Creating UX Analysis Guides and Portfolios

Moderated by Helen Lee Lin as part of the UX Researchers’ Guild
This is an abridgment; view the full video presentation here.
Session 4 – February 11, 2023

Once you have wrapped up fielding your UX project, it’s time to dive into analysis and reporting! In this session, Helen Lee Lin shared examples of UXR portfolios and talked briefly about the level of detail, storytelling, and presentation resources.

UX Analysis Guide

The first thing to look at is the depth of analysis. Here are some tips to ensure that you are on the right path.

While you may be familiar with analysis in the world of academia, expect that you’ll spend far less time analyzing data in industry. For an interview study (8-12 participants), plan on about 2-3 days to synthesize data, which may overlap with study fielding and reporting timeline.

One thing to remember is that stakeholders prize timely insights. But don’t be afraid to advocate for the time you need – just be reasonable. Consider providing “quick insights” or “initial insights” when there is time pressure so stakeholders will know you are actively working on the project. Then, follow up with a deeper dive and higher-fidelity shareout if needed.

Sometimes leads or stakeholders have made a go/no-go decision, and further reporting will no longer be relevant. Keep lines of communication open to understand their timeline and how it corresponds with what you have planned.

While it is preferable to have more than one researcher on a project, you’ll likely be the only one. You probably won’t be tagging/coding data with other raters or attempting inter-rater reliability.

Key Features of UX Analysis

As you’re doing your analysis, here are some key points to consider.

You won’t dissect every bit of information, but you are looking for patterns. You’re looking for information that answers your main research questions. This will help provide direction for your stakeholders in their decision-making. Look for signals that speak to any of those things. 

Perhaps your data scientist has questions they want answered with qualitative data. They might have some specific insights from the data that they’ve collected. But they don’t know why it’s happening. You can help by looking for answers along those lines.

Speak to your stakeholders. Your manager is usually tapped in and can guide you there if necessary. Your product manager usually has these questions in mind as well.

Be on the lookout for surprising and interesting insights. Maybe it was not something you set out to study, but then it turned up in the patterns you’re missing from your data. If something piques your interest, ask yourself: “Is this an example that would make a good story?”

It’s common for a user research team to point out what designers or engineers have done wrong. Look for things that your team or product is doing well. You know these team members have been working for months to design and build what you’re studying. The last thing you want to do is to tear apart their work.

You also want to know what’s working well for the users. You don’t want to break the good experience they are having. As you iterate on the product and start changing things, try to preserve either the general aspect of what was working well for the user, or the experience itself.

Make special note of customer requests and new questions for future research that arise. These can point to the root problem that the users want solved. You may not be able to meet every request, especially if it is just one person that wants it, but identifying the problem will be helpful in the long run.

Think About Your Audience

As you think about the deliverable, the presentation you’re putting together, your audience will generally be your first question. Your presentation may vary if it’s an interview presentation. So, depending on the company you interview with, you may speak only to other researchers. But you could present to an entire product team, including engineers and designers. Ask your recruiter or hiring manager who will be present to know how to direct your presentation. 

If it’s a presentation for your entire workplace, keep it general. Include any research-related details in an appendix for researchers, because those are the questions they will have in mind. But you don’t want to distract the non-researchers with that extra information.

Recruiters typically spend less than 30 seconds on a resume. It might be as little as six or seven seconds to see if they should triage a particular regimen, or if they need to look deeper. When it comes to portfolios, managers might give it a glance for about three minutes before deciding if they see potential in what you can bring to their team or organization. Provide information as succinctly as possible and make it easy to find. Your project may not be deemed relevant at first, but if you have something that piques their interest, that could be the foot in the door to score that interview.

Creating Portfolio Case Studies

The first thing that anyone will see is your title. Give special attention to how you present yourself and choose a title that conveys the expected impact or POV. Try to get information on the users to make it relevant.  Some examples might be:

• “Optimizing new user onboarding…”
• “Understanding young adult creators’ community needs…”
• “Identifying gaps in admins’ moderation tools…”

Play to your strengths. If you have a niche market, include it. Ask yourself: “What makes this case study distinct from others or showcases special skills?” Then be specific: “Diary study and moderated interviews, teens 13-17, APAC market,” for example.

Explain your role in the project. If you were not the lead on the project, clearly identify which parts of the project you owned and what was done by others.

You will want to include the titles, but not the names, of your stakeholders’ and which cross-functional roles you worked with. This could include:

• Common: PM (product manager), EM (engineering manager), ENG (engineers), UXD/PD (designers), UXW/CD (content/writers), DS (data scientists), Marketing/Analysts/Insights
• Also possible: C-suite, Directors, Heads of…, Leads, Strategists

There are several processes for creating your portfolio case study: the collaboration process, the decision-making process, and the research process. Each needs to be addressed and communicated to your stakeholders.

The collaboration process involves identifying how you kept stakeholders well-informed and when you sought any necessary alignment with them. Other questions to consider are: How did you gain stakeholder buy-in, and how did you manage your stakeholders?

The decision-making process presents the steps you took to gather information. What information did you need and how did you gather it? What decisions did you make as you scoped your project, and why?

Finally is the research process.  What were your constraints? Budget, time, recruitment difficulties? What blockers did you run into while fielding? How did you solve or pivot?

If under NDA, keep findings high-level or change variables/numbers. This may mean being vague or simply describing the impact this finding would have. Update your impact as time passes as you see new effects of your research.

No portfolio is perfect. Take a hard look at what you’ve done and plan what you’d do differently next time. Be transparent and honest about mistakes. What did you learn from your mistakes? What steps have you taken, or will take, to improve in the future?

Visually Appealing Portfolios

As researchers, not designers, your content is more important than showing designs. However, visual considerations are still important for your audience’s user experience. Consider the ease of finding key information, including headings, labels, and putting sections in a logical order. Be mindful of font type, the color of the text, and background and alignment, all of which will affect the readability of your text. 

Communicate information but avoid large blocks of text. Break up your text by bolding key phrases or using bullet points. And lastly, proofread for grammar and spelling! Ask friends who are unfamiliar with your work to review for understandability, accidentally omitted information and typos.

Portfolio/Presentation Resources

The following are some presentation resources that Helen has used. Most are free, with some additional resources available for a fee.

• Free slide templates that come with professionally designed layouts, icons, and images
  • Slidescarnival, Slidesgo, Slidesmania, Canva
• Free stock photos
  • Unsplash, Pexels, Pixabay, Nappy.co, Freepik, The Noun Project, more resources here
• Free icons/graphics
  • Icons8, Flaticon, The Noun Project, Drawkit, Storyset, Undraw, Opendoodles, Blush.design, StreamlineHQ
• Other helpful presentation resources
  • Mockdrop, ThisPersonDoesNotExist, DiverseUI, LottieFiles, MOV to GIF

Tips for an Interview Presentation

If you’re doing your presentation as part of a job interview, Helen offers the following tips:

• Reduce text and voice-over details instead
• Spread information out across more slides
• Include checkpoints where you pause for questions
• Include slides that emphasize key takeaways
• Include slides with pullout quotes from participants, or interesting statistics
• Illustrate with screenshots of the UI that you’re discussing and be sure to label them clearly and accurately
• Add more images, graphs, and icons to break up text
• Place additional details in an appendix or appendices

Tips for a Work Presentation

If the deck will be shared throughout your company, here are some additional considerations:

• Add a TL;DR (Too Long, Don’t Read) or executive summary.
  • Summary of key insights
  • Similar to abstracts in academia, but with less focus on background and methods
• (If relevant) include a success story from your participants, and tie it into insights or the recommendations you’ll be making.
• Use example scenarios, give names to the users or personas – make the info “sticky” for your audience.
• Add diagrams and graphs to illustrate concepts
• Include hero quotes and video clips from participants
• Illustrate with screenshots of the UI that you’re discussing
  • Be sure to label them clearly and accurately
  • Tell stakeholders what the product is doing well (don’t tear apart Design and Eng’s hard work)
• Add recommendations – where should your stakeholders go from here?
• Place additional details in an appendix or appendices.

 

About Helen: Helen Lee Lin received her Ph.D. in Social Psychology and has worked in applied research in adolescent literacy and children’s nutrition, and with combat veterans with traumatic brain injuries. She took a career pause while living in Ankara, Turkey for 6 years and then transitioned to UXR in 2018. She volunteered for Hack for LA for six months and broke into industry in January 2021 with a contract at TikTok. She is currently a contract UXR at Meta, working on Facebook Groups.