Hosted by the UX Researchers’ Guild
Moderated by Danielle Green and Jess Vice
This abridged version was written by Booker Harrap. View the full video presentation here.
Session 3 – September 21, 2023

It is clear that the definition of “quantitative skills” varies among researchers (and hiring managers). Even when definitions are aligned, folks have different views on the level of quant required to be a successful UX researcher. 

Danielle Green interviewed dozens of experienced UX researchers about their views on quantitative skills. The results fell into two common themes: 

1) Many researchers lack confidence in their quantitative capabilities. Sometimes these researchers are ashamed because they have no training in statistics, while some are ashamed because they “only know basic things like multiple regression and factor analysis.” Some researchers actively avoid any job descriptions that contain the word “quantitative.” They identify as “qualitative researchers” but are often happy to design, analyze, and report on surveys containing only Likert scale items. 

2) Some researchers self-report “mastery” of quantitative research practices but cannot define terms like “standard deviation” or “confidence interval.” This then naturally leads to the question: What should mastery consist of?

So, are quant skills required or just a bonus for UX researchers? Let’s dive into the discussion of the pros and cons of quantitative skills in UX research.

Pros of Requiring Quant Skills

Minimum Quant Skills Needed in UX Research

• UXRs should be able to calculate estimates of true values given a sample and statistically compare two or more groups. 

The minimum level of quantitative skills would include being able to compute estimates of the true value based on a sample of data, as in reporting confidence intervals, and having the ability to statistically compare two or more groups. This points to basic inferential statistics knowledge, such as familiarity with Anova, Chi-square, and T-tests. Comparing groups usually means proportions and reporting confidence intervals around proportions. 

Quant Skills Contribute to UX Research Success

• The two-part definition of success is 1) stakeholder buy-in and 2) making decisions based on research that generate value for the organization.

Success in this arena boils down to two main ideas. The first is stakeholder buy-in. When you report findings and present them to stakeholders, you want them to take these findings seriously. And second to this is research data-backed decision-making. When the organization takes an action based on your research findings, that action produces value, whether revenue, clicks, signups, etc. Your research insights will turn out to be right most of the time. 

Quant Skills Increase Chances to Be Hired and Retained

• When two resumes are roughly equal, the one with quant will likely win. Business leaders struggle to value qual-only research.

So, why is it good to be familiar with quant skills? The simple answer is getting hired, especially in this job market. All things being equal, the candidate with quant skills will probably get the job over the candidate who lacks those skills. Getting the job is just half the battle; you also want to keep that job. You do this by being seen as a contributor with unique value among your team members. Business leaders can struggle to note the difference between your qualitative insights and the junior PM who talked to five customers this morning and now has a two-year roadmap. Even if you have a Ph.D. in anthropology, they may not recognize the difference between qualitative research and the one who talked to a handful of customers this morning. Quant can help you retain that unique value that you are contributing to the team.

Qual Combined with Quant Skills Improves Operational Efficiency and Effectiveness

• When quant and qual exist together in the same person, the two types of data can be combined more effectively.

When researchers have both quant and qual skillsets, they combine both types of data in a single person’s mind and then deliver more powerful and effective insights. They can do this because they’re able to process both types of data together. If you have both these skills, you don’t have to involve other team members when synthesizing insights. 

Quant Skills Contribute to Accuracy

• If you make decisions based on counts and averages alone, research will be biased to “build it,” which is not advantageous for most orgs.

A final thought centers around accuracy and making the right calls based on research. Whereas those with quant skills may be biased to say that two things, groups, or designs are not different, those with less quantitative skills may report differences that may not be true in the user population.

Philosophically speaking, quantitative researchers tend to be more conservative and less likely to say yes to building an app, making changes, or adopting a more costly option. Less quantitative folks will be more prone to want to try new things. While there is a time and place for both, organizations generally hire researchers to de-risk things and will look for conservative and quantitatively inclined candidates with a sensitivity to those risks.

Cons of Requiring Quant Skills

Accurate (Qualitative Researchers) vs Precise (Quantitative Researchers)

• Because user experience centers around humans, it is impossible to be precise in most scenarios.

Jared Spool points to the difference between accurate and precise; qualitative researchers tend to be accurate, while quantitative researchers tend to be precise. In real-life scenarios, the only time we actually talk about precision is when the client asks about running a survey to statistical significance to which the response is to inform the client of a lack of funds and time. 

But generally speaking, most of the work done as UX researchers deals with humans who are unpredictable by nature. Because of this, it’s usually good, or at least good enough, to be accurate, to represent the trends rather than to be precise. The focus that quant brings a more conservative, granular, and specific approach sometimes causes us to miss the bigger picture. 

Qualitative Researchers Build Collaboration with Data and Teams

• Rather than stepping on toes by taking on someone else’s role, build collaboration with data, analytics, and business analyst teams.

There’s a symbiosis between purely qualitative researchers and purely quantitative data teams. Rather than trying to acquire the skills the quant team already has, can you show up instead as someone who complements them and brings a different perspective? In this way, you can challenge how they look at their jobs and build more collaboration internally, something every company wants.

Since a qual skillset is looking at people patterns, perhaps your time and effort would be better spent focusing on partnering with the business analyst teams already in place rather than trying to be a business analyst. Examining roles and responsibilities within any organization is something all researchers need to better understand. 

Cost-Benefit Analysis of Acquiring Quantitative Skills – Is it Worth It?

• Are quant skills going to be additive? How long will they take to acquire and how much will it cost? What will the real outcome be? Will this effort help me be a better researcher or get that dream job or promotion? 

These are legitimate questions that need to be addressed, especially when quant, or advanced math skills, are not your forte. If these questions make you feel uneasy or uncomfortable, then maybe the effort to acquire such skills is not worth your time and energy. In this case, perhaps it does make more sense to partner a quant person with a qual person, thus allowing each to be the expert in their field.

Data Misrepresentation Without Adequate Quant Training

• Without enough training and accountability, quantitative data can be skewed to tell the story we want. 

Another thing to consider is that without adequate training in quant skills or internal accountability on your part, can you guarantee that the quant numbers you’re sharing are accurate? The numbers might tell the story you want to tell, the one that makes your boss happy, or even the one that guarantees that raise you’ve been aiming for. But is it the correct story? There’s always the chance that you’ve skewed the data without realizing it, especially if you’ve recently been hired or are trying on a new set of skills you haven’t mastered yet. These conclusions need to be left to the experts. Companies would be wise to hire a business analyst or someone with a Ph.D. in numbers who has worked in CPA environments: an expert who can see the gaps and knows what to do about them. Without those quant skills, you, as a newbie, might not even know what could go wrong and how to fix those errors. Do you want that responsibility? Being familiar and careful with the data is crucial.

Quantitative Research and Masked Biases

• The diversity, representation, and accuracy of sources is easy to hide or ignore in quantitative data.

As good as quant tools are, they contain inherent biases that can’t be overlooked. Pollfish is one of those tools. It can be a UX researcher’s best friend, but keep in mind that you don’t know where the user panels are coming from. And with tools such as this, there’s no guarantee you will get the diversity or accurate representation of the groups you are surveying. 

There are unknowns behind the scenes that we don’t understand unless it’s made transparent. In quant specifically, people are more inclined to keep those details in the background because they don’t represent an interesting story. So, instead of getting the full picture, we end up with masked biases that, from the beginning, skew the data. This, of course, can be disconcerting when we’re talking about human data, about real people.

As a UX researcher, you will need to determine whether having quant skills in your toolkit is right for you. These pros and cons can serve as guidelines to make that all-important decision.

. . .

About Danielle Green (she/her): Danielle is a product and UX professional specializing in research and strategy (high growth, product-market fit). She is an Instructor and mentor with eight years in product, and five years leading teams. As a professor of practices at Claremont Graduate University, Danielle teaches the core courses for the User Experience MA in Applied Cognitive Psychology. She is also the founder and director of the Claremont UXR Laboratory (claremontuxrlab.com), a graduate student lab for UX Research, and has extensive experience in many domains, such as: e-commerce, SaaS, Edtech, Virtual Reality, and hardware.

About Jess Vice (they/them): Jess loves working with people and is curious and excited to understand what drives them to make decisions. Jess offers a deep background in qualitative research, user experience best practices, and high-level strategic planning and is particularly good at making meaning from research and using it to create data-informed strategies for creative and development teams. Jess is also intensely aware that the first point of contact is always the internal teams they work with — if a relationship is not built on trust, they know they won’t be a successful researcher and strategist. Jess has been working in marketing and advertising, CRO, SaaS, and product for over 14 years, and is consistently thrilled with how much more there is to learn.

Moderated by Kuldeep Kelkar and Rima Campbell
View the full presentation here.
December 1, 2023

If you’re a UX leader who wants to have more impact, the following strategies will help affect change, increase your influence in your organization, and help get a seat at the table with decision-makers to determine product strategy, corporate strategy, and budgets.

Strategic Influence to Increase Executive Buy-in with Kuldeep Kelkar

Approaching this topic can be overwhelming. But let’s look at two keywords that will help break it down a bit: Executive and Influence. 

What Executives Need

To create strategic influence, first identify key executives within the organization, then contemplate their needs, and lastly, consider their concerns. Following these steps will help you focus your efforts where they will have the greatest impact.

Step 1. Identify Key Executives:

Reflect on two to three key executives at your company whom you want to influence. Consider their roles, responsibilities, and the scope of their influence within the organization. These could be anyone within your organization: a product leader, someone in marketing, your reporting line VP, and so on. 

Step 2. Contemplate Executive Needs:

Beyond the realm of UX, contemplate what these executives might need. What are their broader goals, challenges, and priorities? Think about how your work intersects with their objectives. 

Step 3. Consider Their Concerns: 

What potential issues or challenges keep these executives up at night?  Try to empathize with their position and the pressures they face. 

Ways to Grow Your Influence as a UX Leader

The second keyword in today’s topic is Influence. How can research leaders influence stakeholders? The following suggestions are by no means the only tools you can use to grow your influence. But these are the ones that I have found valuable throughout my career.

Build Credibility 

Building credibility is both an art and a science. Since you are already working with an organization, you can demonstrate tremendous value and build credibility as you execute research; whether in a large study, design validation, or part of a research roadmap.

Forge Strong Relationships 

Forging strong relationships across the organization cannot be overlooked. While this applies to designers or product managers, look for allies in marketing, engineering, HR, or finance. Eventually, an opportunity may come to have a wider influence, but it starts with building those strong relationships. 

Become a Master Storyteller and Share Success Stories

None of us can be influential unless we are master storytellers. I have never interacted with a successful executive who was not also a good storyteller. Learning to share those sound bites and narrate success stories is key to influence. I work with many executives, and those I have tried to emulate are all good storytellers. 

Please note that this goes beyond a PowerPoint deck or research report. Consider adding sound bites and enticing stories into general conversations. Success stories of what you are doing will increase your influence over time. 

Facilitate Workshops 

The next one is near and dear to my heart. As researchers, AI will make us more productive and efficient in defining screeners, analysis, and reporting. But I don’t know that AI can unite us as humans. Most UX researchers and designers are proficient at reading people. We empathize with end users, product managers, and people we work with. As researchers, we are best suited to bring people together. Anytime you have an opportunity, jump in and facilitate a workshop, whether at the divergent or convergent phase of an engagement. Doing so will bring you into strategic conversations you were not previously aware of.

Domain Expertise and Industry Knowledge

The next suggestion centers around domain expertise that is more than the user experience or user research domain. By this, I mean industry knowledge. In the last year, I have worked with product leaders and investors and have worked with several startups. I also speak with the investment community. I have learned that successful leaders within any organization are generally those who have been in that industry and are familiar with competitor pricing, personas, or the ICP, the ideal customer profile. They know what the competitors are doing. 

The more you know about your industry, including pricing trends and growth, the more you know about your competitors. Information for public companies is likely found online. Keep up with the knowledge of whichever industry you work within. Try to speak those sound bites so that people understand that you know how things work within your industry. And again, this is not just about UX but the implications of UX within your industry.

Hire Consultants and Invite Guest Speakers

The next suggestion is to hire consultants and invite wise speakers. You might wonder why this is important when you are a consultant yourself. When I worked at PayPal, before my consulting days, I brought in guest speakers within my organization. Ironically, this made me more influential because I could introduce industry thought leaders to my company. Even if these guest speakers said the same thing I had shared, they offered additional value and validated what I was saying. Doing this will increase your influence.

Mentoring and Being a Continuous Learner

The following two suggestions focus on mentorship and seeking mentors within and across the organization. You don’t have to have a particular agenda in doing this. Being a continuous learner is an important attribute that ties into the need to forge strong relationships. 

Learning from others is crucial, but it is also vital to mentor others. You have experience the larger community can benefit from, even within your company. Every day, product managers, designers, and researchers are being hired. Network with them and be available to educate others to the degree you can.  

Measuring Your Impact and Influence

It is sometimes difficult to measure impact and influence. But you cannot manage something that you cannot measure. I encourage you to figure out ways to measure, improve, and re-measure your current influence. It’s all about understanding where your influence exists. 

Perfect Your Elevator Pitch

The theory of an elevator pitch is envisioning yourself on an elevator with an executive. As you enter the ground floor together, you will have five to 10 floors before exiting. That’s 20 to 30 seconds. That may not seem like a lot of time, but it is enough to do a few simple things.

The elevator pitch typically has three components: sharing who you are, what you do at the company, and why what you do is important. Take advantage of every opportunity to connect with executives, even if for this short period of time.

Think about sound bites to have at the ready. These can be cliffhangers or dramatic headlines, or anything that will cause the other person to pause and want to know more. These hooks are conversation starters that encourage a response and leave an impression.

Speak the Business Language

To connect with executives, you will need to understand the needs of the company and the business metrics that drive performance. On the simplest level, if the person you are speaking to within your industry refers to the end user as a customer, patient, or guest, use those key terms and understand why they are used. 

On a more complex level, take time to research basic concepts of business and finance. You don’t need an MBA or PhD to do this. Someone somewhere in your organization is working on how to beat the bottom line or the beta or gross margin goals that determine how much money the company has to spend on different functions. Every company that I’ve worked for and known has someone who is looking at what the top line number should be for the next four quarters. This forecasting determines how much money there is to spend. The more you understand and speak the business language and understand the company, the more influential you will be simply because you will participate in conversations that you were not part of before. 

Getting a Seat at the Table with Rima Campbell

UX Challenges in Business

A major challenge UX researchers face is getting executive buy-in for UX research. One reason this may not happen as often as we would like concerns the relationship between user experience and key performance indicators. Executives in the C-suite are looking for KPIs and the bottom line and how that impacts their business. The UX professionals and leaders are looking for insights. So, how do they bridge the gap between the C-suite and the user experience? 

McKinsey & Company ran a study about the potential for design-driven growth.¹ They analyzed the impact of the design effort of 300 publicly listed companies over five years across a variety of industries and countries. The report found that design-forward organizations performed at twice the rate of their industry competitors when it came to generating revenue.

How were they able to do this? As explained in the study, “Setting a North Star and combining qualitative data and quantitative measures can demonstrate the value and effectiveness of design.” They started with continuous iteration of their design validation, then consistently embedded research throughout the product development lifecycle, which led to making data-driven decisions by using a higher sample size. They also measured the performance of their design and its impact on the financial success of their business. 
_______________________________

¹ https://www.mckinsey.com/capabilities/mckinsey-design/our-insights/the-business-value-of-design

Competition in the World of UX Research

We live in a world of constant competition in which every executive wants to know their company’s rank among competitors because seeing their ranking can be a motivational booster.

I experienced this firsthand in a study while at Citigroup. The business wanted to increase online credit card applications submitted by 22%. As a researcher, I wanted to make an impact and get executive buy-in for more UX research to grow the budget and the team and be able to embed research across the product development lifecycle. To do this, I had to show metrics and the impact UX research and design make on the business. We looked at the analytics, trying to figure out why people did not submit their credit card applications online, as well as why there was a large call volume related to that issue. Looking at just the analytical data wasn’t giving us the answers. 

So, we looked at the survey where we had interviewed 15 individuals. We chose to simplify our approach by asking participants to do one task: choose a credit card that matched their needs without submitting their application. We also determined that we needed a significant sample size to prove the accuracy of our results.

We ran a baseline benchmarking study where we examined behavioral data and also ran a “talk out loud” study to hear the participant’s frustration along the way. We discovered that the success rate for completing that journey and selecting the current card was 63%; satisfaction was below 60, and the ease of use was equally low. We now understood why participants did not complete that task in the journey. Our comparison experience was unsatisfactory and left participants with a sense of mistrust. When we shared this information, the executive wanted to know what the success rate norm was. We knew why and had statistical data to prove our findings. But how did we know that a 63% success rate was not good?

To verify this, we ran the same study on American Express, Discover, and Wells Fargo and found out that we were, in fact, not doing a good job. To improve the success rate, we needed to improve the customer satisfaction.

As we improved the comparison experience, success, satisfaction, and ease of use increased. We then ran more studies on the prototype and made improvements, iterated design, and tested it to ensure the success rate and satisfaction were on par with our competitors. Within 12 months, we realized that enhancing the experience improved the online application by 22% and reduced the call volume from 63% to 30%. Imagine the impact we created in that particular case study. We got the executive buy-in to do research across the product development lifecycle by starting with this case in that project. 

How to Measure UX Influence

We all agree that measuring influence can be tricky. Consider sharing a survey or questionnaire with stakeholders to collect structured feedback every six months. 

The User Research Team …

• … is regarded as a credible and reliable source within the organization.
• … demonstrates exceptional expertise and knowledge in their field.
• … consistently provides value.
• … regularly introduces innovative ideas that significantly benefit our projects.
• … effectively engages with stakeholders, fostering positive collaborations and outcomes.
• … is frequently recognized within the organization for its contributions to success.
• … plays a crucial role in positively influencing and shaping our organizational culture.
• … exerts influence beyond its immediate domain to other departments and areas.

Being open to their feedback will go a long way in measuring the influence you have on executives and the company as a whole.

Additional Thoughts on the Elevator Pitch

Adding to what Kuldeep shared on this topic, being on an elevator with an executive is a perfect opportunity to talk about UX research and what we’re doing in that world. But if you launch into a long story, the executive will lose interest. 

How do you do an elevator pitch that captures their interest and encourages them to want to know more? Look at sharing specifics, such as, “We ran a benchmark study and Blue Cross Blue Shield won the competition. They scored 77 out of 100, whereas Humana scored the lowest. Their score was 57 (this example is fictitious).”

Sharing this story of scoring and comparison will spark the executive’s curiosity. I could then share that Humana paid a significant amount of money to redesign its website. So why did they score the lowest? Now, with their attention, I am invited to the table. I can go into greater detail to show how this impacts business metrics and KPIs.This will lead to even more discussions because every executive cares about their KPIs and is accountable for business metrics. 

As a UX research leader, you need to know your business metrics and how to tie them to what matters most to your organization. The QXscore will help you do that. It also includes the loyalty or NPS score which most executives resonate with. 

Connecting UX Metrics to Revenue

The bottom line is you don’t have to remember a million scores; you only have to remember one score that is out of a hundred. Having information like this at your fingertips will allow you to capture an executive’s attention. This then trickles up into what we call the 21st-century metrics framework from level four to level one as an impact on the financial goal. Having a deep understanding of a key executive’s ambition and desire, and tying those UX metrics to the business metrics, will make a huge impact and increase your influence.

You Got a Seat at the Table: Now What?

Several organizations ran the measurement program and made a huge impact. One of them is Kimberly Clark. Because they were able to improve before and after from a task success perspective and time spent, they were able to improve the business metrics and show how the UX metrics improve their business metrics and what they end up with. One big success for them is driving prioritization and demonstrating progress as UX leaders and practitioners. In this way, you can impact the prioritization of your project from a product and IT organizational level.  

Scalable Solutions with Customer Value

Another study was with Kroger, where they incorporated scalable solutions with customer value as they ran these measurement programs. Doing so allowed them to get the business to invest more at the beginning of the product development lifecycle and do more discovery, ensuring they were creating the right solution for the customer problem. This occurred at the beginning, not just validating the design in the middle of the product development lifecycle. 

Tying OKRs to QXscore Results

Adobe tied the OKRs to QXscore results and embedded the UX metrics in the design and performance reviews. This desire came from the UX design team to improve their score and the impact they’re having on their organization. So, it was something they included in their performance reviews. 

UX Measurement Program Framework: Activate, Ignite, and Drive

A UX measurement program framework includes three categories: Activate, Ignite, and Drive. 

In the Activate category, there are three components: 

• Identify what matters most to executives;
• Identify a pilot project that focuses on a user journey; and
• Connect UX metrics to the business metrics. 

The Ignite category includes the following steps:

• Know the QXscore and the competitor’s QXscore;
• Educate and build trust with the QXscore; and 
• Prioritize improvements with stakeholders, test and iterate with benchmark.

The last category, Design, incorporates these three elements:

• Visualize, track, and share progress regularly;
• Embed QXscore into team performance review; and 
• Tell inspiring stories regularly.

UX researchers face challenges in today’s enterprise, and focusing on them may seem overwhelming. But don’t give up. Your dedicated efforts will eventually bear fruit. Remember that you can demonstrate value, get executive buy-in, and gain a seat at the table.

About Rima Campbell and Kuldeep Kelkar:

Presenters Rima Campbell and Kuldeep Kelkar worked together as research leaders and consultants at UserZoom (now UserTesting) for the last seven years. They consulted with leaders at Global 500 companies, helping them to conduct effective research, refine products, and increase their influence. Previously, they had long stints leading research in large enterprises; Rima as SVP and Global Research Manager at Citi for 16 years, and Kuldeep as Head of Global UX Research at PayPal for 10 years.

Kuldeep Kelkar
kuldeep@uxreactor.com
https://www.linkedin.com/in/kuldeepkelkar/
https://uxreactor.com/blog/

Rima Campbell
https://www.linkedin.com/in/rima-campbell/
https://www.usertesting.com/resource-library

A UX Researcher’s Guild Online Event
Moderated by Danielle Green and Jess Vice
This abridged version was written by Booker Harrap. View the full video presentation here.
Session 2 – July 14, 2023

As a UX researcher, it’s essential to stay up-to-date with the ever-evolving perspectives on research best practices. Scrutinizing our standard tools and methodologies is important to catch any gaps in our ability to generate sound, actionable insights. In a recent Research Rumble, we took a deep dive into one such tool: personas.

Are Personas Good?

(By Jess Vice)

Distilling Research for Easy Consumption
For design and development teams, who tend to be very visual and think big picture, personas provide a concise summary. Instead of being overwhelmed with data, they get a snapshot, making it easier to assimilate and act upon.

Preventing Self-referential Design
Alan Cooper’s introduction of personas in 1998 aimed to curb the “elastic user” problem, where we imagine what the user might want or who the user might be. Designing based on assumptions is problematic; personas ground these decisions in research.

Maintaining a Research-Backed Focus
Personas help in combating scope creep. They serve as a constant reminder of the real user, discouraging the creation of imaginary users based on our biases or assumptions.

Building Team Alignment
When multiple stakeholders are involved, differing opinions can delay projects. Personas, being research-backed, offer a common ground and guide aligned decision-making.

Cultivating Empathy
With diverse teams contributing to product development, it’s easy to lose sight of the user. Personas serve as a tool to humanize users, fostering empathy, and ensuring that the end product caters to genuine needs.

What’s Wrong with Personas?

(By Danielle Green)

Inclusion of Irrelevant Data
When personas become more of a formality than a functional tool, they risk containing unnecessary information. This dilutes their effectiveness and confuses more than clarifies. 

Too Many Personas
Creating a multitude of personas can paradoxically make them less useful. Instead of clarity, teams face choice paralysis, questioning which persona to prioritize.

Unsubstantiated Foundations
A persona built on minimal or irrelevant data risks being a fictional character rather than a representative of real users.

Risk of Extrapolation
Ironically, while personas aim to curb assumptions, they can sometimes encourage them. Stereotyping of characteristics present in the persona can lead to misplaced assumptions. “Did we just assume that the mother of five who is preparing a family’s meal is a homemaker? Or did we actually ask that question?”

Mental Shortcuts to Different Internal Definitions
Similar to extrapolating beyond the data, the mental shortcuts that personas provide can turn sour when these shortcuts lead to different internal definitions. Maybe memories of the data have warped over time, leaving stakeholders overconfident in what are essentially assumptions.

Designing for the Average, Not the Real
Assuming these challenges of execution and socialization have been overcome – there is a decent amount of data from an appropriately representative sample – what could go wrong? Often, this data is represented by the averages across the various characteristics, creating a persona of a user that doesn’t exist. This can lead to a persona with conflicting items that don’t align.

Creating for the Current Market, Not the Target Market
Maybe the problem of the non-existent average user has been accounted for with a cluster analysis to see which factors hang together, leaving the team with an accurate snapshot of the current user. Perfect! Or is it? There is a difference between the target market and the current market which is not being accounted for by this approach, and this can be a strategic pitfall. Company mission, company values, and a changing marketplace mean it may be very important to design for the ideal user, but this excellent persona of the current user is blinding, and the new market out there that could be served is being ignored. 

An interesting data point here: Despite these concerns, a recent UXPA salary survey revealed that 68% of UX researchers still employ personas.

Why Are Personas Good?

Group Discussion Insights In Favor of Personas

Echoing the Opening Arguments

Cultivating Empathy
It comes as no surprise that this point was echoed by other groups of Rumble attendees.

Maintaining a Research-Backed Focus, and Building Team Alignment
This quote reflects these points well. It also makes a fresh point that personas create company-wide prioritization:

“We need to have consumer surveys with significant data that show that these characteristics, these priorities, these pain points match up with this group that is statistically significant [and] fits within a mainstream user base, right? And we have a general sense of the jobs that they want to get done with our product, right? And so my approach with personas is I will incorporate anything into them that works, that will make it more credible… make it easier for designers to act on it and to keep the company, the product designers, the product managers, the executives–strategically keep everyone aligned on [the] top priorities and the biggest pain-points or use cases that we’re solving for that appeal to a large mainstream user base.”

Fresh Insights

High Utility for Those Involved in the Design Process
This was brought up by two different groups, who called out that personas are particularly useful for stakeholders involved in the design process. Here is a quote:

“[Personas are] very valuable to creatives, designers, and people making decisions on behalf of users and a product and ensuring that they’re building or designing the right thing.”

Replacing Personas… With What?
Two groups independently brought up the following question: “If we throw out personas, what would we replace them with?” It’s a good question that no one had an answer to.

Differentiating Target Markets
A more specific use case for personas mentioned is their utility in differentiating target markets.

“The buyer of a [truck] is very different than the buyer of a [sports car]. [Personas] help us. But then … at a second level, we try to expand our personas because it is possible that that single mom not living in Texas does want to drive a truck. So, we might have who we think is our target, but we try to expand them beyond that.”

Aspirational, Building Towards the Ideal Customer
This callout echoed the opening remarks about considering the ideal user:

“We see a lot of benefit for them being useful for long-term or aspirational use. Who is it that we want to be targeting and can we build towards that sort of ideal customer?

Why Are Personas Bad?

Group Discussion Insights Against Personas

Echoing the Opening Arguments

Risk of Extrapolation
One group specifically referenced the opening remarks:

“As we’ve already discussed, it’s also very easy to get a big idea about something with very little data, which can lead you down a false path.”

Fresh Insights

This is where things got interesting. Although most of the favorable aspects of personas had already been mentioned in the opening arguments, our Rumble attendees had a lot of new insights when speaking about the pitfalls of personas.

Relying on Old Data / Not Updating Persona
One group of Rumble attendees brought up the need to constantly check where the data is sourced from and how recently it was updated. This is a good point; personas should not become stagnant artifacts that provide a false sense of diligence in research-backed decision-making. They must reflect the user we are currently trying to target.

Risk of Not Designing for Limited Abilities
This was a great point that can so easily be overlooked when creating a persona, and focusing on your largest addressable market. All products should keep in mind the need to design for people with ability limitations: not just because it is the ethical (which it very much is!) and the effective thing to do, but companies who fail to do this can find themselves in very hot water, legally or as a matter of public image.

Less Useful for Stakeholders Not Involved in Design
On the flip side of the benefit that personas provide to people involved in designing and building products, personas can have a limited utility for stakeholders not involved in the design lifecycle. Like all presentation of research, it is important to remain intentional in the building of an artifact and who it is designed to help.

Quickly Forgotten After Lots of Work
This can easily happen since personas are such a standard practice. If they are being built as a matter of checking a box in the product development process, and a low effort is made to integrate their use during design and decision-making processes, that can make them effectively a waste of time and resources. Have a plan to revisit research artifacts throughout design!

Limited Resolution of Target Market
Looking back to the benefit personas provide in differentiating target markets, this only happens if the opportunity to do so is noticed and acted on. If you rely on only one persona, there’s a risk of excluding a segment of the market that represents a substantial opportunity. Don’t fall into this trap! As one of our Rumble attendees noted: “It is possible that that single mom living in Texas does want to drive a truck.”

Sometimes They Don’t Account for the “Why”
While personas provide a snapshot of the user’s characteristics and needs, they often lack deeper insights into the underlying reasons or motivations behind certain behaviors or preferences. This “why” aspect is critical for truly understanding the user’s pain-points, desires, and context:–”They may watch TV, but why?”

Potential solution offered by rumble attendee:

Integrate the user journey and product mapping into the persona and vice versa. This underscores the importance of combining personas with other research methods and artifacts to gain a more holistic view of the user.

Conclusion

Personas and UX research are seemingly inseparable; when you think of UX research methods, personas are likely top of mind.  Personas have become a UX tradition, and like all traditions, it’s worth taking a hard look at them and seeing if they still serve us. In this post, we’ve taken a look at the highlights from our recent Research Rumble where we discussed how well, or poorly, they fit into the modern UXer’s toolkit.

. . .

About Danielle Green (she/her): Danielle is a product and UX professional specializing in research and strategy (high growth, product-market fit). She is an Instructor and mentor with eight years in product, and five years leading teams. As a professor of practices at Claremont Graduate University, Danielle teaches the core courses for the User Experience MA in Applied Cognitive Psychology. She is also the founder and director of the Claremont UXR Laboratory (claremontuxrlab.com), a graduate student lab for UX Research, and has extensive experience in many domains, such as: e-commerce, SaaS, Edtech, Virtual Reality, and hardware.

About Jess Vice (they/them): Jess loves working with people and is curious and excited to understand what drives them to make decisions. Jess offers a deep background in qualitative and quantitative research, user experience best practices, and high-level strategic planning and is particularly good at making meaning from research and using it to create data-informed strategies for creative and development teams. Jess is also intensely aware that the first point of contact is always the internal teams they work with — if a relationship is not built on trust, they know they won’t be a successful researcher and strategist. Jess has been working in marketing and advertising, CRO, SaaS, and product for over 14 years, and is consistently thrilled with how much more there is to learn.

About Booker Harrap (he/him): Booker recently graduated with a master’s degree in UX Research from Claremont Graduate University. He is currently a UX Researcher with Atticus Capital and is working with the Games & Interactive Technology Lab on an emotional wellness program for middle school students using gamified mobile applications. Booker is actively seeking employment opportunities in UX research. For more information,  please send an email to hire@uxrguild.com.

Presented by Jeanette Fucella
Director, User Research & Insights, Pendo
This is an abridgment; view the full video presentation here.
February 2, 2023

Modern product teams are simultaneously balancing speed to delivery of new features and the need to test hypotheses, uncover unmet needs, and gain new insights about their users. So many user research questions; so little time! ​Given these competing pressures, how do you determine which studies to prioritize? How should a product team determine how to invest their limited time as well as their customers’ time? When is it worth investing in time-consuming primary research, and when is it ok to “ship it and measure it”?

Jeanette Fucella, Director, User Research and Insights at Pendo, answers these questions and many others in her recent session with the UX Researchers’ Guild.

Scaling the Impact of a Research Team with a Prioritization Framework

When Jeanette first put together this prioritization framework, she thought it might generate a few likes when she posted it online. But then, as she describes it,  “the thing went berserk” and has since been translated into multiple languages. This simple 2×2 matrix has helped Jeanette keep her sanity, and she encourages others to modify it to use in their unique work environments. 

Ratio of Researchers to Designers to Engineers/Developers

According to a study published by the Nielsen/Norman Group some years ago, the ratio of researchers to designers to developers or product can be skewed. This is where the prioritization framework stems from. According to these findings, the most common extended ratio is one researcher to five designers to 50 developers. And these ratios are probably changing with recent layoffs occurring in industry. Whereas in the past, these ratios may have been livable, now they are totally out of proportion. 

With this ratio, it is imperative to prioritize your time because people requesting research, whether designers, engineers, or product managers, outnumber those trying to provide the research. 

In the UX industry, there is a lot of talk about democratization. Whether you’re against or for it, is an irrelevant conversation. Researchers can’t survive without delegating a portion of their research activities and tasks.

This is where the framework comes into play. To use this effectively, however, there must be a change in the way researchers think about and approach their work.

The Role of a UX Researcher – Myth versus Reality

To those who know nothing about a researcher’s role, it might look like a slew of experiments, making amazing things happen while navigating successes and failures. And that would be true.

But what if we reframe how a researcher looks at their role? Instead of coming in to save the day, researchers are frequently putting out the fires to ensure that things don’t metaphorically go up in flames.

As an example, researchers want product managers to do the most effective and rigorous research possible. But what if the product manager is the problem? Then researchers need to explore a solution in the most effective, rigorous way possible. This results in a reframing of who you are as a researcher, and your role in relation to your colleagues, and your work.

The User Research Prioritization Framework

The framework came about as Jeanette discussed with her boss how to decide what to work on personally versus what to delegate to other people. That’s when he suggested a 2×2 matrix. Once she put her ideas into this format, she realized that it resonated with her colleagues and it became a meaningful and effective tool.

The framework begins with a vertical axis representing “Problem Clarity” (with a range from high at the top to low at the bottom). This first axis centers on problem clarity, which researchers deal with regularly.

This problem space is specific to customer problems, as opposed to business and product problems. This then begs the question: how well do you understand the specific concerns of your customers that you as a researcher are trying to solve or address? It might be high; it might be low. These customer problems, often referred to as Jobs to Be Done, are the issues customers are talking about and trying to achieve with or without our product. That’s different from product problems which show up as feedback requests. Product problems exist because a product exists and business problems exist because of the business. 

While there are multiple problems, stakeholders must focus on the customers’ problems within this framework. These other problems do exist and should not be minimized, but where innovation occurs is at the intersection of the three.

All design processes, including the double-diamond shown below, begin with identifying the customer problem which shows WHAT researchers are trying to solve for the customer. But it’s also to identify and mitigate risks as early as possible. Without a clear understanding of the problem, it’s difficult to know what risks might lie ahead. 

(For a more in-depth look at dealing with problems as a researcher, see Jeanette’s article, “The Problem with Problems,” https://dovetailapp.com/outlier/the-problem-with-problems.)

The horizontal axis represents the “Risk of Getting it Wrong” (also from high to low) or in other words, how bad is the risk if you get things wrong? Will customers bail right and left, or will they hardly notice?

The four quadrants created by the two axes, and recommended research methods, are as follows:

1. Research Heavy – Low problem clarity, high risk of getting it wrong. For this quadrant, Jeanette recommends doing Generative Research: JTBD interviews, Contextual inquiry, and Diary studies
2. Design Heavy – High problem clarity, high risk of getting it wrong. Recommendations: Iterative design and Evaluative research: Prototype evaluations, Click tests, and Comprehension tests
3. Research Light – Low problem clarity; low risk of getting it wrong. Recommendations: Generative research: Semi-structured interviews
4. Ship It and Measure – HIgh problem clarity, low risk of getting it wrong. Recommendations: Experimental/evaluative research: A/B testing, Behavioral analysis.

Variations on the Prioritization Framework

The great thing about this quadrant is that researchers can make it their own to fit any given circumstance. The following articles, and modified framework below, demonstrate variations on this framework and how individual researchers have leveraged it to meet individual needs. 

https://uxdesign.cc/building-a-framework-for-prioritizing-user-research-ed46622ead99
https://www.notably.ai/blog/how-to-choose-the-right-qualitative-research-methods
https://dovetailapp.com/blog/research-method-discovery-product-management/

Collaborating with Cross-functional Partners

On the issue of how to collaborate with cross-functional partners, there is no one right answer. It’s very organizationally dependent. Jeanette shared that she has data analysts and scientists that work right around the corner from her who organizationally report differently. Oftentimes, data scientists, especially in an early-stage startup, are focused more on revenue modeling. So that depends on what their remit is. But if you have people for whom that is part of their remit, those are great people to partner with. Part of the challenge is once you’ve set the KPIs or the measurements, remembering to monitor and check it, and then report back to the team.

A researcher’s role is to define what those measurements are and be accurate about what behavioral change accurately measures the success of the new functionality or solution you’re trying to implement.

Thoughts on the Prioritization Framework

Some people may be surprised that they hadn’t seen this framework before. But most would agree that its simplified nature works very well. One use of this matrix might be to plot things on the framework as a group to facilitate a good conversation with stakeholders. Another aspect to consider might revolve around how to layer in the dimension of time or window of opportunity in which research can make a difference, even complicate the prioritization of the work.

Research Impact

Some questions to consider alongside this prioritization framework might include: How impactful will this research project be? How does it align with the business priorities and the business objectives? In our current economic times, the more tightly you can connect the work you do to the priorities of the business, the better. Knowing what the priorities of the business are, and using those to help also prioritize your work, is vital. 

About Jeanette: Jeanette Fuccella started her career at IBM, as a human factors engineer, designer, researcher, strategist, and evangelist. After 16 years in this area, she moved to LexisNexis where she led design ops and research ops initiatives as well as a large global research team. She is now the Director of User Research and Insights at Pendo.

Presented by Lynn Holmes Howe
This is an abridgment; view the full video presentation here.
November 17, 2022

Laying off employees is the single most difficult thing leaders are asked to do. It’s never easy or painless to tell someone that he/she is no longer getting a paycheck. But you can still be empathetic, kind, and supportive.

And if you are on the other side of the desk, the one being laid off, the same sentiments apply: it’s never easy or painless. And although there are some things that are out of your control, there are some concrete suggestions that will make that transition a little more manageable. 

Jonas Cederlöf, of the University of Edinburgh, found that layoffs have negative health consequences for workers and may increase their mortality. “Job loss significantly increases expenditures for antidepressants overall,” he said. In addition to the financial aspect of losing a job, getting laid off can cause serious physical, mental, and emotional health issues, lasting long into the future, putting “a massive strain on public resources, including unemployment, healthcare, and social benefits.” Dr. Harvey Brenner of Johns Hopkins University found that for every 1% increase in the national unemployment rate, there were statistically, 37,887 more deaths, 20,240 more heart failures, 4,227 more mental hospital admissions, and 920 more suicides. 

This can be an emotional and sensitive subject, especially if you or someone in your life has been directly impacted. The following discussion serves as a reminder that there is an empathetic and caring community out there, no matter how challenging this experience might be.

How to Layoff with Empathy

How can employers, leaders, and managers better handle layoffs? Taking examples from recent news stories and personal experiences, here are six steps to help conduct layoffs with empathy and compassion.

1. Don’t let the layoffs come as a surprise. Leadership should start offering color and context of what is happening in the company, and why layoffs might be needed.
   • Open and transparent communication with the entire company is needed even more if the news is bad, and times are uncertain. This is important, not just for the employees being laid off, but also for those who are not impacted and will have an entirely different set of emotions as well. 
   • Hiding information or providing misleading, incomplete information are all likely to break your relationship even with those employees who are allowed to stay with the company.
2. Have a plan before you talk to the employee. It’s going to be a difficult conversation, and it’s important to be ready.
   • Prepare a script, brief outline, or talking points in advance. It’s best to get to the point quickly and deliver the bad news. This preparation will help you be as professional as possible under extremely difficult circumstances.
   • Have a list of answers to anticipated questions.
   • Provide each employee with any necessary paperwork, personalized with their information, whether in person or virtual. Don’t put the employee in a position where they have to ask for this paperwork.
3. As a manager, go the extra mile and make time to meet with laid-off employees one-on-one.
   • Talk through the situation with them. Show that you care and acknowledge the work that they brought to the company. Ask how you can help and listen to their feedback. 
   • After delivering the news, give them a few days to think it through. Give them the option if they are comfortable in doing so, to reach out to you at any time.  
   • While you need to clearly communicate that the decision is final, you can still be available for the impacted employees to talk through things if that would be helpful.
4. Let employees know that you are happy to provide letters of recommendation, and then make sure and write them. If you are aware of the layoffs ahead of time, you could even craft these in advance.
   • Give them proper and well-thought-through letters of recommendation.
   • Write them directly for their potential employers or LinkedIn profiles, as well as allow them to provide your contact information to potential employers for reference.
5. Help them find another job through whatever resources you might have.
   • Provide a reference or offer to make introductions.
   • Ask your network whether they know of any openings and forward them
   • Provide them with a free resume builder, free career counseling, etc.
6. Create a severance package. This can go a long way in easing the impact of being laid off. A generous package might include:
   • Additional weeks of salary
   • Company stock options 
   • Health insurance coverage
   • Pay for accumulated vacation or personal time off  
   • Deposits into 401k account
   • Access to internal job portals

Through the layoff process, managers should inform employees of what is happening and what resources are available, as well as give them access to recruiters, career coaches, and connections within your network of contacts.

As leaders and managers, it’s important to also consider the employees that remain, not only in terms of how they see their former colleagues treated and how the layoffs occurred (obviously, the more poorly handled, the bigger the impacts on morale) but also how they are feeling about the loss. Because it is a loss. Research has shown that remaining employees experience impacts on their physical and mental health similar to those who are actually laid off. They may have survivor guilt, and there will be some uncertainty about their future with the organization. Leaders need to share plans for the future of the company and communicate what’s happening to reassure those employees that things are on track and that they are valued and supported as employees. 

How to Prepare for a Layoff

On the flip side, are those employees being laid off or worried that they might be? How can you best prepare yourself, and what can and should you do if that happens?

The best preparation happens before you get that email or phone call. Here is a check-list to evaluate how prepared you are right now, as well as to point out any gaps: 

• Keep your resume current – update it after a promotion or if you have recently acquired a new skill.
• Keep your LinkedIn profile current. 
• Take time to build out your network. Join relevant job groups and professional organizations that you can tap into if needed.
• Acknowledge recruiters who reach out with related job opportunities, regardless of whether you are interested or not. You just might need those connections in the future. 
• Gather work samples to build a portfolio. 
• Keep copies of your performance reviews. These can serve as reminders of work you have done as you create or update your resume.
• Understand your benefits: healthcare, time off, flex spending, equity, and what happens to those benefits in the event of a layoff. Be on top of what exactly is included in your compensation package.
• Make a career plan and know your next step. Have you been thinking about a pivot? Is there something you are passionate about? Maybe take a look around, if it’s feasible, to explore what would be necessary to get that next job. And if there are rumors of layoffs, consider applying for jobs. Interviews may take months, and starting this process sooner rather than later, can yield great benefits.

I’ve Just Been Laid Off; Now What?

First, take a deep breath and do something that brings you stress relief. Allow yourself time to grieve; you will have a lot of emotions. Reach out and connect with others who understand what you are going through. There will be plenty of details to address. Take care of yourself first.

Understand the exit package details (severance, health coverage, COBRA, 401K) and note important dates of when you will have to take action so you don’t lose whatever your package includes.

Make a list of what you need to update/spruce up – your resume, LinkedIn, etc. Refer to the checklist above. Reach out to your network, and ask for recommendations for your LinkedIn page.

If your former company offers free career counseling as part of the exit package, sign up. You may think you’re doing all right, but emotions and grief will come in waves. Learning how to process the impact of the layoff will be important as you move forward to seek out new employment opportunities. 

Consider signing up for any webinars, classes, or events that interest you both professionally and personally. You will have more time on your hands than you’ve been accustomed to. This will ensure that you have something to do each day.  

File for unemployment on the first day that you are eligible, which is generally the first day you are no longer on the payroll.  Be aware that this can be frustrating if you’ve never had to do it before, and can take longer than you think to get through the paperwork.

While you will want to be diligent in looking for new employment, schedule (and limit!) the time you devote to finding your next work opportunity. If you have the chance to rest and restore before you charge right into your next job, do it. Volunteer, dedicate some time to your side hustle, catch up on medical appointments, commit to your hobbies, or use your free time to take an online course to learn a new skill to level up your resume or just for personal use.

About Lynn Holmes Howe:
Lynn has a B.A. in Psychology from Rutgers and started at Bell Labs in 1983 as a Technical Writer/ Systems Engineer. She transitioned to Human Factors in 1989 and then received her M.S. in Applied Psychology – Human Factors Engineering from Stevens Institute of Technology. She had a long career at AT&T: first as individual contributor, then team lead, then as manager focused on improving the wireless and entertainment buyflow & account management digital user experiences. She was laid off in October 2020 after 37 years. She then joined Glossier as Sr. Manager of UX Research in Feb 2021, focused on digital buyflow UX, and was laid off this past August after 18 months. She lives in coastal New Jersey with her husband, has two adult kids, and enjoys the beach, hiking, reading, NYC, traveling, and volunteering.

Appendix

• A Layoff Guide for Twitter Workers · TWC Newsletter (techworkerscoalition.org)
• What to do when you are laid off: A guide to before, during and after – The Washington Post
• What to Do When You Get Laid Off: 7 Crucial Steps | Career Sidekick
• Guide to Layoffs and Layoff Recovery – Job-Hunt.org
• Tips to consider before turning in your company laptop or phone when you leave your job – The Washington Post
• Life After Layoff: 40 Ideas to Motivate + Inspire During a Job Search | Career Contessa
• Laying off Employees? 10 Tips How to Do It With Empathy (kickresume.com)
• It’s Possible To Humanely Lay Off People With Empathy And Compassion (forbes.com)
• How to Lay Off Employees Script: Do’s and Don’ts – INTOO USA
• Employee layoffs and its impact on employees (thehumancapitalhub.com)
• The WorkPlace Report – Layoffs Can Affect Health, Economic Well-Being of Workers for a Long Time 

Survival Guide to Being Laid Off – Days One to Seven

Day 1:  Take a break, plan, and organize

• Do something kind for yourself: Call a friend and grab a drink or two (or three). Get a massage. Take a hike in a beautiful place. 
• Let yourself feel all your feelings; it’s important to process the emotions associated with this loss; taking at least a day to process them will give you the strength to take the next steps.
• Resist the urge to send out 100 random job applications, and instead start planning and strategizing:  What type of jobs do you want, and why? Think about who you can contact in your network that could introduce you to some companies who are hiring. Think about  anyone at your old company who can give you a testimonial or letter of recommendation.

Day 2:  Get the paperwork out of the way

• Unfortunately, there is a lot of paperwork associated with a layoff. However, it’s one of the most important items for what to do after a layoff, and it’s best to get it out of the way quickly. 
• Get your unemployment paperwork and your health insurance paperwork filled out and squared away. Even if you’ve received a severance package with a lump sum and paid COBRA, it’s a good idea to get this figured out right away. You’ll feel better immediately without these administrative tasks hanging over your head.

Day 3:  Update your resume

• Whether you hope to jump back into the same role as before, or if you plan to make a career change, every job search requires a resume refresh; spend today creating an updated resume that reflects the skills and experience you gleaned in your most recent role. 
• Wherever possible, include data and numbers to show off your accomplishments. This step is never more important that after a layoff, when you are eager to prove your value to a new employer;  If you need help with accurate accomplishments and statistics related to your last job, contact your former boss or coworkers. This is also a great networking opportunity; you can end the call by telling them that you’re actively job searching since you were laid off, and you wanted to ask if they knew any hiring managers or employers who are growing their teams right now.

Day 4:  Write a cover letter

• Take the time on Day 4 to write a fresh cover letter that highlights and expands upon the skills and experience in your resume
• Cover letters, they are a powerful tool in any job seeker’s toolbox;  when faced with candidates who possess similar qualifications, recruiters and hiring managers often turn to cover letters to help them decide which candidate to interview.
• For those who have recently been laid off, a cover letter is also a chance to explain why you are no longer in your most recent role, a line or two will suffice.

Day 5: Start spreading the news

• Once you’ve had a few days to process and take care of the administrative tasks, start reaching out to your contacts and network to let them know you are looking for work.
• Even though you may feel a sense of embarrassment about losing your job, the truth is that layoffs are a fact of life. These things happen and it doesn’t necessarily reflect what kind of employee you were.
• People can’t help you if they don’t know you’re searching, or if they don’t know that you were just laid off. Spreading the word in your professional circles is a great way to succeed faster in your job search, and some studies indicate that up to 85 percent of all jobs are filled through networking.

Day 6:  Account for your availability

• The next big thing to do after being laid off is to plan for how you’ll explain the fact that you’re job searching now. 
• Take the time to decide how you plan to articulate your layoff. Write a short, simple explanation of why you lost your job (“Thanks to budget cuts, ten percent of the company was laid off. Unfortunately, I was one of them.”) and what you hope to do next.
• Keep it brief and positive. Never badmouth or share negative details about your former employer, but do explain the situation and why the layoff occurred (restructuring, financial difficulties, etc.)

Day 7: Use your freedom wisely

• It’s often hard to relax and enjoy yourself during a layoff. Money might be tight, or the stress of the job search could be weighing on you. However, it’s wise to do your best to enjoy this time away from the daily grind. 
• Take advantage of this break to get back in shape, learn a new skill for fun, volunteer, dedicate some time to your side hustle, catch up on medical appointments, commit to your hobbies, or use your free time to take an online course to learn a new skill to level up your resume

A UX Researcher’s Guild Online Event
Moderated by Danielle Green and Jess Vice
This abridged version was written by Booker Harrap. View the full video presentation here.
Session 1 – May 12, 2023

We’ve all heard the buzzword: “research democratization” – the idea of distributing research responsibilities among team members who are not researchers. But is it a strategy that everyone should adopt? Salt Lake City-based researchers Danielle Green and Jess Vice posed this question in an informative and professional discussion. Here are some comments from that event.

“To be good at UX, you need to have empathy and compassion.  If we democratize that, we’ll all do better work and live better lives.”
– Anonymous UX Researcher

For as long as UX Research has existed, the question has been asked, “Who should conduct it?” The term “research democratization” is used to describe the practice of enabling non-researchers to perform research studies. Democratization can amplify the impact of research efforts, and allow teams to meet their research goals. However, allowing inexperienced team members to conduct tests comes with challenges. Below is a summary of the recent “Research Rumble” event where 35 UX professionals were asked to weigh in on the topic. 

In Support of Research Democratization

Multiplying Research Output:

“If you are organized and tactical, allowing you to oversee their research with minimal involvement, you can accomplish more.”
– Research Rumble Attendee

Let’s face it, there are always more research requests than resources. Delegating tactical research activities like usability testing to designers or other team members allows experienced researchers to focus on more strategic endeavors. By removing researchers as the bottleneck, time is saved. Researchers are then free to contribute to high-stakes projects, providing greater business value.

Amplifying Research Impact:

“Research democratization can help propagate the impact of research.”
– Research Rumble Attendee

When stakeholders feel ownership over research findings, it increases user empathy and buy-in. Additionally, stakeholders bring different perspectives and diverse skill sets, which can contribute to well-rounded data over time. This collaborative approach can result in a more comprehensive understanding of users and a more user-focused design process.

Improving Existing Practices:

It’s inevitable–team members outside of the research team are going to talk to users. Rather than spend our energy working against the process, it may be better for researchers to train PMs, designers, and other team members on best practices for gathering valid user feedback.  

Preparing for the Worst

“Research democratization is good planning because if the company has to downsize, it’s good to have some people with training in some research methods.”
– Research Rumble Attendee

This may not seem like a very friendly thought to researchers who want to protect their jobs, but research democratization is a safeguard from a business perspective. Having different team members with research experience ensures that some research capabilities remain within the organization, even when researchers quit or teams are restructured.

Against Research Democratization

Reducing Research Quality:

“Sometimes stakeholders, especially PM’s, just use [research] to validate their own ideas, introducing bias into the research… No data is better than garbage data.”
– Research Rumble Attendee

“Non-researcher research often requires researchers to come in and essentially do their work for them anyways because they didn’t get good data.”
– Research Rumble Attendee

There is a difference between conducting research and extracting valuable insights. One of the greatest assets of a trained researcher is their understanding of the myriad sources of bias, enabling them to design and execute tests that yield accurate data. Their training in experimental design and data analysis means their insights are more likely to reflect reality than delusions based on our assumptions. This objectivism is enhanced by a lack of direct involvement in product design. Conversely, stakeholders are often attached to their designs, feature ideas, or user assumptions. This can lead to research that only confirms what we think we know. Research projects conducted by untrained researchers are at a higher risk of returning useless or inaccurate insights, leaving researchers to repeat the trope: “No data is better than bad data.”

Investing in Operational Tools:

“As an organization gets larger it gets harder to find a balance between systemically coordinated efforts and lean factors.”
– Research Rumble Attendee

“You can only hope that the systems [you put into place] will be utilized by non-researchers doing research.”
– Research Rumble Attendee

To make research democratization successful, non-researchers must have access to appropriate tools and support systems to conduct research effectively and efficiently, and they also have to buy into using these tools and systems (which is not always a simple task). This delegation also creates a workload of instruction and oversight on the part of the researchers. Sometimes, researchers are forced to redo the work of untrained team members, which wastes time. Another important note on this topic is the general and unavoidable difficulty with finding a balance between structured process and agile research, especially in larger organizations.

Recruiting Mishaps:

“Non-researchers may not always have the patience to really recruit properly.”
– Research Rumble Attendee

Impatience, lack of research expertise, and failure to coordinate across teams can lead to recruiting the wrong users or over-recruiting the same users. Recruiting the wrong users invalidates the findings, and over-recruiting the same users can lead to skewed data or annoyed customers.

Sacrificing Other Tasks:

“Where is the extra time coming from?”
– Research Rumble Attendee

One way or another, research takes time. When designers or Product Managers are asked to conduct research, it takes time away from their work. This can lead to situations where, say in the case of a designer, you could wind up with lower-quality designs and lower-quality research.

Endangering Job Security:

Any time we teach other people how to do our jobs, we may be putting our own jobs at risk. More on this below.

Final Thoughts:

Most researchers agree that democratization is not appropriate for high-level strategic research. But for tactical research with lower risk, it might be the perfect fit. It depends on the circumstances, but on most teams, there are non-researchers who are capable of conducting valuable tests (perhaps with a little guidance). If the tools, operations, and oversight are available, you can accomplish more by democratizing research.

As a final thought,  let’s address the issue of job security. If anyone can conduct research, why keep specialized researchers on the payroll? In a recent article by Judd Antin, titled “The UX Research Reckoning is Here” the author states that UX Researchers are losing their jobs because they are not providing business value. He argues that researchers are wasting time on vague, foundational research projects and end up delivering findings that aren’t clearly actionable. Researchers can’t see the forest for the trees – they miss supporting business goals on their zealous missions to understand the user. This isn’t necessarily fair to researchers; this dynamic is often due to systemic issues in company leadership, leaving them cut off from the information and resources needed to inform these high-stakes decisions. Nonetheless, there is a valid point here: researchers need to provide business value. If researchers can focus bandwidth on research that reduces the risk of costly business ventures, it becomes easy to justify in-house researchers as specialized team members. It could be that, by enabling designers and product managers to conduct tests for lower-risk projects, researchers are actually better able to protect their jobs in the long run.

. . .

About Danielle Green (she/her): Danielle is a product and UX professional specializing in research and strategy (high growth, product-market fit). She is an Instructor and mentor with eight years in product, and five years leading teams. As a professor of practices at Claremont Graduate University, Danielle teaches the core courses for the User Experience MA in Applied Cognitive Psychology. She is also the founder and director of the Claremont UXR Laboratory (claremontuxrlab.com), a graduate student lab for UX Research, and has extensive experience in many domains, such as: e-commerce, SaaS, Edtech, Virtual Reality, and hardware.

About Jess Vice (they/them): Jess loves working with people and is curious and excited to understand what drives them to make decisions. Jess offers a deep background in qualitative and quantitative research, user experience best practices, and high-level strategic planning and is particularly good at making meaning from research and using it to create data-informed strategies for creative and development teams. Jess is also intensely aware that the first point of contact is always the internal teams they work with — if a relationship is not built on trust, they know they won’t be a successful researcher and strategist. Jess has been working in marketing and advertising, CRO, SaaS, and product for over 14 years, and is consistently thrilled with how much more there is to learn.

About Booker Harrap (he/him): Booker recently graduated with a master’s degree in UX Research from Claremont Graduate University. He is currently a UX Researcher with Atticus Capital and is working with the Games & Interactive Technology Lab on an emotional wellness program for middle school students using gamified mobile applications. Booker is actively seeking employment opportunities in UX research. For more information,  please send an email to hire@uxrguild.com.

Moderated by Helen Lee Lin as part of the UX Researchers’ Guild
This is an abridgment; view the full video presentation here.
Session 6 – May 6, 2023

When applying for jobs, what one thing are recruiters or hiring managers looking for? The simple, one-word answer is: impact. In this final session of her “Farewell Academia: Hello UXr” series, Helen Lee Lin discussed this all-important topic and how best to frame it in portfolios and resumes.

What is Impact and Why is It Important?

Impact is anything that changed because you contributed or participated in it – any way you’ve managed to move the needle. It doesn’t have to be increasing sales by a certain percentage, reaching a certain number of people, or getting 2,000 users through your product. It can manifest in small ways: whether through administrative, academic, or industry work. 

It may come as a surprise, but impact is one of the things you can control during the job-hunting process. In the last session, recruiter Ploy Hermithian shared that much of the recruiting process is quite superficial. Ploy talked about connecting job candidates to hiring managers based on where people lived or what university they attended. As odd as it might seem, the determining factor as to whether you get your next job could be as random as that, and not the accomplishments on your resume. 

But when you consider that recruiters might take shortcuts like this, it makes you wonder: “What can I do to help control this narrative? How can I ensure they notice those connections?” Presenting yourself well online, and on your Linkedin profile resume, is a huge factor. Make it easy for recruiters and hiring managers to find this information. This will help them see your potential. Because there is a lot of jargon in UX research, recruiters or people working in a business context may not understand what you’re trying to communicate. If you can translate the impact understandably, it will help cement the intention of a line on your CV or resume that much more quickly.

It can also increase your confidence in interviews and, in turn, influence your negotiations for compensation.

How to Talk about Impact

Most of us focus a resume on tasks completed: Made copies; Organized files in the university’s filing system; or Completed forms for IRB approvals, for example. And while these tasks were necessary to complete the job you were hired for, they don’t clearly communicate what was accomplished by doing those duties.

The most common way to show impact on a resume is to Add Numbers – to quantify what you’ve done. For example, if you conducted research, instead of just saying that your duty was to interview participants, include how many people you interviewed. Doing this will show your breadth of experience in how to address unexpected occurrences. 

If you have presented at several conferences, this will show you can convey yourself verbally and reach a broader audience than just within a classroom setting. These numbers will help show how much you’ve done, and for how long, and how many people were affected. This is an easy way to show impact.

For example, instead of listing: “Reported on committee activities, every quarter,” be more specific with: ”Published quarterly report on committee activities to a professional audience of 10,000+ members.”

Another way to extend the discussion of impact on a resume is to go further explain the resulting effect. Ploy Hemrathiran pointed out in the previous session the importance of presenting your research in a way that’s Understandable to a Five-year-old, without a lot of insider jargon. Recruiters are not researchers and may not understand the details of analysis and the different methods you use, nor will they be able to appreciate those subtleties.

For example: let’s say you want to include that you instructed 1,000 students, every semester for three different courses. This would be using the numerical method. A five-year-old might understand the concept of school and that there could be different subjects. But what is that exactly mean? You could simplify it by saying, “Taught students how people interact with each other in a course that lasted for a year.” 

Helen shared that she once worked in a development office as an intern where she was responsible for correctly inputting the addresses of potential donors. That would mean nothing to a five-year-old. But if she said instead, “Contacted potential donors who might be willing to finance the building of an auditorium for all of the students,” that’s something that is going to be more understandable. Of course, don’t use the exact language you would use for a child, because your resume is for adults. But think of noting what the specific effect or goal was instead of leaving it up to people’s imaginations.

Using the previous example, instead of listing, “Reported on committee activities every quarter,” using this method, this could be stated as the following: “Informed colleagues of resources for professional development by regular newsletters.” 

A third method, “Look into the Crystal Ball,” revolves around the idea that expected impact is just as important as impact that has already occurred. To do this, look at the task that was done, and then extrapolate to what you’re expecting to happen. 

Again, with the previous example, “Reported on committee activities every quarter,” could be expressed as “Guided colleagues to funding sources, publication, networking opportunities, potential jobs, and other resources for professional development by writing regular newsletters.”

These examples show the impact or end result at the front of the statement and how this was accomplished towards the end. But you can also reverse the order and have the duty first, and then the impact at the end.

What Does Impact Look Like?

Resumes don’t need to be comprehensive or focused on how you spent most of your time. Nor do you need to list every duty you’ve had at a particular job. Instead, look for ways you have influenced or changed things. 

To find your impact, move past your specific job duties. These can be big or small or even preventative in nature. Consider these impactful tasks below within the following areas: Administrative, Academic, and Industry. Listed below are some examples of how impact might look in these areas. Notice that each statement includes an action verb, putting the focus on the efforts made to complete each task.

• Administrative:
  • Improved an application form to be more understandable, efficient, and logical
  • Pioneered the inclusion of diverse/inclusive answers on demographic questions in the org or team
  • Successfully lobbied for additional budget to _______
  • Prevented delays and saved time with accurate data entry, regular moderation of files
  • Boosted org morale and created networking opportunities by planning social activities
  • Convinced organization to begin using _________ software or vendor
  • Wrote instructions or made a video to help colleagues learn how to do something
• Academic: 
  • Inspired undergrads to pursue a particular career or take on graduate school
  • Communicated complex ideas to layperson audiences on the fly, verbally and in writing
  • Optimized information presentation and application to encourage learning
  • Trained/mentored future scientists, professionals, etc.
  • Led or managed a laboratory of junior scientists, researchers, etc.
  • Collaborated with cross-functional roles to achieve ___
  • Evangelized new tools or methods
  • Introduced different ways of doing or measuring _______
• Industry: 
  • Affected power users, or users whose lives were changed by the product
  • Increased [category of metrics – success rate, satisfaction] rather than specific numbers
  • Provided foundation for design principles
  • Informed product roadmap or feature’s strategic direction
  • Built team empathy for users
  • Uncovered additional questions or research areas
  • Identified opportunities for improvement
  • Shifted the needle on ___ (before and after comparisons)

While these impact statements aren’t necessarily quantifiable, they still have meaning and are impact. People in business or the tech industry might say, “Well, your experience is not the type that we want, and yet you have 15 or 20 years of experience.” Just don’t be the one to discount your accomplishments! If you can help others see your potential, then you already have something to be proud of.

How to Create Impact

To illustrate how to create impact, Helen shared the following story:

“There was an initiative I tried to start at work. We conducted research to decide what it would look like, wrote a proposal, and got buy-in from the levels that were above us. And we actually launched the initiative. But unfortunately, due to several factors, it didn’t take off. One of the two people I was working with had to go on a long leave of absence, and the other one moved off the team so we could no longer execute it together. At the same time, we onboarded some new people and started initiatives for them that superseded the original initiative that I had created. But even so, we did go through all the time and effort to survey people to find out what they wanted, to actually propose the initiative, and then develop how it would look.”

Even though the outcome was not what she had hoped for, Helen still chose to make it a line on her resume and framed it this way: “Designed a peer feedback initiative for fellow UXRs and revamped it based on iterative feedback.” She wasn’t able to take it further and say where it would have gone since they had ended the initiative. But including her efforts as a line item showed the intended impact and the potential for things she could do in the future.

Here are some other considerations for creating impact:

• Don’t settle for the status quo
• Ask ‘Why not?’
• Be one of the first to do something
• Stay at the forefront of trending topics in your field
• Fix something that’s broken
• Mentor or train someone
• Share knowledge, evangelize perspectives
• Write or create something – disseminate it
• Change someone’s mind
• Give someone direction
• Connect people
• Discover what’s missing and fill the gap

Ideas are impact, and following through with them is an even greater way to have increased impact. You can be the first one to do something, and then show everybody else the way, or you can be one of the ones to latch onto it, and then take it even further. In either case, you are contributing to positive impact.

 

About Helen: Helen Lee Lin received her Ph.D. in Social Psychology and has worked in applied research in adolescent literacy and children’s nutrition, and with combat veterans with traumatic brain injuries. She took a career pause while living in Ankara, Turkey for 6 years and then transitioned to UXR in 2018. She volunteered for Hack for LA for six months and broke into industry in January 2021 with a contract at TikTok. She is currently a contract UXR at Meta, working on Facebook Groups.