Hosted by the UX Researchers’ Guild
Moderated by Danielle Green and Jess Vice
View the full video presentation here.
Session 6 – July 26, 2024

Disseminating and socializing research findings is expected of most UX Researchers, but is it really your responsibility to make sure your research findings are driving decisions? At this UX Research Rumble, Jess Vice and Danielle Green from the UX Researchers’ Guild discussed the pros and cons of holding UX Researchers accountable for the impact of their work.

Who’s Responsible for Evangelizing Research Insights?

Danielle kicked off the discussion and advocated that researchers are responsible for how their insights are adopted. At the outset, she clarified that the controversy is not whether researchers should communicate findings to stakeholders, but whether or not researchers should be held accountable in some way.

“The debate is really around whether evangelizing or socializing research findings proactively and convincing your stakeholders that your research findings should be acted upon is part of your job. Said another way, should your performance as a [research] professional be measured based on whether stakeholders adopt your findings or not?,” said Danielle.

Pros of Evangelizing or Socializing Research Insights

Evangelizing or socializing research is a part of many UXR job descriptions, and Danielle advocated that should continue. She summarized four reasons why researchers should be held responsible for convincing others to take action on research findings.

What better success metric?

Being held accountable for the adoption of research findings incentivizes researchers to do impactful work and avoid the “research as a service” dynamic.

How to increase research adoption.

Is it better to do five research projects with minimal adoption or one project with 100% adoption? Her argument stressed that time spent in meetings is valuable if it leads to higher adoption rates of research findings versus conducting multiple projects with minimal adoption, suggesting that fewer projects with greater impact can be more effective.

Team-level performance can address skill issues.

Not everyone can be a master of persuasion and socializing research can be a specialization of a team lead or manager. Danielle noted that not all researchers are naturally persuasive. Instead, she suggested organizations focus on research team-level performance, where some members, like team leads or managers, could take on the role of evangelizing research findings.

You’ll be held accountable anyway.

Whether it is communicated transparently or not, leaders will typically view success in terms of the adoption of research findings (not the amount of good research conducted). The strongest argument presented was that researchers are implicitly judged on the adoption of their findings, especially during times of organizational scrutiny. High visibility from successful evangelization efforts benefits the research team by making their contributions more recognizable within the organization, Danielle concluded.

Cons of Socializing or Championing Research Insights

Jess tackled the negative consequences of holding UX researchers responsible for evangelizing their own research due to the extensive time and energy required. She proposed a specialized role, such as a research evangelist, to handle this task.

“We need research evangelists because we’re not preaching to the converted; first, we must convert…We’re usually at the bottom; we’re not often at the table with high-level decision-makers,” said Jess.

‘But I am le tired.’

Unlike academic research, UXRs are trying to both win people over to the love of research AND get them to consider new data in their upcoming decisions. Individual researchers, especially those with limited support, already have their hands full managing all phases of a research project, from recruiting to analysis. Borrowing from Janelle Ward, Jess argues that adding an impact or adoption metric to measure a researcher’s success, is simply too much.

Consider the org structure.

Researchers are often several tiers away from anyone in a decision-making capacity. Is it right or reasonable to expect them to be able to drive change? Jess pointed out that UX researchers often lack the organizational influence needed to effectively promote their findings. The hierarchical structure of many companies makes it difficult for researchers to reach and impact decision-makers.

Increased cost & time.

Involving more stakeholders in research opens the door for misaligned priorities, rabbit holes, changes of heart, and even more effort on the researcher’s part. Collaborative processes involving multiple stakeholders can be time-consuming and costly. Jess argued that these processes often lead to unproductive discussions about methodology, rather than focusing on the research findings.

Revival meeting feels.

No one wants to be evangelized. How else might we help teams see the value of our work while saving our energy and emotions? Jess challenged the notion that evangelizing is the appropriate approach. She suggested that researchers use their skills in empathy and critical thinking to communicate their findings in a way that resonates with their audience, rather than taking a preachy, evangelistic approach.

Discussions from the Breakout Rooms

Best Way to Communicate the Value of Research

• Researchers must first understand the level of understanding that teams and stakeholders have regarding UX research.
• The challenge is showing value, especially when it doesn’t have a direct output like saving money or avoiding complexity.
• The maturity of the organization plays a significant role in the acceptance of research value.

Strategic Communication and Team Structure

• A strategist at a high level could help align research efforts with organizational goals.
• Aligning incentives is crucial; otherwise, collaboration suffers.
• Team structure and target goals should support integrated collaboration for success.

Champions, rather than Evangelists

• The term “championing” was preferred over “evangelizing.”
• There’s a need for a high-level strategist to align research across the organization.
• Until research is a foundational part of decision-making, researchers must advocate for their own value.
• Partnering with allies in the organization, such as marketing, can help in promoting research findings.

Become the Best Storyteller About Your Work

• Storytelling is a vital skill for researchers to influence others effectively.
• Even if researchers are not naturally persuasive, they should seek coaching to improve their storytelling.
• Effective storytelling can turn research findings into compelling narratives that drive action.

Being Measured on Insight Adoption or Impact

• Measuring the impact of research is challenging, especially when recommendations are not adopted.
• Researchers often need to communicate the risks and trade-offs of decisions, even if they’re not followed.
• The success of research can sometimes be seen in the informed decisions made, even if the recommended actions are not taken.

Research as an Ecosystem

• Research impacts should be considered in a broader, long-term context.
• Spatial and temporal awareness is crucial in understanding the full implications of research findings.
• Communicating these broader impacts can be challenging but is essential for long-term success.

Design Research and UX Research Roles

• UX research often involves usability testing and incremental findings.
• Design research can offer strategic insights and inform product development.
• Both types of research are important and can complement each other.
• Building value through incremental findings can lead to strategic influence over time.

Some Insights Take Time to Make an Impact

• Research findings may not be acted upon immediately but can lead to significant changes in the future.
• Researchers should plant seeds of insights, even if they take time to grow.
• Innovation can arise from past research findings, highlighting the importance of persistence in communicating research value.

Research Findings as a ‘Garden of Insights’

• Viewing research as a garden suggests nurturing insights over time.
• Some findings will yield quick results, while others may take longer to mature.
• This metaphor emphasizes the long-term value of research and the importance of tending to all insights, regardless of immediate impact.

. . .

Danielle: co-founder and President of the Guild; former research and product leader; Professor-Applied Psychology and the Director of Claremont Graduate University’s UX Master’s program.

Jess Vice: an independent UX researcher and strategist; currently the fractional Director of UX at Orca Creative. Jess founded multiple UX teams and mentored UXers one-on-one for more than 10 years.

A Conversation with Author Laura Kalbag
with Lija Hogan, Board of Directors, UX Researchers’ Guild
This is an abridgment; view the full video presentation here.

With the launch of WCAG 2.2 in late 2023 and an increased focus on supporting vulnerable populations, many organizations have been actively working to ensure accessibility to digital experiences and products that enable everyone to navigate the world successfully. 

Laura Kalbag’s book, “Accessibility for Everyone,” is a premier resource for navigating this ever-changing landscape. In the UX Researchers’ Guild’s first Book Group, Laura and Guild board member Lija Hogan shared their insights and knowledge on this timely topic.

Why Accessibility Matters

Laura considers herself fortunate that she was able to learn about accessibility more than a decade ago, as she learned about how to write code and design for the Internet.

Much of her awareness of the need for accessibility, however, came from her background, growing up with a brother who has cerebral palsy and considered himself disabled. Through his experience, she observed how he tried to access the world around him and the difficulty he had with technology. It was obvious to her that designers needed to build platforms that allow for better access to content and information. 

Designers don’t decide who uses their platforms but should try to accommodate as much as possible, and be aware of how difficult this access can be for people, even those who might be using assistive or adaptive technologies to navigate. By being accessibility-focused, technology can bring the world to everyone, no matter their ability. 

Understanding the Accessibility Needs of Users in UX

Many areas of disability need to be considered. One of the primary challenges with making the Internet available to all is knowing how and where to start to make digital products more accessible.

A first consideration is to not think about accessibility in terms of accommodating specific disabilities. Frequently, individuals may have multiple disabilities that interact with each other in a particular way. Instead, think about the needs of each user, what they want, and how they want to interact with what you’re building. That is far more important than identifying the specific disability or technology. Focus on what people need and how to accommodate them. 

One way to do this is to think about four different perspectives on using technology and how to make it easier for people to access information.

(1) The first perspective concerns how people see written information which is more than visually accessing information. Not all people do this with their own eyes – they might have an equivalent experience of “seeing” something by listening to an audio version of the text.

(2) Second is how people hear recorded information. Having captions when a user can’t hear the audio content will be helpful in this area. 

(3) Third is looking at how people find things on a website including how a user interacts with the content, whether with a voice interface, using a mouse, keyboard, or any number of adaptive technologies.

(4) Fourth is how to make information easy to understand.

These four perspectives accommodate a variety of ways that people understand not only the world but also individuals who may have cognitive disabilities or any kind of neurodiversity. 

Breaking it down into those more simple perspectives makes it easier to think about when designing or researching a product rather than trying to cover all of the different possibilities in terms of disability.

Many researchers focus on one area at a time so this might feel counterintuitive. But if you’re testing with unknown users and people in the world, you may not know what they bring with them, as to abilities or disabilities.

Considering How Users Interact with Information

To navigate a combination of disabilities when creating designs, start with a focus, but then be flexible with individual needs that users may bring to the table. Much of that initial focus will be around a product itself or perhaps a particular area or component that you’re designing.  

For example, if the focus is on text, consider more than just making it easy to read and understand. Be aware of how users interact with written text if they have difficulty with fine motor control, which can impact their ability to scroll, drag, or drop components. Users with these difficulties will need an alternative type of interaction for these actions.

It’s easier for the focal point to be on what you’re building. Equally important is how people will access that information. With accessibility in mind, you can explore and design different and better ways to build products.

How to Design for Edges Cases and Those in the Middle

Designing for the edge cases can bring in the middle by default. Focusing on improving a specific element of an experience, whether it’s text, audio, or video, ensures that it’s accessible to everyone. 

When people first find out about accessibility, it may be tempting, for example, to adopt the mindset of creating an entirely different version of a product that is just for screen readers for those who are blind or have limited vision. But doing so implies making a choice for these users, assuming they need and want a different experience, and not considering that they may have their own way of interacting that you don’t necessarily understand yet.

When building better products for accessibility, it’s best to be as accommodating and flexible as possible and not rely on your assumptions. Strive to provide many ways for people to interact with what you’re building in the way they choose.

Pros and Cons of Accessibility Overlays

What is an accessibility overlay? If you browse the web as someone who has accessibility needs, it can best be described as an accessibility widget. It might be a circular button hovering around on a website as you’re scrolling that has a universal human icon on it. This widget is a plug-in that users can add to a website that creates an overlay to automatically make the entire product more accessible. You plug it in, and it does all the work. Because it is automatic, it may seem like an instant solution for all accessibility needs.

The problem is accessibility isn’t like that because people’s needs vary depending on the product. The accessibility is not done by a human, but rather by software that’s doing its best with the limited information it has access to. Your product will also have its own quirks and underlying code with which these widgets interact.

Another issue with an overlay is when users indicate that they want to use it, they are identifying themselves as having accessibility needs or as being disabled which can have privacy implications. Many products of this nature have third-party scripts that send data about the people browsing the sites to various served parties. For individuals who may not want to identify as disabled, this can have repercussions, affecting public services and other privacy concerns.

Often, these overlays don’t help users access website information. Designers who believe these overlays solve the problem may overlook crucial learning and understanding about accessibility and how they can make their product better. It’s best to view overlays as a temporary solution if there are specific needs that they can improve. But be sure they are solving those problems. 

Accessibility is a Right, Not a Privilege

Developing technology with accessibility in mind opens up the world to more people than ever before. Access to products people use is a vital part of today’s infrastructure like the internet itself and how individuals participate in their social lives much of the time. 

Included in this is access to public services, which is now primarily done through the web. But if these services are not accessible to some people because of a disability, that’s a problem. Such access is a human right, not a privilege for the few, that needs to be available to all.

Release of WCAG 2.2

WCAG is the Web Content Accessibility Guidelines and was introduced more than 25 years ago. It has gone through various iterations, with version 2.2 released in late 2023.

Version 2.1 was quite a departure from 2.0 in that it changed the approach of the guides to be less prescriptive about technology and more about how people use it since technology is changing all the time whereas the way people use it is changing less quickly. WCAG 2.2 has added more guidelines on specific use cases that come up in people’s experiences and areas that they might have difficulties with.

The new guidelines about dragging, for instance, is if you have an element that requires being dragged in your interface you must also provide a way of doing it that requires a single pointer interaction, by pointing and clicking with a mouse or using a keyboard. 

Another example is making the clickable area for a link larger, even if the text is small. Doing so allows individuals who use a finger or a mouse or struggle for precise control to click that link as easily as anybody else. Data from behavioral analytics or heat maps can show if such areas need to be expanded. Therefore, if you have these components in what you’re building, make sure they are accessible to people who might struggle with fine motor skills.

The guide can seem overwhelming, but with this recent update, the language is easier to understand. Use it to get an overview of the needs people have as well as to understand accessibility in general. 

But how about affecting change with stakeholders who may not understand the need for accessibility? It all depends on where you are in your organization. If you are a gatekeeper you could point out specific points that need to be more accessible. 

But if you’re trying to do your best work where you are, inform yourself about potential accessibility issues around what you’re building and then work on ways to make improvements. You can still infuse things into your day-to-day work even if no one is telling you what you need to do.

WCAG Standard Levels

Most countries now have regulations around web accessibility based on WCAG guidelines. There are different level standards in WCAG: A, AA, or AAA, where AAA is the highest level of accessibility, and A is the lowest. Conformance at the highest level is also conformance at the lowest level. So if you base what you’re building around the AA standard then you’re usually in a good area. The updated 2.2 guidelines have checklists for all levels and are a great reference of how accessible your experience is, even if you’re not getting a professional audit done by a company. 

See https://www.w3.org/TR/WCAG22/ for more information.

Learning from Users to Understand Accessibility Needs

One of the best ways to know if your website is accessible to people with disabilities is to have it reviewed by people with these limitations. Companies such as Fable (makeitfable.com), Easy Surf (easysurf.ca – located in Canada), and Knowbility (knowbility.org) are all organizations built to do just that.

Discover best practices for remote testing with people who are using assistive and adaptive technologies. While you might focus on doing research with people who are using screen readers, they are often using other assistive and adaptive technologies.

As part of every test, consider asking about what technologies individuals use. There may be something you specifically want to test, but asking this question will inform you about other strategies people use to engage with information on the web. Self-reporting on the part of users can be extremely helpful in understanding how to meet their needs and is a great tool to take into any kind of form of accessibility. Listen to people and don’t make assumptions about their disabilities or how they access information. You may be the expert on research and design, but they are the experts on how they access what you create. Learn to trust what people tell you.

AI and Accessibility

AI has improved many aspects of accessibility including voice-activated interaction on a website to more accurate automatically-generated captions during a live Zoom call. Here again, be aware of privacy considerations. If someone uses their voice to communicate with a device, where else is that recording going without their knowledge? But overall, AI can be valuable in simple tasks and make navigation more accessible. 

While AI may do a good job, don’t assume it does a great job. Automated captions, for instance, need to be checked by a human, particularly with technical topics or areas that have a lot of jargon associated with them. That’s where AI can fail. Also, be aware that AI tends to be biased towards English speakers and male voices and is better at generating captions when speakers have a North American accent.  

A recent article proposed that AI could be used to generate an interface suitable to any particular user, specifically those with disabilities. Users could explain their disabilities and needs, and AI would generate a UI for them. While this would be wonderful if it worked, accuracy will be a huge issue especially if you want reliable information. Think about someone looking for information about a health issue, an area where getting it not quite right could have a disastrous effect. You don’t want AI guessing at it. 

This goes back to the idea of having a specific interface for one person based on a specific disability instead of something that is accommodating to a wide variety of needs, allowing people to choose how they want to interact with it. For this to be effective, users would need to be singled out based on their disability not making it a viable solution. There are solutions that are working and those shouldn’t be thrown aside and replaced by the latest and greatest abilities of AI. 

While AI is improving, believing it can support complete personalization for an individual is unrealistic right now. So what can be done until AI can deliver on that promise? Continue to create the best products with accessibility in mind that you can between now and whenever AI can provide better solutions. 

What Disabilities Need to Be Addressed in Designing for Accessibility?

The Centers for Disease Control (CDC) releases statistics each year on the prevalence of different areas of disability including mobility, self-care, cognitive disability, vision, and hearing. In the past, mobility has been the most prevalent disability in the United States. But in 2023 for the first time, cognitive disability, defined as serious difficulty concentrating, remembering, and making decisions, moved to the #1 spot. A hypothesis is that the inflammation from COVID-19 is negatively impacting the cognitive abilities of younger people. 

Memory is an area people don’t think about in terms of accessibility, which brings to mind the importance of the humble breadcrumb, a reminder of where users have been and where they currently are on a website. Something as small as this makes information more accessible for people with memory issues. Other considerations are to incorporate shorter instructions and more repetition on writing tests. 

Making instructions and navigation easy to understand in terms of text and labels will benefit those with cognitive difficulties as well as people who may not speak the language the interface is in. It also benefits people unfamiliar with terms, especially technical ones, long words, etc. Using plain and simple language is one of the easiest ways to accommodate the needs of a wide variety of people.

___________________________________________________________

¹ https://www.cdc.gov/ncbddd/disabilityandhealth/infographic-disability-impacts-all.html

Barriers to Accessibility

Another question to consider is if people avoid using products because they assume they’re not accessible. There’s no such thing as a hundred percent accessibility. There are too many varieties of needs that overlap in unexpected ways and often, those needs can be in conflict with each other.

Barriers pop up for everyone from time to time. such as when using a tricky interface and an unexpected bug appears. Most people know which tools work for them and will return to these products and services and become loyal to them. But they will also reject something if they have a bad experience or can’t use it. 

If you believe that disabled people are not using your product, it might be because they can’t. People won’t buy your products If you make it impossible for them by putting up barriers.

When to Implement Accessibility Features

Across the development lifecycle, does it make sense to start at the low-fi prototype phase, do it at the high-fidelity phase, or wait until you’re in QA?

The answer is, you can start at any phase. However, the least expensive route is to start as early as possible, even before low-fi. When you’re considering what to build, and what your features and goals are, that’s the time to start thinking about accessibility because it will impact everything you do.

This is why Laura titled her book, Accessibility for Everyone, because it touches every discipline that will be working on a product or has a role within accessibility. Thinking about accessibility and how each stage could be affected is a mindset not unlike thinking about your business goals through all of the different life cycles of a product. 

It will always be more expensive to change something after it is built. If you haven’t thought about accessibility early then you will need to write different code or design a slightly different way of interacting with it, and all of these things are expensive changes to make.

Accessibility and Short-term Disability

What about people who experience temporary disability? Is the design process similar for these individuals as for those with permanent accessibility needs?

This goes back to the idea of making something easy to see, hear, use, and understand. Someone might not have a disability related to motor skills, but if they broke their arm, they would not be able to use things in the way they could before. Suddenly they have an accessibility need. It’s not permanent, but no less inhibiting. 

Laura shared her personal experience of suffering from severe RSI (Repetitive  Strain Injury) in her arm and hand. “On the days when it’s awful, I can’t use a trackpad or a mouse so I have to use a keyboard. I’m suddenly thrown into a world of keyboard accessibility.”  A quick test of a web product is whether you can browse it only using a keyboard because that’s the reality for many people with motor difficulties. 

Impact of Design Systems on Accessibility

Should you incorporate accessibility into your design systems? That’s one of the coping strategies many organizations use to scale accessibility. Design systems can have a positive impact on providing a framework for accessibility in a digital experience.

Laura remembers talking to someone who had worked at Twitter who encouraged accessibility by designing a process that when something failed an automated accessibility test on the software side, it would result in an error which prevented further work until the error was resolved.

This isn’t just about showing people how to do things right but also about educating people on how to understand what the accessibility considerations are for this particular area of what is being built. Rather than accessibility being this amorphous big thing that’s difficult to understand and grasp, break it down into components and focus on the details around a smaller section, making it much easier to understand. You may be working on iterating one component, but as you bring that into the product, you’ve made it step by step a little bit more accessible. 

Education is a big part of making accessibility a reality at an organization and benefits consistency and uniformity. Referring to cognitive or usability difficulties in general, you can learn how to design accessibility into one area and then apply what you learned to other areas to improve the overall product.

The Challenge of Incorporating Accessibility

Don’t be afraid of incorporating accessibility into your products. There are many ways in which accessibility can be intimidating to people. If you’re not disabled, you may feel intimidated by doing or saying the wrong thing or interacting in the wrong way. Do your best and take opportunities to speak to and respect what disabled people share about their experiences. If you can learn what is relevant to what you are working on, a little bit at a time, you will build up a breadth of knowledge very quickly. Your effort will provide breed greater creativity. 

Along with the idea of incorporating accessibility is the question of whether accessibility research should have a distinct role or should all researchers learn how to do research alongside people with disabilities. It is evident that all researchers should include people with disabilities in their research, especially when trying to represent and understand the needs of a widespread audience.

A former student of Lija’s has the job title of “Accessibility UX researcher,” the only person she is aware of with that distinct title. Based on budget alone, most organizations won’t hire a few people whose primary responsibility is UX research, much less someone to address accessibility needs. 

However, with AI becoming a bigger issue, organizations will need more people with that skill set to accommodate a broader range of disabilities. The idea of needing research specialists in this area is likely to grow in the future. These specialists can then guide the generalists to learn how to do best practices of incorporating accessibility in research.

. . .

​Laura Kalbag is a British designer living in Ireland who has worked professionally in the tech industry for nearly 15 years. As someone interested in the entire user experience, she uses writing, speaking, and web development to support her work, with a particular focus on accessibility and privacy. Laura’s book, Accessibility For Everyone, was published by A Book Apart in 2017, and she has given over 100 talks at events worldwide.

Lija Hogan has specialized in usability and user needs research, design, and building UX teams. She teaches qualitative and quantitative UX research methods courses at the University of Michigan – Ann Arbor, on Coursera, and LinkedIn Learning. Lija’s current focus is inclusive research practice, as a Principal Customer Experience Consultant at UserTesting.

*A Job Search Council (JSC) is a mutual support group of peer job seekers who agree to search together using a specific, well-advised process laid out in the book Never Search Alone: The Job Seeker’s Playbook by Phyl Terry. The Guild is forming JSCs because we believe this is an effective program for researchers to define their career objectives and priorities and find support. The Guild is in a unique position to form groups of UX researchers from our community. We believe there is a benefit to career planning with other researchers. Most people join because they are unemployed, but if that’s not the case, you can still use this opportunity to prepare for a future transition.

How does it work?

• Register to join a JSC.
• The Guild will form groups of 6 researchers with similar seniority and employment goals.
• We will notify you when you have been placed in a group. If we don’t select a volunteer moderator, the group will select its own.
• Your moderator will use premade templates to run the meetings. Groups can decide how often and how long to meet. Generally, groups meet weekly for 10-12 weeks.

How does the matching work?

We group people with similar levels of tenure in their careers, employment status (unemployed “fast-seekers” vs. employed “slow seekers”), goals (seeking full-time employment vs. independent consulting), and roles (individual contributors vs. leadership).

What is the difference between joining a JSC with the Guild and Phyl.org?

The Guild creates groups of similarly situated researchers and leaders. Phyl.org creates councils from different disciplines (but usually within product). If you prefer a group with mixed disciplines, apply to join a JSC here: https://www.phyl.org/join-jsc

Participating in a JSC requires approximately 80 hours of work over 3-4 months, but it’s all work that you will do on yourself and your positioning, work that directly impacts the quality of your search, your confidence, and your career. Essentially, you must be willing to:

1. Commit to a time-tested process for searching that centers on articulating your market value. You’ll be tempted to spend all your time doing things “the traditional way,” but we know that, especially in this market, that is not effective.
2. Attend meetings with your team to give each other support and feedback on the exercises you’re working through.
3. You may also choose to join the JSC Slack community at Phyl.org to find connections with like-minded folk, get feedback, and discover new resources. Once a Guild JSC group is formed, the moderator joins Phyl.org, and then the members join if they are interested.

Don’t sign up unless you are committed to doing your work, attending each council meeting, and supporting your peers.

Hosted by the UX Researchers’ Guild
Moderated by Danielle Green and Jess Vice
View the full video presentation here.
Session 5 – February 21, 2024

Do UX Researchers need in-depth domain knowledge to do their jobs effectively? Or can researchers deliver valuable insights without understanding a product’s details or industry jargon? Danielle Green and Jess Vice dove into the pros and cons of this important topic in their most recent Research Rumble.

Domain Knowledge in UX Research

Most people would agree that if a particular project involves a technical or jargon-heavy product, a researcher who knows the industry space will work more efficiently than one who doesn’t. But are there benefits hiring managers are not considering that come with a researcher who is new to a field or product? 

In the top 20 industries that employ UX researchers, such as those involved with hardware and software experience, healthcare, e-commerce, and finance, there is an overwhelming preference for domain knowledge.

Pros of Hiring Domain Expert UX Researchers

This issue of prioritizing domain knowledge might be one of the most important factors hiring managers use in deciding who to hire. In addition, some managers are willing to trade years of experience and expertise in certain methodologies, in favor of industry knowledge. On this pro side, then, what benefits do hiring managers get from domain knowledge that keeps them seeking specialized researchers? 

Domain Knowledge Improves Operational Efficiency

The number one benefit is operational efficiency. A UX researcher who knows the internal jargon or team structure of a company in a particular industry will be able to quickly pick up internal operations at a new company within that same industry. If you worked for a health insurance company before, for example, you know there’s going to be a different team for the website, the member portal, and the provider portal. You have this mental model of how teams may be structured and the different products that may already be at play for that organization.

Domain Knowledge Increases Validity of Data Collection

The next is the validity of the data collected, especially qualitative data. If you know some of the language the user base uses, you can create better surveys and interview experiences and therefore collect better data. Qualitative interviews are also a great example here – a domain expert won’t have to ask participants basic or clarifying questions.

On the flip side, if you don’t know what feature the user is talking about, it’s difficult to ask probing follow-up questions to determine if that feature meets that person’s needs. You can take the naive approach and ask the user to define things for you. But in doing so, you risk losing important details they might provide if you could show that you understand their context. 

Domain Knowledge Improves Qualitative Data Analysis

Domain knowledge can produce better outcomes when it comes to analyzing qualitative data. For example, if you don’t know that two terms are related, you could fail to prioritize a finding or report on a theme that’s more prevalent than you would have otherwise thought.

Danielle had an experience with this when the name of a competitor for a company she was working for was also the name of an app in their suite of products. It took her longer than she would have hoped to realize that users were talking about turning to the competitor product rather than another application in the suite of products, all because she lacked domain knowledge.

Domain Knowledge Promotes Competitive Advantage

Suppose you have extensive experience with a user base or a product space. In that case, you’re able to detect indirect competitors and other market-level issues more quickly than someone without this knowledge. A classic example of indirect competitors is mattress companies, whose biggest competitors are melatonin and other supplements for sleep. So, if you’ve already worked for a pillow company, then you might already know about some of those types of competitors when you go to work for a mattress company. 

Domain Knowledge Enhances Empathy for Users

The next one is empathy for your users. It takes time to build empathy and get to know someone before you step into their shoes. If you have spent time with that user base before, you’ve already begun to cultivate empathy for them.

Danielle worked in telecom years ago and did a lot of field research for contact centers. She realizes now that she is a bit kinder to every customer service rep she has spoken to on the phone since. Developing that empathy for a user base will translate to other industries or companies. 

Domain Knowledge and Its Effect on Data Patterns and Best Practices 

Sometimes researchers with a lot of experience in a particular industry have seen many data patterns and design best practices which can give them a good intuition or first guesses about things. Danielle shared the following example: 

“I once worked at an e-commerce company and the VP they hired over product had worked for many online retailers for over a decade. While he hadn’t worked in anything in our unique situation, he was familiar with much of what we were doing. He remembered a lot of their experimentation results, such as pagination, breadcrumbs, or when to pop open another tab when you click on a product. Some things that worked for previous companies did not work for ours, but many of them did. His knowledge of the space allowed us to quickly get some growth on the team by implementing those changes that were not obvious to us.”

Cons of Hiring Domain Knowledge UX Researchers

Jess Vice learned about the connection between domain knowledge and UX research on their first job interview with a marketing agency:

“The hiring manager was impressed with my writing skills but added that I didn’t know anything about marketing. Thankfully, he said, ‘But, we can teach you that. You seem like you can learn.’ By taking a chance on me, this sort of defined the trajectory for my career.”

How Much Domain Knowledge Makes You an Expert?

The first thing to consider is how to define an expert in any given industry. When is enough knowledge enough, and how do we know what expertise means? 

Malcolm Gladwell made famous the 10,000-hour approach to expertise even though the original study showed that that figure was an average of hours of practice to reach domain expertise, not a precise number. For some people, mastery came after 25,000 hours, whereas others reached that level of expertise with just 4,000 hours. All this is to say that there is no agreed-upon measure or standard to say what expertise is. 

Some people walk boldly into a job interview claiming to be an expert, then prove themselves wrong in short order. It’s important to remember that practice doesn’t make perfect, but perfect practice makes perfect. In expertise, are we talking about quantity or quality, and how well or broadly someone learned a domain? This will not be evident through the interview process alone. 

There has been research that shows after about three years of doing something, we get bored. It can feel like mastery because our brain says, “Enough – I’ve had it. I know everything. I want to move on.” Is it mastery or boredom? It’s hard to tell at times.

If domain expertise cannot be clearly defined, perhaps it should not be a critical factor in the hiring process.

Domain Knowledge and Its Effect on Analogous Thinking 

Interestingly, the last point of the pros is the second point of the cons, showing that the analogous thinking experience in a different field can be a benefit and a con. In the example on the pro side, the new VP had high-level pattern observation across a series of industries that were replicable or applicable to what was being done. Analogous thinking from someone who has no direct experience, but sees patterns clearly may drive innovation and discovery.

However, analogous thinking in the way of a beginner’s mindset is not necessarily beneficial or something to promote. Doing so can greatly delay processes and project timelines rather than pulling on previous experience to find patterns and threads for a thoughtful analysis.

Domain Knowledge and Risk of Decreased Collaboration

Expecting researchers who come into an organization to also be domain experts can create knowledge silos and potentially decrease collaboration. 

Look at it this way: when a company expects an individual researcher to be the expert resource for how a process works, and then asks them to also make continuous discoveries, that puts tremendous pressure on their shoulders. 

Jess shared an experience they had when working in commercial real estate:

“I didn’t know anything about brokerages and how to broker a mortgage on a 700-apartment building. That was way beyond my domain. However, I had bought a house, had been through paperwork for a single homeowner, and had a frame of reference. When I talked to the brokerage team and demonstrated that I had analogous experience in something similar, they were much more willing to open up to me.”

Jess had heard that other researchers approaching the brokering team had negative experiences. But by showing a little bit of ignorance and analogous experience, Jess was better able to collaborate with them. 

“We ended up building a cross-team dependency where they welcomed me coming to them to get the right answer, rather than me assuming that I knew how the industry worked and making best guesses.”

Domain Knowledge: a Barrier to an Exploration Mindset?

Not having domain expertise can encourage an exploration mindset. Think about our previous example of a person who claimed to be a domain expert in an interview: when put in a room with clients, they are less likely to say, “I don’t know,” or to ask for time to explore and to understand more thoroughly. They’re more likely to say, “Here’s the answer; this is what we should do.” They are trying to cover up their lack of knowledge and confidence.

There are many reasons people might respond this way, but do we want someone who comes in with full confidence even if they might be wrong? Or do we want someone who admits they don’t have all the answers but is willing to do the hard work to discover the best path moving forward? 

A key attribute everyone appreciates about a UX researcher is curiosity. When we ask people to be domain experts, are we encouraging them to be curious or to rely on what they already know? In this way, requiring domain expertise might push researchers in the direction of too much knowing and not enough curiosity.

Risk of Focusing on Details Rather than People

As Danielle shared, empathy can be a point for requiring domain knowledge. But this same attribute can be a point against.

When we have domain experts who know something inside and out, they may lose sight of the people they are trying to serve. UX research is all about understanding people and remembering the humans and environments affected by the work being done. 

Jess recently pitched a client on a big project explaining that they needed 6-8 weeks to do research. The liaison representing the client responded by saying, “The CEO says he’s been doing this for 20 years; you can just interview him.” While he may have been doing this for 20 years, he was not an accurate representation of all the people involved in the industry. He is one person in one industry in one particular company with a personal bias and limited experience. Trusting one or two people as authoritative domain experts, as knowledgeable as they may be, can harm the process rather than help it. Researchers must never forget that every field has a mix of people who all have valuable insights.

Factors that Affect the Need for Domain Knowledge

So, do UX researchers need domain knowledge? The answer—a favorite for the industry—is, “It depends.” Whether domain expertise is needed depends on the size of the company, industry, and team. Position level also needs to be taken into consideration. Are we talking about juniors? They probably don’t need domain knowledge and expertise. What about senior directors who are leading others? They probably should have more. The goals of the project should also be looked at. Do we want specificity or speed? Many compromises need to be made and as hiring managers, thinking about the level of expertise needed in a role has a lot of peripheral context that should also be addressed.  

As a researcher who is considering entering an industry, it’s important to think about the context you’re pitching yourself into. Do you need to be a domain expert, or can you make a case for learning once you’re hired? Do you want to create something brand new or are you trying to keep a thing on its wheels and going? There’s a lot of context that defines the need for domain expertise. 

Discussions from the Breakout Rooms

Benefit of Pairing Domain Knowledge with Analogous Thinking

Perhaps it’s best to have two people working side by side: one who is a domain expert and one who brings cross-domain knowledge and more open-mindedness and avoids those biases.

The idea of diverse teams is a compelling one. Together they can balance and support each other. A hiring manager needs to explore who they already have on their team and where the gaps are that they need to fill. Maybe it’s someone who thinks in patterns and across industries and excels at diverse thinking rather than someone else who knows all the details about a particular industry.

In any industry where you are not a domain expert, it’s important to communicate your role as a researcher upfront so expectations are clear. You were hired as an investigator, not to know all the ins and outs of the industry. As Jess once shared with biotech scientists on a job: “I’m going to do my work as a researcher in this space. I need you to be the science experts and tell me when I get it wrong.” 

Building a diverse team is not easy, but it can be very effective. It will require thinking about the organization or the UX team, rather than focusing on any researcher.

In-House Researchers vs Independent Consultants

If domain expertise is required to be an effective UX researcher, then are we also saying that UX research consultants are not effective, or less effective than having an in-house researcher? It may be a rhetorical question, but it still needs to be examined.

Organizations such as the UX Researchers’ Guild can help solve that problem by connecting clients with domain-expert UX researchers, thus avoiding the need for an in-house full-time person. Independent researchers with specific domain knowledge are out there and can often do the same job more effectively and at a lower cost than an in-house consultant.

Domain Expertise as a Gatekeeper

There are situations where domain expertise would be ideal, but if domain expertise is the primary factor being used to choose job candidates, there’s also a high risk that the hiring practices are not looking at a broad, inclusive pool of applicants. And there could be missed opportunities when hiring for domain expertise over character, work ethic, and thinking style.

In contrast, while having domain knowledge doesn’t mean a researcher will necessarily use that expertise in their job, it does demonstrate that they can take on technical and complex topics. And that could be very attractive to a hiring manager. 

Domain Expertise, or Lack of It, is Temporary

It’s important to understand that having, or not having, domain expertise is a temporary condition. On one side, a researcher will naturally learn and grow in an industry the longer they are involved in it. And they will learn about that industry from the inside, where it will be most helpful to the client. 

Domain expertise being a temporary condition can also go in the other direction: How long are you a domain expert in tech if it’s changing every day? Then the idea of using domain expertise as a gatekeeper for hiring can be risky in ever-changing industries by focusing on this one aspect a manager feels an applicant needs to have.

Benefits of Continual Learning in UX Research

The willingness to continue learning is critical, and that mindset is independent of someone’s level of expertise. It’s also important to not get complacent. Just because a person has worked in IT for several years doesn’t mean there isn’t more to learn. Having a passion for an industry can demonstrate a willingness to continue learning about it, but generalized curiosity about a new industry can also be a benefit to companies hiring researchers. 

It would be beneficial to all, both potential candidates and the research team itself, if managers approached the hiring process by asking themselves, “Am I offering a growth role or do I want to hire someone to come in up to speed already and get it done?” Sometimes speed needs to be sacrificed in the interest of the depth that could be brought to the team. 

Value of a Diverse UX Research Team 

Crossing domains or understanding different domains is especially important when tech is involved. Any industry is impacted by changes in technology. Managers must be aware of these changes and how their work will be affected. This is where a non-industry domain team member can be especially helpful. Being tech-savvy in an ever-changing world is crucial to stay on top. Such a team member may need time to get up to speed on domain knowledge, but their expertise in technology will be invaluable and will provide the needed balance.

But how can researchers convince companies of the value of a diverse research team and the results they can bring? That is another conversation for another day. However, it is an important element in the ongoing world of UX research.

Gaining Domain Knowledge After Being Hired as a UX Researcher

One aspect companies can consider as they look at hiring researchers is this: how is knowledge documented and shared with those who don’t have domain expertise? Do these new hires have available resources to educate themselves and get a head start on becoming a domain expert inside the company once hired? The last thing a new employee wants to do is guess every day what’s happening and where to find needed information to do their job. Having a go-to resource will allow researchers to gauge their domain knowledge level and ask informed questions about those information gaps. 

Being Your Own Advocate to Be Hired as a UX Researcher

Because of the current “embarrassment of riches” situation, domain knowledge is now required more often in the job market where there are so many applicants to pick from. But that’s not necessarily a good thing. A generalist can pull in insights from other domains. Make sure you self-advocate and articulate your skills and talents to people who might not understand your background. If your previous experience is not UXR-related but allows you to understand a client’s perspective, don’t be shy about sharing that information. Tell people what to hire you for. 

If you’re a learner and want to be a domain expert, make that clear in the hiring process so managers can see your self-motivation. If you already have domain expertise and want to stay in that space, great. Let them know. Telling people exactly what to hire you for is a thoughtful approach, especially right now with so many choices to be made.

. . .

Jess Vice: UX researcher and strategist; ex-Strategy Director at Struck. Jess also coaches people who feel like they need to get unstuck and educates on big life topics like critical thinking and making decisions.

Danielle: co-founder of the Guild and Research Director; UXR, UXRM, Professor-Applied Psychology, and the Director of Claremont Graduate University’s UX Master’s program.