A few weeks ago, Jess Vice and I hosted a UX Research Rumble on using AI in UXR. We had a great discussion with 100 other UX Researchers about the pros and cons of using AI tools in the workplace.

Attendees expressed interest in learning more about using tools like ChatGPT for UX Research. Many researchers want to know what AI can and cannot do for them at work.

To encourage discussion around this topic, I’m sharing this document I created for my UX graduate students. Think of this as a cheat sheet to give you a sense of what ChatGPT can and cannot do effectively (as of January 2024) when it comes to UX Research.

Download Using AI in UX Research (PDF/2.12 MB)

About Danielle Green (she/her): Danielle is a product and UX professional specializing in research and strategy (high growth, product-market fit). She is an Instructor and mentor with eight years in product, and five years leading teams. As a professor of practices at Claremont Graduate University, Danielle teaches the core courses for the User Experience MA in Applied Cognitive Psychology. She is also the founder and director of the Claremont UXR Laboratory (claremontuxrlab.com), a graduate student lab for UX Research, and has extensive experience in many domains, such as: e-commerce, SaaS, Edtech, Virtual Reality, and hardware.

Organizations and their UX research teams are hard at work finding the way forward in this changing economy. While many teams have downsized, the most mature organizations have invested heavily in differentiating their product or solution offerings by redoubling efforts to connect with customers to understand their evolving goals, needs, expectations, and gaps. Because of all this, freelancers play a key role in this space in the following ways:

Freelancers help organizations fail smarter and faster – and hopefully less frequently
Organizations that want to try new ideas quickly often rely on freelancers to provide extra momentum and time to derisk solutions and course-correct more quickly. Freelancers bring a wealth of experience and skill to figuring strategies that align with organizational goals while driving innovation.  

Fractional leadership: growing the practice one hour at a time
Startups and immature companies may not need the full-time services of a leader. Bringing in fractional leadership enables organizations to bring focused energy and expertise to accelerate UX research maturity. They can map out the territory, build a foundation, and provide guidance on bringing on additional resources when the right opportunities surface.

Freelancers are the global talent pool
Remote work is transforming how organizations build teams and work. Global talent with just the right combination of location, experience, and availability enables organizations to work nimbly and teams that can pivot as needed.

The current circumstances offer a unique opportunity to continue to evolve the practice of UX research. The winners coming out of this downturn will be those organizations that focus on the right problems alongside their customers. UX research freelancers are the right people to provide data and insights that drive better decisions.

I predict that AI will not take over UX research jobs but will instead become the assistant we’ve always needed. There is precedent for this.

The camera was invented in 1837 with the daguerreotype by Louis Jacques Mandé Daguerre. The art world, which was largely relied on to record historical events at the time, panicked that it would take away their jobs. What happened afterward was an explosion of artistic expression that is still rocking the art world today. You see, the camera freed artists from having to record historical events and allowed them to try different styles and themes—abstraction, surrealism, expressionism, etc… The camera also became a tool for artists and photography, a new art form on its own.

AI in art

To create images from prompts, AI generators rely on databases of already existing art and text. These comprise billions of images that have been scraped from the internet and artists aren’t happy about it.

Query: DaVinci style portrait of Taylor Swift

Given my brief Taylor Swift test above, I don’t think artists have anything to worry about yet. It’s important to remember that these AI-generated images are riding on the coattails of real artists. Any perceived value is based on the value that the artist brings—not the technology. A DaVinci is valuable because DaVinci’s “hand” made it.

AI in UX Research

So, what would it look like after the AI panic is over for it to become a tool for the UX research world?

I think AI tools could shine in research doing analysis on the research’s raw data. I know some UXers on LinkedIn have tried this recently and it failed miserably. It’s not there yet. I hope that in the next year or two that research repositories will offer accurate and efficient AI-contextual tagging and maybe even a rough draft of the findings by testing objective.

My company uses Dovetail and I recently met with some of their reps. They anticipate having AI-contextual tagging sometime next year. I expect their competitors are doing the same.

Short-term Cons

There are pitfalls in the meantime. We are in for a rollercoaster of bad AI results, bad actors, and a lack of regulations for the short- to medium-term. It depends on how fast governments get over the politics and create real regulations with enforcement. It could take several years.

Certain parts of the business world think they can replace UX research and real users and instead generate findings from an AI query—basically removing both the user and the UX researcher from user research. This will be a failure on all counts.

• Not our users
• No context for findings
• Diminishing returns as AI-generated findings populate the internet

I’ve said many times that AI-generated UX research reports that don’t come from real users and real researchers aren’t user research. Period. Full stop. Any company that goes down that path does so at its peril. Just as AI is riding the coattails of artists, it is also currently riding the coattails of UX researchers. Flooding the internet with AI-generated “research”, creates a lot of noise that will be the downfall of companies with poor AI research practices.

My advice would be to hang in there and provide real value to your business stakeholders. Make them see that you are providing them with the strategic knowledge they need to succeed. Good luck!

Author’s Note: This article was written by an artist and a UX strategist/researcher without AI-intervention!

About Jillian: Jillian Hudson is NN/g UX certified in Research and Management. She worked as a UX Product Designer/Usability Expert for American Airlines and as a UX Strategist and Researcher for Bank of America, and Wells Fargo Bank. She is currently the Principal UX Research Strategist at RXO. Contact Jillian through the UXr Guild Slack channel.

Guild Origins, Purpose, and Demise

With origins stretching back 2,000 years before the birth of Christ, guilds are best known as medieval associations of craftsmen or merchants who came together for mutual benefit. Typically guild members all practiced the same craft and even lived on the same street in a given city. Important purposes of the guild included overseeing the practice of their craft and maintaining high-quality work, training apprentices, and regulating the flow of work to their self-employed members. Guilds also gave back by performing charitable work among their members, but also in the community at large (1). Beginning in the late 18th century, the power and popularity of Guilds began to fade as a result of industrialization and modernization and the rise of the corporation, which proved to be a more powerful means of organizing people to achieve desired ends.

People Want Freedom and Control

Why would this be a time for guilds to rise again? COVID has shown it is possible for people to work remotely and has given them a taste of the freedom it affords; people want control over their work and personal lives and most prefer working remotely; technological advancements have made it possible to work remotely; finally, many different types of knowledge workers can work relatively autonomously, using modern technology to collaborate.

The Great Resignation

Many people are rethinking their relationship with work, and are considering severing their connection to a single employer in favor of working independently, to preserve the freedom they have experienced in working remotely. Some who do this may seek a different “basis of attachment” (2) so that they are not completely on their own. The Guild is just such an organization, where experts in a specific field can join together, develop professionally, work together, train apprentices, and do good for society.

UXr Guild

The UX Researchers’ Guild mitigates the challenges and risks for researchers who choose to work independently. It is a means to join with other researchers, collaborate, learn, and not be isolated professionally. It is a means to market our cumulative expertise, each new member strengthening the whole, rather than weakening each other through inadvertent competition as sole practitioners. The guild, as a collection of many, allows for greater influence in promoting research standards, mentoring apprentices, and providing charitable aid.

A Guild for Researchers and Those Who Depend on Their Insights

While UXr Guild exists for the benefit of UX Researchers, it also provides benefits for clients who are looking for research expertise. With a stated purpose of maintaining high research standards, the most important standard the Guild maintains is the quality of its members. With rigorous standards for membership, clients can be assured that Guild researchers are excellent practitioners with current experience in a wide variety of industries. If clients can trust the high standards of the Guild, it is no longer necessary to search on their own for researchers and screen them one at a time. The Guild saves a tremendous amount of time and effort usually spent by clients and researchers finding each other, assessing expertise and fit, making contracts, negotiating rates, obtaining approvals, and so on.

Please join us in our mission to define and refine the modern guild, to benefit researchers, clients, and the most vulnerable among us.

Footnotes: 1: Wikipedia (Guild); 2: Adam Grant, “Think Again: The Power of Knowing What You Don’t Know (speech at the Upwork, Work Without Limits, Summit).” Photo credit: Nicholas Hoizey, Unsplash.

Nine design hiring managers share their tips for creating a portfolio and what they’re thinking when they review one.

Melanie Araujo, Founder, Front & Center
Tanner Christensen, Head of Design, Gem
Marc Hemeon, Head of Creative and Digital, Kosas
Anna Iurchenko, Product Designer, Google Health (Moderator)
Abi Jones, UX Manager, Google Health
Cara Kleid, Principal UX Designer, Bill and Melinda Gates Foundation
Austin Knight, Product Designer, Google
Marissa Louie, Director of UX Design, Expedia
Pablo Stanley, Co-founder, Blush, Staff Designer, Lyft
Sean Wen, Product Design, Squarespace

Here are highlights from a wide-ranging conversation:

Q: What are you looking for when you review a portfolio?

Cara: A consistent story. Are you the right person for this role. I want your help categorizing what kind of designer you are.

Melanie: I want you to tell a good story and persuade me. 

Abi: For you to show me what your contribution was, not just pretty pictures; I need to know your perspective to know whether you would fit on my team.

Pablo: Your visual design work is what will pull me in to read more about you. Then once you pass that filter, I’ll focus on how you talk about problems, the people you work with; how you talk about the people you design for, and how you are able to empathize with them. I’m not going to read the book if it doesn’t have a good cover.

Marc: Your work should fit the role you’re going for. There is room for everybody. I just hired an email designer, ad designer, and ux designer. There are lots of specialties today. Just make sure your contact information is on your portfolio, so I can call you. 

Marissa: What is your lens on solving problems in the world (what is your background)? Critical thinking is an interdisciplinary partnership you can have in any company. I’m looking for someone who has a highly developed capacity for critical thinking.

Austin: The container of the work is as important as the work. If you’re applying for a pure interaction design role I’d be less concerned with your visuals, and more with business goals and user goals and how that influenced your work. I like to see what a person is doing outside of work. If you’re a volunteer, I want to hear about your passion.

Some designers think they need motion or other tricks for their portfolio to stand out. Your creativity should show in your design work. Add a little personality, because that lets me get a sense of who you are and your passion. Keep the portfolio simple and straightforward. Then link to other things that might highlight other aspects of you.

Cara: “Your creative brand”—that phrase makes me queasy. I’m not hiring a brand. Who are you, and what value can you bring to my team? We’re trying to figure out what you know, how you do what you do, and where you could fit into our team.

Q: Any suggestions for new grads?

Tanner: I always say if you don’t have experience, your number one job is to do some work (student, freelance, etc.). You can always find side projects to present your capabilities. Find problems in the world, and try to solve them. Build a case study of your design thinking and work. 

Tons of small businesses have been devastated, so volunteer to help them and get some real-world experience. Approach it from a very humble perspective. Sometimes the best experience might be volunteering, because you can choose your projects based on what you want to learn and what you are able to do.

Marissa: If you’re not feeling great about one of your skills, look for work that will strengthen that area. Every designer is different and each case unique. Don’t take it personally if you’re rejected. Keep learning, moving forward, be true to yourself, and try to close gaps. Before you give up on design, seek a mentor who might guide you through.

Melanie: Design is a craft you practice and get better and better. You’re investing in yourself and building your skills with each project. Soft skills are also important. You can make a great design, but if you don’t know how to talk about it and sell it, then it is as if you never did it. You need to back it up with science and data to justify your work. You have to be a bit of a hustler at the beginning of a career.

Austin: Set up your portfolio so that if a hiring manager spends just 1-3 minutes, they will find what you want to show. On just the home page and first case study, get your main points across. Also make sure that the work you show reflects the role you want: mobile, visual, research, etc. Research is the core UX competency, supported by design or engineering. Show design artifacts, original photos of the process, lessons learned. Don’t be reluctant to show things that went wrong.

As a general rule, larger companies have more specialized design roles on larger teams, and smaller companies require more general design skills. The breadth of roles often correlates with the size of the company.

Austin publishes a website on UX design at uxdesignguide.com.

How do you land an interview at a tech company such as Facebook? What is their process, and how can you prepare for each component? Participants Kate Lydon, Head of Design, Nextdoor and Christine Sadrnoori, Product Designer, Facebook had some great suggestions along with the moderator, Amy Stellhorn, Founder and CEO of Big Monocle.

Kate’s process: If someone is referred to her, that is a big plus. Knowing that someone she trusts also knows and likes the designer has a big impact on her. So, before applying, see if you have a connection to the hiring manager.

She loves a cover letter. Surprisingly, not too many people take the time to write one. If they do, it is often boilerplate. However, she sees it as an opportunity for the designer to explain their value proposition. The ideal letter demonstrates how your skills and experience will address the needs of the company and solve their problems. That’s why it needs to be specific for each role. Her advice: make it interesting, real, and personal.

When showing your portfolio, you need to be able to “narrate the story of the work.” She allows about 30 minutes for you to do this. That’s a lot of time, so you should practice, but don’t just give a spiel. Allow some breathing room for questions so that interviewers can draw out aspects that interest them: What was the problem you were addressing? What were the difficulties and design tradeoffs? Why did you choose the solution you did? Who were your collaborators (and do you give them credit)? Did you measure the success of the new design? What was the outcome?

What is the employer looking for? Kate wants great problem solvers and great designers. You can see why narrating the portfolio is so important. It helps show how you define a problem, approach solving it, work with others, and measure the outcome. By the way, she also favors those who have a high degree of self-awareness: aware of your strengths, but also awareness and ability to candidly describe your “areas for growth.” How do you produce good work despite your weaknesses? (Hint: partner with someone who is strong where you are weak.)

When she presents a design challenge to a candidate during an interview, she is not looking for a solution. She wants to hear how fluidly the candidate thinks through the problem and what questions they ask to pull the problem apart. “I want to see the creativity of their thought process.”

Christine’s process: The hiring process at Facebook is now 100% remote. After submitting your resume and portfolio, a recruiter calls to conduct a phone screen. If you pass the screening, two rounds of interviews follow.

The first round of interviews involves an “app critique,” where you choose an app and let the interviewers know what you like and what you would change. You then engage in a conversation about your background; specifically, you describe what interests you and what projects you’ve taken on. Finally, you’re asked if you have any questions about Facebook.

The second round of interviews is called a virtual onsite. You have another app critique with a different group of people, a background conversation with the hiring manager, a problem-solving exercise on a virtual whiteboard, and finally, you give a 30-minute presentation on your most impactful or challenging projects.

When presenting your portfolio, Christine suggests that you balance showing and telling. Don’t go too broad, trying to show all your work. Highlight a few projects and the thinking behind them. Don’t go overboard on the process. The interviewers are all very familiar with the product design process. Focus on what might have been unusual, the tradeoffs you had to make, and how you made decisions.

Wondering how to handle a whiteboard or design challenge? Christine says to expose your design thinking. Use a framework, such as: problem, objective, success criteria. Christine wants to see if you are intentional, proactive, and whether you can navigate with confidence.

Hopefully, these suggestions will give you more confidence, as you know what to expect and can prepare.

Why do so many creative professionals prefer working remotely? The appeal of working offsite can best be explained by the flow of creative work and the havoc wreaked by interruptions.

Flow was defined in a 1990 paper by Mihaly Csikszentmihalyi, as “a state of concentration so focused that it amounts to absolute absorption in an activity.” Designers know that feeling and work hard to achieve a state where all their attention is focused. The problem comes into focus and ideas to solve it start flowing. This rarified state can last a few minutes or a few hours, but unfortunately, it can be broken in an instant by an interruption: a friendly co-worker, an overzealous supervisor, a staff meeting, a phone call, a text or tweet. The cost of these interruptions can be significant.

Even the great composer Tchaikovsky lamented such interruptions: “In the midst of this magic process, it frequently happens that some external interruption wakes me … a ring at the bell, the entrance of my servant, the striking of the clock. Dreadful, indeed, are such interruptions. Sometimes they break the thread of inspiration for a considerable time, so that I have to seek it again–often in vain.” Maybe that’s why Franz Schubert’s Symphony No. 8 is “unfinished.”

To be fair, we are often our own worst enemy. Technology is the great interrupter today, and we each have to tame it in a way that works for us personally. Distraction is the second enemy of creative work. We all need to work very hard to sustain focus and not check a text or tweet or Facebook post. Once we’ve mastered our technology and ourselves, the environment becomes the next big hurdle.

A lot can be said for creating an office culture that respects flow and personal space so that creativity is fostered. Sometimes you find yourself in a space that feels quiet and in which people seem focused. But not everyone thrives in a quiet environment. Some prefer more energy or music. Personal rhythm also affects flow: some can focus better in the morning and others later in the day. 

That brings us back working remotely. The big advantage is that you control the environment and can adjust it to suit your preferences and foster your flow. I invariably hear people say that they do their best creative work when working from home. We have all been forced to adopt work-from-home strategies and many have increased their productivity in this environment. Smart companies will try to preserve the conditions that have enhanced the creativity and productivity of their people. Thus, working from home will undoubtedly play a greater part in the employment landscape of the future.

Sources:

Interruption: The enemy of creative professionals. Jennifer Froleich, Crucial.com.

Flow: The Psychology of Optimal Experience. Mihaly Csikszentmihalyi, Harper and Row, 1990.