O'Reilly logo
live online training icon Live Online training

Usability Testing 101

Planning, recruiting, and running your first study

Elizabeth Allen

“You are not the user” is a classic UX mantra for a reason. Too often, businesses don’t take the time to test their websites or products outside company walls because they assume that their users are just like them. If you’re reading this, the great news is you’ve already bought into the idea that testing with real, representative users is essential. In this three-hour course, you’ll learn how to do usability testing efficiently and effectively so that you get high-quality, honest user insights in a just short period of time.

Through a combination of lectures and hands-on exercises, Elizabeth Allen teaches you techniques and best practices for planning, conducting, and analyzing usability tests. You’ll leave armed with the tools you need to get great data from your users and the confidence to do it all on your own.

What you'll learn-and how you can apply it

By the end of this live, online course, you’ll understand:

  • Why, when, and how to plan for usability testing
  • How to create a warm, encouraging environment for test participants and quickly make them comfortable with participating
  • How to frame your test scenarios and questions so that they don’t lead to biased or misleading findings
  • How to create a basic test setup for running and recording test sessions at little to no cost
  • Easy-to-use techniques for analyzing data from test sessions

And you’ll be able to:

  • Confidently plan and run in-person and remote usability tests
  • Troubleshoot the most common issues and surprises that can come up on test day
  • Use appropriate scenarios, unbiased questions, and active listening to get honest user feedback
  • Read users’ body language for a deeper understanding of their reactions
  • Tease out deep insights from user testing data

This training course is for you because...

  • You’re a designer who wants to start validating your work with real users.
  • You’re a product manager who wants to foster a user-centric culture on your team.
  • You’re interested in starting a career as a UX researcher, and you want a quick overview of the fundamentals.

Prerequisites

  • A basic understanding of what a usability test is and why it’s valuable. Recommended preparation is enough.
  • If you’re working on a project that could benefit from user testing, take time before the course to brainstorm questions about your product that usability testing could help you answer. You’ll use this example during in-class exercises to define a real testing plan for your project.

Recommended preparation:

How to Run a Cheap, Fast, and Incredibly Useful User Test” (article)

Usability Testing Demystified” (article)

Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, “Chapter 1: What Makes Something Usable?” (book chapter)

Usability Testing of Fruit (video)

B|P Game Research Process (video)

Recommended follow-up materials:

UX Research (book)

Interviewing Users (book)

Just Enough Research (book)

Complete Beginner’s Guide to UX Research” (article)

User Research Fundamentals for Designers (video)

About your instructor

  • Dr. Elizabeth Allen is a UX researcher, psychologist, and cat owner based in Toronto, Canada. She is the founder of Brazen, a UX research consultancy that helps companies identify and solve critical UX problems in digital and physical products. She has conducted research for a wide variety of companies over her career, including Shopify, United Airlines, Discover, and Mozilla.

    Elizabeth loves teaching, and has lectured in psychology and statistics at the University of Chicago, and spoken about UX research at the Pratt School of Information and The University of Waterloo. She also speaks regularly at conferences about all things UX research, most recently at UXPA, Web Directions, and Interaction.

    Elizabeth loves teaching, and has lectured in psychology and statistics at the University of Chicago, and spoken about UX research at the Pratt School of Information and The University of Waterloo. She also speaks regularly at conferences about all things UX research, most recently at Web Directions: Respond in Sydney, ConveyUX in Seattle, and Interaction 17 in New York.

Schedule

The timeframes are only estimates and may vary according to how the class is progressing

Course outline

When and why to do usability testing (5 minutes)

  • What is usability testing?
  • Questions that usability testing can help answer
  • What sorts of things you can test

Planning your usability test (10 minutes)

  • The value of planning your test
  • High-level planning: How to craft great research questions
  • Low-level planning: How to translate research questions into user scenarios

Activity 1: Defining research questions and scenarios (15 minutes)

  • You’ll create a short set of research questions and scenarios based on a project you’re currently working on or a mock project proposed by the instructor. (The mock project will be very simple and won’t require any pre-work.) You’ll come up with three high-level research questions that will help you achieve your goal; for each research question, you’ll define one or two scenarios that you could use to get data that addresses it. You’ll then post your questions to the group chat for discussion.

Structuring a test session (10 minutes)

  • Creating a welcoming environment
  • Explaining disclosures
  • Warming up the participant
  • Explaining scenarios and the “think aloud” protocol
  • Wrapping up the session

Q&A (10 minutes)

Break (10 minutes)

Recruiting participants (10 minutes)

  • Who and how many people to test
  • Where and how to recruit participants
  • How to make sure your participants show up on test day

Activity 2: Who are your users and how will you find them? (15 minutes)

  • Drawing on your real-life project or the mock situation provided by the instructor, you’ll think deeply about the characteristics of your target users, and for each characteristic you think of, you’ll explain whether it is based on a hunch or on some data you have. (If you are using a planned product, there may not yet be any data about who the potential users are, so this question could be more about defining who the intended target users will be.) You’ll then brainstorm ideas for places (online and out in the world) where you could try to recruit users for your research and create and post two recruitment spots in the group chat, along with a short (1–2 sentence) rationale for why they make sense for finding your target users.

Logistics of a flawless test day (10 minutes)

  • Recording and note-taking strategies
  • Troubleshooting common issues
  • The importance of a mock session
  • Preparing for and including observers

Q&A (10 minutes)

Break (10 minutes)

  • Getting high-quality data from test sessions (15 minutes)
  • Question formatting: How to ask questions that get unbiased answers
  • Question ordering: How to drill down for deep insights
  • Nonverbal communication and body language
  • Listening actively and effectively

Activity 3: A discussion on being an unbiased moderator (15 minutes)

  • The instructor will post a list of biased moderator quotes from a test session. You’ll rewrite them to be more open and post your answers in the online discussion.

Making sense of the data (20 minutes)

  • Anecdotes versus insights
  • Techniques for formatting and analyzing usability testing data
  • Prioritizing and acting on issues uncovered by usability testing
  • How to decide whether more testing is necessary

Wrap-up and Q&A (15 minutes)