Combining Moderated and Unmoderated Usability Testing

The best way to evaluate the user experience of a product is by testing it with users. Observing people trying to perform tasks in an interface gives you the best understanding of how well it works, and which problems need to be fixed.

Traditionally, usability testing involved moderated sessions, in which a researcher met with a participant in person to observe them performing tasks and to ask them questions about the experience. In recent years, unmoderated usability testing has become available. It enables testing to be conducted more easily, faster, at less cost, and with many more participants.

Moderated Usability Testing

In moderated usability testing, the researcher meets in-person, or through online meeting software, with each participant in individual sessions. The researcher observes and asks questions as the participant attempts to complete tasks in the interface.

The ability to directly observe and interact with the participant is the greatest advantage of moderated usability testing. It allows the researcher to get the best understanding of the participants’ behavior and why they encountered problems. However, the need to be present in each session makes moderated usability testing very time consuming, more expensive, and puts a practical limit on the number of participants that can be included. As a result, most companies only include one or two rounds of moderated usability testing on projects.

Unmoderated Usability Testing

The alterative to moderated testing is unmoderated usability testing. Unmoderated means the researcher is not present when participants complete the study. The researcher sets up the tasks and questions using an online testing tool. The tool generates a link to the study. Participants select the link to go the study and complete it online, on their own time. These tools capture data such as task success rates, task time, errors, survey answers, and audio and video of the task.

The primary advantages of unmoderated usability testing are that it saves time, is less expensive, and allows you to test many participants simultaneously. After setting up the study, and sending the link to the participants, the researcher can work on other things, while the participants complete the study on their own time. Unmoderated tools capture the data and present the consolidated results in useful summaries and visualizations. Because the researcher isn’t required to attend the sessions, and because the data is automatically captured and compiled, hundreds or thousands of participants can be tested without much extra effort.

How to Best Use Moderated and Unmoderated Usability Testing

At this point, you might wonder which of these methods should I use – moderated or unmoderated usability testing? The answer is – you can use both! Add unmoderated usability testing to your design process to quickly get additional user feedback on your design.

Moderated usability testing is extremely valuable, but it’s so time consuming that most projects only include one round of testing. By adding additional rounds of unmoderated testing to the design process you can get additional feedback on your designs very quickly, from many more participants.

Conduct unmoderated testing early in the design process to quickly answer specific design questions. Similar to agile development methodologies, you can create prototypes to answer targeted questions, and then test hundreds of users in a very short period of time. Then make design changes as needed.

Conduct moderated usability testing once you have a more interactive and extensive prototype, when you want to have the ability to directly observe and interview participants. After fixing any problems you found, conduct additional rounds of unmoderated testing to evaluate the final design. Combining moderated and unmoderated testing allows you to take advantage of both methods, and to include more usability testing and validation in your projects.

To truly understand unmoderated testing, it’s helpful to see it firsthand. One option for unmoderated usability testing is the testing tool built into Indigo Studio. With this tool, you can  not only create prototypes, but create unmoderated usability tests  at no extra cost. Other unmoderated usability testing tools charge a per-test fee or require a monthly or yearly subscription that can run up to $10,000 or more. Indigo Studio is a more affordable option – and you can try it for free. Get  Indigo Studio Essential today and see for yourself how easy it is to incorporate unmoderated usability studies into your design process.