Testing software has always been an important phase of software development, but only in the recent past has usability taken on greater importance. This is because with the growth of SaaS offerings and increased global competition, making your software “usable” —  easy to use, an elegant experience or however you want to define a user’s engagement with your software — has skyrocketed in importance.

 A valuable way to understand the usability of your software is to test it, and to do so with real users. But there are many options for how to test, from in-person meetings to remote user testing to moderated or unmoderated user testing. There are also choices about when to test. Should you perform usability testing as you design your software, or should you do it once it’s completely built out? For a deeper look at this topic, read the Value of a Design System whitepaper.  

 This blog will define each of these usability testing approaches and describe the benefits and drawback of each including best practices for designing a remote user test. One thing to keep in mind is that, even as we begin to emerge from this time of pandemic, it’s widely projected that online tools will continue to grow in use. If there’s one thing we’ve learned in the past year, it’s that many things we once thought needed to be done in person can be done with excellent results online — and much faster and at lower cost.

What is Remote User Testing?
When is Remote User Testing Appropriate? 
Remote User Testing Methods — Moderated vs Unmoderated
Cost of Moderated Remote User Testing
Tips for Conducting Remote Usability Testing
User Testing with Indigo.Design
User Testing for Mobile Apps, Web Apps and Desktop Apps
Participate in Remote User Tests from Any Device
Enhanced Usability Testing Analytics Dashboard
Take-a-Remote-Unmoderated-User-Test

Let's get started!

What is Remote User Testing?

Remote user testing is a way to test how easy it is to use your software with people who are in different geographical locations. It can include testing a design prototype of your software before coding begins or even to test an already built-out website.

For those designing software, remote user testing lets a developer or usability analyst get fast feedback before a developer writes the code that turns the design — or prototype — into a completed app. It differs from traditional usability testing that brings users and researchers together in one place to conduct the test.   

When is Remote User Testing Appropriate? 

Remote user testing is appropriate in these situations:

• When evaluators want to test their software’s usability more quickly and at lower cost
• During times of social distancing when meeting physically is discouraged
• When evaluators and prospective users are located in different countries and time zones
• When the resources required for in-person testing make such testing prohibitive
• When participants are unable or unwilling to travel to the test site

Remote User Testing Methods — Moderated vs Unmoderated

While in-person testing is, by definition, moderated, you can choose from two approaches with remote testing: moderated and unmoderated. Here are the differences and suggestions for when each is more effective.

Moderated Remote Testing 

Moderated remote testing is typically conducted one-on-one with a moderator via a collaboration platform with screen-sharing features such as Skype or Webex. It allows you to collect quantitative and qualitative data about a participant’s behavior similar to in-person testing. You can measure time-on-task, error rates, attention and navigational issues. The advantage of moderated remote testing is that you get instant realtime feedback on participant frustration, confusion or dissatisfaction. The feedback mechanism is greatly enhanced if you also use video, now commonly used on many conference calls, as visual cues provide even greater insights into participant comfort level. 

Unmoderated Remote Testing

With an unmoderated remote test, data can be collected at the participant’s convenience. The testers are given access to the prepared test and can choose when to  participate and provide feedback. This lets usability analysts do test sessions with a handful or more of users simultaneously, in their natural environment. The self-reported feedback and comments which are received in response to open-ended questions can be the most valuable data you collect during an unmoderated test. Such automated testing typically allows you to collect quantitative data about participant behavior using a larger sample size than moderated testing methods and varying comparative data.

Some testing platforms provide additional features. Infragistics’ Indigo.Design, for example, provides usability analytics with tracking maps that show the choices a user made, indicating specific clicks on the prototype and where the tester clicked “incorrectly” versus the planned path.

MODERATED VS UNMODERATED USABILITY TESTING

	Moderated	Unmoderated
Type of test	One-on-one	Self-guided
The process	Moderator engages the users as they walk through the session	Users follow instructions and the analytics tool maps their session
Test group size	Small	Small to large
Use cases	Highly specialized domains (e.g. doctors, accountants)	Geographically dispersed audience
Web development stage	Prototyping and onward	Prototyping and onward

Cost of Moderated Remote User Testing

While costs vary for conducting a remote moderated study in the US, the Nielsen Norman Group estimates  that on the low end, a moderated study for 5 participants might cost around $415 plus 32 researcher hours. On the high end, with high recruitment fees and big incentives, it might cost around $1,680 and 48 hours.

Tips for Conducting Remote Usability Testing

There are multiple steps to conduct a successful remote usability test including:

• Define goals and target user
• Recruit users
• Run pilot tests
• Provide guidance
• Summarize the results

Define Goals and Target Users

It’s valuable to define your goals and your target group upfront. Define your target user and decide which specific areas of your app or prototype you plan on testing. Make sure to define a user goal and create task-based scenarios in order to test and validate areas of your product. Once you have this defined, the next step is to recruit participants. 

Recruit Users

Make sure to recruit users who fit your target group and persona as defined in the previous step. Since tests are remote, you are able to recruit people from just about anywhere! This helps you better pinpoint the target audience you wish to test as geography is no longer a constraint. Social media platforms are great options for contacting people that may be interested in testing out your product.  

Run Pilot Tests

No matter how good our plans are, occasionally they will fail us. To avoid disappointment, we must test our user tests!  With just one or two pilot runs of your user test, you will learn and be prepared for any potential misunderstandings and frustrations that users in your larger study might experience. This lets you adjust your script and plans accordingly making the whole experience more pleasant and predictable, while at the same time your data will be cleaner and more accurate.

Provide Guidance

When you invite testers, you’ll want to provide an overview of what you’re trying to accomplish and give them an idea of what the test will involve and how long it may take.

For instance, if this is an MVP, you might want to explain that the experience may feel a little rough or to provide background on the solution itself (why you’ve built it, what you’re looking to do with it) so they’re not distracted by the design.

Don’t worry about your users misplacing these details in their email invitation. Most testing platforms include a place to include these notes within the context of your usability test.

Summarize the Results

Once you’ve received enough testing feedback, go through your results. Note any patterns and areas of friction that will help direct future iterations. Some testing platforms create usability results recap screens so you can quickly determine insights of the entire test, or you can dig deeper to see results from each participant.

CEO's Want User Testing

With user experience and usability still a primary concern of every CEO, management team, and product delivery team, how can you still deliver high quality UX under the restrictions of social distancing? With Indigo.Design’s remote, unmoderated user testing, you can ensure your apps are:

• Useful
• Usable
• Desirable

And you can do this all while maintaining the modern, strict social distancing measures to ensure the safety and health of employees, contractors and, in the case of user testing, the stakeholders and testers of your designs, prototypes and applications.

User Testing with Indigo.Design

With Indigo.Design, you can create cloud-based user tests, and then you simply share them with stakeholders and testers. This means you are testing your designs with real people, getting real-time analytics on the test results, with the safety of social distancing built in.

Here is how it works:

It is really that simple: 3 easy steps to create, share and analyze a remote user test.

Using a tool like Indigo.Design, for unmoderated, remote user testing, you can avoid costly iterations by getting feedback before coding—early and often. It is easy to quickly set-up user testing for your prototypes and see how users engage. And because everything is cloud-based, you can create as many tests as you need at no additional cost and without any additional infrastructure. You can review test results to see how users achieved tasks, screen-by-screen or on video, and users can participate using any device.

So, let's look deeper into on how user testing works using Indigo.Design.

User Testing for Mobile Apps, Web Apps and Desktop Apps

Our approach to creating user tests and analyzing the data from user tests is unique in the market. You can create tests for any of the designs in your cloud workspace. You can either publish your prototypes from Sketch or create them directly in your workspace - for example - upload a prototype from Indigo.Design desktop, or upload a series of screen shots to create a live prototype. When you create a new test, you will see a live test creation experience, where you can add new tasks and then interact with the live prototype to record an expected path. This information will be used when presenting you with the results and segmenting participants.

Everything is saved in real-time so you don’t need to confirm or cancel anything. You can change the default test settings when you set-up the test. We have tried to create some good defaults to speed-up your workflow. A “preview” option lets you take part exactly as a participant would, without recording any results.

Participate in Remote User Tests from Any Device

Indigo.Design solution lets your stakeholders and users take part in a usability test on any device, as long as they have access to a web-browser. You can invite them by sharing a link to the test, which makes it really easy to invite via email, chat or forum post.

Optionally, when someone takes part using the desktop Chrome browser, they can install the Chrome extension that lets us record the user’s session. Not to worry, users are always in control of whether they want to provide camera and voice streams before they start the test. Even if the video is not available, we still collect task analytics and how users interacted with your design.

Did we mention that you can create any number of tests with unlimited participants? Well, you can! This frees you up to test when you want and with anyone you like. Your participants don’t even need to sign-in, further removing any obstacles to participation.

Enhanced User Testing Analytics Dashboard

With the new analytics dashboard for usability tests we've made it easy to not only collect analytics related to participation, but to watch a video replay of the participant’s session.

And while watching videos allows you to really experience the session from the users’ perspective, it’s time-consuming. To address this problem, the analytics dashboard had been redesigned to give you quick insights into overall behavior and also individual behavior— even before you watch any video. We feel this is going to save you a lot of time, allowing you to locate problem areas in your flow quickly.

The user testing dashboard analytics report has four main areas of interest:

1. Task analytics overview- This shows a brief task description, success rate (%), # of participates who succeeded vs. total participants, and the average time spent on each task
2. Completion funnel- this is an aggregate representation of the expected path you defined for each task, and shows how many people use that exact path; where drop-offs occur, and where unexpected interactions are happening.
3. User segments- automatically categorize user participation, based on whether they used the expected path, used an alternate success path or could not complete the task with the design provided.
4. Participation data table- this shows the session details for each participant, with a timeline view (Click-path) that visualizes expected and unexpected interactions. Hovering on any timeline marker provides a quick preview of how and where the user interacted with the design.

If the participant has provided a video replay, you will see a “play video” option listed. Even when there is no video, the click-path option lets you visualize any users’ session.

Take a Remote, Unmoderated User Test!  

Are you ready to see the experience of a remote, unmoderated user test?

*** Click this link Take part in a live usability test ***

Need More Security? Indigo-On Prem is for You.

As mentioned earlier, with Indigo.Design, you can access both your prototypes and inspect in one place. You can get all the power of cloud.indigo.design as a private Indigo Server so that only members from your organization have access. Your private server also supports unmoderated usability testing with user videos to ensure that you have the right design before you generate code. We truly believe in the value of iterating with real customers, and Indigo.Design On-Prem offers a unified platform to design and evaluate your designs as frequently as required.

Contact sales@infragistics.com for a demo of Indigo.Design On-Prem, our private server offering. 

Get Started with Indigo.Design

Don't stop with prototypes; get tests in the hands of real users with our 1-click sharing, and the ability to view prototypes on any device. Harness the power of Indigo.Design Cloud for recording user sessions and task analytics.  With the global pandemic forcing a fundamental change in the way we operate our businesses, user experience is more important than ever.  Using tools like Indigo.Design for remote, unmoderated user testing becomes an imperative.  

If you have any questions or would like to talk about user testing with Indigo.Design, shoot me an email at jasonb@infragistics.com. 

Get started today at Indigo.Design!