Moderated Remote Testing

Benefits, Advantages and Disadvantages

As with either form of remote testing, the primary benefit is ability to work with participants without some of the difficulties of bringing them to a lab.

Advantages:

• You can work with participants from any geographical location
• Participants are working in their own familiar environment
• Observers can watch sessions from their own location

Disadvantages:

• You (usually) cannot see the participant, so cannot gauge body language or other indirect communication.
• Technical difficulties can interfere with the session.
• The material being tested must be on-screen.

An additional advantage to remote moderated testing is the ease with which people with a complex or unusual technical setup to work in their own, familiar, environment. This includes people with disabilities who use assistive technology, or who find travel difficult.

Cost-Effectiveness

The primary cost benefits of moderated remote testing come from eliminating the need for a lab, and for either the moderator or participant to travel. How much of the cost this saves depends on whether you are replacing a rented facility, how much travel would have been required and so on.

Appropriate Uses

Moderated remote testing is most effective when:

• It is difficult to bring participants into a lab, either because of geographical distance, or because they are too busy to recruit easily.
• It is valuable for the participants to work in their own environment.
• You want to intercept visitors to your site, rather than recruit in advance.

Unmoderated Remote Testing

Benefits, Advantages and Disadvantages

The primary benefit of unmoderated remote testing is the ease with which large number of people can participate in the test.

Advantages:

• Quantitative data
• Participants work in their own environment
• Participants set their own schedule
• Lower cost per participant

Disadvantages:

• You cannot watch the participants in real time
• Tasks are fixed in advance, and cannot be adjusted for each participant, or be interview-based
• Task success may rely on self-reporting or web analytics
• Qualitative feedback is limited
• Some participants may be interested only in early the honorarium

Cost-Effectiveness

The cost of unmoderated remote testing include:

• Time to set up and pilot test the test script
• The cost of the tool. This varies from free to a high cost, depending in part on the features of the tool.
• The time to analyze the data collected
• Honorariums are lower, or not paid.

The biggest cost savings of unmoderated remote testing come from not needing a moderator.

Appropriate Uses

Unmoderated remote testing is most effective when:

• You need data from large numbers of participants
• You have very specific questions
• You have clear tasks for participants
• You are more interested in what participants do than in exploring why or how they do it.