Usability testing

Here are some resources from usability tests completed by new media professionals who work for a large publisher.

The tasks below included several parts, most designed for the testers. Only one part in each section (D) was given to the user, on a separate piece of paper.

Subjects followed the "think aloud" protocol, which asked them to talk through what they were doing as they did it.

Subjects were sometimes given an easy task to get them started. See below.

Practice task

A. Purpose

To raise the test subject's confidence.

B. Preparation

Open the web browser to a blank page.

C. Observer notes

Do not take notes for this task.

D. Task for test subject

  1. Go to the Weather.gov web site.
  2. Find local weather for a US city.
  3. Switch to the radar image.

Sample task one

A. Purpose

This task will tell us whether and how users figure out how to add and delete notes in an e-book.

B. Preparation

Log into Tech Comm e-Book as instructor@school.edu/demo and go to any internal page that doesn't have notes at the top.

C. Observer notes

The tester's task is below. Please note what correct and incorrect paths they follow; whether/how/how long/how successfully they use the help menu; what they say about what they're looking for, and why they're clicking on what they're clicking.

D. Task for test subject

  1. Add an annotation on this page that says: "This is a note about this text". Make your annotation available to other users.
  2. Now delete the note you just added.

Sample task two

A. Purpose

These tasks check to see how well the navigational features help users go quickly to sections or features that they need.

B. Preparation

Log into Tech Comm e-Book as instructor@school.edu/demo and go to any internal page that doesn't have notes at the top.

C. Observer notes

Please take notes that help us discover how quickly and directly a user is able to locate each item. Under each step, try to write down everything the tester clicks on. Try to estimate how long each step takes, and if there are any long pauses, be sure to note where those occur, what the tester says they're thinking about, etc.

D. Task for test subject

  1. Locate the first page of Chapter 14.
  2. Chapter 6 links to an Interactive Sample Document called “Evaluating Information from Internet Sources.” Locate that document.
  3. Chapter 9 contains a section called “Analyzing the Writing Situation for Definitions.” Locate that section.
  4. Assume that you have a document that you would like to link in as a supplement to a chapter in the e-book. Find the instructions for doing that.

Introduction

This was read by the testers to the test subjects, no matter what the task.

Thanks for helping us test some features of our new e-books. Your assistance will be really helpful to us. I'm name, and this is name, who will be taking some notes as we work.

We're going to give you a short task to do using one of our e-books. As you do the task, we would like you to speak your thoughts aloud so that we can hear what you're looking for on the screen, why you select the menu items or links you select, and so on. Knowing what you're thinking helps us understand how users need this software to work.

Because we want to learn how well this software will work when a user has to figure it out on his or her own, we'll refrain from answering questions while you do the task. Please simply do the best you can--any problems you have will show us where the software needs to be improved! But we'll be happy to answer questions at the end, and to hear any additional feed back that you have.

Observer notes sheet

Observers used this standardized form. Legal paper was used to allow copious room for notes. Testing was recorded using screen-capturing software which also recorded the test subject's voice from a mike above the monitor.

Task number:

Subject:

Subject expert rating (1-10):

Tester:

Note taker:

Time started (start the recording first):

Notes (please indicate task steps with circled #s):

[lots of space here]

Time finished (stop the recording after):

Recording file name (lastname-firstname-task#-mmdd):

Validate: CSS or XHTML.