HCI Course Part 7 - User Testing

This is part seven in my series about the Human-Computer Interaction course I took through CourseraRead all my posts for the full story.

Assignment 6 User Testing

Now the final pieces fell into place as we created test materials and recruited participants.

Since this was part of an educational experience, the goal was to learn how to do the user testing process the same as we might later do it on the job. As such we had a thorough list of documents to prepare in advance of the testing itself.

Experimental Protocol

The experimental protocol provides details about how the study will be run. It should include enough explicit details to allow the test to be reproduced exactly the same for every participant and exactly the same by a different experimenter at a later date. Scientifically, having an independent group repeat your study later (using your handy experimental protocol as a guide!) and find the same results is very powerful.

What you are likely to find in an experimental protocol:

  • Set up -  materials the experimenter provides, location of test, preparation tasks
  • Order of events - give participant an overview, gain informed consent, etc ...
  • Specific sentences & paragraphs experimenter will use when giving the overview, debriefing, or any other time exactness is appropriate. Consistency is key for valid test results.

Consent Form

As it sounds, the informed consent form is a record of the participant's agreement to take part in the user test. It is also the place to inform the participant about her/his rights and what may be asked of her/him during the study. A copy of the consent form is given to the participant as a record and to provide contact information in case the person has questions or concerns.

In a consent form you'll see:

  • Study title and experimenter's name
  • Description of study
  • Time involvement
  • Risks and benefits (to the participant) of the study
  • Payments - any money, gift cards, t-shirts, gifts, or any other incentives given for participating in the user test or if there are no payments say that explicitly
  • Subjects rights - including spelling out necessities such as the right to stop participating at any time and the participant's privacy rights
  • Media use - state if photos, videos or audio will be recorded, how the media will be used, and list separate place for participant to consent (or not) to each media
  • Contact information - who can the participant contact later with questions
  • Place for participant's signature & date

Study Specific Materials

Obviously these documents vary based on what is needed for the specific user test.

For my user tests I wrote:

  • Task List for Participant - outlining the tasks for the participant to do during the user test
  • Questionnaire - given to the participant after using the prototype to find out more about the experience from that person's point of view
  • Experimenter Notes - a place for me to write notes about each participant's session

Comparison Prototypes

For this particular user study we also redesigned one component of our prototype.

Providing an alternate design can be a powerful way to gain additional constructive feedback. Not only can the participant compare and contrast the two designs, but the presence of two (or more) differing prototypes helps dissolve the preciousness of the main prototype in the eye of the participant. In general people seem less likely to critique something if they feel the creator has spent a significant amount of time and energy and *heart* in building it. Providing an alternate design relieves them of the feeling they are critiquing your "baby" and gives them permission to tell you how they really feel.

In this course we did the entire user test using the main design. When the test and questionnaire were complete we showed the participant the redesign and asked for addition feedback comparing the two designs.

My Car Saver redesign involved modifying the fuel fill up procedure. In the process I had the opportunity to learn more about variables in Axure. Here are the Balsamiq and Azure prototypes for the redesign:

Car Saver redesign: wireframe in Balsamiq
Car Saver redesign: wireframe in Balsamiq
Car Saver redesign: prototype in Axure
Car Saver redesign: prototype in Axure

Let the testing begin!