During the main scenario part your most important goal is to re-create the situation that the user would have had in their "usual" environment.
Explaining the task
Now is the time to explain what needs to be done. The task itself should be concise and easily understandable – for example, “Using the Vodafone website, find where you can choose tablet devices and buy the one that would suit your needs the most.” If this needs further clarification, feel free to explain anything.
Just before the start, ask the participant to repeat the task in their own words. This is a very important check. It might be that what they say isn’t really what they’ve been asked to do. In that case, politely explain again what should be done.
It’s useful to have the main task written down on a piece of paper. You can even give this to the participant as a reminder of what the task is about.
Going through the main task
From now on, you shouldn’t help the user any more. If you see that they’re going in completely the wrong direction, let them. You need to find out if they’ll be able to recognize their mistakes and get back on track.
Participants will ask questions, but you don’t have to remain silent and pretend that you’re not there. Actually, it would be better if, when a question is asked, you ask something back, like “What do you think this is?” or “Do you see how you could proceed?” If they’re stuck, encourage them to keep trying, but also to let you know if they would have kept trying at home or if they would have given up.
As long as the participant is “going with the flow” – i.e. everything is running smoothly and they don’t seem to need to stop and think – don’t interrupt them. You can take notes, but if they’re working quickly, you might miss something crucial, so it might be best just to watch. Only at the first sign of trouble – say, if the participant is taking some time to figure out how to proceed or has already taken the wrong route – should you intervene.
Asking questions
You should always ask open-ended questions. Never ask a closed-ended one like “Did you click that link so that you can …?” Instead ask “Why did you click that link?” or maybe “What did you expect to happen after you clicked that link?”
The reason for avoiding closed-ended questions is that they are based on your assumptions about why something happened. Truth is, it is rare to guess correctly why someone did something. If you ask “Did you do XYZ because …?”, the participant may think “Well, that’s not really why, but you’re close enough” and just answer “Yes.” Even if you’re sure that you know why someone has done something, ask your question in a way that the participant has to elaborate.
Guiding the participant
As already said, participants will ask you questions when they encounter problems. Of course, you shouldn’t give them answers right away. Instead, you should encourage them to describe the situation the way they see it and tell you what they want to achieve next, how they expect to solve the problem and if they see anything that could take them a step closer to the completion of the task.
Only if it’s obvious that they have really hit a brick wall and there’s no way they can proceed, or if they openly say that, in real life, they would have definitely given up at this point, should you tell them what to do. Then this particular step has to be noted as a failure.
When you finish the main task and you’re now doing the small tasks, it makes sense to guide a user to the page where the task should be completed (unless, of course, the small task was to find that particular page). The point is: when doing small tasks, you can be involved in setting up the starting position because the emphasis is on specific small actions and you want to complete as many of them as possible in the remaining time.
Small tasks are also a good indication of whether or not the participant is doing actions relevant to the main task. While the participant is in their “first run” (doing the main task) and starts doing something that isn’t even a part of any of the small tasks, you should ask them why they think that’s relevant for the completion of the main task. If they give you a valid explanation, perhaps that shows that you have somehow omitted a necessary small task. But it might be that the participant has simply wandered off course, in which case you should politely remind them what the task is and ask them to try and stick to that. You don’t have to be absolutely rigid about users not roaming about a bit, but you should be time efficient.
Closing the session
When both the main task and the small tasks have been completed by the participant, the session is over.
Regarding the length of a session, try to keep it to under 60 minutes. Depending on the complexity of the main task and the number of small tasks, it could take anything from 15 minutes to an hour. However, if the participants make the effort come to you to do the testing, having sessions that are only 15 minutes long might make them think they shouldn’t have bothered to come. If the main task is so complex that it really requires sessions longer than an hour, you should rethink the task. It can probably be broken down into two or more tasks.
At the end of the session, take a couple of minutes to get some final thoughts from the participant. Ask them about how satisfying the experience was. How would they describe it? See if they use negative, positive or neutral adjectives. Ask them if the whole procedure was harder or easier than they had expected. Find out if they would recommend the website to others. You can also ask them to try to remember one thing they really liked about the site or one thing that was really bad. If they’ve had any experience with competitors’ websites, ask them how they were different – better, simpler, worse?
After all this, simply say thank you and make sure you have all the necessary contact information from the user and, perhaps, their bank details if they’re not being paid immediately for taking part and you’ll have to transfer their fee to them later.
Missed the previous part?
As always, you can catch-up and see our previous article in the series – Usability: Interaction with the user – Part 1