You can write down comments about certain tasks, you can rate tasks numerically or you can combine both approaches.
Rating the tasks
Exactly what you will do with the data you collect is not really within the scope of this booklet. However, what kind of data you should collect is.
Remember that you have broken down the entire session into main and small tasks. Each of the small tasks should be defined just as the main task is: they should be specific and measurable. In other words, it should be easy to tell if a task has been completed or not and how difficult it was for the participant to complete it.
So, one thing you can definitely rate for each small task is the difficulty. It makes sense to have at least four levels:
- 0 – no difficulties;
- 1 – some difficulty but no intervention by the interviewer;
- 2 – major difficulties with slight intervention;
- 3 – failure. The task is completed in the first three levels but not in the fourth one.
Other factors you can measure are:
- disaster rate: the percentage of participants who thought they had completed the task correctly, while in reality they hadn’t
- the percentage of participants who have completed the task at the first attempt
- the time needed to complete the task
- how crucial this task was to the participant – i.e. if they were unsuccessful, was the failure to complete the task something that they could just live with, or was the fact that it didn’t work a deal-breaker?
You can give more weight to those tasks that were a part of the main task flow, as opposed to those that were only dealt with by the participant because the interviewer asked them to.
Missed the previous part?
As always, you can catch-up and see our previous article in the series – Usability: Interaction with the user – Part 2