In order to complete the evaluation of the medium-fidelity prototype in Appendix X, we also decided to undertake a participant-based evaluation of it. Within the participant-based evaluation, it was decided to use both the cooperative evaluation method and controlled experiments.
Cooperative evaluation was favoured over a co-discovery method, in part due to its cost-effectiveness, and also in part due to our application being completely usable for just a single person. As an example, a co-discovery process might work when evaluating a digital camera (with pairs of participants taking pictures of each other). However, analysis of the prototype resulted in the realisation that there are no features that would benefit from such additional synergies that could arise from participants exploring the application in unison. Hence, co-discovery was decided against.
Moreover, controlled experiments for two specific features were decided to be conducted due to the team having internal disagreements and doubts over variants of two specific features, which are indicated below. Moreover, by doing both controlled experiments and cooperative evaluation, we would get a broad evaluation of the medium-fidelity prototype and further refinement of individual features, thus significantly complementing the prototype's evaluation.
There was a wide overlap between the participants of each user-based evaluation, both in the cooperative evaluation section and in the controlled experiments. To avoid repetition, we describe the participants here.
In total, our entire user-based evaluation consisted of 20 distinct candidates. We describe these candidates in as much detail as possible whilst preserving anonymity:
For planning undertaking the cooperative evaluation, we used Table 10.1 on page 251 of Benyon's (2019) as a reference. In particular, we prepared a set of tasks in advance that the participants would undertake, setting time estimates and time allowances (which, following the advice of Benyon’s (2019), are around 50% longer than the estimate) to give structure to the user evaluation. Moreover, the coordinators of the evaluation compiled a set of questions to ask during and after the evaluation, some of which came from Table 10.1 of Benyon's (2019). We leave the tasks and questions below.
Estimated Time / Allowed Time: 1 minute / 1.5 minutes
Preliminary Objectives:
Estimated Time / Allowed Time: 1.5 minutes / 2 minutes
Preliminary Objectives: