Controlled experiments

Another way of undertaking user evaluation is to set up a controlled experiment. Controlled experiments are appropriate where the designer is interested in particular features of a design, perhaps comparing one design to another to discover which is better. In order to do this with any certainty the experiment needs to be carefully designed and run.

A user study of a particular aspect of the system:

• Comparing which is better?
• Manipulating one aspect to study the effect
• Empirical, interpretive, formative or summative, controlled or field

Advantages and disadvantages of controlled experiments:

A study with users in a controlled ****environment (lab study) is very cheap, easy, and does not require full employment. However, field study, when the study is done with uses in daily life, may be preferred sometimes as it is more akin to real life and results are more long-term, although it is costly to set up.

Research Question

When preparing a controlled experiment, we need to think what is the research question(s) we want to answer:

Etc.

Overall framework

The general framework for controlled experiments is:

1. Decide on independent & dependent variables.
2. Look at literature for possible confounding variables.
3. Decide on experimental design (within-between subject).
4. Decide on who will participate & how many people.