Evaluation

Evaluation is any type of test/measure to see how well something performs. If your system interacts with people, your evaluation will nearly always also involve people!

Examples of evaluations not involving people (we will not cover these):

• Performance of machine-learning algorithm on dataset (only if non-human data!)
• Logical proof on quality of formal mode

Evaluation is part of the process, not just the final product. How ‘good’ does it need to be fore being prototyped? How good before being finished?

What even is a ‘finished’ system? Updates are about changes in related technology, but also about improving after evaluation during deployment!

Through evaluation, we want to know:

What are you evaluating?

• Is it good enough with respect to X?
• How good is X?
• Is X better in system A or B?
• What effect does it have?
• What is wrong with respect to X?

We distinguish 2 types of evaluation, qualitative and quantitative:

Our system can be either evaluated by an expert or a user.