With the rise of opaque and poorly-understood machine learning models in the field of AI, trust is often cited as a key desirable property of the interaction between any user and the AI. The recent rapid growth in the field of explainable AI (XAI) is also, in part, motivated by the need to maintain trust between the human user and the AI. By designing AI that users can and will trust to interact with, the AI can be safely implemented in society.
But, what precisely are the prerequisites for trust between a user and AI? For what goals does the cognitive mechanism of trust exist? How can we design AI which facilitates these prerequisites and goals? And how can we assess whether the prerequisites exist, and whether the purpose behind the trust has been achieved?
In this work, we are interested in formalizing the ‘trust’ transaction between the user and AI, and using this formalization to further our understanding of the requirements behind AI which can be integrated in society.
A common defintion of trust was given by Roger et al. in 1995:
“If A believes that B will act in A’s best interest, and accepts vulnerability to B’s actions, then A trusts B”
Here, A is the trustor, and B is the trustee.
We need trust to:
But, when do we need trust? All circumstances in which we are in any way dependent on other’s actions. Basically always when other people are choosing actions and making decisions that influence you.
Do we need to trust if there is no vulnerability? Trust is most relevant if there is risk! To accept vulnerability to X’s decision/action in high-risk situations, we need to know the type of decision/action and situation. You need to trust any time you rely on someone, and there is a risk (vulnerability) to doing so.
Trust can be seen as a contract: it is specific (if not always explicit) about what actions we anticipate and when.
I.e., “I trust X to do Y in situation Z”. For example, when we say “I trust my doctor”, we are actually saying I trust my doctor to give me advice on medical matters, as we do not trust your doctor to, e.g., fix your plumbing.
In other words, contracts specify what behaviour we anticipate. Even when we talk about ‘general’ trust, there is typically an implicit contract! (I trust my doctor)