Let’s start with a warm-up question. Suppose I have 2 dice: one ordinary die and a loaded one. For the loaded one, the probability of getting 6 is five times as much as for the other sides:
I choose one die, and I throw it 3 times. This gives as a four, then 2 sixes. Can you know for certain which die I threw?
Well, you can’t. A throw with either of the dice could lead to 4-6-6. So let’s rephrase the question: Which die do you think I threw?
So, with the loaded die the probability of seeing a four and two sixes is much higher. Therefore, it is most likely that the loaded die was thrown.
In answering the previous question, you probably chose that parameter (in this case one of the two dice) such that the probability of obtaining the actually observed data is the largest. This is the basic idea of the maximum likelihood principle.
<aside> ℹ️ Maximum likelihood principle: Given a dataset choose the parameter(s) of the model in such a way that the data are most likely.
</aside>
Phrased differently: “With the maximum likelihood principle, you must choose your parameters (like a probability p) in such a way that the probability of actually getting the data from your dataset is the biggest.”
We will now use this principle to know how to get the best estimate for the probability that you will pass your driving test.
Suppose in a year $1000$ people try to pass their driving test:
Note that the number of tries that is needed to pass the test is displayed and some people need more than 4 tries!
You can also see that the distribution among men and women seems different.
Let’s start by estimating the probability of passing the driving tes for women.
Let $X$ be the number of tries need to pass for women, with probability $p$. Suppose every try is independent of the tries before. This probability $p$ is assumed to be equal and constant for all women.
The stochastic model we obtain with these assumptions is that $X$ has a geometric distribution $X\sim Geo(p)$. What would be a good estimate for $p$?
Well, $p$ is just $p=P(X=1)$. Hence, by the Law of Large Numbers, $p\approx\frac{150}{500}=0.3$. However, we do not use all available data in this estimate, so its probably not optimal.