Probability of Type I and Type II errors

Recall:

On the y-axis, the true state of nature, and on the x-axis our decision.

Furthermore, if we reject $H_0$ in favour of $H_1$ when $T\in K$, then

$P(\text{Type I error})=P(T\in K\ | \ H_0\ \text{true})$

Moreover, the the probability of an error of Type II is:

$P(\text{Type II error})=P(T\not\in K\ | \ H_1\ \text{true})$

where $T$ is the test statistic and $K$ the critical region.

Let’s now apply these formulas to a example. Suppose a radar system needs to distinguish between the presence or absence of a target. Let $X$ be the received signal and consider the following two competing hypotheses:

• $H_0$: no target present, $X\sim N(0,1)$ (then, we would get a signal around $0$, with some possible noise)
• $H_1$: target present, $X\sim N(1,1)$ (then, we would get a signal around $1$, with the same level of noise as the previous case)

Suppose that $H_0$ is rejected in favor of $H_1$ if $X>c$. $c$ is some constant (we will talk about its value later on).

We then obtain the following probabilities of Type I and Type II errors:

• $P(\text{false alarm})=P(\text{Type I error})=P(X\ge c\ | \ H_0)=P(Z>c)$
• $P(\text{miss})=P(\text{Type II error})=P(X< c\ | \ H_1)=P(Z<c-1)$

where $Z\sim N(0,1)$.

We see that decreasing the probability of a false alarm comes at the cost of increasing the probability of a miss and vice versa. Graphically:

How can we improve this?

• If we can reduce the noise, say $N(0,0.1)$ and $N(1,0.1)$, we would have two curves that are way more spiked, not so spread out. The overlap will be much less, giving much better results.
• If we can’t reduce the signals, but we can get large signals, say $N(0,1)$ and $N(5,1)$; the red curve would still be the same, but the blue curve shifts 4 units to the right. The overlap will be much less, giving much better results.