Probability vs Statistics

Probability and statistics are deeply connected because all statistical statements are at bottom statements about probability. Despite this the two sometimes feel like very different subjects.

In Probability, there are a few rules and answers all follow logically from the rules, though computations can be tricky. In statistics we apply probability to draw conclusions from data.

Let’s see some examples:

• Probability example. You have a fair coin (equal probability of heads or tails). You will toss it 100 times. What is the probability of 60 or more heads? There is only one answer (about 0.028444) and we will learn how to compute it.
• Statistics example. You have a coin of unknown provenance. To investigate whether it is fair you toss it 100 times and count the number of heads. Let’s say you count 60 heads. Your job as a statistician is to draw a conclusion (inference) from this data. There are many ways to proceed, both in terms of the form the conclusion takes and the probability computations used to justify the conclusion. In fact, different statisticians might draw different conclusions.

Note that in the first example the random process is fully known (probability of heads =.5). The objective is to find the probability of a certain outcome (at least 60 heads) arising from the random process. In the second example, the outcome is known (60 heads) and the objective is to illuminate the unknown random process (the probability of heads).

Probability:

• Calculate probability, distribution and expectations of events
• Relation between multiple probabilistic variables
• Logically self-contained
• A few rules for computing probabilities
• One correct answer

Statistics:

• Summary of data
• Estimate parameters
• Messier and more of an art
• Get experimental data and try to draw probabilistic conclusions