What is a classifier

Classification uses a function $f$ (called a classifier) to map input $x$ to class $y$.

$y = f (x) : f$ takes in $x ∈ X$ and declares its class to be $y ∈ Y$

The problem is two-fold:

• Define the classifier $f$ and its parameters.
• Learn the classification rule using a training set of “labelled data.”

Plug-in Bayes Rule

Classify to class 1 when: $p(y_1|x)>p(y_2|x)$.

May be easier to estimate the class-conditional probability density: $\hat{p}(x|y_1),\ \hat{p}(x|y_2)$.

Use Bayes’ rule to transform one into the other:

$$ p(y|x)=\frac{p(x|y)p(y)}{p(x)} $$

Putting just the direct estimate, $\hat{p}(x|y)$, in results in plug-in Bayes rule.

So, for Bayes rule we need to estimate:

1. the class priors $\hat{p}(y)$
2. the unconditional probabilities $\hat{p}(x)$
3. the class-conditional probabilities $\hat{p}(x|y)$

So, given a training set:

• How can you estimate $\hat{p}(y)$?
• How to estimate the (unconditional) data distribution $\hat{p}(x)$?
• Estimate the class of conditional probabilities