In the first lecture, we look at the following situation:
We had to determine whether a fruit was apple or orange, based on 2 features (weight and greenness).
How can we make this description more distinct? The easiest way is to add more features. For example, some features that may describe our dataset better would be:
But, are they all equally good features? Of course not: the greenness is for sure more relevant than the age, for example.
So, do we need all these features? Probably not, but we need to choose the ones we want: i.e., order them based on importance.
This is what we will discuss: how to get all equally good features, and if not order them based on importance
The number of input variables or features for a dataset is referred to as its dimensionality.
For example, let’s take a very simple dataset containing 2 attributes called Height and Weight. In other words, we have a 2D data set:
If we add another dimension called Age to the same dataset, it becomes a 3-dimensional dataset and any observation lies in the 3-dimensional space.
Likewise, real-world datasets are described by many attributes (they are high-dimensional). The observations of those datasets lie in high-dimensional space, which is hard to imagine.