Classification is the problem of identifying which set of categories a new observation belongs to. To make this definition more clear let’s look at an example : The following figure shows a common classification problem. We have a cloud of points divided into two classes. A new point similar to the preceding ones presents itself, its class unknown. The goal is to assign a class using the fact that we know the class of membership of other points.
How do we classify the new point for which the class of membership is unknown? A simple method is to assign this new point the same class as the closest point belonging to the initial cloud. This is also known as the nearest neighbor method. It is easy to implement but not to use because often eats up valuable computing time when the cloud of points is consistent.
Bayesian procedures, Binary and multi-class classification, Linear classifiers (Logistic regression)