Supervised Learning and Classification

Problem: Given multivariate data, i.e. a number of things, drawn from a sample space T×U, learn a function in T->U, i.e. function to predict the output (dependent, endogenous) attributes in U, given the input (independent, exogenous) attributes in T.

Called "supervised" because examples are given (e.g. by a supervisor or expert) and the aim is to infer a function to mimic the given behaviour.

Supervised Classification is a special case of the above where U is a discrete attribute often called "the class".

This document is online at   http://www.csse.monash.edu.au/~lloyd/Archive/2005-07-Supervised/index.shtml   and contains hyper-links to other resources.

Examples of supervised learning

• Given {(x_i, y_i)} fit a function from some class (e.g. polynomial) to predict y given x.
  (e.g. Usually called polynomial regression.)
  
  
• Time-series: Predict x_i+1 given {x_j | j<i}.
  
  
• Supervised Classification . . .
  
  

Some terms

Training Set: A data set on which to learn the model.

Test Set: A data set given after learning, to test the accuracy of predictions.

Overfitting: When a too complex model is learned - often revealed when errors on the test set are much worse than those on the training set.

Supervised Classification

Supervised classification is a special case of supervised learning where the output-space, U, is a single discrete attribute often called "the class".

(cf Unsupervised classification where the number of classes, their spec's and memberships are all unknown.)

• decision tree (properly classification tree)
  
  
• decison (classification) graph, forest etc.
  
  
• artificial neural network (ANN) (one output node per class)

Now see decision trees