Adaptive linear dimensionality reduction for classification

摘要

Dimensionality reduction is the representation of high-dimensional patterns in a low-dimensional subspace based on a transformation which optimizes a specified criterion in the subspace. For pattern classification, the ideal criteria is the minimum achievable classification error (the Bayes error). Under strict assumptions of the pattern distribution, the Bayes error can be analytically expressed. We use this as a starting point to develop an adaptive algorithm that computes a linear transformation based on the minimization of a cost function that approximates the Bayes error in the subspace. Using kernel estimators we then relax the assumptions and extend the algorithm to more general pattern distributions. Our simulations with three synthetic and one real-data set indicate that the proposed algorithm substantially outperforms Fisher's Linear Discriminant.