A statistical unified framework for rank-based multiple classifier decision combination

摘要

This study presents a theoretical investigation of the rank-based multiple classifier decision combination problem, with the aim of providing a unified framework to understand a variety of such systems. The combination of the decisions of more than one classifiers with the aim of improving overall system performance is a concept of general interest in pattern recognition, as a viable alternative to designing a single sophisticated classifier. The problem of combining the classifier decisions in the raw form of candidate class rankings is formulated as a discrete optimization problem. The objective function to be maximized is selected as the overall probability of correct decision. This formulation introduces a set of observation statistics about the joint behavior of the classifiers which are to be estimated by observing the classifiers operated on a cross-validation test set. The resulting binary programming problem is shown to have a simple and global optimum solution but which also necessitates a prohibitive number of observation statistics. From the objective function expansion, the problem observation space is defined and a method based on partitioning is introduced to reduce its prohibitive dimensionality. Within this partitioning formalism called as the Partitioned Observation Space (POS) theory, the number of behavior observation statistics can be reduced to levels which are feasible to estimate from the available cross-validation test data. It is shown by examples that such specific partitionings can be defined when reasonable assumptions or prior knowledge about the classifiers are incorporated into the problem domain. It is also demonstrated that certain specific partitionings of the classifier observation space effectively lead to the highest rank, Borda count and logistic regression rank-based decision combination methods from the literature. The analysis presented is general and promises to lead to a class of algorithms for rank-based decision combination. The potential of the theory and practical issues in implementation are illustrated by applying it in a real-life phonetic discrimination problem from speech pattern classification with encouraging results.