A new method for ensemble combination based on adaptive decision making

摘要

Ensemble classifiers methods are considered as one of the strongest classification approaches in different applications. A key factor in creating a successful ensemble is to design an optimal combiner that minimizes classification errors. Unfortunately, finding the best fusion methodology that maximizes ensemble performance is a challenge. Most popular ensemble classifiers systems use fixed fusion methods such as average and majority voting. For a given application, there is no clear understanding of why a fusion method might work better than other methods. In this work, decision-making based on Adaptive Power Mean Combiner (APMC) is investigated. A mathematical and experimental framework for ensemble classifier systems based on APMC is proposed. The theoretical results explain why a given algebraic combiner such as average, product, maximum, minimum, and others works best for a given dataset while providing poor results on other datasets. It was shown that selecting an optimal fusion method is strongly dependent on the statistics of base learners’ outputs. Theoretical results for a given ensemble condition show that an improvement in classification accuracy of ∼8% compared to the average fusion method is achieved. Guided by theoretical results, we propose an experimental ensemble-learning algorithm based on surrogate optimization method for searching fusion methods for a given ensemble condition. The performance trend predicted by the mathematical model is validated through several datasets and compared against different fusion methods. Experimental results show the proposed algorithm achieves notable classification results compared to several fusion methods.