A bi-objective optimization method to produce a near-optimal number of classifiers and increase diversity in Bagging

摘要

Bagging is an old and powerful method in ensemble learning which creates an ensemble of classifiers over bootstraps through learning and then generates diverse classifiers. There are two main challenges in bagging method: (1) using bootstraps lead to less diversity compared to other ensemble methods, (2) since one cannot predetermine the number of bootstraps in bagging, some redundant classifiers may be generated which leads to lower classification speed, more need to memory and weakening the efficiency of bagging. In this paper, a new method is proposed based on the above-mentioned challenges which utilizes a multi-objective optimization approach with the two objectives of accuracy and diversity. Taking these two objectives simultaneously into account, some (near-optimal) bags are generated, where these number of bags (the least possible number of bags) are used for training the classifiers in bagging and lead to creating diverse and accurate bags. In this method, diverse bags are generated, while the redundant ones are pruned, simultaneously. The used objective function in calculating diversity is a new method that thoroughly computes the diversity among all bags. Reviewing the literature in this context and to the best of authors’ knowledge, one can imply that the proposed method is the first research that can generate accurate and diverse bags with the least possible number of bags using a multi-objective optimization approach. The classifiers are ultimately learned based on these generated bags. Experimental results by investigating 20 datasets and comparing the proposed method with 7 state-of-the-art methods show that the proposed approach generates fewer classifiers, while has higher accuracy. Moreover, according to the conducted nonparametric statistical tests, it is illustrated that the proposed method significantly outperforms the other methods.