Collaboratively weighted naive Bayes

摘要

Naive Bayes (NB) was once awarded as one of the top 10 data mining algorithms, but the unreliable probability estimation and the unrealistic attribute conditional independence assumption limit its performance. To alleviate these two primary weaknesses simultaneously, instance and attribute weighting has been recently proposed. However, the existing approach learns instance and attribute weights separately, without considering their interactions at all, which restricts the performance of the learned model. Therefore, in this study, we propose a novel approach to learning instance and attribute weights collaboratively and call the resulting model collaboratively weighted naive Bayes (CWNB). In CWNB, we first learn the weight of each training instance iteratively based on its estimated posterior probability loss to make the prior and conditional probabilities more accurate, then we incorporate these two probabilities into the conditional log-likelihood (CLL) formula, and at last we search the optimal weight of each attribute by maximizing the CLL. Extensive experimental results show that CWNB significantly outperforms the standard NB and all the other existing state-of-the-art competitors.