Fast exact maximum likelihood estimation for mixture of language model

作者:

Highlights:

摘要

Language modeling is an effective and theoretically attractive probabilistic framework for text information retrieval. The basic idea of this approach is to estimate a language model of a given document (or document set), and then do retrieval or classification based on this model. A common language modeling approach assumes the data D is generated from a mixture of several language models. The core problem is to find the maximum likelihood estimation of one language model mixture, given the fixed mixture weights and the other language model mixture. The EM algorithm is usually used to find the solution.In this paper, we proof that an exact maximum likelihood estimation of the unknown mixture component exists and can be calculated using the new algorithm we proposed. We further improve the algorithm and provide an efficient algorithm of O(k) complexity to find the exact solution, where k is the number of words occurring at least once in data D. Furthermore, we proof the probabilities of many words are exactly zeros, and the MLE estimation is implemented as a feature selection technique explicitly.

论文关键词:Information retrieval,Language models,Mixture model

论文评审过程:Received 27 June 2007, Revised 26 November 2007, Accepted 11 December 2007, Available online 29 January 2008.

论文官网地址:https://doi.org/10.1016/j.ipm.2007.12.003