A probabilistic model for stemmer generation

作者:

Highlights:

摘要

In this paper we will present a language-independent probabilistic model which can automatically generate stemmers. Stemmers can improve the retrieval effectiveness of information retrieval systems, however the designing and the implementation of stemmers requires a laborious amount of effort due to the fact that documents and queries are often written or spoken in several different languages. The probabilistic model proposed in this paper aims at the development of stemmers used for several languages. The proposed model describes the mutual reinforcement relationship between stems and derivations and then provides a probabilistic interpretation. A series of experiments shows that the stemmers generated by the probabilistic model are as effective as the ones based on linguistic knowledge.

论文关键词:Multi-lingual information retrieval,Stemming,Mutual reinforcement,Web information retrieval

论文评审过程:Available online 21 May 2004.

论文官网地址:https://doi.org/10.1016/j.ipm.2004.04.006