Assessing bias in search engines

摘要

This paper deals with the measurement of bias in search engines on the World Wide Web. Bias is taken to mean the balance and representativeness of items in a collection retrieved from a database for a set of queries. This calls for assessing the degree to which the distribution of items in a collection deviates from the ideal. Ascertaining this ideal poses problems similar to those associated with determining relevance in the measurement of recall and precision. Instead of enlisting subject experts or users to determine such an ideal, a family of comparable search engines is used to approximate it for a set of queries. The distribution is obtained by computing the frequencies of occurrence of the uniform resource locators (URLs) in the collection retrieved by several search engines for the given queries. Bias is assessed by measuring the deviation from the ideal of the distribution produced by a particular search engine.