The pragmatics of information retrieval experimentation, revisited

摘要

The decisions that must be made by an investigator in carrying out an information retrieval experiment are described. Guidance is provided on a number of issues, specifically determining the need for testing, choosing the type of test (laboratory or operational), defining the variables, developing or using databases, finding queries, processing queries, assigning treatments to experimental units, collecting the data, analyzing the data, and presenting the results.