Fading Away: Dilution and User Behaviour

When faced with a poor set of document summaries on the first page of returned search results, a user may respond in various ways: by proceeding on to the next page of results; by entering another query; by switching to another service; or by abandoning their search. We analyse this aspect of searcher behaviour using a commercial search system, comparing a deliberately degraded system to the original one. Our results demonstrate that searchers naturally avoid selecting poor results as answers given the degraded system; however, the depth of the ranking that they view, their query reformulation rate, and the amount of time required to complete search tasks, are all remarkably unchanged. Categories and Subject Descriptors H.3.4 [Information Storage and Retrieval]: Systems and software—performance evaluation. General Terms Experimentation, measurement. Keywords Retrieval experiment, evaluation, system measurement.

• Type: Journal
• Year: 2013
• Where: EUROHCIR
• Authors: Paul Thomas, Falk Scholer, Alistair Moffat
• Keywords: EUROHCIR 2013