Sciweavers

Share
INFOSCALE
2007
ACM

Load-balancing and caching for collection selection architectures

9 years 1 months ago
Load-balancing and caching for collection selection architectures
— To address the rapid growth of the Internet, modern Web search engines have to adopt distributed organizations, where the collection of indexed documents is partitioned among several servers, and query answering is performed as a parallel and distributed task. Collection selection can be a way to reduce the overall computing load, by finding a trade-off between the quality of results retrieved and the cost of solving queries. In this paper, we analyze the relationship between the collection selection strategy, the effect on load balancing and on the caching subsystem, by exploring the design-space of a distributed search engine based on collection selection. In particular, we propose a strategy to perform collection selection in a load-driven way, and a novel caching policy able to incrementally refine the effectiveness of the results returned for each subsequent cache hit. The combination of load-driven collection selection and incremental caching strategies allows our system to...
Diego Puppin, Fabrizio Silvestri, Raffaele Perego,
Added 26 Oct 2010
Updated 26 Oct 2010
Type Conference
Year 2007
Where INFOSCALE
Authors Diego Puppin, Fabrizio Silvestri, Raffaele Perego, Ricardo A. Baeza-Yates
Comments (0)
books