ROAR: increasing the flexibility and performance of distributed search

To search the web quickly, search engines partition the web index over many machines, and consult every partition when answering a query. To increase throughput, replicas are added for each of these machines. The key parameter of these algorithms is the trade-off between replication and partitioning: increasing the partitioning level improves query completion time since more servers handle the query, but may incur non-negligible startup costs for each subquery. Finding the right operating point and adapting to it can significantly improve performance and reduce costs. We introduce Rendezvous On a Ring (ROAR), a novel distributed algorithm that enables on-the-fly re-configuration of the partitioning level. ROAR can add and remove servers without stopping the system, cope with server failures, and provide good load-balancing even with a heterogeneous server pool. We demonstrate these claims using a privacy-preserving search application built upon ROAR. Categories and Subject Descript...

• Type: Conference
• Year: 2009
• Where: SIGCOMM
• Authors: Costin Raiciu, Felipe Huici, Mark Handley, David S. Rosenblum
• Keywords: Communications, Level Improves Query, Non-negligible Startup Costs, Partitioning Level, SIGCOMM 2009