Leveraging a scalable row store to build a distributed text index

Many content-oriented applications require a scalable text index. Building such an index is challenging. In addition to the logic of inserting and searching documents, developers have to worry about issues in a typical distributed environment, such as fault tolerance, incrementally growing the index cluster, and load balancing. We developed a distributed text index called HIndex, by judiciously exploiting the control layer of HBase, which is an open source implementation of Google’s Bigtable. Such leverage enables us to inherit the support on availability, elasticity and load balancing in HBase. We present the design, implementation, and a performance evaluation of HIndex in this paper. Categories and Subject Descriptors H.3.3 [Information Search and Retrieval] General Terms Algorithms, Performance, Design.

• Type: Conference
• Year: 2009
• Where: CIKM
• Authors: Ning Li, Jun Rao, Eugene J. Shekita, Sandeep Tata
• Keywords: CIKM 2009, Database, Load Balancing, Scalable Text Index, Text Index