Fast Matching for All Pairs Similarity Search

12 years 2 months ago
Fast Matching for All Pairs Similarity Search
All pairs similarity search is the problem of finding all pairs of records that have a similarity score above the specified threshold. Many real-world systems like search engines, online social networks, and digital libraries frequently have to solve this problem for datasets having millions of records in a high dimensional space, which are often sparse. The challenge is to design algorithms with feasible time requirements. To meet this challenge, algorithms have been proposed based on the inverted index, which maps each dimension to a list of records with non-zero projection along that dimension. Common to these algorithms is a threephase framework of data preprocessing, pairs matching, and indexing. Matching is the most time-consuming phase. Within this framework, we propose fast matching technique that uses the sparse nature of real-world data to effectively reduce the size of the search space through a systematic set of tighter filtering conditions and heuristic optimizations. ...
Amit C. Awekar, Nagiza F. Samatova
Added 25 May 2010
Updated 25 May 2010
Type Conference
Year 2009
Where WEBI
Authors Amit C. Awekar, Nagiza F. Samatova
Comments (0)