Local Dimensionality Reduction: A New Approach to Indexing High Dimensional Spaces

Many emerging application domains require database systems to support efficient access over highly multidimensional datasets. The current state-of-the-art technique to indexing high dimensional data is to first reduce the dimensionality of the data using Principal Component Analysis and then indexing the reduceddimensionality space using a multidimensional index structure. The above technique, referred to as global dimensionality reduction (GDR), works well when the data set is globally correlated, i.e. most of the variation in the data can be captured by a few dimensions. In practice, datasets are often not globally correlated. In such cases, reducing the data dimensionality using GDR causes significant loss of distance information resulting in a large number of false positives and hence a high query cost. Even when a global correlation does not exist, there may exist subsets of data that are locally correlated. In this paper, we propose a technique called Local Dimensionality Reduct...

• Type: Conference
• Year: 2000
• Where: VLDB
• Authors: Kaushik Chakrabarti, Sharad Mehrotra
• Keywords: Correlated Clusters, Database, Dimensionality, Dimensionality Reduction, VLDB 2000