Deflating the Dimensionality Curse Using Multiple Fractal Dimensions

Nearest neighbor queries are important in many settings, including spatial databases (Find the k closest cities) and multimedia databases (Find the k most similar images). Previous analyses have concluded that nearest neighbor search is hopeless in high dimensions, due to the notorious "curse of dimensionality". However, their precise analysis over real data sets is still an open problem. The typical and often implicit assumption in previous studies is that the data is uniformly distributed, with independence between attributes. However, real data sets overwhelmingly disobey these assumptions; rather, they typically are skewed and exhibit intrinsic ("fractal") dimensionalities that are much lower than their embedding dimension, e.g., due to subtle dependencies between attributes. In this paper, we show how the Hausdorff and Correlation fractal dimensions of a data set can yield extremely accurate formulas that can predict I/O performance to within one standard devi...

• Type: Conference
• Year: 2000
• Where: ICDE
• Authors: Bernd-Uwe Pagel, Flip Korn, Christos Faloutsos
• Keywords: Correlation Fractal Dimensions, Database, ICDE 2000, Nearest Neighbor Search, Real Data Sets