Sciweavers

Share
SSDBM
2006
IEEE

HDF5-FastQuery: Accelerating Complex Queries on HDF Datasets using Fast Bitmap Indices

11 years 5 months ago
HDF5-FastQuery: Accelerating Complex Queries on HDF Datasets using Fast Bitmap Indices
Large scale scientific data is often stored in scientific data formats such as FITS, netCDF and HDF. These storage formats are of particular interest to the scientific user community since they provide multi-dimensional storage and retrieval. However, one of the drawbacks of these storage formats is that they do not support semantic indexing which is important for interactive data analysis where scientists look for features of interests such as “Find all supernova explosions where energy > 105 and temperature > 106 ”. In this paper we present a novel approach called HDF5FastQuery to accelerate the data access of large HDF5 files by introducing multi-dimensional semantic indexing. Our implementation leverages an efficient indexing technology called bitmap indexing that has been widely used in the database community. Bitmap indices are especially well suited for interactive exploration of large-scale readonly data. Storing the bitmap indices into the HDF5 file has the fo...
Luke J. Gosink, John Shalf, Kurt Stockinger, Keshe
Added 12 Jun 2010
Updated 12 Jun 2010
Type Conference
Year 2006
Where SSDBM
Authors Luke J. Gosink, John Shalf, Kurt Stockinger, Kesheng Wu, Wes Bethel
Comments (0)
books