Sciweavers

Share
HIPC
2000
Springer

Meta-data Management System for High-Performance Large-Scale Scientific Data Access

10 years 1 months ago
Meta-data Management System for High-Performance Large-Scale Scientific Data Access
Many scientific applications manipulate large amount of data and, therefore, are parallelized on high-performance computing systems to take advantage of their computational power and memory space. The size of data processed by these large-scale applications can easily overwhelm the disk capacity of most systems. Thus, tertiary storage devices are used to store the data. The parallelization of this type of applications requires understanding of not only the data partition pattern among multiple processors but also the underlying storage architectures and the data storage pattern. In this paper, we present a meta-data management system which uses a database to record the information of datasets and manage these meta data to provide suitable I/O interface. As a result, users specify dataset names instead of data physical location to access data using optimal I/O calls without knowing the underlying storage structure. We use an astrophysics application to demonstrate that the management sy...
Wei-keng Liao, Xiaohui Shen, Alok N. Choudhary
Added 24 Aug 2010
Updated 24 Aug 2010
Type Conference
Year 2000
Where HIPC
Authors Wei-keng Liao, Xiaohui Shen, Alok N. Choudhary
Comments (0)
books