Sciweavers

Share
FAST
2009

Spyglass: Fast, Scalable Metadata Search for Large-Scale Storage Systems

8 years 1 months ago
Spyglass: Fast, Scalable Metadata Search for Large-Scale Storage Systems
The scale of today's storage systems has made it increasingly difficult to find and manage files. To address this, we have developed Spyglass, a file metadata search system that is specially designed for large-scale storage systems. Using an optimized design, guided by an analysis of real-world metadata traces and a user study, Spyglass allows fast, complex searches over file metadata to help users and administrators better understand and manage their files. Spyglass achieves fast, scalable performance through the use of several novel metadata search techniques that exploit metadata search properties. Flexible index control is provided by an index partitioning mechanism that leverages namespace locality. Signature files are used to significantly reduce a query's search space, improving performance and scalability. Snapshot-based metadata collection allows incremental crawling of only modified files. A novel index versioning mechanism provides both fast index updates and &quo...
Andrew W. Leung, Minglong Shao, Timothy Bisson, Sh
Added 17 Feb 2011
Updated 17 Feb 2011
Type Journal
Year 2009
Where FAST
Authors Andrew W. Leung, Minglong Shao, Timothy Bisson, Shankar Pasupathy, Ethan L. Miller
Comments (0)
books