Group file operations for scalable tools and middleware

10 years 1 months ago
Group file operations for scalable tools and middleware
Group file operations are a new, intuitive idiom for tools and middleware - including parallel debuggers and runtimes, performance measurement and steering, and distributed resource management - that require scalable operations on large groups of distributed files. The idiom provides new semantics for using file groups in standard file operations to eliminate costly iteration. A filebased idiom promotes conciseness and portability, and eases adoption. With explicit semantics for aggregation of group results, the idiom addresses a key scalability barrier. We have designed TBON-FS, a new distributed file system that provides scalable group file operations by leveraging tree-based overlay networks (TBONs) for scalable communication and data aggregation. We integrated group file operations into several tools: parallel versions of common utilities including cp, grep, rsync, tail, and top, and the Ganglia Distributed Monitoring System. Our experience verifies the group file operation idiom i...
Michael J. Brim, Barton P. Miller
Added 18 Feb 2011
Updated 18 Feb 2011
Type Journal
Year 2009
Where HIPC
Authors Michael J. Brim, Barton P. Miller
Comments (0)