MS-I/O: A Distributed Multi-Storage I/O System

More and more parallel applications are running in a distributed environment to take advantage of easily available and inexpensive commodity resources. For data intensive applications, employing multiple distributed storage resources has many advantages. In this paper, we present a Multi-Storage I/O System (MS-I/O) that can not only effectively manage various distributed storage resources in the system, but also provide novel high performance storage access schemes. MS-I/O employs many state-of-the-art I/O optimizations such as collective I/O, asynchronous I/O etc. and a number of new techniques such as data location, data replication, subfile, superfile and data access history. In addition, many MS-I/O optimization schemes can work simultaneously within a single data access session, greatly improving the performance. Although I/O optimization techniques can help improve performance, it also complicates I/O system. In addition, most optimization techniques have their limitations. Th...

• Type: Conference
• Year: 2002
• Where: CCGRID
• Authors: Xiaohui Shen, Alok N. Choudhary
• Keywords: CCGRID 2002, Distributed And Parallel Computing, I/o Optimization, I/o Optimization Decisions, I/o System