GrayWulf: Scalable Software Architecture for Data Intensive Computing

Big data presents new challenges to both cluster infrastructure software and parallel application design. We present a set of software services and design principles for data intensive computing with petabyte data sets, named GrayWulf† . These services are intended for deployment on a cluster of commodity servers similar to the well-known Beowulf clusters. We use the Pan-STARRS system currently under development as an example of the architecture and principles in action.

• Type: Conference
• Year: 2009
• Where: HICSS
• Authors: Yogesh Simmhan, Roger S. Barga, Catharine van Ingen, María A. Nieto-Santisteban, Laszlo Dobos, Nolan Li, Michael Shipway, Alexander S. Szalay, Sue Werner, Jim Heasley
• Keywords: Biometrics, Cluster Infrastructure Software, HICSS 2009, Parallel Application Design, Petabyte Data Sets, System Sciences