Sciweavers

Share
USENIX
2008

Free Factories: Unified Infrastructure for Data Intensive Web Services

9 years 14 days ago
Free Factories: Unified Infrastructure for Data Intensive Web Services
We introduce the Free Factory, a platform for deploying data-intensive web services using small clusters of commodity hardware and free software. Independently administered virtual machines called Freegols give application developers the flexibility of a general purpose web server, along with access to distributed batch processing, cache and storage services. Each cluster exploits idle RAM and disk space for cache, and reserves disks in each node for high bandwidth storage. The batch processing service uses a variation of the MapReduce model. Virtualization allows every CPU in the cluster to participate in batch jobs. Each 48-node cluster can achieve 4-8 gigabytes per second of disk I/O. Our intent is to use multiple clusters to process hundreds of simultaneous requests on multi-hundred terabyte data sets. Currently, our applications achieve 1 gigabyte per second of I/O with 123 disks by scheduling batch jobs on two clusters, one of which is located in a remote data center.
Alexander Wait Zaranek, Tom Clegg, Ward Vandewege,
Added 02 Oct 2010
Updated 02 Oct 2010
Type Conference
Year 2008
Where USENIX
Authors Alexander Wait Zaranek, Tom Clegg, Ward Vandewege, George M. Church
Comments (0)
books