Energy Management for MapReduce Clusters

The area of cluster-level energy management has attracted significant research attention over the past few years. One class of techniques to reduce the energy consumption of clusters is to selectively power down nodes during periods of low utilization to increase energy efficiency. One can think of a number of ways of selectively powering down nodes, each with varying impact on the workload response time and overall energy consumption. Since the MapReduce framework is becoming “ubiquitous”, the focus of this paper is on developing a framework for systematically considering various MapReduce node power down strategies, and their impact on the overall energy consumption and workload response time. We closely examine two extreme techniques that can be accommodated in this framework. The first is based on a recently proposed technique called “Covering Set” (CS) that keeps only a small fraction of the nodes powered up during periods of low utilization. At the other extreme is a ...

• Type: Journal
• Year: 2010
• Where: PVLDB
• Authors: Willis Lang, Jignesh M. Patel
• Keywords: Energy Consumption, Low Utilization, Overall Energy Consumption, PVLDB 2010