Energy Proportionality and Performance in Data Parallel Computing Clusters
Energy consumption in datacenters has recently become a major concern due to the rising operational costs andscalability issues. Recent solutions to this problem propose the principle of energy proportionality, i.e., the amount of energy consumedby the server nodes must be proportional to the amount of work performed. For data parallelism and fault tolerancepurposes, most common file systems used in MapReduce-type clusters maintain a set of replicas for each data block. A coveringset is a group of nodes that together contain at least one replica of the data blocks needed for performing computing tasks. In thiswork, we develop and analyze algorithms to maintain energy proportionality by discovering a covering set that minimizesenergy consumption while placing the remaining nodes in lowpower standby mode. Our algorithms can also discover coveringsets in heterogeneous computing environments. In order to allow more data parallelism, we generalize our algorithms so that itcan discover k-covering sets, i.e., a set of nodes that contain at least k replicas of the data blocks. Our experimental results showthat we can achieve substantial energy saving without significant performance loss in diverse cluster configurations and workingenvironments.
- Research Organization:
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
- Sponsoring Organization:
- Computational Research Division
- DOE Contract Number:
- DE-AC02-05CH11231
- OSTI ID:
- 1012478
- Report Number(s):
- LBNL-4533E; TRN: US201110%%113
- Resource Relation:
- Conference: Scientific and Statistical Data Base Management Conference
- Country of Publication:
- United States
- Language:
- English
Similar Records
Center for Technology for Advanced Scientific Componet Software (TASCS)
A Novel Coarsening Method for Scalable and Efficient Mesh Generation