Scalable Regression Tree Learning on Hadoop using OpenPlanet
As scientific and engineering domains attempt to effectively analyze the deluge of data arriving from sensors and instruments, machine learning is becoming a key data mining tool to build prediction models. Regression tree is a popular learning model that combines decision trees and linear regression to forecast numerical target variables based on a set of input features. Map Reduce is well suited for addressing such data intensive learning applications, and a proprietary regression tree algorithm, PLANET, using MapReduce has been proposed earlier. In this paper, we describe an open source implement of this algorithm, OpenPlanet, on the Hadoop framework using a hybrid approach. Further, we evaluate the performance of OpenPlanet using realworld datasets from the Smart Power Grid domain to perform energy use forecasting, and propose tuning strategies of Hadoop parameters to improve the performance of the default configuration by 75% for a training dataset of 17 million tuples on a 64-core Hadoop cluster on FutureGrid.
- Research Organization:
- City of Los Angeles Department
- Sponsoring Organization:
- USDOE Office of Electricity (OE)
- DOE Contract Number:
- OE0000192
- OSTI ID:
- 1332538
- Report Number(s):
- DOE-USC-00192-101
- Resource Relation:
- Conference: International Workshop on MapReduce and its Applications, Delft, the Netherlands June 18- 19, 2012
- Country of Publication:
- United States
- Language:
- English
Similar Records
Large-scale seismic signal analysis with Hadoop
Large-scale seismic waveform quality metric calculation using Hadoop