skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

This content will become publicly available on August 24, 2021

Title: The ASHRAE Great Energy Predictor III competition: Overview and results

Abstract

In late 2019, ASHRAE hosted the Great Energy Predictor III (GEPIII) machine learning competition on the Kaggle platform. This launch marked the third energy prediction competition from ASHRAE and the first since the mid-1990s. In this updated version, the competitors were provided with over 20 million points of training data from 2,380 energy meters collected for 1,448 buildings from 16 sources. This competition’s overall objective was to find the most accurate modeling solutions for the prediction of over 41 million private and public test data points. Furthermore, the competition had 4,370 participants, split across 3,614 teams from 94 countries who submitted 39,403 predictions. In addition to the top five winning workflows, the competitors publicly shared 415 reproducible online machine learning workflow examples (notebooks), including over 40 additional, full solutions. This paper gives a high-level overview of the competition preparation and dataset, competitors and their discussions, machine learning workflows and models generated, winners and their submissions, discussion of lessons learned, and competition outputs and next steps. The most popular and accurate machine learning workflows used large ensembles of mostly gradient boosting tree models, such as LightGBM. Similar to the first predictor competition, preprocessing of the data sets emerged as a keymore » differentiator.« less

Authors:
ORCiD logo [1]; ORCiD logo [2]; ORCiD logo [3]; ORCiD logo [1]; ORCiD logo [4]; ORCiD logo [5]; ORCiD logo [6];  [7]; ORCiD logo [5]; ORCiD logo [8];  [9]
  1. National Univ. of Singapore (NUS) (Singapore). Building and Urban Data Science (BUDS) Lab.
  2. Berkeley Education Alliance for Research in Singapore (BEARS), Singapore, Singapore
  3. University College Dublin (Ireland)
  4. National Univ. of Singapore (NUS) (Singapore). Building and Urban Data Science (BUDS) Lab.; Stanford Univ., CA (United States). Urban Informatics Lab.
  5. Univ. of Texas, Austin, TX (United States). Intelligent Environments Lab. (IEL)
  6. Performance Systems Development of NY, LLC, New York, NY (United States)
  7. Intertek Building Science Solutions, Seattle, WA (United States)
  8. National Renewable Energy Lab. (NREL), Golden, CO (United States)
  9. Texas A&M University, College Station, TX (United States). Energy Systems Lab.
Publication Date:
Research Org.:
National Renewable Energy Lab. (NREL), Golden, CO (United States)
Sponsoring Org.:
USDOE Office of Energy Efficiency and Renewable Energy (EERE), Energy Efficiency Office. Building Technologies Office
OSTI Identifier:
1660254
Report Number(s):
NREL/JA-5500-76088
Journal ID: ISSN 2374-4731; MainId:6941;UUID:d2af0500-d84d-ea11-9c31-ac162d87dfe5;MainAdminID:17381
Grant/Contract Number:  
AC36-08GO28308
Resource Type:
Accepted Manuscript
Journal Name:
Science and Technology for the Built Environment
Additional Journal Information:
Journal Name: Science and Technology for the Built Environment; Journal ID: ISSN 2374-4731
Publisher:
Taylor & Francis
Country of Publication:
United States
Language:
English
Subject:
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION; buildings; ASHRAE; building energy model; machine learning

Citation Formats

Miller, Clayton, Arjunan, Pandarasamy, Kathirgamanathan, Anjukan, Fu, Chun, Roth, Jonathan, Park, June Young, Balbach, Chris, Gowri, Krishnan, Nagy, Zoltan, Fontanini, Anthony D., and Haberl, Jeff. The ASHRAE Great Energy Predictor III competition: Overview and results. United States: N. p., 2020. Web. doi:10.1080/23744731.2020.1795514.
Miller, Clayton, Arjunan, Pandarasamy, Kathirgamanathan, Anjukan, Fu, Chun, Roth, Jonathan, Park, June Young, Balbach, Chris, Gowri, Krishnan, Nagy, Zoltan, Fontanini, Anthony D., & Haberl, Jeff. The ASHRAE Great Energy Predictor III competition: Overview and results. United States. doi:10.1080/23744731.2020.1795514.
Miller, Clayton, Arjunan, Pandarasamy, Kathirgamanathan, Anjukan, Fu, Chun, Roth, Jonathan, Park, June Young, Balbach, Chris, Gowri, Krishnan, Nagy, Zoltan, Fontanini, Anthony D., and Haberl, Jeff. Mon . "The ASHRAE Great Energy Predictor III competition: Overview and results". United States. doi:10.1080/23744731.2020.1795514.
@article{osti_1660254,
title = {The ASHRAE Great Energy Predictor III competition: Overview and results},
author = {Miller, Clayton and Arjunan, Pandarasamy and Kathirgamanathan, Anjukan and Fu, Chun and Roth, Jonathan and Park, June Young and Balbach, Chris and Gowri, Krishnan and Nagy, Zoltan and Fontanini, Anthony D. and Haberl, Jeff},
abstractNote = {In late 2019, ASHRAE hosted the Great Energy Predictor III (GEPIII) machine learning competition on the Kaggle platform. This launch marked the third energy prediction competition from ASHRAE and the first since the mid-1990s. In this updated version, the competitors were provided with over 20 million points of training data from 2,380 energy meters collected for 1,448 buildings from 16 sources. This competition’s overall objective was to find the most accurate modeling solutions for the prediction of over 41 million private and public test data points. Furthermore, the competition had 4,370 participants, split across 3,614 teams from 94 countries who submitted 39,403 predictions. In addition to the top five winning workflows, the competitors publicly shared 415 reproducible online machine learning workflow examples (notebooks), including over 40 additional, full solutions. This paper gives a high-level overview of the competition preparation and dataset, competitors and their discussions, machine learning workflows and models generated, winners and their submissions, discussion of lessons learned, and competition outputs and next steps. The most popular and accurate machine learning workflows used large ensembles of mostly gradient boosting tree models, such as LightGBM. Similar to the first predictor competition, preprocessing of the data sets emerged as a key differentiator.},
doi = {10.1080/23744731.2020.1795514},
journal = {Science and Technology for the Built Environment},
number = ,
volume = ,
place = {United States},
year = {2020},
month = {8}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on August 24, 2021
Publisher's Version of Record

Save / Share:

Works referenced in this record:

A review on the prediction of building energy consumption
journal, August 2012

  • Zhao, Hai-xiang; Magoulès, Frédéric
  • Renewable and Sustainable Energy Reviews, Vol. 16, Issue 6
  • DOI: 10.1016/j.rser.2012.02.049

Application of automated measurement and verification to utility energy efficiency program data
journal, May 2017


CityLearn v1.0: An OpenAI Gym Environment for Demand Response with Deep Reinforcement Learning
conference, November 2019

  • Vázquez-Canteli, José R.; Kämpf, Jérôme; Henze, Gregor
  • BuildSys '19: The 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation
  • DOI: 10.1145/3360322.3360998

Modeling and forecasting building energy consumption: A review of data-driven techniques
journal, July 2019

  • Bourdeau, Mathieu; Zhai, Xiao qiang; Nefzaoui, Elyes
  • Sustainable Cities and Society, Vol. 48
  • DOI: 10.1016/j.scs.2019.101533

More Buildings Make More Generalizable Models—Benchmarking Prediction Methods on Open Electrical Meter Data
journal, August 2019

  • Miller, Clayton
  • Machine Learning and Knowledge Extraction, Vol. 1, Issue 3
  • DOI: 10.3390/make1030056

The Building Data Genome Project: An open, public data set from non-residential building electrical meters
journal, September 2017


XGBoost: A Scalable Tree Boosting System
conference, January 2016

  • Chen, Tianqi; Guestrin, Carlos
  • Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD '16
  • DOI: 10.1145/2939672.2939785

Statistical Analysis of Neural Networks as Applied to Building Energy Prediction
journal, February 2004

  • Dodier, Robert H.; Henze, Gregor P.
  • Journal of Solar Energy Engineering, Vol. 126, Issue 1
  • DOI: 10.1115/1.1637640

A review of data-driven building energy consumption prediction studies
journal, January 2018


Automated measurement and verification: Performance of public domain whole-building electric baseline models
journal, April 2015


Accuracy of automated measurement and verification (M&V) techniques for energy savings in commercial buildings
journal, July 2016


Automated daily pattern filtering of measured building performance data
journal, January 2015


ImageNet Large Scale Visual Recognition Challenge
journal, April 2015

  • Russakovsky, Olga; Deng, Jia; Su, Hao
  • International Journal of Computer Vision, Vol. 115, Issue 3
  • DOI: 10.1007/s11263-015-0816-y