As buildings account for approximately 40% of global energy consumption and associated greenhouse gas emissions, their role in decarbonizing the power grid is crucial. The increased integration of variable energy sources, such as renewables, introduces uncertainties and unprecedented flexibilities, necessitating buildings to adapt their energy demand to enhance grid resiliency. Consequently, buildings must transition from passive energy consumers to active grid assets, providing demand flexibility and energy elasticity while maintaining occupant comfort and health. This fundamental shift demands advanced optimal control methods to manage escalating energy demand and avert power outages. Reinforcement learning (RL) emerges as a promising method to address these challenges. Here, in this paper, we explore ten questions related to the application of RL in buildings, specifically targeting flexible energy management. We consider the growing availability of data, advancements in machine learning algorithms, open-source tools, and the practical deployment aspects associated with software and hardware requirements. Our objective is to deliver a comprehensive introduction to RL, present an overview of existing research and accomplishments, underscore the challenges and opportunities, and propose potential future research directions to expedite the adoption of RL for building energy management.
Nagy, Zoltan, et al. "Ten questions concerning reinforcement learning for building energy management." Building and Environment, vol. 241, Aug. 2023. https://doi.org/10.1016/j.buildenv.2023.110435
Nagy, Zoltan, Henze, Gregor, Dey, Sourav, et al., "Ten questions concerning reinforcement learning for building energy management," Building and Environment 241 (2023), https://doi.org/10.1016/j.buildenv.2023.110435
@article{osti_1984043,
author = {Nagy, Zoltan and Henze, Gregor and Dey, Sourav and Arroyo, Javier and Helsen, Lieve and Zhang, Xiangyu and Chen, Bingqing and Amasyali, Kadir and Kurte, Kuldeep and Zamzam, Ahmed and others},
title = {Ten questions concerning reinforcement learning for building energy management},
annote = {As buildings account for approximately 40% of global energy consumption and associated greenhouse gas emissions, their role in decarbonizing the power grid is crucial. The increased integration of variable energy sources, such as renewables, introduces uncertainties and unprecedented flexibilities, necessitating buildings to adapt their energy demand to enhance grid resiliency. Consequently, buildings must transition from passive energy consumers to active grid assets, providing demand flexibility and energy elasticity while maintaining occupant comfort and health. This fundamental shift demands advanced optimal control methods to manage escalating energy demand and avert power outages. Reinforcement learning (RL) emerges as a promising method to address these challenges. Here, in this paper, we explore ten questions related to the application of RL in buildings, specifically targeting flexible energy management. We consider the growing availability of data, advancements in machine learning algorithms, open-source tools, and the practical deployment aspects associated with software and hardware requirements. Our objective is to deliver a comprehensive introduction to RL, present an overview of existing research and accomplishments, underscore the challenges and opportunities, and propose potential future research directions to expedite the adoption of RL for building energy management.},
doi = {10.1016/j.buildenv.2023.110435},
url = {https://www.osti.gov/biblio/1984043},
journal = {Building and Environment},
issn = {ISSN 0360-1323},
volume = {241},
place = {United States},
publisher = {Elsevier},
year = {2023},
month = {08}}
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States); National Renewable Energy Laboratory (NREL), Golden, CO (United States); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States); Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
Sponsoring Organization:
USDOE Office of Energy Efficiency and Renewable Energy (EERE), Energy Efficiency Office. Building Technologies Office
BuildSys '18: The 5th ACM International Conference on Systems for Built Environments, Proceedings of the 5th Conference on Systems for Built Environmentshttps://doi.org/10.1145/3276774.3276775
BuildSys '19: The 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportationhttps://doi.org/10.1145/3360322.3360849
Vázquez-Canteli, José R.; Kämpf, Jérôme; Henze, Gregor
BuildSys '19: The 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportationhttps://doi.org/10.1145/3360322.3360998
Vazquez-Canteli, Jose R.; Henze, Gregor; Nagy, Zoltan
Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportationhttps://doi.org/10.1145/3408308.3427604
Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportationhttps://doi.org/10.1145/3408308.3427617
von Frankenberg, Nadine; Loftness, Vivian; Bruegge, Bernd
Proceedings of the 8th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportationhttps://doi.org/10.1145/3486611.3486672
Quintana, Matias; Nagy, Zoltan; Tartarini, Federico
Proceedings of the 9th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportationhttps://doi.org/10.1145/3563357.3566167