Data-driven reinforcement learning–based real-time energy management system for plug-in hybrid electric vehicles

Qi, Xuewei; Wu, Guoyuan; Boriboonsomsin, Kanok; Barth, Matthew J.; Gonder, Jeffrey

doi:10.3141/2572-01

Title: Data-driven reinforcement learning–based real-time energy management system for plug-in hybrid electric vehicles

Journal Article · Fri Jan 01 00:00:00 EST 2016 · Transportation Research Record: Journal of the Transportation Research Board

DOI:https://doi.org/10.3141/2572-01· OSTI ID:1239892

Qi, Xuewei ^[1]; Wu, Guoyuan ^[1]; Boriboonsomsin, Kanok ^[1]; Barth, Matthew J. ^[1]; Gonder, Jeffrey ^[2]

Univ. of California, Riverside, CA (United States)
National Renewable Energy Lab. (NREL), Golden, CO (United States)

Plug-in hybrid electric vehicles (PHEVs) show great promise in reducing transportation-related fossil fuel consumption and greenhouse gas emissions. Designing an efficient energy management system (EMS) for PHEVs to achieve better fuel economy has been an active research topic for decades. Most of the advanced systems rely either on a priori knowledge of future driving conditions to achieve the optimal but not real-time solution (e.g., using a dynamic programming strategy) or on only current driving situations to achieve a real-time but nonoptimal solution (e.g., rule-based strategy). This paper proposes a reinforcement learning–based real-time EMS for PHEVs to address the trade-off between real-time performance and optimal energy savings. The proposed model can optimize the power-split control in real time while learning the optimal decisions from historical driving cycles. Here, a case study on a real-world commute trip shows that about a 12% fuel saving can be achieved without considering charging opportunities; further, an 8% fuel saving can be achieved when charging opportunities are considered, compared with the standard binary mode control strategy.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: National Renewable Energy Laboratory (NREL), Golden, CO (United States)

Sponsoring Organization:: U.S. Department of Transportation

Grant/Contract Number:: AC36-08GO28308

OSTI ID:: 1239892

Report Number(s):: NREL/JA-5400-65413

Journal Information:: Transportation Research Record: Journal of the Transportation Research Board, Vol. 2572; ISSN 0361-1981

Publisher:: National Academy of Sciences, Engineering and MedicineCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 81 works

Citation information provided by
Web of Science

References (18)

Approximate Dynamic Programming: Solving the Curses of Dimensionality Powell, Warren B. Wiley Series in Probability and Statistics https://doi.org/10.1002/9781118029176	book	August 2011
Adaptive traffic signal control using approximate dynamic programming Cai, Chen; Wong, Chi Kwong; Heydecker, Benjamin G. Transportation Research Part C: Emerging Technologies, Vol. 17, Issue 5 https://doi.org/10.1016/j.trc.2009.04.005	journal	October 2009
Modeling and Control of a Power-Split Hybrid Vehicle IEEE Transactions on Control Systems Technology, Vol. 16, Issue 6 https://doi.org/10.1109/TCST.2008.919447	journal	November 2008
Commuter Route Optimized Energy Management of Hybrid Electric Vehicles Larsson, Viktor; Johannesson Mardh, Lars; Egardt, Bo IEEE Transactions on Intelligent Transportation Systems, Vol. 15, Issue 3 https://doi.org/10.1109/TITS.2013.2294723	journal	June 2014
Comparative Study on Different Energy Management Strategies for Plug-In Hybrid Electric Vehicles Wang, Ximing; He, Hongwen; Sun, Fengchun Energies, Vol. 6, Issue 11 https://doi.org/10.3390/en6115656	journal	October 2013
A Supervisory Control Strategy for Plug-In Hybrid Electric Vehicles Based on Energy Demand Prediction and Route Preview Tianheng, Feng; Lin, Yang; Qing, Gu IEEE Transactions on Vehicular Technology, Vol. 64, Issue 5 https://doi.org/10.1109/TVT.2014.2336378	journal	May 2015
A Real Time Energy Management Strategy for Plug-in Hybrid Electric Vehicles based on Optimal Control Theory Tribioli, Laura; Barbieri, Michele; Capata, Roberto Energy Procedia, Vol. 45 https://doi.org/10.1016/j.egypro.2014.01.100	journal	January 2014
Energy management of a power-split plug-in hybrid electric vehicle based on genetic algorithm and quadratic programming Chen, Zheng; Mi, Chris Chunting; Xiong, Rui Journal of Power Sources, Vol. 248 https://doi.org/10.1016/j.jpowsour.2013.09.085	journal	February 2014
Energy management of plug-in hybrid electric vehicles with unknown trip length Hou, Cong; Xu, Liangfei; Wang, Hewu Journal of the Franklin Institute, Vol. 352, Issue 2 https://doi.org/10.1016/j.jfranklin.2014.07.009	journal	February 2015
Development and Evaluation of an Intelligent Energy-Management Strategy for Plug-in Hybrid Electric Vehicles Wu, Guoyuan; Boriboonsomsin, Kanok; Barth, Matthew J. IEEE Transactions on Intelligent Transportation Systems, Vol. 15, Issue 3 https://doi.org/10.1109/TITS.2013.2294342	journal	June 2014
Trip-Oriented Energy Management Control Strategy for Plug-In Hybrid Electric Vehicles Yu, Hai; Kuang, Mong; McGee, Ryan IEEE Transactions on Control Systems Technology, Vol. 22, Issue 4, p. 1323-1336 https://doi.org/10.1109/TCST.2013.2278684	journal	July 2014
Fuzzy-based blended control for the energy management of a parallel plug-in hybrid electric vehicle Dubois, Maxime R.; Desrochers, Alain; Denis, Nicolas IET Intelligent Transport Systems, Vol. 9, Issue 1 https://doi.org/10.1049/iet-its.2014.0075	journal	February 2015
Fuzzy energy management strategy for plug-in hev based on driving cycle modeling Wu, Jian 2014 33rd Chinese Control Conference (CCC), Proceedings of the 33rd Chinese Control Conference https://doi.org/10.1109/ChiCC.2014.6895690	conference	July 2014
Analysis of energy management strategies in plug-in hybrid electric vehicles: Application to the GM Chevrolet Volt Tribioli, Laura; Onori, Simona 2013 American Control Conference (ACC) https://doi.org/10.1109/ACC.2013.6580774	conference	June 2013
Trip Based Near Globally Optimal Power Management of Plug-in Hybrid Electric Vehicles Using Gas-Kinetic Traffic Flow Model Gong, Qiuming; Li, Yaoyu; Peng, Zhong-Ren IFAC Proceedings Volumes, Vol. 41, Issue 2 https://doi.org/10.3182/20080706-5-KR-1001.00785	journal	January 2008
Evolutionary algorithm based on-line PHEV energy management system with self-adaptive SOC control Qi, Xuewei; Wu, Guoyuan; Boriboonsomsin, Kanok 2015 IEEE Intelligent Vehicles Symposium (IV) https://doi.org/10.1109/IVS.2015.7225722	conference	June 2015
Trip-oriented Energy Management Control strategy for plug-in hybrid electric vehicles Yu, Hai; Kuang, Ming; McGee, Ryan 2011 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC 2011) https://doi.org/10.1109/cdc.2011.6160609	conference	December 2011
Approximate Dynamic Programming: Solving the Curses of Dimensionality Powell, Warren B. https://doi.org/10.1002/9780470182963	book	January 2007

Cited By (1)

Energy Management Strategy for a Hybrid Electric Vehicle Based on Deep Reinforcement Learning Hu, Yue; Li, Weimin; Xu, Kun Applied Sciences, Vol. 8, Issue 2 https://doi.org/10.3390/app8020187	journal	January 2018

Similar Records

Plug-in hybrid electric vehicles : How does one determine their potential for reducing U.S. oil dependence?

Journal Article · Mon Sep 01 00:00:00 EDT 2008 · WEVA · OSTI ID:1239892

Vyas, A; Santini, D; Duoba, M; +2 more

Cost-Effective and Ecofriendly Plug-In Hybrid Electric Vehicle Charging Management

Journal Article · Sun Jan 01 00:00:00 EST 2017 · Transportation Research Record: Journal of the Transportation Research Board · OSTI ID:1239892

Kontou, Eleftheria; Yin, Yafeng; Ge, Ying-en

Using Global Positioning System Travel Data to Assess Real-World Energy Use of Plug-In Hybrid Electric Vehicles

Journal Article · Mon Jan 01 00:00:00 EST 2007 · Transportation Research Record: Journal of the Transportation Research Board · OSTI ID:1239892

Gonder, J; Markel, T; Thornton, M; +1 more

Related Subjects

33 ADVANCED PROPULSION SYSTEMS
plug-in hybrid electric vehicles (PHEV)
energy management system (EMS)
approximate dynamic programming
reinforcement learning (RL)

Title: Data-driven reinforcement learning–based real-time energy management system for plug-in hybrid electric vehicles

Citation Formats

References (18)

Cited By (1)

Similar Records

Related Subjects