A junction-tree based learning algorithm to optimize network wide traffic control: A coordinated multi-agent framework
Abstract
Our study develops a novel reinforcement learning algorithm for the challenging coordinated signal control problem. Traffic signals are modeled as intelligent agents interacting with the stochastic traffic environment. The model is built on the framework of coordinated reinforcement learning. The Junction Tree Algorithm (JTA) based reinforcement learning is proposed to obtain an exact inference of the best joint actions for all the coordinated intersections. Moreover, the algorithm is implemented and tested with a network containing 18 signalized intersections in VISSIM. Finally, our results show that the JTA based algorithm outperforms independent learning (Q-learning), real-time adaptive learning, and fixed timing plans in terms of average delay, number of stops, and vehicular emissions at the network level.
- Authors:
-
- Purdue Univ., West Lafayette, IN (United States)
- Publication Date:
- Research Org.:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
- Sponsoring Org.:
- USDOE, National Science Foundation (NSF)
- OSTI Identifier:
- 1265896
- Grant/Contract Number:
- AC05-00OR22725; 1004528; 104IPY04
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Transportation Research Part C: Emerging Technologies
- Additional Journal Information:
- Journal Volume: 58; Journal Issue: PC; Journal ID: ISSN 0968-090X
- Publisher:
- Elsevier
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; Traffic Control; Learning Algorithm; Multi-agent control; Connected Vehicle
Citation Formats
Zhu, Feng, Aziz, H. M. Abdul, Qian, Xinwu, and Ukkusuri, Satish V. A junction-tree based learning algorithm to optimize network wide traffic control: A coordinated multi-agent framework. United States: N. p., 2015.
Web. doi:10.1016/j.trc.2014.12.009.
Zhu, Feng, Aziz, H. M. Abdul, Qian, Xinwu, & Ukkusuri, Satish V. A junction-tree based learning algorithm to optimize network wide traffic control: A coordinated multi-agent framework. United States. https://doi.org/10.1016/j.trc.2014.12.009
Zhu, Feng, Aziz, H. M. Abdul, Qian, Xinwu, and Ukkusuri, Satish V. Sat .
"A junction-tree based learning algorithm to optimize network wide traffic control: A coordinated multi-agent framework". United States. https://doi.org/10.1016/j.trc.2014.12.009. https://www.osti.gov/servlets/purl/1265896.
@article{osti_1265896,
title = {A junction-tree based learning algorithm to optimize network wide traffic control: A coordinated multi-agent framework},
author = {Zhu, Feng and Aziz, H. M. Abdul and Qian, Xinwu and Ukkusuri, Satish V.},
abstractNote = {Our study develops a novel reinforcement learning algorithm for the challenging coordinated signal control problem. Traffic signals are modeled as intelligent agents interacting with the stochastic traffic environment. The model is built on the framework of coordinated reinforcement learning. The Junction Tree Algorithm (JTA) based reinforcement learning is proposed to obtain an exact inference of the best joint actions for all the coordinated intersections. Moreover, the algorithm is implemented and tested with a network containing 18 signalized intersections in VISSIM. Finally, our results show that the JTA based algorithm outperforms independent learning (Q-learning), real-time adaptive learning, and fixed timing plans in terms of average delay, number of stops, and vehicular emissions at the network level.},
doi = {10.1016/j.trc.2014.12.009},
journal = {Transportation Research Part C: Emerging Technologies},
number = PC,
volume = 58,
place = {United States},
year = {Sat Jan 31 00:00:00 EST 2015},
month = {Sat Jan 31 00:00:00 EST 2015}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 41 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Reinforcement Learning for True Adaptive Traffic Signal Control
journal, May 2003
- Abdulhai, Baher; Pringle, Rob; Karakoulas, Grigoris J.
- Journal of Transportation Engineering, Vol. 129, Issue 3
Store-and-forward based methods for the signal control problem in large-scale congested urban road networks
journal, April 2009
- Aboudolas, K.; Papageorgiou, M.; Kosmatopoulos, E.
- Transportation Research Part C: Emerging Technologies, Vol. 17, Issue 2
Unified Framework for Dynamic Traffic Assignment and Signal Control with Cell Transmission Model
journal, January 2012
- Aziz, H. M. Abdul; Ukkusuri, Satish V.
- Transportation Research Record: Journal of the Transportation Research Board, Vol. 2311, Issue 1
A Distributed Approach for Coordination of Traffic Signal Agents
journal, January 2005
- Bazzan, Ana L. C.
- Autonomous Agents and Multi-Agent Systems, Vol. 10, Issue 1
Learning in groups of traffic signals
journal, June 2010
- Bazzan, Ana L. C.; de Oliveira, Denise; da Silva, Bruno C.
- Engineering Applications of Artificial Intelligence, Vol. 23, Issue 4
System Optimal Signal Optimization Formulation
journal, January 2006
- Beard, Christopher; Ziliaskopoulos, Athanasios
- Transportation Research Record: Journal of the Transportation Research Board, Vol. 1978
The real-time urban traffic control system CRONOS: Algorithm and experiments
journal, February 2006
- Boillot, Florence; Midenet, Sophie; Pierrelée, Jean-Claude
- Transportation Research Part C: Emerging Technologies, Vol. 14, Issue 1
A multivariable regulator approach to traffic-responsive network-wide signal control
journal, February 2002
- Diakaki, Christina; Papageorgiou, Markos; Aboudolas, Kostas
- Control Engineering Practice, Vol. 10, Issue 2
Towards multi-agent reinforcement learning for integrated network of optimal traffic controllers (MARLIN-OTC)
journal, April 2010
- El-Tantawy, Samah; Abdulhai, Baher
- Transportation Letters, Vol. 2, Issue 2
A multiagent system for optimizing urban traffic
conference, January 2003
- France, J.; Ghorbani, A. A.
- 2003 IEEE/WIC International Conference on Intelligent Agent Technology, IEEE/WIC International Conference on Intelligent Agent Technology, 2003. IAT 2003.
Model predictive control for optimal coordination of ramp metering and variable speed limits
journal, June 2005
- Hegyi, Andreas; De Schutter, Bart; Hellendoorn, Hans
- Transportation Research Part C: Emerging Technologies, Vol. 13, Issue 3
Traffic Signal Optimization with Greedy Randomized Tabu Search Algorithm
journal, August 2012
- Hu, Ta-Yin; Chen, Li-Wen
- Journal of Transportation Engineering, Vol. 138, Issue 8
Evaluating the impacts of urban corridor traffic signal optimization on vehicle emissions and fuel consumption
journal, March 2012
- Kwak, Jaeyoung; Park, Byungkyu; Lee, Jaesup
- Transportation Planning and Technology, Vol. 35, Issue 2
An Enhanced 0–1 Mixed-Integer LP Formulation for Traffic Signal Control
journal, December 2004
- Lin, W. -H.; Wang, C.
- IEEE Transactions on Intelligent Transportation Systems, Vol. 5, Issue 4
A Cell-Based Traffic Control Formulation: Strategies and Benefits of Dynamic Timing Plans
journal, May 2001
- Lo, Hong K.
- Transportation Science, Vol. 35, Issue 2
Utopia
book, January 1990
- Mauro, V.; Di Taranto, C.
- Control, Computers, Communications in Transportation
Traffic signal control using reinforcement learning and the max-plus algorithm as a coordinating strategy
conference, September 2012
- Medina, Juan C.; Benekohal, Rahim F.
- 2012 15th International IEEE Conference on Intelligent Transportation Systems - (ITSC 2012)
Genetic reinforcement learning for cooperative traffic signal control
conference, January 1994
- Mikami, S.; Kakazu, Y.
- Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence
A real-time traffic signal control system: architecture, algorithms, and analysis
journal, December 2001
- Mirchandani, Pitu; Head, Larry
- Transportation Research Part C: Emerging Technologies, Vol. 9, Issue 6
Traffic Lights Control with Adaptive Group Formation Based on Swarm Intelligence
book, January 2006
- de Oliveira, Denise; Bazzan, Ana L. C.
- Ant Colony Optimization and Swarm Intelligence
Using cooperative mediation to coordinate traffic lights: a case study
conference, January 2005
- de Oliveira, Denise; Bazzan, Ana L. C.; Lesser, Victor
- Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems - AAMAS '05
A Mathematical Logic Approach for the Transformation of the Linear Conditional Piecewise Functions of Dispersion-and-Store and Cell Transmission Traffic Flow Models into Linear Mixed-Integer Form
journal, February 2009
- Pavlis, Yannis; Recker, Will
- Transportation Science, Vol. 43, Issue 1
Tree consistency and bounds on the performance of the max-product algorithm and its generalizations
journal, April 2004
- Wainwright, Martin; Jaakkola, Tommi; Willsky, Alan
- Statistics and Computing, Vol. 14, Issue 2
Simulation and optimization of traffic in a city
conference, January 2004
- Wiering, M.; Vreeken, J.; van Veenen, J.
- IEEE Intelligent Vehicles Symposium, 2004
A Novel Signal-Scheduling Algorithm With Quality-of-Service Provisioning for an Isolated Intersection
journal, September 2008
- Wunderlich, R.; Elhanany, I.
- IEEE Transactions on Intelligent Transportation Systems, Vol. 9, Issue 3
Works referencing / citing this record:
A Simultaneous Solution for Reserve Capacity Maximization and Delay Minimization Problems in Signalized Road Networks
journal, May 2019
- Baskan, Ozgur; Ceylan, Huseyin; Ozan, Cenk
- Journal of Advanced Transportation, Vol. 2019