A junction-tree based learning algorithm to optimize network wide traffic control: A coordinated multi-agent framework

Zhu, Feng; Aziz, H. M.  Abdul; Qian, Xinwu; Ukkusuri, Satish V.

doi:10.1016/j.trc.2014.12.009

Title: A junction-tree based learning algorithm to optimize network wide traffic control: A coordinated multi-agent framework

Journal Article · Sat Jan 31 00:00:00 EST 2015 · Transportation Research Part C: Emerging Technologies

DOI:https://doi.org/10.1016/j.trc.2014.12.009· OSTI ID:1265896

Zhu, Feng ^[1]; Aziz, H. M. Abdul ^[1]; Qian, Xinwu ^[1]; Ukkusuri, Satish V. ^[1]

Purdue Univ., West Lafayette, IN (United States)

Our study develops a novel reinforcement learning algorithm for the challenging coordinated signal control problem. Traffic signals are modeled as intelligent agents interacting with the stochastic traffic environment. The model is built on the framework of coordinated reinforcement learning. The Junction Tree Algorithm (JTA) based reinforcement learning is proposed to obtain an exact inference of the best joint actions for all the coordinated intersections. Moreover, the algorithm is implemented and tested with a network containing 18 signalized intersections in VISSIM. Finally, our results show that the JTA based algorithm outperforms independent learning (Q-learning), real-time adaptive learning, and fixed timing plans in terms of average delay, number of stops, and vehicular emissions at the network level.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE, National Science Foundation (NSF)

Grant/Contract Number:: AC05-00OR22725; 1004528; 104IPY04

OSTI ID:: 1265896

Journal Information:: Transportation Research Part C: Emerging Technologies, Vol. 58, Issue PC; ISSN 0968-090X

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 41 works

Citation information provided by
Web of Science

References (26)

Reinforcement Learning for True Adaptive Traffic Signal Control Abdulhai, Baher; Pringle, Rob; Karakoulas, Grigoris J. Journal of Transportation Engineering, Vol. 129, Issue 3 https://doi.org/10.1061/(ASCE)0733-947X(2003)129:3(278)	journal	May 2003
Store-and-forward based methods for the signal control problem in large-scale congested urban road networks Aboudolas, K.; Papageorgiou, M.; Kosmatopoulos, E. Transportation Research Part C: Emerging Technologies, Vol. 17, Issue 2 https://doi.org/10.1016/j.trc.2008.10.002	journal	April 2009
Unified Framework for Dynamic Traffic Assignment and Signal Control with Cell Transmission Model Aziz, H. M. Abdul; Ukkusuri, Satish V. Transportation Research Record: Journal of the Transportation Research Board, Vol. 2311, Issue 1 https://doi.org/10.3141/2311-07	journal	January 2012
A Distributed Approach for Coordination of Traffic Signal Agents Bazzan, Ana L. C. Autonomous Agents and Multi-Agent Systems, Vol. 10, Issue 1 https://doi.org/10.1007/s10458-004-6975-9	journal	January 2005
Learning in groups of traffic signals Bazzan, Ana L. C.; de Oliveira, Denise; da Silva, Bruno C. Engineering Applications of Artificial Intelligence, Vol. 23, Issue 4 https://doi.org/10.1016/j.engappai.2009.11.009	journal	June 2010
System Optimal Signal Optimization Formulation Beard, Christopher; Ziliaskopoulos, Athanasios Transportation Research Record: Journal of the Transportation Research Board, Vol. 1978 https://doi.org/10.3141/1978-15	journal	January 2006
The real-time urban traffic control system CRONOS: Algorithm and experiments Boillot, Florence; Midenet, Sophie; Pierrelée, Jean-Claude Transportation Research Part C: Emerging Technologies, Vol. 14, Issue 1 https://doi.org/10.1016/j.trc.2006.05.001	journal	February 2006
A multivariable regulator approach to traffic-responsive network-wide signal control Diakaki, Christina; Papageorgiou, Markos; Aboudolas, Kostas Control Engineering Practice, Vol. 10, Issue 2 https://doi.org/10.1016/S0967-0661(01)00121-6	journal	February 2002
Towards multi-agent reinforcement learning for integrated network of optimal traffic controllers (MARLIN-OTC) El-Tantawy, Samah; Abdulhai, Baher Transportation Letters, Vol. 2, Issue 2 https://doi.org/10.3328/TL.2010.02.02.89-110	journal	April 2010
A multiagent system for optimizing urban traffic France, J.; Ghorbani, A. A. 2003 IEEE/WIC International Conference on Intelligent Agent Technology, IEEE/WIC International Conference on Intelligent Agent Technology, 2003. IAT 2003. https://doi.org/10.1109/IAT.2003.1241110	conference	January 2003
Model predictive control for optimal coordination of ramp metering and variable speed limits Hegyi, Andreas; De Schutter, Bart; Hellendoorn, Hans Transportation Research Part C: Emerging Technologies, Vol. 13, Issue 3 https://doi.org/10.1016/j.trc.2004.08.001	journal	June 2005
Traffic Signal Optimization with Greedy Randomized Tabu Search Algorithm Hu, Ta-Yin; Chen, Li-Wen Journal of Transportation Engineering, Vol. 138, Issue 8 https://doi.org/10.1061/(ASCE)TE.1943-5436.0000404	journal	August 2012
Graphical Models Jordan, Michael I. Statistical Science, Vol. 19, Issue 1 https://doi.org/10.1214/088342304000000026	journal	February 2004
Evaluating the impacts of urban corridor traffic signal optimization on vehicle emissions and fuel consumption Kwak, Jaeyoung; Park, Byungkyu; Lee, Jaesup Transportation Planning and Technology, Vol. 35, Issue 2 https://doi.org/10.1080/03081060.2011.651877	journal	March 2012
An Enhanced 0–1 Mixed-Integer LP Formulation for Traffic Signal Control Lin, W. -H.; Wang, C. IEEE Transactions on Intelligent Transportation Systems, Vol. 5, Issue 4 https://doi.org/10.1109/TITS.2004.838217	journal	December 2004
A Cell-Based Traffic Control Formulation: Strategies and Benefits of Dynamic Timing Plans Lo, Hong K. Transportation Science, Vol. 35, Issue 2 https://doi.org/10.1287/trsc.35.2.148.10136	journal	May 2001
Utopia Mauro, V.; Di Taranto, C. Control, Computers, Communications in Transportation https://doi.org/10.1016/B978-0-08-037025-5.50042-6	book	January 1990
Traffic signal control using reinforcement learning and the max-plus algorithm as a coordinating strategy Medina, Juan C.; Benekohal, Rahim F. 2012 15th International IEEE Conference on Intelligent Transportation Systems - (ITSC 2012) https://doi.org/10.1109/ITSC.2012.6338911	conference	September 2012
Genetic reinforcement learning for cooperative traffic signal control Mikami, S.; Kakazu, Y. Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence https://doi.org/10.1109/ICEC.1994.350012	conference	January 1994
A real-time traffic signal control system: architecture, algorithms, and analysis Mirchandani, Pitu; Head, Larry Transportation Research Part C: Emerging Technologies, Vol. 9, Issue 6 https://doi.org/10.1016/S0968-090X(00)00047-4	journal	December 2001
Traffic Lights Control with Adaptive Group Formation Based on Swarm Intelligence de Oliveira, Denise; Bazzan, Ana L. C. Ant Colony Optimization and Swarm Intelligence https://doi.org/10.1007/11839088_61	book	January 2006
Using cooperative mediation to coordinate traffic lights: a case study de Oliveira, Denise; Bazzan, Ana L. C.; Lesser, Victor Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems - AAMAS '05 https://doi.org/10.1145/1082473.1082544	conference	January 2005
A Mathematical Logic Approach for the Transformation of the Linear Conditional Piecewise Functions of Dispersion-and-Store and Cell Transmission Traffic Flow Models into Linear Mixed-Integer Form Pavlis, Yannis; Recker, Will Transportation Science, Vol. 43, Issue 1 https://doi.org/10.1287/trsc.1080.0254	journal	February 2009
Tree consistency and bounds on the performance of the max-product algorithm and its generalizations Wainwright, Martin; Jaakkola, Tommi; Willsky, Alan Statistics and Computing, Vol. 14, Issue 2 https://doi.org/10.1023/B:STCO.0000021412.33763.d5	journal	April 2004
Simulation and optimization of traffic in a city Wiering, M.; Vreeken, J.; van Veenen, J. IEEE Intelligent Vehicles Symposium, 2004 https://doi.org/10.1109/IVS.2004.1336426	conference	January 2004
A Novel Signal-Scheduling Algorithm With Quality-of-Service Provisioning for an Isolated Intersection Wunderlich, R.; Elhanany, I. IEEE Transactions on Intelligent Transportation Systems, Vol. 9, Issue 3 https://doi.org/10.1109/TITS.2008.928266	journal	September 2008

Cited By (1)

A Simultaneous Solution for Reserve Capacity Maximization and Delay Minimization Problems in Signalized Road Networks Baskan, Ozgur; Ceylan, Huseyin; Ozan, Cenk Journal of Advanced Transportation, Vol. 2019 https://doi.org/10.1155/2019/6203137	journal	May 2019

Similar Records

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Journal Article · Sun Apr 07 00:00:00 EDT 2024 · Symmetry · OSTI ID:1265896

Kumarasamy, Vijayalakshmi K.; Saroj, Abhilasha Jairam; Liang, Yu; +4 more

Traffic Signal Optimization by Integrating Reinforcement Learning and Digital Twins

Conference · Tue Aug 01 00:00:00 EDT 2023 · OSTI ID:1265896

Kumarasamy, Vijayalakshmi K.; Saroj, Abhilasha; Liang, Yu; +4 more

Investigating the Impact of Connected Vehicle Market Share on the Performance of Reinforcement-Learning Based Traffic Signal Control

Technical Report · Sun Jun 30 00:00:00 EDT 2019 · OSTI ID:1265896

Aziz, H M Abdul; Wang, Hong; Young, Stanley; +1 more

Related Subjects

97 MATHEMATICS AND COMPUTING
Traffic Control
Learning Algorithm
Multi-agent control
Connected Vehicle

Title: A junction-tree based learning algorithm to optimize network wide traffic control: A coordinated multi-agent framework

Citation Formats

References (26)

Cited By (1)

Similar Records

Related Subjects