The Confluence of Networks, Games, and Learning a Game-Theoretic Framework for Multiagent Decision Making Over Networks

Li, Tao; Peng, Guanze; Zhu, Quanyan; Basar, Tamer

doi:10.1109/mcs.2022.3171478

The Confluence of Networks, Games, and Learning a Game-Theoretic Framework for Multiagent Decision Making Over Networks

Journal Article · 01 August 2022 · IEEE Control Systems Magazine

DOI:https://doi.org/10.1109/mcs.2022.3171478· OSTI ID:1980446

Li, Tao ^[1]; Peng, Guanze ^[1]; Zhu, Quanyan ^[2]; Basar, Tamer ^[3]

New York University, New York, USA
Department of Electrical and Computer Engineering, New York University, New York, USA
University of Illinois Urbana-Champaign, Illinois, USA

Not provided.

Research Organization:: The Ohio State Univ., Columbus, OH (United States)

Sponsoring Organization:: USDOE Office of Nuclear Energy (NE)

DOE Contract Number:: NE0008986

OSTI ID:: 1980446

Journal Information:: IEEE Control Systems Magazine, Vol. 42, Issue 4; ISSN 1066-033X

Publisher:: IEEE

Country of Publication:: United States

Language:: English

References (89)

On the Rate of Convergence of Continuous-Time Fictitious Play Harris, Christopher Games and Economic Behavior, Vol. 22, Issue 2 https://doi.org/10.1006/game.1997.0582	journal	February 1998
Convergent multiple-timescales reinforcement learning algorithms in normal form games Leslie, David S.; Collins, E. J. The Annals of Applied Probability, Vol. 13, Issue 4 https://doi.org/10.1214/aoap/1069786497	journal	November 2003
Uncoupled Dynamics Do Not Lead to Nash Equilibrium Hart, Sergiu; Mas-Colell, Andreu American Economic Review, Vol. 93, Issue 5 https://doi.org/10.1257/000282803322655581	journal	November 2003
Evolutionary stable strategies and game dynamics Taylor, Peter D.; Jonker, Leo B. Mathematical Biosciences, Vol. 40, Issue 1-2 https://doi.org/10.1016/0025-5564(78)90077-9	journal	July 1978
The Complexity of Computing a Nash Equilibrium Daskalakis, Constantinos; Goldberg, Paul W.; Papadimitriou, Christos H. SIAM Journal on Computing, Vol. 39, Issue 1 https://doi.org/10.1137/070699652	journal	January 2009
Stochastic Approximations and Differential Inclusions Benaïm, Michel; Hofbauer, Josef; Sorin, Sylvain SIAM Journal on Control and Optimization, Vol. 44, Issue 1 https://doi.org/10.1137/S0363012904439301	journal	January 2005
Reexamination of the perfectness concept for equilibrium points in extensive games Selten, R. International Journal of Game Theory, Vol. 4, Issue 1 https://doi.org/10.1007/BF01766400	journal	March 1975
Online Learning and Online Convex Optimization Shalev-Shwartz, Shai Foundations and Trends® in Machine Learning, Vol. 4, Issue 2 https://doi.org/10.1561/2200000018	journal	January 2011
Existence and Uniqueness of Equilibrium Points for Concave N-Person Games Rosen, J. B. Econometrica, Vol. 33, Issue 3 https://doi.org/10.2307/1911749	journal	July 1965
Stochastic Nash Equilibrium Problems: Models, Analysis, and Algorithms Lei, Jinlong; Shanbhag, Uday V. IEEE Control Systems, Vol. 42, Issue 4 https://doi.org/10.1109/MCS.2022.3171481	journal	August 2022
Adaptive multi-robot coordination: A game-theoretic perspective Kaminka, Gal A.; Erusalimchik, Dan; Kraus, Sarit 2010 IEEE International Conference on Robotics and Automation https://doi.org/10.1109/ROBOT.2010.5509316	conference	May 2010
Nash Equilibria, Variational Inequalities, and Dynamical Systems Cavazzuti, E.; Pappalardo, M.; Passacantando, M. Journal of Optimization Theory and Applications, Vol. 114, Issue 3 https://doi.org/10.1023/A:1016056327692	journal	September 2002
Distributed Generalized Nash Equilibrium Seeking: An Operator-Theoretic Perspective Belgioioso, Giuseppe; Yi, Peng; Grammatico, Sergio IEEE Control Systems, Vol. 42, Issue 4 https://doi.org/10.1109/MCS.2022.3171480	journal	August 2022
The replicator equation and other game dynamics Cressman, Ross; Tao, Yi Proceedings of the National Academy of Sciences, Vol. 111, Issue supplement_3 https://doi.org/10.1073/pnas.1400823111	journal	July 2014
Stochastic Approximations and Differential Inclusions, Part II: Applications Benaïm, Michel; Hofbauer, Josef; Sorin, Sylvain Mathematics of Operations Research, Vol. 31, Issue 4 https://doi.org/10.1287/moor.1060.0213	journal	November 2006
Riemannian game dynamics Mertikopoulos, Panayotis; Sandholm, William H. Journal of Economic Theory, Vol. 177 https://doi.org/10.1016/j.jet.2018.06.002	journal	September 2018
Time Average Replicator and Best-Reply Dynamics Hofbauer, Josef; Sorin, Sylvain; Viossat, Yannick Mathematics of Operations Research, Vol. 34, Issue 2 https://doi.org/10.1287/moor.1080.0359	journal	May 2009
Individual Q-Learning in Normal Form Games Leslie, David S.; Collins, E. J. SIAM Journal on Control and Optimization, Vol. 44, Issue 2 https://doi.org/10.1137/S0363012903437976	journal	January 2005
Actor-Critic--Type Learning Algorithms for Markov Decision Processes Konda, Vijaymohan R.; Borkar, Vivek S. SIAM Journal on Control and Optimization, Vol. 38, Issue 1 https://doi.org/10.1137/S036301299731669X	journal	January 1999
Generalised weakened fictitious play Leslie, David S.; Collins, E. J. Games and Economic Behavior, Vol. 56, Issue 2 https://doi.org/10.1016/j.geb.2005.08.005	journal	August 2006
Evolutionary game dynamics Hofbauer, Josef; Sigmund, Karl Bulletin of the American Mathematical Society, Vol. 40, Issue 4 https://doi.org/10.1090/S0273-0979-03-00988-1	journal	July 2003
A differential game approach to distributed demand side management in smart grid Zhu, Quanyan; Han, Zhu; Basar, Tamer 2012 IEEE International Conference on Communications (ICC) https://doi.org/10.1109/ICC.2012.6364562	conference	June 2012
Dependable Demand Response Management in the Smart Grid: A Stackelberg Game Approach Maharjan, Sabita; Zhu, Quanyan; Zhang, Yan IEEE Transactions on Smart Grid, Vol. 4, Issue 1 https://doi.org/10.1109/TSG.2012.2223766	journal	March 2013
Interference Aware Routing Game for Cognitive Radio Multi-Hop Networks Zhu, Quanyan; Yuan, Zhou; Song, Ju Bin IEEE Journal on Selected Areas in Communications, Vol. 30, Issue 10 https://doi.org/10.1109/JSAC.2012.121115	journal	November 2012
Game theory meets network security and privacy Manshaei, Mohammad Hossein; Zhu, Quanyan; Alpcan, Tansu ACM Computing Surveys, Vol. 45, Issue 3 https://doi.org/10.1145/2480741.2480742	journal	June 2013
The Logic of Animal Conflict Smith, J. Maynard; Price, G. R. Nature, Vol. 246, Issue 5427 https://doi.org/10.1038/246015a0	journal	November 1973
Heterogeneous learning in zero-sum stochastic games with incomplete information Zhu, Quanyan; Tembine, Hamidou; Basar, Tamer 49th IEEE Conference on Decision and Control (CDC) https://doi.org/10.1109/CDC.2010.5718053	conference	December 2010
Quantal Response Equilibria for Normal Form Games McKelvey, Richard D.; Palfrey, Thomas R. Games and Economic Behavior, Vol. 10, Issue 1 https://doi.org/10.1006/game.1995.1023	journal	July 1995
Introductory Lectures on Convex Optimization Nesterov, Yurii Applied Optimization https://doi.org/10.1007/978-1-4419-8853-9	book	January 2004
Primal-dual subgradient methods for convex problems Nesterov, Yurii Mathematical Programming, Vol. 120, Issue 1 https://doi.org/10.1007/s10107-007-0149-x	journal	June 2007
On the Convergence of Fictitious Play Krishna, Vijay; Sjöström, Tomas Mathematics of Operations Research, Vol. 23, Issue 2 https://doi.org/10.1287/moor.23.2.479	journal	May 1998
Learning in Games via Reinforcement and Regularization Mertikopoulos, Panayotis; Sandholm, William H. Mathematics of Operations Research, Vol. 41, Issue 4 https://doi.org/10.1287/moor.2016.0778	journal	November 2016
Learning in games with continuous action sets and unknown payoff functions Mertikopoulos, Panayotis; Zhou, Zhengyuan Mathematical Programming, Vol. 173, Issue 1-2 https://doi.org/10.1007/s10107-018-1254-8	journal	March 2018
A Large-Scale Markov Game Approach to Dynamic Protection of Interdependent Infrastructure Networks Huang, Linan; Chen, Juntao; Zhu, Quanyan Lecture Notes in Computer Science https://doi.org/10.1007/978-3-319-68711-7_19	book	January 2017
A Dynamic Game Approach to Strategic Design of Secure and Resilient Infrastructure Network Chen, Juntao; Touati, Corinne; Zhu, Quanyan IEEE Transactions on Information Forensics and Security, Vol. 15 https://doi.org/10.1109/TIFS.2019.2924130	journal	January 2020
Demand Response Management in the Smart Grid in a Large Population Regime Maharjan, Sabita; Zhu, Quanyan; Zhang, Yan IEEE Transactions on Smart Grid, Vol. 7, Issue 1 https://doi.org/10.1109/TSG.2015.2431324	journal	January 2016
A Game-Theoretic Framework for Resilient and Distributed Generation Control of Renewable Energies in Microgrids Chen, Juntao; Zhu, Quanyan IEEE Transactions on Smart Grid, Vol. 8, Issue 1 https://doi.org/10.1109/TSG.2016.2598771	journal	January 2017
Best response dynamics for continuous games Barron, E.; Goebel, R.; Jensen, R. Proceedings of the American Mathematical Society, Vol. 138, Issue 3 https://doi.org/10.1090/S0002-9939-09-10170-3	journal	November 2009
On Passivity, Reinforcement Learning, and Higher Order Learning in Multiagent Finite Games Gao, Bolin; Pavel, Lacra IEEE Transactions on Automatic Control, Vol. 66, Issue 1 https://doi.org/10.1109/TAC.2020.2978037	journal	January 2021
Heterogeneous Multi-Layer Adversarial Network Design for the IoT-Enabled Infrastructures Chen, Juntao; Touati, Corinne; Zhu, Quanyan GLOBECOM 2017 - 2017 IEEE Global Communications Conference https://doi.org/10.1109/GLOCOM.2017.8254620	conference	December 2017
Interdependent network formation games with an application to critical infrastructures No authors listed 2016 American Control Conference (ACC) https://doi.org/10.1109/ACC.2016.7525354	conference	July 2016
Perturbations of Set-Valued Dynamical Systems, with Applications to Game Theory Benaïm, Michel; Hofbauer, Josef; Sorin, Sylvain Dynamic Games and Applications, Vol. 2, Issue 2 https://doi.org/10.1007/s13235-012-0040-0	journal	February 2012
A Game-Theoretic Approach to Secure Control of Communication-Based Train Control Systems Under Jamming Attacks Xu, Zhiheng; Zhu, Quanyan Proceedings of the 1st International Workshop on Safe Control of Connected and Autonomous Vehicles https://doi.org/10.1145/3055378.3055381	conference	April 2017
Games with randomly disturbed payoffs: A new rationale for mixed-strategy equilibrium points Harsanyi, John C. International Journal of Game Theory, Vol. 2, Issue 1 https://doi.org/10.1007/BF01737554	journal	December 1973
On Best-Response Dynamics in Potential Games Swenson, Brian; Murray, Ryan; Kar, Soummya SIAM Journal on Control and Optimization, Vol. 56, Issue 4 https://doi.org/10.1137/17M1139461	journal	January 2018
Regular potential games Swenson, Brian; Murray, Ryan; Kar, Soummya Games and Economic Behavior, Vol. 124 https://doi.org/10.1016/j.geb.2020.09.005	journal	November 2020
Dynamic games for secure and resilient control system design Huang, Yunhan; Chen, Juntao; Huang, Linan National Science Review, Vol. 7, Issue 7 https://doi.org/10.1093/nsr/nwz218	journal	January 2020
On Multi-Phase and Multi-Stage Game-Theoretic Modeling of Advanced Persistent Threats Zhu, Quanyan; Rass, Stefan IEEE Access, Vol. 6 https://doi.org/10.1109/ACCESS.2018.2814481	journal	January 2018
Eavesdropping and jamming in next-generation wireless networks: A game-theoretic approach Zhu, Quanyan; Saad, Walid; Han, Zhu 2011 - MILCOM 2011 Military Communications Conference https://doi.org/10.1109/MILCOM.2011.6127463	conference	November 2011
Learning in perturbed asymmetric games Hofbauer, Josef; Hopkins, Ed Games and Economic Behavior, Vol. 52, Issue 1 https://doi.org/10.1016/j.geb.2004.06.006	journal	July 2005
The Evolution of Conventions Young, H. Peyton Econometrica, Vol. 61, Issue 1 https://doi.org/10.2307/2951778	journal	January 1993
Performance of Dynamic Secure Routing Game Song, Ju Bin; Zhu, Quanyan Game Theory for Networking Applications https://doi.org/10.1007/978-3-319-93058-9_4	book	August 2018
Technologies for 5G Networks: Challenges and Opportunities Al-Falahy, Naser; Alani, Omar Y. IT Professional, Vol. 19, Issue 1 https://doi.org/10.1109/MITP.2017.9	journal	January 2017
A Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning Geramifard, Alborz Foundations and Trends® in Machine Learning, Vol. 6, Issue 4 https://doi.org/10.1561/2200000042	journal	January 2013
Human-level control through deep reinforcement learning Mnih, Volodymyr; Kavukcuoglu, Koray; Silver, David Nature, Vol. 518, Issue 7540 https://doi.org/10.1038/nature14236	journal	February 2015
On Convergence Rate of Adaptive Multiscale Value Function Approximation for Reinforcement Learning Li, Tao; Zhu, Quanyan 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP) https://doi.org/10.1109/MLSP.2019.8918816	conference	October 2019
Multilayer Cyber-Physical Security and Resilience for Smart Grid Zhu, Quanyan Smart Grid Control https://doi.org/10.1007/978-3-319-98310-3_14	book	September 2018
A Differential Game Approach to Decentralized Virus-Resistant Weight Adaptation Policy Over Complex Networks Huang, Yunhan; Zhu, Quanyan IEEE Transactions on Control of Network Systems, Vol. 7, Issue 2 https://doi.org/10.1109/TCNS.2019.2931862	journal	June 2020
On Gradient-Based Learning in Continuous Games Mazumdar, Eric; Ratliff, Lillian J.; Sastry, S. Shankar SIAM Journal on Mathematics of Data Science, Vol. 2, Issue 1 https://doi.org/10.1137/18M1231298	journal	January 2020
Best response dynamics for continuous zero--sum games Hofbauer, Josef; Sorin, Sylvain Discrete & Continuous Dynamical Systems - B, Vol. 6, Issue 1 https://doi.org/10.3934/dcdsb.2006.6.215	journal	January 2006
Designing Games for Distributed Optimization Li, Na; Marden, Jason R. IEEE Journal of Selected Topics in Signal Processing, Vol. 7, Issue 2 https://doi.org/10.1109/JSTSP.2013.2246511	journal	April 2013
On the Global Convergence of Stochastic Fictitious Play Hofbauer, Josef; Sandholm, William H. Econometrica, Vol. 70, Issue 6 https://doi.org/10.1111/j.1468-0262.2002.00440.x	journal	November 2002
Best-response dynamics in zero-sum stochastic games Leslie, David S.; Perkins, Steven; Xu, Zibo Journal of Economic Theory, Vol. 189 https://doi.org/10.1016/j.jet.2020.105095	journal	September 2020
The theory of dynamic programming Bellman, Richard Bulletin of the American Mathematical Society, Vol. 60, Issue 6 https://doi.org/10.1090/S0002-9904-1954-09848-8	journal	January 1954
A variational perspective on accelerated methods in optimization Wibisono, Andre; Wilson, Ashia C.; Jordan, Michael I. Proceedings of the National Academy of Sciences, Vol. 113, Issue 47 https://doi.org/10.1073/pnas.1614734113	journal	November 2016
Generative adversarial networks Goodfellow, Ian; Pouget-Abadie, Jean; Mirza, Mehdi Communications of the ACM, Vol. 63, Issue 11 https://doi.org/10.1145/3422622	journal	October 2020
Stationary Markov perfect equilibria in discounted stochastic games He, Wei; Sun, Yeneng Journal of Economic Theory, Vol. 169 https://doi.org/10.1016/j.jet.2017.01.007	journal	May 2017
Multi-robot coordination using switching of methods for deriving equilibrium in game theory Inujima, Wataru; Nakano, Kazushi; Hosokawa, Shu 2013 10th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology https://doi.org/10.1109/ECTICon.2013.6559667	conference	May 2013
Distributed Nash Equilibrium Seeking: Continuous-Time Control-Theoretic Approaches Hu, Guoqiang; Pang, Yipeng; Sun, Chao IEEE Control Systems, Vol. 42, Issue 4 https://doi.org/10.1109/MCS.2022.3171479	journal	August 2022
Nash Equilibrium Seeking in Noncooperative Games Frihauf, P.; Krstic, M.; Basar, T. IEEE Transactions on Automatic Control, Vol. 57, Issue 5 https://doi.org/10.1109/TAC.2011.2173412	journal	May 2012
A Passivity-Based Approach to Nash Equilibrium Seeking Over Networks Gadjov, Dian; Pavel, Lacra IEEE Transactions on Automatic Control, Vol. 64, Issue 3 https://doi.org/10.1109/TAC.2018.2833140	journal	March 2019
Relaxation techniques and asynchronous algorithms for on-line computation of non-cooperative equilibria Başar, Tamer Journal of Economic Dynamics and Control, Vol. 11, Issue 4 https://doi.org/10.1016/S0165-1889(87)80006-4	journal	December 1987
Cyber-Security in Critical Infrastructures Rass, Stefan; Schauer, Stefan; König, Sandra Advanced Sciences and Technologies for Security Applications https://doi.org/10.1007/978-3-030-46908-5	book	June 2020
Why Both Game Theory and Reliability Theory Are Important in Defending Infrastructure against Intelligent Attacks Bier, Vicki M.; Cox, Louis A.; Azaiez, M. Naceur International Series in Operations Research & Management Science https://doi.org/10.1007/978-0-387-87767-9_1	book	October 2008
Factored Markov Game Theory for Secure Interdependent Infrastructure Networks Huang, Linan; Chen, Juntao; Zhu, Quanyan Game Theory for Security and Risk Management https://doi.org/10.1007/978-3-319-75268-6_5	book	January 2018
A dynamic games approach to proactive defense strategies against Advanced Persistent Threats in cyber-physical systems Huang, Linan; Zhu, Quanyan Computers & Security, Vol. 89 https://doi.org/10.1016/j.cose.2019.101660	journal	February 2020
Learning Stationary Correlated Equilibria in Constrained General-Sum Stochastic Games Hakami, Vesal; Dehghan, Mehdi IEEE Transactions on Cybernetics, Vol. 46, Issue 7 https://doi.org/10.1109/TCYB.2015.2453165	journal	July 2016
Toward System-Optimal Routing in Traffic Networks: A Reverse Stackelberg Game Approach Groot, Noortje; De Schutter, Bart; Hellendoorn, Hans IEEE Transactions on Intelligent Transportation Systems, Vol. 16, Issue 1 https://doi.org/10.1109/TITS.2014.2322312	journal	February 2015
Payoff-Based Dynamics for Multiplayer Weakly Acyclic Games Marden, Jason R.; Young, H. Peyton; Arslan, Gürdal SIAM Journal on Control and Optimization, Vol. 48, Issue 1 https://doi.org/10.1137/070680199	journal	January 2009
Performance Analysis of Trial and Error Algorithms Gaveau, Jerome; Le Martret, Christophe J.; Assaad, Mohamad IEEE Transactions on Parallel and Distributed Systems, Vol. 31, Issue 6 https://doi.org/10.1109/TPDS.2020.2964256	journal	June 2020
Selecting efficient correlated equilibria through distributed learning Marden, Jason R. Games and Economic Behavior, Vol. 106 https://doi.org/10.1016/j.geb.2017.09.007	journal	November 2017
Learning efficient Nash equilibria in distributed systems Pradelski, Bary S. R.; Young, H. Peyton Games and Economic Behavior, Vol. 75, Issue 2 https://doi.org/10.1016/j.geb.2012.02.017	journal	July 2012
Learning by trial and error Young, H. Peyton Games and Economic Behavior, Vol. 65, Issue 2 https://doi.org/10.1016/j.geb.2008.02.011	journal	March 2009
A game-theoretical approach to incentive design in collaborative intrusion detection networks Zhu, Quanyan; Fung, Carol; Boutaba, Raouf 2009 International Conference on Game Theory for Networks https://doi.org/10.1109/GAMENETS.2009.5137424	conference	May 2009
On the Secure and Reconfigurable Multi-Layer Network Design for Critical Information Dissemination in the Internet of Battlefield Things (IoBT) Farooq, Muhammad Junaid; Zhu, Quanyan IEEE Transactions on Wireless Communications, Vol. 17, Issue 4 https://doi.org/10.1109/TWC.2018.2799860	journal	April 2018
Modeling, Analysis, and Mitigation of Dynamic Botnet Formation in Wireless IoT Networks Farooq, Muhammad Junaid; Zhu, Quanyan IEEE Transactions on Information Forensics and Security, Vol. 14, Issue 9 https://doi.org/10.1109/TIFS.2019.2898817	journal	September 2019
Achieving Pareto Optimality Through Distributed Learning Marden, Jason R.; Young, H. Peyton; Pao, Lucy Y. SIAM Journal on Control and Optimization, Vol. 52, Issue 5 https://doi.org/10.1137/110850694	journal	January 2014
On convergence rates of game theoretic reinforcement learning algorithms Hu, Zhisheng; Zhu, Minghui; Chen, Ping Automatica, Vol. 104 https://doi.org/10.1016/j.automatica.2019.02.032	journal	June 2019
A survey of game theoretic approach for adversarial machine learning Zhou, Yan; Kantarcioglu, Murat; Xi, Bowei WIREs Data Mining and Knowledge Discovery, Vol. 9, Issue 3 https://doi.org/10.1002/widm.1259	journal	April 2018

Similar Records

Game-Theoretic Frameworks for Epidemic Spreading and Human Decision-Making: A Review

Journal Article · 2022 · Dynamic Games and Applications · OSTI ID:1976758

Huang, Yunhan; Zhu, Quanyan

The role of information structures in game-theoretic multi-agent learning

Journal Article · 2022 · Annual Reviews in Control · OSTI ID:1976878

Li, Tao; Zhao, Yuhan; Zhu, Quanyan

A game theory perspective on environmental assessment: What games are played and what does this tell us about decision making rationality and legitimacy?

Journal Article · 2016 · Environmental Impact Assessment Review · OSTI ID:22589243

Bond, Alan; Pope, Jenny; Morrison-Saunders, Angus; +1 more

Related Subjects

Automation & Control Systems

The Confluence of Networks, Games, and Learning a Game-Theoretic Framework for Multiagent Decision Making Over Networks

Citation Formats

References (89)

Similar Records

Related Subjects