Task-oriented machine learning surrogates for tipping points of agent-based models

Fabiani, Gianluca; Evangelou, Nikolaos; Cui, Tianqi; Bello-Rivas, Juan M.; Martin-Linares, Cristina P.; Siettos, Constantinos; Kevrekidis, Ioannis G.

doi:10.1038/s41467-024-48024-7

Title: Task-oriented machine learning surrogates for tipping points of agent-based models

Journal Article · 14 May 2024 · Nature Communications

DOI: https://doi.org/10.1038/s41467-024-48024-7 · OSTI ID:2470388

^[1]; Evangelou, Nikolaos ^[2]; Cui, Tianqi ^[2]; Bello-Rivas, Juan M. ^[2]; Martin-Linares, Cristina P. ^[2];

^[3];

^[2]

Scuola Superiore Meridionale, Naples (Italy); Johns Hopkins University, Baltimore, MD (United States)
Johns Hopkins University, Baltimore, MD (United States)
Università degli Studi di Napoli Federico II, Naples (Italy)

We present a machine learning framework bridging manifold learning, neural networks, Gaussian processes, and Equation-Free multiscale approach, for the construction of different types of effective reduced order models from detailed agent-based simulators and the systematic multiscale numerical analysis of their emergent dynamics. The specific tasks of interest here include the detection of tipping points, and the uncertainty quantification of rare events near them. Our illustrative examples are an event-driven, stochastic financial market model describing the mimetic behavior of traders, and a compartmental stochastic epidemic model on an Erdös-Rényi network. We contrast the pros and cons of the different types of surrogate models and the effort involved in learning them. Importantly, the proposed framework reveals that, around the tipping points, the emergent dynamics of both benchmark examples can be effectively described by a one-dimensional stochastic differential equation, thus revealing the intrinsic dimensionality of the normal form of the specific type of the tipping point. This allows a significant reduction in the computational cost of the tasks of interest.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Johns Hopkins University, Baltimore, MD (United States)

Sponsoring Organization:: US Air Force Office of Scientific Research (AFOSR); USDOE; la Caixa Foundation Fellowship

OSTI ID:: 2470388

Journal Information:: Nature Communications, Journal Name: Nature Communications Journal Issue: 1 Vol. 15; ISSN 2041-1723

Publisher:: Nature Publishing GroupCopyright Statement

Country of Publication:: United States

Language:: English

References (66)

Nonlinear principal component analysis using autoassociative neural networks Kramer, Mark A. AIChE Journal, Vol. 37, Issue 2 https://doi.org/10.1002/aic.690370209	journal	February 1991
Simulations of thermodynamics and kinetics on rough energy landscapes with milestoning Bello‐Rivas, Juan M.; Elber, Ron Journal of Computational Chemistry, Vol. 37, Issue 6 https://doi.org/10.1002/jcc.24039	journal	August 2015
Molecular enhanced sampling with autoencoders: On-the-fly collective variable discovery and accelerated free energy landscape exploration: Molecular Enhanced Sampling with Autoencoders: On-The-Fly Collective Variable Discovery and Accelerated Free Energy Landscape Exploration Chen, Wei; Ferguson, Andrew L. Journal of Computational Chemistry, Vol. 39, Issue 25 https://doi.org/10.1002/jcc.25520	journal	September 2018
Approximation by superpositions of a sigmoidal function Cybenko, G. Mathematics of Control, Signals, and Systems, Vol. 2, Issue 4 https://doi.org/10.1007/BF02551274	journal	December 1989
Learning black- and gray-box chemotactic PDEs/closures from agent based Monte Carlo simulation data Lee, Seungjoon; Psarellis, Yorgos M.; Siettos, Constantinos I. Journal of Mathematical Biology, Vol. 87, Issue 1 https://doi.org/10.1007/s00285-023-01946-0	journal	June 2023
Enabling Equation-Free Modeling via Diffusion Maps Chin, Tracy; Ruth, Jacob; Sanford, Clayton Journal of Dynamics and Differential Equations, Vol. 36, Issue S1 https://doi.org/10.1007/s10884-021-10127-w	journal	January 2022
Numerical Bifurcation Analysis of PDEs From Lattice Boltzmann Model Simulations: a Parsimonious Machine Learning Approach Galaris, Evangelos; Fabiani, Gianluca; Gallos, Ioannis Journal of Scientific Computing, Vol. 92, Issue 2 https://doi.org/10.1007/s10915-022-01883-y	journal	June 2022
Nurturing breakthroughs: lessons from complexity theory Sornette, D. Journal of Economic Interaction and Coordination, Vol. 3, Issue 2 https://doi.org/10.1007/s11403-008-0040-8	journal	April 2008
Linking Machine Learning with Multiscale Numerics: Data-Driven Discovery of Homogenized Equations Arbabi, Hassan; Bunder, Judith E.; Samaey, Giovanni JOM, Vol. 72, Issue 12 https://doi.org/10.1007/s11837-020-04399-8	journal	October 2020
Manners makyth modellers Aris, Rutherford Chemical Engineering Science, Vol. 46, Issue 7 https://doi.org/10.1016/0009-2509(91)87003-U	journal	January 1991
Multilayer feedforward networks are universal approximators Hornik, Kurt; Stinchcombe, Maxwell; White, Halbert Neural Networks, Vol. 2, Issue 5 https://doi.org/10.1016/0893-6080(89)90020-8	journal	January 1989
Modelling of nonlinear process dynamics using Kohonen's neural networks, fuzzy systems and Chebyshev series Alexandridis, A. P.; Siettos, C. I.; Sarimveis, H. K. Computers & Chemical Engineering, Vol. 26, Issue 4-5 https://doi.org/10.1016/S0098-1354(01)00785-2	journal	May 2002
Time series properties of an artificial stock market LeBaron, Blake; Arthur, W. Brian; Palmer, Richard Journal of Economic Dynamics and Control, Vol. 23, Issue 9-10 https://doi.org/10.1016/S0165-1889(98)00081-5	journal	September 1999
Diffusion maps, spectral clustering and reaction coordinates of dynamical systems Nadler, Boaz; Lafon, Stéphane; Coifman, Ronald R. Applied and Computational Harmonic Analysis, Vol. 21, Issue 1, p. 113-127 https://doi.org/10.1016/j.acha.2005.07.004	journal	July 2006
Parsimonious representation of nonlinear dynamical systems through manifold learning: A chemotaxis case study Dsilva, Carmeline J.; Talmon, Ronen; Coifman, Ronald R. Applied and Computational Harmonic Analysis, Vol. 44, Issue 3 https://doi.org/10.1016/j.acha.2015.06.008	journal	May 2018
The role of agent-based models in wildlife ecology and management McLane, Adam J.; Semeniuk, Christina; McDermid, Gregory J. Ecological Modelling, Vol. 222, Issue 8 https://doi.org/10.1016/j.ecolmodel.2011.01.020	journal	April 2011
Modeling human decisions in coupled human and natural systems: Review of agent-based models An, Li Ecological Modelling, Vol. 229 https://doi.org/10.1016/j.ecolmodel.2011.07.010	journal	March 2012
Machine learning of linear differential equations using Gaussian processes Raissi, Maziar; Perdikaris, Paris; Karniadakis, George Em Journal of Computational Physics, Vol. 348 https://doi.org/10.1016/j.jcp.2017.07.050	journal	November 2017
Solving and learning nonlinear PDEs with Gaussian processes Chen, Yifan; Hosseini, Bamdad; Owhadi, Houman Journal of Computational Physics, Vol. 447 https://doi.org/10.1016/j.jcp.2021.110668	journal	December 2021
Data-driven control of agent-based models: An Equation/Variable-free machine learning approach Patsatzis, Dimitrios G.; Russo, Lucia; Kevrekidis, Ioannis G. Journal of Computational Physics, Vol. 478 https://doi.org/10.1016/j.jcp.2023.111953	journal	April 2023
Double Diffusion Maps and their Latent Harmonics for scientific computations in latent space Evangelou, Nikolaos; Dietrich, Felix; Chiavazzo, Eliodoro Journal of Computational Physics, Vol. 485 https://doi.org/10.1016/j.jcp.2023.112072	journal	July 2023
Modeling a large population of traders: Mimesis and stability Omurtag, Ahmet; Sirovich, Lawrence Journal of Economic Behavior & Organization, Vol. 61, Issue 4 https://doi.org/10.1016/j.jebo.2004.07.016	journal	December 2006
Extreme learning machine: Theory and applications Huang, Guang-Bin; Zhu, Qin-Yu; Siew, Chee-Kheong Neurocomputing, Vol. 70, Issue 1-3 https://doi.org/10.1016/j.neucom.2005.12.126	journal	December 2006
Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics Vlachas, P. R.; Pathak, J.; Hunt, B. R. Neural Networks, Vol. 126 https://doi.org/10.1016/j.neunet.2020.02.016	journal	June 2020
Catastrophic regime shifts in ecosystems: linking theory to observation Scheffer, Marten; Carpenter, Stephen R. Trends in Ecology & Evolution, Vol. 18, Issue 12 https://doi.org/10.1016/j.tree.2003.09.002	journal	December 2003
Economics: Meltdown modelling Buchanan, Mark Nature, Vol. 460, Issue 7256 https://doi.org/10.1038/460680a	journal	August 2009
The economy needs agent-based modelling Farmer, J. Doyne; Foley, Duncan Nature, Vol. 460, Issue 7256 https://doi.org/10.1038/460685a	journal	August 2009
Foreseeing tipping points Scheffer, Marten Nature, Vol. 467, Issue 7314 https://doi.org/10.1038/467411a	journal	September 2010
Early-warning signals for critical transitions Scheffer, Marten; Bascompte, Jordi; Brock, William A. Nature, Vol. 461, Issue 7260 https://doi.org/10.1038/nature08227	journal	September 2009
Next generation reservoir computing Gauthier, Daniel J.; Bollt, Erik; Griffith, Aaron Nature Communications, Vol. 12, Issue 1 https://doi.org/10.1038/s41467-021-25801-2	journal	September 2021
Agent-based modelling of reactive vaccination of workplaces and schools against COVID-19 Faucher, Benjamin; Assab, Rania; Roux, Jonathan Nature Communications, Vol. 13, Issue 1 https://doi.org/10.1038/s41467-022-29015-y	journal	March 2022
Ecosystem tipping points in an evolving world Dakos, Vasilis; Matthews, Blake; Hendry, Andrew P. Nature Ecology & Evolution, Vol. 3, Issue 3 https://doi.org/10.1038/s41559-019-0797-2	journal	February 2019
Physics-informed machine learning Karniadakis, George Em; Kevrekidis, Ioannis G.; Lu, Lu Nature Reviews Physics, Vol. 3, Issue 6 https://doi.org/10.1038/s42254-021-00314-5	journal	May 2021
Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators Lu, Lu; Jin, Pengzhan; Pang, Guofei Nature Machine Intelligence, Vol. 3, Issue 3 https://doi.org/10.1038/s42256-021-00302-5	journal	March 2021
Multiscale simulations of complex systems by learning their effective dynamics Vlachas, Pantelis R.; Arampatzis, Georgios; Uhler, Caroline Nature Machine Intelligence, Vol. 4, Issue 4 https://doi.org/10.1038/s42256-022-00464-w	journal	April 2022
Data-driven discovery of intrinsic dynamics Floryan, Daniel; Graham, Michael D. Nature Machine Intelligence, Vol. 4, Issue 12 https://doi.org/10.1038/s42256-022-00575-4	journal	December 2022
Equation-free Model Reduction in Agent-based Computations: Coarse-grained Bifurcation and Variable-free Rare Event Analysis Liu, Ping; Siettos, C. I.; Gear, C. W. Mathematical Modelling of Natural Phenomena, Vol. 10, Issue 3 https://doi.org/10.1051/mmnp/201510307	journal	January 2015
Exploration of effective potential landscapes using coarse reverse integration Frewen, Thomas A.; Hummer, Gerhard; Kevrekidis, Ioannis G. The Journal of Chemical Physics, Vol. 131, Issue 13 https://doi.org/10.1063/1.3207882	journal	October 2009
Coarse-scale PDEs from fine-scale observations via machine learning Lee, Seungjoon; Kooshkbaghi, Mahdi; Spiliotis, Konstantinos Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 30, Issue 1 https://doi.org/10.1063/1.5126869	journal	January 2020
On learning Hamiltonian systems from data Bertalan, Tom; Dietrich, Felix; Mezić, Igor Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 29, Issue 12 https://doi.org/10.1063/1.5128231	journal	December 2019
An end-to-end deep learning approach for extracting stochastic dynamical systems withα-stable Lévy noise Fang, Cheng; Lu, Yubin; Gao, Ting Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 32, Issue 6 https://doi.org/10.1063/5.0089832	journal	June 2022
Time-series forecasting using manifold learning, radial basis function interpolation, and geometric harmonics Papaioannou, Panagiotis G.; Talmon, Ronen; Kevrekidis, Ioannis G. Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 32, Issue 8 https://doi.org/10.1063/5.0094887	journal	August 2022
Learning effective stochastic differential equations from microscopic simulations: Linking stochastic numerics to deep learning Dietrich, Felix; Makeev, Alexei; Kevrekidis, George Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 33, Issue 2 https://doi.org/10.1063/5.0113632	journal	February 2023
Parsimonious physics-informed random projection neural networks for initial value problems of ODEs and index-1 DAEs Fabiani, Gianluca; Galaris, Evangelos; Russo, Lucia Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 33, Issue 4 https://doi.org/10.1063/5.0135903	journal	April 2023
Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps Coifman, R. R.; Lafon, S.; Lee, A. B. Proceedings of the National Academy of Sciences, Vol. 102, Issue 21 https://doi.org/10.1073/pnas.0500334102	journal	May 2005
Detecting intrinsic slow variables in stochastic dynamical systems by anisotropic diffusion maps Singer, A.; Erban, R.; Kevrekidis, I. G. Proceedings of the National Academy of Sciences, Vol. 106, Issue 38 https://doi.org/10.1073/pnas.0905547106	journal	August 2009
Discovering governing equations from data by sparse identification of nonlinear dynamical systems Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan Proceedings of the National Academy of Sciences, Vol. 113, Issue 15 https://doi.org/10.1073/pnas.1517384113	journal	March 2016
DISCRETE- vs. CONTINUOUS-TIME NONLINEAR SIGNAL PROCESSING OF Cu ELECTRODISSOLUTION DATA Rico-MartÍNez, R.; Krischer, K.; Kevrekidis, I. G. Chemical Engineering Communications, Vol. 118, Issue 1 https://doi.org/10.1080/00986449208936084	journal	November 1992
On the efficiency of the equation-free closure of statistical moments: dynamical properties of a stochastic epidemic model on Erdős–Rényi networks Reppas, A. I.; De Decker, Y.; Siettos, C. I. Journal of Statistical Mechanics: Theory and Experiment, Vol. 2012, Issue 08 https://doi.org/10.1088/1742-5468/2012/08/P08020	journal	August 2012
Dimension reduction of noisy interacting systems Zagli, Niccolò; Pavliotis, Grigorios A.; Lucarini, Valerio Physical Review Research, Vol. 5, Issue 1 https://doi.org/10.1103/PhysRevResearch.5.013078	journal	February 2023
Functional-link net computing: theory, system architecture, and functionalities Pao, Y. -H.; Takefuji, Y. Computer, Vol. 25, Issue 5 https://doi.org/10.1109/2.144401	journal	May 1992
Training feedforward networks with the Marquardt algorithm Hagan, M. T.; Menhaj, M. B. IEEE Transactions on Neural Networks, Vol. 5, Issue 6 https://doi.org/10.1109/72.329697	journal	January 1994
Pattern-Oriented Modeling of Agent-Based Complex Systems: Lessons from Ecology Grimm, V. Science, Vol. 310, Issue 5750 https://doi.org/10.1126/science.1116681	journal	November 2005
Nonlinear Dimensionality Reduction by Locally Linear Embedding Roweis, S. T. Science, Vol. 290, Issue 5500 https://doi.org/10.1126/science.290.5500.2323	journal	December 2000
The Isomap Algorithm and Topological Stability Balasubramanian, M. Science, Vol. 295, Issue 5552 https://doi.org/10.1126/science.295.5552.7a	journal	January 2002
Exceeding 1.5°C global warming could trigger multiple climate tipping points Armstrong McKay, David I.; Staal, Arie; Abrams, Jesse F. Science, Vol. 377, Issue 6611 https://doi.org/10.1126/science.abn7950	journal	September 2022
Diffusion Maps, Reduction Coordinates, and Low Dimensional Representation of Stochastic Systems Coifman, R. R.; Kevrekidis, I. G.; Lafon, S. Multiscale Modeling & Simulation, Vol. 7, Issue 2 https://doi.org/10.1137/070696325	journal	January 2008
Attractor Modeling and Empirical Nonlinear Model Reduction of Dissipative Dynamical Systems Bollt, Erik International Journal of Bifurcation and Chaos, Vol. 17, Issue 04 https://doi.org/10.1142/S021812740701777X	journal	April 2007
FRED (A Framework for Reconstructing Epidemic Dynamics): an open-source software system for modeling infectious diseases and control strategies using census-based populations Grefenstette, John J.; Brown, Shawn T.; Rosenfeld, Roni BMC Public Health, Vol. 13, Issue 1 https://doi.org/10.1186/1471-2458-13-940	journal	October 2013
An equation-free approach to agent-based computation: Bifurcation analysis and control of stationary states Siettos, C. I.; Gear, C. W.; Kevrekidis, I. G. EPL (Europhysics Letters), Vol. 99, Issue 4 https://doi.org/10.1209/0295-5075/99/48007	journal	August 2012
Modeling the 2014 Ebola Virus Epidemic 2013 Agent-Based Simulations, Temporal Analysis and Future Predictions for Liberia and Sierra Leone Siettos, Constantinos; Anastassopoulou, Cleo; Russo, Lucia PLoS Currents https://doi.org/10.1371/currents.outbreaks.8d5984114855fc425e699e1a18cdc6c9	journal	January 2015
Little Italy: An Agent-Based Approach to the Estimation of Contact Patterns- Fitting Predicted Matrices to Serological Data Iozzi, Fabrizio; Trusiano, Francesco; Chinazzi, Matteo PLoS Computational Biology, Vol. 6, Issue 12 https://doi.org/10.1371/journal.pcbi.1001021	journal	December 2010
Covasim: An agent-based model of COVID-19 dynamics and interventions Kerr, Cliff C.; Stuart, Robyn M.; Mistry, Dina PLOS Computational Biology, Vol. 17, Issue 7 https://doi.org/10.1371/journal.pcbi.1009149	journal	July 2021
Policies to Reduce Influenza in the Workplace: Impact Assessments Using an Agent-Based Model Kumar, Supriya; Grefenstette, John J.; Galloway, David American Journal of Public Health, Vol. 103, Issue 8 https://doi.org/10.2105/AJPH.2013.301269	journal	August 2013
From interacting agents to density-based modeling with stochastic PDEs Helfmann, Luzie; Djurdjevac Conrad, Nataša; Djurdjevac, Ana Communications in Applied Mathematics and Computational Science, Vol. 16, Issue 1 https://doi.org/10.2140/camcos.2021.16.1	journal	January 2021
Equation-Free, Coarse-Grained Multiscale Computation: Enabling Mocroscopic Simulators to Perform System-Level Analysis Gear, C. William; Hyman, James M.; Kevrekidid, Panagiotis G. Communications in Mathematical Sciences, Vol. 1, Issue 4 https://doi.org/10.4310/CMS.2003.v1.n4.a5	journal	January 2003

Similar Records

Anomalous physical transport in complex networks

Journal Article · 2010 · Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics · OSTI ID:1505821

Nicolaides, Christos; Cueto-Felgueroso, Luis; Juanes, Ruben

AN INDUCTIVE MAPPING WITH CONVOLUTIONAL REPRESENTATIONS FOR HUMAN SETTLEMENT DETECTION: PRELIMINARY RESULTS

Conference · 2017 · OSTI ID:1408008

Lunga, Dalton; Patlolla, Dilip Reddy; Yang, Lexie; +2 more

The expressivity of classical and quantum neural networks on entanglement entropy

Journal Article · 2024 · European Physical Journal. C, Particles and Fields (Online) · OSTI ID:2580057

Wu, Chih-Hung; Yen, Ching-Che

Related Subjects

97 MATHEMATICS AND COMPUTING
Applied mathematics
Computational science
Science & Technology

Title: Task-oriented machine learning surrogates for tipping points of agent-based models

Citation Formats

References (66)

Similar Records

Related Subjects