Time-series forecasting using manifold learning, radial basis function interpolation, and geometric harmonics
Abstract
We address a three-tier numerical framework based on nonlinear manifold learning for the forecasting of high-dimensional time series, relaxing the “curse of dimensionality” related to the training phase of surrogate/machine learning models. At the first step, we embed the high-dimensional time series into a reduced low-dimensional space using nonlinear manifold learning (local linear embedding and parsimonious diffusion maps). Then, we construct reduced-order surrogate models on the manifold (here, for our illustrations, we used multivariate autoregressive and Gaussian process regression models) to forecast the embedded dynamics. Finally, we solve the pre-image problem, thus lifting the embedded time series back to the original high-dimensional space using radial basis function interpolation and geometric harmonics. The proposed numerical data-driven scheme can also be applied as a reduced-order model procedure for the numerical solution/propagation of the (transient) dynamics of partial differential equations (PDEs). In conclusion, we assess the performance of the proposed scheme via three different families of problems: (a) the forecasting of synthetic time series generated by three simplistic linear and weakly nonlinear stochastic models resembling electroencephalography signals, (b) the prediction/propagation of the solution profiles of a linear parabolic PDE and the Brusselator model (a set of two nonlinear parabolic PDEs), and (c) themore »
- Authors:
-
[2];
[3];
[1]
- University of Naples Federico II (Italy)
- Technion-Israel Institute of Technology, Haifa (Israel)
- Johns Hopkins University, Baltimore, MD (United States)
- Publication Date:
- Research Org.:
- Johns Hopkins University, Baltimore, MD (United States)
- Sponsoring Org.:
- USDOE; US Air Force Office of Scientific Research (AFOSR)
- OSTI Identifier:
- 1982387
- Alternate Identifier(s):
- OSTI ID: 1880176
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Chaos: An Interdisciplinary Journal of Nonlinear Science
- Additional Journal Information:
- Journal Volume: 32; Journal Issue: 8; Journal ID: ISSN 1054-1500
- Publisher:
- American Institute of Physics (AIP)
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSICS; 97 MATHEMATICS AND COMPUTING; Mathematics; Physics; Dynamical systems; Machine learning; Numerical methods; Gaussian processes; Time series analysis
Citation Formats
Papaioannou, Panagiotis G., Talmon, Ronen, Kevrekidis, Ioannis G., and Siettos, Constantinos. Time-series forecasting using manifold learning, radial basis function interpolation, and geometric harmonics. United States: N. p., 2022.
Web. doi:10.1063/5.0094887.
Papaioannou, Panagiotis G., Talmon, Ronen, Kevrekidis, Ioannis G., & Siettos, Constantinos. Time-series forecasting using manifold learning, radial basis function interpolation, and geometric harmonics. United States. https://doi.org/10.1063/5.0094887
Papaioannou, Panagiotis G., Talmon, Ronen, Kevrekidis, Ioannis G., and Siettos, Constantinos. Mon .
"Time-series forecasting using manifold learning, radial basis function interpolation, and geometric harmonics". United States. https://doi.org/10.1063/5.0094887. https://www.osti.gov/servlets/purl/1982387.
@article{osti_1982387,
title = {Time-series forecasting using manifold learning, radial basis function interpolation, and geometric harmonics},
author = {Papaioannou, Panagiotis G. and Talmon, Ronen and Kevrekidis, Ioannis G. and Siettos, Constantinos},
abstractNote = {We address a three-tier numerical framework based on nonlinear manifold learning for the forecasting of high-dimensional time series, relaxing the “curse of dimensionality” related to the training phase of surrogate/machine learning models. At the first step, we embed the high-dimensional time series into a reduced low-dimensional space using nonlinear manifold learning (local linear embedding and parsimonious diffusion maps). Then, we construct reduced-order surrogate models on the manifold (here, for our illustrations, we used multivariate autoregressive and Gaussian process regression models) to forecast the embedded dynamics. Finally, we solve the pre-image problem, thus lifting the embedded time series back to the original high-dimensional space using radial basis function interpolation and geometric harmonics. The proposed numerical data-driven scheme can also be applied as a reduced-order model procedure for the numerical solution/propagation of the (transient) dynamics of partial differential equations (PDEs). In conclusion, we assess the performance of the proposed scheme via three different families of problems: (a) the forecasting of synthetic time series generated by three simplistic linear and weakly nonlinear stochastic models resembling electroencephalography signals, (b) the prediction/propagation of the solution profiles of a linear parabolic PDE and the Brusselator model (a set of two nonlinear parabolic PDEs), and (c) the forecasting of a real-world data set containing daily time series of ten key foreign exchange rates spanning the time period 3 September 2001–29 October 2020.},
doi = {10.1063/5.0094887},
journal = {Chaos: An Interdisciplinary Journal of Nonlinear Science},
number = 8,
volume = 32,
place = {United States},
year = {Mon Aug 08 00:00:00 EDT 2022},
month = {Mon Aug 08 00:00:00 EDT 2022}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Reduced Models in Chemical Kinetics via Nonlinear Data-Mining
journal, January 2014
- Chiavazzo, Eliodoro; Gear, Charles; Dsilva, Carmeline
- Processes, Vol. 2, Issue 1
Predicting Spatio-temporal Time Series Using Dimension Reduced Local States
journal, October 2019
- Isensee, Jonas; Datseris, George; Parlitz, Ulrich
- Journal of Nonlinear Science, Vol. 30, Issue 3
Time Series Forecasting with Gaussian Processes Needs Priors
book, January 2021
- Corani, Giorgio; Benavoli, Alessio; Zaffalon, Marco
- Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track
A divide and conquer method for the symmetric tridiagonal eigenproblem
journal, June 1980
- Cuppen, J. J. M.
- Numerische Mathematik, Vol. 36, Issue 2
Observing spatio-temporal dynamics of excitable media using reservoir computing
journal, April 2018
- Zimmermann, Roland S.; Parlitz, Ulrich
- Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 28, Issue 4
Can social microblogging be used to forecast intraday exchange rates?
journal, October 2013
- Papaioannou, Panagiotis; Russo, Lucia; Papaioannou, George
- NETNOMICS: Economic Research and Electronic Networking, Vol. 14, Issue 1-2
Dynamic mode decomposition of numerical and experimental data
journal, July 2010
- Schmid, Peter J.
- Journal of Fluid Mechanics, Vol. 656
The Isomap Algorithm and Topological Stability
journal, January 2002
- Balasubramanian, M.
- Science, Vol. 295, Issue 5552
On the Koopman Operator of Algorithms
journal, January 2020
- Dietrich, Felix; Thiem, Thomas N.; Kevrekidis, Ioannis G.
- SIAM Journal on Applied Dynamical Systems, Vol. 19, Issue 2
Data-Driven Reduction for a Class of Multiscale Fast-Slow Stochastic Dynamical Systems
journal, January 2016
- Dsilva, Carmeline J.; Talmon, Ronen; Gear, C. William
- SIAM Journal on Applied Dynamical Systems, Vol. 15, Issue 3
Manifold parametrizations by eigenfunctions of the Laplacian and heat kernels
journal, February 2008
- Jones, P. W.; Maggioni, M.; Schul, R.
- Proceedings of the National Academy of Sciences, Vol. 105, Issue 6
Earthquake-explosion discrimination using diffusion maps
journal, September 2016
- Rabin, N.; Bregman, Y.; Lindenbaum, O.
- Geophysical Journal International, Vol. 207, Issue 3
A Global Geometric Framework for Nonlinear Dimensionality Reduction
journal, December 2000
- Tenenbaum, J. B.
- Science, Vol. 290, Issue 5500
To infinity and some glimpses of beyond
journal, November 2017
- Kevrekidis, Panayotis G.; Siettos, Constantinos I.; Kevrekidis, Yannis G.
- Nature Communications, Vol. 8, Issue 1
Nonlinear principal component analysis using autoassociative neural networks
journal, February 1991
- Kramer, Mark A.
- AIChE Journal, Vol. 37, Issue 2
A nonlinear manifold-based reduced order model for multiscale analysis of heterogeneous hyperelastic materials
journal, May 2016
- Bhattacharjee, Satyaki; Matouš, Karel
- Journal of Computational Physics, Vol. 313
Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks
journal, May 2018
- Vlachas, Pantelis R.; Byeon, Wonmin; Wan, Zhong Y.
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 474, Issue 2213
ARPACK Users' Guide
book, January 1998
- Lehoucq, R. B.; Sorensen, D. C.; Yang, C.
- Software, Environments, and Tools
Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach
journal, January 2018
- Pathak, Jaideep; Hunt, Brian; Girvan, Michelle
- Physical Review Letters, Vol. 120, Issue 2
The Runge phenomenon and spatially variable shape parameters in RBF interpolation
journal, August 2007
- Fornberg, Bengt; Zuev, Julia
- Computers & Mathematics with Applications, Vol. 54, Issue 3
Diffusion maps
journal, July 2006
- Coifman, Ronald R.; Lafon, Stéphane
- Applied and Computational Harmonic Analysis, Vol. 21, Issue 1
The Gap-Tooth Scheme for Homogenization Problems
journal, January 2005
- Samaey, Giovanni; Roose, Dirk; Kevrekidis, Ioannis G.
- Multiscale Modeling & Simulation, Vol. 4, Issue 1
25 years of time series forecasting
journal, January 2006
- De Gooijer, Jan G.; Hyndman, Rob J.
- International Journal of Forecasting, Vol. 22, Issue 3
LSTM: A Search Space Odyssey
journal, October 2017
- Greff, Klaus; Srivastava, Rupesh K.; Koutnik, Jan
- IEEE Transactions on Neural Networks and Learning Systems, Vol. 28, Issue 10
A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series
journal, November 2006
- Marcellino, Massimiliano; Stock, James H.; Watson, Mark W.
- Journal of Econometrics, Vol. 135, Issue 1-2
Patch dynamics with buffers for homogenization problems
journal, March 2006
- Samaey, Giovanni; Kevrekidis, Ioannis G.; Roose, Dirk
- Journal of Computational Physics, Vol. 213, Issue 1
datafold: data-driven models for point clouds and time series on manifolds
journal, July 2020
- Lehmberg, Daniel; Dietrich, Felix; Köster, Gerta
- Journal of Open Source Software, Vol. 5, Issue 51
Coarse-grained variables for particle-based models: diffusion maps and animal swarming simulations
journal, September 2014
- Liu, Ping; Safford, Hannah R.; Couzin, Iain D.
- Computational Particle Mechanics, Vol. 1, Issue 4
Fredholm and Volterra Integral Equations of the Second Kind
journal, January 1990
- Press, William H.; Teukolsky, Saul A.
- Computers in Physics, Vol. 4, Issue 5
A data-driven reduced-order model of nonlinear processes based on diffusion maps and artificial neural networks
journal, October 2020
- Koronaki, E. D.; Nikas, A. M.; Boudouvis, A. G.
- Chemical Engineering Journal, Vol. 397
Intrinsic Isometric Manifold Learning with Application to Localization
journal, January 2019
- Schwartz, Ariel; Talmon, Ronen
- SIAM Journal on Imaging Sciences, Vol. 12, Issue 3
Nonlinear Component Analysis as a Kernel Eigenvalue Problem
journal, July 1998
- Schölkopf, Bernhard; Smola, Alexander; Müller, Klaus-Robert
- Neural Computation, Vol. 10, Issue 5
Gaussian processes for time-series modelling
journal, February 2013
- Roberts, S.; Osborne, M.; Ebden, M.
- Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 371, Issue 1984
Time series forecasting for nonlinear and non-stationary processes: a review and comparative study
journal, January 2015
- Cheng, Changqing; Sa-Ngasoongsong, Akkarapol; Beyca, Omer
- IIE Transactions, Vol. 47, Issue 10
Applied Koopman Theory for Partial Differential Equations and Data-Driven Modeling of Spatio-Temporal Systems
journal, December 2018
- Nathan Kutz, J.; Proctor, J. L.; Brunton, S. L.
- Complexity, Vol. 2018
Molecular enhanced sampling with autoencoders: On-the-fly collective variable discovery and accelerated free energy landscape exploration: Molecular Enhanced Sampling with Autoencoders: On-The-Fly Collective Variable Discovery and Accelerated Free Energy Landscape Exploration
journal, September 2018
- Chen, Wei; Ferguson, Andrew L.
- Journal of Computational Chemistry, Vol. 39, Issue 25
Nonlinear Dimensionality Reduction by Locally Linear Embedding
journal, December 2000
- Roweis, S. T.
- Science, Vol. 290, Issue 5500
Inverting nonlinear dimensionality reduction with scale-free radial basis function interpolation
journal, July 2014
- Monnig, Nathan D.; Fornberg, Bengt; Meyer, François G.
- Applied and Computational Harmonic Analysis, Vol. 37, Issue 1
The MATLAB ODE Suite
journal, January 1997
- Shampine, Lawrence F.; Reichelt, Mark W.
- SIAM Journal on Scientific Computing, Vol. 18, Issue 1
"Coarse" stability and bifurcation analysis using time-steppers: A reaction-diffusion example
journal, August 2000
- Theodoropoulos, C.; Qian, Y. -H.; Kevrekidis, I. G.
- Proceedings of the National Academy of Sciences, Vol. 97, Issue 18
Carry Trades and Global Foreign Exchange Volatility
journal, March 2012
- Menkhoff, Lukas; Sarno, Lucio; Schmeling, Maik
- The Journal of Finance, Vol. 67, Issue 2
Reduced-space Gaussian Process Regression for data-driven probabilistic forecast of chaotic dynamical systems
journal, April 2017
- Wan, Zhong Yi; Sapsis, Themistoklis P.
- Physica D: Nonlinear Phenomena, Vol. 345
Partial directed coherence: a new concept in neural structure determination
journal, May 2001
- Baccalá, Luiz A.; Sameshima, Koichi
- Biological Cybernetics, Vol. 84, Issue 6
Parsimonious representation of nonlinear dynamical systems through manifold learning: A chemotaxis case study
journal, May 2018
- Dsilva, Carmeline J.; Talmon, Ronen; Coifman, Ronald R.
- Applied and Computational Harmonic Analysis, Vol. 44, Issue 3
A comparative study of Reservoir Computing strategies for monthly time series prediction
journal, June 2010
- Wyffels, F.; Schrauwen, B.
- Neurocomputing, Vol. 73, Issue 10-12
Emergent Spaces for Coupled Oscillators
journal, May 2020
- Thiem, Thomas N.; Kooshkbaghi, Mahdi; Bertalan, Tom
- Frontiers in Computational Neuroscience, Vol. 14
SciPy 1.0: fundamental algorithms for scientific computing in Python
journal, February 2020
- Virtanen, Pauli; Gommers, Ralf; Oliphant, Travis E.
- Nature Methods
A nonlinear dimensionality reduction framework using smooth geodesics
journal, March 2019
- Gajamannage, Kelum; Paffenroth, Randy; Bollt, Erik M.
- Pattern Recognition, Vol. 87
A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
journal, June 2015
- Williams, Matthew O.; Kevrekidis, Ioannis G.; Rowley, Clarence W.
- Journal of Nonlinear Science, Vol. 25, Issue 6
Facing the high-dimensions: Inverse projection with radial basis functions
journal, May 2015
- Amorim, Elisa; Vital Brazil, Emilio; Mena-Chalco, Jesús
- Computers & Graphics, Vol. 48
Dynamic mode decomposition for financial trading strategies
journal, April 2016
- Mann, Jordan; Kutz, J. Nathan
- Quantitative Finance, Vol. 16, Issue 11
Detecting intrinsic slow variables in stochastic dynamical systems by anisotropic diffusion maps
journal, August 2009
- Singer, A.; Erban, R.; Kevrekidis, I. G.
- Proceedings of the National Academy of Sciences, Vol. 106, Issue 38
The Sharpe Ratio
journal, October 1994
- Sharpe, William F.
- The Journal of Portfolio Management, Vol. 21, Issue 1
Coarse-scale PDEs from fine-scale observations via machine learning
journal, January 2020
- Lee, Seungjoon; Kooshkbaghi, Mahdi; Spiliotis, Konstantinos
- Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 30, Issue 1
Data Structures for Statistical Computing in Python
conference, January 2010
- McKinney, Wes
- Proceedings of the Python in Science Conference
Investigating Causal Relations by Econometric Models and Cross-spectral Methods
journal, August 1969
- Granger, C. W. J.
- Econometrica, Vol. 37, Issue 3
Convolutional autoencoder and conditional random fields hybrid for predicting spatial-temporal chaos
journal, December 2019
- Herzog, S.; Wörgötter, F.; Parlitz, U.
- Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 29, Issue 12
A Limited Memory Algorithm for Bound Constrained Optimization
journal, September 1995
- Byrd, Richard H.; Lu, Peihuang; Nocedal, Jorge
- SIAM Journal on Scientific Computing, Vol. 16, Issue 5
Statsmodels: Econometric and Statistical Modeling with Python
conference, January 2010
- Seabold, Skipper; Perktold, Josef
- Proceedings of the Python in Science Conference
Reservoir computing approaches to recurrent neural network training
journal, August 2009
- Lukoševičius, Mantas; Jaeger, Herbert
- Computer Science Review, Vol. 3, Issue 3
Discovering governing equations from data by sparse identification of nonlinear dynamical systems
journal, March 2016
- Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan
- Proceedings of the National Academy of Sciences, Vol. 113, Issue 15
Dynamic Mode Decomposition
book, January 2016
- Kutz, J. Nathan; Brunton, Steven L.; Brunton, Bingni W.
- Society for Industrial and Applied Mathematics
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
journal, June 2003
- Belkin, Mikhail; Niyogi, Partha
- Neural Computation, Vol. 15, Issue 6
Detecting strange attractors in turbulence
book, January 1981
- Takens, Floris
- Lecture Notes in Mathematics
Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization
journal, December 1997
- Zhu, Ciyou; Byrd, Richard H.; Lu, Peihuang
- ACM Transactions on Mathematical Software, Vol. 23, Issue 4
Solving Ill-Posed Linear Systems with GMRES and a Singular Preconditioner
journal, January 2012
- Eldén, Lars; Simoncini, Valeria
- SIAM Journal on Matrix Analysis and Applications, Vol. 33, Issue 4
Equation-free: The computer-aided analysis of complex multiscale systems
journal, January 2004
- Kevrekidis, Ioannis G.; Gear, C. William; Hummer, Gerhard
- AIChE Journal, Vol. 50, Issue 7
Analysis of Fluid Flows via Spectral Properties of the Koopman Operator
journal, January 2013
- Mezić, Igor
- Annual Review of Fluid Mechanics, Vol. 45, Issue 1
Coarse Brownian dynamics for nematic liquid crystals: Bifurcation, projective integration, and control via stochastic simulation
journal, June 2003
- Siettos, C. I.; Graham, M. D.; Kevrekidis, I. G.
- The Journal of Chemical Physics, Vol. 118, Issue 22
Symmetry Breaking Instabilities in Dissipative Systems. II
journal, February 1968
- Prigogine, I.; Lefever, R.
- The Journal of Chemical Physics, Vol. 48, Issue 4
A Flexible Inner-Outer Preconditioned GMRES Algorithm
journal, March 1993
- Saad, Youcef
- SIAM Journal on Scientific Computing, Vol. 14, Issue 2
The gap-tooth method in particle simulations
journal, September 2003
- Gear, C. William; Li, Ju; Kevrekidis, Ioannis G.
- Physics Letters A, Vol. 316, Issue 3-4
Robust kernel Isomap
journal, March 2007
- Choi, Heeyoul; Choi, Seungjin
- Pattern Recognition, Vol. 40, Issue 3
Gaussian Processes in Machine Learning
book, January 2004
- Rasmussen, Carl Edward
- Advanced Lectures on Machine Learning
Equation-Free, Coarse-Grained Multiscale Computation: Enabling Mocroscopic Simulators to Perform System-Level Analysis
journal, January 2003
- Gear, C. William; Hyman, James M.; Kevrekidid, Panagiotis G.
- Communications in Mathematical Sciences, Vol. 1, Issue 4
A Nonlinear Causality Estimator Based on Non-Parametric Multiplicative Regression
journal, June 2016
- Nicolaou, Nicoletta; Constandinou, Timothy G.
- Frontiers in Neuroinformatics, Vol. 10
Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps
journal, May 2005
- Coifman, R. R.; Lafon, S.; Lee, A. B.
- Proceedings of the National Academy of Sciences, Vol. 102, Issue 21
Attractor Modeling and Empirical Nonlinear Model Reduction of Dissipative Dynamical Systems
journal, April 2007
- Bollt, Erik
- International Journal of Bifurcation and Chaos, Vol. 17, Issue 04
Geometric harmonics: A novel tool for multiscale out-of-sample extension of empirical functions
journal, July 2006
- Coifman, Ronald R.; Lafon, Stéphane
- Applied and Computational Harmonic Analysis, Vol. 21, Issue 1
Diffusion maps, spectral clustering and reaction coordinates of dynamical systems
journal, July 2006
- Nadler, Boaz; Lafon, Stéphane; Coifman, Ronald R.
- Applied and Computational Harmonic Analysis, Vol. 21, Issue 1, p. 113-127
Diffusion Maps, Reduction Coordinates, and Low Dimensional Representation of Stochastic Systems
journal, January 2008
- Coifman, R. R.; Kevrekidis, I. G.; Lafon, S.
- Multiscale Modeling & Simulation, Vol. 7, Issue 2
Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics
journal, June 2020
- Vlachas, P. R.; Pathak, J.; Hunt, B. R.
- Neural Networks, Vol. 126
Data-driven model reduction, Wiener projections, and the Koopman-Mori-Zwanzig formalism
journal, January 2021
- Lin, Kevin K.; Lu, Fei
- Journal of Computational Physics, Vol. 424
Symmetric Gauge Functions and Unitarily Invariant Norms
journal, January 1960
- Mirsky, L.
- The Quarterly Journal of Mathematics, Vol. 11, Issue 1
Connecting dots: from local covariance to empirical intrinsic geometry and locally linear embedding
journal, January 2019
- Malik, John; Shen, Chao; Wu, Hau-Tieng
- Pure and Applied Analysis, Vol. 1, Issue 4