Direct Estimation of Parameters in ODE Models Using WENDy: Weak-Form Estimation of Nonlinear Dynamics
Abstract
Abstract We introduce the Weak-form Estimation of Nonlinear Dynamics (WENDy) method for estimating model parameters for non-linear systems of ODEs. Without relying on any numerical differential equation solvers, WENDy computes accurate estimates and is robust to large (biologically relevant) levels of measurement noise. For low dimensional systems with modest amounts of data, WENDy is competitive with conventional forward solver-based nonlinear least squares methods in terms of speed and accuracy. For both higher dimensional systems and stiff systems, WENDy is typically both faster (often by orders of magnitude) and more accurate than forward solver-based approaches. The core mathematical idea involves an efficient conversion of the strong form representation of a model to its weak form, and then solving a regression problem to perform parameter inference. The core statistical idea rests on the Errors-In-Variables framework, which necessitates the use of the iteratively reweighted least squares algorithm. Further improvements are obtained by using orthonormal test functions, created from a set of $$$$C^{\infty }$$$$ bump functions of varying support sizes.We demonstrate the high robustness and computational efficiency by applying WENDy to estimate parameters in some common models from population biology, neuroscience, and biochemistry, including logistic growth, Lotka-Volterra, FitzHugh-Nagumo, Hindmarsh-Rose, and a Protein Transduction Benchmark model. Software and code for reproducing the examples is available at https://github.com/MathBioCU/WENDy .
- Publication Date:
- Research Org.:
- Univ. of Colorado, Boulder, CO (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- OSTI Identifier:
- 2007689
- Alternate Identifier(s):
- OSTI ID: 2008430
- Grant/Contract Number:
- SC0023346
- Resource Type:
- Published Article
- Journal Name:
- Bulletin of Mathematical Biology
- Additional Journal Information:
- Journal Name: Bulletin of Mathematical Biology Journal Volume: 85 Journal Issue: 11; Journal ID: ISSN 0092-8240
- Publisher:
- Springer Science + Business Media
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING; data-driven modeling; parameter estimation; parameter inference; weak form; test functions
Citation Formats
Bortz, David M., Messenger, Daniel A., and Dukic, Vanja. Direct Estimation of Parameters in ODE Models Using WENDy: Weak-Form Estimation of Nonlinear Dynamics. United States: N. p., 2023.
Web. doi:10.1007/s11538-023-01208-6.
Bortz, David M., Messenger, Daniel A., & Dukic, Vanja. Direct Estimation of Parameters in ODE Models Using WENDy: Weak-Form Estimation of Nonlinear Dynamics. United States. https://doi.org/10.1007/s11538-023-01208-6
Bortz, David M., Messenger, Daniel A., and Dukic, Vanja. Thu .
"Direct Estimation of Parameters in ODE Models Using WENDy: Weak-Form Estimation of Nonlinear Dynamics". United States. https://doi.org/10.1007/s11538-023-01208-6.
@article{osti_2007689,
title = {Direct Estimation of Parameters in ODE Models Using WENDy: Weak-Form Estimation of Nonlinear Dynamics},
author = {Bortz, David M. and Messenger, Daniel A. and Dukic, Vanja},
abstractNote = {Abstract We introduce the Weak-form Estimation of Nonlinear Dynamics (WENDy) method for estimating model parameters for non-linear systems of ODEs. Without relying on any numerical differential equation solvers, WENDy computes accurate estimates and is robust to large (biologically relevant) levels of measurement noise. For low dimensional systems with modest amounts of data, WENDy is competitive with conventional forward solver-based nonlinear least squares methods in terms of speed and accuracy. For both higher dimensional systems and stiff systems, WENDy is typically both faster (often by orders of magnitude) and more accurate than forward solver-based approaches. The core mathematical idea involves an efficient conversion of the strong form representation of a model to its weak form, and then solving a regression problem to perform parameter inference. The core statistical idea rests on the Errors-In-Variables framework, which necessitates the use of the iteratively reweighted least squares algorithm. Further improvements are obtained by using orthonormal test functions, created from a set of $$C^{\infty }$$ C ∞ bump functions of varying support sizes.We demonstrate the high robustness and computational efficiency by applying WENDy to estimate parameters in some common models from population biology, neuroscience, and biochemistry, including logistic growth, Lotka-Volterra, FitzHugh-Nagumo, Hindmarsh-Rose, and a Protein Transduction Benchmark model. Software and code for reproducing the examples is available at https://github.com/MathBioCU/WENDy .},
doi = {10.1007/s11538-023-01208-6},
journal = {Bulletin of Mathematical Biology},
number = 11,
volume = 85,
place = {United States},
year = {Thu Oct 05 00:00:00 EDT 2023},
month = {Thu Oct 05 00:00:00 EDT 2023}
}
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1007/s11538-023-01208-6
https://doi.org/10.1007/s11538-023-01208-6
Other availability
Search WorldCat to find libraries that may hold this journal
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Quasi-maximum likelihood estimation and inference in dynamic models with time-varying covariances
journal, January 1992
- Bollerslev, Tim; Wooldridge, Jeffrey M.
- Econometric Reviews, Vol. 11, Issue 2
A Spline Least Squares Method for Numerical Parameter Estimation in Differential Equations
journal, March 1982
- Varah, J. M.
- SIAM Journal on Scientific and Statistical Computing, Vol. 3, Issue 1
Model selection in systems and synthetic biology
journal, August 2013
- Kirk, Paul; Thorne, Thomas; Stumpf, Michael PH
- Current Opinion in Biotechnology, Vol. 24, Issue 4
Bayesian ranking of biochemical system models
journal, December 2007
- Vyshemirsky, Vladislav; Girolami, Mark A.
- Bioinformatics, Vol. 24, Issue 6
The influence of numerical error on parameter estimation and uncertainty quantification for advective PDE models
journal, May 2019
- Nardini, John T.; Bortz, D. M.
- Inverse Problems, Vol. 35, Issue 6
Variational system identification of the partial differential equations governing the physics of pattern-formation: Inference under varying fidelity and noise
journal, November 2019
- Wang, Z.; Huan, X.; Garikipati, K.
- Computer Methods in Applied Mechanics and Engineering, Vol. 356
Parameter estimation for differential equations: a generalized smoothing approach: Parameter Estimation for Differential Equations
journal, October 2007
- Ramsay, J. O.; Hooker, G.; Campbell, D.
- Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 69, Issue 5
A unified approach for sparse dynamical system inference from temporal measurements
journal, January 2018
- Pantazis, Yannis; Tsamardinos, Ioannis
- Bioinformatics, Vol. 35, Issue 18
A Joint estimation approach to sparse additive ordinary differential equations
journal, August 2022
- Zhang, Nan; Nanshan, Muye; Cao, Jiguo
- Statistics and Computing, Vol. 32, Issue 5
More about process identification
journal, July 1965
- Loeb, J.; Cahen, G.
- IEEE Transactions on Automatic Control, Vol. 10, Issue 3
A new method for the identification of systems
journal, August 1969
- Bellman, Richard
- Mathematical Biosciences, Vol. 5, Issue 1-2
Parametric Estimation of Ordinary Differential Equations With Orthogonality Conditions
journal, January 2014
- Brunel, Nicolas J-B.; Clairon, Quentin; d’Alché-Buc, Florence
- Journal of the American Statistical Association, Vol. 109, Issue 505
Spatial Regression With Partial Differential Equation Regularisation
journal, March 2021
- Sangalli, Laura M.
- International Statistical Review, Vol. 89, Issue 3
Sparse model selection via integral terms
journal, August 2017
- Schaeffer, Hayden; McCalla, Scott G.
- Physical Review E, Vol. 96, Issue 2
Parameter Estimation for Differential Equation Models Using a Framework of Measurement Error in Regression Models
journal, December 2008
- Liang, Hua; Wu, Hulin
- Journal of the American Statistical Association, Vol. 103, Issue 484
Learning sparse nonlinear dynamics via mixed-integer optimization
journal, January 2023
- Bertsimas, Dimitris; Gurnee, Wes
- Nonlinear Dynamics, Vol. 111, Issue 7
Weak SINDy: Galerkin-Based Data-Driven Model Selection
journal, January 2021
- Messenger, Daniel A.; Bortz, David M.
- Multiscale Modeling & Simulation, Vol. 19, Issue 3
Estimation of ordinary differential equation parameters using constrained local polynomial regression
journal, January 2014
- Ding, A. Adam; Wu, Hulin
- Statistica Sinica
Robust and optimal sparse regression for nonlinear PDE models
journal, October 2019
- Gurevich, Daniel R.; Reinbold, Patrick A. K.; Grigoriev, Roman O.
- Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 29, Issue 10
Weak SINDy for partial differential equations
journal, October 2021
- Messenger, Daniel A.; Bortz, David M.
- Journal of Computational Physics, Vol. 443
Parameter estimation of ODE’s via nonparametric estimators
journal, January 2008
- Brunel, Nicolas J-B.
- Electronic Journal of Statistics, Vol. 2, Issue 0
Differential equations in data analysis
journal, November 2020
- Dattner, Itai
- WIREs Computational Statistics, Vol. 13, Issue 6
Spatiotemporal system reconstruction using Fourier spectral operators and structure selection techniques
journal, December 2008
- Xu, Daolin; Khanmohamadi, Omid
- Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 18, Issue 4
Computational modeling of the dynamics of the MAP kinase cascade activated by surface and internalized EGF receptors
journal, April 2002
- Schoeberl, Birgit; Eichler-Jonsson, Claudia; Gilles, Ernst Dieter
- Nature Biotechnology, Vol. 20, Issue 4
Generation of finite difference formulas on arbitrarily spaced grids
journal, January 1988
- Fornberg, Bengt
- Mathematics of Computation, Vol. 51, Issue 184
Impulses and Physiological States in Theoretical Models of Nerve Membrane
journal, July 1961
- FitzHugh, Richard
- Biophysical Journal, Vol. 1, Issue 6
Identification of Systems Described by Partial Differential Equations
journal, June 1966
- Perdreauville, F. J.; Goodson, R. E.
- Journal of Basic Engineering, Vol. 88, Issue 2
Theory and application of the modulating function method—I. Review and theory of the method and theory of the spline-type modulating functions
journal, January 1993
- Preisig, H. A.; Rippin, D. W. T.
- Computers & Chemical Engineering, Vol. 17, Issue 1
Generalized sensitivity functions for size-structured population models
journal, May 2015
- Keck, Dustin D.; Bortz, David M.
- Journal of Inverse and Ill-posed Problems, Vol. 24, Issue 3
Using noisy or incomplete data to discover models of spatiotemporal dynamics
journal, January 2020
- Reinbold, Patrick A. K.; Gurevich, Daniel R.; Grigoriev, Roman O.
- Physical Review E, Vol. 101, Issue 1
Accurate model selection computations
journal, December 2006
- Bortz, D. M.
- Journal of Critical Care, Vol. 21, Issue 4
Spatiotemporal system identification on nonperiodic domains using Chebyshev spectral operators and system reduction algorithms
journal, August 2009
- Khanmohamadi, Omid; Xu, Daolin
- Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 19, Issue 3
Discovering governing equations from data by sparse identification of nonlinear dynamical systems
journal, March 2016
- Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan
- Proceedings of the National Academy of Sciences, Vol. 113, Issue 15
Explicit Estimation of Derivatives from data and Differential Equations by Gaussian Process Regression
journal, January 2021
- Wang, Hongqiao; Zhou, X.
- International Journal for Uncertainty Quantification, Vol. 11, Issue 4
Parameter estimation in continuous-time dynamic models using principal differential analysis
journal, February 2006
- Poyton, A. A.; Varziri, M. S.; McAuley, K. B.
- Computers & Chemical Engineering, Vol. 30, Issue 4
Data-driven discovery of partial differential equations
journal, April 2017
- Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L.
- Science Advances, Vol. 3, Issue 4
PySINDy: A comprehensive Python package for robust sparse system identification
journal, January 2022
- Kaptanoglu, Alan; de Silva, Brian; Fasel, Urban
- Journal of Open Source Software, Vol. 7, Issue 69
The growth of mixed populations: Two species competing for a common food supply
book, January 1978
- Lotka, Alfred J.
- Lecture Notes in Biomathematics
ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems
journal, April 2020
- Wenk, Philippe; Abbati, Gabriele; Osborne, Michael A.
- Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, Issue 04
Modelling and parameter inference of predator–prey dynamics in heterogeneous environments using the direct integral approach
journal, January 2017
- Dattner, Itai; Miller, Ezer; Petrenko, Margarita
- Journal of The Royal Society Interface, Vol. 14, Issue 126
Inference of dynamic systems from noisy and sparse data via manifold-constrained Gaussian processes
journal, April 2021
- Yang, Shihao; Wong, Samuel W. K.; Kou, S. C.
- Proceedings of the National Academy of Sciences, Vol. 118, Issue 15
Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control
journal, April 2022
- Fasel, U.; Kutz, J. N.; Brunton, B. W.
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 478, Issue 2260
A model of neuronal bursting using three coupled first order differential equations
journal, March 1984
- Hindmarsh, J. L.; Rose, R. M.
- Proceedings of the Royal Society of London. Series B. Biological Sciences, Vol. 221, Issue 1222, p. 87-102
An analysis of variance test for normality (complete samples)
journal, December 1965
- Shapiro, S. S.; Wilk, M. B.
- Biometrika, Vol. 52, Issue 3-4
Learning anisotropic interaction rules from individual trajectories in a heterogeneous cellular population
journal, October 2022
- Messenger, Daniel A.; Wheeler, Graycen E.; Liu, Xuedong
- Journal of The Royal Society Interface, Vol. 19, Issue 195
Bayesian uncertainty quantification for data-driven equation learning
journal, October 2021
- Martina-Perez, Simon; Simpson, Matthew J.; Baker, Ruth E.
- Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 477, Issue 2254
Uncertainty in predictions of disease spread and public health responses to bioterrorism and emerging diseases
journal, October 2006
- Elderd, B. D.; Dukic, V. M.; Dwyer, G.
- Proceedings of the National Academy of Sciences, Vol. 103, Issue 42
Learning mean-field equations from particle data using WSINDy
journal, November 2022
- Messenger, Daniel A.; Bortz, David M.
- Physica D: Nonlinear Phenomena, Vol. 439