Direct Estimation of Parameters in ODE Models Using WENDy: Weak-Form Estimation of Nonlinear Dynamics

Bortz, David M.; Messenger, Daniel A.; Dukic, Vanja

doi:10.1007/s11538-023-01208-6

Title: Direct Estimation of Parameters in ODE Models Using WENDy: Weak-Form Estimation of Nonlinear Dynamics

Journal Article · 04 October 2023 · Bulletin of Mathematical Biology

DOI: https://doi.org/10.1007/s11538-023-01208-6 · OSTI ID:2007689

; Messenger, Daniel A.; Dukic, Vanja

Abstract We introduce the Weak-form Estimation of Nonlinear Dynamics (WENDy) method for estimating model parameters for non-linear systems of ODEs. Without relying on any numerical differential equation solvers, WENDy computes accurate estimates and is robust to large (biologically relevant) levels of measurement noise. For low dimensional systems with modest amounts of data, WENDy is competitive with conventional forward solver-based nonlinear least squares methods in terms of speed and accuracy. For both higher dimensional systems and stiff systems, WENDy is typically both faster (often by orders of magnitude) and more accurate than forward solver-based approaches. The core mathematical idea involves an efficient conversion of the strong form representation of a model to its weak form, and then solving a regression problem to perform parameter inference. The core statistical idea rests on the Errors-In-Variables framework, which necessitates the use of the iteratively reweighted least squares algorithm. Further improvements are obtained by using orthonormal test functions, created from a set of $$$$C^{\infty }$$$$ $C^{\infty}$ bump functions of varying support sizes.We demonstrate the high robustness and computational efficiency by applying WENDy to estimate parameters in some common models from population biology, neuroscience, and biochemistry, including logistic growth, Lotka-Volterra, FitzHugh-Nagumo, Hindmarsh-Rose, and a Protein Transduction Benchmark model. Software and code for reproducing the examples is available at https://github.com/MathBioCU/WENDy .

View Journal Article

Cite

Export

Save

Research Organization:: Univ. of Colorado, Boulder, CO (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: SC0023346

OSTI ID:: 2007689

Journal Information:: Bulletin of Mathematical Biology, Journal Name: Bulletin of Mathematical Biology Journal Issue: 11 Vol. 85; ISSN 0092-8240

Publisher:: Springer Science + Business MediaCopyright Statement

Country of Publication:: United States

Language:: English

References (49)

Differential equations in data analysis Dattner, Itai WIREs Computational Statistics, Vol. 13, Issue 6 https://doi.org/10.1002/wics.1534	journal	November 2020
Distributions Duistermaat, J. J.; Kolk, J. A. C. https://doi.org/10.1007/978-0-8176-4675-2	book	January 2010
The growth of mixed populations: Two species competing for a common food supply Lotka, Alfred J. Lecture Notes in Biomathematics https://doi.org/10.1007/978-3-642-50151-7_12	book	January 1978
Learning sparse nonlinear dynamics via mixed-integer optimization Bertsimas, Dimitris; Gurnee, Wes Nonlinear Dynamics, Vol. 111, Issue 7 https://doi.org/10.1007/s11071-022-08178-9	journal	January 2023
A Joint estimation approach to sparse additive ordinary differential equations Zhang, Nan; Nanshan, Muye; Cao, Jiguo Statistics and Computing, Vol. 32, Issue 5 https://doi.org/10.1007/s11222-022-10117-y	journal	August 2022
A new method for the identification of systems Bellman, Richard Mathematical Biosciences, Vol. 5, Issue 1-2 https://doi.org/10.1016/0025-5564(69)90042-X	journal	August 1969
Theory and application of the modulating function method—I. Review and theory of the method and theory of the spline-type modulating functions Preisig, H. A.; Rippin, D. W. T. Computers & Chemical Engineering, Vol. 17, Issue 1 https://doi.org/10.1016/0098-1354(93)80001-4	journal	January 1993
Impulses and Physiological States in Theoretical Models of Nerve Membrane FitzHugh, Richard Biophysical Journal, Vol. 1, Issue 6 https://doi.org/10.1016/S0006-3495(61)86902-6	journal	July 1961
Variational system identification of the partial differential equations governing the physics of pattern-formation: Inference under varying fidelity and noise Wang, Z.; Huan, X.; Garikipati, K. Computer Methods in Applied Mechanics and Engineering, Vol. 356 https://doi.org/10.1016/j.cma.2019.07.007	journal	November 2019
Parameter estimation in continuous-time dynamic models using principal differential analysis Poyton, A. A.; Varziri, M. S.; McAuley, K. B. Computers & Chemical Engineering, Vol. 30, Issue 4 https://doi.org/10.1016/j.compchemeng.2005.11.008	journal	February 2006
Model selection in systems and synthetic biology Kirk, Paul; Thorne, Thomas; Stumpf, Michael PH Current Opinion in Biotechnology, Vol. 24, Issue 4 https://doi.org/10.1016/j.copbio.2013.03.012	journal	August 2013
Weak SINDy for partial differential equations Messenger, Daniel A.; Bortz, David M. Journal of Computational Physics, Vol. 443 https://doi.org/10.1016/j.jcp.2021.110525	journal	October 2021
Accurate model selection computations Bortz, D. M. Journal of Critical Care, Vol. 21, Issue 4 https://doi.org/10.1016/j.jcrc.2006.10.029	journal	December 2006
Learning mean-field equations from particle data using WSINDy Messenger, Daniel A.; Bortz, David M. Physica D: Nonlinear Phenomena, Vol. 439 https://doi.org/10.1016/j.physd.2022.133406	journal	November 2022
Computational modeling of the dynamics of the MAP kinase cascade activated by surface and internalized EGF receptors Schoeberl, Birgit; Eichler-Jonsson, Claudia; Gilles, Ernst Dieter Nature Biotechnology, Vol. 20, Issue 4 https://doi.org/10.1038/nbt0402-370	journal	April 2002
Spatiotemporal system reconstruction using Fourier spectral operators and structure selection techniques Xu, Daolin; Khanmohamadi, Omid Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 18, Issue 4 https://doi.org/10.1063/1.3030611	journal	December 2008
Spatiotemporal system identification on nonperiodic domains using Chebyshev spectral operators and system reduction algorithms Khanmohamadi, Omid; Xu, Daolin Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 19, Issue 3 https://doi.org/10.1063/1.3180843	journal	August 2009
Robust and optimal sparse regression for nonlinear PDE models Gurevich, Daniel R.; Reinbold, Patrick A. K.; Grigoriev, Roman O. Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 29, Issue 10 https://doi.org/10.1063/1.5120861	journal	October 2019
Uncertainty in predictions of disease spread and public health responses to bioterrorism and emerging diseases Elderd, B. D.; Dukic, V. M.; Dwyer, G. Proceedings of the National Academy of Sciences, Vol. 103, Issue 42 https://doi.org/10.1073/pnas.0600816103	journal	October 2006
Discovering governing equations from data by sparse identification of nonlinear dynamical systems Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan Proceedings of the National Academy of Sciences, Vol. 113, Issue 15 https://doi.org/10.1073/pnas.1517384113	journal	March 2016
Inference of dynamic systems from noisy and sparse data via manifold-constrained Gaussian processes Yang, Shihao; Wong, Samuel W. K.; Kou, S. C. Proceedings of the National Academy of Sciences, Vol. 118, Issue 15 https://doi.org/10.1073/pnas.2020397118	journal	April 2021
Parametric Estimation of Ordinary Differential Equations With Orthogonality Conditions Brunel, Nicolas J-B.; Clairon, Quentin; d’Alché-Buc, Florence Journal of the American Statistical Association, Vol. 109, Issue 505 https://doi.org/10.1080/01621459.2013.841583	journal	January 2014
Quasi-maximum likelihood estimation and inference in dynamic models with time-varying covariances Bollerslev, Tim; Wooldridge, Jeffrey M. Econometric Reviews, Vol. 11, Issue 2 https://doi.org/10.1080/07474939208800229	journal	January 1992
The influence of numerical error on parameter estimation and uncertainty quantification for advective PDE models Nardini, John T.; Bortz, D. M. Inverse Problems, Vol. 35, Issue 6 https://doi.org/10.1088/1361-6420/ab10bb	journal	May 2019
Generation of finite difference formulas on arbitrarily spaced grids Fornberg, Bengt Mathematics of Computation, Vol. 51, Issue 184 https://doi.org/10.1090/S0025-5718-1988-0935077-0	journal	January 1988
Bayesian ranking of biochemical system models Vyshemirsky, Vladislav; Girolami, Mark A. Bioinformatics, Vol. 24, Issue 6 https://doi.org/10.1093/bioinformatics/btm607	journal	December 2007
A unified approach for sparse dynamical system inference from temporal measurements Pantazis, Yannis; Tsamardinos, Ioannis Bioinformatics, Vol. 35, Issue 18 https://doi.org/10.1093/bioinformatics/btz065	journal	January 2018
An analysis of variance test for normality (complete samples) Shapiro, S. S.; Wilk, M. B. Biometrika, Vol. 52, Issue 3-4 https://doi.org/10.1093/biomet/52.3-4.591	journal	December 1965
Modelling and parameter inference of predator–prey dynamics in heterogeneous environments using the direct integral approach Dattner, Itai; Miller, Ezer; Petrenko, Margarita Journal of The Royal Society Interface, Vol. 14, Issue 126 https://doi.org/10.1098/rsif.2016.0525	journal	January 2017
Learning anisotropic interaction rules from individual trajectories in a heterogeneous cellular population Messenger, Daniel A.; Wheeler, Graycen E.; Liu, Xuedong Journal of The Royal Society Interface, Vol. 19, Issue 195 https://doi.org/10.1098/rsif.2022.0412	journal	October 2022
Bayesian uncertainty quantification for data-driven equation learning Martina-Perez, Simon; Simpson, Matthew J.; Baker, Ruth E. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 477, Issue 2254 https://doi.org/10.1098/rspa.2021.0426	journal	October 2021
Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control Fasel, U.; Kutz, J. N.; Brunton, B. W. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 478, Issue 2260 https://doi.org/10.1098/rspa.2021.0904	journal	April 2022
A model of neuronal bursting using three coupled first order differential equations Hindmarsh, J. L.; Rose, R. M. Proceedings of the Royal Society of London. Series B. Biological Sciences, Vol. 221, Issue 1222, p. 87-102 https://doi.org/10.1098/rspb.1984.0024	journal	March 1984
Using noisy or incomplete data to discover models of spatiotemporal dynamics Reinbold, Patrick A. K.; Gurevich, Daniel R.; Grigoriev, Roman O. Physical Review E, Vol. 101, Issue 1 https://doi.org/10.1103/PhysRevE.101.010203	journal	January 2020
Sparse model selection via integral terms Schaeffer, Hayden; McCalla, Scott G. Physical Review E, Vol. 96, Issue 2 https://doi.org/10.1103/PhysRevE.96.023302	journal	August 2017
More about process identification Loeb, J.; Cahen, G. IEEE Transactions on Automatic Control, Vol. 10, Issue 3 https://doi.org/10.1109/TAC.1965.1098172	journal	July 1965
Spatial Regression With Partial Differential Equation Regularisation Sangalli, Laura M. International Statistical Review, Vol. 89, Issue 3 https://doi.org/10.1111/insr.12444	journal	March 2021
Parameter estimation for differential equations: a generalized smoothing approach: Parameter Estimation for Differential Equations Ramsay, J. O.; Hooker, G.; Campbell, D. Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 69, Issue 5 https://doi.org/10.1111/j.1467-9868.2007.00610.x	journal	October 2007
Identification of Systems Described by Partial Differential Equations Perdreauville, F. J.; Goodson, R. E. Journal of Basic Engineering, Vol. 88, Issue 2 https://doi.org/10.1115/1.3645880	journal	June 1966
Data-driven discovery of partial differential equations Rudy, Samuel H.; Brunton, Steven L.; Proctor, Joshua L. Science Advances, Vol. 3, Issue 4 https://doi.org/10.1126/sciadv.1602614	journal	April 2017
A Spline Least Squares Method for Numerical Parameter Estimation in Differential Equations Varah, J. M. SIAM Journal on Scientific and Statistical Computing, Vol. 3, Issue 1 https://doi.org/10.1137/0903003	journal	March 1982
Weak SINDy: Galerkin-Based Data-Driven Model Selection Messenger, Daniel A.; Bortz, David M. Multiscale Modeling & Simulation, Vol. 19, Issue 3 https://doi.org/10.1137/20M1343166	journal	January 2021
Parameter Estimation for Differential Equation Models Using a Framework of Measurement Error in Regression Models Liang, Hua; Wu, Hulin Journal of the American Statistical Association, Vol. 103, Issue 484 https://doi.org/10.1198/016214508000000797	journal	December 2008
Parameter estimation of ODE’s via nonparametric estimators Brunel, Nicolas J-B. Electronic Journal of Statistics, Vol. 2, Issue 0 https://doi.org/10.1214/07-EJS132	journal	January 2008
Generalized sensitivity functions for size-structured population models Keck, Dustin D.; Bortz, David M. Journal of Inverse and Ill-posed Problems, Vol. 24, Issue 3 https://doi.org/10.1515/jiip-2014-0041	journal	May 2015
ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems Wenk, Philippe; Abbati, Gabriele; Osborne, Michael A. Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, Issue 04 https://doi.org/10.1609/aaai.v34i04.6106	journal	April 2020
Explicit Estimation of Derivatives from data and Differential Equations by Gaussian Process Regression Wang, Hongqiao; Zhou, X. International Journal for Uncertainty Quantification, Vol. 11, Issue 4 https://doi.org/10.1615/Int.J.UncertaintyQuantification.2021034382	journal	January 2021
PySINDy: A comprehensive Python package for robust sparse system identification Kaptanoglu, Alan; de Silva, Brian; Fasel, Urban Journal of Open Source Software, Vol. 7, Issue 69 https://doi.org/10.21105/joss.03994	journal	January 2022
Estimation of ordinary differential equation parameters using constrained local polynomial regression Ding, A. Adam; Wu, Hulin Statistica Sinica https://doi.org/10.5705/ss.2012.304	journal	January 2014

Similar Records

PMC$_\infty$: Infinite-Order Scale-Setting method using the Principle of Maximum Conformality and preserving the Intrinsic Conformality

Journal Article · 2022 · SciPost Physics Proceedings · OSTI ID:1873211

Di Giustino, Leonardo; Brodsky, Stanley; Wu, Xing-Gang; +1 more

Hard exclusive $\pi^{+}n$ electro production beam spin asymmetries off the proton in the GPD and TDA regimes

Journal Article · 2022 · SciPost Physics Proceedings · OSTI ID:1875722

Diehl, Stefan

A dispersive analysis of $$\varvec{\eta '\rightarrow \pi ^+\pi ^-\gamma }$$ and $$\varvec{\eta '\rightarrow \ell ^+\ell ^-\gamma }$$

Journal Article · 2022 · European Physical Journal. C, Particles and Fields (Online) · OSTI ID:1867523

Holz, Simon; Hanhart, Christoph; Hoferichter, Martin; +1 more

Related Subjects

97 MATHEMATICS AND COMPUTING
data-driven modeling
parameter estimation
parameter inference
test functions
weak form

Title: Direct Estimation of Parameters in ODE Models Using WENDy: Weak-Form Estimation of Nonlinear Dynamics

Citation Formats

References (49)

Similar Records

Related Subjects