Data-model integration plays a critical role in assessing and improving our capacity to predict ecosystem dynamics. Similarly, the ability to attach quantitative statements of uncertainty around model forecasts is crucial for model assessment and interpretation and for setting field research priorities. Bayesian methods provide a rigorous data assimilation framework for these applications, especially for problems with multiple data constraints. However, the Markov chain Monte Carlo (MCMC) techniques underlying most Bayesian calibration can be prohibitive for computationally demanding models and large datasets. We employ an alternative method, Bayesian model emulation of sufficient statistics, that can approximate the full joint posterior density, is more amenable to parallelization, and provides an estimate of parameter sensitivity. Analysis involved informative priors constructed from a meta-analysis of the primary literature and specification of both model and data uncertainties, and it introduced novel approaches to autocorrelation corrections on multiple data streams and emulating the sufficient statistics surface. We report the integration of this method within an ecological workflow management software, Predictive Ecosystem Analyzer (PEcAn), and its application and validation with two process-based terrestrial ecosystem models: SIPNET and ED2. In a test against a synthetic dataset, the emulator was able to retrieve the true parameter values. A comparison of the emulator approach to standard brute-force MCMC involving multiple data constraints showed that the emulator method was able to constrain the faster and simpler SIPNET model's parameters with comparable performance to the brute-force approach but reduced computation time by more than 2 orders of magnitude. The emulator was then applied to calibration of the ED2 model, whose complexity precludes standard (brute-force) Bayesian data assimilation techniques. Both models are constrained after assimilation of the observational data with the emulator method, reducing the uncertainty around their predictions. Performance metrics showed increased agreement between model predictions and data. Our study furthers efforts toward reducing model uncertainties, showing that the emulator method makes it possible to efficiently calibrate complex models.
Fer, Istem, et al. "Linking big models to big data: efficient ecosystem model calibration through Bayesian model emulation." Biogeosciences (Online), vol. 15, no. 19, Oct. 2018. https://doi.org/10.5194/bg-15-5801-2018
Fer, Istem, Kelly, Ryan, Moorcroft, Paul R., Richardson, Andrew D., Cowdery, Elizabeth M., & Dietze, Michael C. (2018). Linking big models to big data: efficient ecosystem model calibration through Bayesian model emulation. Biogeosciences (Online), 15(19). https://doi.org/10.5194/bg-15-5801-2018
Fer, Istem, Kelly, Ryan, Moorcroft, Paul R., et al., "Linking big models to big data: efficient ecosystem model calibration through Bayesian model emulation," Biogeosciences (Online) 15, no. 19 (2018), https://doi.org/10.5194/bg-15-5801-2018
@article{osti_1483479,
author = {Fer, Istem and Kelly, Ryan and Moorcroft, Paul R. and Richardson, Andrew D. and Cowdery, Elizabeth M. and Dietze, Michael C.},
title = {Linking big models to big data: efficient ecosystem model calibration through Bayesian model emulation},
annote = {Data-model integration plays a critical role in assessing and improving our capacity to predict ecosystem dynamics. Similarly, the ability to attach quantitative statements of uncertainty around model forecasts is crucial for model assessment and interpretation and for setting field research priorities. Bayesian methods provide a rigorous data assimilation framework for these applications, especially for problems with multiple data constraints. However, the Markov chain Monte Carlo (MCMC) techniques underlying most Bayesian calibration can be prohibitive for computationally demanding models and large datasets. We employ an alternative method, Bayesian model emulation of sufficient statistics, that can approximate the full joint posterior density, is more amenable to parallelization, and provides an estimate of parameter sensitivity. Analysis involved informative priors constructed from a meta-analysis of the primary literature and specification of both model and data uncertainties, and it introduced novel approaches to autocorrelation corrections on multiple data streams and emulating the sufficient statistics surface. We report the integration of this method within an ecological workflow management software, Predictive Ecosystem Analyzer (PEcAn), and its application and validation with two process-based terrestrial ecosystem models: SIPNET and ED2. In a test against a synthetic dataset, the emulator was able to retrieve the true parameter values. A comparison of the emulator approach to standard brute-force MCMC involving multiple data constraints showed that the emulator method was able to constrain the faster and simpler SIPNET model's parameters with comparable performance to the brute-force approach but reduced computation time by more than 2 orders of magnitude. The emulator was then applied to calibration of the ED2 model, whose complexity precludes standard (brute-force) Bayesian data assimilation techniques. Both models are constrained after assimilation of the observational data with the emulator method, reducing the uncertainty around their predictions. Performance metrics showed increased agreement between model predictions and data. Our study furthers efforts toward reducing model uncertainties, showing that the emulator method makes it possible to efficiently calibrate complex models.},
doi = {10.5194/bg-15-5801-2018},
url = {https://www.osti.gov/biblio/1483479},
journal = {Biogeosciences (Online)},
issn = {ISSN 1726-4189},
number = {19},
volume = {15},
place = {United States},
publisher = {European Geosciences Union},
year = {2018},
month = {10}}
Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 222, Issue 594-604https://doi.org/10.1098/rsta.1922.0009