skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: The multi-assumption architecture and testbed (MAAT v1.0): R code for generating ensembles with dynamic model structure and analysis of epistemic uncertainty from multiple sources

Abstract

Abstract. Computer models are ubiquitous tools used to represent systems across many scientific and engineering domains. For any given system, many computer models exist, each built on different assumptions and demonstrating variability in the ways in which these systems can be represented. This variability is known as epistemic uncertainty, i.e. uncertainty in our knowledge of how these systems operate. Two primary sources of epistemic uncertainty are (1) uncertain parameter values and (2) uncertain mathematical representations of the processes that comprise the system. Many formal methods exist to analyse parameter-based epistemic uncertainty, while process-representation-based epistemic uncertainty is often analysed post hoc, incompletely, informally, or is ignored. In this model description paper we present the multi-assumption architecture and testbed (MAAT v1.0) designed to formally and completely analyse process-representation-based epistemic uncertainty. MAAT is a modular modelling code that can simply and efficiently vary model structure (process representation), allowing for the generation and running of large model ensembles that vary in process representation, parameters, parameter values, and environmental conditions during a single execution of the code. MAAT v1.0 approaches epistemic uncertainty through sensitivity analysis, assigning variability in model output to processes (process representation and parameters) or to individual parameters. In this model description paper we describemore » MAAT and, by using a simple groundwater model example, verify that the sensitivity analysis algorithms have been correctly implemented. The main system model currently coded in MAAT is a unified, leaf-scale enzyme kinetic model of C3 photosynthesis. In the Appendix we describe the photosynthesis model and the unification of multiple representations of photosynthetic processes. The numerical solution to leaf-scale photosynthesis is verified and examples of process variability in temperature response functions are provided. For rapid application to new systems, the MAAT algorithms for efficient variation of model structure and sensitivity analysis are agnostic of the specific system model employed. Therefore MAAT provides a tool for the development of novel or toy models in many domains, i.e. not only photosynthesis, facilitating rapid informal and formal comparison of alternative modelling approaches.« less

Authors:
ORCiD logo [1];  [2]; ORCiD logo [1]; ORCiD logo [3]; ORCiD logo [1];  [4]; ORCiD logo [5]; ORCiD logo [5]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  2. Florida State Univ., Tallahassee, FL (United States)
  3. Univ. of New South Wales, Sydney, NSW (Australia)
  4. Western Sydney Univ., Penrith, NSW (Australia)
  5. Brookhaven National Lab. (BNL), Upton, NY (United States)
Publication Date:
Research Org.:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States); Brookhaven National Laboratory (BNL), Upton, NY (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Biological and Environmental Research (BER)
OSTI Identifier:
1465033
Alternate Identifier(s):
OSTI ID: 1466608
Report Number(s):
BNL-207962-2018-JAAM
Journal ID: ISSN 1991-9603
Grant/Contract Number:  
AC05-00OR22725; SC0012704
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Geoscientific Model Development (Online)
Additional Journal Information:
Journal Volume: 11; Journal Issue: 8; Journal ID: ISSN 1991-9603
Publisher:
European Geosciences Union
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; 54 ENVIRONMENTAL SCIENCES

Citation Formats

Walker, Anthony P., Ye, Ming, Lu, Dan, De Kauwe, Martin G., Gu, Lianhong, Medlyn, Belinda E., Rogers, Alistair, and Serbin, Shawn P. The multi-assumption architecture and testbed (MAAT v1.0): R code for generating ensembles with dynamic model structure and analysis of epistemic uncertainty from multiple sources. United States: N. p., 2018. Web. doi:10.5194/gmd-11-3159-2018.
Walker, Anthony P., Ye, Ming, Lu, Dan, De Kauwe, Martin G., Gu, Lianhong, Medlyn, Belinda E., Rogers, Alistair, & Serbin, Shawn P. The multi-assumption architecture and testbed (MAAT v1.0): R code for generating ensembles with dynamic model structure and analysis of epistemic uncertainty from multiple sources. United States. https://doi.org/10.5194/gmd-11-3159-2018
Walker, Anthony P., Ye, Ming, Lu, Dan, De Kauwe, Martin G., Gu, Lianhong, Medlyn, Belinda E., Rogers, Alistair, and Serbin, Shawn P. 2018. "The multi-assumption architecture and testbed (MAAT v1.0): R code for generating ensembles with dynamic model structure and analysis of epistemic uncertainty from multiple sources". United States. https://doi.org/10.5194/gmd-11-3159-2018. https://www.osti.gov/servlets/purl/1465033.
@article{osti_1465033,
title = {The multi-assumption architecture and testbed (MAAT v1.0): R code for generating ensembles with dynamic model structure and analysis of epistemic uncertainty from multiple sources},
author = {Walker, Anthony P. and Ye, Ming and Lu, Dan and De Kauwe, Martin G. and Gu, Lianhong and Medlyn, Belinda E. and Rogers, Alistair and Serbin, Shawn P.},
abstractNote = {Abstract. Computer models are ubiquitous tools used to represent systems across many scientific and engineering domains. For any given system, many computer models exist, each built on different assumptions and demonstrating variability in the ways in which these systems can be represented. This variability is known as epistemic uncertainty, i.e. uncertainty in our knowledge of how these systems operate. Two primary sources of epistemic uncertainty are (1) uncertain parameter values and (2) uncertain mathematical representations of the processes that comprise the system. Many formal methods exist to analyse parameter-based epistemic uncertainty, while process-representation-based epistemic uncertainty is often analysed post hoc, incompletely, informally, or is ignored. In this model description paper we present the multi-assumption architecture and testbed (MAAT v1.0) designed to formally and completely analyse process-representation-based epistemic uncertainty. MAAT is a modular modelling code that can simply and efficiently vary model structure (process representation), allowing for the generation and running of large model ensembles that vary in process representation, parameters, parameter values, and environmental conditions during a single execution of the code. MAAT v1.0 approaches epistemic uncertainty through sensitivity analysis, assigning variability in model output to processes (process representation and parameters) or to individual parameters. In this model description paper we describe MAAT and, by using a simple groundwater model example, verify that the sensitivity analysis algorithms have been correctly implemented. The main system model currently coded in MAAT is a unified, leaf-scale enzyme kinetic model of C3 photosynthesis. In the Appendix we describe the photosynthesis model and the unification of multiple representations of photosynthetic processes. The numerical solution to leaf-scale photosynthesis is verified and examples of process variability in temperature response functions are provided. For rapid application to new systems, the MAAT algorithms for efficient variation of model structure and sensitivity analysis are agnostic of the specific system model employed. Therefore MAAT provides a tool for the development of novel or toy models in many domains, i.e. not only photosynthesis, facilitating rapid informal and formal comparison of alternative modelling approaches.},
doi = {10.5194/gmd-11-3159-2018},
url = {https://www.osti.gov/biblio/1465033}, journal = {Geoscientific Model Development (Online)},
issn = {1991-9603},
number = 8,
volume = 11,
place = {United States},
year = {Fri Aug 10 00:00:00 EDT 2018},
month = {Fri Aug 10 00:00:00 EDT 2018}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 11 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

A unified approach for process‐based hydrologic modeling: 1. Modeling concept
journal, April 2015


Forest water use and water use efficiency at elevated CO 2 : a model-data intercomparison at two contrasting temperate forest FACE sites
journal, March 2013


Climate model genealogy: CLIMATE MODEL GENEALOGY
journal, April 2011


Using ecosystem experiments to improve vegetation models
journal, May 2015


A simulation model for the transient effects of climate change on forest landscapes
journal, January 1993


Resistances along the CO2 diffusion pathway inside leaves
journal, April 2009


Tracking the origins of the Kok effect, 70 years after its discovery
journal, March 2017


Improving canopy processes in the Community Land Model version 4 (CLM4) using global flux fields empirically inferred from FLUXNET data
journal, January 2011


Optimal plant water economy: Optimal plant water economy
journal, October 2016


A framework for dealing with uncertainty due to model structure error
journal, November 2006


Optimum Aerodynamic Design Using the Navier-Stokes Equations
journal, January 1998


Biophysical drivers of seasonal variability in Sphagnum gross primary production in a northern temperate bog
journal, May 2017


The Influence of Light and Carbon Dioxide on Photosynthesis
journal, July 1937


Models of soil organic matter decomposition: the SoilR package, version 1.0
journal, January 2012


Modelling photosynthesis of cotton grown in elevated CO2
journal, April 1992


Photosynthesis and nitrogen relationships in leaves of C3 plants
journal, January 1989


Selecting a climate model subset to optimise key ensemble properties
journal, January 2018


A roadmap for improving the representation of photosynthesis in Earth system models
journal, November 2016


A manifesto for the equifinality thesis
journal, March 2006


Managing complexity in simulations of land surface and near-surface processes
journal, April 2016


Challenges in Combining Projections from Multiple Climate Models
journal, May 2010


The use of the multi-model ensemble in probabilistic climate projections
journal, June 2007

  • Tebaldi, Claudia; Knutti, Reto
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 365, Issue 1857
  • https://doi.org/10.1098/rsta.2007.2076

Towards a comprehensive assessment of model structural adequacy: ASSESSMENT OF MODEL STRUCTURAL ADEQUACY
journal, August 2012


GSSHA: Model To Simulate Diverse Stream Flow Producing Processes
journal, May 2004


Modelling respiration of vegetation: evidence for a general temperature-dependent Q10
journal, February 2001


Plant responses to increasing CO 2 reduce estimates of climate impacts on drought severity
journal, August 2016


Analysis of variance designs for model output
journal, March 1999


Spatiotemporal patterns of terrestrial gross primary production: A review: GPP Spatiotemporal Patterns
journal, August 2015


Physiological and environmental regulation of stomatal conductance, photosynthesis and transpiration: a model that includes a laminar boundary layer
journal, April 1991


A simple calibrated model of Amazon rainforest productivity based on leaf biochemical properties
journal, October 1995


Uncertainties in CMIP5 Climate Projections due to Carbon Cycle Feedbacks
journal, January 2014


The Coordination of Leaf Photosynthesis Links C and N Fluxes in C3 Plant Species
journal, June 2012


Optimal stomatal behavior with competition for water and risk of hydraulic impairment
journal, October 2016


Changes in the chloroplastic CO 2 concentration explain much of the observed Kok effect: a model
journal, March 2017


The ECMWF Ensemble Prediction System: Methodology and validation
journal, January 1996


Temperature response of parameters of a biochemically based model of photosynthesis. II. A review of experimental data
journal, September 2002


Temperature responses of mesophyll conductance differ greatly between species: Temperature responses of mesophyll conductance
journal, October 2014


Effects of parameter uncertainties on the modeling of terrestrial biosphere dynamics: PARAMETER-BASED UNCERTAINTY OF A DGVM
journal, September 2005


Coupled response of stomatal and mesophyll conductance to light enhances photosynthesis of shade leaves under sunflecks: Mesophyll conductance response to light
journal, November 2016


The Twentieth Century Reanalysis Project
journal, January 2011

  • Compo, G. P.; Whitaker, J. S.; Sardeshmukh, P. D.
  • Quarterly Journal of the Royal Meteorological Society, Vol. 137, Issue 654
  • https://doi.org/10.1002/qj.776

Modeling for Understanding v. Modeling for Numbers
journal, November 2016


Global climatic drivers of leaf size
journal, August 2017


An Empirical Model of Stomatal Conductance
journal, January 1984


C3 and C4 photosynthesis models: An overview from the perspective of crop modelling
journal, December 2009


Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index
journal, February 2010


Equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling?
journal, October 2008


Temperature acclimation in a biochemical model of photosynthesis: a reanalysis of data from 36 species
journal, September 2007


A global scale mechanistic model of photosynthetic capacity (LUNA V1.0)
journal, January 2016


A biochemical model of photosynthetic CO2 assimilation in leaves of C3 species
journal, June 1980


Uncertainty in the environmental modelling process – A framework and guidance
journal, November 2007


Improved temperature response functions for models of Rubisco-limited photosynthesis
journal, February 2001


A comparison of three different canopy radiation models commonly used in plant modelling
journal, January 2003


Long-Term Response of Nutrient-Limited Forests to CO"2 Enrichment; Equilibrium Behavior of Plant-Soil Models
journal, November 1993


The ECMWF Ensemble Prediction System: Methodology and validation
journal, January 1996


Coordination theory of leaf nitrogen distribution in a canopy
journal, February 1993


A canopy conductance and photosynthesis model for use in a GCM land surface scheme
journal, December 1998


Modelling Stomatal Behaviour and and Photosynthesis of Eucalyptus grandis
journal, January 1990


Reconciling the optimal and empirical approaches to modelling stomatal conductance
journal, October 2012


The Twentieth Century Reanalysis Project
text, January 2011


The ECMWF Ensemble Prediction System - methodology and validation
text, January 1994


Challenges in combining projections from multiple climate models
text, January 2010


Selecting a climate model subset to optimise key ensemble properties
text, January 2018


A framework for dealing with uncertainty due to model structure error
journal, November 2006


Uncertainty in the environmental modelling process – A framework and guidance
journal, November 2007


C3 and C4 photosynthesis models: An overview from the perspective of crop modelling
journal, December 2009


Tracking the origins of the Kok effect, 70 years after its discovery
journal, March 2017


The Coordination of Leaf Photosynthesis Links C and N Fluxes in C3 Plant Species
journal, June 2012


The ECMWF Ensemble Prediction System - methodology and validation
text, January 1994


A global scale mechanistic model of photosynthetic capacity (LUNA V1.0)
journal, January 2016


Works referencing / citing this record:

Advancing global change biology through experimental manipulations: Where have we been and where might we go?
journal, November 2019


Linking big models to big data: efficient ecosystem model calibration through Bayesian model emulation
journal, January 2018


Advancing global change biology through experimental manipulations: Where have we been and where might we go?
journal, November 2019