skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: The multi-assumption architecture and testbed (MAAT v1.0): R code for generating ensembles with dynamic model structure and analysis of epistemic uncertainty from multiple sources

Abstract

Abstract. Here, computer models are ubiquitous tools used to represent systems across many scientific and engineering domains. For any given system, many computer models exist, each built on different assumptions and demonstrating variability in the ways in which these systems can be represented. This variability is known as epistemic uncertainty, i.e. uncertainty in our knowledge of how these systems operate. Two primary sources of epistemic uncertainty are (1) uncertain parameter values and (2) uncertain mathematical representations of the processes that comprise the system. Many formal methods exist to analyse parameter-based epistemic uncertainty, while process-representation-based epistemic uncertainty is often analysed post hoc, incompletely, informally, or is ignored. In this model description paper we present the multi-assumption architecture and testbed (MAAT v1.0) designed to formally and completely analyse process-representation-based epistemic uncertainty. MAAT is a modular modelling code that can simply and efficiently vary model structure (process representation), allowing for the generation and running of large model ensembles that vary in process representation, parameters, parameter values, and environmental conditions during a single execution of the code. MAAT v1.0 approaches epistemic uncertainty through sensitivity analysis, assigning variability in model output to processes (process representation and parameters) or to individual parameters. In this model description paper wemore » describe MAAT and, by using a simple groundwater model example, verify that the sensitivity analysis algorithms have been correctly implemented. The main system model currently coded in MAAT is a unified, leaf-scale enzyme kinetic model of C 3 photosynthesis. In the Appendix we describe the photosynthesis model and the unification of multiple representations of photosynthetic processes. The numerical solution to leaf-scale photosynthesis is verified and examples of process variability in temperature response functions are provided. For rapid application to new systems, the MAAT algorithms for efficient variation of model structure and sensitivity analysis are agnostic of the specific system model employed. Therefore MAAT provides a tool for the development of novel or toy models in many domains, i.e. not only photosynthesis, facilitating rapid informal and formal comparison of alternative modelling approaches.« less

Authors:
ORCiD logo [1];  [2]; ORCiD logo [1]; ORCiD logo [3]; ORCiD logo [1];  [4]; ORCiD logo [5]; ORCiD logo [5]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  2. Florida State Univ., Tallahassee, FL (United States)
  3. Univ. of New South Wales, Sydney, NSW (Australia)
  4. Western Sydney Univ., Penrith, NSW (Australia)
  5. Brookhaven National Lab. (BNL), Upton, NY (United States)
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Brookhaven National Lab. (BNL), Upton, NY (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Biological and Environmental Research (BER) (SC-23)
OSTI Identifier:
1465033
Alternate Identifier(s):
OSTI ID: 1466608
Report Number(s):
[BNL-207962-2018-JAAM]
[Journal ID: ISSN 1991-9603]
Grant/Contract Number:  
[AC05-00OR22725; SC0012704]
Resource Type:
Accepted Manuscript
Journal Name:
Geoscientific Model Development (Online)
Additional Journal Information:
[Journal Name: Geoscientific Model Development (Online); Journal Volume: 11; Journal Issue: 8]; Journal ID: ISSN 1991-9603
Publisher:
European Geosciences Union
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; 54 ENVIRONMENTAL SCIENCES

Citation Formats

Walker, Anthony P., Ye, Ming, Lu, Dan, De Kauwe, Martin G., Gu, Lianhong, Medlyn, Belinda E., Rogers, Alistair, and Serbin, Shawn P. The multi-assumption architecture and testbed (MAAT v1.0): R code for generating ensembles with dynamic model structure and analysis of epistemic uncertainty from multiple sources. United States: N. p., 2018. Web. doi:10.5194/gmd-11-3159-2018.
Walker, Anthony P., Ye, Ming, Lu, Dan, De Kauwe, Martin G., Gu, Lianhong, Medlyn, Belinda E., Rogers, Alistair, & Serbin, Shawn P. The multi-assumption architecture and testbed (MAAT v1.0): R code for generating ensembles with dynamic model structure and analysis of epistemic uncertainty from multiple sources. United States. doi:10.5194/gmd-11-3159-2018.
Walker, Anthony P., Ye, Ming, Lu, Dan, De Kauwe, Martin G., Gu, Lianhong, Medlyn, Belinda E., Rogers, Alistair, and Serbin, Shawn P. Fri . "The multi-assumption architecture and testbed (MAAT v1.0): R code for generating ensembles with dynamic model structure and analysis of epistemic uncertainty from multiple sources". United States. doi:10.5194/gmd-11-3159-2018. https://www.osti.gov/servlets/purl/1465033.
@article{osti_1465033,
title = {The multi-assumption architecture and testbed (MAAT v1.0): R code for generating ensembles with dynamic model structure and analysis of epistemic uncertainty from multiple sources},
author = {Walker, Anthony P. and Ye, Ming and Lu, Dan and De Kauwe, Martin G. and Gu, Lianhong and Medlyn, Belinda E. and Rogers, Alistair and Serbin, Shawn P.},
abstractNote = {Abstract. Here, computer models are ubiquitous tools used to represent systems across many scientific and engineering domains. For any given system, many computer models exist, each built on different assumptions and demonstrating variability in the ways in which these systems can be represented. This variability is known as epistemic uncertainty, i.e. uncertainty in our knowledge of how these systems operate. Two primary sources of epistemic uncertainty are (1) uncertain parameter values and (2) uncertain mathematical representations of the processes that comprise the system. Many formal methods exist to analyse parameter-based epistemic uncertainty, while process-representation-based epistemic uncertainty is often analysed post hoc, incompletely, informally, or is ignored. In this model description paper we present the multi-assumption architecture and testbed (MAAT v1.0) designed to formally and completely analyse process-representation-based epistemic uncertainty. MAAT is a modular modelling code that can simply and efficiently vary model structure (process representation), allowing for the generation and running of large model ensembles that vary in process representation, parameters, parameter values, and environmental conditions during a single execution of the code. MAAT v1.0 approaches epistemic uncertainty through sensitivity analysis, assigning variability in model output to processes (process representation and parameters) or to individual parameters. In this model description paper we describe MAAT and, by using a simple groundwater model example, verify that the sensitivity analysis algorithms have been correctly implemented. The main system model currently coded in MAAT is a unified, leaf-scale enzyme kinetic model of C3 photosynthesis. In the Appendix we describe the photosynthesis model and the unification of multiple representations of photosynthetic processes. The numerical solution to leaf-scale photosynthesis is verified and examples of process variability in temperature response functions are provided. For rapid application to new systems, the MAAT algorithms for efficient variation of model structure and sensitivity analysis are agnostic of the specific system model employed. Therefore MAAT provides a tool for the development of novel or toy models in many domains, i.e. not only photosynthesis, facilitating rapid informal and formal comparison of alternative modelling approaches.},
doi = {10.5194/gmd-11-3159-2018},
journal = {Geoscientific Model Development (Online)},
number = [8],
volume = [11],
place = {United States},
year = {2018},
month = {8}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 1 work
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

A unified approach for process‐based hydrologic modeling: 1. Modeling concept
journal, April 2015

  • Clark, Martyn P.; Nijssen, Bart; Lundquist, Jessica D.
  • Water Resources Research, Vol. 51, Issue 4
  • DOI: 10.1002/2015WR017198

Forest water use and water use efficiency at elevated CO 2 : a model-data intercomparison at two contrasting temperate forest FACE sites
journal, March 2013

  • De Kauwe, Martin G.; Medlyn, Belinda E.; Zaehle, Sönke
  • Global Change Biology, Vol. 19, Issue 6
  • DOI: 10.1111/gcb.12164

Climate model genealogy: CLIMATE MODEL GENEALOGY
journal, April 2011


Using ecosystem experiments to improve vegetation models
journal, May 2015

  • Medlyn, Belinda E.; Zaehle, Sönke; De Kauwe, Martin G.
  • Nature Climate Change, Vol. 5, Issue 6
  • DOI: 10.1038/nclimate2621

A simulation model for the transient effects of climate change on forest landscapes
journal, January 1993


Resistances along the CO2 diffusion pathway inside leaves
journal, April 2009

  • Evans, J. R.; Kaldenhoff, R.; Genty, B.
  • Journal of Experimental Botany, Vol. 60, Issue 8
  • DOI: 10.1093/jxb/erp117

Tracking the origins of the Kok effect, 70 years after its discovery
journal, March 2017

  • Tcherkez, Guillaume; Gauthier, Paul; Buckley, Thomas N.
  • New Phytologist, Vol. 214, Issue 2
  • DOI: 10.1111/nph.14527

Improving canopy processes in the Community Land Model version 4 (CLM4) using global flux fields empirically inferred from FLUXNET data
journal, January 2011

  • Bonan, Gordon B.; Lawrence, Peter J.; Oleson, Keith W.
  • Journal of Geophysical Research, Vol. 116, Issue G2
  • DOI: 10.1029/2010JG001593

Optimal plant water economy: Optimal plant water economy
journal, October 2016

  • Buckley, Thomas N.; Sack, Lawren; Farquhar, Graham D.
  • Plant, Cell & Environment, Vol. 40, Issue 6
  • DOI: 10.1111/pce.12823

A framework for dealing with uncertainty due to model structure error
journal, November 2006


Optimum Aerodynamic Design Using the Navier-Stokes Equations
journal, January 1998

  • Jameson, A.; Martinelli, L.; Pierce, N. A.
  • Theoretical and Computational Fluid Dynamics, Vol. 10, Issue 1-4
  • DOI: 10.1007/s001620050060

Biophysical drivers of seasonal variability in Sphagnum gross primary production in a northern temperate bog
journal, May 2017

  • Walker, Anthony P.; Carter, Kelsey R.; Gu, Lianhong
  • Journal of Geophysical Research: Biogeosciences, Vol. 122, Issue 5
  • DOI: 10.1002/2016JG003711

The Influence of Light and Carbon Dioxide on Photosynthesis
journal, July 1937


Global sensitivity analysis in hydrological modeling: Review of concepts, methods, theoretical framework, and applications
journal, April 2015


Biochemical Limitations to Carbon Assimilation in C 3 Plants—A Retrospective Analysis of the A/C i Curves from 109 Species
journal, January 1993


Models of soil organic matter decomposition: the SoilR package, version 1.0
journal, January 2012

  • Sierra, C. A.; Müller, M.; Trumbore, S. E.
  • Geoscientific Model Development, Vol. 5, Issue 4
  • DOI: 10.5194/gmd-5-1045-2012

Modelling photosynthesis of cotton grown in elevated CO2
journal, April 1992


Photosynthesis and nitrogen relationships in leaves of C3 plants
journal, January 1989


Reconciling the optimal and empirical approaches to modelling stomatal conductance: RECONCILING OPTIMAL AND EMPIRICAL STOMATAL MODELS
journal, January 2011


Selecting a climate model subset to optimise key ensemble properties
journal, January 2018

  • Herger, Nadja; Abramowitz, Gab; Knutti, Reto
  • Earth System Dynamics, Vol. 9, Issue 1
  • DOI: 10.5194/esd-9-135-2018

A roadmap for improving the representation of photosynthesis in Earth system models
journal, November 2016

  • Rogers, Alistair; Medlyn, Belinda E.; Dukes, Jeffrey S.
  • New Phytologist, Vol. 213, Issue 1
  • DOI: 10.1111/nph.14283

A manifesto for the equifinality thesis
journal, March 2006


Managing complexity in simulations of land surface and near-surface processes
journal, April 2016


Challenges in Combining Projections from Multiple Climate Models
journal, May 2010

  • Knutti, Reto; Furrer, Reinhard; Tebaldi, Claudia
  • Journal of Climate, Vol. 23, Issue 10
  • DOI: 10.1175/2009JCLI3361.1

The use of the multi-model ensemble in probabilistic climate projections
journal, June 2007

  • Tebaldi, Claudia; Knutti, Reto
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 365, Issue 1857
  • DOI: 10.1098/rsta.2007.2076

Towards a comprehensive assessment of model structural adequacy: ASSESSMENT OF MODEL STRUCTURAL ADEQUACY
journal, August 2012

  • Gupta, Hoshin V.; Clark, Martyn P.; Vrugt, Jasper A.
  • Water Resources Research, Vol. 48, Issue 8
  • DOI: 10.1029/2011WR011044

GSSHA: Model To Simulate Diverse Stream Flow Producing Processes
journal, May 2004


Modelling respiration of vegetation: evidence for a general temperature-dependent Q10
journal, February 2001


The relationship of leaf photosynthetic traits - V cmax and J max - to leaf nitrogen, leaf phosphorus, and specific leaf area: a meta-analysis and modeling study
journal, July 2014

  • Walker, Anthony P.; Beckerman, Andrew P.; Gu, Lianhong
  • Ecology and Evolution, Vol. 4, Issue 16
  • DOI: 10.1002/ece3.1173

Analysis of variance designs for model output
journal, March 1999


Spatiotemporal patterns of terrestrial gross primary production: A review: GPP Spatiotemporal Patterns
journal, August 2015

  • Anav, Alessandro; Friedlingstein, Pierre; Beer, Christian
  • Reviews of Geophysics, Vol. 53, Issue 3
  • DOI: 10.1002/2015RG000483

Physiological and environmental regulation of stomatal conductance, photosynthesis and transpiration: a model that includes a laminar boundary layer
journal, April 1991

  • Collatz, G. James; Ball, J. Timothy; Grivet, Cyril
  • Agricultural and Forest Meteorology, Vol. 54, Issue 2-4
  • DOI: 10.1016/0168-1923(91)90002-8

A simple calibrated model of Amazon rainforest productivity based on leaf biochemical properties
journal, October 1995


Uncertainties in CMIP5 Climate Projections due to Carbon Cycle Feedbacks
journal, January 2014

  • Friedlingstein, Pierre; Meinshausen, Malte; Arora, Vivek K.
  • Journal of Climate, Vol. 27, Issue 2
  • DOI: 10.1175/JCLI-D-12-00579.1

The Coordination of Leaf Photosynthesis Links C and N Fluxes in C3 Plant Species
journal, June 2012


Optimal stomatal behavior with competition for water and risk of hydraulic impairment
journal, October 2016

  • Wolf, Adam; Anderegg, William R. L.; Pacala, Stephen W.
  • Proceedings of the National Academy of Sciences, Vol. 113, Issue 46
  • DOI: 10.1073/pnas.1615144113

Changes in the chloroplastic CO 2 concentration explain much of the observed Kok effect: a model
journal, March 2017

  • Farquhar, Graham D.; Busch, Florian A.
  • New Phytologist, Vol. 214, Issue 2
  • DOI: 10.1111/nph.14512

The ECMWF Ensemble Prediction System: Methodology and validation
journal, January 1996

  • Molteni, F.; Buizza, R.; Palmer, T. N.
  • Quarterly Journal of the Royal Meteorological Society, Vol. 122, Issue 529
  • DOI: 10.1002/qj.49712252905

Temperature response of parameters of a biochemically based model of photosynthesis. II. A review of experimental data
journal, September 2002


Temperature responses of mesophyll conductance differ greatly between species: Temperature responses of mesophyll conductance
journal, October 2014

  • von CAEMMERER, Susanne; Evans, John R.
  • Plant, Cell & Environment, Vol. 38, Issue 4
  • DOI: 10.1111/pce.12449

Effects of parameter uncertainties on the modeling of terrestrial biosphere dynamics: PARAMETER-BASED UNCERTAINTY OF A DGVM
journal, September 2005

  • Zaehle, S.; Sitch, S.; Smith, B.
  • Global Biogeochemical Cycles, Vol. 19, Issue 3
  • DOI: 10.1029/2004GB002395

Coupled response of stomatal and mesophyll conductance to light enhances photosynthesis of shade leaves under sunflecks: Mesophyll conductance response to light
journal, November 2016

  • Campany, Courtney E.; Tjoelker, Mark G.; von Caemmerer, Susanne
  • Plant, Cell & Environment, Vol. 39, Issue 12
  • DOI: 10.1111/pce.12841

The Twentieth Century Reanalysis Project
journal, January 2011

  • Compo, G. P.; Whitaker, J. S.; Sardeshmukh, P. D.
  • Quarterly Journal of the Royal Meteorological Society, Vol. 137, Issue 654
  • DOI: 10.1002/qj.776

Modeling for Understanding v. Modeling for Numbers
journal, November 2016


Global climatic drivers of leaf size
journal, August 2017


An Empirical Model of Stomatal Conductance
journal, January 1984

  • Farquhar, Gd; Wong, Sc
  • Functional Plant Biology, Vol. 11, Issue 3
  • DOI: 10.1071/PP9840191

C3 and C4 photosynthesis models: An overview from the perspective of crop modelling
journal, December 2009


Reversible jump Markov chain Monte Carlo computation and Bayesian model determination
journal, January 1995


Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index
journal, February 2010

  • Saltelli, Andrea; Annoni, Paola; Azzini, Ivano
  • Computer Physics Communications, Vol. 181, Issue 2
  • DOI: 10.1016/j.cpc.2009.09.018

Equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling?
journal, October 2008

  • Vrugt, Jasper A.; ter Braak, Cajo J. F.; Gupta, Hoshin V.
  • Stochastic Environmental Research and Risk Assessment, Vol. 23, Issue 7
  • DOI: 10.1007/s00477-008-0274-y

Mesophyll conductance in Zea mays responds transiently to CO 2 availability: implications for transpiration efficiency in C 4 crops
journal, December 2017

  • Kolbe, Allison R.; Cousins, Asaph B.
  • New Phytologist, Vol. 217, Issue 4
  • DOI: 10.1111/nph.14942

The impact of alternative trait-scaling hypotheses for the maximum photosynthetic carboxylation rate ( V cmax ) on global gross primary production
journal, June 2017

  • Walker, Anthony P.; Quaife, Tristan; van Bodegom, Peter M.
  • New Phytologist, Vol. 215, Issue 4
  • DOI: 10.1111/nph.14623

Temperature acclimation in a biochemical model of photosynthesis: a reanalysis of data from 36 species
journal, September 2007


A global scale mechanistic model of photosynthetic capacity (LUNA V1.0)
journal, January 2016


A biochemical model of photosynthetic CO2 assimilation in leaves of C3 species
journal, June 1980

  • Farquhar, G. D.; von Caemmerer, S.; Berry, J. A.
  • Planta, Vol. 149, Issue 1
  • DOI: 10.1007/BF00386231

Uncertainty in the environmental modelling process – A framework and guidance
journal, November 2007

  • Refsgaard, Jens Christian; van der Sluijs, Jeroen P.; Højberg, Anker Lajer
  • Environmental Modelling & Software, Vol. 22, Issue 11
  • DOI: 10.1016/j.envsoft.2007.02.004

Improved temperature response functions for models of Rubisco-limited photosynthesis
journal, February 2001


A comparison of three different canopy radiation models commonly used in plant modelling
journal, January 2003

  • Wang, Ying Ping
  • Functional Plant Biology, Vol. 30, Issue 2
  • DOI: 10.1071/FP02117

Long-Term Response of Nutrient-Limited Forests to CO"2 Enrichment; Equilibrium Behavior of Plant-Soil Models
journal, November 1993

  • Comins, H. N.; McMurtrie, R. E.
  • Ecological Applications, Vol. 3, Issue 4
  • DOI: 10.2307/1942099