skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Practical Use of Computationally Frugal Model Analysis Methods

Journal Article · · Ground Water
DOI:https://doi.org/10.1111/gwat.12330· OSTI ID:1286771
 [1];  [2];  [3];  [4];  [5];  [6];  [7];  [8]
  1. U.S. Geological Survey, Boulder, CO (United States); Univ. of Kansas, Lawrence, KS (United States)
  2. Univ. of Adelaide, SA (Australia)
  3. National Center for Atmospheric Research, Boulder, CO (United States)
  4. Florida State Univ., Tallahassee, FL (United States)
  5. Colorado State Univ., Fort Collins, CO (United States)
  6. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  7. Univ. of Darmstadt (Germany)
  8. California State Univ. (CalState), Chico, CA (United States)
+ Show Author Affiliations

Abstract Three challenges compromise the utility of mathematical models of groundwater and other environmental systems: (1) a dizzying array of model analysis methods and metrics make it difficult to compare evaluations of model adequacy, sensitivity, and uncertainty; (2) the high computational demands of many popular model analysis methods (requiring 1000's, 10,000 s, or more model runs) make them difficult to apply to complex models; and (3) many models are plagued by unrealistic nonlinearities arising from the numerical model formulation and implementation. This study proposes a strategy to address these challenges through a careful combination of model analysis and implementation methods. In this strategy, computationally frugal model analysis methods (often requiring a few dozen parallelizable model runs) play a major role, and computationally demanding methods are used for problems where (relatively) inexpensive diagnostics suggest the frugal methods are unreliable. We also argue in favor of detecting and, where possible, eliminating unrealistic model nonlinearities—this increases the realism of the model itself and facilitates the application of frugal methods. Literature examples are used to demonstrate the use of frugal methods and associated diagnostics. We suggest that the strategy proposed in this paper would allow the environmental sciences community to achieve greater transparency and falsifiability of environmental models, and obtain greater scientific insight from ongoing and future modeling efforts.

Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
USDOE Laboratory Directed Research and Development (LDRD) Program
Grant/Contract Number:
AC05-00OR22725; SC0008272; 0911074; 21-66885
OSTI ID:
1286771
Alternate ID(s):
OSTI ID: 1400448
Journal Information:
Ground Water, Vol. 54, Issue 2; ISSN 0017-467X
Publisher:
Wiley - NGWACopyright Statement
Country of Publication:
United States
Language:
English
Citation Metrics:
Cited by: 48 works
Citation information provided by
Web of Science

References (48)

Exploring Regression Structure Using Nonparametric Functional Estimation journal September 1993
Uncertainty in science and its role in climate policy
  • Smith, Leonard A.; Stern, Nicholas
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 369, Issue 1956 https://doi.org/10.1098/rsta.2011.0149
journal December 2011
Hyperresolution global land surface modeling: Meeting a grand challenge for monitoring Earth's terrestrial water: OPINION journal May 2011
Testing Alternative Ground Water Models Using Cross-Validation and Other Methods journal September 2007
Two statistics for evaluating parameter identifiability and error reduction journal March 2009
Technical note: Method of Morris effectively reduces the computational demands of global sensitivity analysis for distributed watershed models journal January 2013
A short exploration of structural noise: A SHORT EXPLORATION OF STRUCTURAL NOISE journal May 2010
Comment on “Two statistics for evaluating parameter identifiability and error reduction” by John Doherty and Randall J. Hunt journal January 2010
Pursuing the method of multiple working hypotheses for hydrological modeling: HYPOTHESIS TESTING IN HYDROLOGY journal September 2011
Quantifying Data Worth Toward Reducing Predictive Uncertainty journal August 2010
Response to Comment on “Two statistics for evaluating parameter identifiability and error reduction” journal January 2010
Ancient numerical daemons of conceptual hydrological modeling: 2. Impact of time stepping schemes on model analysis and prediction journal October 2010
Measuring Uncertainty Importance: Investigation and Comparison of Alternative Approaches journal October 2006
Effective Groundwater Model Calibration book January 2007
Review of surrogate modeling in water resources: REVIEW journal July 2012
Estimation of Aquifer Parameters Under Transient and Steady State Conditions: 1. Maximum Likelihood Method Incorporating Prior Information journal February 1986
Simulation of Water-Table Aquifers Using Specified Saturated Thickness journal February 2014
Analysis of regression confidence intervals and Bayesian credible intervals for uncertainty quantification: REGRESSION CONFIDENCE AND BAYESIAN CREDIBLE INTERVALS journal September 2012
Parameter and observation importance in modelling virus transport in saturated porous media—investigations in a homogenous system journal November 2005
Ignorance is bliss: Or seven reasons not to use uncertainty analysis: OPINION journal May 2006
Decision tree for choosing an uncertainty analysis methodology: a wiki experiment journal January 2006
Towards a comprehensive assessment of model structural adequacy: ASSESSMENT OF MODEL STRUCTURAL ADEQUACY journal August 2012
Sensitivity analysis, calibration, and testing of a distributed hydrological model using error-based weighting and one objective function: HYDROLOGICAL MODEL CALIBRATION journal June 2009
Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences journal February 1994
Beyond optimality: Multistakeholder robustness tradeoffs for regional water portfolio planning under deep uncertainty journal October 2014
Flood forecasting using a fully distributed model: application of the TOPKAPI model to the Upper Xixian Catchment journal January 2005
Evaluating model structure adequacy: The case of the Maggia Valley groundwater system, southern Switzerland: Evaluating Model Structure Adequacy journal January 2013
Bees, lies and evidence-based policy journal February 2013
Assessment and management of risk in subsurface hydrology: A review and perspective journal January 2013
Probabilistic sensitivity analysis of complex models: a Bayesian approach journal August 2004
Are Models Too Simple? Arguments for Increased Parameterization journal May 2007
Assessing parameter importance of the Common Land Model based on qualitative and quantitative sensitivity analysis journal January 2013
Evolutionary multiobjective optimization in water resources: The past, present, and future journal January 2013
Methods for using groundwater model predictions to guide hydrogeologic data collection, with application to the Death Valley regional groundwater flow system: USING MODEL PREDICTIONS TO GUIDE DATA COLLECTION journal January 2003
Nonlinear sensitivity analysis of multiparameter model systems journal January 1978
Comparing sensitivity analysis methods to advance lumped watershed model identification and evaluation journal January 2007
Sensitivity analysis of simulation models: an overview journal January 2010
A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output journal February 1999
Knowledge, transparency, and refutability in groundwater models, an example from the Death Valley regional groundwater flow system journal January 2013
Model smoothing strategies to remove microscale discontinuities and spurious secondary optima in objective functions in hydrological calibration: MODEL SMOOTHING IN HYDROLOGY journal March 2007
Reducing the computational cost of automatic calibration through model preemption: MODEL PREEMPTION APPROACH IN AUTOMATIC CALIBRATION journal November 2010
Distributed Evaluation of Local Sensitivity Analysis (DELSA), with application to hydrologic models: Distributed evaluation of local sensitivity analysis journal January 2014
A hybrid regularized inversion methodology for highly parameterized environmental models: HYBRID REGULARIZATION METHODOLOGY journal October 2005
Two-Dimensional Advective Transport in Ground-Water Flow Parameter Estimation journal November 1996
Effective and efficient global optimization for conceptual rainfall-runoff models journal April 1992
Time-varying sensitivity analysis clarifies the effects of watershed model formulation on model behavior: TIME-VARYING SENSITIVITY OF WATERSHED MODELS journal March 2013
The Logic of Scientific Discovery journal November 1959
OPR-PPR, a computer program for assessing data importance to model predictions using linear statistics report January 2007

Cited By (9)

Why Should Practitioners be Concerned about Predictive Uncertainty of Groundwater Management Models? journal October 2016
Terrainbento 1.0: a Python package for multi-model analysis in long-term drainage basin evolution journal January 2019
Reconnecting Stochastic Methods With Hydrogeological Applications: A Utilitarian Uncertainty Analysis and Risk Assessment Approach for the Design of Optimal Monitoring Networks journal March 2018
The evolution of process-based hydrologic models: historical challenges and the collective quest for physical realism journal January 2017
Characterizing and reducing equifinality by constraining a distributed catchment model with regional signatures, local observations, and process understanding journal January 2017
Sensitivity Analysis and Calibration of an Integrated Hydrologic Model in an Irrigated Agricultural Basin With a Groundwater‐Dependent Ecosystem journal September 2019
Comparison of Newton-type and SCE optimisation algorithms for the calibration of conceptual hydrological models journal July 2016
A Robust Gauss‐Newton Algorithm for the Optimization of Hydrological Models: From Standard Gauss‐Newton to Robust Gauss‐Newton journal November 2018
terrainbento 1.0: a Python package for multi-model analysis in long-term drainage basin evolution posted_content October 2018

Similar Records

Related Subjects

97 MATHEMATICS AND COMPUTING
54 ENVIRONMENTAL SCIENCES
Model calibration
Sensitivity analysis
Uncertainty
Bayesian
regression
transparency
falsifiability