Skip to main content
U.S. Department of Energy
Office of Scientific and Technical Information

Practical Use of Computationally Frugal Model Analysis Methods

Journal Article · · Ground Water
DOI:https://doi.org/10.1111/gwat.12330· OSTI ID:1286771
 [1];  [2];  [3];  [4];  [5];  [6];  [7];  [8]
  1. U.S. Geological Survey, Boulder, CO (United States); Univ. of Kansas, Lawrence, KS (United States)
  2. Univ. of Adelaide, SA (Australia)
  3. National Center for Atmospheric Research, Boulder, CO (United States)
  4. Florida State Univ., Tallahassee, FL (United States)
  5. Colorado State Univ., Fort Collins, CO (United States)
  6. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  7. Univ. of Darmstadt (Germany)
  8. California State Univ. (CalState), Chico, CA (United States)

Computationally frugal methods of model analysis can provide substantial benefits when developing models of groundwater and other environmental systems. Model analysis includes ways to evaluate model adequacy and to perform sensitivity and uncertainty analysis. Frugal methods typically require 10s of parallelizable model runs; their convenience allows for other uses of the computational effort. We suggest that model analysis be posed as a set of questions used to organize methods that range from frugal to expensive (requiring 10,000 model runs or more). This encourages focus on method utility, even when methods have starkly different theoretical backgrounds. We note that many frugal methods are more useful when unrealistic process-model nonlinearities are reduced. Inexpensive diagnostics are identified for determining when frugal methods are advantageous. Examples from the literature are used to demonstrate local methods and the diagnostics. We suggest that the greater use of computationally frugal model analysis methods would allow questions such as those posed in this work to be addressed more routinely, allowing the environmental sciences community to obtain greater scientific insight from the many ongoing and future modeling efforts

Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
USDOE; ORNL LDRD Director's R&D
Grant/Contract Number:
AC05-00OR22725; SC0008272
OSTI ID:
1286771
Alternate ID(s):
OSTI ID: 1400448
Journal Information:
Ground Water, Journal Name: Ground Water Journal Issue: 2 Vol. 54; ISSN 0017-467X
Publisher:
Wiley - NGWACopyright Statement
Country of Publication:
United States
Language:
English

References (49)

Effective Groundwater Model Calibration book January 2007
Distributed Evaluation of Local Sensitivity Analysis (DELSA), with application to hydrologic models: Distributed evaluation of local sensitivity analysis journal January 2014
Beyond optimality: Multistakeholder robustness tradeoffs for regional water portfolio planning under deep uncertainty journal October 2014
Applied Regression Analysis book January 1998
Decision tree for choosing an uncertainty analysis methodology: a wiki experiment journal January 2006
Time-varying sensitivity analysis clarifies the effects of watershed model formulation on model behavior: TIME-VARYING SENSITIVITY OF WATERSHED MODELS journal March 2013
Nonlinear sensitivity analysis of multiparameter model systems journal January 1978
Evolutionary multiobjective optimization in water resources: The past, present, and future journal January 2013
Assessment and management of risk in subsurface hydrology: A review and perspective journal January 2013
Parameter and observation importance in modelling virus transport in saturated porous media—investigations in a homogenous system journal November 2005
Two statistics for evaluating parameter identifiability and error reduction journal March 2009
Comment on “Two statistics for evaluating parameter identifiability and error reduction” by John Doherty and Randall J. Hunt journal January 2010
Response to Comment on “Two statistics for evaluating parameter identifiability and error reduction” journal January 2010
Knowledge, transparency, and refutability in groundwater models, an example from the Death Valley regional groundwater flow system journal January 2013
Sensitivity analysis of simulation models: an overview journal January 2010
Methods for using groundwater model predictions to guide hydrogeologic data collection, with application to the Death Valley regional groundwater flow system: USING MODEL PREDICTIONS TO GUIDE DATA COLLECTION journal January 2003
A hybrid regularized inversion methodology for highly parameterized environmental models: HYBRID REGULARIZATION METHODOLOGY journal October 2005
Ignorance is bliss: Or seven reasons not to use uncertainty analysis: OPINION journal May 2006
Model smoothing strategies to remove microscale discontinuities and spurious secondary optima in objective functions in hydrological calibration: MODEL SMOOTHING IN HYDROLOGY journal March 2007
Sensitivity analysis, calibration, and testing of a distributed hydrological model using error-based weighting and one objective function: HYDROLOGICAL MODEL CALIBRATION journal June 2009
A short exploration of structural noise: A SHORT EXPLORATION OF STRUCTURAL NOISE journal May 2010
Ancient numerical daemons of conceptual hydrological modeling: 2. Impact of time stepping schemes on model analysis and prediction journal October 2010
Reducing the computational cost of automatic calibration through model preemption: MODEL PREEMPTION APPROACH IN AUTOMATIC CALIBRATION journal November 2010
Pursuing the method of multiple working hypotheses for hydrological modeling: HYPOTHESIS TESTING IN HYDROLOGY journal September 2011
Hyperresolution global land surface modeling: Meeting a grand challenge for monitoring Earth's terrestrial water: OPINION journal May 2011
Towards a comprehensive assessment of model structural adequacy: ASSESSMENT OF MODEL STRUCTURAL ADEQUACY journal August 2012
Analysis of regression confidence intervals and Bayesian credible intervals for uncertainty quantification: REGRESSION CONFIDENCE AND BAYESIAN CREDIBLE INTERVALS journal September 2012
Review of surrogate modeling in water resources: REVIEW journal July 2012
Evaluating model structure adequacy: The case of the Maggia Valley groundwater system, southern Switzerland: Evaluating Model Structure Adequacy journal January 2013
Effective and efficient global optimization for conceptual rainfall-runoff models journal April 1992
Estimation of Aquifer Parameters Under Transient and Steady State Conditions: 1. Maximum Likelihood Method Incorporating Prior Information journal February 1986
Bees, lies and evidence-based policy journal February 2013
The Logic of Scientific Discovery journal November 1959
A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output journal February 1999
Exploring Regression Structure Using Nonparametric Functional Estimation journal September 1993
Uncertainty in science and its role in climate policy
  • Smith, Leonard A.; Stern, Nicholas
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 369, Issue 1956 https://doi.org/10.1098/rsta.2011.0149
journal December 2011
Simulation of Water-Table Aquifers Using Specified Saturated Thickness journal February 2014
Probabilistic sensitivity analysis of complex models: a Bayesian approach journal August 2004
Measuring Uncertainty Importance: Investigation and Comparison of Alternative Approaches journal October 2006
Two-Dimensional Advective Transport in Ground-Water Flow Parameter Estimation journal November 1996
Are Models Too Simple? Arguments for Increased Parameterization journal May 2007
Testing Alternative Ground Water Models Using Cross-Validation and Other Methods journal September 2007
Quantifying Data Worth Toward Reducing Predictive Uncertainty journal August 2010
Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences journal February 1994
OPR-PPR, a computer program for assessing data importance to model predictions using linear statistics report January 2007
Comparing sensitivity analysis methods to advance lumped watershed model identification and evaluation journal January 2007
Flood forecasting using a fully distributed model: application of the TOPKAPI model to the Upper Xixian Catchment journal January 2005
Assessing parameter importance of the Common Land Model based on qualitative and quantitative sensitivity analysis journal January 2013
Technical note: Method of Morris effectively reduces the computational demands of global sensitivity analysis for distributed watershed models journal January 2013

Cited By (15)

A Bayesian approach to improved calibration and prediction of groundwater models with structural error journal November 2015
Making the most out of a hydrological model data set: Sensitivity analyses to open the model black-box: ANALYSES TO OPEN THE MODEL BLACK-BOX journal September 2017
Reconnecting Stochastic Methods With Hydrogeological Applications: A Utilitarian Uncertainty Analysis and Risk Assessment Approach for the Design of Optimal Monitoring Networks journal March 2018
Signature-Domain Calibration of Hydrological Models Using Approximate Bayesian Computation: Empirical Analysis of Fundamental Properties journal June 2018
Why Should Practitioners be Concerned about Predictive Uncertainty of Groundwater Management Models? journal October 2016
The Fast and the Robust: Trade‐Offs Between Optimization Robustness and Cost in the Calibration of Environmental Models journal November 2018
A Robust Gauss‐Newton Algorithm for the Optimization of Hydrological Models: From Standard Gauss‐Newton to Robust Gauss‐Newton journal November 2018
A Robust Gauss‐Newton Algorithm for the Optimization of Hydrological Models: Benchmarking Against Industry‐Standard Algorithms journal November 2018
Model Variable Augmentation (MVA) for Diagnostic Assessment of Sensitivity Analysis Results journal April 2019
Sensitivity Analysis and Calibration of an Integrated Hydrologic Model in an Irrigated Agricultural Basin With a Groundwater‐Dependent Ecosystem journal September 2019
Comparison of Newton-type and SCE optimisation algorithms for the calibration of conceptual hydrological models journal July 2016
Terrainbento 1.0: a Python package for multi-model analysis in long-term drainage basin evolution journal January 2019
terrainbento 1.0: a Python package for multi-model analysis in long-term drainage basin evolution posted_content October 2018
Characterizing and reducing equifinality by constraining a distributed catchment model with regional signatures, local observations, and process understanding journal January 2017
The evolution of process-based hydrologic models: historical challenges and the collective quest for physical realism journal January 2017

Similar Records

Resource frugal optimizer for quantum machine learning
Journal Article · Wed Aug 23 00:00:00 EDT 2023 · Quantum Science and Technology · OSTI ID:2228652

Muddling toward frugality
Book · Sat Dec 31 23:00:00 EST 1977 · OSTI ID:6779192