Advancing Diagnostic Model Evaluation to Better Understand Water Shortage Mechanisms in Institutionally Complex River Basins
Abstract
Abstract Water resources systems models enable valuable inferences on consequential system stressors by representing both the geophysical processes determining the movement of water and the human elements distributing it to its various competing uses. This study contributes a diagnostic evaluation framework that pairs exploratory modeling with global sensitivity analysis to enhance our ability to make inferences on water scarcity vulnerabilities in institutionally complex river basins. Diagnostic evaluation of models representing institutionally complex river basins with many stakeholders poses significant challenges. First, it needs to exploit a large and diverse suite of simulations to capture important human‐natural system interactions as well as institutionally aware behavioral mechanisms. Second, it needs to have performance metrics that are consequential and draw on decision‐relevant model outputs that adequately capture the multisector concerns that emerge from diverse basin stakeholders. We demonstrate the proposed model diagnostic framework by evaluating how potential interactions between changing hydrologic conditions and human demands influence the frequencies and durations of water shortages of varying magnitudes experienced by hundreds of users in a subbasin of the Colorado River. We show that the dominant factors shaping these effects vary both across users and, for an individual user, across percentiles of shortage magnitude. These differencesmore »
- Authors:
-
[1];
[2];
[1]
- School of Civil and Environmental Engineering Cornell University Ithaca NY USA
- Department of Engineering Systems and Environment University of Virginia Charlottesville VA USA
- Publication Date:
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1786802
- Resource Type:
- Publisher's Accepted Manuscript
- Journal Name:
- Water Resources Research
- Additional Journal Information:
- Journal Name: Water Resources Research Journal Volume: 56 Journal Issue: 10; Journal ID: ISSN 0043-1397
- Publisher:
- American Geophysical Union (AGU)
- Country of Publication:
- United States
- Language:
- English
Citation Formats
Hadjimichael, Antonia, Quinn, Julianne, and Reed, Patrick. Advancing Diagnostic Model Evaluation to Better Understand Water Shortage Mechanisms in Institutionally Complex River Basins. United States: N. p., 2020.
Web. doi:10.1029/2020WR028079.
Hadjimichael, Antonia, Quinn, Julianne, & Reed, Patrick. Advancing Diagnostic Model Evaluation to Better Understand Water Shortage Mechanisms in Institutionally Complex River Basins. United States. https://doi.org/10.1029/2020WR028079
Hadjimichael, Antonia, Quinn, Julianne, and Reed, Patrick. Fri .
"Advancing Diagnostic Model Evaluation to Better Understand Water Shortage Mechanisms in Institutionally Complex River Basins". United States. https://doi.org/10.1029/2020WR028079.
@article{osti_1786802,
title = {Advancing Diagnostic Model Evaluation to Better Understand Water Shortage Mechanisms in Institutionally Complex River Basins},
author = {Hadjimichael, Antonia and Quinn, Julianne and Reed, Patrick},
abstractNote = {Abstract Water resources systems models enable valuable inferences on consequential system stressors by representing both the geophysical processes determining the movement of water and the human elements distributing it to its various competing uses. This study contributes a diagnostic evaluation framework that pairs exploratory modeling with global sensitivity analysis to enhance our ability to make inferences on water scarcity vulnerabilities in institutionally complex river basins. Diagnostic evaluation of models representing institutionally complex river basins with many stakeholders poses significant challenges. First, it needs to exploit a large and diverse suite of simulations to capture important human‐natural system interactions as well as institutionally aware behavioral mechanisms. Second, it needs to have performance metrics that are consequential and draw on decision‐relevant model outputs that adequately capture the multisector concerns that emerge from diverse basin stakeholders. We demonstrate the proposed model diagnostic framework by evaluating how potential interactions between changing hydrologic conditions and human demands influence the frequencies and durations of water shortages of varying magnitudes experienced by hundreds of users in a subbasin of the Colorado River. We show that the dominant factors shaping these effects vary both across users and, for an individual user, across percentiles of shortage magnitude. These differences hold even for users sharing diversion locations, demand levels or water right seniority. Our findings underline the importance of detailed institutional representation for such basins, as institutions strongly shape how dominant factors of stakeholder vulnerabilities propagate through the complex network of users.},
doi = {10.1029/2020WR028079},
journal = {Water Resources Research},
number = 10,
volume = 56,
place = {United States},
year = {Fri Oct 16 00:00:00 EDT 2020},
month = {Fri Oct 16 00:00:00 EDT 2020}
}
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1029/2020WR028079
https://doi.org/10.1029/2020WR028079
Other availability
Search WorldCat to find libraries that may hold this journal
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Robustness Metrics: How Are They Calculated, When Should They Be Used and Why Do They Give Different Results?
journal, February 2018
- McPhail, C.; Maier, H. R.; Kwakkel, J. H.
- Earth's Future, Vol. 6, Issue 2
The future of water resources systems analysis: Toward a scientific framework for sustainable water management: THE FUTURE OF WATER RESOURCES SYSTEMS ANALYSIS
journal, August 2015
- Brown, Casey M.; Lund, Jay R.; Cai, Ximing
- Water Resources Research, Vol. 51, Issue 8
How Should Robustness Be Defined for Water Systems Planning under Change?
journal, October 2015
- Herman, Jonathan D.; Reed, Patrick M.; Zeff, Harrison B.
- Journal of Water Resources Planning and Management, Vol. 141, Issue 10
Drought 2002 in Colorado: An Unprecedented Drought or a Routine Drought?
journal, August 2005
- Pielke, Roger A.; Doesken, Nolan; Bliss, Odilia
- pure and applied geophysics, Vol. 162, Issue 8-9
Riverware: a Generalized tool for Complex Reservoir System Modeling1
journal, August 2001
- Zagona, Edith A.; Fulp, Terrance J.; Shane, Richard
- JAWRA Journal of the American Water Resources Association, Vol. 37, Issue 4
How Long Does a 15-Year Drought Last? On the Correlation of Rare Events
journal, January 2019
- Rhee, George; Salazar, Jimmy; Grigg, Corwin
- Journal of Climate, Vol. 32, Issue 5
Addressing ten questions about conceptual rainfall–runoff models with global sensitivity analyses in R
journal, October 2013
- Shin, Mun-Ju; Guillaume, Joseph H. A.; Croke, Barry F. W.
- Journal of Hydrology, Vol. 503
Credibility, salience, and legitimacy of boundary objects: water managers' assessment of a simulation model in an immersive decision theater
journal, April 2010
- White, Dave D.; Wutich, Amber; Larson, Kelli L.
- Science and Public Policy, Vol. 37, Issue 3
Colorado River flow dwindles as warming-driven loss of reflective snow energizes evaporation
journal, February 2020
- Milly, P. C. D.; Dunne, K. A.
- Science, Vol. 367, Issue 6483
What do we mean by sensitivity analysis? The need for comprehensive characterization of “global” sensitivity in Earth and Environmental systems models: A Critical Look at Sensitivity Analysis
journal, May 2015
- Razavi, Saman; Gupta, Hoshin V.
- Water Resources Research, Vol. 51, Issue 5
Navigating financial and supply reliability tradeoffs in regional drought management portfolios
journal, June 2014
- Zeff, Harrison B.; Kasprzyk, Joseph R.; Herman, Jonathan D.
- Water Resources Research, Vol. 50, Issue 6
Cooperation and lack thereof on management of the Yarmouk River
journal, December 2009
- Haddadin, Munther
- Water International, Vol. 34, Issue 4
Human–water interface in hydrological modelling: current status and future directions
journal, January 2017
- Wada, Yoshihide; Bierkens, Marc F. P.; de Roo, Ad
- Hydrology and Earth System Sciences, Vol. 21, Issue 8
Assessing the Risk of Persistent Drought Using Climate Model Simulations and Paleoclimate Data
journal, October 2014
- Ault, Toby R.; Cole, Julia E.; Overpeck, Jonathan T.
- Journal of Climate, Vol. 27, Issue 20
The future of hydrology: An evolving science for a changing world: OPINION
journal, May 2010
- Wagener, Thorsten; Sivapalan, Murugesu; Troch, Peter A.
- Water Resources Research, Vol. 46, Issue 5
The twenty-first century Colorado River hot drought and implications for the future: COLORADO RIVER FLOW LOSS
journal, March 2017
- Udall, Bradley; Overpeck, Jonathan
- Water Resources Research, Vol. 53, Issue 3
Knowledge systems for sustainable development
journal, May 2003
- Cash, David W.; Clark, William C.; Alcock, Frank
- Proceedings of the National Academy of Sciences, Vol. 100, Issue 14
Defining Robustness, Vulnerabilities, and Consequential Scenarios for Diverse Stakeholder Interests in Institutionally Complex River Basins
journal, July 2020
- Hadjimichael, Antonia; Quinn, Julianne; Wilson, Erin
- Earth's Future, Vol. 8, Issue 7
A nonparametric stochastic approach for multisite disaggregation of annual to daily streamflow: NON-PARAMETRIC DAILY DISAGGREGATION
journal, August 2010
- Nowak, Kenneth; Prairie, James; Rajagopalan, Balaji
- Water Resources Research, Vol. 46, Issue 8
Climate Change Impacts on the Water Balance of the Colorado Headwaters: High-Resolution Regional Climate Model Simulations
journal, June 2014
- Rasmussen, Roy; Ikeda, Kyoko; Liu, Changhai
- Journal of Hydrometeorology, Vol. 15, Issue 3
Pursuing the method of multiple working hypotheses for hydrological modeling: HYPOTHESIS TESTING IN HYDROLOGY
journal, September 2011
- Clark, Martyn P.; Kavetski, Dmitri; Fenicia, Fabrizio
- Water Resources Research, Vol. 47, Issue 9
Classifying and communicating uncertainties in model-based policy analysis
journal, January 2010
- Kwakkel, Jan H.; Walker, Warren E.; Marchau, Vincent A. W. J.
- International Journal of Technology, Policy and Management, Vol. 10, Issue 4
Global sensitivity analysis in hydrological modeling: Review of concepts, methods, theoretical framework, and applications
journal, April 2015
- Song, Xiaomeng; Zhang, Jianyun; Zhan, Chesheng
- Journal of Hydrology, Vol. 523
Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information
journal, April 1998
- Gupta, Hoshin Vijai; Sorooshian, Soroosh; Yapo, Patrice Ogou
- Water Resources Research, Vol. 34, Issue 4
Dealing With Uncertainty in Policy Analysis and Policymaking
journal, March 2003
- Walker, W. E.; Marchau, V. A. W. J.
- Integrated Assessment, Vol. 4, Issue 1
The role of groundwater trading in spatial water management
journal, November 2014
- Palazzo, Amanda; Brozović, Nicholas
- Agricultural Water Management, Vol. 145
Institutional Ecology, `Translations' and Boundary Objects: Amateurs and Professionals in Berkeley's Museum of Vertebrate Zoology, 1907-39
journal, August 1989
- Star, Susan Leigh; Griesemer, James R.
- Social Studies of Science, Vol. 19, Issue 3
The Continuum of Hydroclimate Variability in Western North America during the Last Millennium
journal, August 2013
- Ault, Toby R.; Cole, Julia E.; Overpeck, Jonathan T.
- Journal of Climate, Vol. 26, Issue 16
This is Not a Boundary Object: Reflections on the Origin of a Concept
journal, August 2010
- Leigh Star, Susan
- Science, Technology, & Human Values, Vol. 35, Issue 5
Global sensitivity measures from given data
journal, May 2013
- Plischke, Elmar; Borgonovo, Emanuele; Smith, Curtis L.
- European Journal of Operational Research, Vol. 226, Issue 3
Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates
journal, February 2001
- Sobol′, I. M.
- Mathematics and Computers in Simulation, Vol. 55, Issue 1-3
Towards reduced uncertainty in conceptual rainfall-runoff modelling: dynamic identifiability analysis
journal, January 2003
- Wagener, T.; McIntyre, N.; Lees, M. J.
- Hydrological Processes, Vol. 17, Issue 2
Water Resource Systems Models: Their Role in Planning
journal, May 1992
- Loucks, Daniel P.
- Journal of Water Resources Planning and Management, Vol. 118, Issue 3
Understanding the time‐varying importance of different uncertainty sources in hydrological modelling using global sensitivity analysis
journal, August 2016
- Pianosi, Francesca; Wagener, Thorsten
- Hydrological Processes, Vol. 30, Issue 22
Large Ensemble Analytic Framework for Consequence‐Driven Discovery of Climate Change Scenarios
journal, March 2018
- Lamontagne, Jonathan R.; Reed, Patrick M.; Link, Robert
- Earth's Future, Vol. 6, Issue 3
On the Causes of Declining Colorado River Streamflows
journal, September 2018
- Xiao, Mu; Udall, Bradley; Lettenmaier, Dennis P.
- Water Resources Research, Vol. 54, Issue 9
Reconciling theory with observations: elements of a diagnostic approach to model evaluation
journal, August 2008
- Gupta, Hoshin V.; Wagener, Thorsten; Liu, Yuqiong
- Hydrological Processes, Vol. 22, Issue 18
A new uncertainty importance measure
journal, June 2007
- Borgonovo, E.
- Reliability Engineering & System Safety, Vol. 92, Issue 6
Decision-making under great uncertainty: environmental management in an era of global change
journal, August 2011
- Polasky, Stephen; Carpenter, Stephen R.; Folke, Carl
- Trends in Ecology & Evolution, Vol. 26, Issue 8
Why so many published sensitivity analyses are false: A systematic review of sensitivity analysis practices
journal, April 2019
- Saltelli, Andrea; Aleksankina, Ksenia; Becker, William
- Environmental Modelling & Software, Vol. 114
Colorado's Decision Support Systems: Data-Centered Water Resources Planning and Administration
conference, April 2012
- Malers, Steven A.; Bennett, Ray R.; Nutting-Lane, Catherine
- Watershed Management and Operations Management Conferences 2000, Watershed Management and Operations Management 2000
Catchment response to bark beetle outbreak and dust-on-snow in the Colorado Rocky Mountains
journal, April 2015
- Livneh, Ben; Deems, Jeffrey S.; Buma, Brian
- Journal of Hydrology, Vol. 523
Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences
journal, February 1994
- Oreskes, N.; Shrader-Frechette, K.; Belitz, K.
- Science, Vol. 263, Issue 5147
Five Hundred Years of Hydrological Drought in the Upper Colorado River Basin
journal, June 2007
- Timilsena, Janak; Piechota, Thomas C.; Hidalgo, Hugo
- Journal of the American Water Resources Association, Vol. 43, Issue 3
Computer-assisted reasoning
journal, January 2001
- Bankes, S. C.; Lempert, R. J.; Popper, S. W.
- Computing in Science & Engineering, Vol. 3, Issue 2
Statsmodels: Econometric and Statistical Modeling with Python
conference, January 2010
- Seabold, Skipper; Perktold, Josef
- Proceedings of the Python in Science Conference
Battling Arrow’s Paradox to Discover Robust Water Management Alternatives
journal, February 2016
- Kasprzyk, Joseph R.; Reed, Patrick M.; Hadka, David M.
- Journal of Water Resources Planning and Management, Vol. 142, Issue 2
Exploratory Modeling for Policy Analysis
journal, June 1993
- Bankes, Steve
- Operations Research, Vol. 41, Issue 3
Structuring and evaluating decision support processes to enhance the robustness of complex human–natural systems
journal, January 2020
- Moallemi, Enayat A.; Zare, Fateme; Reed, Patrick M.
- Environmental Modelling & Software, Vol. 123
Can Exploratory Modeling of Water Scarcity Vulnerabilities and Robustness Be Scenario Neutral?
journal, November 2020
- Quinn, J. D.; Hadjimichael, A.; Reed, P. M.
- Earth's Future, Vol. 8, Issue 11
Adapting agriculture to drought and extreme events
journal, November 2012
- Lal, R.; Delgado, J. A.; Gulliford, J.
- Journal of Soil and Water Conservation, Vol. 67, Issue 6
Debates—Does Information Theory Provide a New Paradigm for Earth Science? Causality, Interaction, and Feedback
journal, February 2020
- Goodwell, Allison E.; Jiang, Peishi; Ruddell, Benjamin L.
- Water Resources Research, Vol. 56, Issue 2
Decision-centric adaptation appraisal for water management across Colorado’s Continental Divide
journal, January 2015
- Yates, David N.; Miller, Kathleen A.; Wilby, Robert L.
- Climate Risk Management, Vol. 10
How to evaluate a monitoring system for adaptive policies: criteria for signposts selection and their model-based evaluation
journal, January 2019
- Raso, Luciano; Kwakkel, Jan; Timmermans, Jos
- Climatic Change, Vol. 153, Issue 1-2
What Is Controlling Our Control Rules? Opening the Black Box of Multireservoir Operating Policies Using Time‐Varying Sensitivity Analysis
journal, July 2019
- Quinn, J. D.; Reed, P. M.; Giuliani, M.
- Water Resources Research, Vol. 55, Issue 7
Toward Understanding Direct Absorption and Grain Size Feedbacks by Dust Radiative Forcing in Snow With Coupled Snow Physical and Radiative Transfer Modeling
journal, August 2019
- Skiles, S. McKenzie; Painter, Thomas H.
- Water Resources Research, Vol. 55, Issue 8
A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code
journal, May 1979
- McKay, M. D.; Beckman, R. J.; Conover, W. J.
- Technometrics, Vol. 21, Issue 2
Sensitivity analysis of environmental models: A systematic review with practical workflow
journal, May 2016
- Pianosi, Francesca; Beven, Keith; Freer, Jim
- Environmental Modelling & Software, Vol. 79
A new decision sciences for complex systems
journal, May 2002
- Lempert, R. J.
- Proceedings of the National Academy of Sciences, Vol. 99, Issue Supplement 3
Reservoir Operations Management Using a Water Resources Model
conference, April 2012
- Parsons, Rick; Bennett, Ray
- Operations Management Conference 2006, Operating Reservoirs in Changing Conditions
Comparing sensitivity analysis methods to advance lumped watershed model identification and evaluation
journal, January 2007
- Tang, Y.; Reed, P.; Wagener, T.
- Hydrology and Earth System Sciences, Vol. 11, Issue 2
A hidden Markov model combined with climate indices for multidecadal streamflow simulation
journal, October 2014
- Bracken, C.; Rajagopalan, B.; Zagona, E.
- Water Resources Research, Vol. 50, Issue 10
CalSim: Generalized Model for Reservoir System Analysis
journal, November 2004
- Draper, Andrew J.; Munévar, Armin; Arora, Sushil K.
- Journal of Water Resources Planning and Management, Vol. 130, Issue 6
A multimodel ensemble approach to assessment of climate change impacts on the hydrology and water resources of the Colorado River Basin
journal, January 2007
- Christensen, N. S.; Lettenmaier, D. P.
- Hydrology and Earth System Sciences, Vol. 11, Issue 4
Sedimentation and sustainability of western American reservoirs: SEDIMENTATION AND SUSTAINABILITY
journal, December 2010
- Graf, William L.; Wohl, Ellen; Sinha, Tushar
- Water Resources Research, Vol. 46, Issue 12
SALib: An open-source Python library for Sensitivity Analysis
journal, January 2017
- Herman, Jon; Usher, Will
- The Journal of Open Source Software, Vol. 2, Issue 9
Time-varying sensitivity analysis clarifies the effects of watershed model formulation on model behavior: TIME-VARYING SENSITIVITY OF WATERSHED MODELS
journal, March 2013
- Herman, J. D.; Reed, P. M.; Wagener, T.
- Water Resources Research, Vol. 49, Issue 3