A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model
Sensitivity analysis (SA) is a commonly used approach for identifying important parameters that dominate model behaviors. We use a newly developed software package, a Problem Solving environment for Uncertainty Analysis and Design Exploration (PSUADE), to evaluate the effectiveness and efficiency of ten widely used SA methods, including seven qualitative and three quantitative ones. All SA methods are tested using a variety of sampling techniques to screen out the most sensitive (i.e., important) parameters from the insensitive ones. The Sacramento Soil Moisture Accounting (SACSMA) model, which has thirteen tunable parameters, is used for illustration. The South Branch Potomac River basin near Springfield, West Virginia in the U.S. is chosen as the study area. The key findings from this study are: (1) For qualitative SA methods, Correlation Analysis (CA), Regression Analysis (RA), and Gaussian Process (GP) screening methods are shown to be not effective in this example. Morris OneAtaTime (MOAT) screening is the most efficient, needing only 280 samples to identify the most important parameters, but it is the least robust method. Multivariate Adaptive Regression Splines (MARS), Delta Test (DT) and SumOfTrees (SOT) screening methods need about 400–600 samples for the same purpose. Monte Carlo (MC), Orthogonal Array (OA) and Orthogonal Arraymore »
 Authors:

^{[1]};
^{[1]};
^{[1]};
^{[2]};
^{[2]};
^{[3]};
^{[1]};
^{[1]};
^{[1]}
 Beijing Normal Univ., Beijing (China)
 Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
 Univ. of California, Irvine, CA (United States)
 Publication Date:
 Grant/Contract Number:
 2010CB428402; 41075075
 Type:
 Accepted Manuscript
 Journal Name:
 Environmental Modelling and Software
 Additional Journal Information:
 Journal Volume: 51; Journal Issue: C; Journal ID: ISSN 13648152
 Publisher:
 Elsevier
 Research Org:
 Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
 Sponsoring Org:
 USDOE
 Country of Publication:
 United States
 Language:
 English
 Subject:
 97 MATHEMATICS AND COMPUTING; 58 GEOSCIENCES; uncertainty quantification; sensitivity analysis; parameter screening; spacefilling sampling; PSUADE
 OSTI Identifier:
 1201664
Gan, Yanjun, Duan, Qingyun, Gong, Wei, Tong, Charles, Sun, Yunwei, Chu, Wei, Ye, Aizhong, Miao, Chiyuan, and Di, Zhenhua. A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model. United States: N. p.,
Web. doi:10.1016/j.envsoft.2013.09.031.
Gan, Yanjun, Duan, Qingyun, Gong, Wei, Tong, Charles, Sun, Yunwei, Chu, Wei, Ye, Aizhong, Miao, Chiyuan, & Di, Zhenhua. A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model. United States. doi:10.1016/j.envsoft.2013.09.031.
Gan, Yanjun, Duan, Qingyun, Gong, Wei, Tong, Charles, Sun, Yunwei, Chu, Wei, Ye, Aizhong, Miao, Chiyuan, and Di, Zhenhua. 2014.
"A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model". United States.
doi:10.1016/j.envsoft.2013.09.031. https://www.osti.gov/servlets/purl/1201664.
@article{osti_1201664,
title = {A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model},
author = {Gan, Yanjun and Duan, Qingyun and Gong, Wei and Tong, Charles and Sun, Yunwei and Chu, Wei and Ye, Aizhong and Miao, Chiyuan and Di, Zhenhua},
abstractNote = {Sensitivity analysis (SA) is a commonly used approach for identifying important parameters that dominate model behaviors. We use a newly developed software package, a Problem Solving environment for Uncertainty Analysis and Design Exploration (PSUADE), to evaluate the effectiveness and efficiency of ten widely used SA methods, including seven qualitative and three quantitative ones. All SA methods are tested using a variety of sampling techniques to screen out the most sensitive (i.e., important) parameters from the insensitive ones. The Sacramento Soil Moisture Accounting (SACSMA) model, which has thirteen tunable parameters, is used for illustration. The South Branch Potomac River basin near Springfield, West Virginia in the U.S. is chosen as the study area. The key findings from this study are: (1) For qualitative SA methods, Correlation Analysis (CA), Regression Analysis (RA), and Gaussian Process (GP) screening methods are shown to be not effective in this example. Morris OneAtaTime (MOAT) screening is the most efficient, needing only 280 samples to identify the most important parameters, but it is the least robust method. Multivariate Adaptive Regression Splines (MARS), Delta Test (DT) and SumOfTrees (SOT) screening methods need about 400–600 samples for the same purpose. Monte Carlo (MC), Orthogonal Array (OA) and Orthogonal Array based Latin Hypercube (OALH) are appropriate sampling techniques for them; (2) For quantitative SA methods, at least 2777 samples are needed for Fourier Amplitude Sensitivity Test (FAST) to identity parameter main effect. McKay method needs about 360 samples to evaluate the main effect, more than 1000 samples to assess the twoway interaction effect. OALH and LPτ (LPTAU) sampling techniques are more appropriate for McKay method. For the Sobol' method, the minimum samples needed are 1050 to compute the firstorder and total sensitivity indices correctly. These comparisons show that qualitative SA methods are more efficient but less accurate and robust than quantitative ones.},
doi = {10.1016/j.envsoft.2013.09.031},
journal = {Environmental Modelling and Software},
number = C,
volume = 51,
place = {United States},
year = {2014},
month = {1}
}