R&D for computational cognitive and social models : foundations for model evaluation through verification and validation (final LDRD report).
Sandia National Laboratories is investing in projects that aim to develop computational modeling and simulation applications that explore human cognitive and social phenomena. While some of these modeling and simulation projects are explicitly research oriented, others are intended to support or provide insight for people involved in high consequence decision-making. This raises the issue of how to evaluate computational modeling and simulation applications in both research and applied settings where human behavior is the focus of the model: when is a simulation 'good enough' for the goals its designers want to achieve? In this report, we discuss two years' worth of review and assessment of the ASC program's approach to computational model verification and validation, uncertainty quantification, and decision making. We present a framework that extends the principles of the ASC approach into the area of computational social and cognitive modeling and simulation. In doing so, we argue that the potential for evaluation is a function of how the modeling and simulation software will be used in a particular setting. In making this argument, we move from strict, engineering and physics oriented approaches to V&V to a broader project of model evaluation, which asserts that the systematic, rigorous, and transparent accumulation of evidence about a model's performance under conditions of uncertainty is a reasonable and necessary goal for model evaluation, regardless of discipline. How to achieve the accumulation of evidence in areas outside physics and engineering is a significant research challenge, but one that requires addressing as modeling and simulation tools move out of research laboratories and into the hands of decision makers. This report provides an assessment of our thinking on ASC Verification and Validation, and argues for further extending V&V research in the physical and engineering sciences toward a broader program of model evaluation in situations of high consequence decision-making.
- Research Organization:
- Sandia National Laboratories
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC04-94AL85000
- OSTI ID:
- 945901
- Report Number(s):
- SAND2008-6453
- Country of Publication:
- United States
- Language:
- English
Similar Records
Verification and validation as applied epistemology.
Why Models Don%3CU%2B2019%3Et Forecast.
Verification, Validation, and Predictive Capability in Computational Engineering and Physics
Conference
·
Sun Apr 01 00:00:00 EDT 2007
·
OSTI ID:952371
Why Models Don%3CU%2B2019%3Et Forecast.
Conference
·
Sun Aug 01 00:00:00 EDT 2010
·
OSTI ID:1022212
Verification, Validation, and Predictive Capability in Computational Engineering and Physics
Technical Report
·
Fri Jan 31 23:00:00 EST 2003
·
OSTI ID:809603