Statistical Validation of Engineering and Scientific Models: A Maximum Likelihood Based Metric

Technical Report · Tue Jan 01 04:00:00 EST 2002

DOI:https://doi.org/10.2172/791881· OSTI ID:791881

HILLS, RICHARD GUY ^[1]; TRUCANO, TIMOTHY G

---

1. Sandia National Laboratories

Two major issues associated with model validation are addressed here. First, we present a maximum likelihood approach to define and evaluate a model validation metric. The advantage of this approach is it is more easily applied to nonlinear problems than the methods presented earlier by Hills and Trucano (1999, 2001); the method is based on optimization for which software packages are readily available; and the method can more easily be extended to handle measurement uncertainty and prediction uncertainty with different probability structures. Several examples are presented utilizing this metric. We show conditions under which this approach reduces to the approach developed previously by Hills and Trucano (2001). Secondly, we expand our earlier discussions (Hills and Trucano, 1999, 2001) on the impact of multivariate correlation and the effect of this on model validation metrics. We show that ignoring correlation in multivariate data can lead to misleading results, such as rejecting a good model when sufficient evidence to do so is not available.

• Research Organization: Sandia National Labs., Albuquerque, NM (US); Sandia National Labs., Livermore, CA (US)
• Sponsoring Organization: US Department of Energy (US)
• DOE Contract Number: AC04-94AL85000
• OSTI ID: 791881
• Report Number(s): SAND2001-1783
• Country of Publication: United States
• Language: English

Related Subjects

99 GENERAL AND MISCELLANEOUS
FORECASTING
MATHEMATICAL MODELS
MAXIMUM-LIKELIHOOD FIT
METRICS
NONLINEAR PROBLEMS
PROBABILITY
VALIDATION