Graph-Based Similarity Metrics for Comparing Simulation Model Causal Structures
- Sandia National Laboratories (SNL), Albuquerque, NM, and Livermore, CA (United States)
The causal structure of a simulation is a major determinant of both its character and behavior, yet most methods we use to compare simulations focus only on simulation outputs. We introduce a method that combines graphical representation with information theoretic metrics to quantitatively compare the causal structures of models. The method applies to agent-based simulations as well as system dynamics models and facilitates comparison within and between types. Comparing models based on their causal structures can illuminate differences in assumptions made by the models, allowing modelers to (1) better situate their models in the context of existing work, including highlighting novelty, (2) explicitly compare conceptual theory and assumptions to simulated theory and assumptions, and (3) investigate potential causal drivers of divergent behavior between models. We demonstrate the method by comparing two epidemiology models at different levels of aggregation.
- Research Organization:
- Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
- Sponsoring Organization:
- USDOE National Nuclear Security Administration (NNSA); DARPA
- DOE Contract Number:
- NA0003525;
- OSTI ID:
- 1884926
- Report Number(s):
- SAND2022-11300; 709379
- Country of Publication:
- United States
- Language:
- English
Similar Records
Feedback density and causal complexity of simulation model structure
Causal Evaluations for Identifying Differences between Observations and Earth System Models
Verification, Validation, and Calibration Through a Causal Lens
Journal Article
·
2021
· Journal of Simulation
·
OSTI ID:1828789
Causal Evaluations for Identifying Differences between Observations and Earth System Models
Technical Report
·
2021
·
OSTI ID:1820528
Verification, Validation, and Calibration Through a Causal Lens
Conference
·
2024
·
OSTI ID:2352475