Graph-Based Similarity Metrics for Comparing Simulation Model Causal Structures
- Sandia National Laboratories (SNL), Albuquerque, NM, and Livermore, CA (United States)
The causal structure of a simulation is a major determinant of both its character and behavior, yet most methods we use to compare simulations focus only on simulation outputs. We introduce a method that combines graphical representation with information theoretic metrics to quantitatively compare the causal structures of models. The method applies to agent-based simulations as well as system dynamics models and facilitates comparison within and between types. Comparing models based on their causal structures can illuminate differences in assumptions made by the models, allowing modelers to (1) better situate their models in the context of existing work, including highlighting novelty, (2) explicitly compare conceptual theory and assumptions to simulated theory and assumptions, and (3) investigate potential causal drivers of divergent behavior between models. We demonstrate the method by comparing two epidemiology models at different levels of aggregation.
- Research Organization:
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Sponsoring Organization:
- USDOE National Nuclear Security Administration (NNSA); Defense Advanced Research Projects Agency (DARPA)
- DOE Contract Number:
- NA0003525
- OSTI ID:
- 1884926
- Report Number(s):
- SAND2022-11300; 709379
- Country of Publication:
- United States
- Language:
- English
Similar Records
Explaining and predicting human behavior and social dynamics in simulated virtual worlds: reproducibility, generalizability, and robustness of causal discovery methods
Causally‐Informed Deep Learning to Improve Climate Models and Projections