skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Evaluating Emulation-based Models of Distributed Computing Systems

Abstract

Emulation-based models of distributed computing systems are collections of virtual ma- chines, virtual networks, and other emulation components configured to stand in for oper- ational systems when performing experimental science, training, analysis of design alterna- tives, test and evaluation, or idea generation. As with any tool, we should carefully evaluate whether our uses of emulation-based models are appropriate and justified. Otherwise, we run the risk of using a model incorrectly and creating meaningless results. The variety of uses of emulation-based models each have their own goals and deserve thoughtful evaluation. In this paper, we enumerate some of these uses and describe approaches that one can take to build an evidence-based case that a use of an emulation-based model is credible. Predictive uses of emulation-based models, where we expect a model to tell us something true about the real world, set the bar especially high and the principal evaluation method, called validation , is comensurately rigorous. We spend the majority of our time describing and demonstrating the validation of a simple predictive model using a well-established methodology inherited from decades of development in the compuational science and engineering community.

Authors:
 [1];  [1];  [2]
  1. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States). Cyber Initiatives
  2. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States). Emulytics Initiatives
Publication Date:
Research Org.:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1398865
Report Number(s):
SAND-2017-10634
657469
DOE Contract Number:  
AC04-94AL85000
Resource Type:
Technical Report
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Jones, Stephen T., Gabert, Kasimir G., and Tarman, Thomas D. Evaluating Emulation-based Models of Distributed Computing Systems. United States: N. p., 2017. Web. doi:10.2172/1398865.
Jones, Stephen T., Gabert, Kasimir G., & Tarman, Thomas D. Evaluating Emulation-based Models of Distributed Computing Systems. United States. https://doi.org/10.2172/1398865
Jones, Stephen T., Gabert, Kasimir G., and Tarman, Thomas D. Tue . "Evaluating Emulation-based Models of Distributed Computing Systems". United States. https://doi.org/10.2172/1398865. https://www.osti.gov/servlets/purl/1398865.
@article{osti_1398865,
title = {Evaluating Emulation-based Models of Distributed Computing Systems},
author = {Jones, Stephen T. and Gabert, Kasimir G. and Tarman, Thomas D.},
abstractNote = {Emulation-based models of distributed computing systems are collections of virtual ma- chines, virtual networks, and other emulation components configured to stand in for oper- ational systems when performing experimental science, training, analysis of design alterna- tives, test and evaluation, or idea generation. As with any tool, we should carefully evaluate whether our uses of emulation-based models are appropriate and justified. Otherwise, we run the risk of using a model incorrectly and creating meaningless results. The variety of uses of emulation-based models each have their own goals and deserve thoughtful evaluation. In this paper, we enumerate some of these uses and describe approaches that one can take to build an evidence-based case that a use of an emulation-based model is credible. Predictive uses of emulation-based models, where we expect a model to tell us something true about the real world, set the bar especially high and the principal evaluation method, called validation , is comensurately rigorous. We spend the majority of our time describing and demonstrating the validation of a simple predictive model using a well-established methodology inherited from decades of development in the compuational science and engineering community.},
doi = {10.2172/1398865},
url = {https://www.osti.gov/biblio/1398865}, journal = {},
number = ,
volume = ,
place = {United States},
year = {2017},
month = {8}
}