skip to main content


Title: Validating the simulation of large-scale parallel applications using statistical characteristics

Simulation is a widely adopted method to analyze and predict the performance of large-scale parallel applications. Validating the hardware model is highly important for complex simulations with a large number of parameters. Common practice involves calculating the percent error between the projected and the real execution time of a benchmark program. However, in a high-dimensional parameter space, this coarse-grained approach often suffers from parameter insensitivity, which may not be known a priori. Moreover, the traditional approach cannot be applied to the validation of software models, such as application skeletons used in online simulations. In this work, we present a methodology and a toolset for validating both hardware and software models by quantitatively comparing fine-grained statistical characteristics obtained from execution traces. Although statistical information has been used in tasks like performance optimization, this is the first attempt to apply it to simulation validation. Lastly, our experimental results show that the proposed evaluation approach offers significant improvement in fidelity when compared to evaluation using total execution time, and the proposed metrics serve as reliable criteria that progress toward automating the simulation tuning process.
 [1] ;  [2] ;  [2] ;  [3]
  1. Univ. of Central Florida, Orlando, FL (United States)
  2. Sandia National Lab. (SNL-CA), Livermore, CA (United States)
  3. Univ. of Central Florida, Orlando, FL (United States); Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Publication Date:
Report Number(s):
Journal ID: ISSN 2376-3639; 583296
Grant/Contract Number:
Accepted Manuscript
Journal Name:
ACM Transactions on Modeling and Performance Evaluation of Computing Systems
Additional Journal Information:
Journal Volume: 1; Journal Issue: 1; Journal ID: ISSN 2376-3639
Association for Computing Machinery
Research Org:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Sandia National Laboratories, Livermore, CA (United States)
Sponsoring Org:
USDOE National Nuclear Security Administration (NNSA)
Country of Publication:
United States
97 MATHEMATICS AND COMPUTING; measurement; experimentation; simulation evaluation; evaluation metrics; software skeleton
OSTI Identifier: