skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Hierarchical Model Validation of Symbolic Performance Models of Scientific Kernels

Abstract

Multi-resolution validation of hierarchical performance models of scientific applications is critical primarily for two reasons. First, the step-by-step validation determines the correctness of all essential components or phases in a science simulation. Second, a model that is validated at multiple resolution levels is the very first step to generate predictive performance models, for not only existing systems but also for emerging systems and future problem sizes. We present the design and validation of hierarchical performance models of two scientific benchmarks using a new technique called the modeling assertions (MA). Our MA prototype framework generates symbolic performance models that can be evaluated efficiently by generating the equivalent model representations in Octave and MATLAB. The multi-resolution modeling and validation is conducted on two contemporary, massively-parallel systems, XT3 and Blue Gene/L system. The workload distribution and the growth rates predictions generated by the MA models are confirmed by the experimental data collected on the MPP platforms. In addition, the physical memory requirements that are generated by the MA models are verified by the runtime values on the Blue Gene/L system, which has 512 MBytes and 256 MBytes physical memory capacity in its two unique execution modes.

Authors:
 [1];  [1]
  1. ORNL
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
931654
DOE Contract Number:
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Journal Volume: 4128; Conference: International Euro-Par Conference 2006, Dresden, Germany, 20060829, 20060901
Country of Publication:
United States
Language:
English

Citation Formats

Alam, Sadaf R, and Vetter, Jeffrey S. Hierarchical Model Validation of Symbolic Performance Models of Scientific Kernels. United States: N. p., 2006. Web. doi:10.1007/11823285_8.
Alam, Sadaf R, & Vetter, Jeffrey S. Hierarchical Model Validation of Symbolic Performance Models of Scientific Kernels. United States. doi:10.1007/11823285_8.
Alam, Sadaf R, and Vetter, Jeffrey S. Sun . "Hierarchical Model Validation of Symbolic Performance Models of Scientific Kernels". United States. doi:10.1007/11823285_8.
@article{osti_931654,
title = {Hierarchical Model Validation of Symbolic Performance Models of Scientific Kernels},
author = {Alam, Sadaf R and Vetter, Jeffrey S},
abstractNote = {Multi-resolution validation of hierarchical performance models of scientific applications is critical primarily for two reasons. First, the step-by-step validation determines the correctness of all essential components or phases in a science simulation. Second, a model that is validated at multiple resolution levels is the very first step to generate predictive performance models, for not only existing systems but also for emerging systems and future problem sizes. We present the design and validation of hierarchical performance models of two scientific benchmarks using a new technique called the modeling assertions (MA). Our MA prototype framework generates symbolic performance models that can be evaluated efficiently by generating the equivalent model representations in Octave and MATLAB. The multi-resolution modeling and validation is conducted on two contemporary, massively-parallel systems, XT3 and Blue Gene/L system. The workload distribution and the growth rates predictions generated by the MA models are confirmed by the experimental data collected on the MPP platforms. In addition, the physical memory requirements that are generated by the MA models are verified by the runtime values on the Blue Gene/L system, which has 512 MBytes and 256 MBytes physical memory capacity in its two unique execution modes.},
doi = {10.1007/11823285_8},
journal = {},
number = ,
volume = 4128,
place = {United States},
year = {Sun Jan 01 00:00:00 EST 2006},
month = {Sun Jan 01 00:00:00 EST 2006}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:
  • Evaluation of model performance may consist of objectively comparing corresponding pairs of observed and predicted values. This defensible provided bias has been identified and removed from the data and that the data are distributed normally or may be transformed so as to be distributed normally. Recent modifications to the TERPED computer code have made this program applicable to the analysis and transformation of small and large data sets. TERPED was used to perform Kolmogorov-Smirnov and graphical analyses of mormalized crosswind - integrated ground-level air concentration measurements of zinc sulfide particles released from a height of 111 m at Hanford, Washingtonmore » as reported by Nickola. Our analysis demonstrates that the data may be considered to be distributed log-normally (..cap alpha.. = 0.014), but not normally (..cap alpha.. = 0.86). Simulations of particulate transport used each of three methods of stability determination - (1) vertical temperature gradient, (2) standard deviation of wind direction, and (3) vertical temperature gradient and wind speed - and each of five dispersion parameterizations. Model performance was evaluated by computing the root mean squared difference of log-transformed value pairs, and it was shown that method (2) and Julich (100m) dispersion values gave the best results. For comparison, the models were evaluated using the nontransformed observed and predicted values, and it was shown that application of this inappropriate approach yields ambiguous results that suggest markedly different conclusions.« less
  • Performance and workload modeling has numerous uses at every stage of the high-end computing lifecycle: design, integration, procurement, installation and tuning. Despite the tremendous usefulness of performance models, their construction remains largely a manual, complex, and time-consuming exercise. We propose a new approach to the model construction, called modeling assertions (MA), which borrows advantages from both the empirical and analytical modeling techniques. This strategy has many advantages over traditional methods: incremental construction of realistic performance models, straightforward model validation against empirical data, and intuitive error bounding on individual model terms. We demonstrate this new technique on the NAS parallel CGmore » and SP benchmarks by constructing high fidelity models for the floating-point operation cost, memory requirements, and MPI message volume. These models are driven by a small number of key input parameters thereby allowing efficient design space exploration of future problem sizes and architectures.« less
  • Distributed shared memory architectures (DSM`s) such as the Origin 2000 are being implemented which extend the concept of single-processor cache hierarchies across an entire physically-distributed multiprocessor machine. The scalability of a DSM machine is inherently tied to memory hierarchy performance, including such issues as latency hiding techniques in the architecture, global cache-coherence protocols, memory consistency models and, of course, the inherent locality of reference in algorithms of interest. In this paper, we characterize application performance with a {open_quotes}memory-centric{close_quotes} view. Using a simple mean value analysis (MVA) strategy and empirical performance data, we infer the contribution of each level in themore » memory system to the application`s overall cycles per instruction (cpi). We account for the overlap of processor execution with memory accesses - a key parameter which is not directly measurable on the Origin systems. We infer the separate contributions of three major architecture features in the memory subsystem of the Origin 2000: cache size, outstanding loads-under-miss, and memory latency.« less
  • Large data sets cross-classified according to multiple factors are available in epidemiology and other disciplines. Their analysis often calls for finding a small set of best hierarchical log-linear models to serve as a basis for further analysis. This selection can be based on an Akaike information type statistic. Fitting all possible models to find a best set is usually not feasible for as few as five factors (7581 possible models). By noting that the set of hierarchical models and their relationships can be represented by a graph, a graph traversal algorithm is developed that requires fitting a fraction of allmore » models to find an exact subset of the best models. The algorithm classifies (in the best set/not in the best set) as many models as possible on the basis of each fit. A data structure implementing the graph of model nodes keeps track of the information required by the algorithm. 7 refs., 4 figs.« less
  • In this paper, the authors characterize application performance with a memory-centric view. Using a simple strategy and performance data measured by on-chip hardware performance counters, they model the performance of a simple memory hierarchy and infer the contribution of each level in the memory system to an application`s overall cycles per instruction (cpi). They account for the overlap of processor execution with memory accesses--a key parameter not directly measurable on most systems. They infer the separate contributions of three major architecture features in the memory subsystem of the Origin 2000: cache size, outstanding loads-under-miss, and memory latency.