skip to main content

DOE PAGESDOE PAGES

Title: PANORAMA: An approach to performance modeling and diagnosis of extreme-scale workflows

Here we report that computational science is well established as the third pillar of scientific discovery and is on par with experimentation and theory. However, as we move closer toward the ability to execute exascale calculations and process the ensuing extreme-scale amounts of data produced by both experiments and computations alike, the complexity of managing the compute and data analysis tasks has grown beyond the capabilities of domain scientists. Therefore, workflow management systems are absolutely necessary to ensure current and future scientific discoveries. A key research question for these workflow management systems concerns the performance optimization of complex calculation and data analysis tasks. The central contribution of this article is a description of the PANORAMA approach for modeling and diagnosing the run-time performance of complex scientific workflows. This approach integrates extreme-scale systems testbed experimentation, structured analytical modeling, and parallel systems simulation into a comprehensive workflow framework called Pegasus for understanding and improving the overall performance of complex scientific workflows.
Authors:
 [1] ;  [2] ;  [3] ;  [4] ;  [5] ;  [3] ;  [3] ;  [1] ;  [6] ;  [5] ;  [5] ;  [5] ;  [5] ;  [3] ;  [1]
  1. Univ. of Southern California, Los Angeles, CA (United States)
  2. Rensselaer Polytechnic Inst., Troy, NY (United States)
  3. Univ. of North Carolina, Chapel Hill, NC (United States)
  4. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  5. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  6. Univ. of Southern California, Los Angeles, CA (United States); AGH - Univ. of Science and Technology, Krakow (Poland)
Publication Date:
Grant/Contract Number:
AC05-00OR22725; SC0012636
Type:
Accepted Manuscript
Journal Name:
International Journal of High Performance Computing Applications
Additional Journal Information:
Journal Name: International Journal of High Performance Computing Applications; Journal ID: ISSN 1094-3420
Publisher:
SAGE
Research Org:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Spallation Neutron Source (SNS)
Sponsoring Org:
USDOE Office of Science (SC), Basic Energy Sciences (BES) (SC-22)
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; performance modeling; extreme scale; scientific workflow
OSTI Identifier:
1265426