Panorama 360 (Final Report)

Technical Report · Thu Jun 22 04:00:00 EDT 2023

DOI:https://doi.org/10.2172/1846090· OSTI ID:1846090

Deelman, Ewa ^[1]

---

1. Univ. of Southern California, Los Angeles, CA (United States); University of Southern California

This is the final technical report for the DOE-funded Panorama 360 project. Panorama 360 provided a resource for the collection, analysis, and sharing of performance data about end-to-end scientific workflows executing on DOE facilities. The work focused on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: 1. A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); 2. A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; 3. A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and 4. Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

• Research Organization: Univ. of Southern California, Los Angeles, CA (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
• Contributing Organization: Renaissance Computing Institute (RENCI); Oak Ridge National Laboratory (ORNL); Lawrence Berkeley National Laboratory (LBNL)
• DOE Contract Number: SC0012636
• OSTI ID: 1846090
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
Anomaly Detection
Distributed Computing Resources
Performance Modeling
Scientific Workflows