Panorama 360 (Final Report)

Technical Report · Thu Mar 31 04:00:00 EDT 2022

DOI:https://doi.org/10.2172/1856320· OSTI ID:1856320

Deelman, Ewa ^[1]

---

1. Univ. of Southern California, Los Angeles, CA (United States); The University Of North Carolina at Chapel Hill

This final technical report from the lead institution, USC grant #DE-SC0012636, serves as the final technical report for collaborative institution UNC-CH grant #DE-SC0012390. The goal was to develop a repository and associated capabilities for data collection, ingestion, and analysis for a broad class of DOE applications that span experimental and simulation science workflows. In particular, this work focuses on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: (1) A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); (2) A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; (3) A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and (4) Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

• Research Organization: Univ. of North Carolina, Chapel Hill, NC (United States); Univ. of Southern California, Los Angeles, CA (United States)
• Sponsoring Organization: USDOE Office of Science (SC)
• DOE Contract Number: SC0012390; SC0012636
• OSTI ID: 1856320
• Report Number(s): SOE-UNC-CH-12390
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
data
repository
workflow