Summit Darshan Archival Dataset
- ORNL-OLCF
Summit Darshan Archival Dataset contains 2021 Summit Darshan log data for 25 applications and is grouped into science domains. The dataset is processed, and all the propriety fields are anonymized. The resultant data is converted into a tabular structure and saved in parquet file format. In this notebook, we demonstrate how to access the data. Data Organization: The data is organized into two directories: Darshan total (`darshan_total`): List all the high levels generated by the `darshan-parser --total` command on `.darshan` files. There is one parquet file for each application. Note: `uid` and `exe` field are masked Darshan detail (`darshan_detail`): This data contains detailed job level log information extracted by command `darshan-parser` on the raw `.darshan` files. The data is sorted by directory hierarchy in the order of `year/month/day (2021/12/07)`. For instance, to get the data for a `job_id` 3819766 of application `App11`, which was executed on `2021-12-07`can be accessed as follows. Note:`uid` and `filename` fields are masked
- Research Organization:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
- Sponsoring Organization:
- Oak Ridge Leadership Computing Facility, Oak Ridge National Laboratory
- DOE Contract Number:
- AC05-00OR22725
- OSTI ID:
- 2305496
- Country of Publication:
- United States
- Language:
- English
Similar Records
April 2020 Darshan counters from the Summit supercomputer
Performance Evaluation of Darshan 3.0.0 on the Cray XC30
Darshan for HEP applications
Dataset
·
Tue May 03 00:00:00 EDT 2022
·
OSTI ID:1865904
Performance Evaluation of Darshan 3.0.0 on the Cray XC30
Technical Report
·
Fri Apr 01 00:00:00 EDT 2016
·
OSTI ID:1250469
Darshan for HEP applications
Conference
·
Sun Dec 31 23:00:00 EST 2023
·
OSTI ID:2447365