skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: DXT: Darshan eXtended Tracing

Abstract

As modern supercomputers evolve to exascale, their I/O subsystems are becoming increasingly complex, making optimization of I/O for scientific applications a daunting task. Although I/O profiling tools facilitate the process of optimizing application I/O performance, legacy profiling tools lack flexibility in their level of detail and ability to correlate traces with other sources of data. Additionally, a lack of robust trace analysis tools makes it difficult to derive actionable insights from large-scale I/O traces. Darshan is an HPC I/O characterization tool that records statistics in a lightweight manner that makes it appropriate for full-time production deployment. However, Darshan’s default characterization mechanism records information at a fixed granularity. We augment Darshan by proposing Darshan eXtended Tracing (DXT) for more detailed profiling of I/O software stacks. DXT enables users and administrators to vary the level of fidelity captured by Darshan at run time without modifying or recompiling applications. This capability facilitates systematic analysis on the I/O behavior of applications and can provide useful application kernel I/O traces to help advance parallel I/O research. We have demonstrated the power of DXT by obtaining a wide range of useful statistics for multiple case studies, and we further show that DXT is able to domore » so same with negligible overhead.« less

Authors:
; ; ; ; ; ; ;
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1490709
DOE Contract Number:  
AC02-05CH11231
Resource Type:
Conference
Resource Relation:
Conference: Cray Users Group Meeting 2017, Redmond, CA (United States), 8-11 May 2017
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Xu, Cong, Snyder, Shane, Kulkarni, Omkar, Venkatesan, Vishwanath, Carns, Phillip, Byna, Surendra, Sisneros, Robert, and Chadalavada, Kalyana. DXT: Darshan eXtended Tracing. United States: N. p., 2019. Web.
Xu, Cong, Snyder, Shane, Kulkarni, Omkar, Venkatesan, Vishwanath, Carns, Phillip, Byna, Surendra, Sisneros, Robert, & Chadalavada, Kalyana. DXT: Darshan eXtended Tracing. United States.
Xu, Cong, Snyder, Shane, Kulkarni, Omkar, Venkatesan, Vishwanath, Carns, Phillip, Byna, Surendra, Sisneros, Robert, and Chadalavada, Kalyana. Wed . "DXT: Darshan eXtended Tracing". United States. https://www.osti.gov/servlets/purl/1490709.
@article{osti_1490709,
title = {DXT: Darshan eXtended Tracing},
author = {Xu, Cong and Snyder, Shane and Kulkarni, Omkar and Venkatesan, Vishwanath and Carns, Phillip and Byna, Surendra and Sisneros, Robert and Chadalavada, Kalyana},
abstractNote = {As modern supercomputers evolve to exascale, their I/O subsystems are becoming increasingly complex, making optimization of I/O for scientific applications a daunting task. Although I/O profiling tools facilitate the process of optimizing application I/O performance, legacy profiling tools lack flexibility in their level of detail and ability to correlate traces with other sources of data. Additionally, a lack of robust trace analysis tools makes it difficult to derive actionable insights from large-scale I/O traces. Darshan is an HPC I/O characterization tool that records statistics in a lightweight manner that makes it appropriate for full-time production deployment. However, Darshan’s default characterization mechanism records information at a fixed granularity. We augment Darshan by proposing Darshan eXtended Tracing (DXT) for more detailed profiling of I/O software stacks. DXT enables users and administrators to vary the level of fidelity captured by Darshan at run time without modifying or recompiling applications. This capability facilitates systematic analysis on the I/O behavior of applications and can provide useful application kernel I/O traces to help advance parallel I/O research. We have demonstrated the power of DXT by obtaining a wide range of useful statistics for multiple case studies, and we further show that DXT is able to do so same with negligible overhead.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2019},
month = {1}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: