skip to main content

DOE PAGESDOE PAGES

Title: An Efficient Format for Nearly Constant-Time Access to Arbitrary Time Intervals in Large Trace Files

A powerful method to aid in understanding the performance of parallel applications uses log or trace files containing time-stamped events and states (pairs of events). These trace files can be very large, often hundreds or even thousands of megabytes. Because of the cost of accessing and displaying such files, other methods are often used that reduce the size of the tracefiles at the cost of sacrificing detail or other information. This paper describes a hierarchical trace file format that provides for display of an arbitrary time window in a time independent of the total size of the file and roughly proportional to the number of events within the time window. This format eliminates the need to sacrifice data to achieve a smaller trace file size (since storage is inexpensive, it is necessary only to make efficient use of bandwidth to that storage). The format can be used to organize a trace file or to create a separate file of annotations that may be used with conventional trace files. We present an analysis of the time to access all of the events relevant to an interval of time and we describe experiments demonstrating the performance of this file format.
Authors:
 [1] ;  [1] ;  [1]
  1. Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL, USA
Publication Date:
OSTI Identifier:
1197991
Grant/Contract Number:
AC02-06CH11357
Type:
Published Article
Journal Name:
Scientific Programming
Additional Journal Information:
Journal Volume: 16; Journal Issue: 2-3; Related Information: CHORUS Timestamp: 2016-08-23 10:36:07; Journal ID: ISSN 1058-9244
Publisher:
Hindawi Publishing Corporation
Sponsoring Org:
USDOE
Country of Publication:
Egypt
Language:
English