Tracking Files Using the Kepler Provenance Framework

Conference · Thu Jan 01 04:00:00 EST 2009

OSTI ID:963935

Pierre, Mouallem ^[1]; Vouk, Mladen ^[1]; Klasky, Scott A ^[2]; Tchoua, Roselyne B ^[2]; Podhorszki, Norbert ^[2]

---

1. North Carolina State University
2. ORNL

Workflow Management Systems (WFMS), such as Kepler, are proving to be an important tool in scientific problem solving. They can automate and manage complex processes and huge amounts of data produced by petascale simulations. Typically, the produced data need to be properly visualized and analyzed by scientists in order to achieve the desired scientific goals. Both run-time and post analysis may benefit from, even require, additional meta-data - provenance information. One of the challenges in this context is the tracking of the data files that can be produced in very large numbers during stages of the workflow, such as visualizations. The Kepler provenance framework collects all or part of the raw information flowing through the workflow graph. This information then needs to be further parsed to extract meta-data of interest. This can be done through add-on tools and algorithms. We show how to automate tracking specific information such as data files locations.

• Research Organization: Oak Ridge National Laboratory (ORNL); Center for Computational Sciences
• Sponsoring Organization: SC USDOE - Office of Science (SC)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 963935
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
99 GENERAL AND MISCELLANEOUS
ALGORITHMS
COMPUTER GRAPHICS
DATA ANALYSIS
DATA BASE MANAGEMENT
K CODES
MEMORY MANAGEMENT