Tracking Files in the Kepler Provenance Framework

Conference · Thu Jan 01 04:00:00 EST 2009

OSTI ID:982189

Mouallem, Pierre A ^[1]; Klasky, Scott A ^[1]; Podhorszki, Norbert ^[2]; Vouk, Mladen ^[3]

---

1. ORNL
2. University of California, Davis
3. North Carolina State University

Workflow Management Systems (WFMS), such as Kepler, are proving to be an important tool in scientific problem solving. They can automate and manage complex processes and huge amounts of data produced by petascale simulations. Typically, the produced data need to be properly visualized and analyzed by scientists in order to achieve the desired scientific goals. Both run-time and post analysis may benefit from, even require, additional meta-data - provenance information. One of the challenges in this context is the tracking of the data files that can be produced in very large numbers during stages of the workflow, such as visualizations. The Kepler provenance framework collects all or part of the raw information flowing through the workflow graph. This information then needs to be further parsed to extract meta-data of interest. This can be done through add-on tools and algorithms. We show how to automate tracking specific information such as data files locations.

• Research Organization: Oak Ridge National Laboratory (ORNL)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 982189
• Country of Publication: United States
• Language: English

Related Subjects

99 GENERAL AND MISCELLANEOUS
ALGORITHMS
COMPUTERS
MANAGEMENT
PROGRAMMING