When Workflow Management Systems and Logging Systems Meet: Analyzing Large-Scale Execution Traces
This poster shows the benefits of integrating a workflow management system with logging and log mining capabilities. By combing two existing, mature technologies: Pegasus-WMS and Netlogger, we are able to efficiently process execution logs of earthquake science workflows consisting of hundreds of thousands to one million tasks. In particular we show results of processing logs of CyberShake, a workflow application running on the TeraGrid. Client-side tools allow scientists to quickly gather statistics about a workflow run and find out which tasks executed, where they were executed, what was their runtime, etc. These statistics can be used to understand the performance characteristics of a workflow and help tune the execution parameters of the workflow management system. This poster shows the scalability of the system presenting results of uploading task execution records into the system and by showing results of querying the system for overall workflow performance information.
- Research Organization:
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
- Sponsoring Organization:
- Computational Research Division
- DOE Contract Number:
- DE-AC02-05CH11231
- OSTI ID:
- 1050631
- Report Number(s):
- LBNL-4777E; TRN: US201218%%848
- Resource Relation:
- Conference: SC08, Austin, TX, USA, 11/15/08 - 11/21/08
- Country of Publication:
- United States
- Language:
- English
Similar Records
Towards Lightweight Data Integration Using Multi-Workflow Provenance and Data Observability
Autonomic Management of Application Workflows on Hybrid Computing Infrastructure