Scalable Analysis of Distributed Workflow Traces
Bacterial response to nitric oxide (NO) is of major importance since NO is an obligatory intermediate of the nitrogen cycle. Transcriptional regulation of the dissimilatory nitric oxides metabolism in bacteria is Large-scale workflows are becoming increasingly important in both the scientific research and business domains. Science and commerce have both experienced an explosion in the sheer amount of data that must be analyzed. An important tool for analyzing these huge datasets is a compute cluster of hundreds or thousands of machines. However, debugging and tuning clusters requires specialized tools. Current cluster performance tools are more oriented towards tightly coupled parallel applications. We describe how the NetLogger Toolkit methodology is more appropriate for this class of cluster computing, and describe our new automatic work flow anomaly detection component. We also describe how this methodology is being used in the Nearby Supernova Factory (SN factory) project at Lawrence Berkeley National Laboratory.
- Research Organization:
- Ernest Orlando Lawrence Berkeley NationalLaboratory, Berkeley, CA (US)
- Sponsoring Organization:
- USDOE Director. Office of Science. Office of AdvancedScientific Computing Research. Mathematical Information and ComputationalSciences Division
- DOE Contract Number:
- AC02-05CH11231
- OSTI ID:
- 861968
- Report Number(s):
- LBNL--57060; BnR: KJ0102000
- Country of Publication:
- United States
- Language:
- English
Similar Records
Chimbuko: A Workflow-Level Scalable Performance Trace Analysis Tool
ExaWorks: Workflows for Exascale