Supercomputing and Scientific Workflows Gaps and Requirements
Over the past decade, workflows have been successfully applied to a number of scientific domains with great success. Workflow engines are now commonly used across scientific disciplines to automate mundane tasks, collect provenance, and orchestrate complex processes. However, workflows have not yet made significant strides managing fine-grain, concurrent tasks directly on supercomputing platforms. As scientific computing becomes an increasingly important discovery method and high performance computing environments become more complex, addressing this gap becomes critical. Using a simple use case as motivation, this paper describes the current barriers to using workflow engines in a supercomputing environment and outlines the new capabilities that must be provided if workflows are to be successfully applied in this context.
- Research Organization:
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-76RL01830
- OSTI ID:
- 1024538
- Report Number(s):
- PNNL-SA-78245; KJ0403000; TRN: US201119%%450
- Resource Relation:
- Conference: IEEE World Congress on Services, July 4-9, 2011, Washington, DC, 208 - 211
- Country of Publication:
- United States
- Language:
- English
Similar Records
Accelerating Scientific Workflows on HPC Platforms with In Situ Processing
A Multi-Dimensional Classification Model for Scientific Workflow Characteristics