Services + Components = Data Intensive Scientific Workflow Applications with MeDICi
Scientific applications are often structured as workflows that execute a series of distributed software modules to analyze large data sets. Such workflows are typically constructed using general-purpose scripting languages to coordinate the execution of the various modules and to exchange data sets between them. While such scripts provide a cost-effective approach for simple workflows, as the workflow structure becomes complex and evolves, the scripts quickly become complex and difficult to modify. This makes them a major barrier to easily and quickly deploying new algorithms and exploiting new, scalable hardware platforms. In this paper, we describe the MeDICi Workflow technology that is specifically designed to reduce the complexity of workflow application development, and to efficiently handle data intensive workflow applications. MeDICi integrates standard component-based and service-based technologies, and employs an efficient integration mechanism to ensure large data sets can be efficiently processed. We illustrate the use of MeDICi with a climate data processing example that we have built, and describe some of the new features
- Research Organization:
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-76RL01830
- OSTI ID:
- 1011819
- Report Number(s):
- PNNL-SA-65181; TRN: US201109%%629
- Resource Relation:
- Conference: Component Based Software Engineering: 12th International Symposium (CBSE 2009), June 24-26, 2009, East Stroudsburg, PA. Lecture Notes in Computer Science, 5582:227-241
- Country of Publication:
- United States
- Language:
- English
Similar Records
Middleware Case Study: MeDICi
Kepler + MeDICi