DataSpaces: An Interaction and Coordination Framework for Coupled Simulation Workflows

Conference · Fri Jan 01 04:00:00 EST 2010

OSTI ID:982175

Docan, Ciprian ^[1]; Klasky, Scott A ^[2]; Parashar, Manish ^[2]

---

1. Rutgers University
2. ORNL

Emerging high-performance distributed computing environments are enabling new end-to-end formulations in science and engineering that involve multiple interacting processes and data-intensive application workflows. For example, current fusion simulation efforts are exploring coupled models and codes that simultaneously simulate separate application processes, such as the core and the edge turbulence, and run on different high performance computing resources. These components need to interact, at runtime, with each other and with services for data monitoring, data analysis and visualization, and data archiving. As a result, they require efficient support for dynamic and flexible couplings and interactions, which remains a challenge. This paper presents Data-Spaces, a flexible interaction and coordination substrate that addresses this challenge. DataSpaces essentially implements a semantically specialized virtual shared space abstraction that can be associatively accessed by all components and services in the application workflow. It enables live data to be extracted from running simulation components, indexes this data online, and then allows it to be monitored, queried and accessed by other components and services via the space using semantically meaningful operators. The underlying data transport is asynchronous, low-overhead and largely memory-to-memory. The design, implementation, and experimental evaluation of DataSpaces using a coupled fusion simulation workflow is presented.

• Research Organization: Oak Ridge National Laboratory (ORNL)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 982175
• Country of Publication: United States
• Language: English

Related Subjects

70 PLASMA PHYSICS AND FUSION TECHNOLOGY
97 MATHEMATICS AND COMPUTING
COMPUTERIZED SIMULATION
D CODES
DATA ANALYSIS
DATA TRANSMISSION
MEMORY MANAGEMENT
PERFORMANCE
THERMONUCLEAR REACTIONS