A Network-Aware Distributed Storage Cache for Data Intensive Environments
Modern scientific computing involves organizing, moving, visualizing, and analyzing massive amounts of data at multiple sites around the world. The technologies, the middleware services, and the architectures that are used to build useful high-speed, wide area distributed systems, constitute the field of data intensive computing. In this paper the authors describe an architecture for data intensive applications where they use a high-speed distributed data cache as a common element for all of the sources and sinks of data. This cache-based approach provides standard interfaces to a large, application-oriented, distributed, on-line, transient storage system. They describe their implementation of this cache, how they have made it network aware, and how they do dynamic load balancing based on the current network conditions. They also show large increases in application throughput by access to knowledge of the network conditions.
- Research Organization:
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
- Sponsoring Organization:
- USDOE Director, Office of Science. Office of Advanced Scientific Computing Research; Defense Advanced Research Project Agency (US)
- DOE Contract Number:
- AC03-76SF00098
- OSTI ID:
- 764335
- Report Number(s):
- LBNL-42896; R&D Project: 429650; TRN: AH200033%%293
- Resource Relation:
- Other Information: PBD: 23 Dec 1999
- Country of Publication:
- United States
- Language:
- English
Similar Records
Data Locality Enhancement of Dynamic Simulations for Exascale Computing (Final Report)
Using High-Speed WANs and Network Data Caches to Enable Remote and Distributed Visualization