Efficient Bulk Data Replication for the Earth System Grid

Conference · Wed Mar 10 00:00:00 EST 2010

OSTI ID:986313

Sim, Alex ^[1]; Gunter, Dan ^[1]; Natarajan, Vijaya ^[1]; Shoshani, Arie ^[1]; Williams, Dean ^[2]; Long, Jeff ^[2]; Hick, Jason ^[3]; Lee, Jason ^[3]; Dart, Eli ^[4]

---

1. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
2. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
3. National Energy Research Scientific Computing Center, Berkeley, CA (United States)
4. Energy Sciences Network, Berkeley, CA (United States)

The Earth System Grid (ESG) community faces the difficult challenge of managing the distribution of massive data sets to thousands of scientists around the world. To move data replicas efficiently, the ESG has developed a data transfer management tool called the Bulk Data Mover (BDM). We describe the performance results of the current system and plans towards extending the techniques developed so far for the up- coming project, in which the ESG will employ advanced networks to move multi-TB datasets with the ulti- mate goal of helping researchers understand climate change and its potential impacts on world ecology and society.

• Research Organization: Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
• Sponsoring Organization: USDOE; Computational Research Division
• DOE Contract Number: AC02-05CH11231
• OSTI ID: 986313
• Report Number(s): LBNL-3821E; TRN: US201017%%197
• Resource Relation: Conference: International Symposium on Grid Computing,
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
CLIMATES
DISTRIBUTION
ECOLOGY
MANAGEMENT
PERFORMANCE
REPLICAS
Earth System Grid
climate data
Bulk Data Movement
data transfer
NetLogger