A first look at scalable I/O in Linux commands
- ORNL
Data created from and used by terascale and petascale applications continues to increase, but our ability to handle and manage these files is still limited by the capabilities of the standard serialized Linux command set. This paper introduces the Center for Computational Sciences (NCCS) at Oak Ridge National Laboratory (ORNL) efforts towards providing parallelized and more efficient versions of the commonly used Linux commands. The design and implementation details as well as performance analysis of an in-house developed distributed parallelized version of the cp tool, spdcp is presented. Tests show that our spdcp utility can achieve 73 times more performance than its serialized counterpart. In addition, we introduce current work to extend this approach to other tools.
- Research Organization:
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). National Center for Computational Sciences (NCCS)
- Sponsoring Organization:
- USDOE Office of Science (SC)
- DOE Contract Number:
- DE-AC05-00OR22725
- OSTI ID:
- 929789
- Resource Relation:
- Conference: The 9th LCI International Conference on High-Performance Clustered Computing, Urbana- Champaign, IL, USA, 20080428, 20080501
- Country of Publication:
- United States
- Language:
- English
Similar Records
PETASCALE DATA STORAGE INSTITUTE (PDSI) Final Report
PREPARING FOR EXASCALE: ORNL Leadership Computing Application Requirements and Strategy