Characterizing output bottlenecks in a supercomputer

Conference · Sat Dec 31 23:00:00 EST 2011

OSTI ID:1096349

Xie, Bing ^[1]; Chase, Jeffrey ^[1]; Dillow, David A ^[2]; Drokin, Oleg ^[3]; Klasky, Scott A ^[2]; Oral, H Sarp ^[2]; Podhorszki, Norbert ^[2]

---

1. Duke University
2. ORNL
3. Intel Corporation

Supercomputer I/O loads are often dominated by writes. HPC (High Performance Computing) file systems are designed to absorb these bursty outputs at high bandwidth through massive parallelism. However, the delivered write bandwidth often falls well below the peak. This paper characterizes the data absorption behavior of a center-wide shared Lustre parallel file system on the Jaguar supercomputer. We use a statistical methodology to address the challenges of accurately measuring a shared machine under production load and to obtain the distribution of bandwidth across samples of compute nodes, storage targets, and time intervals. We observe and quantify limitations from competing traffic, contention on storage servers and I/O routers, concurrency limitations in the client compute node operating systems, and the impact of variance (stragglers) on coupled output such as striping. We then examine the implications of our results for application performance and the design of I/O middleware systems on shared supercomputers.

• Research Organization: Oak Ridge National Laboratory (ORNL); Center for Computational Sciences
• Sponsoring Organization: SC USDOE - Office of Science (SC)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1096349
• Country of Publication: United States
• Language: English