The Design, Deployment, and Evaluation of the CORAL Pre-Exascale Systems

Conference · Thu Nov 01 00:00:00 EDT 2018

DOI:https://doi.org/10.1109/SC.2018.00055· OSTI ID:1564142

Vazhkudai, Sudharshan  ^[1]; de Supinski, Bronis R. ^[2]; Bland, Buddy  ^[1]; Geist II, Al  ^[1]; Grinberg, Leopold ^[3]; Yin, Junqi  ^[1]; Sexton, James ^[3]; Kahle, Jim ^[3]; Zimmer, Christopher  ^[1]; Atchley, Scott  ^[1]; Oral, H  ^[1]; Maxwell, Don  ^[1]; Melesse Vergara, Veronica  ^[1]; Bertsch, Adam ^[2]; Goldstone, Robin ^[2]; Joubert, Wayne  ^[1]; Chambreau, Chris ^[2]; Applehans, David ^[3]; Blackmore, Robert ^[3]; Casses, Ben ^[2] ; Chochia, George ^[3]; Davison, Gene ^[3]; Ezell, Matthew  ^[1]; Gooding, Tom ^[3]; Gonsiorowski, Elsa ^[2]; Hanson, Bill ^[3]; Hartner, Bill ^[3]; Karlin, Ian ^[2]; Leininger, Matthew ^[2]; Leverman, Dustin B.  ^[1]; Marroquin, Chris ^[3]; Moody, Adam ^[2]; Ohmacht, Martin ^[3]; Pankajakshan, Ramesh ^[2]; Pizzano, Fernando ^[3]; Rogers II, Jim  ^[1]; Rosenburg, Bryan ^[3]; Schmidt, Drew ^[1]; Shankar, Mallikarjun (Arjun)  ^[1]; Wang, Feiyi  ^[1]; Watson, Py ^[2]; Walkup, Bob ^[3]; Weems, Lance ^[2]

---

1. ORNL
2. Lawrence Livermore National Laboratory (LLNL)
3. IBM

CORAL, the Collaboration of Oak Ridge, Argonne and Livermore, is fielding two similar IBM systems, Summit and Sierra, with NVIDIA GPUs that will replace the existing Titan and Sequoia systems. Summit and Sierra are currently ranked No. 1 and No. 3, respectively on the Top500 list. We discuss the design and key differences of the systems. Our evaluation of the systems highlights the following. Applications that fit in HBM see the most benefit and may prefer more GPUs; however, for some applications, the CPU-GPU bandwidth is more important than the number of GPUs. The node-local burst buffer scales linearly, and can achieve a 4X improvement over the parallel file system for large jobs; smaller jobs, however, may benefit from writing directly to the PFS. Finally, several CPU, network and memory bound analytics and GPU-bound deep learning codes achieve up to a 11X and 79X speedup/node, respectively over Titan.

• Research Organization: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1564142
• Resource Relation: Conference: International Conference for High Performance Computing, Networking, Storage, and Analysis (SC18) - Dallas, Texas, United States of America - 11/11/2018 4:00:00 PM-11/16/2018 10:00:00 AM
• Country of Publication: United States
• Language: English