Balancing Performance and Portability with Containers in HPC: An OpenSHMEM Example

Conference · Tue Jan 01 00:00:00 EST 2019

DOI:https://doi.org/10.1007/978-3-319-73814-7_9· OSTI ID:1468263

Naughton III, Thomas  ^[1]; Sorrillo, Lawrence  ^[1]; Simpson, Adam B.  ^[1]; Imam, Neena  ^[1]

---

1. ORNL

There is a growing interest in using Linux containers to streamline software development and application deployment. A container enables the user to bundle the salient elements of the software stack from an application’s perspective. In this paper, we discuss initial experiences in using the Open MPI implementation of OpenSHMEM with containers on HPC resources. We provide a brief overview of two container runtimes, Docker & Singularity, highlighting elements that are of interest for HPC users. The Docker platform offers a rich set of services that are widely used in enterprise environments, whereas Singularity is an emerging container runtime that is specifically written for use on HPC systems. We describe our procedure for container assembly and deployment that strives to maintain the portability of the container-based application. We show performance results for the Graph500 benchmark running along the typical continuum of development testbed up to full production supercomputer (ORNL’s Titan). The results show consistent performance between the native and Singularity (container) tests. The results also showed an unexplained drop in performance when using the Cray Gemini network with Open MPI’s OpenSHMEM, which was unrelated to the container usage.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1468263
• Resource Relation: Journal Volume: 10679; Conference: OpenSHMEM 2017: Fourth workshop on OpenSHMEM and Related Technologies - Annapolis, Maryland, United States of America - 8/7/2017 8:00:00 AM-8/9/2017 8:00:00 AM
• Country of Publication: United States
• Language: English