Locality-Aware Parallel Process Mapping for Multi-Core HPC Systems

Conference · Sat Jan 01 04:00:00 EST 2011

OSTI ID:1024716

Hursey, Joshua J ^[1]; Squyres, Jeffrey M. ^[2]; Dontje, Terry ^[3]

---

1. ORNL
2. Cisco Systems, Inc.
3. Oracle

High Performance Computing (HPC) systems are composed of servers containing an ever-increasing number of cores. With such high processor core counts, non-uniform memory access (NUMA) architectures are almost universally used to reduce inter-processor and memory communication bottlenecks by distributing processors and memory throughout a server-internal networking topology. Application studies have shown that the tuning of processes placement in a server s NUMA networking topology to the application can have a dramatic impact on performance. The performance implications are magnified when running a parallel job across multiple server nodes, especially with large scale HPC applications. This paper presents the Locality-Aware Mapping Algorithm (LAMA) for distributing the individual processes of a parallel application across processing resources in an HPC system, paying particular attention to the internal server NUMA topologies. The algorithm is able to support both homogeneous and heterogeneous hardware systems, and dynamically adapts to the available hardware and user-specified process layout at run-time. As implemented in Open MPI, the LAMA provides 362,880 mapping permutations and is able to naturally scale out to additional hardware resources as they become available in future architectures.

• Research Organization: Oak Ridge National Laboratory (ORNL); Center for Computational Sciences
• Sponsoring Organization: SC USDOE - Office of Science (SC)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1024716
• Country of Publication: United States
• Language: English

Related Subjects

99 GENERAL AND MISCELLANEOUS
ALGORITHMS
COMMUNICATIONS
COMPUTER NETWORKS
COMPUTERS
PERFORMANCE
PROCESSING
TOPOLOGY
TUNING