Invited Paper: Benchmarking and Optimizing Data Movement on Emerging Heterogeneous Architectures

Bienz, Amanda

doi:10.1109/ipdpsw59300.2023.00104

Invited Paper: Benchmarking and Optimizing Data Movement on Emerging Heterogeneous Architectures

Conference · 04 August 2023 · 2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)

DOI:https://doi.org/10.1109/ipdpsw59300.2023.00104· OSTI ID:2205621

Bienz, Amanda ^[1]

Univ. of New Mexico, Albuquerque, NM (United States)

As supercomputers evolve, nodes are continually increasing in complexity. As a result, each generation of parallel systems brings new performance challenges. For instance, on recent systems inter-node communication has outperformed inter-socket, resulting in poor performance of many node-aware communication optimizations. Communication optimizations are critical for the performance and scalability of parallel applications, but are dependent on the parallel architecture, which varies significantly among recent generations of supercomputers. Furthermore, this paper investigates the performance of various paths of data movement on recent generations of systems, and analyzes the increased complexity of communication, particularly on recent heterogeneous systems. The paper also introduces MPI Advance, a communication library that enables optimizations to be created based on benchmark analysis of each emerging system.

🛈

OSTI does not have a digital full text copy available. For more information, please see document availability, search WorldCat, or search Google Scholar.

Research Organization:: Univ. of New Mexico, Albuquerque, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA)

DOE Contract Number:: NA0003966

OSTI ID:: 2205621

Journal Information:: 2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Conference: 2023 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), St. Petersburg, FL (United States), 15-19 May 2023

Country of Publication:: United States

Language:: English

References (18)

Node-Aware Improvements to Allreduce Bienz, Amanda; Olson, Luke; Gropp, William 2019 IEEE/ACM Workshop on Exascale MPI (ExaMPI) https://doi.org/10.1109/ExaMPI49596.2019.00008	conference	November 2019
MPI Collectives and Datatypes for Hierarchical All-to-all Communication Träff, Jesper Larsson; Rougier, Antoine Proceedings of the 21st European MPI Users' Group Meeting https://doi.org/10.1145/2642769.2642770	conference	September 2014
Designing a Highly-Scalable Operating System: The Blue Gene/L Story Moreira, Jose; Brutman, Michael; Castanos, Jose ACM/IEEE SC 2006 Conference (SC'06) https://doi.org/10.1109/SC.2006.23	conference	November 2006
A Locality-Aware Bruck Allgather Bienz, Amanda; Gautam, Shreeman; Kharel, Amun Proceedings of the 29th European MPI Users' Group Meeting https://doi.org/10.1145/3555819.3555825	conference	September 2022
Hierarchical Collectives in MPICH2 Zhu, Hao; Goodell, David; Gropp, William Recent Advances in Parallel Virtual Machine and Message Passing Interface https://doi.org/10.1007/978-3-642-03770-2_41	book	January 2009
Optimization of Collective Reduction Operations Rabenseifner, Rolf Computational Science - ICCS 2004 https://doi.org/10.1007/978-3-540-24685-5_1	book	January 2004
Improving Performance Models for Irregular Point-to-Point Communication Bienz, Amanda; Gropp, William D.; Olson, Luke N. EuroMPI'18: 25th European MPI Users' Group Meeting, Proceedings of the 25th European MPI Users' Group Meeting https://doi.org/10.1145/3236367.3236368	conference	September 2018
Exascale science: the next frontier in high performance computing Pawlowski, Stephen S. Proceedings of the 24th ACM International Conference on Supercomputing - ICS '10 https://doi.org/10.1145/1810085.1810087	conference	January 2010
The IBM Blue Gene/Q Compute Chip Haring, Ruud; Ohmacht, Martin; Fox, Thomas IEEE Micro, Vol. 32, Issue 2 https://doi.org/10.1109/MM.2011.108	journal	March 2012
Kernel-assisted and topology-aware MPI collective communications on multicore/many-core platforms Ma, Teng; Bosilca, George; Bouteiller, Aurelien Journal of Parallel and Distributed Computing, Vol. 73, Issue 7 https://doi.org/10.1016/j.jpdc.2013.01.015	journal	July 2013
Designing topology-aware collective communication algorithms for large scale InfiniBand clusters: Case studies with Scatter and Gather Kandalla, Krishna; Subramoni, Hari; Vishnu, Abhinav 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW) https://doi.org/10.1109/IPDPSW.2010.5470853	conference	April 2010
Decomposing MPI Collectives for Exploiting Multi-lane Communication Traff, Jesper Larsson; Hunold, Sascha 2020 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER49012.2020.00037	conference	September 2020
Stepping up to Summit Hines, Jonathan Computing in Science & Engineering, Vol. 20, Issue 2 https://doi.org/10.1109/MCSE.2018.021651341	journal	March 2018
Efficient algorithms for all-to-all communications in multiport message-passing systems Bruck, J.; Kipnis, S. IEEE Transactions on Parallel and Distributed Systems, Vol. 8, Issue 11 https://doi.org/10.1109/71.642949	journal	January 1997
The Exascale Era is Upon Us: The Frontier supercomputer may be the first to reach 1,000,000,000,000,000,000 operations per second Schneider, David IEEE Spectrum, Vol. 59, Issue 1 https://doi.org/10.1109/MSPEC.2022.9676353	journal	January 2022
Optimization of Collective Communication Operations in MPICH Thakur, Rajeev; Rabenseifner, Rolf; Gropp, William The International Journal of High Performance Computing Applications, Vol. 19, Issue 1 https://doi.org/10.1177/1094342005051521	journal	February 2005
MPI collective communications on the blue gene/p supercomputer: algorithms and optimizations Faraj, Ahmad; Kumar, Sameer; Smih, Brian Proceedings of the 23rd international conference on Conference on Supercomputing - ICS '09 https://doi.org/10.1145/1542275.1542344	conference	January 2009
Modeling MPI Communication Performance on SMP Nodes: Is it Time to Retire the Ping Pong Test Gropp, William; Olson, Luke N.; Samfass, Philipp Proceedings of the 23rd European MPI Users' Group Meeting on - EuroMPI 2016 https://doi.org/10.1145/2966884.2966919	conference	January 2016

Similar Records

Modeling Data Movement Performance on Heterogeneous Architectures

Journal Article · 2021 · IEEE Conference on High Performance Extreme Computing (Online) · OSTI ID:1846269

Bienz, Amanda; Olson, Luke N.; Gropp, William D.; +1 more

Characterizing the performance of node-aware strategies for irregular point-to-point communication on heterogeneous architectures

Journal Article · 2023 · Parallel Computing · OSTI ID:1976144

Lockhart, Shelby; Bienz, Amanda; Gropp, William D.; +1 more

Related Subjects

HPC
benchmarking
communication
heterogeneous
scalability
exascale
performance

Invited Paper: Benchmarking and Optimizing Data Movement on Emerging Heterogeneous Architectures

Citation Formats

References (18)

Similar Records

Related Subjects