skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Communication Characterization and Optimization of Applications Using Topology-Aware Task Mapping on Large Supercomputers

Abstract

On large supercomputers, the job scheduling systems may assign a non-contiguous node allocation for user applications depending on available resources. With parallel applications using MPI (Message Passing Interface), the default process ordering does not take into account the actual physical node layout available to the application. This contributes to non-locality in terms of physical network topology and impacts communication performance of the application. In order to mitigate such performance penalties, this work describes techniques to identify suitable task mapping that takes the layout of the allocated nodes as well as the application's communication behavior into account. During the first phase of this research, we instrumented and collected performance data to characterize communication behavior of critical US DOE (United States - Department of Energy) applications using an augmented version of the mpiP tool. Subsequently, we developed several reordering methods (spectral bisection, neighbor join tree etc.) to combine node layout and application communication data for optimized task placement. We developed a tool called mpiAproxy to facilitate detailed evaluation of the various reordering algorithms without requiring full application executions. This work presents a comprehensive performance evaluation (14,000 experiments) of the various task mapping techniques in lowering communication costs on Titan, the leadership classmore » supercomputer at Oak Ridge National Laboratory.« less

Authors:
 [1];  [1];  [1];  [1]
  1. ORNL
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1261293
DOE Contract Number:  
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: ACM/SPEC International Conference on Performance Engineering, Delft, Netherlands, 20160312, 20160318
Country of Publication:
United States
Language:
English

Citation Formats

Sreepathi, Sarat, D'Azevedo, Eduardo, Philip, Bobby, and Worley, Patrick H. Communication Characterization and Optimization of Applications Using Topology-Aware Task Mapping on Large Supercomputers. United States: N. p., 2016. Web.
Sreepathi, Sarat, D'Azevedo, Eduardo, Philip, Bobby, & Worley, Patrick H. Communication Characterization and Optimization of Applications Using Topology-Aware Task Mapping on Large Supercomputers. United States.
Sreepathi, Sarat, D'Azevedo, Eduardo, Philip, Bobby, and Worley, Patrick H. Fri . "Communication Characterization and Optimization of Applications Using Topology-Aware Task Mapping on Large Supercomputers". United States. https://www.osti.gov/servlets/purl/1261293.
@article{osti_1261293,
title = {Communication Characterization and Optimization of Applications Using Topology-Aware Task Mapping on Large Supercomputers},
author = {Sreepathi, Sarat and D'Azevedo, Eduardo and Philip, Bobby and Worley, Patrick H},
abstractNote = {On large supercomputers, the job scheduling systems may assign a non-contiguous node allocation for user applications depending on available resources. With parallel applications using MPI (Message Passing Interface), the default process ordering does not take into account the actual physical node layout available to the application. This contributes to non-locality in terms of physical network topology and impacts communication performance of the application. In order to mitigate such performance penalties, this work describes techniques to identify suitable task mapping that takes the layout of the allocated nodes as well as the application's communication behavior into account. During the first phase of this research, we instrumented and collected performance data to characterize communication behavior of critical US DOE (United States - Department of Energy) applications using an augmented version of the mpiP tool. Subsequently, we developed several reordering methods (spectral bisection, neighbor join tree etc.) to combine node layout and application communication data for optimized task placement. We developed a tool called mpiAproxy to facilitate detailed evaluation of the various reordering algorithms without requiring full application executions. This work presents a comprehensive performance evaluation (14,000 experiments) of the various task mapping techniques in lowering communication costs on Titan, the leadership class supercomputer at Oak Ridge National Laboratory.},
doi = {},
url = {https://www.osti.gov/biblio/1261293}, journal = {},
number = ,
volume = ,
place = {United States},
year = {2016},
month = {1}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: