skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Memory Compression Techniques for Network Address Management in MPI

Abstract

MPI allows applications to treat processes as a logical collection of integer ranks for each MPI communicator, while internally translating these logical ranks into actual network addresses. In current MPI implementations the management and lookup of such network addresses use memory sizes that are proportional to the number of processes in each communicator. In this paper, we propose a new mechanism, called AV-Rankmap, for managing such translation. AV-Rankmap takes advantage of logical patterns in rank-address mapping that most applications naturally tend to have, and it exploits the fact that some parts of network address structures are naturally more performance critical than others. It uses this information to compress the memory used for network address management. We demonstrate that AV-Rankmap can achieve performance similar to or better than that of other MPI implementations while using significantly less memory.

Authors:
; ; ; ; ; ;
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1366311
DOE Contract Number:
AC02-06CH11357
Resource Type:
Conference
Resource Relation:
Conference: 31st IEEE International Parallel and Distributed Processing Symposium , 05/29/17 - 06/02/17, Orlando, FL, USA
Country of Publication:
United States
Language:
English

Citation Formats

Guo, Yanfei, Archer, Charles J., Blocksome, Michael, Parker, Scott, Bland, Wesley, Raffenetti, Ken, and Balaji, Pavan. Memory Compression Techniques for Network Address Management in MPI. United States: N. p., 2017. Web.
Guo, Yanfei, Archer, Charles J., Blocksome, Michael, Parker, Scott, Bland, Wesley, Raffenetti, Ken, & Balaji, Pavan. Memory Compression Techniques for Network Address Management in MPI. United States.
Guo, Yanfei, Archer, Charles J., Blocksome, Michael, Parker, Scott, Bland, Wesley, Raffenetti, Ken, and Balaji, Pavan. Mon . "Memory Compression Techniques for Network Address Management in MPI". United States. doi:. https://www.osti.gov/servlets/purl/1366311.
@article{osti_1366311,
title = {Memory Compression Techniques for Network Address Management in MPI},
author = {Guo, Yanfei and Archer, Charles J. and Blocksome, Michael and Parker, Scott and Bland, Wesley and Raffenetti, Ken and Balaji, Pavan},
abstractNote = {MPI allows applications to treat processes as a logical collection of integer ranks for each MPI communicator, while internally translating these logical ranks into actual network addresses. In current MPI implementations the management and lookup of such network addresses use memory sizes that are proportional to the number of processes in each communicator. In this paper, we propose a new mechanism, called AV-Rankmap, for managing such translation. AV-Rankmap takes advantage of logical patterns in rank-address mapping that most applications naturally tend to have, and it exploits the fact that some parts of network address structures are naturally more performance critical than others. It uses this information to compress the memory used for network address management. We demonstrate that AV-Rankmap can achieve performance similar to or better than that of other MPI implementations while using significantly less memory.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Mon May 29 00:00:00 EDT 2017},
month = {Mon May 29 00:00:00 EDT 2017}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: