Unraveling Network-induced Memory Contention: Deeper Insights with Machine Learning

Groves, Taylor Liles; Grant, Ryan; Gonzales, Aaron; Arnold, Dorian

doi:10.1109/tpds.2017.2773483

Title: Unraveling Network-induced Memory Contention: Deeper Insights with Machine Learning

Full Record
Other Related Research

Abstract

Remote Direct Memory Access (RDMA) is expected to be an integral communication mechanism for future exascale systems enabling asynchronous data transfers, so that applications may fully utilize CPU resources while simultaneously sharing data amongst remote nodes. We examine Network-induced Memory Contention (NiMC) on Infiniband networks. We expose the interactions between RDMA, main-memory and cache, when applications and out-of-band services compete for memory resources. We then explore NiMCs resulting impact on application-level performance. For a range of hardware technologies and HPC workloads, we quantify NiMC and show that NiMCs impact grows with scale resulting in up to 3X performance degradation at scales as small as 8K processes even in applications that previously have been shown to be performance resilient in the presence of noise. In addition, this work examines the problem of predicting NiMC's impact on applications by leveraging machine learning and easily accessible performance counters. This approach provides additional insights about the root cause of NiMC and facilitates dynamic selection of potential solutions. Finally, we evaluated three potential techniques to reduce NiMCs impact, namely hardware offloading, core reservation and network throttling.

Authors:

Groves, Taylor Liles ^[1]; Grant, Ryan ^[2]; Gonzales, Aaron ^[3]; Arnold, Dorian ^[4]

Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Univ. of New Mexico, Albuquerque, NM (United States)
Emory Univ., Atlanta, GA (United States)

Publication Date:: Tue Nov 21 00:00:00 EST 2017

Research Org.:: Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Org.:: USDOE National Nuclear Security Administration (NNSA)

OSTI Identifier:: 1411596

Report Number(s):: SAND-2017-2071J
Journal ID: ISSN 1045-9219; 651197; TRN: US1800239

Grant/Contract Number:: AC04-94AL85000

Resource Type:: Accepted Manuscript

Journal Name:: IEEE Transactions on Parallel and Distributed Systems

Additional Journal Information:: Journal Volume: 29; Journal Issue: 8; Journal ID: ISSN 1045-9219

Publisher:: IEEE

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; measurement; performance; memory contention; networks; asynchronous communication; machine learning

Citation Formats


                    Groves, Taylor Liles, Grant, Ryan, Gonzales, Aaron, and Arnold, Dorian. Unraveling Network-induced Memory Contention: Deeper Insights with Machine Learning.  United States: N. p., 2017. 
Web.  doi:10.1109/tpds.2017.2773483.

Copy to clipboard


                    Groves, Taylor Liles, Grant, Ryan, Gonzales, Aaron, & Arnold, Dorian. Unraveling Network-induced Memory Contention: Deeper Insights with Machine Learning.  United States.  https://doi.org/10.1109/tpds.2017.2773483

Copy to clipboard


                    Groves, Taylor Liles, Grant, Ryan, Gonzales, Aaron, and Arnold, Dorian. Tue .  
"Unraveling Network-induced Memory Contention: Deeper Insights with Machine Learning".  United States.  https://doi.org/10.1109/tpds.2017.2773483.  https://www.osti.gov/servlets/purl/1411596.

Copy to clipboard


                    
@article{osti_1411596,

  title        = {Unraveling Network-induced Memory Contention: Deeper Insights with Machine Learning},

  author       = {Groves, Taylor Liles and Grant, Ryan and Gonzales, Aaron and Arnold, Dorian},

  abstractNote = {Remote Direct Memory Access (RDMA) is expected to be an integral communication mechanism for future exascale systems enabling asynchronous data transfers, so that applications may fully utilize CPU resources while simultaneously sharing data amongst remote nodes. We examine Network-induced Memory Contention (NiMC) on Infiniband networks. We expose the interactions between RDMA, main-memory and cache, when applications and out-of-band services compete for memory resources. We then explore NiMCs resulting impact on application-level performance. For a range of hardware technologies and HPC workloads, we quantify NiMC and show that NiMCs impact grows with scale resulting in up to 3X performance degradation at scales as small as 8K processes even in applications that previously have been shown to be performance resilient in the presence of noise. In addition, this work examines the problem of predicting NiMC's impact on applications by leveraging machine learning and easily accessible performance counters. This approach provides additional insights about the root cause of NiMC and facilitates dynamic selection of potential solutions. Finally, we evaluated three potential techniques to reduce NiMCs impact, namely hardware offloading, core reservation and network throttling.},

  doi          = {10.1109/tpds.2017.2773483},

  journal      = {IEEE Transactions on Parallel and Distributed Systems},

  number       = 8,

  volume       = 29,

  place        = {United States},

  year         = {Tue Nov 21 00:00:00 EST 2017},

  month        = {Tue Nov 21 00:00:00 EST 2017}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1109/tpds.2017.2773483

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 4 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Software-Driven Network Architecture for Synchronous Data Acquisition

Technical Report McMillian, Gary ; McMillian, Brett ; DeWitt, Michael

Experiments at facilities such as the Facility for Rare Isotope Beams (FRIB) involve thousands of detector elements that produce raw experimental data at rates in excess of a GB/sec using new Flash ADC (FADC) based detectors. Data acquisition systems convert detector signals into a digital form in real-time. The data from each detector element is labeled with a precisely synchronized timestamp and transmitted to buffers. The total charge, the number of coincident elements, or other information summaries are used to determine whether something interesting has happened, which constitutes a trigger. If justified by the trigger, the data from these elementsmore »« less
Exploring the All-to-All Collective Optimization Space with ConnectX CORE-Direct

Conference Venkata, Manjunath Gorentla ; Graham, Richard L. ; Ladd, Joshua ; ... - 2012 41st International Conference on Parallel Processing; 10-13 Sept. 2012; Pittsburgh, PA, USA

The all-to-all collective communication operation is used by many scientific applications, and is one of the most time consuming and challenging collective operation to optimize. The algorithms for all-to-all operations typically fall into two classes, logarithmic and linear scaling algorithms, with Bruck's algorithm, a logarithmic scaling algorithm, used in many small-data all-to-all implementations. The recent addition of InfiniBand CORE-Direct support for network management of collective communications offers new opportunities for optimizing all-to-all operation as well as supporting truly asynchronous implementations of these operations. This paper presents several new enhancements to the Bruck small-data algorithm that leverage CORE-Direct and other InfiniBandmore »« less
https://doi.org/10.1109/ICPP.2012.28
Data Locality Enhancement of Dynamic Simulations for Exascale Computing (Final Report)

Technical Report Shen, Xipeng

The development of modern processors exhibits two trends that complicate the optimizations of modern software. The first is the increasing sensitivity of processors' throughput to irregularities in computation. With more processors produced through a massive integration of simple cores, future systems will increasingly favor regular data-level parallel computations, but deviate from the needs of applications with complex patterns. Some evidences are already shown on Graphic Processing Units (GPU): Irregular data accesses (e.g., indirect references A[D[i]]) and conditional branches are limiting many GPU applications' performance at a level an order of magnitude lower than the peak of GPU. The second hardwaremore »« less
https://doi.org/10.2172/1576175

Full Text Available
Proactive Data Containers for Scientific Storage (Final Report)

Technical Report Soumagne, Jerome ; Warren, Richard ; Mu, Jingqing ; ...

Emerging HPC systems are expected to be deployed with an unprecedented level of complexity, due to a deep system memory/storage hierarchy and heterogeneity of the storage hardware. This hierarchy is expected to range from CPU cache through several levels of volatile memory to nonvolatile memory, traditional hard disks, and tape. Simple and efficient methods of data management and movement through this hierarchy is critical for scientific applications using exascale systems. Existing storage system and I/O (SSIO) technologies face severe challenges in dealing with these requirements. POSIX and MPI I/O standards that are the basis for existing I/O libraries and parallelmore »« less
https://doi.org/10.2172/1577855

Full Text Available
Dynamic Time-Variant Connection Management for PGAS Models on InfiniBand

Conference Vishnu, Abhinav ; Krishnan, Manoj Kumar ; Balaji, Pavan

InfiniBand (IB) has established itself as a promising network infrastructure for high-end cluster computing systems as evidenced by its usage in the Top500 supercomputers today. While the IB standard describes multiple communication models (including reliable-connection (RC), and unreliable datagram (UD)), most of its promising features such as remote direct memory access (RDMA), hardware atomics and network fault tolerance are only available for the RC model which requires connections between communicating process pairs. In the past, several researchers have proposed on-demand connection management techniques that establish connections when there is a need to communicate, and not before. While such techniques workmore »« less

Similar Records