NUMA-Aware Thread Scheduling for Big Data Transfers over Terabits Network Infrastructure

doi:10.1155/2018/4120561

Title: NUMA-Aware Thread Scheduling for Big Data Transfers over Terabits Network Infrastructure

Abstract

The evergrowing trend of big data has led scientists to share and transfer the simulation and analytical data across the geodistributed research and computing facilities. However, the existing data transfer frameworks used for data sharing lack the capability to adopt the attributes of the underlying parallel file systems (PFS). LADS (Layout-Aware Data Scheduling) is an end-to-end data transfer tool optimized for terabit network using a layout-aware data scheduling via PFS. However, it does not consider the NUMA (Nonuniform Memory Access) architecture. In this paper, we propose a NUMA-aware thread and resource scheduling for optimized data transfer in terabit network. First, we propose distributed RMA buffers to reduce memory controller contention in CPU sockets and then schedule the threads based on CPU socket and NUMA nodes inside CPU socket to reduce memory access latency. We design and implement the proposed resource and thread scheduling in the existing LADS framework. Experimental results showed from 21.7% to 44% improvement with memory-level optimizations in the LADS framework as compared to the baseline without any optimization.

Authors:

^[1];

^[2]; Atchley, Scott ^[3]

Sogang Univ., Seoul (Korea)
Ajou Univ., Suwon (Korea)
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

Publication Date:: 2018-05-07

Research Org.:: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)

Sponsoring Org.:: USDOE Office of Science (SC)

OSTI Identifier:: 1565699

Resource Type:: Accepted Manuscript

Journal Name:: Scientific Programming

Additional Journal Information:: [ Journal Volume: 2018]; Journal ID: ISSN 1058-9244

Publisher:: Hindawi

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; Computer Science

Citation Formats


                    Kim, Taeuk, Khan, Awais, Kim, Youngjae, Kasu, Preethika, and Atchley, Scott. NUMA-Aware Thread Scheduling for Big Data Transfers over Terabits Network Infrastructure.  United States: N. p., 2018. 
        Web.  doi:10.1155/2018/4120561.

Copy to clipboard


                    Kim, Taeuk, Khan, Awais, Kim, Youngjae, Kasu, Preethika, & Atchley, Scott. NUMA-Aware Thread Scheduling for Big Data Transfers over Terabits Network Infrastructure.  United States.  doi:10.1155/2018/4120561.

Copy to clipboard


                    Kim, Taeuk, Khan, Awais, Kim, Youngjae, Kasu, Preethika, and Atchley, Scott. Mon .  
        "NUMA-Aware Thread Scheduling for Big Data Transfers over Terabits Network Infrastructure".  United States.  doi:10.1155/2018/4120561.  https://www.osti.gov/servlets/purl/1565699.

Copy to clipboard


                    
@article{osti_1565699,

  title        = {NUMA-Aware Thread Scheduling for Big Data Transfers over Terabits Network Infrastructure},

  author       = {Kim, Taeuk and Khan, Awais and Kim, Youngjae and Kasu, Preethika and Atchley, Scott},

  abstractNote = {The evergrowing trend of big data has led scientists to share and transfer the simulation and analytical data across the geodistributed research and computing facilities. However, the existing data transfer frameworks used for data sharing lack the capability to adopt the attributes of the underlying parallel file systems (PFS). LADS (Layout-Aware Data Scheduling) is an end-to-end data transfer tool optimized for terabit network using a layout-aware data scheduling via PFS. However, it does not consider the NUMA (Nonuniform Memory Access) architecture. In this paper, we propose a NUMA-aware thread and resource scheduling for optimized data transfer in terabit network. First, we propose distributed RMA buffers to reduce memory controller contention in CPU sockets and then schedule the threads based on CPU socket and NUMA nodes inside CPU socket to reduce memory access latency. We design and implement the proposed resource and thread scheduling in the existing LADS framework. Experimental results showed from 21.7% to 44% improvement with memory-level optimizations in the LADS framework as compared to the baseline without any optimization.},

  doi          = {10.1155/2018/4120561},

  journal      = {Scientific Programming},

  number       = ,

  volume       = [2018],

  place        = {United States},

  year         = {2018},

  month        = {5}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

DOI: 10.1155/2018/4120561

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Analysis of NUMA effects in modern multicore systems for the design of high-performance data transfer applications
journal, September 2017

Li, Tan; Ren, Yufei; Yu, Dantong
Future Generation Computer Systems, Vol. 74
DOI: 10.1016/j.future.2017.04.001

RAMSYS: Resource-Aware Asynchronous Data Transfer with Multicore SYStems
journal, May 2017

Li, Tan; Ren, Yufei; Yu, Dantong
IEEE Transactions on Parallel and Distributed Systems, Vol. 28, Issue 5
DOI: 10.1109/TPDS.2016.2619344

Similar Records in DOE PAGES and OSTI.GOV collections:

Optimizing End-to-End Big Data Transfers over Terabits Network Infrastructure

Journal Article Kim, Youngjae ; Atchley, Scott ; Vallee, Geoffroy R. ; ... - IEEE Transactions on Parallel and Distributed Systems

While future terabit networks hold the promise of significantly improving big-data motion among geographically distributed data centers, significant challenges must be overcome even on today's 100 gigabit networks to realize end-to-end performance. Multiple bottlenecks exist along the end-to-end path from source to sink, for instance, the data storage infrastructure at both the source and sink and its interplay with the wide-area network are increasingly the bottleneck to achieving high performance. In this study, we identify the issues that lead to congestion on the path of an end-to-end data transfer in the terabit network environment, and we present a new bulkmore »« less
Cited by 1
DOI: 10.1109/TPDS.2016.2550439

Full Text Available
FTLADS: Object-Logging Based Fault-Tolerant Big Data Transfer System Using Layout Aware Data Scheduling

Journal Article Kasu, Preethika ; Kim, Taeuk ; Um, Jung-Ho ; ... - IEEE Access

The layout-aware data scheduling (LADS) data movement framework optimizes congestion for end-to-end data transfers. During data transfer, LADS can avoid congested storage elements by exploiting the underlying storage layout at each endpoint. This improves the I/O bandwidth and hence the data transfer rate across high-speed networks. However, the absence of fault tolerance (FT) in LADS results in data retransmission overhead and may lead to possible data integrity issues upon faults. In this paper, we propose object-logging FT mechanisms to avoid transmitting the objects that are successfully written into the parallel file system (PFS) at the sink end. Depending on themore »« less
DOI: 10.1109/ACCESS.2019.2905158

Full Text Available
Hot spot analysis in large scale shared memory multiprocessors

Conference Harzallah, K ; Sevcik, K C

Scalable multiprocessors that support a shared-memory image to application programmers are typically based on physical memory modules that are distributed. Consequently, the access times for a particular processor to various parts of physical memory differ. In this paper, the authors explore the implications of this non-uniformity in memory access times. In particular, they study the effect of hot-spots in hierarchical large scale NUMA multiprocessors. Hot-spot analysis is of interest because coordinated threads of parallel programs lead to hot spots whose impact on performance may be substantial or even dominant. They have developed an analytical model of access latencies and contentionmore »« less
Characterizing and Mitigating Work Time Inflation in Task Parallel Programs

Journal Article Olivier, Stephen L. ; de Supinski, Bronis R. ; Schulz, Martin ; ... - Scientific Programming

Task parallelism raises the level of abstraction in shared memory parallel programming to simplify the development of complex applications. However, task parallel applications can exhibit poor performance due to thread idleness, scheduling overheads, and work time inflation – additional time spent by threads in a multithreaded computation beyond the time required to perform the same work in a sequential computation. We identify the contributions of each factor to lost efficiency in various task parallel OpenMP applications and diagnose the causes of work time inflation in those applications. Increased data access latency can cause significant work time inflation in NUMA systems.more »« less
DOI: 10.1155/2013/898597
Unraveling Network-induced Memory Contention: Deeper Insights with Machine Learning

Journal Article Groves, Taylor Liles ; Grant, Ryan ; Gonzales, Aaron ; ... - IEEE Transactions on Parallel and Distributed Systems

Remote Direct Memory Access (RDMA) is expected to be an integral communication mechanism for future exascale systems enabling asynchronous data transfers, so that applications may fully utilize CPU resources while simultaneously sharing data amongst remote nodes. We examine Network-induced Memory Contention (NiMC) on Infiniband networks. We expose the interactions between RDMA, main-memory and cache, when applications and out-of-band services compete for memory resources. We then explore NiMCs resulting impact on application-level performance. For a range of hardware technologies and HPC workloads, we quantify NiMC and show that NiMCs impact grows with scale resulting in up to 3X performance degradation atmore »« less
DOI: 10.1109/tpds.2017.2773483

Full Text Available

Similar Records

Title: NUMA-Aware Thread Scheduling for Big Data Transfers over Terabits Network Infrastructure

Abstract

Citation Formats

Analysis of NUMA effects in modern multicore systems for the design of high-performance data transfer applications journal, September 2017

RAMSYS: Resource-Aware Asynchronous Data Transfer with Multicore SYStems journal, May 2017

Analysis of NUMA effects in modern multicore systems for the design of high-performance data transfer applications
journal, September 2017

RAMSYS: Resource-Aware Asynchronous Data Transfer with Multicore SYStems
journal, May 2017