skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC Systems with Eley

Abstract

Here, burst buffer is an effective solution for reducing the data transfer time and the I/O interference in HPC systems. Extending Burst Buffers (BBs) to handle Big Data applications is challenging because BBs must account for the large data inputs of Big Data applications and the Quality-of-Service (QoS) of HPC applications which are considered as first-class citizens in HPC systems. Existing BBs focus on only intermediate data of Big Data applications and incur a high performance degradation of both Big Data and HPC applications. We present Eley, a burst buffer solution that helps to accelerate the performance of Big Data applications while guaranteeing the QoS of HPC applications. To achieve this goal, Eley embraces interference-aware prefetching technique that makes reading data input faster while introducing low interference for HPC applications. Evaluations using a wide range of Big Data and HPC applications demonstrate that Eley improves the performance of Big Data applications by up to 30% compared to existing BBs while maintaining the QoS of HPC applications.

Authors:
 [1];  [2];  [3]
  1. Univ. Rennes, Rennes (France); Argonne National Lab. (ANL), Lemont, IL (United States)
  2. Shenzhen Univ., Shenzhen (China)
  3. Inria, Nantes (France)
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
National Science Foundation (NSF); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21); Shenzhen University
OSTI Identifier:
1467628
Grant/Contract Number:  
AC02-06CH11357
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Future Generations Computer Systems
Additional Journal Information:
Journal Volume: 86; Journal Issue: C; Journal ID: ISSN 0167-739X
Publisher:
Elsevier
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Big Data; Burst Buffers; HPC; Interference; MapReduce; Parallel File Systems; Prefetch

Citation Formats

Yildiz, Orcun, Zhou, Amelie Chi, and Ibrahim, Shadi. Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC Systems with Eley. United States: N. p., 2018. Web. doi:10.1016/j.future.2018.03.029.
Yildiz, Orcun, Zhou, Amelie Chi, & Ibrahim, Shadi. Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC Systems with Eley. United States. doi:10.1016/j.future.2018.03.029.
Yildiz, Orcun, Zhou, Amelie Chi, and Ibrahim, Shadi. Thu . "Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC Systems with Eley". United States. doi:10.1016/j.future.2018.03.029. https://www.osti.gov/servlets/purl/1467628.
@article{osti_1467628,
title = {Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC Systems with Eley},
author = {Yildiz, Orcun and Zhou, Amelie Chi and Ibrahim, Shadi},
abstractNote = {Here, burst buffer is an effective solution for reducing the data transfer time and the I/O interference in HPC systems. Extending Burst Buffers (BBs) to handle Big Data applications is challenging because BBs must account for the large data inputs of Big Data applications and the Quality-of-Service (QoS) of HPC applications which are considered as first-class citizens in HPC systems. Existing BBs focus on only intermediate data of Big Data applications and incur a high performance degradation of both Big Data and HPC applications. We present Eley, a burst buffer solution that helps to accelerate the performance of Big Data applications while guaranteeing the QoS of HPC applications. To achieve this goal, Eley embraces interference-aware prefetching technique that makes reading data input faster while introducing low interference for HPC applications. Evaluations using a wide range of Big Data and HPC applications demonstrate that Eley improves the performance of Big Data applications by up to 30% compared to existing BBs while maintaining the QoS of HPC applications.},
doi = {10.1016/j.future.2018.03.029},
journal = {Future Generations Computer Systems},
issn = {0167-739X},
number = C,
volume = 86,
place = {United States},
year = {2018},
month = {3}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share: