Compiler-Directed File Layout Optimization for Hierarchical Storage Systems

Ding, Wei; Zhang, Yuanrui; Kandemir, Mahmut; Son, Seung Woo

doi:10.1155/2013/167581

Title: Compiler-Directed File Layout Optimization for Hierarchical Storage Systems

Full Record
Other Related Research

Abstract

File layout of array data is a critical factor that effects the behavior of storage caches, and has so far taken not much attention in the context of hierarchical storage systems. The main contribution of this paper is a compiler-driven file layout optimization scheme for hierarchical storage caches. This approach, fully automated within an optimizing compiler, analyzes a multi-threaded application code and determines a file layout for each disk-resident array referenced by the code, such that the performance of the target storage cache hierarchy is maximized. We tested our approach using 16 I/O intensive application programs and compared its performance against two previously proposed approaches under different cache space management schemes. Our experimental results show that the proposed approach improves the execution time of these parallel applications by 23.7% on average.

Authors:

Ding, Wei ^[1]; Zhang, Yuanrui ^[2]; Kandemir, Mahmut ^[1]; Son, Seung Woo ^[3]

The Pennsylvania State University, University Park, PA, USA
Intel Corporation, University Park, PA, USA
Northwestern University, Evanston, IL, USA

Publication Date:: 2013-01-01

Sponsoring Org.:: USDOE

OSTI Identifier:: 1227907

Grant/Contract Number:: SC0002156

Resource Type:: Published Article

Journal Name:: Scientific Programming

Additional Journal Information:: Journal Name: Scientific Programming Journal Volume: 21 Journal Issue: 3-4; Journal ID: ISSN 1058-9244

Publisher:: Hindawi Publishing Corporation

Country of Publication:: Egypt

Language:: English

Citation Formats


                    Ding, Wei, Zhang, Yuanrui, Kandemir, Mahmut, and Son, Seung Woo. Compiler-Directed File Layout Optimization for Hierarchical Storage Systems.  Egypt: N. p., 2013. 
Web.  doi:10.1155/2013/167581.

Copy to clipboard


                    Ding, Wei, Zhang, Yuanrui, Kandemir, Mahmut, & Son, Seung Woo. Compiler-Directed File Layout Optimization for Hierarchical Storage Systems.  Egypt.  https://doi.org/10.1155/2013/167581

Copy to clipboard


                    Ding, Wei, Zhang, Yuanrui, Kandemir, Mahmut, and Son, Seung Woo. Tue .  
"Compiler-Directed File Layout Optimization for Hierarchical Storage Systems".  Egypt.  https://doi.org/10.1155/2013/167581.

Copy to clipboard


                    
@article{osti_1227907,

  title        = {Compiler-Directed File Layout Optimization for Hierarchical Storage Systems},

  author       = {Ding, Wei and Zhang, Yuanrui and Kandemir, Mahmut and Son, Seung Woo},

  abstractNote = {File layout of array data is a critical factor that effects the behavior of storage caches, and has so far taken not much attention in the context of hierarchical storage systems. The main contribution of this paper is a compiler-driven file layout optimization scheme for hierarchical storage caches. This approach, fully automated within an optimizing compiler, analyzes a multi-threaded application code and determines a file layout for each disk-resident array referenced by the code, such that the performance of the target storage cache hierarchy is maximized. We tested our approach using 16 I/O intensive application programs and compared its performance against two previously proposed approaches under different cache space management schemes. Our experimental results show that the proposed approach improves the execution time of these parallel applications by 23.7% on average.},

  doi          = {10.1155/2013/167581},

  journal      = {Scientific Programming},

  number       = 3-4,

  volume       = 21,

  place        = {Egypt},

  year         = {2013},

  month        = {1}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Publisher's Version of Record
https://doi.org/10.1155/2013/167581

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Final Report on Data Locality Enhancement of Dynamic Simulations for Exascale Computing

Technical Report Shen, Xipeng

The development of modern processors exhibits two trends that complicate the optimizations of modern software. The first is the increasing sensitivity of processors' throughput to irregularities in computation. With more processors produced through a massive integration of simple cores, future systems will increasingly favor regular data-level parallel computations, but deviate from the needs of applications with complex patterns. Some evidences are already shown on Graphic Processing Units (GPU): Irregular data accesses (e.g., indirect references A[D[i]]) and conditional branches are limiting many GPU applications' performance at a level an order of magnitude lower than the peak of GPU. The second hardwaremore »« less
https://doi.org/10.2172/1576175

Full Text Available
An Optimizing Compiler for Petascale I/O on Leadership-Class Architectures

Technical Report Kandemir, Mahmut Taylan ; Choudary, Alok ; Thakur, Rajeev

In high-performance computing (HPC), parallel I/O architectures usually have very complex hierarchies with multiple layers that collectively constitute an I/O stack, including high-level I/O libraries such as PnetCDF and HDF5, I/O middleware such as MPI-IO, and parallel file systems such as PVFS and Lustre. Our DOE project explored automated instrumentation and compiler support for I/O intensive applications. Our project made significant progress towards understanding the complex I/O hierarchies of high-performance storage systems (including storage caches, HDDs, and SSDs), and designing and implementing state-of-the-art compiler/runtime system technology that targets I/O intensive HPC applications that target leadership class machine. This final reportmore »« less
https://doi.org/10.2172/1123486

Full Text Available
A Fault-oblivious Extreme-scale Execution Environment

Technical Report Sadayappan, Ponnuswamy

Exascale computing systems will provide a thousand-fold increase in parallelism and a proportional increase in failure rate relative to today's machines. Systems software for exascale machines must provide the infrastructure to support existing applications while simultaneously enabling efficient execution of new programming models that naturally express dynamic, adaptive, irregular computation; coupled simulations; and massive data analysis in a highly unreliable hardware environment with billions of threads of execution. We propose a new approach to the data and work distribution model provided by system software based on the unifying formalism of an abstract file system. The proposed hierarchical data model providesmore »« less
https://doi.org/10.2172/1307122

Full Text Available
Dynamic Non-Hierarchical File Systems for Exascale Storage

Technical Report Long, Darrell E. ; Miller, Ethan L

This constitutes the final report for “Dynamic Non-Hierarchical File Systems for Exascale Storage”. The ultimate goal of this project was to improve data management in scientific computing and high-end computing (HEC) applications, and to achieve this goal we proposed: to develop the first, HEC-targeted, file system featuring rich metadata and provenance collection, extreme scalability, and future storage hardware integration as core design goals, and to evaluate and develop a flexible non-hierarchical file system interface suitable for providing more powerful and intuitive data management interfaces to HEC and scientific computing users. Data management is swiftly becoming a serious problem in themore »« less
https://doi.org/10.2172/1170868

Full Text Available
Compiler generation and autotuning of communication-avoiding operators for geometric multigrid

Conference Basu, Protonu ; Venkat, Anand ; Hall, Mary ; ...

This paper describes a compiler approach to introducing communication-avoiding optimizations in geometric multigrid (GMG), one of the most popular methods for solving partial differential equations. Communication-avoiding optimizations reduce vertical communication through the memory hierarchy and horizontal communication across processes or threads, usually at the expense of introducing redundant computation. We focus on applying these optimizations to the smooth operator, which successively reduces the error and accounts for the largest fraction of the GMG execution time. Our compiler technology applies both novel and known transformations to derive an implementation comparable to manually-tuned code. To make the approach portable, an underlying autotuningmore »« less
https://doi.org/10.1109/HiPC.2013.6799131

Full Text Available

Similar Records