skip to main content
DOE Patents title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Architecture and method for a burst buffer using flash technology

Abstract

A parallel supercomputing cluster includes compute nodes interconnected in a mesh of data links for executing an MPI job, and solid-state storage nodes each linked to a respective group of the compute nodes for receiving checkpoint data from the respective compute nodes, and magnetic disk storage linked to each of the solid-state storage nodes for asynchronous migration of the checkpoint data from the solid-state storage nodes to the magnetic disk storage. Each solid-state storage node presents a file system interface to the MPI job, and multiple MPI processes of the MPI job write the checkpoint data to a shared file in the solid-state storage in a strided fashion, and the solid-state storage node asynchronously migrates the checkpoint data from the shared file in the solid-state storage to the magnetic disk storage and writes the checkpoint data to the magnetic disk storage in a sequential fashion.

Inventors:
; ; ; ; ;
Issue Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1243041
Patent Number(s):
9,286,261
Application Number:
13/676,000
Assignee:
EMC Corporation (Hopkinton, MA) Los Alamos National Security, LLC (Los Alamos, NM)
DOE Contract Number:  
AC52-06NA25396
Resource Type:
Patent
Resource Relation:
Patent File Date: 2012 Nov 13
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Tzelnic, Percy, Faibish, Sorin, Gupta, Uday K., Bent, John, Grider, Gary Alan, and Chen, Hsing-bung. Architecture and method for a burst buffer using flash technology. United States: N. p., 2016. Web.
Copy to clipboard
Tzelnic, Percy, Faibish, Sorin, Gupta, Uday K., Bent, John, Grider, Gary Alan, & Chen, Hsing-bung. Architecture and method for a burst buffer using flash technology. United States.
Copy to clipboard
Tzelnic, Percy, Faibish, Sorin, Gupta, Uday K., Bent, John, Grider, Gary Alan, and Chen, Hsing-bung. Tue . "Architecture and method for a burst buffer using flash technology". United States. https://www.osti.gov/servlets/purl/1243041.
Copy to clipboard
@article{osti_1243041,
title = {Architecture and method for a burst buffer using flash technology},
author = {Tzelnic, Percy and Faibish, Sorin and Gupta, Uday K. and Bent, John and Grider, Gary Alan and Chen, Hsing-bung},
abstractNote = {A parallel supercomputing cluster includes compute nodes interconnected in a mesh of data links for executing an MPI job, and solid-state storage nodes each linked to a respective group of the compute nodes for receiving checkpoint data from the respective compute nodes, and magnetic disk storage linked to each of the solid-state storage nodes for asynchronous migration of the checkpoint data from the solid-state storage nodes to the magnetic disk storage. Each solid-state storage node presents a file system interface to the MPI job, and multiple MPI processes of the MPI job write the checkpoint data to a shared file in the solid-state storage in a strided fashion, and the solid-state storage node asynchronously migrates the checkpoint data from the shared file in the solid-state storage to the magnetic disk storage and writes the checkpoint data to the magnetic disk storage in a sequential fashion.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2016},
month = {3}
}
Copy to clipboard
Patent:
View Patent
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

DASH-IO: an empirical study of flash-based IO for HPC
conference, January 2010

  • He, Jiahua; Bennett, Jeffrey; Snavely, Allan
  • Proceedings of the 2010 TeraGrid Conference on - TG '10, Article No. 10
  • DOI: 10.1145/1838574.1838584

...and eat it too: high read performance in write-optimized HPC I/O middleware file formats
conference, January 2009

  • Polte, Milo; Wingate, Meghan; Wolf, Matthew
  • Proceedings of the 4th Annual Workshop on Petascale Data Storage - PDSW '09
  • DOI: 10.1145/1713072.1713079

Enhancing Checkpoint Performance with Staging IO and SSD
conference, May 2010

  • Ouyang, Xiangyong; Marcarelli, Sonya; Panda, Dhabaleswar K.
  • 2010 International Workshop on Storage Network Architecture and Parallel I/Os (SNAPI)
  • DOI: 10.1109/SNAPI.2010.10

PLFS: a checkpoint filesystem for parallel applications
conference, January 2009

An abstract-device interface for implementing portable parallel-I/O interfaces
conference, January 1996

  • Thakur, R.; Gropp, W.; Lusk, E.
  • Proceedings of 6th Symposium on the Frontiers of Massively Parallel Computation (Frontiers '96)
  • DOI: 10.1109/FMPC.1996.558080

Evaluation of active storage strategies for the lustre parallel file system
conference, January 2007

  • Piernas, Juan; Nieplocha, Jarek; Felix, Evan J.
  • Proceedings of the 2007 ACM/IEEE conference on Supercomputing - SC '07
  • DOI: 10.1145/1362622.1362660

A Cost-Effective, High Bandwidth Server I/O network Architecture for Cluster Systems
conference, March 2007

  • Chen, Hsing-bung; Grider, Gary; Fields, Parks
  • 2007 IEEE International Parallel and Distributed Processing Symposium
  • DOI: 10.1109/IPDPS.2007.370221

Integration Experiences and Performance Studies of A COTS Parallel Archive System
conference, September 2010

  • Chen, Hsing-bung; Grider, Gary; Scott, Cody
  • 2010 IEEE International Conference on Cluster Computing (CLUSTER)
  • DOI: 10.1109/CLUSTER.2010.23

PaScal-- A New Parallel and Scalable Server IO Networking Infrastructure for Supporting Global Storage/File Systems in Large-size Linux Clusters
conference, January 2006

  • Grider, G.; Nunez, J.
  • 2006 IEEE International Performance Computing and Communications Conference
  • DOI: 10.1109/.2006.1629424

Flexibility, manageability, and performance in a Grid storage appliance
conference, January 2002

  • Bent, J.; Venkataramani, V.; LeRoy, N.
  • Proceedings 11th IEEE International Symposium on High Performance Distributed Computing
  • DOI: 10.1109/HPDC.2002.1029898

Distributed-and-split data-control extension to SCSI for scalable storage area networks
conference, January 2002

Pageserver: High-Performance SSD-Based Checkpointing of Transactional Distributed Memory
conference, March 2010

  • Gerhold, Steffen; Kaemmer, Nico; Weggerle, Alexander
  • 2010 Second International Conference on Computer Engineering and Applications
  • DOI: 10.1109/ICCEA.2010.53

Hybrid checkpointing using emerging nonvolatile memories for future exascale systems
journal, July 2011

  • Dong, Xiangyu; Xie, Yuan; Muralimanohar, Naveen
  • ACM Transactions on Architecture and Code Optimization, Vol. 8, Issue 2
  • DOI: 10.1145/1970386.1970387

Evaluating the benefits of an extended memory hierarchy for parallel streamline algorithms
conference, October 2011

  • Camp, David; Childs, Hank; Chourasia, Amit
  • 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV)
  • DOI: 10.1109/LDAV.2011.6092318

Provisioning a Multi-tiered Data Staging Area for Extreme-Scale Machines
conference, June 2011

  • Prabhakar, Ramya; Vazhkudai, Sudharshan S.; Kim, Youngjae
  • 2011 31st International Conference on Distributed Computing Systems (ICDCS)
  • DOI: 10.1109/ICDCS.2011.33

Functional Partitioning to Optimize End-to-End Performance on Many-core Architectures
conference, November 2010

  • Li, Min; Vazhkudai, Sudharshan S.; Butt, Ali R.
  • 2010 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis
  • DOI: 10.1109/SC.2010.28

Managing storage space in a flash and disk hybrid storage system
conference, September 2009

  • Xiaojian Wu, ; Reddy, A. L. N.
  • amp; Simulation of Computer and Telecommunication Systems (MASCOTS), 2009 IEEE International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems
  • DOI: 10.1109/MASCOT.2009.5366764

Exploiting Concurrency to Improve Latency and throughput in a Hybrid Storage System
conference, August 2010

  • Wu, Xiaojian; Reddy, A. L. Narasimha
  • Simulation of Computer and Telecommunication Systems (MASCOTS), 2010 IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems
  • DOI: 10.1109/MASCOTS.2010.11

Umbrella file system: Storage management across heterogeneous devices
journal, March 2009

  • Garrison, John A.; Reddy, A. L. Narasimha
  • ACM Transactions on Storage, Vol. 5, Issue 1
  • DOI: 10.1145/1502777.1502780

Incorporating Network RAM and Flash into Fast Backing Store for Clusters
conference, September 2011

  • Newhall, Tia; Woos, Douglas
  • 2011 IEEE International Conference on Cluster Computing (CLUSTER)
  • DOI: 10.1109/CLUSTER.2011.22

The Conquest file system : Better performance through a disk/persistent-RAM hybrid design
journal, August 2006

  • Wang, An-I Andy; Kuenning, Geoff; Reiher, Peter
  • ACM Transactions on Storage, Vol. 2, Issue 3
  • DOI: 10.1145/1168910.1168914

Azor: Using Two-Level Block Selection to Improve SSD-Based I/O Caches
conference, July 2011

  • Klonatos, Yannis; Makatos, Thanos; Marazakis, Manolis
  • 2011 6th IEEE International Conference on Networking, Architecture, and Storage (NAS), 2011 IEEE Sixth International Conference on Networking, Architecture, and Storage
  • DOI: 10.1109/NAS.2011.50

Using Active NVRAM for Cloud I/O
conference, October 2011

  • Kannan, Sudarsun; Milojicic, Dejan; Talwar, Vanish
  • 2011 6th Open Cirrus Summit (OCS), 2011 Sixth Open Cirrus Summit
  • DOI: 10.1109/OCS.2011.12

A comprehensive study of energy efficiency and performance of flash-based SSD
journal, April 2011

  • Park, Seonyeong; Kim, Youngjae; Urgaonkar, Bhuvan
  • Journal of Systems Architecture, Vol. 57, Issue 4, p. 354-365
  • DOI: 10.1016/j.sysarc.2011.01.005

Making a case for distributed file systems at Exascale
conference, January 2011

  • Raicu, Ioan; Foster, Ian T.; Beckman, Pete
  • Proceedings of the third international workshop on Large-scale system and application performance - LSAP '11
  • DOI: 10.1145/1996029.1996034

Jitter-free co-processing on a prototype exascale storage stack
conference, April 2012

  • Bent, John; Faibish, Sorin; Ahrens, Jim
  • 2012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST)
  • DOI: 10.1109/MSST.2012.6232382

Verifying Scientific Simulations via Comparative and Quantitative Visualization
journal, November 2010

  • Ahrens, James; Heitmann, Katrin; Petersen, Mark
  • IEEE Computer Graphics and Applications, Vol. 30, Issue 6
  • DOI: 10.1109/MCG.2010.100

Scalable I/O forwarding framework for high-performance computing systems
conference, August 2009

  • Ali, Nawab; Carns, Philip; Iskra, Kamil
  • 2009 IEEE International Conference on Cluster Computing and Workshops
  • DOI: 10.1109/CLUSTR.2009.5289188

Design issues for a shingled write disk system
conference, May 2010

  • Amer, Ahmed; Long, Darrell D. E.; Miller, Ethan L.
  • 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST)
  • DOI: 10.1109/MSST.2010.5496991

The ParaView Coprocessing Library: A scalable, general purpose in situ visualization library
conference, October 2011

  • Fabian, Nathan; Moreland, Kenneth; Thompson, David
  • 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV)
  • DOI: 10.1109/LDAV.2011.6092322

Managing Variability in the IO Performance of Petascale Storage Systems
conference, November 2010

  • Lofstead, Jay; Zheng, Fang; Liu, Qing
  • 2010 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis
  • DOI: 10.1109/SC.2010.32

Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS)
conference, January 2008

  • Lofstead, Jay F.; Klasky, Scott; Schwan, Karsten
  • Proceedings of the 6th international workshop on Challenges of large applications in distributed environments - CLADE '08
  • DOI: 10.1145/1383529.1383533

Design, Modeling, and Evaluation of a Scalable Multi-level Checkpointing System
conference, November 2010

  • Moody, Adam; Bronevetsky, Greg; Mohror, Kathryn
  • 2010 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis
  • DOI: 10.1109/SC.2010.18

GIGA+: scalable directories for shared file systems
conference, January 2007

  • Patil, Swapnil V.; Gibson, Garth A.; Lang, Sam
  • Proceedings of the 2nd international workshop on Petascale data storage held in conjunction with Supercomputing '07 - PDSW '07
  • DOI: 10.1145/1374596.1374604

Scalable parallel building blocks for custom data analysis
conference, October 2011

  • Peterka, Tom; Ross, Robert; Gyulassy, Attila
  • 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV)
  • DOI: 10.1109/LDAV.2011.6092324

Visualization by Proxy: A Novel Framework for Deferred Interaction with Volume Data
journal, November 2010

  • Tikhonova, A.; Correa, C. D.
  • IEEE Transactions on Visualization and Computer Graphics, Vol. 16, Issue 6
  • DOI: 10.1109/TVCG.2010.215

Toward simulation-time data analysis and I/O acceleration on leadership-class systems
conference, October 2011

  • Vishwanath, Venkatram; Hereld, Mark; Papka, Michael E.
  • 2011 IEEE Symposium on Large Data Analysis and Visualization (LDAV)
  • DOI: 10.1109/LDAV.2011.6092178

Parallel In Situ Coupling of Simulation with a Fully Featured Visualization System
null, January 2011

In-situ Sampling of a Large-Scale Particle Simulation for Interactive Visualization and Analysis
journal, June 2011

Remote Large Data Visualization in the ParaView Framework
null, January 2006

On the role of burst buffers in leadership-class storage systems
conference, April 2012

  • Liu, Ning; Cope, Jason; Carns, Philip
  • 2012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST)
  • DOI: 10.1109/MSST.2012.6232369

Storage challenges at Los Alamos National Lab
conference, April 2012

A higher order estimate of the optimum checkpoint interval for restart dumps
journal, February 2006

A Self-Organizing Storage Cluster for Parallel Data-Intensive Applications
conference, January 2004

  • Hong Tang, ; Gulbeden, A.
  • Proceedings of the ACM/IEEE SC2004 Conference
  • DOI: 10.1109/SC.2004.9
    [ × clear filter / sort ]

    Similar Records in DOE Patents and OSTI.GOV collections: