skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Method and apparatus for offloading compute resources to a flash co-processing appliance

Patent ·
OSTI ID:1223101
; ; ; ; ;

Solid-State Drive (SSD) burst buffer nodes are interposed into a parallel supercomputing cluster to enable fast burst checkpoint of cluster memory to or from nearby interconnected solid-state storage with asynchronous migration between the burst buffer nodes and slower more distant disk storage. The SSD nodes also perform tasks offloaded from the compute nodes or associated with the checkpoint data. For example, the data for the next job is preloaded in the SSD node and very fast uploaded to the respective compute node just before the next job starts. During a job, the SSD nodes perform fast visualization and statistical analysis upon the checkpoint data. The SSD nodes can also perform data reduction and encryption of the checkpoint data.

Research Organization:
Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Sponsoring Organization:
USDOE
DOE Contract Number:
AC52-06NA25396
Assignee:
EMC Corporation (Hopkinton, MA)
Patent Number(s):
9,158,540
Application Number:
13/676,019
OSTI ID:
1223101
Resource Relation:
Patent File Date: 2012 Nov 13
Country of Publication:
United States
Language:
English

References (44)

Data storage system having separate data transfer section and message network patent October 2006
Distributed maintenance of snapshot copies by a primary processor managing metadata and a secondary processor providing read-write access to a production dataset patent March 2010
Methods, systems, and computer program products for providing access to shared storage by computing grids and clusters with large numbers of nodes patent March 2010
Network file server sharing local caches of file access information in data processors assigned to respective file systems patent June 2010
Techniques for using flash-based memory as a write cache and a vault patent September 2010
Pre-allocation and hierarchical mapping of data blocks distributed from a first processor to a second processor for use in a file system patent May 2011
Efficient read/write algorithms and associated mapping for block-level data reduction processes patent March 2012
Techniques for using flash-based memory in recovery processing patent October 2012
Apparatus For Enhancing Performance Of A Parallel Processing Environment, And Associated Methods patent-application July 2010
PLFS: a checkpoint filesystem for parallel applications conference January 2009
A Cost-Effective, High Bandwidth Server I/O network Architecture for Cluster Systems conference March 2007
PaScal-- A New Parallel and Scalable Server IO Networking Infrastructure for Supporting Global Storage/File Systems in Large-size Linux Clusters conference January 2006
Distributed-and-split data-control extension to SCSI for scalable storage area networks conference January 2002
Hybrid checkpointing using emerging nonvolatile memories for future exascale systems journal July 2011
Evaluation of active storage strategies for the lustre parallel file system conference January 2007
Evaluating the benefits of an extended memory hierarchy for parallel streamline algorithms conference October 2011
Managing storage space in a flash and disk hybrid storage system
  • amp; Simulation of Computer and Telecommunication Systems (MASCOTS), 2009 IEEE International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems https://doi.org/10.1109/MASCOT.2009.5366764
conference September 2009
Exploiting Concurrency to Improve Latency and throughput in a Hybrid Storage System
  • Wu, Xiaojian; Reddy, A. L. Narasimha
  • Simulation of Computer and Telecommunication Systems (MASCOTS), 2010 IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems https://doi.org/10.1109/MASCOTS.2010.11
conference August 2010
Incorporating Network RAM and Flash into Fast Backing Store for Clusters conference September 2011
The Conquest file system : Better performance through a disk/persistent-RAM hybrid design journal August 2006
Azor: Using Two-Level Block Selection to Improve SSD-Based I/O Caches
  • Klonatos, Yannis; Makatos, Thanos; Marazakis, Manolis
  • 2011 6th IEEE International Conference on Networking, Architecture, and Storage (NAS), 2011 IEEE Sixth International Conference on Networking, Architecture, and Storage https://doi.org/10.1109/NAS.2011.50
conference July 2011
Using Active NVRAM for Cloud I/O conference October 2011
A comprehensive study of energy efficiency and performance of flash-based SSD journal April 2011
Making a case for distributed file systems at Exascale conference January 2011
Jitter-free co-processing on a prototype exascale storage stack conference April 2012
Verifying Scientific Simulations via Comparative and Quantitative Visualization journal November 2010
Design issues for a shingled write disk system conference May 2010
Managing Variability in the IO Performance of Petascale Storage Systems
  • Lofstead, Jay; Zheng, Fang; Liu, Qing
  • 2010 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2010.32
conference November 2010
Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS)
  • Lofstead, Jay F.; Klasky, Scott; Schwan, Karsten
  • Proceedings of the 6th international workshop on Challenges of large applications in distributed environments - CLADE '08 https://doi.org/10.1145/1383529.1383533
conference January 2008
Design, Modeling, and Evaluation of a Scalable Multi-level Checkpointing System
  • Moody, Adam; Bronevetsky, Greg; Mohror, Kathryn
  • 2010 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2010.18
conference November 2010
GIGA+: scalable directories for shared file systems
  • Patil, Swapnil V.; Gibson, Garth A.; Lang, Sam
  • Proceedings of the 2nd international workshop on Petascale data storage held in conjunction with Supercomputing '07 - PDSW '07 https://doi.org/10.1145/1374596.1374604
conference January 2007
Scalable parallel building blocks for custom data analysis conference October 2011
The Case of the Missing Supercomputer Performance: Achieving Optimal Performance on the 8,192 Processors of ASCI Q conference January 2003
Visualization by Proxy: A Novel Framework for Deferred Interaction with Volume Data journal November 2010
Toward simulation-time data analysis and I/O acceleration on leadership-class systems conference October 2011
In-situ Sampling of a Large-Scale Particle Simulation for Interactive Visualization and Analysis journal June 2011
Remote Large Data Visualization in the ParaView Framework null January 2006
On the role of burst buffers in leadership-class storage systems conference April 2012
DataStager: scalable data staging services for petascale applications journal June 2010
Modeling a Leadership-Scale Storage System book January 2012
Storage challenges at Los Alamos National Lab conference April 2012
Scalable I/O forwarding framework for high-performance computing systems conference August 2009
Pageserver: High-Performance SSD-Based Checkpointing of Transactional Distributed Memory conference March 2010
Enhancing Checkpoint Performance with Staging IO and SSD
  • Ouyang, Xiangyong; Marcarelli, Sonya; Panda, Dhabaleswar K.
  • 2010 International Workshop on Storage Network Architecture and Parallel I/Os (SNAPI) https://doi.org/10.1109/SNAPI.2010.10
conference May 2010

Similar Records

Architecture and method for a burst buffer using flash technology
Patent · Tue Mar 15 00:00:00 EDT 2016 · OSTI ID:1223101
+3 more
SCR-Exa: Enhanced Scalable Checkpoint Restart (SCR) Library for Next Generation Exascale Computing
Technical Report · Mon Feb 21 00:00:00 EST 2022 · OSTI ID:1223101
Optimizing checkpoint data placement with guaranteed burst buffer endurance in large-scale hierarchical storage systems
Journal Article · Wed Feb 01 00:00:00 EST 2017 · Journal of Parallel and Distributed Computing · OSTI ID:1223101
+1 more

Related Subjects

97 MATHEMATICS AND COMPUTING