Burst buffer dynamic logical volume sizing in high performance computing environment

Gooding, Thomas M.; Hermsmeier, David L.; Ma, Jin; Mincher, Gary J.; Rosenburg, Bryan S.

Title: Burst buffer dynamic logical volume sizing in high performance computing environment

Abstract

A method, data processing system and program product utilize dynamic logical storage volume sizing for burst buffers or other local storage for computing nodes to optimize job stage in, execution and/or stage out.

Inventors:: Gooding, Thomas M.; Hermsmeier, David L.; Ma, Jin; Mincher, Gary J.; Rosenburg, Bryan S.

Issue Date:: 2019-05-14

Research Org.:: International Business Machines Corp., Armonk, NY (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1568335

Patent Number(s):: 10289329

Application Number:: 15/433,998

Assignee:: International Business Machines Corporation (Armonk, NY)

Patent Classifications (CPCs):: G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING

H - ELECTRICITY H04 - ELECTRIC COMMUNICATION TECHNIQUE H04L - TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION

Show more

G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING
G06F3/067 - {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]}

H - ELECTRICITY H04 - ELECTRIC COMMUNICATION TECHNIQUE H04L - TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
H04L67/1097 - {for distributed storage of data in a network, e.g. network file system [NFS], transport mechanisms for storage area networks [SAN] or network attached storage [NAS]

G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING
G06F3/0631 - {by allocating resources to storage systems}
G06F3/0605 - {by facilitating the interaction with a user or administrator}

Show less

DOE Contract Number:: B604142

Resource Type:: Patent

Resource Relation:: Patent File Date: 02/15/2017

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Gooding, Thomas M., Hermsmeier, David L., Ma, Jin, Mincher, Gary J., and Rosenburg, Bryan S. Burst buffer dynamic logical volume sizing in high performance computing environment.  United States: N. p., 2019. 
        Web.

Copy to clipboard


                    Gooding, Thomas M., Hermsmeier, David L., Ma, Jin, Mincher, Gary J., & Rosenburg, Bryan S. Burst buffer dynamic logical volume sizing in high performance computing environment.  United States.

Copy to clipboard


                    Gooding, Thomas M., Hermsmeier, David L., Ma, Jin, Mincher, Gary J., and Rosenburg, Bryan S. Tue .  
        "Burst buffer dynamic logical volume sizing in high performance computing environment".  United States.  https://www.osti.gov/servlets/purl/1568335.

Copy to clipboard


                    
@article{osti_1568335,

  title        = {Burst buffer dynamic logical volume sizing in high performance computing environment},

  author       = {Gooding, Thomas M. and Hermsmeier, David L. and Ma, Jin and Mincher, Gary J. and Rosenburg, Bryan S.},

  abstractNote = {A method, data processing system and program product utilize dynamic logical storage volume sizing for burst buffers or other local storage for computing nodes to optimize job stage in, execution and/or stage out.},

  doi          = {},

  journal      = {},
number       = ,

  volume       = ,

  place        = {United States},

  year         = {2019},

  month        = {5}

}

Copy to clipboard

Patent:

View Patent

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Dynamic allocation of a high-speed memory pool between a cluster file system and a burst buffer appliance
patent, October 2016

Faibish, Sorin; Bent, John M.; Tzelnic, Percy
US Patent Document 9,465,549
URL: http://patft.uspto.gov/netacgi/nph-Parser?patentnumber=9465549

Architecture and method for a burst buffer using flash technology
patent, March 2016

Tzelnic, Percy; Faibish, Sorin; Gupta, Uday K.
US Patent Document 9,286,261
URL: http://patft.uspto.gov/netacgi/nph-Parser?patentnumber=9286261

Method and apparatus for offloading compute resources to a flash co-processing appliance
patent, October 2015

Tzelnic, Percy; Faibish, Sorin; Gupta, Uday K.
US Patent Document 9,158,540
URL: http://patft.uspto.gov/netacgi/nph-Parser?patentnumber=9158540

Similar Records in DOE Patents and OSTI.GOV collections:

Distributed metadata in a high performance computing environment

Patent Bent, John M.; Faibish, Sorin; Zhang, Zhenhua; ...

A computer-executable method, system, and computer program product for managing meta-data in a distributed storage system, wherein the distributed storage system includes one or more burst buffers enabled to operate with a distributed key-value store, the co computer-executable method, system, and computer program product comprising receiving a request for meta-data associated with a block of data stored in a first burst buffer of the one or more burst buffers in the distributed storage system, wherein the meta data is associated with a key-value, determining which of the one or more burst buffers stores the requested metadata, and upon determination thatmore » « less
Full Text Available
Sequence optimizations in a high-performance computing environment

Patent Zhang, Yongzhi

Embodiments are directed to techniques to determine dataflow graph instructions comprising one or more pick/switch instruction pairs and generate a reverse static single assignment graph based on the dataflow graph instructions, the reverse static single assignment graph comprising strongly connected components, each of the strongly connected components associated with at least one of the one or more pick/switch instruction pairs. Embodiments also include traversing the reverse static single assignment graph depth-first, and replace pick/switch instructions associated with strongly connected components having configuration values with compound instructions.
Full Text Available
Architecture and method for a burst buffer using flash technology

Patent Tzelnic, Percy; Faibish, Sorin; Gupta, Uday K.; ...

A parallel supercomputing cluster includes compute nodes interconnected in a mesh of data links for executing an MPI job, and solid-state storage nodes each linked to a respective group of the compute nodes for receiving checkpoint data from the respective compute nodes, and magnetic disk storage linked to each of the solid-state storage nodes for asynchronous migration of the checkpoint data from the solid-state storage nodes to the magnetic disk storage. Each solid-state storage node presents a file system interface to the MPI job, and multiple MPI processes of the MPI job write the checkpoint data to a shared filemore » « less
Full Text Available
Parallel storage system with burst buffer appliance for storage of partitioned key-value store across a plurality of storage tiers

Patent Bent, John M.; Faibish, Sorin; Grider, Gary; ...

Parallel storage systems are provided with a burst buffer appliance for storage of a partitioned key-value store across a plurality of storage tiers. The burst buffer appliance is configured to store key-value data on first and second storage tiers comprising respective disjoint subsets of storage. One or more servers in the partitioned key-value store provide (i) an interface between one or more local key-value stores and one or more additional storage architectures, and/or (ii) a key value interface that provides access to at least one hardware key-value store. The local key-values are optionally sorted both locally by the local storemore » « less
Full Text Available
Identifying a largest logical plane from a plurality of logical planes formed of compute nodes of a subcommunicator in a parallel computer

Patent Davis, Kristan D.; Faraj, Daniel A.

In a parallel computer, a largest logical plane from a plurality of logical planes formed of compute nodes of a subcommunicator may be identified by: identifying, by each compute node of the subcommunicator, all logical planes that include the compute node; calculating, by each compute node for each identified logical plane that includes the compute node, an area of the identified logical plane; initiating, by a root node of the subcommunicator, a gather operation; receiving, by the root node from each compute node of the subcommunicator, each node's calculated areas as contribution data to the gather operation; and identifying, bymore » « less
Full Text Available

Similar Records

Title: Burst buffer dynamic logical volume sizing in high performance computing environment

Abstract

Citation Formats

Dynamic allocation of a high-speed memory pool between a cluster file system and a burst buffer appliance patent, October 2016

Architecture and method for a burst buffer using flash technology patent, March 2016

Method and apparatus for offloading compute resources to a flash co-processing appliance patent, October 2015

Dynamic allocation of a high-speed memory pool between a cluster file system and a burst buffer appliance
patent, October 2016

Architecture and method for a burst buffer using flash technology
patent, March 2016

Method and apparatus for offloading compute resources to a flash co-processing appliance
patent, October 2015