Storing files in a parallel computing system based on user or application specification

Faibish, Sorin; Bent, John M.; Nick, Jeffrey M.; Grider, Gary; Torres, Aaron

Advanced Search OptionsAdvanced Search queries use a traditional Term Search. For more info, see our FAQ.

All Fields:

Patent Title:

Abstract:

Assignee:

Inventor(s):

Patent Number:

Patent Classification (CPC):

All Classifications
A - human necessities
A01 - agriculture
A21 - baking
A22 - butchering
A23 - foods or foodstuffs
A24 - tobacco
A41 - wearing apparel
A42 - headwear
A43 - footwear
A44 - haberdashery
A45 - hand or travelling articles
A46 - brushware
A47 - furniture
A61 - medical or veterinary science
A62 - life-saving
A63 - sports
A99 - subject matter not otherwise provided for in this section
B - performing operations
B01 - physical or chemical processes or apparatus in general
B02 - crushing, pulverising, or disintegrating
B03 - separation of solid materials using liquids or using pneumatic tables or jigs
B04 - centrifugal apparatus or machines for carrying-out physical or chemical processes
B05 - spraying or atomising in general
B06 - generating or transmitting mechanical vibrations in general
B07 - separating solids from solids
B08 - cleaning
B09 - disposal of solid waste
B21 - mechanical metal-working without essentially removing material
B22 - casting
B23 - machine tools
B24 - grinding
B25 - hand tools
B26 - hand cutting tools
B27 - working or preserving wood or similar material
B28 - working cement, clay, or stone
B29 - working of plastics
B30 - presses
B31 - making articles of paper, cardboard or material worked in a manner analogous to paper
B32 - layered products
B33 - additive manufacturing technology
B41 - printing
B42 - bookbinding
B43 - writing or drawing implements
B44 - decorative arts
B60 - vehicles in general
B61 - railways
B62 - land vehicles for travelling otherwise than on rails
B63 - ships or other waterborne vessels
B64 - aircraft
B65 - conveying
B66 - hoisting
B67 - opening, closing {or cleaning} bottles, jars or similar containers
B68 - saddlery
B81 - microstructural technology
B82 - nanotechnology
B99 - subject matter not otherwise provided for in this section
C - chemistry
C01 - inorganic chemistry
C02 - treatment of water, waste water, sewage, or sludge
C03 - glass
C04 - cements
C05 - fertilisers
C06 - explosives
C07 - organic chemistry
C08 - organic macromolecular compounds
C09 - dyes
C10 - petroleum, gas or coke industries
C11 - animal or vegetable oils, fats, fatty substances or waxes
C12 - biochemistry
C13 - sugar industry
C14 - skins
C21 - metallurgy of iron
C22 - metallurgy
C23 - coating metallic material
C25 - electrolytic or electrophoretic processes
C30 - crystal growth
C40 - combinatorial technology
C99 - subject matter not otherwise provided for in this section
D - textiles
D01 - natural or man-made threads or fibres
D02 - yarns
D03 - weaving
D04 - braiding
D05 - sewing
D06 - treatment of textiles or the like
D07 - ropes
D10 - indexing scheme associated with sublasses of section d, relating to textiles
D21 - paper-making
D99 - subject matter not otherwise provided for in this section
E - fixed constructions
E01 - construction of roads, railways, or bridges
E02 - hydraulic engineering
E03 - water supply
E04 - building
E05 - locks
E06 - doors, windows, shutters, or roller blinds in general
E21 - earth drilling
E99 - subject matter not otherwise provided for in this section
F - mechanical engineering
F01 - machines or engines in general
F02 - combustion engines
F03 - machines or engines for liquids
F04 - positive - displacement machines for liquids
F05 - indexing schemes relating to engines or pumps in various subclasses of classes f01-f04
F15 - fluid-pressure actuators
F16 - engineering elements and units
F17 - storing or distributing gases or liquids
F21 - lighting
F22 - steam generation
F23 - combustion apparatus
F24 - heating
F25 - refrigeration or cooling
F26 - drying
F27 - furnaces
F28 - heat exchange in general
F41 - weapons
F42 - ammunition
F99 - subject matter not otherwise provided for in this section
G - physics
G01 - measuring
G02 - optics
G03 - photography
G04 - horology
G05 - controlling
G06 - computing
G07 - checking-devices
G08 - signalling
G09 - education
G10 - musical instruments
G11 - information storage
G12 - instrument details
G16 - information and communication technology [ict] specially adapted for specific application fields
G21 - nuclear physics
G99 - subject matter not otherwise provided for in this section
H - electricity
H01 - basic electric elements
H02 - generation
H03 - basic electronic circuitry
H04 - electric communication technique
H05 - electric techniques not otherwise provided for
H99 - subject matter not otherwise provided for in this section
Y - new / cross sectional technologies
Y02 - technologies or applications for mitigation or adaptation against climate change
Y04 - information or communication technologies having an impact on other technology areas
Y10 - technical subjects covered by former uspc

More Options ...

Title: Storing files in a parallel computing system based on user or application specification

Abstract

Techniques are provided for storing files in a parallel computing system based on a user-specification. A plurality of files generated by a distributed application in a parallel computing system are stored by obtaining a specification from the distributed application indicating how the plurality of files should be stored; and storing one or more of the plurality of files in one or more storage nodes of a multi-tier storage system based on the specification. The plurality of files comprise a plurality of complete files and/or a plurality of sub-files. The specification can optionally be processed by a daemon executing on one or more nodes in a multi-tier storage system. The specification indicates how the plurality of files should be stored, for example, identifying one or more storage nodes where the plurality of files should be stored.

Inventors:: Faibish, Sorin; Bent, John M.; Nick, Jeffrey M.; Grider, Gary; Torres, Aaron

Issue Date:: Tue Mar 29 00:00:00 EDT 2016

Research Org.:: Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1244229

Patent Number(s):: 9298733

Application Number:: 13/536,289

Assignee:: EMC Corporation (Hopkinton, MA) Los Alamos National Security, LLC (Los Alamos, NM)

Patent Classifications (CPCs):: G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING

Show more

G - PHYSICS G06 - COMPUTING G06F - ELECTRIC DIGITAL DATA PROCESSING
G06F16/1827 - {Management specifically adapted to NAS
G06F16/1858 - {Parallel file systems, i.e. file systems supporting multiple processors}

Show less

DOE Contract Number:: AC52-06NA25396

Resource Type:: Patent

Resource Relation:: Patent File Date: 2012 Jun 28

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Faibish, Sorin, Bent, John M., Nick, Jeffrey M., Grider, Gary, and Torres, Aaron. Storing files in a parallel computing system based on user or application specification.  United States: N. p., 2016. 
        Web.

Copy to clipboard


                    Faibish, Sorin, Bent, John M., Nick, Jeffrey M., Grider, Gary, & Torres, Aaron. Storing files in a parallel computing system based on user or application specification.  United States.

Copy to clipboard


                    Faibish, Sorin, Bent, John M., Nick, Jeffrey M., Grider, Gary, and Torres, Aaron. Tue .  
        "Storing files in a parallel computing system based on user or application specification".  United States.  https://www.osti.gov/servlets/purl/1244229.

Copy to clipboard


                    
@article{osti_1244229,

  title        = {Storing files in a parallel computing system based on user or application specification},

  author       = {Faibish, Sorin and Bent, John M. and Nick, Jeffrey M. and Grider, Gary and Torres, Aaron},

  abstractNote = {Techniques are provided for storing files in a parallel computing system based on a user-specification. A plurality of files generated by a distributed application in a parallel computing system are stored by obtaining a specification from the distributed application indicating how the plurality of files should be stored; and storing one or more of the plurality of files in one or more storage nodes of a multi-tier storage system based on the specification. The plurality of files comprise a plurality of complete files and/or a plurality of sub-files. The specification can optionally be processed by a daemon executing on one or more nodes in a multi-tier storage system. The specification indicates how the plurality of files should be stored, for example, identifying one or more storage nodes where the plurality of files should be stored.},

  doi          = {},

  journal      = {},
number       = ,

  volume       = ,

  place        = {United States},

  year         = {Tue Mar 29 00:00:00 EDT 2016},

  month        = {Tue Mar 29 00:00:00 EDT 2016}

}

Copy to clipboard

Patent:

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

PLFS: a checkpoint filesystem for parallel applications
conference, January 2009

Bent, John; Gibson, Garth; Grider, Gary
https://doi.org/10.1145/1654059.1654081

Network integrated data compression system
patent, July 2008

Suzuki, Norihisa; Marrin, Mark
US Patent Document 7,394,408
URL: https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/7394408

Policy based tiered data deduplication strategy
patent, July 2009

Anglin, Matthew J.; Cannon, David M.; Dawson, Colin S.
US Patent Document 7,567,188
URL: https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/7567188

Method and apparatus for server share migration and server recovery using hierarchical storage management
patent, September 2009

Therrien, David G.; Pownell, James E.; VanderSpek, Adrian T.
US Patent Document 7,593,966
URL: https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/7593966

Systems and methods for managing portions of files in multi-tier storage systems
patent, January 2013

Mamidi, Murthy V.; Malige, Raghupathi; Ravi, Gautham
US Patent Document 8,352,429
URL: https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/8352429

System and method for configuring a parallel-processing database system
patent-application, May 2004

Baylis, David; Chapman, Richard; Smith, Jake
US Patent Application 10/293483; 20040098373
URL: https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/20040098373

Data Loading Systems and Methods
patent-application, December 2011

Abadi, Daniel; Abouzied, Azza
US Patent Application 13/032538; 20110302226
URL: https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/20110302226

Similar Records in DOE Patents and OSTI.GOV collections:

Storing files in a parallel computing system based on user-specified parser function

Patent Faibish, Sorin; Bent, John M; Tzelnic, Percy; ...

Techniques are provided for storing files in a parallel computing system based on a user-specified parser function. A plurality of files generated by a distributed application in a parallel computing system are stored by obtaining a parser from the distributed application for processing the plurality of files prior to storage; and storing one or more of the plurality of files in one or more storage nodes of the parallel computing system based on the processing by the parser. The plurality of files comprise one or more of a plurality of complete files and a plurality of sub-files. The parser canmore » « less
Full Text Available
Storing files in a parallel computing system using list-based index to identify replica files

Patent Faibish, Sorin; Bent, John M.; Tzelnic, Percy; ...

Improved techniques are provided for storing files in a parallel computing system using a list-based index to identify file replicas. A file and at least one replica of the file are stored in one or more storage nodes of the parallel computing system. An index for the file comprises at least one list comprising a pointer to a storage location of the file and a storage location of the at least one replica of the file. The file comprises one or more of a complete file and one or more sub-files. The index may also comprise a checksum value formore » « less
Full Text Available
Parallel file system with metadata distributed across partitioned key-value store c

Patent Bent, John M.; Faibish, Sorin; Grider, Gary; ...

Improved techniques are provided for storing metadata associated with a plurality of sub-files associated with a single shared file in a parallel file system. The shared file is generated by a plurality of applications executing on a plurality of compute nodes. A compute node implements a Parallel Log Structured File System (PLFS) library to store at least one portion of the shared file generated by an application executing on the compute node and metadata for the at least one portion of the shared file on one or more object storage servers. The compute node is also configured to implement amore » « less
Full Text Available
Request queues for interactive clients in a shared file system of a parallel computing system

Patent Bent, John M.; Faibish, Sorin

Interactive requests are processed from users of log-in nodes. A metadata server node is provided for use in a file system shared by one or more interactive nodes and one or more batch nodes. The interactive nodes comprise interactive clients to execute interactive tasks and the batch nodes execute batch jobs for one or more batch clients. The metadata server node comprises a virtual machine monitor; an interactive client proxy to store metadata requests from the interactive clients in an interactive client queue; a batch client proxy to store metadata requests from the batch clients in a batch client queue;more » « less
Full Text Available
Methods and apparatus for multi-resolution replication of files in a parallel computing system using semantic information

Patent Faibish, Sorin; Bent, John M.; Tzelnic, Percy; ...

Techniques are provided for storing files in a parallel computing system using different resolutions. A method is provided for storing at least one file generated by a distributed application in a parallel computing system. The file comprises one or more of a complete file and a sub-file. The method comprises the steps of obtaining semantic information related to the file; generating a plurality of replicas of the file with different resolutions based on the semantic information; and storing the file and the plurality of replicas of the file in one or more storage nodes of the parallel computing system. Themore » « less
Full Text Available

Similar Records

Title: Storing files in a parallel computing system based on user or application specification

Abstract

Citation Formats

PLFS: a checkpoint filesystem for parallel applications conference, January 2009

Network integrated data compression system patent, July 2008

Policy based tiered data deduplication strategy patent, July 2009

Method and apparatus for server share migration and server recovery using hierarchical storage management patent, September 2009

Systems and methods for managing portions of files in multi-tier storage systems patent, January 2013

System and method for configuring a parallel-processing database system patent-application, May 2004

Data Loading Systems and Methods patent-application, December 2011

PLFS: a checkpoint filesystem for parallel applications
conference, January 2009

Network integrated data compression system
patent, July 2008

Policy based tiered data deduplication strategy
patent, July 2009

Method and apparatus for server share migration and server recovery using hierarchical storage management
patent, September 2009

Systems and methods for managing portions of files in multi-tier storage systems
patent, January 2013

System and method for configuring a parallel-processing database system
patent-application, May 2004

Data Loading Systems and Methods
patent-application, December 2011