Distributed metadata in a high performance computing environment

Title: Distributed metadata in a high performance computing environment

A computer-executable method, system, and computer program product for managing meta-data in a distributed storage system, wherein the distributed storage system includes one or more burst buffers enabled to operate with a distributed key-value store, the co computer-executable method, system, and computer program product comprising receiving a request for meta-data associated with a block of data stored in a first burst buffer of the one or more burst buffers in the distributed storage system, wherein the meta data is associated with a key-value, determining which of the one or more burst buffers stores the requested metadata, and upon determination that a first burst buffer of the one or more burst buffers stores the requested metadata, locating the key-value in a portion of the distributed key-value store accessible from the first burst buffer.

Inventors:: Bent, John M.; Faibish, Sorin; Zhang, Zhenhua; Liu, Xuezhao; Tang, Haiying

Issue Date:: 2017-07-11

OSTI Identifier:: 1369068

Assignee:: EMC IP Holding Company LLC LANL

Patent Number(s):: 9,703,788

Application Number:: 14/230,800

Contract Number:: AC52-06NA25396

Resource Relation:: Patent File Date: 2014 Mar 31

Research Org:: Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Sponsoring Org:: USDOE

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Full Text
View Full Text

Other works cited in this record:

Storing and retrieving objects on a computer network in a distributed database
patent, April 2009

Meadway, Mike; Tharp, Stanford
US Patent Document 7,523,130
URL: http://patft.uspto.gov/netacgi/nph-Parser?Sect2=PTO1&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&d=PALL&RefSrch=yes&Query=PN%2F7523130

Message Flow Control in a Multi-Node Computer System
patent-application, December 2009

Barsness, Eric L.; Darrington, David L.; Peters, Amanda
US Patent Application\t12/144783; 20090319621
URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220090319621%22.PGNR.&OS=DN/20090319621&RS=DN/20090319621

Policy-Based Storage Structure Distribution
patent-application, March 2011

Moyne, Jean-Noel; Subramani, Suresh; Kim, Tom M.
US Patent Application 12/762249; 20110066591
URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220110066591%22.PGNR.&OS=DN/20110066591&RS=DN/20110066591

Query Execution Systems and Methods
patent-application, December 2011

Abadi, Daniel; Bajda-Pawlikowski, Kamil
US Patent Application\t13/032551; 20110302151
URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220110302151%22.PGNR.&OS=DN/20110302151&RS=DN/20110302151

Scalable Distributed Metadata File System using Key-Value Stores
patent-application, November 2012

Dalton, Michael W.
US Patent Application 13/455891; 20120284317
URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220120284317%22.PGNR.&OS=DN/20120284317&RS=DN/20120284317

Active Non-Volatile Memory Post-Processing
patent-application, August 2013

Kannan, Sudarsun; Milojicic, Dejan S.; Talwar, Vanish
US Patent Application 13/404619; 20130227194
URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220130227194%22.PGNR.&OS=DN/20130227194&RS=DN/20130227194

Distributed Object Storage System Comprising Performance Optimizations
patent-application, October 2013

De Schrijvr, Frederik; Slootmaekers, Romain Raymond Agnes; Stougie, Bastiaan
US Patent Application 13/976061; 20130275381
URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220130275381%22.PGNR.&OS=DN/20130275381&RS=DN/20130275381

Load Balancing for Distributed Key-Value Store
patent-application, December 2013

Shiga, Kenta; Lin, Wujuan
US Patent Application 13/489897; 20130332608
URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220130332608%22.PGNR.&OS=DN/20130332608&RS=DN/20130332608

Data Storage Architecture and System For High Performance Computing
patent-application, April 2014

Nowoczynski, Paul; Cope, Jason Micah; Manning, Gordon
US Patent Application 14/028292; 20140108707
URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220140108707%22.PGNR.&OS=DN/20140108707&RS=DN/20140108707

System and Method for Compressing Data in a Database
patent-application, September 2014

Velury, Ramabrahmam; Tan, Jihui; Zhou, Guangcheng
US Patent Application\t13/804321; 20140279964
URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220140279964%22.PGNR.&OS=DN/20140279964&RS=DN/20140279964

Systems and Methods For Implementing Distributed Databases Using Many-Core Processors
patent-application, September 2014

Rawson, Ryan; Newman, Alexander
US Patent Document 14/209454; 20140280375
URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220140280375%22.PGNR.&OS=DN/20140280375&RS=DN/20140280375

Method and System For Data Transfer Between Compute Clusters And File System
patent-application, November 2014

Uppu, Pavan Kumar; Cope, Jason Micah; Nowoczynski, Paul
US Patent Application14/045170; 20140351300
URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220140351300%22.PGNR.&OS=DN/20140351300&RS=DN/20140351300

Read Operations In A Tree-Based Distributed File System
patent-application, February 2016

Aron, Mohit; Shanmuganathan, Ganesha
US Patent Application 14/450798; 20160034507
URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220160034507%22.PGNR.&OS=DN/20160034507&RS=DN/20160034507

Similar records in DOepatents and OSTI.GOV collections:

Distributed metadata servers for cluster file systems using shared low latency persistent key-value metadata store

Patent Bent, John M. ; Faibish, Sorin ; Pedone, Jr., James M. ; Tzelnic, Percy ; Ting, Dennis P. J. ; Ionkov, Latchesar A. ; Grider, Gary

A cluster file system is provided having a plurality of distributed metadata servers with shared access to one or more shared low latency persistent key-value metadata stores. A metadata server comprises an abstract storage interface comprising a software interface module that communicates with at least one shared persistent key-value metadata store providing a key-value interface for persistent storage of key-value metadata. The software interface module provides the key-value metadata to the at least one shared persistent key-value metadata store in a key-value format. The shared persistent key-value metadata store is accessed by a plurality of metadata servers. A metadata requestmore »« less
Full Text Available December 2017
Parallel sort with a ranged, partitioned key-value store in a high perfomance computing environment

Patent Bent, John M. ; Faibish, Sorin ; Grider, Gary ; Torres, Aaron ; Poole, Stephen W.

Improved sorting techniques are provided that perform a parallel sort using a ranged, partitioned key-value store in a high performance computing (HPC) environment. A plurality of input data files comprising unsorted key-value data in a partitioned key-value store are sorted. The partitioned key-value store comprises a range server for each of a plurality of ranges. Each input data file has an associated reader thread. Each reader thread reads the unsorted key-value data in the corresponding input data file and performs a local sort of the unsorted key-value data to generate sorted key-value data. A plurality of sorted, ranged subsets ofmore »« less
Full Text Available January 2016
Debugging a high performance computing program

Patent Gooding, Thomas M.

Methods, apparatus, and computer program products are disclosed for debugging a high performance computing program by gathering lists of addresses of calling instructions for a plurality of threads of execution of the program, assigning the threads to groups in dependence upon the addresses, and displaying the groups to identify defective threads.
Full Text Available August 2013
Matrix multiplication operations with data pre-conditioning in a high performance computing architecture

Patent Eichenberger, Alexandre E ; Gschwind, Michael K ; Gunnels, John A

Mechanisms for performing matrix multiplication operations with data pre-conditioning in a high performance computing architecture are provided. A vector load operation is performed to load a first vector operand of the matrix multiplication operation to a first target vector register. A load and splat operation is performed to load an element of a second vector operand and replicating the element to each of a plurality of elements of a second target vector register. A multiply add operation is performed on elements of the first target vector register and elements of the second target vector register to generate a partial productmore »« less
Full Text Available November 2013
Complex matrix multiplication operations with data pre-conditioning in a high performance computing architecture

Patent Eichenberger, Alexandre E ; Gschwind, Michael K ; Gunnels, John A

Mechanisms for performing a complex matrix multiplication operation are provided. A vector load operation is performed to load a first vector operand of the complex matrix multiplication operation to a first target vector register. The first vector operand comprises a real and imaginary part of a first complex vector value. A complex load and splat operation is performed to load a second complex vector value of a second vector operand and replicate the second complex vector value within a second target vector register. The second complex vector value has a real and imaginary part. A cross multiply add operation ismore »« less
Full Text Available February 2014