Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

A Software Defined Storage Approach to Exascale Storage Services (Final Technical Report)

Technical Report ·
DOI:https://doi.org/10.2172/1544789· OSTI ID:1544789
 [1];  [2];  [3];  [4];  [5];  [5];  [2];  [2];  [2];  [2];  [3];  [3];  [4];  [4];  [4]
  1. The HDF Group, Champaign, IL (United States); The HDF Group
  2. Argonne National Laboratory (ANL), Argonne, IL (United States)
  3. Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
  4. Carnegie Mellon Univ., Pittsburgh, PA (United States)
  5. The HDF Group, Champaign, IL (United States)
+ Show Author Affiliations
The Mochi project began in 2015 in recognition of a set of changes that were set to alter the landscape of storage and I/O for DOE science. On the one hand, an influx of new classes of applications including an initial set of data intensive and machine learning applications were operating with new types of data and in new ways, as compared to traditional computational science simulation codes. Simultaneously, new memory, storage, and networking technologies were driving a need to adapt software designs to a rapidly changing design space. The goal of the three-year Mochi project was to develop and demonstrate an approach for composing storage services which provides new levels of functionality, performance, and reliability for science applications at extreme scale. The team, including a mix of Laboratory, university, and industry collaborators, focused on systems and applications of interest to DOE Office of Science, tracking developments at NERSC and the LCFs as well as application trends within ECP and SciDAC. Technical work focused on three themes: composition (understand the building blocks needed to provide storage services that meet DOE mission needs and how these components are flexibly composed), application interfacing (what service compositions are most effective for common DOE application use cases), and consistency and synchronization (taking advantage of knowledge about, and from, the system and scientific applications to provide better service than would otherwise be possible). The project was highly successful, producing 13 publications and tens of thousands of lines of open source software that is now being used by multiple ECP teams, HPC software vendors, and international collaborators. The uptake of Mochi technologies speaks to both the value of the service composition model and the quality of architecture and software developed by the team. Through continued funding under ASCR and ECP, the Mochi team and its technologies are on track to further contribute to DOE science successes into the exascale timeframe.
Research Organization:
The HDF Group, Champaign, IL (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR). Scientific Discovery through Advanced Computing (SciDAC)
DOE Contract Number:
SC0016474
OSTI ID:
1544789
Report Number(s):
DOE-HDF--16474
Country of Publication:
United States
Language:
English

Similar Records

Large Scale Computing and Storage Requirements for Biological and Environmental Research
Technical Report · Wed Sep 30 00:00:00 EDT 2009 · OSTI ID:974254
Science-Driven Computing: NERSC's Plan for 2006-2010
Technical Report · Mon May 16 00:00:00 EDT 2005 · OSTI ID:876211
Large Scale Computing and Storage Requirements for Basic Energy Sciences Research
Technical Report · Thu Mar 31 00:00:00 EDT 2011 · OSTI ID:1050635

Related Subjects

97 MATHEMATICS AND COMPUTING