Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Metall: A persistent memory allocator for data-centric analytics

Journal Article · · Parallel Computing
 [1];  [2];  [3];  [1];  [1]
  1. Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
  2. Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States); Virginia Polytechnic Institute and State University (Virginia Tech), Blacksburg, VA (United States)
  3. University of Maryland Baltimore County (UMBC), Baltimore, MD (United States)
+ Show Author Affiliations

Data analytics applications transform raw input data into analytics-specific data structures before performing analytics. Unfortunately, such data ingestion steps are often more expensive than analytics. In addition, various types of NVRAM devices are already used in many HPC systems today. Such devices will be useful for storing and reusing data structures beyond a single process life cycle. We developed Metall, a persistent memory allocator built on top of the memory-mapped file mechanism. Metall enables applications to transparently allocate custom C++ data structures into various types of persistent memories. Metall incorporates a concise and high-performance memory management algorithm inspired by Supermalloc and the rich C++ interface developed by Boost.Interprocess library. On a dynamic graph construction workload, Metall achieved up to 11.7x and 48.3x performance improvements over Boost.Interprocess and memkind (PMEM kind), respectively. We also demonstrate Metall’s high adaptability by integrating Metall into a graph processing framework, GraphBLAS Template Library. Here this study’s outcomes indicate that Metall will be a strong tool for accelerating future large-scale data analytics by allowing applications to leverage persistent memory efficiently.

Research Organization:
Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
Sponsoring Organization:
USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC); USDOE Laboratory Directed Research and Development (LDRD) Program
Grant/Contract Number:
AC52-07NA27344
OSTI ID:
1962476
Report Number(s):
LLNL-JRNL-825588; 1032778
Journal Information:
Parallel Computing, Journal Name: Parallel Computing Journal Issue: N/A Vol. 111; ISSN 0167-8191
Publisher:
ElsevierCopyright Statement
Country of Publication:
United States
Language:
English

References (6)

Graphs over time: densification laws, shrinking diameters and possible explanations
  • Leskovec, Jure; Kleinberg, Jon; Faloutsos, Christos
  • Proceeding of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining - KDD '05 https://doi.org/10.1145/1081870.1081893
conference January 2005
Graph evolution: Densification and shrinking diameters journal March 2007
Signed networks in social media conference April 2010
Exascale computing and big data journal June 2015
Persistent Memory Programming on Conventional Hardware journal August 2019
A Prompt Report on the Performance of Intel Optane DC Persistent Memory Module journal May 2020

Similar Records

SharP Unified Memory Allocator: An Intent-Based Memory Allocator for Extreme-Scale Systems
Conference · Wed Aug 01 00:00:00 EDT 2018 · OSTI ID:1468045
Meta Allocator
Software · Fri Nov 30 19:00:00 EST 2018 · OSTI ID:code-24530
Makalu: fast recoverable allocation of non-volatile memory
Journal Article · Wed Oct 19 00:00:00 EDT 2016 · SIGPLAN · OSTI ID:1425366

Related Subjects

97 MATHEMATICS AND COMPUTING
graph processing
memory allocator
persistent memory