Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

UMap: An application-oriented user level memory mapping library

Journal Article · · International Journal of High Performance Computing Applications
 [1];  [1];  [2];  [2];  [2];  [2]
  1. KTH Royal Inst. of Technology, Stockholm (Sweden)
  2. Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
+ Show Author Affiliations
Exploiting the prominent role of complex memories in exascale node architecture, the UMap page fault handler offers new capabilities to access large memory-mapped data sets directly. UMap provides flexible configuration options to customize page handling to each application, including analysis of massive observational and simulation data sets. The high-performance design features I/O decoupling, dynamic load balancing, and application-level controls. Page faults triggered by application threads and processes accessing data mapped to a UMapp’ed region are handled via the Linux userfaultfd protocol, an asynchronous message-oriented kernel-user communication mechanism that avoids the context switch penalty of traditional signal fault handlers. UMap is fully open source. Here, in this paper, we give an overview of the UMap library architecture, its extensible plugin architecture, and the use/performance of UMap in emerging heterogeneous memory hierarchies such as near-node Non-volatile Memory (NVM) and network attached memories. We highlight new capabilities in two pagefault management plugins, the NetworkStore and SparseStore. We demonstrate the integration between UMap and multiple ECP products including Caliper, Metall, ZFP, Mochi, and Ripples.
Research Organization:
Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
Sponsoring Organization:
USDOE; USDOE National Nuclear Security Administration (NNSA)
Grant/Contract Number:
AC52-07NA27344
OSTI ID:
2479926
Alternate ID(s):
OSTI ID: 2519315
Report Number(s):
LLNL--JRNL-860428; 1091831
Journal Information:
International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications Journal Issue: 2 Vol. 39; ISSN 1094-3420
Publisher:
SAGECopyright Statement
Country of Publication:
United States
Language:
English

References (19)

DI-MMAP—a scalable memory-map runtime for out-of-core data-intensive applications journal October 2013
Mochi: Composing Data Services for High-Performance Computing Environments journal January 2020
Metall: A persistent memory allocator for data-centric analytics journal July 2022
Community Structure in Large Networks: Natural Cluster Sizes and the Absence of Large Well-Defined Clusters journal January 2009
Fast and Scalable Implementations of Influence Maximization Algorithms conference September 2019
FAM-Graph: Graph Analytics on Disaggregated Memory conference May 2022
RAM as a Network Managed Resource conference May 2018
UMap: Enabling Application-driven Optimizations for Page Management conference November 2019
Evaluating Emerging CXL-enabled Memory Pooling for HPC Systems conference November 2022
On the Memory Underutilization: Exploring Disaggregated Memory on HPC Systems conference September 2020
Caliper: Performance Introspection for HPC Software Stacks
  • Boehme, David; Gamblin, Todd; Beckingsale, David
  • SC16: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2016.46
conference November 2016
Enabling Scalable and Extensible Memory-Mapped Datastores in Userspace journal April 2022
Fixed-Rate Compressed Floating-Point Arrays journal December 2014
Ligra journal February 2013
Efficient Memory-Mapped I/O on Fast Storage Device journal May 2016
Userland CO-PAGER
  • Li, Feng; Waddington, Daniel G.; Song, Fengguang
  • Proceedings of the 3rd International Conference on High Performance Compilation, Computing and Communications https://doi.org/10.1145/3318265.3318272
conference March 2019
SplitFS conference October 2019
Hailstorm
  • Bindschaedler, Laurent; Goel, Ashvin; Zwaenepoel, Willy
  • Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems https://doi.org/10.1145/3373376.3378504
conference March 2020
A Quantitative Approach for Adopting Disaggregated Memory in HPC Systems
  • Wahlgren, Jacob; Schieffer, Gabin; Gokhale, Maya
  • Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1145/3581784.3607108
conference November 2023

Similar Records

Enabling Scalable and Extensible Memory-mapped Datastores in Userspace
Journal Article · 2021 · IEEE Transactions on Parallel and Distributed Systems · OSTI ID:1829975
DRAGON: breaking GPU memory capacity limits with direct NVM access
Conference · 2018 · OSTI ID:1489577

Related Subjects

97 MATHEMATICS AND COMPUTING
Computer science
computer science