Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

HPC s Pivot to Data

Conference ·
OSTI ID:1159414
 [1];  [1];  [2];  [3];  [2];  [1];  [1];  [1];  [1];  [2];  [1];  [1];  [3]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
  2. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)
  3. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Energy Sciences Network (ESnet)
+ Show Author Affiliations
Computer centers such as NERSC and OLCF have traditionally focused on delivering computational capability that enables breakthrough innovation in a wide range of science domains. Accessing that computational power has required services and tools to move the data from input and output to computation and storage. A ''pivot to data'' is occurring in HPC. Data transfer tools and services that were previously peripheral are becoming integral to scientific workflows. Emerging requirements from high-bandwidth detectors, high-throughput screening techniques, highly concur- rent simulations, increased focus on uncertainty quantification, and an emerging open-data policy posture toward published research are among the data-drivers shaping the networks, file systems, databases, and overall compute and data environment. In this paper we explain the pivot to data in HPC through user requirements and the changing resources provided by HPC with particular focus on data movement. For WAN data transfers we present the results of a study of network performance between centers.
Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
Sponsoring Organization:
USDOE Office of Science (SC)
DOE Contract Number:
AC05-00OR22725
OSTI ID:
1159414
Country of Publication:
United States
Language:
English

Similar Records

Enabling Low-Overhead HT-HPC Workflows at Extreme Scale using GNU Parallel
Conference · Thu Oct 31 20:00:00 EDT 2024 · OSTI ID:2538327
Performance Analysis, Modeling and Scaling of HPC Applications and Tools
Technical Report · Tue Jan 12 23:00:00 EST 2016 · OSTI ID:1240983
ExaFEL: extreme-scale real-time data processing for X-ray free electron laser science
Journal Article · Wed Oct 02 20:00:00 EDT 2024 · Frontiers in High Performance Computing · OSTI ID:2999940

Related Subjects

97 MATHEMATICS AND COMPUTING
Data
WAN
Workflows