skip to main content

Title: HPC s Pivot to Data

Computer centers such as NERSC and OLCF have traditionally focused on delivering computational capability that enables breakthrough innovation in a wide range of science domains. Accessing that computational power has required services and tools to move the data from input and output to computation and storage. A ''pivot to data'' is occurring in HPC. Data transfer tools and services that were previously peripheral are becoming integral to scientific workflows. Emerging requirements from high-bandwidth detectors, high-throughput screening techniques, highly concur- rent simulations, increased focus on uncertainty quantification, and an emerging open-data policy posture toward published research are among the data-drivers shaping the networks, file systems, databases, and overall compute and data environment. In this paper we explain the pivot to data in HPC through user requirements and the changing resources provided by HPC with particular focus on data movement. For WAN data transfers we present the results of a study of network performance between centers.
Authors:
 [1] ;  [1] ;  [2] ;  [3] ;  [2] ;  [1] ;  [1] ;  [1] ;  [1] ;  [2] ;  [1] ;  [1] ;  [3]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
  2. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)
  3. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Energy Sciences Network (ESnet)
Publication Date:
OSTI Identifier:
1159414
DOE Contract Number:
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: Cray Users Group Meeting, Lugano (Switzerland), 3-8 May 2014
Research Org:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
Sponsoring Org:
USDOE Office of Science (SC)
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING Data; WAN; Workflows