Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Performance Analysis, Design Considerations, and Applications of Extreme-Scale In Situ Infrastructures

Journal Article · · International Conference for High Performance Computing, Networking, Storage and Analysis (Online)
DOI:https://doi.org/10.1109/SC.2016.78· OSTI ID:1379776
 [1];  [1];  [2];  [3];  [4];  [5];  [6];  [5];  [5];  [3];  [5];  [1];  [3];  [7];  [8];  [4];  [5];  [2];  [3];  [5] more »;  [5] « less
  1. Kitware, Inc., Clifton Park, NY (United States)
  2. Intelligent Light, Rutherford, NJ (United States)
  3. Georgia Inst. of Technology, Atlanta, GA (United States)
  4. Argonne National Lab. (ANL), Argonne, IL (United States)
  5. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  6. Univ. of Colorado, Boulder, CO (United States)
  7. Univ. of Colorado, Boulder, CO (United States); Cenaero, Charleroi (Belgium)
  8. Computational Science and Engineering, LLC, Annapolis, MD (United States)
+ Show Author Affiliations

A key trend facing extreme-scale computational science is the widening gap between computational and I/O rates, and the challenge that follows is how to best gain insight from simulation data when it is increasingly impractical to save it to persistent storage for subsequent visual exploration and analysis. One approach to this challenge is centered around the idea of in situ processing, where visualization and analysis processing is performed while data is still resident in memory. Our paper examines several key design and performance issues related to the idea of in situ processing at extreme scale on modern platforms: Scalability, overhead, performance measurement and analysis, comparison and contrast with a traditional post hoc approach, and interfacing with simulation codes. Finally, we illustrate these principles in practice with studies, conducted on large-scale HPC platforms, that include a miniapplication and multiple science application codes, one of which demonstrates in situ methods in use at greater than 1M-way concurrency.

Research Organization:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
Grant/Contract Number:
AC02-05CH11231
OSTI ID:
1379776
Alternate ID(s):
OSTI ID: 1595254
OSTI ID: 1468608
Journal Information:
International Conference for High Performance Computing, Networking, Storage and Analysis (Online), Journal Name: International Conference for High Performance Computing, Networking, Storage and Analysis (Online) Vol. 2017; ISSN 2167-4337
Publisher:
IEEECopyright Statement
Country of Publication:
United States
Language:
English

Cited By (1)

Scalable In situ Analysis of Molecular Dynamics Simulations
  • Malakar, Preeti; Knight, Christopher; Munson, Todd
  • Proceedings of the In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization - ISAV'17 https://doi.org/10.1145/3144769.3144772
conference January 2017

Similar Records

Performance Analysis, Design Considerations, and Applications of Extreme-Scale In Situ Infrastructures
Conference · Wed Sep 05 00:00:00 EDT 2018 · OSTI ID:1468608
Scalable Analysis Methods and In Situ Infrastructure for Extreme Scale Knowledge Discovery
Technical Report · Sun Jul 24 00:00:00 EDT 2016 · OSTI ID:1421430

Related Subjects

97 MATHEMATICS AND COMPUTING
analytical models
computational modeling
concurrent computing
data models
data visualization
libraries
scientific computing