skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure

Abstract

This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies (i.e ., automated code analysis and instrumentation), in-situ infrastructure (i.e ., ADIOS) and big data analysis engines (i.e ., SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. Finally, the in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.

Authors:
ORCiD logo [1];  [2]; ORCiD logo [1]; ORCiD logo [3]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Environmental Science Division
  2. Univ. of Tennessee, Knoxville, TN (United States). Dept. of Computer Science and Electric Engineering
  3. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Computer Science and Mathematics Division
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
Sponsoring Org.:
USDOE Office of Science (SC), Biological and Environmental Research (BER) (SC-23)
OSTI Identifier:
1463975
Grant/Contract Number:  
AC05-00OR22725
Resource Type:
Accepted Manuscript
Journal Name:
Journal of Computer and Communications
Additional Journal Information:
Journal Volume: 5; Journal Issue: 14; Journal ID: ISSN 2327-5219
Publisher:
Scientific Research Publishing, Inc.
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; In-Situ Data Analysis; Source Code Analysis; Data Staging; ADIOS; Earth System Model; Machine Learning; SciKit-Learn; E3SM

Citation Formats

Wang, Dali, Luo, X., Yuan, Fengming, and Podhorszki, Norbert. A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure. United States: N. p., 2017. Web. doi:10.4236/jcc.2017.514007.
Wang, Dali, Luo, X., Yuan, Fengming, & Podhorszki, Norbert. A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure. United States. doi:10.4236/jcc.2017.514007.
Wang, Dali, Luo, X., Yuan, Fengming, and Podhorszki, Norbert. Fri . "A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure". United States. doi:10.4236/jcc.2017.514007. https://www.osti.gov/servlets/purl/1463975.
@article{osti_1463975,
title = {A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure},
author = {Wang, Dali and Luo, X. and Yuan, Fengming and Podhorszki, Norbert},
abstractNote = {This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies (i.e ., automated code analysis and instrumentation), in-situ infrastructure (i.e ., ADIOS) and big data analysis engines (i.e ., SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. Finally, the in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.},
doi = {10.4236/jcc.2017.514007},
journal = {Journal of Computer and Communications},
number = 14,
volume = 5,
place = {United States},
year = {2017},
month = {12}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Figures / Tables:

Figure 1 Figure 1: A procedure of in-situ data analysis.

Save / Share:
Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.