skip to main content

DOE PAGESDOE PAGES

Title: A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure

This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies (i.e ., automated code analysis and instrumentation), in-situ infrastructure (i.e ., ADIOS) and big data analysis engines (i.e ., SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. Finally, the in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.
Authors:
ORCiD logo [1] ;  [2] ; ORCiD logo [1] ; ORCiD logo [3]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Environmental Science Division
  2. Univ. of Tennessee, Knoxville, TN (United States). Dept. of Computer Science and Electric Engineering
  3. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Computer Science and Mathematics Division
Publication Date:
Grant/Contract Number:
AC05-00OR22725
Type:
Accepted Manuscript
Journal Name:
Journal of Computer and Communications
Additional Journal Information:
Journal Volume: 5; Journal Issue: 14; Journal ID: ISSN 2327-5219
Publisher:
Scientific Research
Research Org:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org:
USDOE Office of Science (SC), Biological and Environmental Research (BER) (SC-23)
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; In-Situ Data Analysis; Source Code Analysis; Data Staging; ADIOS; Earth System Model; Machine Learning; SciKit-Learn; E3SM
OSTI Identifier:
1463975

Wang, Dali, Luo, X., Yuan, Fengming, and Podhorszki, Norbert. A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure. United States: N. p., Web. doi:10.4236/jcc.2017.514007.
Wang, Dali, Luo, X., Yuan, Fengming, & Podhorszki, Norbert. A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure. United States. doi:10.4236/jcc.2017.514007.
Wang, Dali, Luo, X., Yuan, Fengming, and Podhorszki, Norbert. 2017. "A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure". United States. doi:10.4236/jcc.2017.514007. https://www.osti.gov/servlets/purl/1463975.
@article{osti_1463975,
title = {A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure},
author = {Wang, Dali and Luo, X. and Yuan, Fengming and Podhorszki, Norbert},
abstractNote = {This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies (i.e ., automated code analysis and instrumentation), in-situ infrastructure (i.e ., ADIOS) and big data analysis engines (i.e ., SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. Finally, the in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.},
doi = {10.4236/jcc.2017.514007},
journal = {Journal of Computer and Communications},
number = 14,
volume = 5,
place = {United States},
year = {2017},
month = {12}
}