A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure

Wang, Dali; Luo, X.; Yuan, Fengming; Podhorszki, Norbert

doi:10.4236/jcc.2017.514007

Title: A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure

Abstract

This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies (i.e ., automated code analysis and instrumentation), in-situ infrastructure (i.e ., ADIOS) and big data analysis engines (i.e ., SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. Finally, the in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.

Authors:

^[1]; Luo, X. ^[2];

^[1];

^[3]

Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Environmental Science Division
Univ. of Tennessee, Knoxville, TN (United States). Dept. of Computer Science and Electric Engineering
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Computer Science and Mathematics Division

Publication Date:: Fri Dec 29 00:00:00 EST 2017

Research Org.:: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)

Sponsoring Org.:: USDOE Office of Science (SC), Biological and Environmental Research (BER)

OSTI Identifier:: 1463975

Grant/Contract Number:: AC05-00OR22725

Resource Type:: Accepted Manuscript

Journal Name:: Journal of Computer and Communications

Additional Journal Information:: Journal Volume: 5; Journal Issue: 14; Journal ID: ISSN 2327-5219

Publisher:: Scientific Research Publishing, Inc.

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; In-Situ Data Analysis; Source Code Analysis; Data Staging; ADIOS; Earth System Model; Machine Learning; SciKit-Learn; E3SM

Citation Formats


                    Wang, Dali, Luo, X., Yuan, Fengming, and Podhorszki, Norbert. A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure.  United States: N. p., 2017. 
Web.  doi:10.4236/jcc.2017.514007.

Copy to clipboard


                    Wang, Dali, Luo, X., Yuan, Fengming, & Podhorszki, Norbert. A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure.  United States.  https://doi.org/10.4236/jcc.2017.514007

Copy to clipboard


                    Wang, Dali, Luo, X., Yuan, Fengming, and Podhorszki, Norbert. Fri .  
"A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure".  United States.  https://doi.org/10.4236/jcc.2017.514007.  https://www.osti.gov/servlets/purl/1463975.

Copy to clipboard


                    
@article{osti_1463975,

  title        = {A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure},

  author       = {Wang, Dali and Luo, X. and Yuan, Fengming and Podhorszki, Norbert},

  abstractNote = {This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies (i.e ., automated code analysis and instrumentation), in-situ infrastructure (i.e ., ADIOS) and big data analysis engines (i.e ., SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. Finally, the in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.},

  doi          = {10.4236/jcc.2017.514007},

  journal      = {Journal of Computer and Communications},

  number       = 14,

  volume       = 5,

  place        = {United States},

  year         = {Fri Dec 29 00:00:00 EST 2017},

  month        = {Fri Dec 29 00:00:00 EST 2017}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.4236/jcc.2017.514007

Other availability

Search WorldCat to find libraries that may hold this journal

Figures / Tables:

Figure 1: A procedure of in-situ data analysis.

All figures and tables (5 total)

Save / Share:

Export Metadata

Save to My Library

Figures / Tables found in this record:

Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.

Similar Records in DOE PAGES and OSTI.GOV collections:

Asynchronous In Situ Connected-Components Analysis for Complex Fluid flows, In: ISAV '16 Proceedings of the 2nd Workshop on In Situ Infrastructures for Enabling Extreme-scale Analysis and Visualization

Conference McClure, James E. ; Berrill, Mark A. ; Prins, Jan F. ; ... - PROCEEDINGS OF ISAV 2016: 2ND WORKSHOP ON IN SITU INFRASTRUCTURES FOR ENABLING EXTREME-SCALE ANALYSIS AND VISUALIZATION

The simulation of multiscale physics is an important challenge for scientific computing. For this class of problem, large three-dimensional simulations are performed to advance scientific inquiry. On massively parallel computing systems, the volume of data generated by such approaches can become a productivity bottleneck if the raw data generated from the simulation is analyzed in a post-processing step. To address this, we present a physics-based framework for in situ data reduction that is theoretically grounded in multiscale averaging theory. We show how task parallelism can be exploited to concurrently perform a variety of analysis tasks with data-dependent costs, including themore »« less
https://doi.org/10.1109/ISAV.2016.8
Enhancing the Earth System Grid Authentication Infrastructure through Single Sign-On and Autoprovisioning

Conference Siebenlist, Frank ; Bernholdt, David E. ; Williams, Dean N.

Climate scientists face an overarching need to efficiently access and manipulate climate model data. Increasingly, researchers must assemble and analyze large datasets that are archived in different formats on disparate platforms and must extract portions of datasets to compute statistical or diagnostic metrics in place. The need for a common virtual environment in which to access both climate model datasets and analysis tools is therefore keenly felt. The software infrastructure to support such an environment must not only provide ready access to climate data but must also facilitate the use of visualization software, diagnostic algorithms, and related resources. To thismore »« less
Utilizing rare earth elements as tracers in high TDS reservoir brines in CCS applications

Journal Article McLing, Travis ; Smith, William ; Smith, Robert - Energy Procedia (Online)

In this paper we report the result of research associated with the testing of a procedures necessary for utilizing natural occurring trace elements, specifically the Rare Earth Elements (REE) as geochemical tracers in Carbon Capture and Storage (CCS) applications. Trace elements, particularly REE may be well suited to serve as in situ tracers for monitoring geochemical conditions and the migration of CO₂-charged waters within CCS storage systems. We have been conducting studies to determine the efficacy of using REE as a tracer and characterization tool in the laboratory, at a CCS analogue site in Soda Springs, Idaho, and at amore »« less
Cited by 5
https://doi.org/10.1016/j.egypro.2014.11.426

Full Text Available
Distributed Resources for the Earth System Grid Federation (ESGF) Advanced Management (DREAM). Final Report

Technical Report Ames, Sasha ; Balaji, V. ; Christensen, Cameron ; ...

Distributed Resources for the Earth System Grid Federation (ESGF) Advanced Management (DREAM) is a proposed system that will enable data from an infinite number of diverse sources to be organized and accessed from anywhere using any handheld or other computer device. The approach offers a powerful roadmap for the creation and integration of a unified knowledge base of an entire ecosystem, including its many geophysical, geographical, social, political, agricultural, energy, transportation, and cyber aspects. The resulting aggregation of data has the potential to generate an informational universe of unprecedented size that has never before been possible due to the prohibitivemore »« less
https://doi.org/10.2172/1618423

Full Text Available
Plant science decadal vision 2020–2030: Reimagining the potential of plants for a healthy and sustainable future

Journal Article Henkhaus, Natalie ; Bartlett, Madelaine ; Gang, David ; ... - Plant Direct

Abstract Plants, and the biological systems around them, are key to the future health of the planet and its inhabitants. The Plant Science Decadal Vision 2020–2030 frames our ability to perform vital and far‐reaching research in plant systems sciences, essential to how we value participants and apply emerging technologies. We outline a comprehensive vision for addressing some of our most pressing global problems through discovery, practical applications, and education. The Decadal Vision was developed by the participants at the Plant Summit 2019, a community event organized by the Plant Science Research Network. The Decadal Vision describes a holistic vision formore »« less
https://doi.org/10.1002/pld3.252

Similar Records