skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: An effective online data monitoring and saving strategy for large-scale climate simulations

Abstract

Large-scale climate simulation models have been developed and widely used to generate historical data and study future climate scenarios. These simulation models often have to run for a couple of months to understand the changes in the global climate over the course of decades. This long-duration simulation process creates a huge amount of data with both high temporal and spatial resolution information; however, how to effectively monitor and record the climate changes based on these large-scale simulation results that are continuously produced in real time still remains to be resolved. Due to the slow process of writing data to disk, the current practice is to save a snapshot of the simulation results at a constant, slow rate although the data generation process runs at a very high speed. This study proposes an effective online data monitoring and saving strategy over the temporal and spatial domains with the consideration of practical storage and memory capacity constraints. Finally, our proposed method is able to intelligently select and record the most informative extreme values in the raw data generated from real-time simulations in the context of better monitoring climate changes.

Authors:
 [1];  [2];  [2];  [1];  [3]
  1. Univ. of Wisconsin, Madison, WI (United States). Dept. of Industrial and Systems Engineering
  2. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Climate Change Science Inst.
  3. Xi'an Jiaotong Univ., Xi'an (China). School of Management. State Key Lab. for Manufacturing Systems Engineering
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Univ. of Wisconsin, Madison, WI (United States); Xi'an Jiaotong Univ., Xi'an (China)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21); National Science Foundation (NSF); National Natural Science Foundation of China (NNSFC)
OSTI Identifier:
1423034
Grant/Contract Number:  
AC05-00OR22725; CMMI-1362529; 71402133; 71602155; 71572138; 11501209
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Quality Technology & Quantitative Management
Additional Journal Information:
Journal Name: Quality Technology & Quantitative Management; Journal ID: ISSN 1684-3703
Country of Publication:
United States
Language:
English
Subject:
54 ENVIRONMENTAL SCIENCES; 97 MATHEMATICS AND COMPUTING; big data; local extrema; raw simulation data; spatial and temporal domains

Citation Formats

Xian, Xiaochen, Archibald, Rick, Mayer, Benjamin, Liu, Kaibo, and Li, Jian. An effective online data monitoring and saving strategy for large-scale climate simulations. United States: N. p., 2018. Web. doi:10.1080/16843703.2017.1414112.
Xian, Xiaochen, Archibald, Rick, Mayer, Benjamin, Liu, Kaibo, & Li, Jian. An effective online data monitoring and saving strategy for large-scale climate simulations. United States. doi:10.1080/16843703.2017.1414112.
Xian, Xiaochen, Archibald, Rick, Mayer, Benjamin, Liu, Kaibo, and Li, Jian. Mon . "An effective online data monitoring and saving strategy for large-scale climate simulations". United States. doi:10.1080/16843703.2017.1414112.
@article{osti_1423034,
title = {An effective online data monitoring and saving strategy for large-scale climate simulations},
author = {Xian, Xiaochen and Archibald, Rick and Mayer, Benjamin and Liu, Kaibo and Li, Jian},
abstractNote = {Large-scale climate simulation models have been developed and widely used to generate historical data and study future climate scenarios. These simulation models often have to run for a couple of months to understand the changes in the global climate over the course of decades. This long-duration simulation process creates a huge amount of data with both high temporal and spatial resolution information; however, how to effectively monitor and record the climate changes based on these large-scale simulation results that are continuously produced in real time still remains to be resolved. Due to the slow process of writing data to disk, the current practice is to save a snapshot of the simulation results at a constant, slow rate although the data generation process runs at a very high speed. This study proposes an effective online data monitoring and saving strategy over the temporal and spatial domains with the consideration of practical storage and memory capacity constraints. Finally, our proposed method is able to intelligently select and record the most informative extreme values in the raw data generated from real-time simulations in the context of better monitoring climate changes.},
doi = {10.1080/16843703.2017.1414112},
journal = {Quality Technology & Quantitative Management},
number = ,
volume = ,
place = {United States},
year = {Mon Jan 22 00:00:00 EST 2018},
month = {Mon Jan 22 00:00:00 EST 2018}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on January 22, 2019
Publisher's Version of Record

Save / Share: