Remote visual analysis of large turbulence databases at multiple scales

Pulido, Jesus; Livescu, Daniel; Kanov, Kalin; Burns, Randal; Canada, Curtis; Ahrens, James; Hamann, Bernd

doi:10.1016/j.jpdc.2018.05.011

Title: Remote visual analysis of large turbulence databases at multiple scales

Abstract

The remote analysis and visualization of raw large turbulence datasets is challenging. Current accurate direct numerical simulations (DNS) of turbulent flows generate datasets with billions of points per time-step and several thousand time-steps per simulation. Until recently, the analysis and visualization of such datasets was restricted to scientists with access to large supercomputers. The public Johns Hopkins Turbulence database simplifies access to multi-terabyte turbulence datasets and facilitates the computation of statistics and extraction of features through the use of commodity hardware. In this paper, we present a framework designed around wavelet-based compression for high-speed visualization of large datasets and methods supporting multi-resolution analysis of turbulence. By integrating common technologies, this framework enables remote access to tools available on supercomputers and over 230 terabytes of DNS data over the Web. Finally, the database toolset is expanded by providing access to exploratory data analysis tools, such as wavelet decomposition capabilities and coherent feature extraction.

Authors:

Pulido, Jesus ^[1]; Livescu, Daniel ^[2]; Kanov, Kalin ^[3]; Burns, Randal ^[3]; Canada, Curtis ^[2]; Ahrens, James ^[2]; Hamann, Bernd ^[4]

Univ. of California, Davis, CA (United States). Dept. of Computer Science; Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Johns Hopkins Univ., Baltimore, MD (United States)
Univ. of California, Davis, CA (United States). Dept. of Computer Science

Publication Date:: Fri Jun 15 00:00:00 EDT 2018

Research Org.:: Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Sponsoring Org.:: USDOE National Nuclear Security Administration (NNSA)

OSTI Identifier:: 1457276

Alternate Identifier(s):: OSTI ID: 1582730

Report Number(s):: LA-UR-17-20757
Journal ID: ISSN 0743-7315

Grant/Contract Number:: AC52-06NA25396

Resource Type:: Accepted Manuscript

Journal Name:: Journal of Parallel and Distributed Computing

Additional Journal Information:: Journal Volume: 120; Journal ID: ISSN 0743-7315

Publisher:: Elsevier

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; Computer Science; Databases; Wavelets; Data Reduction; Remote Visualization; Distributed Systems; Turbulence

Citation Formats


                    Pulido, Jesus, Livescu, Daniel, Kanov, Kalin, Burns, Randal, Canada, Curtis, Ahrens, James, and Hamann, Bernd. Remote visual analysis of large turbulence databases at multiple scales.  United States: N. p., 2018. 
Web.  doi:10.1016/j.jpdc.2018.05.011.

Copy to clipboard


                    Pulido, Jesus, Livescu, Daniel, Kanov, Kalin, Burns, Randal, Canada, Curtis, Ahrens, James, & Hamann, Bernd. Remote visual analysis of large turbulence databases at multiple scales.  United States.  https://doi.org/10.1016/j.jpdc.2018.05.011

Copy to clipboard


                    Pulido, Jesus, Livescu, Daniel, Kanov, Kalin, Burns, Randal, Canada, Curtis, Ahrens, James, and Hamann, Bernd. Fri .  
"Remote visual analysis of large turbulence databases at multiple scales".  United States.  https://doi.org/10.1016/j.jpdc.2018.05.011.  https://www.osti.gov/servlets/purl/1457276.

Copy to clipboard


                    
@article{osti_1457276,

  title        = {Remote visual analysis of large turbulence databases at multiple scales},

  author       = {Pulido, Jesus and Livescu, Daniel and Kanov, Kalin and Burns, Randal and Canada, Curtis and Ahrens, James and Hamann, Bernd},

  abstractNote = {The remote analysis and visualization of raw large turbulence datasets is challenging. Current accurate direct numerical simulations (DNS) of turbulent flows generate datasets with billions of points per time-step and several thousand time-steps per simulation. Until recently, the analysis and visualization of such datasets was restricted to scientists with access to large supercomputers. The public Johns Hopkins Turbulence database simplifies access to multi-terabyte turbulence datasets and facilitates the computation of statistics and extraction of features through the use of commodity hardware. In this paper, we present a framework designed around wavelet-based compression for high-speed visualization of large datasets and methods supporting multi-resolution analysis of turbulence. By integrating common technologies, this framework enables remote access to tools available on supercomputers and over 230 terabytes of DNS data over the Web. Finally, the database toolset is expanded by providing access to exploratory data analysis tools, such as wavelet decomposition capabilities and coherent feature extraction.},

  doi          = {10.1016/j.jpdc.2018.05.011},

  journal      = {Journal of Parallel and Distributed Computing},

  number       = ,

  volume       = 120,

  place        = {United States},

  year         = {Fri Jun 15 00:00:00 EDT 2018},

  month        = {Fri Jun 15 00:00:00 EDT 2018}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (Publisher)

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1016/j.jpdc.2018.05.011

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 6 works

Citation information provided by
Web of Science

Figures / Tables:

Figure 1: Architecture overview of the JHTDB. Multiple N server nodes create a database cluster to serve K users. The web services module provides an API where remote users may place requests to the database.

All figures and tables (12 total)

Save / Share:

Export Metadata

Save to My Library

Figures / Tables found in this record:

Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.

Similar Records in DOE PAGES and OSTI.GOV collections:

RVA. 3-D Visualization and Analysis Software to Support Management of Oil and Gas Resources

Technical Report Keefer, Donald A. ; Shaffer, Eric G. ; Storsved, Brynne ; ...

A free software application, RVA, has been developed as a plugin to the US DOE-funded ParaView visualization package, to provide support in the visualization and analysis of complex reservoirs being managed using multi-fluid EOR techniques. RVA, for Reservoir Visualization and Analysis, was developed as an open-source plugin to the 64 bit Windows version of ParaView 3.14. RVA was developed at the University of Illinois at Urbana-Champaign, with contributions from the Illinois State Geological Survey, Department of Computer Science and National Center for Supercomputing Applications. RVA was designed to utilize and enhance the state-of-the-art visualization capabilities within ParaView, readily allowing jointmore »« less
https://doi.org/10.2172/1238338

Full Text Available
Feature-Based Statistical Analysis of Combustion Simulation Data

Journal Article Bennett, J ; Krishnamoorthy, V ; Liu, S ; ... - IEEE Transactions on Visualization and Computer Graphics

We present a new framework for feature-based statistical analysis of large-scale scientific data and demonstrate its effectiveness by analyzing features from Direct Numerical Simulations (DNS) of turbulent combustion. Turbulent flows are ubiquitous and account for transport and mixing processes in combustion, astrophysics, fusion, and climate modeling among other disciplines. They are also characterized by coherent structure or organized motion, i.e. nonlocal entities whose geometrical features can directly impact molecular mixing and reactive processes. While traditional multi-point statistics provide correlative information, they lack nonlocal structural information, and hence, fail to provide mechanistic causality information between organized fluid motion and mixing andmore »« less
https://doi.org/10.1109/TVCG.2011.199

Full Text Available
Final Report: Phase II Nevada Water Resources Data, Modeling, and Visualization (DMV) Center

Technical Report Jackman, Thomas ; Minor, Timothy ; Pohll, Gregory

Water is unquestionably a critical resource throughout the United States. In the semi-arid west -- an area stressed by increase in human population and sprawl of the built environment -- water is the most important limiting resource. Crucially, science must understand factors that affect availability and distribution of water. To sustain growing consumptive demand, science needs to translate understanding into reliable and robust predictions of availability under weather conditions that could be average but might be extreme. These predictions are needed to support current and long-term planning. Similar to the role of weather forecast and climate prediction, water prediction overmore »« less
https://doi.org/10.2172/1088043

Full Text Available
Middle East, North Africa and Western Eurasia Seismic Research Database

Conference O'Boyle, J L ; Ruppert, S D ; Hauk, T F ; ...

The National Nuclear Security Administration (NNSA) Ground-Based Nuclear Explosion Monitoring Research and Engineering (GNEM R&E) Program at Lawrence Livermore National Laboratory (LLNL) has made significant progress populating a comprehensive Seismic Research Database (SRDB) used for deriving seismic calibration parameters for the Middle East, North Africa and Western Eurasia (ME/NA/WE) regions. In addition to an overview of select individual information products, we present an overview of our visualization, integration, validation, and organizational processes. Development of these processes and the LLNL SRDB was necessitated by both the very large amount of data and information involved (over 15 terabytes) and the varied datamore »« less
Full Text Available
SciDAC Institute for Ultra-Scale Visualization: Activity Recognition for Ultra-Scale Visualization

Technical Report Silver, Deborah

Understanding the science behind ultra-scale simulations requires extracting meaning from data sets of hundreds of terabytes or more. Developing scalable parallel visualization algorithms is a key step enabling scientists to interact and visualize their data at this scale. However, at extreme scales, the datasets are so huge, there is not even enough time to view the data, let alone explore it with basic visualization methods. Automated tools are necessary for knowledge discovery -- to help sift through the information and isolate characteristic patterns, thereby enabling the scientist to study local interactions, the origin of features and their evolution in largemore »« less
https://doi.org/10.2172/1162059

Full Text Available

Similar Records