A Pipeline for Large Data Processing Using Regular Sampling for Unstructured Grids
- Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
- Virginia Polytechnic Inst. and State Univ. (Virginia Tech), Blacksburg, VA (United States)
- Univ. of Texas, Austin, TX (United States)
Large simulation data requires a lot of time and computational resources to compute, store, analyze, visualize, and run user studies. Today, the largest cost of a supercomputer is not hardware but maintenance, in particular energy consumption. Our goal is to balance energy consumption and cognitive value of visualizations of resulting data. This requires us to go through the entire processing pipeline, from simulation to user studies. To reduce the amount of resources, data can be sampled or compressed. While this adds more computation time, the computational overhead is negligible compared to the simulation time. We built a processing pipeline at the example of regular sampling. The reasons for this choice are two-fold: using a simple example reduces unnecessary complexity as we know what to expect from the results. Furthermore, it provides a good baseline for future, more elaborate sampling methods. We measured time and energy for each test we did, and we conducted user studies in Amazon Mechanical Turk (AMT) for a range of different results we produced through sampling.
- Research Organization:
- Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC)
- DOE Contract Number:
- AC52-06NA25396
- OSTI ID:
- 1357102
- Report Number(s):
- LA--UR-17-23903
- Country of Publication:
- United States
- Language:
- English
Similar Records
U.S. residential consumer product information: Validation of methods for post-stratification weighting of Amazon Mechanical Turk surveys
Online and Scalable Data Compression Pipeline with Guarantees on Quantities of Interest
Technical Report
·
Mon Apr 01 00:00:00 EDT 2013
·
OSTI ID:1171750
Online and Scalable Data Compression Pipeline with Guarantees on Quantities of Interest
Conference
·
Sun Oct 01 00:00:00 EDT 2023
·
OSTI ID:2251609