skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Visualization and Analysis for Near-Real-Time Decision Making in Distributed Workflows

Abstract

Data driven science is becoming increasingly more common, complex, and is placing tremendous stresses on visualization and analysis frameworks. Data sources producing 10GB per second (and more) are becoming increasingly commonplace in both simulation, sensor and experimental sciences. These data sources, which are often distributed around the world, must be analyzed by teams of scientists that are also distributed. Enabling scientists to view, query and interact with such large volumes of data in near-real-time requires a rich fusion of visualization and analysis techniques, middleware and workflow systems. Here, this paper discusses initial research into visualization and analysis of distributed data workflows that enables scientists to make near-real-time decisions of large volumes of time varying data.

Authors:
 [1];  [2];  [1];  [1];  [3];  [4];  [5];  [5];  [6];  [7];  [7];  [7];  [8]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  2. Univ. of Oregon, Eugene, OR (United States); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  3. Stony Brook Univ., NY (United States); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  4. Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
  5. Georgia Inst. of Technology, Atlanta, GA (United States)
  6. Univ. of Oregon, Eugene, OR (United States); Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  7. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  8. A*STAR Computational Resource Centre
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1379523
Grant/Contract Number:
AC02-05CH11231
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016
Additional Journal Information:
Conference: 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Chicago, IL (United States), 23-27 May 2016
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Data visualization; Data models; Computational modeling; Distributed databases; Plasmas; Middleware; Wide area networks

Citation Formats

Pugmire, David, Kress, James, Choi, Jong, Klasky, Scott, Kurc, Tahsin, Churchill, Randy Michael, Wolf, Matthew, Eisenhower, Greg, Childs, Hank, Wu, Kesheng, Sim, Alexander, Gu, Junmin, and Low, Jonathan. Visualization and Analysis for Near-Real-Time Decision Making in Distributed Workflows. United States: N. p., 2016. Web. doi:10.1109/IPDPSW.2016.175.
Pugmire, David, Kress, James, Choi, Jong, Klasky, Scott, Kurc, Tahsin, Churchill, Randy Michael, Wolf, Matthew, Eisenhower, Greg, Childs, Hank, Wu, Kesheng, Sim, Alexander, Gu, Junmin, & Low, Jonathan. Visualization and Analysis for Near-Real-Time Decision Making in Distributed Workflows. United States. doi:10.1109/IPDPSW.2016.175.
Pugmire, David, Kress, James, Choi, Jong, Klasky, Scott, Kurc, Tahsin, Churchill, Randy Michael, Wolf, Matthew, Eisenhower, Greg, Childs, Hank, Wu, Kesheng, Sim, Alexander, Gu, Junmin, and Low, Jonathan. 2016. "Visualization and Analysis for Near-Real-Time Decision Making in Distributed Workflows". United States. doi:10.1109/IPDPSW.2016.175. https://www.osti.gov/servlets/purl/1379523.
@article{osti_1379523,
title = {Visualization and Analysis for Near-Real-Time Decision Making in Distributed Workflows},
author = {Pugmire, David and Kress, James and Choi, Jong and Klasky, Scott and Kurc, Tahsin and Churchill, Randy Michael and Wolf, Matthew and Eisenhower, Greg and Childs, Hank and Wu, Kesheng and Sim, Alexander and Gu, Junmin and Low, Jonathan},
abstractNote = {Data driven science is becoming increasingly more common, complex, and is placing tremendous stresses on visualization and analysis frameworks. Data sources producing 10GB per second (and more) are becoming increasingly commonplace in both simulation, sensor and experimental sciences. These data sources, which are often distributed around the world, must be analyzed by teams of scientists that are also distributed. Enabling scientists to view, query and interact with such large volumes of data in near-real-time requires a rich fusion of visualization and analysis techniques, middleware and workflow systems. Here, this paper discusses initial research into visualization and analysis of distributed data workflows that enables scientists to make near-real-time decisions of large volumes of time varying data.},
doi = {10.1109/IPDPSW.2016.175},
journal = {Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016},
number = ,
volume = ,
place = {United States},
year = 2016,
month = 8
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:
  • The ability to reach an interpretation while fundamental exploration and production processes are underway holds the promise of substantially improving outcomes while acquiring data, drilling wells and managing reservoirs. This article presents real-time interpretation benefits and technology associated with it. Related technology topics discussed are: parallel processing; information access technology; 3-D visualization; integration framework; and the interpretation system of the future.
  • Purpose: This study presents an adaptive error detection algorithm (AEDA) for real-timein vivo point dosimetry during high dose rate (HDR) or pulsed dose rate (PDR) brachytherapy (BT) where the error identification, in contrast to existing approaches, does not depend on an a priori reconstruction of the dosimeter position. Instead, the treatment is judged based on dose rate comparisons between measurements and calculations of the most viable dosimeter position provided by the AEDA in a data driven approach. As a result, the AEDA compensates for false error cases related to systematic effects of the dosimeter position reconstruction. Given its nearly exclusivemore » dependence on stable dosimeter positioning, the AEDA allows for a substantially simplified and time efficient real-time in vivo BT dosimetry implementation. Methods: In the event of a measured potential treatment error, the AEDA proposes the most viable dosimeter position out of alternatives to the original reconstruction by means of a data driven matching procedure between dose rate distributions. If measured dose rates do not differ significantly from the most viable alternative, the initial error indication may be attributed to a mispositioned or misreconstructed dosimeter (false error). However, if the error declaration persists, no viable dosimeter position can be found to explain the error, hence the discrepancy is more likely to originate from a misplaced or misreconstructed source applicator or from erroneously connected source guide tubes (true error). Results: The AEDA applied on twoin vivo dosimetry implementations for pulsed dose rate BT demonstrated that the AEDA correctly described effects responsible for initial error indications. The AEDA was able to correctly identify the major part of all permutations of simulated guide tube swap errors and simulated shifts of individual needles from the original reconstruction. Unidentified errors corresponded to scenarios where the dosimeter position was sufficiently symmetric with respect to error and no-error source position constellations. The AEDA was able to correctly identify all false errors represented by mispositioned dosimeters contrary to an error detection algorithm relying on the original reconstruction. Conclusions: The study demonstrates that the AEDA error identification during HDR/PDR BT relies on a stable dosimeter position rather than on an accurate dosimeter reconstruction, and the AEDA’s capacity to distinguish between true and false error scenarios. The study further shows that the AEDA can offer guidance in decision making in the event of potential errors detected with real-timein vivo point dosimetry.« less
  • In the beginning of nuclear material safeguards, emphasis was placed on safe detection of diversion of nuclear material. Later, the aspect of timely detection became equally important. Since there is a trade-off between these two objectives, the question of an appropriate compromise was raised. In this paper, a decision theoretical framework is presented in which the objectives of the two players, inspector and inspectee, are expressed in terms of general utility functions. Within this framework, optimal safeguards strategies are defined, and furthermore, conditions are formulated under which the optimization criteria corresponding to the objectives mentioned above can be justified.
  • A critical evaluation of the many environmental issues facing EPA Region 3 has established five major priorities: (1) ozone pollution (and its precursors); (2) impacts of acidification (acid deposition and acid mine drainage); (3) eutrophication of the Chesapeake Bay from atmospheric nitrogen deposition; (4) Cities/Urban Environment (ozone, particulate matter (PM), air toxics are some of the air components); and (5) Climate Change. Recognizing the complex nature of the systems controlling these issues, Region III's Air Protection Division (APD) is developing a decision support tool, i.e., the Decision Consequence Model (DCM), that will integrate and automate the analysis of environmental impactsmore » in a manner that allows them to holistically address these regional priorities. Using this tool the authors intend to consider the interdependency of pollutants and their environmental impacts in order to support real-time decision making. The purpose of this paper is to outline the basic concept of the DCM and to present an example set of environmental indicators to illustrate how the DCM will be used to evaluate environmental impacts. The authors will discuss their process of indicator development, and present an example suite of indicators to provide a concrete example of the concepts presented above and, to illustrate the utility of the DCM to simultaneously evaluate multiple effects of a single pollutant. They will discuss the type of indicators chosen for this example as well as the general criteria the DCM indicators must satisfy. The framework that was developed to construct the indicators is discussed and used to calculate the example indicators. The yearly magnitudes of these example indicators are calculated for various multi-year periods to show their behavior over time.« less