skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Where Big Data and Prediction Meet

Abstract

Our ability to assemble and analyze massive data sets, often referred to under the title of “big data”, is an increasingly important tool for shaping national policy. This in turn has introduced issues from privacy concerns to cyber security. But as IBM’s John Kelly emphasized in the last Innovation, making sense of the vast arrays of data will require radically new computing tools. In the past, technologies and tools for analysis of big data were viewed as quite different from the traditional realm of high performance computing (HPC) with its huge models of phenomena such as global climate or supporting the nuclear test moratorium. Looking ahead, this will change with very positive benefits for both worlds. Societal issues such as global security, economic planning and genetic analysis demand increased understanding that goes beyond existing data analysis and reduction. The modeling world often produces simulations that are complex compositions of mathematical models and experimental data. This has resulted in outstanding successes such as the annual assessment of the state of the US nuclear weapons stockpile without underground nuclear testing. Ironically, while there were historically many test conducted, this body of data provides only modest insight into the underlying physics of themore » system. A great deal of emphasis was thus placed on the level of confidence we can develop for the predictions. As data analytics and simulation come together, there is a growing need to assess the confidence levels in both data being gathered and the complex models used to make predictions. An example of this is assuring the security or optimizing the performance of critical infrastructure systems such as the power grid. If one wants to understand the vulnerabilities of the system or impacts of predicted threats, full scales tests of the grid against threat scenarios are unlikely. Preventive measures would need to be predicated on well-defined margins of confidence in order to take mitigating actions that could have wide ranging impacts. There is a rich opportunity for interaction and exchange between the HPC simulation and data analytics communities.« less

Authors:
 [1];  [2];  [3];  [4];  [5]
  1. Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
  2. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
  3. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
  4. USDOE, Washington, DC (United States)
  5. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Publication Date:
Research Org.:
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1169890
Report Number(s):
LLNL-TR-660223
DOE Contract Number:
AC52-07NA27344
Resource Type:
Technical Report
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE

Citation Formats

Ahrens, James, Brase, Jim M., Hart, Bill, Kusnezov, Dimitri, and Shalf, John. Where Big Data and Prediction Meet. United States: N. p., 2014. Web. doi:10.2172/1169890.
Ahrens, James, Brase, Jim M., Hart, Bill, Kusnezov, Dimitri, & Shalf, John. Where Big Data and Prediction Meet. United States. doi:10.2172/1169890.
Ahrens, James, Brase, Jim M., Hart, Bill, Kusnezov, Dimitri, and Shalf, John. 2014. "Where Big Data and Prediction Meet". United States. doi:10.2172/1169890. https://www.osti.gov/servlets/purl/1169890.
@article{osti_1169890,
title = {Where Big Data and Prediction Meet},
author = {Ahrens, James and Brase, Jim M. and Hart, Bill and Kusnezov, Dimitri and Shalf, John},
abstractNote = {Our ability to assemble and analyze massive data sets, often referred to under the title of “big data”, is an increasingly important tool for shaping national policy. This in turn has introduced issues from privacy concerns to cyber security. But as IBM’s John Kelly emphasized in the last Innovation, making sense of the vast arrays of data will require radically new computing tools. In the past, technologies and tools for analysis of big data were viewed as quite different from the traditional realm of high performance computing (HPC) with its huge models of phenomena such as global climate or supporting the nuclear test moratorium. Looking ahead, this will change with very positive benefits for both worlds. Societal issues such as global security, economic planning and genetic analysis demand increased understanding that goes beyond existing data analysis and reduction. The modeling world often produces simulations that are complex compositions of mathematical models and experimental data. This has resulted in outstanding successes such as the annual assessment of the state of the US nuclear weapons stockpile without underground nuclear testing. Ironically, while there were historically many test conducted, this body of data provides only modest insight into the underlying physics of the system. A great deal of emphasis was thus placed on the level of confidence we can develop for the predictions. As data analytics and simulation come together, there is a growing need to assess the confidence levels in both data being gathered and the complex models used to make predictions. An example of this is assuring the security or optimizing the performance of critical infrastructure systems such as the power grid. If one wants to understand the vulnerabilities of the system or impacts of predicted threats, full scales tests of the grid against threat scenarios are unlikely. Preventive measures would need to be predicated on well-defined margins of confidence in order to take mitigating actions that could have wide ranging impacts. There is a rich opportunity for interaction and exchange between the HPC simulation and data analytics communities.},
doi = {10.2172/1169890},
journal = {},
number = ,
volume = ,
place = {United States},
year = 2014,
month = 9
}

Technical Report:

Save / Share:
  • In this presentation, the speaker discusses the apparent separation between the workplace and the community, the separation of the occupational-health problem from the community environmental-health problem and the question of jurisdiction which arises out of such attempts to separate the issue. He stresses the different origins of environmental and occupational-health considerations, reminding his audience that occupational-health concerns began as a surgical speciality, for treating workers injured on the job. In the community, concerns grew over the purity of public water supplies and adequate sanitation. Chronic problems involving both the workplace and the community include air and water pollution. The productionmore » of synthetic chemicals is continuously growing, and with this there is constant concern over exposure of workers during manufacture, use, and cleanup. The author recommends that occupational and community environmental professionals coordinate actions, and share information.« less
  • This is part of 10 special reports on the environmental assessment of stationary source NOx combustion modification technologies program. The program has two main objectives: (1) to identify the multimedia environmental impact of stationary combustion sources and NOx combustion modification controls; and (2) to identify the most cost-effective, environmentally sound NOx combustion modification controls for attaining and maintaining current and projected NO2 air quality standards to the year 2000. The emissions characterization effort documented in Volume I of this report provides supporting data to subsequent efforts. Volume II contains the appendices to this report. They are presented as follows: Regionalmore » fuel consumption tables for 1974; National fuel consumption tables for 1985 and 2000; National criteria pollutant emissions for 1974; National emissions inventory for 1974; Regional stationary source combustion controls; National source pollution potential assessment for 1974; Pollution potential summaries for the energy projection scenarios in 1985 and 2000, and Pollution potential trends to the year 2000 for the energy projection scenarios. These appendices support the report documentation given in Volume I.« less
  • A coordination meeting of the program was held at Argonne National Laboratory on September 17-19, 1986. Representatives from the participating laboratories and from the fusion technology community met to discuss nuclear data needs for fusion. Most of the standing nuclear data requests for fusion were discussed in considerable detail, and the status of the relevant data was reviewed. Task force groups were organized along disciplinary lines to address many of the issues which confront the program. Plans were laid for several collaborative endeavors, including technical projects to address specific data problems and an intercomparison of methods and codes in themore » area of nuclear modeling.« less
  • This review was prepared during a coordination meeting held at Oak Ridge National Laboratory on September 28-29, 1983. Participants included research scientists working for this program, a representative from the OFE's Coordination of Magnetic Fusion Energy (MFE) Nuclear Data Needs Activities, and invited specialists.
  • Experimental data acquired under the BES program to meet Fusion Data Needs from its inception (1975) to 1983 are summarized by element, reaction, energy, and contributing laboratory. Cases where the data have been incorporated in ENDF evaluations are noted. Work in nuclear model code development under this program is summarized.