skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Knowledge Discovery from Sensor Data For Scientific Applications

Abstract

Sensor networks consist of distributed autonomous devices that cooperatively monitor an environment. Sensors are equipped with capacities to store information in memory, process this information and communicate with their neighbors. Processing data streams generated from wireless sensor networks has raised new research challenges over the last few years due to the huge numbers of data streams to be managed continuously and at a very high rate. The book provides the reader with a comprehensive overview of stream data processing, including famous prototype implementations like the Nile system and the TinyOS operating system.

Authors:
 [1];  [1];  [1];  [1];  [1]
  1. ORNL
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Laboratory Directed Research and Development (LDRD) Program
OSTI Identifier:
932049
DOE Contract Number:
DE-AC05-00OR22725
Resource Type:
Book
Country of Publication:
United States
Language:
English
Subject:
99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; SENSORS; DATA TRANSMISSION; DATA PROCESSING; DATA ANALYSIS; INFORMATION RETRIEVAL

Citation Formats

Ganguly, Auroop R, Omitaomu, Olufemi A, Fang, Yi, Khan, Shiraj, and Bhaduri, Budhendra L. Knowledge Discovery from Sensor Data For Scientific Applications. United States: N. p., 2007. Web.
Ganguly, Auroop R, Omitaomu, Olufemi A, Fang, Yi, Khan, Shiraj, & Bhaduri, Budhendra L. Knowledge Discovery from Sensor Data For Scientific Applications. United States.
Ganguly, Auroop R, Omitaomu, Olufemi A, Fang, Yi, Khan, Shiraj, and Bhaduri, Budhendra L. Mon . "Knowledge Discovery from Sensor Data For Scientific Applications". United States. doi:.
@article{osti_932049,
title = {Knowledge Discovery from Sensor Data For Scientific Applications},
author = {Ganguly, Auroop R and Omitaomu, Olufemi A and Fang, Yi and Khan, Shiraj and Bhaduri, Budhendra L},
abstractNote = {Sensor networks consist of distributed autonomous devices that cooperatively monitor an environment. Sensors are equipped with capacities to store information in memory, process this information and communicate with their neighbors. Processing data streams generated from wireless sensor networks has raised new research challenges over the last few years due to the huge numbers of data streams to be managed continuously and at a very high rate. The book provides the reader with a comprehensive overview of stream data processing, including famous prototype implementations like the Nile system and the TinyOS operating system.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Mon Jan 01 00:00:00 EST 2007},
month = {Mon Jan 01 00:00:00 EST 2007}
}

Book:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this book.

Save / Share:
  • Evolving threat situations in a post 9/11 world demand faster and more reliable decisions to thwart the adversary. One critical path to enhanced threat cognizance is through online knowledge discovery based on dynamic, heterogeneous data available from strategically placed wide-area sensor networks. The knowledge discovery process needs to coordinate adaptive predictive analysis with real-time analysis and decision support systems. The ability to detect precursors and signatures of rare events and change from massive and disparate data in real-time may require a paradigm shift in the science of knowledge discovery. This chapter describes a case study in the area of transportationmore » security to describe both the key challenges, as well as the possible solution strategies, in this high-priority area. A suite of knowledge discovery tools developed for the purpose is described along with a discussion on future requirements.« less
  • No abstract prepared.
  • Knowledge discovery from large and complex scientific data is a challenging task. With the ability to measure and simulate more processes at increasingly finer spatial and temporal scales, the growing number of data dimensions and data objects presents tremendous challenges for effective data analysis and data exploration methods and tools. The combination and close integration of methods from scientific visualization, information visualization, automated data analysis, and other enabling technologies"such as efficient data management" supports knowledge discovery from multi-dimensional scientific data. This paper surveys two distinct applications in developmental biology and accelerator physics, illustrating the effectiveness of the described approach.
  • Recent advances in distributed sensor network technology and increasing deployment of such networks to monitor wide variety of dynamic phenomena has opened new opportunities to the knowledge discovery community. Extracting knowledge and emerging patterns from sensor data is a nontrivial task. The challenges for the knowledge discovery community are expected to be immense. On one hand, dynamic data streams or events require real-time analysis methodologies and systems, while on the other hand centralized processing through high end computing is also required for generating offline predictive insights, which in turn can facilitate real-time analysis. In addition, emerging societal problems require knowledgemore » discovery solutions that are designed to investigate anomalies, changes, extremes and nonlinear processes, and departures from the normal. Keeping in view the requirements of the emerging field of knowledge discovery from sensor data, we took initiative to develop a community of researchers with common interests and scientific goals, which culminated into the organization of Sensor-KDD series of workshops in conjunction with the prestigious ACM SIGKDD International Conference of Knowledge Discovery and Data Mining. In this chapter we summarize the events of the Second ACM-SIGKDD International Workshop on Knowledge Discovery form Sensor Data (Sensor-KDD 2008).« less
  • Big data is prevalent in HPC computing. Many HPC projects rely on complex workflows to analyze terabytes or petabytes of data. These workflows often require running over thousands of CPU cores and performing simultaneous data accesses, data movements, and computation. It is challenging to analyze the performance involving terabytes or petabytes of workflow data or measurement data of the executions, from complex workflows over a large number of nodes and multiple parallel task executions. To help identify performance bottlenecks or debug the performance issues in large-scale scientific applications and scientific clusters, we have developed a performance analysis framework, using state-ofthe-more » art open-source big data processing tools. Our tool can ingest system logs and application performance measurements to extract key performance features, and apply the most sophisticated statistical tools and data mining methods on the performance data. It utilizes an efficient data processing engine to allow users to interactively analyze a large amount of different types of logs and measurements. To illustrate the functionality of the big data analysis framework, we conduct case studies on the workflows from an astronomy project known as the Palomar Transient Factory (PTF) and the job logs from the genome analysis scientific cluster. Our study processed many terabytes of system logs and application performance measurements collected on the HPC systems at NERSC. The implementation of our tool is generic enough to be used for analyzing the performance of other HPC systems and Big Data workows.« less