Network Traffic Analysis With Query Driven VisualizationSC 2005HPC Analytics Results
Our analytics challenge is to identify, characterize, and visualize anomalous subsets of large collections of network connection data. We use a combination of HPC resources, advanced algorithms, and visualization techniques. To effectively and efficiently identify the salient portions of the data, we rely on a multi-stage workflow that includes data acquisition, summarization (feature extraction), novelty detection, and classification. Once these subsets of interest have been identified and automatically characterized, we use a state-of-the-art-high-dimensional query system to extract data subsets for interactive visualization. Our approach is equally useful for other large-data analysis problems where it is more practical to identify interesting subsets of the data for visualization than to render all data elements. By reducing the size of the rendering workload, we enable highly interactive and useful visualizations. As a result of this work we were able to analyze six months worth of data interactively with response times two orders of magnitude shorter than with conventional methods.
- Research Organization:
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
- Sponsoring Organization:
- USDOE Director. Office of Science. Office of AdvancedScientific Computing Research, National Nuclear Security Adminsitrationand Department of Homeland Security National Visualization and AnalysisCenter
- DOE Contract Number:
- DE-AC02-05CH11231
- OSTI ID:
- 878116
- Report Number(s):
- LBNL-58768; R&D Project: KWC035; BnR: YN1901000; TRN: US200609%%178
- Resource Relation:
- Conference: Supercomputing 2005, Seattle, Washingoton,November 12-18, 2005
- Country of Publication:
- United States
- Language:
- English
Similar Records
An Application of Multivariate Statistical Analysis for Query-Driven Visualization
Query-driven visualization of large data sets