Accelerating Network Traffic Analytics Using Query-DrivenVisualization
Realizing operational analytics solutions where large and complex data must be analyzed in a time-critical fashion entails integrating many different types of technology. This paper focuses on an interdisciplinary combination of scientific data management and visualization/analysis technologies targeted at reducing the time required for data filtering, querying, hypothesis testing and knowledge discovery in the domain of network connection data analysis. We show that use of compressed bitmap indexing can quickly answer queries in an interactive visual data analysis application, and compare its performance with two alternatives for serial and parallel filtering/querying on 2.5 billion records worth of network connection data collected over a period of 42 weeks. Our approach to visual network connection data exploration centers on two primary factors: interactive ad-hoc and multiresolution query formulation and execution over n dimensions and visual display of then-dimensional histogram results. This combination is applied in a case study to detect a distributed network scan and to then identify the set of remote hosts participating in the attack. Our approach is sufficiently general to be applied to a diverse set of data understanding problems as well as used in conjunction with a diverse set of analysis and visualization tools.
- Research Organization:
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
- Sponsoring Organization:
- USDOE Director. Office of Science. Advanced ScientificComputing Research, Los Alamos National Laboratory
- DOE Contract Number:
- DE-AC02-05CH11231
- OSTI ID:
- 898855
- Report Number(s):
- LBNL-59891; R&D Project: K11107; BnR: KJ0101030; TRN: US200706%%347
- Resource Relation:
- Conference: IEEE Symposium on Visual Analytics Science andTechnology, Baltimore, MD, USA, Oct 31 - Nov 2,2006
- Country of Publication:
- United States
- Language:
- English
Similar Records
Interactive Analysis of Large Network Data Collections UsingQuery-Driven Visualization
Query-driven visualization of large data sets