Interactive Analysis of Large Network Data Collections UsingQuery-Driven Visualization
Realizing operational analytics solutions where large and complex data must be analyzed in a time-critical fashion entails integrating many different types of technology. Considering the extreme scale of contemporary datasets, one significant challenge is to reduce the duty cycle in the analytics discourse process. This paper focuses on an interdisciplinary combination of scientific data management and visualization/analysis technologies targeted at reducing the duty cyclein hypothesis testing and knowledge discovery. We present an application of such a combination in the problem domain of network traffic data analysis. Our performance experiment results, including both serial and parallel scalability tests, show that the combination can dramatically decrease the analytics duty cycle for this particular application. The combination is effectively applied to the analysis of network traffic data to detect slow and distributed scans, which is a difficult-to-detect form of cyber attack. Our approach is sufficiently general to be applied to a diverse set of data understanding problems as well as used in conjunction with a diverse set of analysis and visualization tools.
- Research Organization:
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
- Sponsoring Organization:
- USDOE Director. Office of Science. Advanced ScientificComputing Research; Department of Homeland Security NationalVisualization and Analytics Center, National Nuclear SecurityAdministration
- DOE Contract Number:
- DE-AC02-05CH11231
- OSTI ID:
- 891627
- Report Number(s):
- LBNL-59166; R&D Project: K11107; BnR: KJ0101030; TRN: US200622%%269
- Country of Publication:
- United States
- Language:
- English
Similar Records
Data Intensive Architecture for Scalable Cyber Analytics
Flexible visualization of a 3rd party Intrusion Prevention (Security) tool: A use case with the ELK stack