Combining Tensor Decompositions and Graph Analytics to Provide Cyber Situational Awareness at HPC Scale

Conference · Wed Sep 25 00:00:00 EDT 2019

DOI:https://doi.org/10.1109/HPEC.2019.8916559· OSTI ID:1580600

Ezick, James ^[1]; Henretty, Tom ^[1]; Baskaran, Muthu ^[1]; Lethin, Richard ^[1]; Feo, John T. ^[2]; Tuan, Tai-Ching ^[3]; Coley, Christopher ^[4]; Leonard, Leslie ^[5]; Agrawal, Rajeev ^[5]; Parsons, Ben ^[6]; Glodek, William ^[7]

---

1. Reservoir Lab, Inc
2. BATTELLE (PACIFIC NW LAB)
3. University of Maryland at College Park
4. Laboratory for Physical Sciences
5. Information Technology Laboratory
6. U.S. Army Engineer Research and Development Center
7. BreakPoint Labs

This paper describes MADHAT (Multidimensional Anomaly Detection fusing HPC, Analytics, and Tensors), an integrated workflow that demonstrates the applicability of HPC resources to the problem of maintaining cyber situational awareness. MADHAT combines two high-performance packages: ENSIGN for large-scale sparse tensor decompositions and HAGGLE for graph analytics. Tensor decompositions isolate coherent patterns of network behavior in ways that common clustering methods based on distance metrics cannot. Parallelized graph analysis then uses directed queries on a representation that combines the elements of identified patterns with other available information (such as additional log fields, domain knowledge, network topology, whitelists and blacklists, prior feedback, and published alerts) to confirm or reject a threat hypothesis, collect context, and raise alerts. MADHAT was developed using the collaborative HPC Architecture for Cyber Situational Awareness (HACSAW) research environment and evaluated on structured network sensor logs collected from Defense Research and Engineering Network (DREN) sites using HPC resources at the U.S. Army Engineer Research and Development Center DoD Supercomputing Resource Center (ERDC DSRC). To date, MADHAT has analyzed logs with over 650 million entries.

• Research Organization: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 1580600
• Report Number(s): PNNL-SA-143871
• Country of Publication: United States
• Language: English

Related Subjects

Cyber security
Tensor decomposition
graph analytics