skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: General-purpose Unsupervised Cyber Anomaly Detection via Non-negative Tensor Factorization

Journal Article · · Digital Threats: Research and Practice
DOI:https://doi.org/10.1145/3519602· OSTI ID:2282529
ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [2]; ORCiD logo [1]
  1. Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
  2. Amazon (United States)
+ Show Author Affiliations

Distinguishing malicious anomalous activities from unusual but benign activities is a fundamental challenge for cyber defenders. Prior studies have shown that statistical user behavior analysis yields accurate detections by learning behavior profiles from observed user activity. These unsupervised models are able to generalize to unseen types of attacks by detecting deviations from normal behavior, without knowledge of specific attack signatures. However, approaches proposed to date based on probabilistic matrix factorization are limited by the information conveyed in a two-dimensional space. Non-negative tensor factorization, on the other hand, is a powerful unsupervised machine learning method that naturally models multi-dimensional data, capturing complex and multi-faceted details of behavior profiles. Herein, our new unsupervised statistical anomaly detection methodology matches or surpasses state-of-the-art supervised learning baselines across several challenging and diverse cyber application areas, including detection of compromised user credentials, botnets, spam e-mails, and fraudulent credit card transactions.

Research Organization:
Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Sponsoring Organization:
USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE National Nuclear Security Administration (NNSA)
Grant/Contract Number:
89233218CNA000001; 20190020DR; 20210043DR
OSTI ID:
2282529
Alternate ID(s):
OSTI ID: 1889984
Report Number(s):
LA-UR-21-29195; LA-UR-22-21176
Journal Information:
Digital Threats: Research and Practice, Vol. 4, Issue 1; ISSN 2692-1626
Publisher:
Association for Computing Machinery (ACM)Copyright Statement
Country of Publication:
United States
Language:
English

References (37)

Feature bagging for outlier detection conference January 2005
UGR‘16: A new dataset for the evaluation of cyclostationarity-based network IDSs journal March 2018
Real-time botnet detection using nonnegative tucker decomposition conference April 2019
The harmonic mean p -value for combining dependent tests journal January 2019
GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection conference June 2019
Some mathematical notes on three-mode factor analysis journal September 1966
Anomaly Detection with Score Functions Based on the Reconstruction Error of the Kernel PCA book January 2014
Newton-based optimization for Kullback–Leibler nonnegative tensor factorizations journal February 2015
Multi-Dimensional Anomalous Entity Detection via Poisson Tensor Factorization conference November 2020
Improving Detection Accuracy for Imbalanced Network Intrusion Classification using Cluster-based Under-sampling with Random Forests conference May 2019
Unified Host and Network Data Set book September 2018
Anomaly-based Intrusion Detection and Prevention Using Adaptive Boosting in Software-defined Network conference October 2019
The Expression of a Tensor or a Polyadic as a Sum of Products journal April 1927
A Probabilistic-driven Ensemble Approach to Perform Event Classification in Intrusion Detection System
  • Saia, Roberto; Carta, Salvatore; Recupero, Diego Reforgiato
  • Proceedings of the 10th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management https://doi.org/10.5220/0006893801410148
conference January 2018
Semialgebraic Geometry of Nonnegative Tensor Rank journal January 2016
L-SVM: A radius-margin-based SVM algorithm with LogDet regularization journal July 2018
Anomaly Detection Using Nonnegative Matrix Factorization book January 2008
Classification of Red Team Authentication Events in an Enterprise Network book September 2018
Introducing DeepBalance: Random deep belief network ensembles to address class imbalance conference December 2017
Semantic Nonnegative Matrix Factorization with Automatic Model Determination for Topic Modeling conference December 2020
Tensor Ensemble Learning for Multidimensional data conference November 2018
Bayesian Poisson Tensor Factorization for Inferring Multilateral Relations from Sparse Dyadic Event Counts
  • Schein, Aaron; Paisley, John; Blei, David M.
  • Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD '15 https://doi.org/10.1145/2783258.2783414
conference January 2015
Time of Day Anomaly Detection conference October 2018
Towards a Reliable Comparison and Evaluation of Network Intrusion Detection Systems Based on Machine Learning Approaches journal March 2020
Determination of latent dimensionality in international trade flow journal October 2020
Nonnegative approximations of nonnegative tensors journal July 2009
Adversarially Learned Anomaly Detection conference November 2018
Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination journal September 2015
Cyber Security through Multidimensional Data Decompositions conference April 2016
Poisson factorization for peer-based anomaly detection conference September 2016
Isolation-Based Anomaly Detection journal March 2012
Deep Learning for Anomaly Detection: A Review journal March 2022
Semi-Supervised Multivariate Statistical Network Monitoring for Learning Security Threats journal August 2019
A Local Feature Engineering Strategy to Improve Network Anomaly Detection journal October 2020
Learning the parts of objects by non-negative matrix factorization journal October 1999
On Tensors, Sparsity, and Nonnegative Factorizations journal January 2012
Reciprocal rank fusion outperforms condorcet and individual rank learning methods
  • Cormack, Gordon V.; Clarke, Charles L. A.; Buettcher, Stefan
  • Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval https://doi.org/10.1145/1571941.1572114
conference July 2009

Similar Records

Related Subjects

97 MATHEMATICS AND COMPUTING
computing methodologies → anomaly detection
factorization methods
security and privacy → intrusion detection systems
anomaly detection
poisson tensor factorization
non-negative tensor factorization
unsupervised learning
cyber security
CPD
malware
data fusion
ensemble learning
GPU