Systems and methods for fast detection of elephant flows in network traffic
Abstract
In a system for efficiently detecting large/elephant flows in a network, the rate at which the received packets are sampled is adjusted according to a top flow detection likelihood computed for a cache of flows identified in the arriving network traffic. After observing packets sampled from the network, Dirichlet-Categorical inference is employed to calculate a posterior distribution that captures uncertainty about the sizes of each flow, yielding a top flow detection likelihood. The posterior distribution is used to find the most likely subset of elephant flows. The technique rapidly converges to the optimal sampling rate at a speed O(1/n), where n is the number of packet samples received, and the only hyperparameter required is the targeted detection likelihood.
- Inventors:
- Issue Date:
- Research Org.:
- Reservoir Labs, Inc., New York, NY (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1805526
- Patent Number(s):
- 10924418
- Application Number:
- 16/270,089
- Assignee:
- Reservoir Labs, Inc. (New York, NY)
- Patent Classifications (CPCs):
-
H - ELECTRICITY H04 - ELECTRIC COMMUNICATION TECHNIQUE H04L - TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- DOE Contract Number:
- SC0011358
- Resource Type:
- Patent
- Resource Relation:
- Patent File Date: 02/07/2019
- Country of Publication:
- United States
- Language:
- English
Citation Formats
Gudibanda, Aditya, and Ros-Giralt, Jordi. Systems and methods for fast detection of elephant flows in network traffic. United States: N. p., 2021.
Web.
Gudibanda, Aditya, & Ros-Giralt, Jordi. Systems and methods for fast detection of elephant flows in network traffic. United States.
Gudibanda, Aditya, and Ros-Giralt, Jordi. Tue .
"Systems and methods for fast detection of elephant flows in network traffic". United States. https://www.osti.gov/servlets/purl/1805526.
@article{osti_1805526,
title = {Systems and methods for fast detection of elephant flows in network traffic},
author = {Gudibanda, Aditya and Ros-Giralt, Jordi},
abstractNote = {In a system for efficiently detecting large/elephant flows in a network, the rate at which the received packets are sampled is adjusted according to a top flow detection likelihood computed for a cache of flows identified in the arriving network traffic. After observing packets sampled from the network, Dirichlet-Categorical inference is employed to calculate a posterior distribution that captures uncertainty about the sizes of each flow, yielding a top flow detection likelihood. The posterior distribution is used to find the most likely subset of elephant flows. The technique rapidly converges to the optimal sampling rate at a speed O(1/n), where n is the number of packet samples received, and the only hyperparameter required is the targeted detection likelihood.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2021},
month = {2}
}
Works referenced in this record:
Loading a flow table with neural network determined information
patent, March 2018
- Viljoen, Nicolaas J.
- US Patent Document 9,929,933
Sampling based on large flow detection for network visibility monitoring
patent, October 2018
- Conlon, Declan; Whiteside, Raymond Scott; Volpe, Thomas A.
- US Patent Document 10,097,464
Systems and Methods for Pacing Data Flows
patent-application, September 2016
- Burnette, John; Hadorn, Ben; Harrang, Jeffrey
- US Patent Application 15/060486; 20160261510
Large flow detection for network visibility monitoring
patent, May 2018
- Volpe, Thomas A.; Whiteside, Raymond Scott; Conlon, Declan
- US Patent Document 9,979,624
Network visibility monitoring
patent, June 2018
- Whiteside, Raymond Scott; Volpe, Thomas A.; Conlon, Declan
- US Patent Document 10,003,515
Using a neural network to determine how to direct a flow
patent, November 2018
- Viljoen, Nicolaas J.
- US Patent Document 10,129,135
Historically large flows in network visibility monitoring
patent, July 2018
- Whiteside, Raymond Scott; Volpe, Thomas A.
- US Patent Document 10,033,613
Traffic-aware sampling rate adjustment within a network device
patent, January 2016
- Tagore, Kalyana Prakash Ravindranath
- US Patent Document 9,246,828
Determining Sampling Rate from Randomly Sampled Events
patent-application, March 2015
- Agarwal, Kanak B.; Carter, John B.; Dixon, Colin K.
- US Patent Application 14/034862; 20150089045
System and Method for Sampling Network Traffic
patent-application, June 2010
- Duffield, Nicholas; Breslaw, Lee M.; Ee, Cheng
- US Patent Application 12/342957; 20100161791
Systems and Methods for Identifying Candidate Flows in Data Packet Networks
patent-application, August 2019
- Miller, Michelle; Burnette, John M.; Hadorn, Ben
- US Patent Application 16/258467; 20190238461