Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: A Network Management System for Handling Scientific Data Flows

Abstract

Large scientific data transfers often occur at high rates causing increased burstiness in Internet traffic. To limit the adverse effects of these high-rate large-sized flows, which are referred to as $$\alpha $$ flows, on delay-sensitive audio/video flows, a network management system called Alpha Flow Traffic Engineering System (AFTES) is proposed for intra-domain traffic engineering. An offline approach is used in which AFTES analyzes NetFlow records collected by routers, extracts source–destination address prefixes of $$\alpha $$ flows, and uses these prefixes to configure firewall filters at ingress routers of a provider’s network to redirect future $$\alpha $$ flows to traffic-engineered paths and isolated queues. The effectiveness of this scheme was evaluated through an analysis of 7 months of NetFlow data obtained from an ESnet router. For this data set, 91 % of bytes generated by $$\alpha $$ flows during high-rate intervals would have been directed had AFTES been deployed. The negative aspect of using address prefixes in firewall filters, i.e., the redirection of $$\beta $$ flows to $$\alpha $$-flow paths/queues, was also quantified.

Authors:
 [1];  [2];  [1];  [1];  [1]
+ Show Author Affiliations
  1. Univ. of Virginia, Charlottesville, VA (United States)
  2. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Publication Date:
Research Org.:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1532178
Grant/Contract Number:  
AC02-05CH11231
Resource Type:
Accepted Manuscript
Journal Name:
Journal of Network and Systems Management
Additional Journal Information:
Journal Volume: 24; Journal Issue: 1; Journal ID: ISSN 1064-7570
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; NetFlow traffic analysis; Elephant flows; Scientific computing; Research and education networks (RENs); MPLS; Virtual circuits

Citation Formats

Yan, Zhenzhen, Tracy, Chris, Veeraraghavan, Malathi, Jin, Tian, and Liu, Zhengyang. A Network Management System for Handling Scientific Data Flows. United States: N. p., 2014. Web. doi:10.1007/s10922-014-9336-2.
Copy to clipboard
Yan, Zhenzhen, Tracy, Chris, Veeraraghavan, Malathi, Jin, Tian, & Liu, Zhengyang. A Network Management System for Handling Scientific Data Flows. United States. https://doi.org/10.1007/s10922-014-9336-2
Copy to clipboard
Yan, Zhenzhen, Tracy, Chris, Veeraraghavan, Malathi, Jin, Tian, and Liu, Zhengyang. Sat . "A Network Management System for Handling Scientific Data Flows". United States. https://doi.org/10.1007/s10922-014-9336-2. https://www.osti.gov/servlets/purl/1532178.
Copy to clipboard
@article{osti_1532178,
title = {A Network Management System for Handling Scientific Data Flows},
author = {Yan, Zhenzhen and Tracy, Chris and Veeraraghavan, Malathi and Jin, Tian and Liu, Zhengyang},
abstractNote = {Large scientific data transfers often occur at high rates causing increased burstiness in Internet traffic. To limit the adverse effects of these high-rate large-sized flows, which are referred to as $\alpha $ flows, on delay-sensitive audio/video flows, a network management system called Alpha Flow Traffic Engineering System (AFTES) is proposed for intra-domain traffic engineering. An offline approach is used in which AFTES analyzes NetFlow records collected by routers, extracts source–destination address prefixes of $\alpha $ flows, and uses these prefixes to configure firewall filters at ingress routers of a provider’s network to redirect future $\alpha $ flows to traffic-engineered paths and isolated queues. The effectiveness of this scheme was evaluated through an analysis of 7 months of NetFlow data obtained from an ESnet router. For this data set, 91 % of bytes generated by $\alpha $ flows during high-rate intervals would have been directed had AFTES been deployed. The negative aspect of using address prefixes in firewall filters, i.e., the redirection of $\beta $ flows to $\alpha $-flow paths/queues, was also quantified.},
doi = {10.1007/s10922-014-9336-2},
journal = {Journal of Network and Systems Management},
number = 1,
volume = 24,
place = {United States},
year = {Sat Oct 11 00:00:00 EDT 2014},
month = {Sat Oct 11 00:00:00 EDT 2014}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1007/s10922-014-9336-2
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Citation Metrics:
Cited by: 4 works
Citation information provided by
Web of Science
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Estimating the Heavy Tail Index from Scaling Properties
journal, January 1999

  • Crovella, Mark E.; Taqqu, Murad S.
  • Methodology And Computing In Applied Probability, Vol. 1, Issue 1, p. 55-79
  • DOI: 10.1023/A:1010012224103

Estimating flow distributions from sampled flow statistics
journal, October 2005

  • Duffield, N.; Lund, C.; Thorup, M.
  • IEEE/ACM Transactions on Networking, Vol. 13, Issue 5
  • DOI: 10.1109/TNET.2005.852874

A Survey on the Path Computation Element (PCE) Architecture
journal, January 2013

  • Paolucci, Francesco; Cugini, Filippo; Giorgetti, Alessio
  • IEEE Communications Surveys & Tutorials, Vol. 15, Issue 4
  • DOI: 10.1109/SURV.2013.011413.00087

An overview of routing optimization for internet traffic engineering
journal, January 2008

Internet traffic engineering using multi-protocol label switching (MPLS)
journal, September 2002

A measurement study of correlations of Internet flow characteristics
journal, January 2006

A pragmatic definition of elephants in internet backbone traffic
conference, January 2002

  • Papagiannaki, K.; Taft, N.; Bhattacharyya, S.
  • Proceedings of the second ACM SIGCOMM Workshop on Internet measurment - IMW '02
  • DOI: 10.1145/637201.637227

Real-time Detection of Grid Bulk Transfer Traffic
conference, January 2006

  • Paisley, J.; Sventek, J.
  • 2006 IEEE/IFIP Network Operations and Management Symposium NOMS 2006
  • DOI: 10.1109/NOMS.2006.1687539

c-Through: part-time optics in data centers
journal, August 2010

  • Wang, Guohui; Andersen, David G.; Kaminsky, Michael
  • ACM SIGCOMM Computer Communication Review, Vol. 40, Issue 4
  • DOI: 10.1145/1851275.1851222

Helios: a hybrid electrical/optical switch architecture for modular data centers
journal, August 2010

  • Farrington, Nathan; Porter, George; Radhakrishnan, Sivasankar
  • ACM SIGCOMM Computer Communication Review, Vol. 40, Issue 4
  • DOI: 10.1145/1851275.1851223

Identifying high-rate flows based on Bayesian single sampling
conference, April 2010

  • Zhang, Yu; Fang, Binxing; Zhang, Yongzheng
  • 2010 2nd International Conference on Computer Engineering and Technology
  • DOI: 10.1109/ICCET.2010.5486097

A methodology for studying persistency aspects of internet flows
journal, April 2005

  • Wallerich, Jörg; Dreger, Holger; Feldmann, Anja
  • ACM SIGCOMM Computer Communication Review, Vol. 35, Issue 2
  • DOI: 10.1145/1064413.1064417

Inverting sampled traffic
journal, February 2006

Characterization of high-rate large-sized flows
conference, May 2014

  • Jin, Tian; Tracy, Chris; Veeraraghavan, Malathi
  • 2014 IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom)
  • DOI: 10.1109/BlackSeaCom.2014.6849008

Programming your network at run-time for big data applications
conference, January 2012

  • Wang, Guohui; Ng, T. S. Eugene; Shaikh, Anees
  • Proceedings of the first workshop on Hot topics in software defined networks - HotSDN '12
  • DOI: 10.1145/2342441.2342462

Experiences with 100Gbps network applications
conference, January 2012

  • Balman, Mehmet; Pouyoul, Eric; Yao, Yushu
  • Proceedings of the fifth international workshop on Data-Intensive Distributed Computing Date - DIDC '12
  • DOI: 10.1145/2286996.2287004

Identifying elephant flows through periodically sampled packets
conference, January 2004

  • Mori, Tatsuya; Uchida, Masato; Kawahara, Ryoichi
  • Proceedings of the 4th ACM SIGCOMM conference on Internet measurement - IMC '04
  • DOI: 10.1145/1028788.1028803

A Survey on Internet Traffic Identification
journal, January 2009

  • Callado, Arthur; Kamienski, Carlos; Szabo, Geza
  • IEEE Communications Surveys & Tutorials, Vol. 11, Issue 3
  • DOI: 10.1109/SURV.2009.090304

ElephantTrap: A low cost device for identifying large flows
conference, August 2007

  • Lu, Yi; Prabhakar, Balaji; Bonomi, Flavio
  • 15th Annual IEEE Symposium on High-Performance Interconnects (HOTI 2007)
  • DOI: 10.1109/HOTI.2007.13

Traffic engineering with MPLS in the Internet
journal, January 2000

  • Xipeng Xiao, ; Hannan, A.; Bailey, B.
  • IEEE Network, Vol. 14, Issue 2
  • DOI: 10.1109/65.826369

Beyond MLU: An application-centric comparison of traffic engineering schemes
conference, April 2011

  • Sharma, Abhigyan; Mishra, Aditya; Kumar, Vikas
  • IEEE INFOCOM 2011 - IEEE Conference on Computer Communications, 2011 Proceedings IEEE INFOCOM
  • DOI: 10.1109/INFCOM.2011.5935255

An Architecture for Differentiated Services
report, December 1998

Simple and Accurate Identification of High-Rate Flows by Packet Sampling
conference, April 2006

  • Kamiyama, N.; Mori, T.
  • Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications
  • DOI: 10.1109/INFOCOM.2006.324

Wide-area Internet traffic patterns and characteristics
journal, January 1997

  • Thompson, K.; Miller, G. J.; Wilder, R.
  • IEEE Network, Vol. 11, Issue 6
  • DOI: 10.1109/65.642356

A survey of techniques for internet traffic classification using machine learning
journal, January 2008

  • Nguyen, Thuy T. T.; Armitage, Grenville
  • IEEE Communications Surveys & Tutorials, Vol. 10, Issue 4
  • DOI: 10.1109/SURV.2008.080406

Application-awareness in SDN
conference, January 2013

  • Qazi, Zafar Ayyub; Lee, Jeongkeun; Jin, Tao
  • Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM - SIGCOMM '13
  • DOI: 10.1145/2486001.2491700

Tracking elephant flows in internet backbone traffic with an FPGA-based cache
conference, August 2009

  • Zadnik, Martin; Canini, Marco; Moore, Andrew W.
  • 2009 International Conference on Field Programmable Logic and Applications (FPL)
  • DOI: 10.1109/FPL.2009.5272387

Self-management of hybrid optical and packet switching networks
conference, May 2011

  • Fioreze, Tiago; Pras, Aiko
  • 2011 IFIP/IEEE International Symposium on Integrated Network Management (IM 2011), 12th IFIP/IEEE International Symposium on Integrated Network Management (IM 2011) and Workshops
  • DOI: 10.1109/INM.2011.5990527

On using virtual circuits for GridFTP transfers
conference, November 2012

  • Liu, Z.; Veeraraghavan, M.; Yan, Z.
  • 2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis
  • DOI: 10.1109/SC.2012.84

Understanding Internet traffic streams: dragonflies and tortoises
journal, October 2002

Connection-level analysis and modeling of network traffic
conference, January 2001

  • Sarvotham, Shriram; Riedi, Rudolf; Baraniuk, Richard
  • Proceedings of the First ACM SIGCOMM Workshop on Internet Measurement Workshop - IMW '01
  • DOI: 10.1145/505213.505215

Providing and verifying advanced IP services in hierarchical DiffServ networks-the case of GEANT
journal, April 2004

  • Liakopoulos, Athanassios; Maglaris, Basil; Bouras, Christos
  • International Journal of Communication Systems, Vol. 17, Issue 4
  • DOI: 10.1002/dac.645
    Sort by:
    [ × clear filter / sort ]

    Similar Records in DOE PAGES and OSTI.GOV collections: