skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: HEP Science Network Requirements--Final Report

Abstract

The Energy Sciences Network (ESnet) is the primary provider of network connectivity for the US Department of Energy Office of Science, the single largest supporter of basic research in the physical sciences in the United States. In support of the Office of Science programs, ESnet regularly updates and refreshes its understanding of the networking requirements of the instruments, facilities, scientists, and science programs that it serves. This focus has helped ESnet to be a highly successful enabler of scientific discovery for over 20 years. In August 2009 ESnet and the Office of High Energy Physics (HEP), of the DOE Office of Science, organized a workshop to characterize the networking requirements of the programs funded by HEP. The International HEP community has been a leader in data intensive science from the beginning. HEP data sets have historically been the largest of all scientific data sets, and the communty of interest the most distributed. The HEP community was also the first to embrace Grid technologies. The requirements identified at the workshop are summarized below, and described in more detail in the case studies and the Findings section: (1) There will be more LHC Tier-3 sites than orginally thought, and likely more Tier-2more » to Tier-2 traffic than was envisioned. It it not yet known what the impact of this will be on ESnet, but we will need to keep an eye on this traffic. (2) The LHC Tier-1 sites (BNL and FNAL) predict the need for 40-50 Gbps of data movement capacity in 2-5 years, and 100-200 Gbps in 5-10 years for HEP program related traffic. Other key HEP sites include LHC Tier-2 and Tier-3 sites, many of which are located at universities. To support the LHC, ESnet must continue its collaborations with university and international networks. (3) While in all cases the deployed 'raw' network bandwidth must exceed the user requirements in order to meet the data transfer and reliability requirements, network engineering for trans-Atlantic connectivity is more complex than network engineering for intra-US connectivity. This is because transoceanic circuits have lower reliability and longer repair times when compared with land-based circuits. Therefore, trans-Atlantic connectivity requires greater deployed bandwidth and diversity to ensure reliability and service continuity of the user-level required data transfer rates. (4) Trans-Atlantic traffic load and patterns must be monitored, and projections adjusted if necessary. There is currently a shutdown planned for the LHC in 2012 that may affect projections of trans-Atlantic bandwidth requirements. (5) There is a significant need for network tuning and troubleshooting during the establishment of new LHC Tier-2 and Tier-3 facilities. ESnet will work with the HEP community to help new sites effectively use the network. (6) SLAC is building the CCD camera for the LSST. This project will require significant bandwidth (up to 30Gbps) to NCSA over the next few years. (7) The accelerator modeling program at SLAC could require the movement of 1PB simulation data sets from the Leadership Computing Facilities at Argonne and Oak Ridge to SLAC. The data sets would need to be moved overnight, and moving 1PB in eight hours requires more than 300Gbps of throughput. This requirement is dependent on the deployment of analysis capabilities at SLAC, and is about five years away. (8) It is difficult to achieve high data transfer throughput to sites in China. Projects that need to transfer data in or out of China are encouraged to deploy test and measurement infrastructure (e.g. perfSONAR) and allow time for performance tuning.« less

Authors:
; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; more »; ; ; ; ; ; ; ; ; « less
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
Computational Research Division
OSTI Identifier:
983267
Report Number(s):
LBNL-3397E
TRN: US1004507
DOE Contract Number:  
DE-AC02-05CH11231
Resource Type:
Technical Report
Country of Publication:
United States
Language:
English
Subject:
99 GENERAL AND MISCELLANEOUS; 72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS; 79 ASTRONOMY AND ASTROPHYSICS; 97 MATHEMATICS AND COMPUTING; ACCELERATORS; BNL; CAMERAS; CAPACITY; HIGH ENERGY PHYSICS; PERFORMANCE; RELIABILITY; REPAIR; SHUTDOWN; SIMULATION; STANFORD LINEAR ACCELERATOR CENTER; TUNING; High Energy Physics High Speed Networking Network Requirements

Citation Formats

Bakken, Jon, Barczyk, Artur, Blatecky, Alan, Boehnlein, Amber, Carlson, Rich, Chekanov, Sergei, Cotter, Steve, Cottrell, Les, Crawford, Glen, Crawford, Matt, Dart, Eli, Dattoria, Vince, Ernst, Michael, Fisk, Ian, Gardner, Rob, Johnston, Bill, Kent, Steve, Lammel, Stephan, Loken, Stewart, Metzger, Joe, Mount, Richard, Ndousse-Fetter, Thomas, Newman, Harvey, Schopf, Jennifer, Sekine, Yukiko, Stone, Alan, Tierney, Brian, Tull, Craig, and Zurawski, Jason. HEP Science Network Requirements--Final Report. United States: N. p., 2010. Web. doi:10.2172/983267.
Bakken, Jon, Barczyk, Artur, Blatecky, Alan, Boehnlein, Amber, Carlson, Rich, Chekanov, Sergei, Cotter, Steve, Cottrell, Les, Crawford, Glen, Crawford, Matt, Dart, Eli, Dattoria, Vince, Ernst, Michael, Fisk, Ian, Gardner, Rob, Johnston, Bill, Kent, Steve, Lammel, Stephan, Loken, Stewart, Metzger, Joe, Mount, Richard, Ndousse-Fetter, Thomas, Newman, Harvey, Schopf, Jennifer, Sekine, Yukiko, Stone, Alan, Tierney, Brian, Tull, Craig, & Zurawski, Jason. HEP Science Network Requirements--Final Report. United States. doi:10.2172/983267.
Bakken, Jon, Barczyk, Artur, Blatecky, Alan, Boehnlein, Amber, Carlson, Rich, Chekanov, Sergei, Cotter, Steve, Cottrell, Les, Crawford, Glen, Crawford, Matt, Dart, Eli, Dattoria, Vince, Ernst, Michael, Fisk, Ian, Gardner, Rob, Johnston, Bill, Kent, Steve, Lammel, Stephan, Loken, Stewart, Metzger, Joe, Mount, Richard, Ndousse-Fetter, Thomas, Newman, Harvey, Schopf, Jennifer, Sekine, Yukiko, Stone, Alan, Tierney, Brian, Tull, Craig, and Zurawski, Jason. Tue . "HEP Science Network Requirements--Final Report". United States. doi:10.2172/983267. https://www.osti.gov/servlets/purl/983267.
@article{osti_983267,
title = {HEP Science Network Requirements--Final Report},
author = {Bakken, Jon and Barczyk, Artur and Blatecky, Alan and Boehnlein, Amber and Carlson, Rich and Chekanov, Sergei and Cotter, Steve and Cottrell, Les and Crawford, Glen and Crawford, Matt and Dart, Eli and Dattoria, Vince and Ernst, Michael and Fisk, Ian and Gardner, Rob and Johnston, Bill and Kent, Steve and Lammel, Stephan and Loken, Stewart and Metzger, Joe and Mount, Richard and Ndousse-Fetter, Thomas and Newman, Harvey and Schopf, Jennifer and Sekine, Yukiko and Stone, Alan and Tierney, Brian and Tull, Craig and Zurawski, Jason},
abstractNote = {The Energy Sciences Network (ESnet) is the primary provider of network connectivity for the US Department of Energy Office of Science, the single largest supporter of basic research in the physical sciences in the United States. In support of the Office of Science programs, ESnet regularly updates and refreshes its understanding of the networking requirements of the instruments, facilities, scientists, and science programs that it serves. This focus has helped ESnet to be a highly successful enabler of scientific discovery for over 20 years. In August 2009 ESnet and the Office of High Energy Physics (HEP), of the DOE Office of Science, organized a workshop to characterize the networking requirements of the programs funded by HEP. The International HEP community has been a leader in data intensive science from the beginning. HEP data sets have historically been the largest of all scientific data sets, and the communty of interest the most distributed. The HEP community was also the first to embrace Grid technologies. The requirements identified at the workshop are summarized below, and described in more detail in the case studies and the Findings section: (1) There will be more LHC Tier-3 sites than orginally thought, and likely more Tier-2 to Tier-2 traffic than was envisioned. It it not yet known what the impact of this will be on ESnet, but we will need to keep an eye on this traffic. (2) The LHC Tier-1 sites (BNL and FNAL) predict the need for 40-50 Gbps of data movement capacity in 2-5 years, and 100-200 Gbps in 5-10 years for HEP program related traffic. Other key HEP sites include LHC Tier-2 and Tier-3 sites, many of which are located at universities. To support the LHC, ESnet must continue its collaborations with university and international networks. (3) While in all cases the deployed 'raw' network bandwidth must exceed the user requirements in order to meet the data transfer and reliability requirements, network engineering for trans-Atlantic connectivity is more complex than network engineering for intra-US connectivity. This is because transoceanic circuits have lower reliability and longer repair times when compared with land-based circuits. Therefore, trans-Atlantic connectivity requires greater deployed bandwidth and diversity to ensure reliability and service continuity of the user-level required data transfer rates. (4) Trans-Atlantic traffic load and patterns must be monitored, and projections adjusted if necessary. There is currently a shutdown planned for the LHC in 2012 that may affect projections of trans-Atlantic bandwidth requirements. (5) There is a significant need for network tuning and troubleshooting during the establishment of new LHC Tier-2 and Tier-3 facilities. ESnet will work with the HEP community to help new sites effectively use the network. (6) SLAC is building the CCD camera for the LSST. This project will require significant bandwidth (up to 30Gbps) to NCSA over the next few years. (7) The accelerator modeling program at SLAC could require the movement of 1PB simulation data sets from the Leadership Computing Facilities at Argonne and Oak Ridge to SLAC. The data sets would need to be moved overnight, and moving 1PB in eight hours requires more than 300Gbps of throughput. This requirement is dependent on the deployment of analysis capabilities at SLAC, and is about five years away. (8) It is difficult to achieve high data transfer throughput to sites in China. Projects that need to transfer data in or out of China are encouraged to deploy test and measurement infrastructure (e.g. perfSONAR) and allow time for performance tuning.},
doi = {10.2172/983267},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Tue Apr 27 00:00:00 EDT 2010},
month = {Tue Apr 27 00:00:00 EDT 2010}
}

Technical Report:

Save / Share: