skip to main content

DOE PAGESDOE PAGES

  • DOE PAGES
  • Accepted Manuscript: Bridging the gap between peak and average loads on science networks

Title: Bridging the gap between peak and average loads on science networks

Backbone networks are typically overprovisioned in order to support peak loads. Research and education networks (RENs), for example, are often designed to operate at 20–30% of capacity. Thus, Internet2 upgrades its backbone interconnects when the weekly 95th-percentile load is reliably above 30% of link capacity, and analysis of ESnet traffic between major laboratories shows a substantial gap between peak and average utilization. As science data volumes increase exponentially, it is unclear whether this overprovisioning trend can continue into the future. Even if overprovisioning is possible, it may not be the most cost-effective (and desirable) approach going forward. Under the current mode of free access to RENs, traffic at peak load may include both flows that need to be transferred in near-real time–for example, for computation and instrument monitoring and steering–and flows that are less time-critical, for example, archival and storage replication operations. Thus, peak load does not necessarily indicate the capacity that is absolutely required at that moment. We thus examine how data transfers are impacted when the average network load is increased while the network capacity is kept at the current levels. We also classify data transfers into on-demand (time-critical) and best-effort (less time-critical) and study the impact onmore » both classes for different proportions of both the number of on-demand transfers and amount of bandwidth allocated for on-demand transfers. For our study, we use real transfer logs from production GridFTP servers to do simulation-based experiments as well as real experiments on a testbed. We find that when the transfer load is doubled and the network capacity is fixed at the current level, the gap between peak and average throughput decreases by an average of 18% in the simulation experiments and 16% in the testbed experiments, and the average slowdown experienced by the data transfers is under 1.5×. Moreover, when transfers are classified as on-demand or best-effort, on-demand transfers experience almost no slowdown and the mean slowdown experienced by best-effort transfers is under 2× in the simulation experiments and under 1.2× in the testbed experiments.« less
Authors:
 [1] ; ORCiD logo [2] ;  [3] ;  [4]
+ Show Author Affiliations
  1. Univ. of Chicago, Chicago, IL (United States)
  2. Hongik Univ. (South Korea)
  3. Argonne National Lab. (ANL), Lemont, IL (United States)
  4. Univ. of Chicago, Chicago, IL (United States); Argonne National Lab. (ANL), Lemont, IL (United States)
Publication Date:
Grant/Contract Number:
AC02-06CH11357
Type:
Accepted Manuscript
Journal Name:
Future Generations Computer Systems
Additional Journal Information:
Journal Volume: 79; Journal Issue: P1; Journal ID: ISSN 0167-739X
Publisher:
Elsevier
Research Org:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org:
USDOE Office of Science (SC)
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Network utilization; Data transfer scheduling; Network planning
OSTI Identifier:
1421948
  • Citation Metrics
  • Cited by: 1work
  • Citation information provided by
    Web of Science