The Open High Throughput Computing Content Delivery Network

Journal Article · Tue Sep 17 00:00:00 EDT 2019 · EPJ Web of Conferences

DOI:https://doi.org/10.1051/epjconf/201921404023· OSTI ID:1490833

Dykstra, Dave  ^[1]; Bockelman, Brian ^[2]; Blomer, Jakob ^[3]; Field, Laurence ^[3]

---

1. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
2. Univ. of Nebraska, Lincoln, NE (United States)
3. European Organization for Nuclear Research (CERN), Geneva (Switzerland)

LHC experiments make extensive use of web proxy caches, especially for software distribution via the CernVM File System and for conditions data via the Frontier Distributed Database Caching system. Since many jobs read the same data, cache hit rates are high and hence most of the traffic flows efficiently over Local Area Networks. However, it is not always possible to have local web caches, particularly for opportunistic cases whereexperiments have little control over site services. The Open High Throughput Computing (HTC) Content Delivery Network (CDN), openhtc.io, aims to address this by using web proxy caches from a commercial CDN provider. Cloudflare provides a simple interface for registering DNS aliases of any web server and does reverse proxy web caching on those aliases. The openhtc.io domain is hosted on Cloudflare's free tier CDN which has no bandwidth limit and makes use of data centers throughout the world, so the average performance for clients is much improved compared to reading from CERN or a Tier 1. The load on WLCG servers is also significantly reduced. WLCG Web Proxy Auto Discovery is used to select local web caches when they are available and otherwise select openhtc.io caching. This paper describes the Open HTC CDN in detail and provides initial results from its use for LHC@Home and USCMS opportunistic computing.

• Research Organization: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
• Sponsoring Organization: USDOE Office of Science (SC), High Energy Physics (HEP)
• Grant/Contract Number: AC02-07CH11359
• OSTI ID: 1490833
• Report Number(s): FERMILAB-CONF-18-582-CD; CMS-CR-2018-287; oai:inspirehep.net:1712632
• Journal Information: EPJ Web of Conferences, Vol. 214; Conference: 23rd International Conference on Computing in High Energy and Nuclear Physics, Sofia, Bulgaria, 07/09-07/13/2018; ISSN 2100-014X
• Publisher: EDP Sciences
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING