XRootd, disk-based, caching proxy for optimization of data access, data placement and data replication

Conference · Wed Jan 01 00:00:00 EST 2014 · J.Phys.Conf.Ser.

DOI:https://doi.org/10.1088/1742-6596/513/4/042044· OSTI ID:1831852

Bauerdick, L. A.T.  ^[1]; Bloom, K.  ^[2]; Bockelman, B.  ^[2]; Bradley, D. C. ^[3]; Dasu, S.  ^[3]; Dost, J. M. ^[4]; Sfiligoi, I. ^[4]; Tadel, A. ^[4]; Tadel, M. ^[4]; Wuerthwein, F.  ^[4]; Yagil, A.  ^[4]

---

1. Fermilab
2. Nebraska U.
3. Wisconsin U., Madison
4. UC, San Diego

Following the success of the XRootd-based US CMS data federation, the AAA project investigated extensions of the federation architecture by developing two sample implementations of an XRootd, disk-based, caching proxy. The first one simply starts fetching a whole file as soon as a file open request is received and is suitable when completely random file access is expected or it is already known that a whole file be read. The second implementation supports on-demand downloading of partial files. Extensions to the Hadoop Distributed File System have been developed to allow for an immediate fallback to network access when local HDFS storage fails to provide the requested block. Both cache implementations are in pre-production testing at UCSD.

• Research Organization: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
• Sponsoring Organization: USDOE Office of Science (SC), High Energy Physics (HEP)
• Contributing Organization: CMS
• DOE Contract Number: AC02-07CH11359
• OSTI ID: 1831852
• Report Number(s): FERMILAB-CONF-14-639-CMS; oai:inspirehep.net:1302118
• Journal Information: J.Phys.Conf.Ser., Vol. 513
• Country of Publication: United States
• Language: English