Scale tests of the new DUNE data pipeline

Conference · 31 December 2023 · EPJ Web Conf.

DOI:https://doi.org/10.1051/epjconf/202429501046· OSTI ID:2396726

Timm, Steven ^[1]

---

1. Fermilab

In preparation for the second runs of the ProtoDUNE detectors at CERN (NP02 and NP04)[1], DUNE has established a new data pipeline for bringing the data from the EHN-1 experimental hall at CERN to primary tape storage at Fermilab and CERN, and then spreading it out to a distributed disk data store at many locations around the world. This system includes a new Ingest Daemon and a new Declaration Daemon. The Rucio[2] replica catalog, and FTS3 transport are used to transport all files. All file metadata is declared to the new MetaCat[3] metadata service. All of these new components have been successfully tested at a scale equal to the expected output of the detector data acquisition system (~2-4 GB/s), and the expected network bandwidth out of the experimental hall. We present the procedure that was used to test and the results of the test.

• Research Organization: Fermi National Accelerator Laboratory (FNAL), Batavia, IL (United States)
• Sponsoring Organization: USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25)
• Contributing Organization: DUNE
• DOE Contract Number: AC02-07CH11359
• OSTI ID: 2396726
• Report Number(s): FERMILAB-CONF-23-533-CSAID; oai:inspirehep.net:2785155
• Journal Information: EPJ Web Conf., Journal Name: EPJ Web Conf. Vol. 295
• Country of Publication: United States
• Language: English