Pooling the resources of the CMS Tier-1 sites

Apyan, A.; Badillo, J.; Cruz, J. Diaz; Gadrat, S.; Gutsche, O.; Holzman, B.; Lahiff, A.; Magini, N.; Mason, D.; Perez, A.; Stober, F.; Taneja, S.; Taze, M.; Wissing, C.

doi:10.1088/1742-6596/664/4/042056

Title: Pooling the resources of the CMS Tier-1 sites

Full Record
Other Related Research

Abstract

The CMS experiment at the LHC relies on 7 Tier-1 centres of the WLCG to perform the majority of its bulk processing activity, and to archive its data. During the first run of the LHC, these two functions were tightly coupled as each Tier-1 was constrained to process only the data archived on its hierarchical storage. This lack of flexibility in the assignment of processing workflows occasionally resulted in uneven resource utilisation and in an increased latency in the delivery of the results to the physics community.The long shutdown of the LHC in 2013-2014 was an opportunity to revisit this mode of operations, disentangling the processing and archive functionalities of the Tier-1 centres. The storage services at the Tier-1s were redeployed breaking the traditional hierarchical model: each site now provides a large disk storage to host input and output data for processing, and an independent tape storage used exclusively for archiving. Movement of data between the tape and disk endpoints is not automated, but triggered externally through the WLCG transfer management systems.With this new setup, CMS operations actively controls at any time which data is available on disk for processing and which data should be sent to archive. Thanks tomore »« less

Authors:

Apyan, A. ^[1]; Badillo, J. ^[2]; Cruz, J. Diaz ^[3]; Gadrat, S. ^[4]; Gutsche, O. ^[3]; Holzman, B. ^[3]; Lahiff, A. ^[5]; Magini, N. ^[3]; Mason, D. ^[3]; Perez, A. ^[6]; Stober, F. ^[7]; Taneja, S. ^[8]; Taze, M. ^[9]; Wissing, C. ^[10]

Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
Univ. de Los Andes, Bogota (Colombia)
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Centre de Calcul de l'Institut National de Physique Nucleaire et de Physique des Particules (IN2P3), Villeurbanne Cedex (France)
Rutherford Appleton Lab., Didcot (United Kingdom)
CIEMAT, Madrid (Spain)
Karlsruhe Institut fuer Technologie, Karlsruhe (Germany)
Univ. di Bologna, Bologna (Italy)
Cukurova Univ., Adana (Turkey)
DESY, Hamburg (Germany)

Publication Date:: Wed Dec 23 00:00:00 EST 2015

Research Org.:: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)

Sponsoring Org.:: USDOE Office of Science (SC), High Energy Physics (HEP)

OSTI Identifier:: 1250777

Report Number(s):: FERMILAB-CONF-15-447-CD
Journal ID: ISSN 1742-6588; 1413883

Grant/Contract Number:: AC02-07CH11359

Resource Type:: Accepted Manuscript

Journal Name:: Journal of Physics. Conference Series

Additional Journal Information:: Journal Volume: 664; Journal Issue: 4; Conference: 21st International Conference on Computing in High Energy and Nuclear Physics, Okinawa (Japan), 13-17 Apr 2015; Journal ID: ISSN 1742-6588

Publisher:: IOP Publishing

Country of Publication:: United States

Language:: English

Subject:: 96 KNOWLEDGE MANAGEMENT AND PRESERVATION

Citation Formats


                    Apyan, A., Badillo, J., Cruz, J. Diaz, Gadrat, S., Gutsche, O., Holzman, B., Lahiff, A., Magini, N., Mason, D., Perez, A., Stober, F., Taneja, S., Taze, M., and Wissing, C. Pooling the resources of the CMS Tier-1 sites.  United States: N. p., 2015. 
Web.  doi:10.1088/1742-6596/664/4/042056.

Copy to clipboard


                    Apyan, A., Badillo, J., Cruz, J. Diaz, Gadrat, S., Gutsche, O., Holzman, B., Lahiff, A., Magini, N., Mason, D., Perez, A., Stober, F., Taneja, S., Taze, M., & Wissing, C. Pooling the resources of the CMS Tier-1 sites.  United States.  https://doi.org/10.1088/1742-6596/664/4/042056

Copy to clipboard


                    Apyan, A., Badillo, J., Cruz, J. Diaz, Gadrat, S., Gutsche, O., Holzman, B., Lahiff, A., Magini, N., Mason, D., Perez, A., Stober, F., Taneja, S., Taze, M., and Wissing, C. Wed .  
"Pooling the resources of the CMS Tier-1 sites".  United States.  https://doi.org/10.1088/1742-6596/664/4/042056.  https://www.osti.gov/servlets/purl/1250777.

Copy to clipboard


                    
@article{osti_1250777,

  title        = {Pooling the resources of the CMS Tier-1 sites},

  author       = {Apyan, A. and Badillo, J. and Cruz, J. Diaz and Gadrat, S. and Gutsche, O. and Holzman, B. and Lahiff, A. and Magini, N. and Mason, D. and Perez, A. and Stober, F. and Taneja, S. and Taze, M. and Wissing, C.},

  abstractNote = {The CMS experiment at the LHC relies on 7 Tier-1 centres of the WLCG to perform the majority of its bulk processing activity, and to archive its data. During the first run of the LHC, these two functions were tightly coupled as each Tier-1 was constrained to process only the data archived on its hierarchical storage. This lack of flexibility in the assignment of processing workflows occasionally resulted in uneven resource utilisation and in an increased latency in the delivery of the results to the physics community.The long shutdown of the LHC in 2013-2014 was an opportunity to revisit this mode of operations, disentangling the processing and archive functionalities of the Tier-1 centres. The storage services at the Tier-1s were redeployed breaking the traditional hierarchical model: each site now provides a large disk storage to host input and output data for processing, and an independent tape storage used exclusively for archiving. Movement of data between the tape and disk endpoints is not automated, but triggered externally through the WLCG transfer management systems.With this new setup, CMS operations actively controls at any time which data is available on disk for processing and which data should be sent to archive. Thanks to the high-bandwidth connectivity guaranteed by the LHCOPN, input data can be freely transferred between disk endpoints as needed to take advantage of free CPU, turning the Tier-1s into a large pool of shared resources. The output data can be validated before archiving them permanently, and temporary data formats can be produced without wasting valuable tape resources. Lastly, the data hosted on disk at Tier-1s can now be made available also for user analysis since there is no risk any longer of triggering chaotic staging from tape.In this contribution, we describe the technical solutions adopted for the new disk and tape endpoints at the sites, and we report on the commissioning and scale testing of the service. We detail the procedures implemented by CMS computing operations to actively manage data on disk at Tier-1 sites, and we give examples of the benefits brought to CMS workflows by the additional flexibility of the new system.},

  doi          = {10.1088/1742-6596/664/4/042056},

  journal      = {Journal of Physics. Conference Series},

  number       = 4,

  volume       = 664,

  place        = {United States},

  year         = {Wed Dec 23 00:00:00 EST 2015},

  month        = {Wed Dec 23 00:00:00 EST 2015}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1088/1742-6596/664/4/042056

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

CMS Data transfer operations after the first years of LHC collisions

Conference Kaselis, Rapolas ; Piperov, S. ; Magini, N. ; ... - J.Phys.Conf.Ser.

CMS experiment utilizes distributed computing infrastructure and its performance heavily depends on the fast and smooth distribution of data between different CMS sites. Data must be transferred from the Tier-0 (CERN) to the Tier-1s for processing, storing and archiving, and time and good quality are vital to avoid overflowing CERN storage buffers. At the same time, processed data has to be distributed from Tier-1 sites to all Tier-2 sites for physics analysis while Monte Carlo simulations sent back to Tier-1 sites for further archival. At the core of all transferring machinery is PhEDEx (Physics Experiment Data Export) data transfer system.more »« less
https://doi.org/10.1088/1742-6596/396/4/042033

Full Text Available
Large scale and low latency analysis facilities for the CMS experiment: Development and operational aspects

Conference Riahi, H. ; Gowdy, S. ; Kreuzer, P. ; ... - J.Phys.Conf.Ser.

While a majority of CMS data analysis activities rely on the distributed computing infrastructure on the WLCG Grid, dedicated local computing facilities have been deployed to address particular requirements in terms of latency and scale. The CMS CERN Analysis Facility (CAF) was primarily designed to host a large variety of latency-critical workflows. These break down into alignment and calibration, detector commissioning and diagnosis, and high-interest physics analysis requiring fast turnaround. In order to reach the goal for fast turnaround tasks, the Workload Management group has designed a CRABServer based system to fit with two main needs: to provide a simple,more »« less
https://doi.org/10.1088/1742-6596/331/7/072030

Full Text Available
ATLAS Data Carousel

Conference Barisits, Martin ; Elmsheuser, Johannes ; Borodin, M ; ...

The ATLAS experiment at CERN's LHC stores detector and simulation data in raw and derived data formats across more than 150 Grid sites world-wide, currently in total about 200PB on disk and 250PB on tape. Data have different access characteristics due to various computational workflows, and can be accessed from different media, such as remote I/O, disk cache on hard disk drives or SSDs. Also, larger data centers provide the majority of offline storage capability via tape systems. For the High-Luminosity LHC (HL-LHC), the estimated data storage requirements are several factors bigger than the present forecast of available resources, basedmore »« less
https://doi.org/10.1051/epjconf/202024504035

Full Text Available
Integration of Russian Tier-1 Grid Center with High Performance Computers at NRC-KI for LHC experiments and beyond HENP

Journal Article Belyaev, A. ; Berezhnaya, A. ; Betev, L. ; ... - Journal of Physics. Conference Series

The LHC experiments are preparing for the precision measurements and further discoveries that will be made possible by higher LHC energies from April 2015 (LHC Run2). The need for simulation, data processing and analysis would overwhelm the expected capacity of grid infrastructure computing facilities deployed by the Worldwide LHC Computing Grid (WLCG). To meet this challenge the integration of the opportunistic resources into LHC computing model is highly important. The Tier-1 facility at Kurchatov Institute (NRC-KI) in Moscow is a part of WLCG and it will process, simulate and store up to 10% of total data obtained from ALICE, ATLASmore »« less
Cited by 2
https://doi.org/10.1088/1742-6596/664/9/092018

Full Text Available
Monitoring data transfer latency in CMS computing operations

Journal Article Bonacorsi, Daniele ; Diotalevi, Tommaso ; Magini, Nicolo ; ... - Journal of Physics. Conference Series

During the first LHC run, the CMS experiment collected tens of Petabytes of collision and simulated data, which need to be distributed among dozens of computing centres with low latency in order to make efficient use of the resources. While the desired level of throughput has been successfully achieved, it is still common to observe transfer workflows that cannot reach full completion in a timely manner due to a small fraction of stuck files which require operator intervention.For this reason, in 2012 the CMS transfer management system, PhEDEx, was instrumented with a monitoring system to measure file transfer latencies, andmore »« less
Cited by 2
https://doi.org/10.1088/1742-6596/664/3/032033

Full Text Available

Similar Records