CMS distributed data analysis with CRAB3
Abstract
The CMS Remote Analysis Builder (CRAB) is a distributed workflow management tool which facilitates analysis tasks by isolating users from the technical details of the Grid infrastructure. Throughout LHC Run 1, CRAB has been successfully employed by an average of 350 distinct users each week executing about 200,000 jobs per day.CRAB has been significantly upgraded in order to face the new challenges posed by LHC Run 2. Components of the new system include 1) a lightweight client, 2) a central primary server which communicates with the clients through a REST interface, 3) secondary servers which manage user analysis tasks and submit jobs to the CMS resource provisioning system, and 4) a central service to asynchronously move user data from temporary storage in the execution site to the desired storage location. Furthermore, the new system improves the robustness, scalability and sustainability of the service.Here we provide an overview of the new system, operation, and user support, report on its current status, and identify lessons learned from the commissioning phase and production roll-out.
- Authors:
-
- Univ. e INFN, Sezione di Milano-Bicocca (Rome)
- Vilnius Univ., Vilnius (Lithuania)
- Univ. e INFN, Sezione di Trieste (Italy)
- Univ. of Nebraska-Lincoln, Lincoln, NE (United States)
- CIEMAT, Madrid (Spain)
- Univ. e INFN, Sezione di Perugia (Italy)
- Bulgarian Academy of Sciences, Sofia (Bulgaria)
- Univ. Estadual Paulista (Brazil)
- Univ. of Malaya (Malaysia)
- Vanderbilt Univ., Nashville, TN (United States)
- European Organization for Nuclear Research (CERN), Geneva (Switzerland)
- Univ. of California, San Diego, CA (United States)
- Univ. of Notre Dame, Notre Dame, IN (United States)
- Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
- Publication Date:
- Research Org.:
- Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), High Energy Physics (HEP)
- OSTI Identifier:
- 1250780
- Report Number(s):
- FERMILAB-CONF-15-606-CD
Journal ID: ISSN 1742-6588; 1413959
- Grant/Contract Number:
- AC02-07CH11359
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Journal of Physics. Conference Series
- Additional Journal Information:
- Journal Volume: 664; Journal Issue: 6; Conference: 21st International Conference on Computing in High Energy and Nuclear Physics, Okinawa (Japan), 13-17 Apr 2015; Journal ID: ISSN 1742-6588
- Publisher:
- IOP Publishing
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 97 MATHEMATICS AND COMPUTING
Citation Formats
Mascheroni, M., Balcas, J., Belforte, S., Bockelman, B. P., Hernandez, J. M., Ciangottini, D., Konstantinov, P. B., Silva, J. M. D., Ali, M. A. B. M., Melo, A. M., Riahi, H., Tanasijczuk, A. J., Yusli, M. N. B., Wolf, M., Woodard, A. E., and Vaandering, E. CMS distributed data analysis with CRAB3. United States: N. p., 2015.
Web. doi:10.1088/1742-6596/664/6/062038.
Mascheroni, M., Balcas, J., Belforte, S., Bockelman, B. P., Hernandez, J. M., Ciangottini, D., Konstantinov, P. B., Silva, J. M. D., Ali, M. A. B. M., Melo, A. M., Riahi, H., Tanasijczuk, A. J., Yusli, M. N. B., Wolf, M., Woodard, A. E., & Vaandering, E. CMS distributed data analysis with CRAB3. United States. https://doi.org/10.1088/1742-6596/664/6/062038
Mascheroni, M., Balcas, J., Belforte, S., Bockelman, B. P., Hernandez, J. M., Ciangottini, D., Konstantinov, P. B., Silva, J. M. D., Ali, M. A. B. M., Melo, A. M., Riahi, H., Tanasijczuk, A. J., Yusli, M. N. B., Wolf, M., Woodard, A. E., and Vaandering, E. Wed .
"CMS distributed data analysis with CRAB3". United States. https://doi.org/10.1088/1742-6596/664/6/062038. https://www.osti.gov/servlets/purl/1250780.
@article{osti_1250780,
title = {CMS distributed data analysis with CRAB3},
author = {Mascheroni, M. and Balcas, J. and Belforte, S. and Bockelman, B. P. and Hernandez, J. M. and Ciangottini, D. and Konstantinov, P. B. and Silva, J. M. D. and Ali, M. A. B. M. and Melo, A. M. and Riahi, H. and Tanasijczuk, A. J. and Yusli, M. N. B. and Wolf, M. and Woodard, A. E. and Vaandering, E.},
abstractNote = {The CMS Remote Analysis Builder (CRAB) is a distributed workflow management tool which facilitates analysis tasks by isolating users from the technical details of the Grid infrastructure. Throughout LHC Run 1, CRAB has been successfully employed by an average of 350 distinct users each week executing about 200,000 jobs per day.CRAB has been significantly upgraded in order to face the new challenges posed by LHC Run 2. Components of the new system include 1) a lightweight client, 2) a central primary server which communicates with the clients through a REST interface, 3) secondary servers which manage user analysis tasks and submit jobs to the CMS resource provisioning system, and 4) a central service to asynchronously move user data from temporary storage in the execution site to the desired storage location. Furthermore, the new system improves the robustness, scalability and sustainability of the service.Here we provide an overview of the new system, operation, and user support, report on its current status, and identify lessons learned from the commissioning phase and production roll-out.},
doi = {10.1088/1742-6596/664/6/062038},
journal = {Journal of Physics. Conference Series},
number = 6,
volume = 664,
place = {United States},
year = {Wed Dec 23 00:00:00 EST 2015},
month = {Wed Dec 23 00:00:00 EST 2015}
}
Web of Science