Recent evolution of the offline computing model of the NOvA experiment

Habig, Alec; Norman, A.; Group, Craig

doi:10.1088/1742-6596/664/3/032011

Title: Recent evolution of the offline computing model of the NOvA experiment

Full Record
Other Related Research

Abstract

The NOvA experiment at Fermilab is a long-baseline neutrino experiment designed to study ν_e appearance in a ν_μ beam. Over the last few years there has been intense work to streamline the computing infrastructure in preparation for data, which started to flow in from the far detector in Fall 2013. Major accomplishments for this effort include migration to the use of off-site resources through the use of the Open Science Grid and upgrading the file-handling framework from simple disk storage to a tiered system using a comprehensive data management and delivery system to find and access files on either disk or tape storage. NOvA has already produced more than 6.5 million files and more than 1 PB of raw data and Monte Carlo simulation files which are managed under this model. In addition, the current system has demonstrated sustained rates of up to 1 TB/hour of file transfer by the data handling system. NOvA pioneered the use of new tools and this paved the way for their use by other Intensity Frontier experiments at Fermilab. Most importantly, the new framework places the experiment's infrastructure on a firm foundation, and is ready to produce the files needed for first physics.

Authors:

Habig, Alec ^[1]; Norman, A. ^[2]; Group, Craig ^[3]

Univ. of Minnesota Duluth, Duluth, MN (United States)
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Univ. of Virginia, Charlottesville, VA (United States)

Publication Date:: Wed Dec 23 00:00:00 EST 2015

Research Org.:: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)

Sponsoring Org.:: USDOE Office of Science (SC), High Energy Physics (HEP)

OSTI Identifier:: 1250771

Report Number(s):: FERMILAB-CONF-15-198-CD-ND
Journal ID: ISSN 1742-6588; 1413812

Grant/Contract Number:: AC02-07CH11359

Resource Type:: Journal Article: Accepted Manuscript

Journal Name:: Journal of Physics. Conference Series

Additional Journal Information:: Journal Volume: 664; Journal Issue: 3; Conference: 21st International Conference on Computing in High Energy and Nuclear Physics, Okinawa (Japan), 13-17 Apr 2015; Journal ID: ISSN 1742-6588

Publisher:: IOP Publishing

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Habig, Alec, Norman, A., and Group, Craig. Recent evolution of the offline computing model of the NOvA experiment.  United States: N. p., 2015. 
        Web.  doi:10.1088/1742-6596/664/3/032011.

Copy to clipboard


                    Habig, Alec, Norman, A., & Group, Craig. Recent evolution of the offline computing model of the NOvA experiment.  United States.  https://doi.org/10.1088/1742-6596/664/3/032011

Copy to clipboard


                    Habig, Alec, Norman, A., and Group, Craig. 2015.  
        "Recent evolution of the offline computing model of the NOvA experiment".  United States.  https://doi.org/10.1088/1742-6596/664/3/032011.  https://www.osti.gov/servlets/purl/1250771.

Copy to clipboard


                    
@article{osti_1250771,

  title        = {Recent evolution of the offline computing model of the NOvA experiment},

  author       = {Habig, Alec and Norman, A. and Group, Craig},

  abstractNote = {The NOvA experiment at Fermilab is a long-baseline neutrino experiment designed to study νe appearance in a νμ beam. Over the last few years there has been intense work to streamline the computing infrastructure in preparation for data, which started to flow in from the far detector in Fall 2013. Major accomplishments for this effort include migration to the use of off-site resources through the use of the Open Science Grid and upgrading the file-handling framework from simple disk storage to a tiered system using a comprehensive data management and delivery system to find and access files on either disk or tape storage. NOvA has already produced more than 6.5 million files and more than 1 PB of raw data and Monte Carlo simulation files which are managed under this model. In addition, the current system has demonstrated sustained rates of up to 1 TB/hour of file transfer by the data handling system. NOvA pioneered the use of new tools and this paved the way for their use by other Intensity Frontier experiments at Fermilab. Most importantly, the new framework places the experiment's infrastructure on a firm foundation, and is ready to produce the files needed for first physics.},

  doi          = {10.1088/1742-6596/664/3/032011},

  url          = {https://www.osti.gov/biblio/1250771},
  journal      = {Journal of Physics. Conference Series},

  issn         = {1742-6588},

  number       = 3,

  volume       = 664,

  place        = {United States},

  year         = {Wed Dec 23 00:00:00 EST 2015},

  month        = {Wed Dec 23 00:00:00 EST 2015}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1088/1742-6596/664/3/032011

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar records in OSTI.GOV collections:

Distributed data access in the sequential access model at the D0 experiment at Fermilab

Conference Terekhov, Igor; White, Victoria

The authors present the Sequential Access Model (SAM), which is the data handling system for D0, one of two primary High Energy Experiments at Fermilab. During the next several years, the D0 experiment will store a total of about 1 PByte of data, including raw detector data and data processed at various levels. The design of SAM is not specific to the D0 experiment and carries few assumptions about the underlying mass storage level; its ideas are applicable to any sequential data access. By definition, in the sequential access mode a user application needs to process a stream of data,more »« less
https://doi.org/10.1109/HPDC.2000.868672

Full Text Available
NOvA Event Building, Buffering and Data-Driven Triggering From Within the DAQ System

Conference Fischler, M.; Green, C.; Kowalkowski, J.; ... - J.Phys.Conf.Ser.

To make its core measurements, the NOvA experiment needs to make real-time data-driven decisions involving beam-spill time correlation and other triggering issues. NOvA-DDT is a prototype Data-Driven Triggering system, built using the Fermilab artdaq generic DAQ/Event-building tools set. This provides the advantages of sharing online software infrastructure with other Intensity Frontier experiments, and of being able to use any offline analysis module--unchanged--as a component of the online triggering decisions. The NOvA-artdaq architecture chosen has significant advantages, including graceful degradation if the triggering decision software fails or cannot be done quickly enough for some fraction of the time-slice ``events.'' We havemore »« less
https://doi.org/10.1088/1742-6596/396/1/012020

Full Text Available
Data handling with SAM and art at the NOvA experiment

Journal Article Aurisano, A.; Backhouse, C.; Davies, G.; ... - Journal of Physics. Conference Series

During operations, NOvA produces between 5,000 and 7,000 raw files per day with peaks in excess of 12,000. These files must be processed in several stages to produce fully calibrated and reconstructed analysis files. In addition, many simulated neutrino interactions must be produced and processed through the same stages as data. To accommodate the large volume of data and Monte Carlo, production must be possible both on the Fermilab grid and on off-site farms, such as the ones accessible through the Open Science Grid. To handle the challenge of cataloging these files and to facilitate their off-line processing, we havemore »« less
https://doi.org/10.1088/1742-6596/664/4/042001

Full Text Available
The D0 Experiment Data Grid: SAM

Journal Article Lueking, L.; Loebel-Carpenter, L.; Merritt, W.; ... - Lect.Notes Comput.Sci.

SAM (Sequential Access through Meta-data) is a data grid and data cataloging system developed for the D0 high energy physics (HEP) experiment at Fermilab. Since March 2001 , D0 has been acquiring data in real time from the detector and will archive up to 1/2 Petabyte a year of simulated, raw detector and processed physics data. SAM catalogs the event and calibration data, provides distributed file delivery and caching services, and manages the processing and analysis jobs for the hundreds of D0 collaborators around the world. The D0 applications are data-intensive and the physics analysis programs execute on the ordermore »« less
https://doi.org/10.1007/3-540-45644-9_17

Full Text Available
Langston University - High Energy Physics (LU-HEP)

Technical Report Snow, Dr., Joel

This final report is presented by Langston University (LU) for the project entitled "Langston University High Energy Physics" (LUHEP) under the direction of principal investigator (PI) and project director Professor Joel Snow. The project encompassed high energy physics research performed at hadron colliders. The PI is a collaborator on the DZero experiment at Fermi National Accelerator Laboratory in Batavia, IL, USA and the ATLAS experiment at CERN in Geneva, Switzerland and was during the entire project period from April 1, 1999 until May 14, 2012. Both experiments seek to understand the fundamental constituents of the physical universe and the forcesmore »« less
https://doi.org/10.2172/1048139

Full Text Available

Similar Records