skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: CMS DAQ event builder based on Gigabit Ethernet

Abstract

The CMS Data Acquisition System is designed to build and filter events originating from 476 detector data sources at a maximum trigger rate of 100 KHz. Different architectures and switch technologies have been evaluated to accomplish this purpose. Events will be built in two stages: the first stage will be a set of event builders called FED Builders. These will be based on Myrinet technology and will pre-assemble groups of about 8 data sources. The second stage will be a set of event builders called Readout Builders. These will perform the building of full events. A single Readout Builder will build events from 72 sources of 16 KB fragments at a rate of 12.5 KHz. In this paper we present the design of a Readout Builder based on TCP/IP over Gigabit Ethernet and the optimization that was required to achieve the design throughput. This optimization includes architecture of the Readout Builder, the setup of TCP/IP, and hardware selection.

Authors:
; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ;
Publication Date:
Research Org.:
Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
919586
Report Number(s):
FERMILAB-CONF-07-364-E
TRN: US0806448
DOE Contract Number:
AC02-07CH11359
Resource Type:
Conference
Resource Relation:
Conference: Prepared for 15th IEEE Real Time Conference 2007 (RT 07), Batavia, Illinois, 29 Apr - 4 May 2007
Country of Publication:
United States
Language:
English
Subject:
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND TECHNOLOGY; COMPUTER ARCHITECTURE; BUILDERS; DATA ACQUISITION SYSTEMS; DESIGN; OPTIMIZATION; Instrumentation

Citation Formats

Bauer, G., /MIT, Boyer, V., /CERN, Branson, J., /UC, San Diego, Brett, A., Cano, E., Carboni, A., Ciganek, M., Cittolin, S., /CERN, Erhan, Samim, /CERN /UCLA, Gigi, D., Glege, F., and /CERN /INFN, Legnaro /CERN /CERN /Kyungpook Natl. U. /MIT /UC, San Diego /CERN. CMS DAQ event builder based on Gigabit Ethernet. United States: N. p., 2007. Web.
Bauer, G., /MIT, Boyer, V., /CERN, Branson, J., /UC, San Diego, Brett, A., Cano, E., Carboni, A., Ciganek, M., Cittolin, S., /CERN, Erhan, Samim, /CERN /UCLA, Gigi, D., Glege, F., & /CERN /INFN, Legnaro /CERN /CERN /Kyungpook Natl. U. /MIT /UC, San Diego /CERN. CMS DAQ event builder based on Gigabit Ethernet. United States.
Bauer, G., /MIT, Boyer, V., /CERN, Branson, J., /UC, San Diego, Brett, A., Cano, E., Carboni, A., Ciganek, M., Cittolin, S., /CERN, Erhan, Samim, /CERN /UCLA, Gigi, D., Glege, F., and /CERN /INFN, Legnaro /CERN /CERN /Kyungpook Natl. U. /MIT /UC, San Diego /CERN. Sun . "CMS DAQ event builder based on Gigabit Ethernet". United States. doi:. https://www.osti.gov/servlets/purl/919586.
@article{osti_919586,
title = {CMS DAQ event builder based on Gigabit Ethernet},
author = {Bauer, G. and /MIT and Boyer, V. and /CERN and Branson, J. and /UC, San Diego and Brett, A. and Cano, E. and Carboni, A. and Ciganek, M. and Cittolin, S. and /CERN and Erhan, Samim and /CERN /UCLA and Gigi, D. and Glege, F. and /CERN /INFN, Legnaro /CERN /CERN /Kyungpook Natl. U. /MIT /UC, San Diego /CERN},
abstractNote = {The CMS Data Acquisition System is designed to build and filter events originating from 476 detector data sources at a maximum trigger rate of 100 KHz. Different architectures and switch technologies have been evaluated to accomplish this purpose. Events will be built in two stages: the first stage will be a set of event builders called FED Builders. These will be based on Myrinet technology and will pre-assemble groups of about 8 data sources. The second stage will be a set of event builders called Readout Builders. These will perform the building of full events. A single Readout Builder will build events from 72 sources of 16 KB fragments at a rate of 12.5 KHz. In this paper we present the design of a Readout Builder based on TCP/IP over Gigabit Ethernet and the optimization that was required to achieve the design throughput. This optimization includes architecture of the Readout Builder, the setup of TCP/IP, and hardware selection.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Sun Apr 01 00:00:00 EDT 2007},
month = {Sun Apr 01 00:00:00 EDT 2007}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:
  • The CMS Data Acquisition system is designed to build and filter events originating from approximately 500 data sources from the detector at a maximum Level 1 trigger rate of 100 kHz and with an aggregate throughput of 100 GByte/s. For this purpose different architectures and switch technologies have been evaluated. Events will be built in two stages: the first stage, the FED Builder, will be based on Myrinet technology and will pre-assemble groups of about 8 data sources. The next stage, the Readout Builder, will perform the building of full events. The requirement of one Readout Builder is to buildmore » events at 12.5 kHz with average size of 16 kBytes from 64 sources. In this paper we present the prospects of a Readout Builder based on TCP/IP over Gigabit Ethernet. Various Readout Builder architectures that we are considering are discussed. The results of throughput measurements and scaling performance are outlined as well as the preliminary estimates of the final performance. All these studies have been carried out at our test-bed farms that are made up of a total of 130 dual Xeon PCs interconnected with Myrinet and Gigabit Ethernet networking and switching technologies.« less
  • The present data acquisition system of the CDF experiment has to be upgraded for the higher luminosities expected during the Run 11 (1999+) data-taking period. The core of the system, consisting of a control network based on reflective memories will remain the same. The network used for data transfers, however, will have to be changed. We have investigated ATM as a possible replacement technology for the current Ultranet switch. We present preliminary results on this new ATM-based event builder system.
  • The Electronic Systems Engineering department of the Computing Division at the Fermi National Accelerator Laboratory has assembled a pixel test beam telescope for high energy physics detector research and development. The telescope features CMS PSI46 readout chips and a data acquisition and control system known as the Compact And Programmable daTa Acquisition Node or CAPTAN. The CAPTAN is a flexible and powerful system that meets the readout and control demands of a variety of pixel and strip detectors for high energy physics applications. The CAPTAN functions in a gigabit Ethernet network, which facilitates the coordination of the multiple pixel planesmore » of the telescope. Through the use of the CAPTAN hardware, a unified telescope system is attained encompassing both the CMS PSI46 pixel components and adevice under test. This paper discusses results from the telescope project including mechanical design, alignment procedure, and attainble precision.« less
  • The Data Acquisition system of the Compact Muon Solenoid experiment at CERN assembles events at a rate of 100 kHz, transporting event data at an aggregate throughput of 100 GB/s. By the time the LHC restarts after the 2013/14 shut-down, the current computing and networking infrastructure will have reached the end of their lifetime. This paper presents design studies for an upgrade of the CMS event builder based on advanced networking technologies such as 10/40 Gb/s Ethernet and Infiniband. The results of performance measurements with small-scale test setups are shown.
  • The data acquisition system (DAQ) of the CMS experiment at the CERN Large Hadron Collider assembles events at a rate of 100 kHz, transporting event data at an aggregate throughput of to the high-level trigger farm. The DAQ architecture is based on state-of-the-art network technologies for the event building. For the data concentration, 10/40 Gbit/s Ethernet technologies are used together with a reduced TCP/IP protocol implemented in FPGA for a reliable transport between custom electronics and commercial computing hardware. A 56 Gbit/s Infiniband FDR Clos network has been chosen for the event builder. This paper presents the implementation and performancemore » of the event-building system.« less