skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: ProIO: An Event-Based I/O Stream Format for Protobuf Messages

Abstract

ProIO is a new event-oriented streaming data format which utilizes Google’s Protocol Buffers (protobuf) to be flexible and highly language-neutral. The ProIO concept is described here along with its software implementations. The performance of the ProIO concept for a dataset with Monte-Carlo event records used in high-energy physics was benchmarked and compared/contrasted with ROOT I/O. Various combinations of general-purpose compression and variable-length integer encoding available in protobuf were used to investigate the relationship between I/O performance and size-on-disk in a few key scenarios.

Authors:
; ; ;
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science - Office of High Energy Physics
OSTI Identifier:
1513238
DOE Contract Number:  
AC02-06CH11357
Resource Type:
Journal Article
Journal Name:
Computer Physics Communications
Additional Journal Information:
Journal Volume: 241
Country of Publication:
United States
Language:
English
Subject:
Event; I/O; Protobuf; Stream; data format; data storage; proio

Citation Formats

Blyth, David, Alcaraz, Jose de Jesus, Binet, S, and Chekanov, S. V. ProIO: An Event-Based I/O Stream Format for Protobuf Messages. United States: N. p., 2019. Web. doi:10.1016/j.cpc.2019.03.018.
Blyth, David, Alcaraz, Jose de Jesus, Binet, S, & Chekanov, S. V. ProIO: An Event-Based I/O Stream Format for Protobuf Messages. United States. doi:10.1016/j.cpc.2019.03.018.
Blyth, David, Alcaraz, Jose de Jesus, Binet, S, and Chekanov, S. V. Thu . "ProIO: An Event-Based I/O Stream Format for Protobuf Messages". United States. doi:10.1016/j.cpc.2019.03.018.
@article{osti_1513238,
title = {ProIO: An Event-Based I/O Stream Format for Protobuf Messages},
author = {Blyth, David and Alcaraz, Jose de Jesus and Binet, S and Chekanov, S. V.},
abstractNote = {ProIO is a new event-oriented streaming data format which utilizes Google’s Protocol Buffers (protobuf) to be flexible and highly language-neutral. The ProIO concept is described here along with its software implementations. The performance of the ProIO concept for a dataset with Monte-Carlo event records used in high-energy physics was benchmarked and compared/contrasted with ROOT I/O. Various combinations of general-purpose compression and variable-length integer encoding available in protobuf were used to investigate the relationship between I/O performance and size-on-disk in a few key scenarios.},
doi = {10.1016/j.cpc.2019.03.018},
journal = {Computer Physics Communications},
number = ,
volume = 241,
place = {United States},
year = {2019},
month = {8}
}