Investigation of fast and efficient lossless compression algorithms for macromolecular crystallography experiments

Bernstein, Herbert J.; Jakoncic, Jean

doi:10.1107/S160057752400359X

Investigation of fast and efficient lossless compression algorithms for macromolecular crystallography experiments

Journal Article · Wed Jun 05 00:00:00 EDT 2024 · Journal of Synchrotron Radiation (Online)

DOI:https://doi.org/10.1107/S160057752400359X· OSTI ID:2370561

;

Structural biology experiments benefit significantly from state-of-the-art synchrotron data collection. One can acquire macromolecular crystallography (MX) diffraction data on large-area photon-counting pixel-array detectors at framing rates exceeding 1000 frames per second, using 200 Gbps network connectivity, or higher when available. In extreme cases this represents a raw data throughput of about 25 GB s ⁻¹ , which is nearly impossible to deliver at reasonable cost without compression. Our field has used lossless compression for decades to make such data collection manageable. Many MX beamlines are now fitted with DECTRIS Eiger detectors, all of which are delivered with optimized compression algorithms by default, and they perform well with current framing rates and typical diffraction data. However, better lossless compression algorithms have been developed and are now available to the research community. Here one of the latest and most promising lossless compression algorithms is investigated on a variety of diffraction data like those routinely acquired at state-of-the-art MX beamlines.

View Journal Article

Research Organization:: Brookhaven National Laboratory (BNL), Upton, NY (United States). National Synchrotron Light Source II (NSLS-II)

Sponsoring Organization:: National Institute of General Medical Sciences (NIGMS); USDOE; USDOE Office of Science (SC), Basic Energy Sciences (BES); USDOE Office of Science (SC), Biological and Environmental Research (BER)

Grant/Contract Number:: SC0012704

OSTI ID:: 2370561

Alternate ID(s):: OSTI ID: 2372825

Report Number(s):: BNL--225684-2024-JAAM; PII: S160057752400359X

Journal Information:: Journal of Synchrotron Radiation (Online), Journal Name: Journal of Synchrotron Radiation (Online) Journal Issue: 4 Vol. 31; ISSN 1600-5775; ISSN JSYRES

Publisher:: International Union of Crystallography (IUCr)Copyright Statement

Country of Publication:: Denmark

Language:: English

References (18)

Transmission of Information ¹ Hartley, R. V. L. Bell System Technical Journal, Vol. 7, Issue 3 https://doi.org/10.1002/j.1538-7305.1928.tb01236.x	journal	July 1928
A Mathematical Theory of Communication Shannon, C. E. Bell System Technical Journal, Vol. 27, Issue 3 https://doi.org/10.1002/j.1538-7305.1948.tb01338.x	journal	July 1948
Continuous sample rotation data collection for protein crystallography with the PILATUS detector Brönnimann, Ch; Eikenberry, E. F.; Horisberger, R. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 510, Issue 1-2 https://doi.org/10.1016/S0168-9002(03)01673-5	journal	September 2003
X-rays and Crystals Bragg, W. H. Nature, Vol. 90, Issue 2243 https://doi.org/10.1038/090219a0	journal	October 1912
Best practices for high data-rate macromolecular crystallography (HDRMX) Bernstein, Herbert J.; Andrews, Lawrence C.; Diaz, Jorge A. Structural Dynamics, Vol. 7, Issue 1 https://doi.org/10.1063/1.5128498	journal	January 2020
Meeting Report: Workshop on Beamline Integration and Data Formatting Donath, Tilman; Rissi, Michael; Billich, Heiner Synchrotron Radiation News, Vol. 26, Issue 5 https://doi.org/10.1080/08940886.2013.832589	journal	September 2013
Future trends in synchrotron science at NSLS-II Hill, John; Campbell, Stuart; Carini, Gabriella Journal of Physics: Condensed Matter, Vol. 32, Issue 37 https://doi.org/10.1088/1361-648X/ab7b19	journal	June 2020
ImageNCIF: an initiative to standardise image formats Hammersley, A. P. Acta Crystallographica Section A Foundations of Crystallography, Vol. 52, Issue a1 https://doi.org/10.1107/S0108767396076532	journal	August 1996
Bragg coherent diffraction imaging with the CITIUS charge-integrating detector Grimes, Michael; Pauwels, Kristof; Schülli, Tobias U. Journal of Applied Crystallography, Vol. 56, Issue 4 https://doi.org/10.1107/S1600576723004314	journal	June 2023
Solution scattering at the Life Science X-ray Scattering (LiX) beamline Yang, Lin; Antonelli, Stephen; Chodankar, Shirish Journal of Synchrotron Radiation, Vol. 27, Issue 3 https://doi.org/10.1107/S1600577520002362	journal	March 2020
AMX – the highly automated macromolecular crystallography (17-ID-1) beamline at the NSLS-II Schneider, Dieter K.; Soares, Alexei S.; Lazo, Edwin O. Journal of Synchrotron Radiation, Vol. 29, Issue 6 https://doi.org/10.1107/S1600577522009377	journal	October 2022
A simple technique to classify diffraction data from dynamic proteins according to individual polymorphs Nguyen, Thu; Phan, Kim L.; Kozakov, Dima Acta Crystallographica Section D Structural Biology, Vol. 78, Issue 3 https://doi.org/10.1107/S2059798321013425	journal	February 2022
Elucidating polymorphs of crystal structures by intensity-based hierarchical clustering analysis of multiple diffraction data sets Matsuura, Hiroaki; Sakai, Naoki; Toma-Fukai, Sachiko Acta Crystallographica Section D Structural Biology, Vol. 79, Issue 10 https://doi.org/10.1107/S2059798323007039	journal	September 2023
A Technique for High-Performance Data Compression No authors listed Computer, Vol. 17, Issue 6 https://doi.org/10.1109/MC.1984.1659158	journal	June 1984
Certain Factors Affecting Telegraph Speed Nyquist, H. Transactions of the American Institute of Electrical Engineers, Vol. XLIII https://doi.org/10.1109/T-AIEE.1924.5060996	journal	January 1924
A universal algorithm for sequential data compression Ziv, J.; Lempel, A. IEEE Transactions on Information Theory, Vol. 23, Issue 3 https://doi.org/10.1109/TIT.1977.1055714	journal	May 1977
Zstandard Compression and the 'application/zstd' Media Type Collet, Y.; Kucherawy, M. https://doi.org/10.17487/RFC8878	report	February 2021
Lossless Image Compression Techniques: A State-of-the-Art Survey Rahman, Md. Atiqur; Hamada, Mohamed Symmetry, Vol. 11, Issue 10 https://doi.org/10.3390/sym11101274	journal	October 2019

Similar Records

The use of a mini-κ goniometer head in macromolecular crystallography diffraction experiments

Journal Article · Mon Jul 01 00:00:00 EDT 2013 · Acta Crystallographica. Section D: Biological Crystallography · OSTI ID:22347851

Gold Standard for macromolecular crystallography diffraction data

Journal Article · Thu Jul 09 20:00:00 EDT 2020 · IUCrJ · OSTI ID:1638110

Related Subjects

36 MATERIALS SCIENCE
high framing rate
lossless compression
macromolecular crystallography diffraction

Investigation of fast and efficient lossless compression algorithms for macromolecular crystallography experiments

Citation Formats

References (18)

Similar Records

Related Subjects