Raw diffraction data preservation and reuse: Overview, update on practicalities and metadata requirements

Kroon-Batenburg, Loes M. J.; Helliwell, John R.; McMahon, Brian; Terwilliger, Thomas Charles

doi:10.1107/S2052252516018315

Title: Raw diffraction data preservation and reuse: Overview, update on practicalities and metadata requirements

Journal Article · Sun Jan 01 00:00:00 UTC 2017 · IUCrJ

DOI: https://doi.org/10.1107/S2052252516018315 · OSTI ID:1338781

Kroon-Batenburg, Loes M. J. ^[1];

^[2];

^[3];

^[4]

Utrecht Univ., Utrecht (The Netherlands)
Univ. of Manchester, Manchester (United Kingdom)
International Union of Crystallography, Chester (United Kingdom)
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

A topical review is presented of the rapidly developing interest in and storage options for the preservation and reuse of raw data within the scientific domain of the IUCr and its Commissions, each of which operates within a great diversity of instrumentation. A résumé is included of the case for raw diffraction data deposition. An overall context is set by highlighting the initiatives of science policy makers towards an `Open Science' model within which crystallographers will increasingly work in the future; this will bring new funding opportunities but also new codes of procedure within open science frameworks. Skills education and training for crystallographers will need to be expanded. Overall, there are now the means and the organization for the preservation of raw crystallographic diffraction dataviadifferent types of archive, such as at universities, discipline-specific repositories (Integrated Resource for Reproducibility in Macromolecular Crystallography, Structural Biology Data Grid), general public data repositories (Zenodo, ResearchGate) and centralized neutron and X-ray facilities. Formulation of improved metadata descriptors for the raw data types of each of the IUCr Commissions is in progress; some detailed examples are provided. Lastly, a number of specific case studies are presented, including an example research thread that provides complete open access to raw data.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Los Alamos National Laboratory (LANL)

Sponsoring Organization:: National Institutes of Health; USDOE

Grant/Contract Number:: AC52-06NA25396

OSTI ID:: 1338781

Report Number(s):: LA-UR-16-26332

Journal Information:: IUCrJ, Journal Name: IUCrJ Journal Issue: 1 Vol. 4; ISSN 2052-2525; ISSN IUCRAJ

Publisher:: International Union of CrystallographyCopyright Statement

Country of Publication:: United States

Language:: English

References (32)

Cisplatin binding to proteins: A structural perspective Messori, Luigi; Merlino, Antonello Coordination Chemistry Reviews, Vol. 315 https://doi.org/10.1016/j.ccr.2016.01.010	journal	May 2016
Safeguarding Structural Data Repositories against Bad Apples Minor, Wladek; Dauter, Zbigniew; Helliwell, John R. Structure, Vol. 24, Issue 2 https://doi.org/10.1016/j.str.2015.12.010	journal	February 2016
Data publication with the structural biology data grid supports live analysis Meyer, Peter A.; Socias, Stephanie; Key, Jason Nature Communications, Vol. 7, Issue 1 https://doi.org/10.1038/ncomms10882	journal	March 2016
The Protein Data Bank Berman, H. M. Nucleic Acids Research, Vol. 28, Issue 1 https://doi.org/10.1093/nar/28.1.235	journal	January 2000
EVAL15 : a diffraction data integration method based on ab initio predicted profiles Schreurs, Antoine M. M.; Xian, Xinyi; Kroon-Batenburg, Loes M. J. Journal of Applied Crystallography, Vol. 43, Issue 1 https://doi.org/10.1107/S0021889809043234	journal	December 2009
Estimation of errors in diffraction data measured by CCD area detectors Waterman, David; Evans, Gwyndaf Journal of Applied Crystallography, Vol. 43, Issue 6 https://doi.org/10.1107/S0021889810033418	journal	October 2010
CrystFEL : a software suite for snapshot serial crystallography White, Thomas A.; Kirian, Richard A.; Martin, Andrew V. Journal of Applied Crystallography, Vol. 45, Issue 2 https://doi.org/10.1107/S0021889812002312	journal	March 2012
Experience with exchange and archiving of raw data: comparison of data from two diffractometers and four software packages on a series of lysozyme crystals Tanley, Simon W. M.; Schreurs, Antoine M. M.; Helliwell, John R. Journal of Applied Crystallography, Vol. 46, Issue 1 https://doi.org/10.1107/S0021889812044172	journal	December 2012
The crystallographic information file (CIF): a new standard archive file for crystallography Hall, S. R.; Allen, F. H.; Brown, I. D. Acta Crystallographica Section A Foundations of Crystallography, Vol. 47, Issue 6 https://doi.org/10.1107/S010876739101067X	journal	November 1991
Some 60 new space-group corrections Marsh, Richard E.; Kapon, Moshe; Hu, Shengzhi Acta Crystallographica Section B Structural Science, Vol. 58, Issue 1 https://doi.org/10.1107/S0108768101017128	journal	January 2001
XDS Kabsch, Wolfgang Acta Crystallographica Section D Biological Crystallography, Vol. 66, Issue 2 https://doi.org/10.1107/S0907444909047337	journal	January 2010
Data processing and analysis with the autoPROC toolbox Vonrhein, Clemens; Flensburg, Claus; Keller, Peter Acta Crystallographica Section D Biological Crystallography, Vol. 67, Issue 4 https://doi.org/10.1107/S0907444911007773	journal	March 2011
Publication guidelines for structural modelling of small-angle scattering data from biomolecules in solution Jacques, David A.; Guss, J. Mitchell; Svergun, Dmitri I. Acta Crystallographica Section D Biological Crystallography, Vol. 68, Issue 6 https://doi.org/10.1107/S0907444912012073	journal	May 2012
New Python-based methods for data processing Sauter, Nicholas K.; Hattne, Johan; Grosse-Kunstleve, Ralf W. Acta Crystallographica Section D Biological Crystallography, Vol. 69, Issue 7 https://doi.org/10.1107/S0907444913000863	journal	June 2013
Better models by discarding data? Diederichs, K.; Karplus, P. A. Acta Crystallographica Section D Biological Crystallography, Vol. 69, Issue 7 https://doi.org/10.1107/S0907444913001121	journal	June 2013
The role of structural bioinformatics resources in the era of integrative structural biology Gutmanas, Aleksandras; Oldfield, Thomas J.; Patwardhan, Ardan Acta Crystallographica Section D Biological Crystallography, Vol. 69, Issue 5 https://doi.org/10.1107/S0907444913001157	journal	April 2013
Towards data format standardization for X-ray absorption spectroscopy Ravel, B.; Hester, J. R.; Solé, V. A. Journal of Synchrotron Radiation, Vol. 19, Issue 6 https://doi.org/10.1107/S0909049512036886	journal	October 2012
Experiences with archived raw diffraction images data: capturing cisplatin after chemical conversion of carboplatin in high salt conditions for a protein crystal Tanley, Simon W. M.; Diederichs, Kay; Kroon-Batenburg, Loes M. J. Journal of Synchrotron Radiation, Vol. 20, Issue 6 https://doi.org/10.1107/S0909049513020724	journal	October 2013
Experiences with making diffraction image data available: what metadata do we need to archive? Kroon-Batenburg, Loes M. J.; Helliwell, John R. Acta Crystallographica Section D Biological Crystallography, Vol. 70, Issue 10 https://doi.org/10.1107/S1399004713029817	journal	September 2014
How to make deposition of images a reality Guss, J. Mitchell; McMahon, Brian Acta Crystallographica Section D Biological Crystallography, Vol. 70, Issue 10 https://doi.org/10.1107/S1399004714005185	journal	September 2014
Operation of the Australian Store.Synchrotron for macromolecular crystallography Meyer, Grischa R.; Aragão, David; Mudie, Nathan J. Acta Crystallographica Section D Biological Crystallography, Vol. 70, Issue 10 https://doi.org/10.1107/S1399004714016174	journal	September 2014
Continuous mutual improvement of macromolecular structure models in the PDB and of X-ray crystallographic software: the dual role of deposited experimental data Terwilliger, Thomas C.; Bricogne, Gerard Acta Crystallographica Section D Biological Crystallography, Vol. 70, Issue 10 https://doi.org/10.1107/S1399004714017040	journal	September 2014
Archiving raw crystallographic data Terwilliger, Thomas C. Acta Crystallographica Section D Biological Crystallography, Vol. 70, Issue 10 https://doi.org/10.1107/S139900471402118X	journal	September 2014
Crystallography and chemistry should always go together: a cautionary tale of protein complexes with cisplatin and carboplatin Shabalin, Ivan; Dauter, Zbigniew; Jaskolski, Mariusz Acta Crystallographica Section D Biological Crystallography, Vol. 71, Issue 9 https://doi.org/10.1107/S139900471500629X	journal	August 2015
Response from Tanley et al. to Crystallography and chemistry should always go together: a cautionary tale of protein complexes with cisplatin and carboplatin Tanley, Simon W. M.; Diederichs, Kay; Kroon- Batenburg, Loes M. J. Acta Crystallographica Section D Biological Crystallography, Vol. 71, Issue 9 https://doi.org/10.1107/S1399004715014340	journal	August 2015
The NeXus data format Könnecke, Mark; Akeroyd, Frederick A.; Bernstein, Herbert J. Journal of Applied Crystallography, Vol. 48, Issue 1 https://doi.org/10.1107/S1600576714027575	journal	January 2015
Specification of the Crystallographic Information File format, version 2.0 Bernstein, Herbert J.; Bollinger, John C.; Brown, I. David Journal of Applied Crystallography, Vol. 49, Issue 1 https://doi.org/10.1107/S1600576715021871	journal	February 2016
Extension of the sasCIF format and its applications for data processing and deposition Kachala, Michael; Westbrook, John; Svergun, Dmitri Journal of Applied Crystallography, Vol. 49, Issue 1 https://doi.org/10.1107/S1600576715024942	journal	February 2016
Re-refinement of 4g4a: room-temperature X-ray diffraction study of cisplatin and its binding to His15 of HEWL after 14 months chemical exposure in the presence of DMSO Tanley, Simon W. M.; Schreurs, Antoine M. M.; Kroon-Batenburg, Loes M. J. Acta Crystallographica Section F Structural Biology Communications, Vol. 72, Issue 3 https://doi.org/10.1107/S2053230X16000856	journal	February 2016
A public database of macromolecular diffraction experiments Grabowski, Marek; Langner, Karol M.; Cymborowski, Marcin Acta Crystallographica Section D Structural Biology, Vol. 72, Issue 11 https://doi.org/10.1107/S2059798316014716	journal	October 2016
Using a Core Scientific Metadata Model in Large-Scale Facilities Matthews, Brian; Sufi, Shoaib; Flannery, Damian International Journal of Digital Curation, Vol. 5, Issue 1 https://doi.org/10.2218/ijdc.v5i1.146	journal	June 2010
A Robust, Format-Agnostic Scientific Data Transfer Framework Hester, James Data Science Journal, Vol. 15, Issue 0 https://doi.org/10.5334/dsj-2016-012	journal	January 2016

Cited By (19)

Synchrotron Big Data Science Wang, Chunpeng; Steiner, Ullrich; Sepe, Alessandro Small, Vol. 14, Issue 46 https://doi.org/10.1002/smll.201802291	journal	September 2018
Principles and methods used to grow and optimize crystals of protein–metallodrug adducts, to determine metal binding sites and to assign metal ligands Russo Krauss, Irene; Ferraro, Giarita; Pica, Andrea Metallomics, Vol. 9, Issue 11 https://doi.org/10.1039/c7mt00219j	journal	January 2017
FACT and FAIR with Big Data allows objectivity in science: The view of crystallography Helliwell, John R. Structural Dynamics, Vol. 6, Issue 5 https://doi.org/10.1063/1.5124439	journal	September 2019
The Integrated Resource for Reproducibility in Macromolecular Crystallography: Experiences of the first four years Grabowski, Marek; Cymborowski, Marcin; Porebski, Przemyslaw J. Structural Dynamics, Vol. 6, Issue 6 https://doi.org/10.1063/1.5128672	journal	November 2019
Chemical Crystallography: when are ‘ bad data’ ‘good data’? Thompson, Amber L. Crystallography Reviews, Vol. 25, Issue 1 https://doi.org/10.1080/0889311x.2019.1569643	journal	November 2018
Metrics for crystallographic diffraction- and fit-data: a review of existing ones and the need for new ones Henn, J. Crystallography Reviews, Vol. 25, Issue 2 https://doi.org/10.1080/0889311x.2019.1607845	journal	March 2019
FACT and FAIR with big data allows objectivity in science: the view of crystallography Helliwell, John R. Acta Crystallographica Section A Foundations and Advances, Vol. 75, Issue a1 https://doi.org/10.1107/s010876731909946x	journal	July 2019
Sharing powder diffraction raw data: challenges and benefits Aranda, Miguel A. G. Journal of Applied Crystallography, Vol. 51, Issue 6 https://doi.org/10.1107/s160057671801556x	journal	November 2018
The science is in the data Helliwell, John R.; McMahon, Brian; Guss, J. Mitchell IUCrJ, Vol. 4, Issue 6 https://doi.org/10.1107/s2052252517013690	journal	October 2017
Findable Accessible Interoperable Re-usable (FAIR) diffraction data are coming to protein crystallography Helliwell, John R.; Minor, Wladek; Weiss, Manfred S. IUCrJ, Vol. 6, Issue 3 https://doi.org/10.1107/s2052252519005918	journal	April 2019
Findable Accessible Interoperable Re-usable (FAIR) diffraction data are coming to protein crystallography Helliwell, John R.; Minor, Wladek; Weiss, Manfred S. Acta Crystallographica Section F Structural Biology Communications, Vol. 75, Issue 5 https://doi.org/10.1107/s2053230x19005909	journal	April 2019
checkCIF validation ALERTS: what they mean and how to respond Spek, Anthony L. Acta Crystallographica Section E Crystallographic Communications, Vol. 76, Issue 1 https://doi.org/10.1107/s2056989019016244	journal	January 2020
Findable Accessible Interoperable Re-usable (FAIR) diffraction data are coming to protein crystallography Helliwell, John R.; Minor, Wladek; Weiss, Manfred S. Acta Crystallographica Section D Structural Biology, Vol. 75, Issue 5 https://doi.org/10.1107/s2059798319004844	journal	April 2019
FACT and FAIR with big data allows objectivity in science: the view of crystallography [PPT Presentation] Helliwell, John R. https://doi.org/10.5281/zenodo.3355112	posted_content	July 2019
FACT and FAIR with big data allows objectivity in science: the view of crystallography Helliwell, John R. Zenodo https://doi.org/10.5281/zenodo.3355113	text	July 2019
Crystallography and Databases Bruno, Ian; Gražulis, Saulius; Helliwell, John R. Data Science Journal, Vol. 16 https://doi.org/10.5334/dsj-2017-038	journal	August 2017
The Life Cycle of Structural Biology Data Morris, Chris Data Science Journal, Vol. 17 https://doi.org/10.5334/dsj-2018-026	journal	January 2018
Metrics for crystallographic diffraction- and fit-data: a review of existing ones and the need for new ones Henn, J. Figshare https://doi.org/10.6084/m9.figshare.8194661	text	January 2019
Metrics for crystallographic diffraction- and fit-data: a review of existing ones and the need for new ones Henn, J. Figshare https://doi.org/10.6084/m9.figshare.8194661.v1	text	January 2019

Similar Records

The Living Publication

Technical Report · Tue May 01 00:00:00 UTC 2012 · OSTI ID:1043003

Terwilliger, Thomas C.; Helliwell, John R.; McMahon, Brian

Fundamental cosmology from ANDES precision spectroscopy

Journal Article · Fri May 05 00:00:00 UTC 2023 · Monthly Notices of the Royal Astronomical Society · OSTI ID:1974254

Marques, C. M. J.; Martins, C. J. A. P.; Alves, C. S.

ANDES (ARGONNE NUCLEAR DATA EXPLORATION SOFTWARE) - Physics Tools

Software · Mon Aug 24 00:00:00 UTC 2020 · OSTI ID:code-74414

Santiago-Gonzalez, Daniel

Related Subjects

96 KNOWLEDGE MANAGEMENT AND PRESERVATION
Biological Science

Title: Raw diffraction data preservation and reuse: Overview, update on practicalities and metadata requirements

Citation Formats

References (32)

Cited By (19)

Similar Records

Related Subjects