An Adaptable Seismic Data Format
Abstract
We present ASDF, the Adaptable Seismic Data Format, a modern and practical data format for all branches of seismology and beyond. The growing volume of freely available data coupled with ever expanding computational power opens avenues to tackle larger and more complex problems. Current bottlenecks include inefficient resource usage and insufficient data organization. Properly scaling a problem requires the resolution of both these challenges, and existing data formats are no longer up to the task. ASDF stores any number of synthetic, processed or unaltered waveforms in a single file. A key improvement compared to existing formats is the inclusion of comprehensive meta information, such as event or station information, in the same file. Additionally, it is also usable for any non-waveform data, for example, cross-correlations, adjoint sources or receiver functions. Last but not least, full provenance information can be stored alongside each item of data, thereby enhancing reproducibility and accountability. Any data set in our proposed format is self-describing and can be readily exchanged with others, facilitating collaboration. The utilization of the HDF5 container format grants efficient and parallel I/O operations, integrated compression algorithms and check sums to guard against data corruption. To not reinvent the wheel and to buildmore »
- Authors:
-
- Ludwig Maximilian Univ., Munich (Germany). Dept. of Earth and Environmental Sciences
- Princeton Univ., NJ (United States). Dept. of Geosciences
- Univ. of Toronto, ON (Canada). Dept. of Physics
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Univ. of Nice Sophia Antipolis, Valbonne (France). Géoazur Lab.
- Princeton Univ., NJ (United States). Dept. of Geosciences, Program in Applied & Computational Mathematics
- Publication Date:
- Research Org.:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
- Sponsoring Org.:
- USDOE Office of Science (SC); National Science Foundation (NSF); Natural Sciences and Engineering Research Council of Canada (NSERC)
- OSTI Identifier:
- 1559753
- Grant/Contract Number:
- AC05-00OR22725; 1112906; 487237
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Geophysical Journal International
- Additional Journal Information:
- Journal Volume: 207; Journal Issue: 2; Journal ID: ISSN 0956-540X
- Publisher:
- Oxford University Press
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 58 GEOSCIENCES; Time-series analysis; Seismic tomography; Computational seismology; Wave propagation
Citation Formats
Krischer, Lion, Smith, James, Lei, Wenjie, Lefebvre, Matthieu, Ruan, Youyi, de Andrade, Elliott Sales, Podhorszki, Norbert, Bozdağ, Ebru, and Tromp, Jeroen. An Adaptable Seismic Data Format. United States: N. p., 2016.
Web. doi:10.1093/gji/ggw319.
Krischer, Lion, Smith, James, Lei, Wenjie, Lefebvre, Matthieu, Ruan, Youyi, de Andrade, Elliott Sales, Podhorszki, Norbert, Bozdağ, Ebru, & Tromp, Jeroen. An Adaptable Seismic Data Format. United States. https://doi.org/10.1093/gji/ggw319
Krischer, Lion, Smith, James, Lei, Wenjie, Lefebvre, Matthieu, Ruan, Youyi, de Andrade, Elliott Sales, Podhorszki, Norbert, Bozdağ, Ebru, and Tromp, Jeroen. Thu .
"An Adaptable Seismic Data Format". United States. https://doi.org/10.1093/gji/ggw319. https://www.osti.gov/servlets/purl/1559753.
@article{osti_1559753,
title = {An Adaptable Seismic Data Format},
author = {Krischer, Lion and Smith, James and Lei, Wenjie and Lefebvre, Matthieu and Ruan, Youyi and de Andrade, Elliott Sales and Podhorszki, Norbert and Bozdağ, Ebru and Tromp, Jeroen},
abstractNote = {We present ASDF, the Adaptable Seismic Data Format, a modern and practical data format for all branches of seismology and beyond. The growing volume of freely available data coupled with ever expanding computational power opens avenues to tackle larger and more complex problems. Current bottlenecks include inefficient resource usage and insufficient data organization. Properly scaling a problem requires the resolution of both these challenges, and existing data formats are no longer up to the task. ASDF stores any number of synthetic, processed or unaltered waveforms in a single file. A key improvement compared to existing formats is the inclusion of comprehensive meta information, such as event or station information, in the same file. Additionally, it is also usable for any non-waveform data, for example, cross-correlations, adjoint sources or receiver functions. Last but not least, full provenance information can be stored alongside each item of data, thereby enhancing reproducibility and accountability. Any data set in our proposed format is self-describing and can be readily exchanged with others, facilitating collaboration. The utilization of the HDF5 container format grants efficient and parallel I/O operations, integrated compression algorithms and check sums to guard against data corruption. To not reinvent the wheel and to build upon past developments, we use existing standards like QuakeML, StationXML, W3C PROV and HDF5 wherever feasible. Usability and tool support are crucial for any new format to gain acceptance. We developed mature C/Fortran and Python based APIs coupling ASDF to the widely used SPECFEM3D_GLOBE and ObsPy toolkits.},
doi = {10.1093/gji/ggw319},
journal = {Geophysical Journal International},
number = 2,
volume = 207,
place = {United States},
year = {Thu Aug 25 00:00:00 EDT 2016},
month = {Thu Aug 25 00:00:00 EDT 2016}
}
Web of Science
Works referenced in this record:
Big Data: Astronomical or Genomical?
journal, July 2015
- Stephens, Zachary D.; Lee, Skylar Y.; Faghri, Faraz
- PLOS Biology, Vol. 13, Issue 7
The adjoint method in seismology
journal, August 2006
- Fichtner, A.; Bunge, H. -P.; Igel, H.
- Physics of the Earth and Planetary Interiors, Vol. 157, Issue 1-2
Seismic tomography, adjoint methods, time reversal and banana-doughnut kernels: Seismic tomography, adjoint methods, time reversal and banana-doughnut kernels
journal, December 2004
- Tromp, Jeroen; Tape, Carl; Liu, Qinya
- Geophysical Journal International, Vol. 160, Issue 1
Spectral-element simulations of global seismic wave propagation-I. Validation
journal, May 2002
- Komatitsch, Dimitri; Tromp, Jeroen
- Geophysical Journal International, Vol. 149, Issue 2
Seismic tomography of the southern California crust based on spectral-element and adjoint methods
journal, January 2010
- Tape, Carl; Liu, Qinya; Maggi, Alessia
- Geophysical Journal International, Vol. 180, Issue 1
NetCDF: an interface for scientific data access
journal, July 1990
- Rew, R.; Davis, G.
- IEEE Computer Graphics and Applications, Vol. 10, Issue 4
ObsPy: A Python Toolbox for Seismology
journal, May 2010
- Beyreuther, M.; Barsch, R.; Krischer, L.
- Seismological Research Letters, Vol. 81, Issue 3
Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks: HELLO ADIOS
journal, August 2013
- Liu, Qing; Logan, Jeremy; Tian, Yuan
- Concurrency and Computation: Practice and Experience, Vol. 26, Issue 7
Ambient Noise Levels in the Continental United States
journal, August 2004
- McNamara, D. E.
- Bulletin of the Seismological Society of America, Vol. 94, Issue 4
Spectral-element simulations of global seismic wave propagation-II. Three-dimensional models, oceans, rotation and self-gravitation
journal, July 2002
- Komatitsch, Dimitri; Tromp, Jeroen
- Geophysical Journal International, Vol. 150, Issue 1
Routine Data Processing in Earthquake Seismology With Sample Data, Exercises and Software
book, January 2010
- Havskov, Jens; Ottemoller, Lars
ObsPy: a bridge for seismology into the scientific Python ecosystem
journal, January 2015
- Krischer, Lion; Megies, Tobias; Barsch, Robert
- Computational Science & Discovery, Vol. 8, Issue 1
Works referencing / citing this record:
ITalian ACcelerometric Archive (ITACA), version 3.1
collection, January 2020
- D'Amico, Maria Clara; Felicetta, Chiara; Russo, Emiliano
- Istituto Nazionale di Geofisica e Vulcanologia (INGV)
A Detailed Earthquake Catalog for the San Jacinto Fault‐Zone Region in Southern California
journal, July 2019
- White, Malcolm C. A.; Ben‐Zion, Yehuda; Vernon, Frank L.
- Journal of Geophysical Research: Solid Earth, Vol. 124, Issue 7
Seismic wavefield imaging of Earth’s interior across scales
journal, December 2019
- Tromp, Jeroen
- Nature Reviews Earth & Environment, Vol. 1, Issue 1
Influence of reservoir geology on seismic response during decameter-scale hydraulic stimulations in crystalline rock
journal, January 2020
- Villiger, Linus; Gischig, Valentin Samuel; Doetsch, Joseph
- Solid Earth, Vol. 11, Issue 2
Influence of reservoir geology on seismic response during decameter-scale hydraulic stimulations in crystalline rock
text, January 2020
- Villiger, Linus; Gischig, Valentin Samuel; Doetsch, Joseph
- RWTH Aachen University
ITalian ACcelerometric Archive (ITACA), version 3.2
collection, January 2022
- Russo, Emiliano; Felicetta, Chiara; D'Amico, Maria Clara
- Istituto Nazionale di Geofisica e Vulcanologia (INGV)
Influence of reservoir geology on seismic response during decameter-scale hydraulic stimulations in crystalline rock
text, January 2020
- Linus, Villiger,; S., Gischig, Valentin; Joseph, Doetsch,
- ETH Zurich