skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Enabling modern data discovery for atmospheric measurements

Abstract

The Atmospheric Radiation Measurement (ARM) user facility is a US Department of Energy Office of Science user facility that is managed and operated through a collaborative effort led by nine US Department of Energy national laboratories. The ARM Data Center, located at Oak Ridge National Laboratory, is responsible for the timely collection, processing, and delivery of data products to the scientific community. The ARM Data Center holds more than 11,000 data products, including metadata collected from field campaigns, instruments, value-added products, and principal investigator–contributed data. These data sets are checked for successful transfer (for most data, this transfer is carried out automatically via the network; however, some of the largest data sets and some of the most remote sites require manual shipping of hard disks) and both the data and metadata are processed to a standard format, which is an ARM-standardized structure, via the Network Common Data Form. The Network Common Data Form is a self-describing binary format with many compatible software tools. Once processed, the data are cataloged, stored in the ARM Data Archive, and made discoverable through association with an array of metadata-characterizing information, such as location and measurement classification. These metadata enable powerful search capabilities through themore » ARM Data Center Data Discovery interface. This paper discusses the workflow of how the new discovery system has been redesigned from user requirements and how the data are distributed to the scientific community.« less

Authors:
ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1];  [1]; ORCiD logo [1]; ORCiD logo [1]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Environmental Sciences Division
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Biological and Environmental Research (BER)
OSTI Identifier:
1807242
Grant/Contract Number:  
AC05-00OR22725
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Earth Science Informatics
Additional Journal Information:
Journal Volume: 14; Journal Issue: 3; Journal ID: ISSN 1865-0473
Publisher:
Springer
Country of Publication:
United States
Language:
English
Subject:
54 ENVIRONMENTAL SCIENCES; 97 MATHEMATICS AND COMPUTING; ARM data center; Metadata; Data archive; FAIR data; Metadata management; Data search

Citation Formats

Guntupally, Kavya, Dumas, Kyle, Prakash, Giri, Devarakonda, Ranjeet, Darnell, Wade, Davis, Maggie, and Cederwall, Richard. Enabling modern data discovery for atmospheric measurements. United States: N. p., 2021. Web. doi:10.1007/s12145-021-00635-0.
Guntupally, Kavya, Dumas, Kyle, Prakash, Giri, Devarakonda, Ranjeet, Darnell, Wade, Davis, Maggie, & Cederwall, Richard. Enabling modern data discovery for atmospheric measurements. United States. https://doi.org/10.1007/s12145-021-00635-0
Guntupally, Kavya, Dumas, Kyle, Prakash, Giri, Devarakonda, Ranjeet, Darnell, Wade, Davis, Maggie, and Cederwall, Richard. 2021. "Enabling modern data discovery for atmospheric measurements". United States. https://doi.org/10.1007/s12145-021-00635-0. https://www.osti.gov/servlets/purl/1807242.
@article{osti_1807242,
title = {Enabling modern data discovery for atmospheric measurements},
author = {Guntupally, Kavya and Dumas, Kyle and Prakash, Giri and Devarakonda, Ranjeet and Darnell, Wade and Davis, Maggie and Cederwall, Richard},
abstractNote = {The Atmospheric Radiation Measurement (ARM) user facility is a US Department of Energy Office of Science user facility that is managed and operated through a collaborative effort led by nine US Department of Energy national laboratories. The ARM Data Center, located at Oak Ridge National Laboratory, is responsible for the timely collection, processing, and delivery of data products to the scientific community. The ARM Data Center holds more than 11,000 data products, including metadata collected from field campaigns, instruments, value-added products, and principal investigator–contributed data. These data sets are checked for successful transfer (for most data, this transfer is carried out automatically via the network; however, some of the largest data sets and some of the most remote sites require manual shipping of hard disks) and both the data and metadata are processed to a standard format, which is an ARM-standardized structure, via the Network Common Data Form. The Network Common Data Form is a self-describing binary format with many compatible software tools. Once processed, the data are cataloged, stored in the ARM Data Archive, and made discoverable through association with an array of metadata-characterizing information, such as location and measurement classification. These metadata enable powerful search capabilities through the ARM Data Center Data Discovery interface. This paper discusses the workflow of how the new discovery system has been redesigned from user requirements and how the data are distributed to the scientific community.},
doi = {10.1007/s12145-021-00635-0},
url = {https://www.osti.gov/biblio/1807242}, journal = {Earth Science Informatics},
issn = {1865-0473},
number = 3,
volume = 14,
place = {United States},
year = {2021},
month = {6}
}

Works referenced in this record:

Big Federal Data Centers Implementing FAIR Data Principles: ARM Data Center Example
conference, December 2019


Spring Boot based REST API to Improve Data Quality Report Generation for Big Scientific Data: ARM Data Center Example
conference, December 2018


HPC infrastructure to support the next-generation ARM facility data operations
conference, December 2016


Automated Indexing of Structured Scientific Metadata Using Apache Solr
conference, December 2020