Enabling modern data discovery for atmospheric measurements
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Environmental Sciences Division
The Atmospheric Radiation Measurement (ARM) user facility is a US Department of Energy Office of Science user facility that is managed and operated through a collaborative effort led by nine US Department of Energy national laboratories. The ARM Data Center, located at Oak Ridge National Laboratory, is responsible for the timely collection, processing, and delivery of data products to the scientific community. The ARM Data Center holds more than 11,000 data products, including metadata collected from field campaigns, instruments, value-added products, and principal investigator–contributed data. These data sets are checked for successful transfer (for most data, this transfer is carried out automatically via the network; however, some of the largest data sets and some of the most remote sites require manual shipping of hard disks) and both the data and metadata are processed to a standard format, which is an ARM-standardized structure, via the Network Common Data Form. The Network Common Data Form is a self-describing binary format with many compatible software tools. Once processed, the data are cataloged, stored in the ARM Data Archive, and made discoverable through association with an array of metadata-characterizing information, such as location and measurement classification. These metadata enable powerful search capabilities through the ARM Data Center Data Discovery interface. This paper discusses the workflow of how the new discovery system has been redesigned from user requirements and how the data are distributed to the scientific community.
- Research Organization:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC), Biological and Environmental Research (BER)
- Grant/Contract Number:
- AC05-00OR22725
- OSTI ID:
- 1807242
- Journal Information:
- Earth Science Informatics, Journal Name: Earth Science Informatics Journal Issue: 3 Vol. 14; ISSN 1865-0473
- Publisher:
- SpringerCopyright Statement
- Country of Publication:
- United States
- Language:
- English
HPC infrastructure to support the next-generation ARM facility data operations
|
conference | December 2016 |
Spring Boot based REST API to Improve Data Quality Report Generation for Big Scientific Data: ARM Data Center Example
|
conference | December 2018 |
Big Federal Data Centers Implementing FAIR Data Principles: ARM Data Center Example
|
conference | December 2019 |
Automated Indexing of Structured Scientific Metadata Using Apache Solr
|
conference | December 2020 |
Similar Records
ACRF data collection and processing infrastructure.
Building a Global Federation System for Climate Change Research: The Earth System Grid Center for Enabling Technologies (ESG-CET)