skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: ARM Data File Standards Version 1.2

Abstract

The U.S. Department of Energy (DOE)’s Atmospheric Radiation Measurement (ARM) Climate Research Facility performs routine in situ and remote-sensing observations to provide a detailed and accurate description of the Earth atmosphere in diverse climate regimes. The result is a huge archive of diverse data sets containing observational and derived data, currently accumulating at a rate of 30 terabytes (TB) of data and 150,000 different files per month (http://www.archive.arm.gov/stats/). Continuing the current processing while scaling this to even larger sizes is extremely important to the ARM Facility and requires consistent metadata and data standards. The standards described in this document will enable development of automated analysis and discovery tools for the ever growing data volumes. It will enable consistent analysis of the multiyear data, allow for development of automated monitoring and data health status tools, and allow future capabilities of delivering data on demand that can be tailored explicitly for the user needs. This analysis ability will only be possible if the data follows a minimum set of standards. This document proposes a hierarchy of required and recommended standards.

Authors:
 [1]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Publication Date:
Research Org.:
DOE Office of Science Atmospheric Radiation Measurement (ARM) Program (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Biological and Environmental Research (BER) (SC-23)
OSTI Identifier:
1253898
Report Number(s):
DOE/SC-ARM-14-010
DOE Contract Number:  
AC05-7601830
Resource Type:
Technical Report
Country of Publication:
United States
Language:
English
Subject:
54 ENVIRONMENTAL SCIENCES; file formats; datastreams; data products; value-added products; standards; file naming conventions; metadata; field names; global attributes; quality control; exceptions

Citation Formats

Palanisamy, Giri. ARM Data File Standards Version 1.2. United States: N. p., 2016. Web. doi:10.2172/1253898.
Palanisamy, Giri. ARM Data File Standards Version 1.2. United States. doi:10.2172/1253898.
Palanisamy, Giri. Sun . "ARM Data File Standards Version 1.2". United States. doi:10.2172/1253898. https://www.osti.gov/servlets/purl/1253898.
@article{osti_1253898,
title = {ARM Data File Standards Version 1.2},
author = {Palanisamy, Giri},
abstractNote = {The U.S. Department of Energy (DOE)’s Atmospheric Radiation Measurement (ARM) Climate Research Facility performs routine in situ and remote-sensing observations to provide a detailed and accurate description of the Earth atmosphere in diverse climate regimes. The result is a huge archive of diverse data sets containing observational and derived data, currently accumulating at a rate of 30 terabytes (TB) of data and 150,000 different files per month (http://www.archive.arm.gov/stats/). Continuing the current processing while scaling this to even larger sizes is extremely important to the ARM Facility and requires consistent metadata and data standards. The standards described in this document will enable development of automated analysis and discovery tools for the ever growing data volumes. It will enable consistent analysis of the multiyear data, allow for development of automated monitoring and data health status tools, and allow future capabilities of delivering data on demand that can be tailored explicitly for the user needs. This analysis ability will only be possible if the data follows a minimum set of standards. This document proposes a hierarchy of required and recommended standards.},
doi = {10.2172/1253898},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2016},
month = {5}
}