U.S. Department of EnergyOffice of Scientific and Technical Information
Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions
Abstract
The National Ecological Observatory Network (NEON) is a continental-scale observatory with sites across the US collecting standardized ecological observations that will operate for multiple decades. To maximize the utility of NEON data, we envision edge computing systems that gather, calibrate, aggregate, and ingest measurements in an integrated fashion. Edge systems will employ machine learning methods to cross-calibrate, gap-fill and provision data in near-real time to the NEON Data Portal and to High Performance Computing (HPC) systems, running ensembles of Earth system models (ESMs) that assimilate the data. For the first time gridded EC data products and response functions promise to offset pervasive observational biases through evaluating, benchmarking, optimizing parameters, and training new machine learning parameterizations within ESMs all at the same model-grid scale. Leveraging open-source software for EC data analysis, we are already building software infrastructure for integration of near-real time data streams into the International Land Model Benchmarking (ILAMB) package for use by the wider research community. We will present a perspective on the design and integration of end-to-end infrastructure for data acquisition, edge computing, HPC simulation, analysis, and validation, where Artificial Intelligence (AI) approaches are used throughout the distributed workflow to improve accuracy and computational performance.
- Authors:
-
[3];
[3];
[3];
[3]
- National Ecological Observatory Network Program
- Lawrence Berkeley National Laboratory (LBNL)
- ORNL
- Pennsylvania State University
- University of Wisconsin Madison
- National Center for Atmospheric Research (NCAR)
- Publication Date:
- Research Org.:
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Biological and Environmental Research (BER)
- OSTI Identifier:
- 1819607
- DOE Contract Number:
- AC05-00OR22725
- Resource Type:
- Conference
- Resource Relation:
- Conference: 2020 Smoky Mountains Computational Sciences and Engineering Conference (SMC2020) - Kingsport, Tennessee, United States of America - 8/26/2020 8:00:00 AM-8/28/2020 8:00:00 AM
- Country of Publication:
- United States
- Language:
- English
Citation Formats
Durden, David J., Metzger, Stefan, Chu, Housen, Collier, Nathaniel, Davis, Kenneth J., Desai, Ankur R., Kumar, Jitendra, Wieder, William R., Xu, Min, and Hoffman, Forrest. Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions. United States: N. p., 2020.
Web.
Durden, David J., Metzger, Stefan, Chu, Housen, Collier, Nathaniel, Davis, Kenneth J., Desai, Ankur R., Kumar, Jitendra, Wieder, William R., Xu, Min, & Hoffman, Forrest. Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions. United States.
Durden, David J., Metzger, Stefan, Chu, Housen, Collier, Nathaniel, Davis, Kenneth J., Desai, Ankur R., Kumar, Jitendra, Wieder, William R., Xu, Min, and Hoffman, Forrest. 2020.
"Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions". United States. https://www.osti.gov/servlets/purl/1819607.
@article{osti_1819607,
title = {Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions},
author = {Durden, David J. and Metzger, Stefan and Chu, Housen and Collier, Nathaniel and Davis, Kenneth J. and Desai, Ankur R. and Kumar, Jitendra and Wieder, William R. and Xu, Min and Hoffman, Forrest},
abstractNote = {The National Ecological Observatory Network (NEON) is a continental-scale observatory with sites across the US collecting standardized ecological observations that will operate for multiple decades. To maximize the utility of NEON data, we envision edge computing systems that gather, calibrate, aggregate, and ingest measurements in an integrated fashion. Edge systems will employ machine learning methods to cross-calibrate, gap-fill and provision data in near-real time to the NEON Data Portal and to High Performance Computing (HPC) systems, running ensembles of Earth system models (ESMs) that assimilate the data. For the first time gridded EC data products and response functions promise to offset pervasive observational biases through evaluating, benchmarking, optimizing parameters, and training new machine learning parameterizations within ESMs all at the same model-grid scale. Leveraging open-source software for EC data analysis, we are already building software infrastructure for integration of near-real time data streams into the International Land Model Benchmarking (ILAMB) package for use by the wider research community. We will present a perspective on the design and integration of end-to-end infrastructure for data acquisition, edge computing, HPC simulation, analysis, and validation, where Artificial Intelligence (AI) approaches are used throughout the distributed workflow to improve accuracy and computational performance.},
doi = {},
url = {https://www.osti.gov/biblio/1819607},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2020},
month = {12}
}