Title: Evaluating the Potential and Challenges of an Uncertainty Quantification Method for Long Short–Term Memory Models for Soil Moisture Predictions

Abstract

Abstract Recently, recurrent deep networks have shown promise to harness newly available satellite‐sensed data for long‐term soil moisture projections. However, to be useful in forecasting, deep networks must also provide uncertainty estimates. Here we evaluated Monte Carlo dropout with an input‐dependent data noise term (MCD+N), an efficient uncertainty estimation framework originally developed in computer vision, for hydrologic time series predictions. MCD+N simultaneously estimates a heteroscedastic input‐dependent data noise term (a trained error model attributable to observational noise) and a network weight uncertainty term (attributable to insufficiently constrained model parameters). Although MCD+N has appealing features, many heuristic approximations were employed during its derivation, and rigorous evaluations and evidence of its asserted capability to detect dissimilarity were lacking. To address this, we provided an in‐depth evaluation of the scheme's potential and limitations. We showed that for reproducing soil moisture dynamics recorded by the Soil Moisture Active Passive (SMAP) mission, MCD+N indeed gave a good estimate of predictive error, provided that we tuned a hyperparameter and used a representative training data set. The input‐dependent term responded strongly to observational noise, while the model term clearly acted as a detector for physiographic dissimilarity from the training data, behaving as intended. However, when the trainingmore » and test data were characteristically different, the input‐dependent term could be misled, undermining its reliability. Additionally, due to the data‐driven nature of the model, data noise also influences network weight uncertainty, and therefore the two uncertainty terms are correlated. Overall, this approach has promise, but care is needed to interpret the results.« less

Authors:

Fang, Kuai  ^[1];

• Search DOE PAGES for author "Fang, Kuai"
• Search DOE PAGES for ORCID "0000-0001-5246-5131"

Kifer, Daniel ^[2]; Lawson, Kathryn  ^[2];

• Search DOE PAGES for author "Lawson, Kathryn"
• Search DOE PAGES for ORCID "0000-0003-0075-7911"

Shen, Chaopeng  ^[2]

• Search DOE PAGES for author "Shen, Chaopeng"
• Search DOE PAGES for ORCID "0000-0002-0685-1901"

---

+ Show Author Affiliations

1. Pennsylvania State Univ., University Park, PA (United States); Stanford Univ., CA (United States)
2. Pennsylvania State Univ., University Park, PA (United States)

Publication Date:

Mon Nov 09 00:00:00 EST 2020

Research Org.:

Pennsylvania State Univ., University Park, PA (United States)

Sponsoring Org.:

USDOE Advanced Research Projects Agency - Energy (ARPA-E); National Science Foundation (NSF)

OSTI Identifier:

1755316

Alternate Identifier(s):

OSTI ID: 1786917

Grant/Contract Number:  

SC0016605; EAR #1832294

Resource Type:

Accepted Manuscript

Journal Name:

Water Resources Research

Additional Journal Information:

Journal Volume: 56; Journal Issue: 12; Journal ID: ISSN 0043-1397

Publisher:

American Geophysical Union (AGU)

Country of Publication:

United States

Language:

English

Subject:

54 ENVIRONMENTAL SCIENCES; Soil moisture; Monte Carlo dropout; LSTM; uncertainty; deep learning; Bayesian inference