Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles

Journal Article · Fri Aug 04 00:00:00 EDT 2023 · Physica. D, Nonlinear Phenomena

DOI:https://doi.org/10.1016/j.physd.2023.133852· OSTI ID:2584756

Maulik, Romit ^[1]; Egele, Romain ^[2]; Raghavan, Krishnan ^[3]; Balaprakash, Prasanna ^[4]

---

1. Pennsylvania State Univ., University Park, PA (United States)
2. Université Paris-Saclay, Gif-sur-Yvette (France)
3. Argonne National Laboratory (ANL), Argonne, IL (United States)
4. Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DNN) based algorithmic approaches. However, most DNN models do not provide uncertainty estimates, which are crucial for establishing the trustworthiness of these techniques in downstream decision making tasks and scenarios. In recent years, ensemble-based methods have achieved significant success for the uncertainty quantification in DNNs on a number of benchmark problems. However, their performance on real-world applications remains under-explored. In this work, we present an automated approach to DNN discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification. Specifically, we propose the use of a scalable neural and hyperparameter architecture search for discovering an ensemble of DNN models for complex dynamical systems. We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly. This is achieved by using genetic algorithms and Bayesian optimization for sampling the search space of neural network architectures and hyperparameters. Subsequently, a model selection approach is used to identify candidate models for an ensemble set construction. Afterwards, a variance decomposition approach is used to estimate the uncertainty of the predictions from the ensemble. We demonstrate the feasibility of this framework for two tasks — forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature. In conclusion, we demonstrate superior performance from the ensemble in contrast with individual high-performing models and other benchmarks.

• Research Organization: Argonne National Laboratory (ANL), Argonne, IL (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities (SUF)
• Grant/Contract Number: AC02-06CH11357
• OSTI ID: 2584756
• Alternate ID(s): OSTI ID: 1995927
• Journal Information: Physica. D, Nonlinear Phenomena, Journal Name: Physica. D, Nonlinear Phenomena Vol. 454; ISSN 0167-2789
• Publisher: Elsevier
• Country of Publication: United States
• Language: English

Related Subjects

Deep ensembles
Neural architecture and hyperparameter search
Scientific machine learning