Strictly Enforcing Invertibility and Conservation in CNN-Based Super Resolution for Scientific Datasets
- a Pacific Northwest National Laboratory, Richland, Washington
- a Pacific Northwest National Laboratory, Richland, Washington, b ClimateAI, Inc., San Francisco, California
Recently, deep convolutional neural networks (CNNs) have revolutionized image “super resolution” (SR), dramatically outperforming past methods for enhancing image resolution. They could be a boon for the many scientific fields that involve imaging or any regularly gridded datasets: satellite remote sensing, radar meteorology, medical imaging, numerical modeling, and so on. Unfortunately, while SR-CNNs produce visually compelling results, they do not necessarily conserve physical quantities between their low-resolution inputs and high-resolution outputs when applied to scientific datasets. Here, a method for “downsampling enforcement” in SR-CNNs is proposed. A differentiable operator is derived that, when applied as the final transfer function of a CNN, ensures the high-resolution outputs exactly reproduce the low-resolution inputs under 2D-average downsampling while improving performance of the SR schemes. The method is demonstrated across seven modern CNN-based SR schemes on several benchmark image datasets, and applications to weather radar, satellite imager, and climate model data are shown. The approach improves training time and performance while ensuring physical consistency between the super-resolved and low-resolution data.
Significance StatementRecent advancements in using deep learning to increase the resolution of images have substantial potential across the many scientific fields that use images and image-like data. Most image super-resolution research has focused on the visual quality of outputs, however, and is not necessarily well suited for use with scientific data where known physics constraints may need to be enforced. Here, we introduce a method to modify existing deep neural network architectures so that they strictly conserve physical quantities in the input field when “super resolving” scientific data and find that the method can improve performance across a wide range of datasets and neural networks. Integration of known physics and adherence to established physical constraints into deep neural networks will be a critical step before their potential can be fully realized in the physical sciences.
- Research Organization:
- Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
- Sponsoring Organization:
- USDOE
- Grant/Contract Number:
- AC05-76RL01830
- OSTI ID:
- 1958686
- Alternate ID(s):
- OSTI ID: 1962738
- Report Number(s):
- PNNL-SA-157590; e210012
- Journal Information:
- Artificial Intelligence for the Earth Systems, Journal Name: Artificial Intelligence for the Earth Systems Journal Issue: 1 Vol. 2; ISSN 2769-7525
- Publisher:
- American Meteorological SocietyCopyright Statement
- Country of Publication:
- United States
- Language:
- English