Deep learning to estimate permeability using geophysical data
Abstract
Time-lapse electrical resistivity tomography (ERT) is a popular geophysical method to estimate three-dimensional (3D) permeability fields from electrical potential difference measurements. Traditional inversion and data assimilation methods are used to ingest this ERT data into hydrogeophysical models to estimate permeability. Due to ill-posedness and the curse of dimensionality, existing inversion strategies provide poor estimates and low resolution of the 3D permeability field. Recent advances in deep learning provide us with powerful algorithms to overcome this challenge. This paper presents a deep learning (DL) framework to estimate the 3D subsurface permeability from time-lapse ERT data. To test the feasibility of the proposed framework, we train DL-enabled inverse models on simulation data. Each measurement in both synthetic and field data is standardized by removing the mean and scaling the time-series to unit variance. This pre-processing step is necessary to bring simulation data closer to field observations. Subsurface process models based on hydrogeophysics are used to generate this synthetic data. Training performed on limited simulation data resulted in the DL model over-fitting. An advanced data augmentation based on mixup is implemented to generate additional training samples to overcome this issue. This mixup technique creates weakly labeled (low-fidelity) samples from strongly labeled (high-fidelity) data.more »
- Authors:
-
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
- Virginia Polytechnic Inst. and State Univ. (Virginia Tech), Blacksburg, VA (United States)
- Publication Date:
- Research Org.:
- Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Biological and Environmental Research (BER). Earth and Environmental Systems Science Division; USDOE
- OSTI Identifier:
- 1883390
- Alternate Identifier(s):
- OSTI ID: 1960964
- Report Number(s):
- PNNL-SA-175440
Journal ID: ISSN 0309-1708
- Grant/Contract Number:
- AC05-76RL01830; AC02-05CH11231
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Advances in Water Resources
- Additional Journal Information:
- Journal Volume: 167; Journal ID: ISSN 0309-1708
- Publisher:
- Elsevier
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 58 GEOSCIENCES; Artificial neural networks; Ensemble methods; Deep learning; Hydrogeophysics; Electrical resistivity tomography; Permeability; Multi-physics; Dimensionality reduction; Deep neural networks; Scalable deep learning
Citation Formats
Mudunuru, Maruti K., Cromwell, Erol L. D., Wang, Hongsheng, and Chen, Xingyuan. Deep learning to estimate permeability using geophysical data. United States: N. p., 2022.
Web. doi:10.1016/j.advwatres.2022.104272.
Mudunuru, Maruti K., Cromwell, Erol L. D., Wang, Hongsheng, & Chen, Xingyuan. Deep learning to estimate permeability using geophysical data. United States. https://doi.org/10.1016/j.advwatres.2022.104272
Mudunuru, Maruti K., Cromwell, Erol L. D., Wang, Hongsheng, and Chen, Xingyuan. Fri .
"Deep learning to estimate permeability using geophysical data". United States. https://doi.org/10.1016/j.advwatres.2022.104272. https://www.osti.gov/servlets/purl/1883390.
@article{osti_1883390,
title = {Deep learning to estimate permeability using geophysical data},
author = {Mudunuru, Maruti K. and Cromwell, Erol L. D. and Wang, Hongsheng and Chen, Xingyuan},
abstractNote = {Time-lapse electrical resistivity tomography (ERT) is a popular geophysical method to estimate three-dimensional (3D) permeability fields from electrical potential difference measurements. Traditional inversion and data assimilation methods are used to ingest this ERT data into hydrogeophysical models to estimate permeability. Due to ill-posedness and the curse of dimensionality, existing inversion strategies provide poor estimates and low resolution of the 3D permeability field. Recent advances in deep learning provide us with powerful algorithms to overcome this challenge. This paper presents a deep learning (DL) framework to estimate the 3D subsurface permeability from time-lapse ERT data. To test the feasibility of the proposed framework, we train DL-enabled inverse models on simulation data. Each measurement in both synthetic and field data is standardized by removing the mean and scaling the time-series to unit variance. This pre-processing step is necessary to bring simulation data closer to field observations. Subsurface process models based on hydrogeophysics are used to generate this synthetic data. Training performed on limited simulation data resulted in the DL model over-fitting. An advanced data augmentation based on mixup is implemented to generate additional training samples to overcome this issue. This mixup technique creates weakly labeled (low-fidelity) samples from strongly labeled (high-fidelity) data. The weakly labeled training data is then used to develop DL-enabled inverse models and reduce over-fitting. As both time-lapse ERT (1133048 features/realization) and 3D permeability (585453 features/realization) data samples are from a high-dimensional space, principal component analysis (PCA) is employed to reduce dimensionality. Encoded ERT and encoded permeability are generated using the trained PCA estimators. A deep neural network is then trained to map the encoded ERT to encoded permeability. This mixup training and unsupervised learning allowed us to build a fast and reasonably accurate DL-based inverse model under limited simulation data. Results show that proposed weak supervised learning can capture salient spatial features in the 3D permeability field. Quantitatively, the average mean squared error (in terms of the natural log) on the strongly labeled training, validation, and test datasets is less than 0.5. The R2-score (global metric) is greater than 0.75, and the percent error in each cell (local metric) is less than 10%. Finally, an added benefit in terms of computational cost is that the proposed DL-based inverse model is at least O(104) times faster than running a forward model once it is trained. Data generation, DL model training, and hyperparameter tuning to identify optimal neural network architectures utilized high-performance computing resources while the DL inference is performed on a standard laptop. Approximately, O(105) processor hours are used for generating data and DL tuning and training. We acknowledge that the data generation and DL model development are expensive. But once a DL model is trained, it can be re-used for inversion rapidly for the given system, with set physics and domain. Note that traditional inversion may require multiple forward model simulations (e.g., in the order of 10 to 1000), which are very expensive. This computational savings ≈ O(105) – O(107)) makes the proposed DL-based inverse model attractive for subsurface imaging and real-time ERT monitoring applications due to fast and yet reasonably accurate estimations of permeability field.},
doi = {10.1016/j.advwatres.2022.104272},
journal = {Advances in Water Resources},
number = ,
volume = 167,
place = {United States},
year = {Fri Jul 15 00:00:00 EDT 2022},
month = {Fri Jul 15 00:00:00 EDT 2022}
}
Works referenced in this record:
Advancing process-based watershed hydrological research using near-surface geophysics: a vision for, and review of, electrical and magnetic geophysical methods
journal, August 2008
- Robinson, D. A.; Binley, A.; Crook, N.
- Hydrological Processes, Vol. 22, Issue 18
Discovering State‐Parameter Mappings in Subsurface Models Using Generative Adversarial Networks
journal, October 2018
- Sun, Alexander Y.
- Geophysical Research Letters, Vol. 45, Issue 20
Neural Style Transfer: A Review
journal, November 2020
- Jing, Yongcheng; Yang, Yezhou; Feng, Zunlei
- IEEE Transactions on Visualization and Computer Graphics, Vol. 26, Issue 11
Advances in interpretation of subsurface processes with time-lapse electrical imaging
journal, August 2014
- Singha, K.; Day-Lewis, F. D.; Johnson, T.
- Hydrological Processes, Vol. 29, Issue 6, p. 1549-1576
Application of ensemble-based data assimilation techniques for aquifer characterization using tracer data at Hanford 300 area: Tracer Data Assimilation at Hanford 300 Area
journal, October 2013
- Chen, Xingyuan; Hammond, Glenn E.; Murray, Chris J.
- Water Resources Research, Vol. 49, Issue 10
Evaluating the performance of parallel subsurface simulators: An illustrative example with PFLOTRAN: Evaluating the Parallel Performance of Pflotran
journal, January 2014
- Hammond, G. E.; Lichtner, P. C.; Mills, R. T.
- Water Resources Research, Vol. 50, Issue 1
Four-dimensional electrical conductivity monitoring of stage-driven river water intrusion: Accounting for water table effects using a transient mesh boundary and conditional inversion constraints: 4-D RIVER WATER INTRUSION IMAGING
journal, August 2015
- Johnson, Tim; Versteeg, Roelof; Thomle, Jon
- Water Resources Research, Vol. 51, Issue 8
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
journal, February 2019
- Raissi, M.; Perdikaris, P.; Karniadakis, G. E.
- Journal of Computational Physics, Vol. 378
Hydro-, bio-geophysics
journal, July 2004
- al Hagrey, Said A.; Meissner, Rolf; Werban, Ulrike
- The Leading Edge, Vol. 23, Issue 7
Biogeophysics: A new frontier in Earth science research
journal, January 2009
- Atekwana, Estella A.; Slater, Lee D.
- Reviews of Geophysics, Vol. 47, Issue 4
Machine learning to discover mineral trapping signatures due to CO2 injection
journal, July 2021
- Ahmmed, Bulbul; Karra, Satish; Vesselinov, Velimir V.
- International Journal of Greenhouse Gas Control, Vol. 109
Less is more: Sampling chemical space with active learning
journal, June 2018
- Smith, Justin S.; Nebgen, Ben; Lubbers, Nicholas
- The Journal of Chemical Physics, Vol. 148, Issue 24
Estimating Watershed Subsurface Permeability From Stream Discharge Data Using Deep Neural Networks
journal, February 2021
- Cromwell, Erol; Shuai, Pin; Jiang, Peishi
- Frontiers in Earth Science, Vol. 9
PFLOTRAN-SIP: A PFLOTRAN Module for Simulating Spectral-Induced Polarization of Electrical Impedance Data
journal, December 2020
- Ahmmed, Bulbul; Mudunuru, Maruti Kumar; Karra, Satish
- Energies, Vol. 13, Issue 24
Near Surface Electrical Characterization of Hydraulic Conductivity: From Petrophysical Properties to Aquifer Geometries—A Review
journal, July 2007
- Slater, Lee
- Surveys in Geophysics, Vol. 28, Issue 2-3
Deep transfer learning and data augmentation improve glucose levels prediction in type 2 diabetes patients
journal, July 2021
- Deng, Yixiang; Lu, Lu; Aponte, Laura
- npj Digital Medicine, Vol. 4, Issue 1
Trends, prospects and challenges in quantifying flow and transport through fractured rocks
journal, February 2005
- Neuman, Shlomo P.
- Hydrogeology Journal, Vol. 13, Issue 1
Borehole and surface-based hydrogeophysics
journal, February 2005
- Gu�rin, Roger
- Hydrogeology Journal, Vol. 13, Issue 1
A comparative simulation study of coupled THM processes and their effect on fractured rock permeability around nuclear waste repositories
journal, October 2008
- Rutqvist, Jonny; Barr, Deborah; Birkholzer, Jens T.
- Environmental Geology, Vol. 57, Issue 6
Characterizing flow and transport in fractured geological media: A review
journal, August 2002
- Berkowitz, Brian
- Advances in Water Resources, Vol. 25, Issue 8-12
The Microbial Community Structure in Petroleum-Contaminated Sediments Corresponds to Geophysical Signatures
journal, May 2007
- Allen, Jonathan P.; Atekwana, Estella A.; Atekwana, Eliot A.
- Applied and Environmental Microbiology, Vol. 73, Issue 9
DART-PFLOTRAN: An ensemble-based data assimilation system for estimating subsurface flow and transport model parameters
journal, August 2021
- Jiang, Peishi; Chen, Xingyuan; Chen, Kewei
- Environmental Modelling & Software, Vol. 142
Physically based regularization of hydrogeophysical inverse problems for improved imaging of process-driven systems: POD-BASED imaging of process-driven systems
journal, October 2013
- Oware, E. K.; Moysey, S. M. J.; Khan, T.
- Water Resources Research, Vol. 49, Issue 10
Multiscale geophysical imaging of the critical zone: Geophysical Imaging of the Critical Zone
journal, January 2015
- Parsekian, A. D.; Singha, K.; Minsley, B. J.
- Reviews of Geophysics, Vol. 53, Issue 1
Machine learning to identify geologic factors associated with production in geothermal fields: a case-study using 3D geologic data, Brady geothermal field, Nevada
journal, June 2021
- Siler, Drew L.; Pepin, Jeff D.; Vesselinov, Velimir V.
- Geothermal Energy, Vol. 9, Issue 1
The emergence of hydrogeophysics for improved understanding of subsurface processes over multiple scales: The Emergence of Hydrogeophysics
journal, June 2015
- Binley, Andrew; Hubbard, Susan S.; Huisman, Johan A.
- Water Resources Research, Vol. 51, Issue 6
Improved hydrogeophysical characterization and monitoring through parallel modeling and inversion of time-domain resistivity andinduced-polarization data
journal, July 2010
- Johnson, Timothy C.; Versteeg, Roelof J.; Ward, Andy
- Geophysics, Vol. 75, Issue 4, p. WA27-WA41
What is principal component analysis?
journal, March 2008
- Ringnér, Markus
- Nature Biotechnology, Vol. 26, Issue 3
Understanding Mixup Training Methods
journal, January 2018
- Liang, Daojun; Yang, Feng; Zhang, Tian
- IEEE Access, Vol. 6
How can Big Data and machine learning benefit environment and water management: a survey of methods, applications, and future directions
journal, July 2019
- Sun, Alexander Y.; Scanlon, Bridget R.
- Environmental Research Letters, Vol. 14, Issue 7
Neural network modeling of in situ fluid-filled pore size distributions in subsurface shale reservoirs under data constraints
journal, March 2019
- Li, Hao; Misra, Siddharth; He, Jiabo
- Neural Computing and Applications, Vol. 32, Issue 8
PFLOTRAN User Manual: A Massively Parallel Reactive Flow and Transport Model for Describing Surface and Subsurface Processes
report, January 2015
- Lichtner, Peter; Hammond, Glenn; Lu, Chuan
Comparison of invasive and non-invasive electromagnetic methods in soil water content estimation of a dike model
journal, April 2009
- Preko, Kwasi; Scheuermann, Alexander; Wilhelm, Helmut
- Journal of Geophysics and Engineering, Vol. 6, Issue 2
A Survey on Deep Transfer Learning
book, January 2018
- Tan, Chuanqi; Sun, Fuchun; Kong, Tao
- Artificial Neural Networks and Machine Learning – ICANN 2018
mpi4py: Status Update After 12 Years of Development
journal, July 2021
- Dalcin, Lisandro; Fang, Yao-Lung L.
- Computing in Science & Engineering, Vol. 23, Issue 4
Shale gas and non-aqueous fracturing fluids: Opportunities and challenges for supercritical CO2
journal, June 2015
- Middleton, Richard S.; Carey, J. William; Currier, Robert P.
- Applied Energy, Vol. 147
Geophysical Signatures of Microbial Activity at Hydrocarbon Contaminated Sites: A Review
journal, November 2009
- Atekwana, Estella A.; Atekwana, Eliot A.
- Surveys in Geophysics, Vol. 31, Issue 2
A Bayesian approach for inverse modeling, data assimilation, and conditional simulation of spatial random fields
journal, October 2010
- Rubin, Yoram; Chen, Xingyuan; Murakami, Haruko
- Water Resources Research, Vol. 46, Issue 10
The Data Assimilation Research Testbed: A Community Facility
journal, September 2009
- Anderson, Jeffrey; Hoar, Tim; Raeder, Kevin
- Bulletin of the American Meteorological Society, Vol. 90, Issue 9
PFLOTRAN: Reactive Flow & Transport Code for Use on Laptops to Leadership-Class Supercomputers
book, March 2012
- E. Hammond, G.; C. Lichtner, P.; Lu, C.
- Groundwater Reactive Transport Models
Understanding hydraulic fracturing: a multi-scale problem
journal, October 2016
- Hyman, J. D.; Jiménez-Martínez, J.; Viswanathan, H. S.
- Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 374, Issue 2078
Multiple-point geostatistics for modeling subsurface heterogeneity: A comprehensive review: REVIEW OF MULTIPLE POI
journal, November 2008
- Hu, L. Y.; Chugunova, T.
- Water Resources Research, Vol. 44, Issue 11
A System Model for Geologic Sequestration of Carbon Dioxide
journal, February 2009
- Stauffer, Philip H.; Viswanathan, Hari S.; Pawar, Rajesh J.
- Environmental Science & Technology, Vol. 43, Issue 3
Machine learning in geo- and environmental sciences: From small to large scale
journal, August 2020
- Tahmasebi, Pejman; Kamrava, Serveh; Bai, Tao
- Advances in Water Resources, Vol. 142
Principal component analysis
journal, January 2014
- Bro, Rasmus; Smilde, Age K.
- Anal. Methods, Vol. 6, Issue 9
PFLOTRAN-E4D: A parallel open source PFLOTRAN module for simulating time-lapse electrical resistivity data
journal, February 2017
- Johnson, Timothy C.; Hammond, Glenn E.; Chen, Xingyuan
- Computers & Geosciences, Vol. 99
Machine learning for hydrologic sciences: An introductory overview
journal, May 2021
- Xu, Tianfang; Liang, Feng
- WIREs Water, Vol. 8, Issue 5
The cross-scale science of CO2 capture and storage: from pore scale to regional scale
journal, January 2012
- Middleton, Richard S.; Keating, Gordon N.; Stauffer, Philip H.
- Energy & Environmental Science, Vol. 5, Issue 6
Sequential geophysical and flow inversion to characterize fracture networks in subsurface systems: MUDUNURU
journal, September 2017
- Mudunuru, Maruti Kumar; Karra, Satish; Makedonska, Nataliia
- Statistical Analysis and Data Mining: The ASA Data Science Journal, Vol. 10, Issue 5
Unsupervised machine learning based on non-negative tensor factorization for analyzing reactive-mixing
journal, October 2019
- Vesselinov, V. V.; Mudunuru, M. K.; Karra, S.
- Journal of Computational Physics, Vol. 395
3-D Geologic Controls of Hydrothermal Fluid Flow at Brady geothermal field, Nevada, USA
journal, July 2021
- Siler, Drew L.; Pepin, Jeff D.
- Geothermics, Vol. 94
Regression-based reduced-order models to predict transient thermal output for enhanced geothermal systems
journal, November 2017
- Mudunuru, M. K.; Karra, S.; Harp, D. R.
- Geothermics, Vol. 70
A survey on Image Data Augmentation for Deep Learning
journal, July 2019
- Shorten, Connor; Khoshgoftaar, Taghi M.
- Journal of Big Data, Vol. 6, Issue 1