Boosting Barlow Twins Reduced Order Modeling for Machine Learning‐Based Surrogate Models in Multiphase Flow Problems

Kadeethum, T.; Silva, V. L. S.; Salinas, P.; Pain, C. C.; Yoon, H.

doi:10.1029/2023WR035778

Title: Boosting Barlow Twins Reduced Order Modeling for Machine Learning‐Based Surrogate Models in Multiphase Flow Problems

Journal Article · 17 October 2024 · Water Resources Research

DOI: https://doi.org/10.1029/2023WR035778 · OSTI ID:2475447

^[1]; Silva, V. L. S. ^[2]; Salinas, P. ^[3]; Pain, C. C. ^[2];

^[1]

Sandia National Laboratories Geoscience Research &, Applications Albuquerque NM USA
Imperial College London London UK
OpenGoSim Leicester UK

Abstract We present an innovative approach called boosting Barlow Twins reduced order modeling (BBT‐ROM) to enhance the reliability of machine learning surrogate models for multiphase flow problems. BBT‐ROM builds upon Barlow Twins reduced order modeling that leverages self‐supervised learning to effectively handle linear and nonlinear manifolds by constructing well‐structured latent spaces of input parameters and output quantities. To address the challenge of high contrast data in multiphase flow problems due to injection wells and faults, we employ a boosting algorithm within BBT‐ROM. This algorithm sequentially trains a set of weak models (i.e., inaccurate models), improving prediction accuracy through ensemble learning. To evaluate the performance of BBT‐ROM, we conduct three three‐dimensional multiphase flow problems, including waterflooding and geologic carbon storage (GCS), with varying numbers of input parameter cases and model domain features. The results demonstrate that BBT‐ROM excels at predicting non‐wetting phase saturation (e.g., oil or saturation) and fluid pressure, with average relative errors ranging from 0.5% to 3%. Importantly, BBT‐ROM showcases robustness when faced with limited input parameter space during GCS testing.

View Journal Article

Cite

Export

Save

Sponsoring Organization:: USDOE

OSTI ID:: 2475447

Journal Information:: Water Resources Research, Journal Name: Water Resources Research Journal Issue: 10 Vol. 60; ISSN 0043-1397

Publisher:: American Geophysical Union (AGU)Copyright Statement

Country of Publication:: United States

Language:: English

References (59)

Effects of spatial heterogeneity and material anisotropy on the fracture pattern and macroscopic effective toughness of Mancos Shale in Brazilian tests Na, SeonHong; Sun, WaiChing; Ingraham, Mathew D. Journal of Geophysical Research: Solid Earth, Vol. 122, Issue 8 https://doi.org/10.1002/2016jb013374	journal	August 2017
Assessment of CO ₂ Storage Potential in Naturally Fractured Reservoirs With Dual‐Porosity Models March, Rafael; Doster, Florian; Geiger, Sebastian Water Resources Research, Vol. 54, Issue 3 https://doi.org/10.1002/2017wr022159	journal	March 2018
Scaling behavior of convective mixing, with application to geological storage of CO2 Hassanzadeh, Hassan; Pooladi-Darvish, Mehran; Keith, David W. AIChE Journal, Vol. 53, Issue 5 https://doi.org/10.1002/aic.11157	journal	January 2007
Improving the convergence behaviour of a fixed-point-iteration solver for multiphase flow in porous media Salinas, P.; Pavlidis, D.; Xie, Z. International Journal for Numerical Methods in Fluids, Vol. 84, Issue 8 https://doi.org/10.1002/fld.4357	journal	December 2016
Improving the robustness of the control volume finite element method with application to multiphase porous media flow Salinas, Pablo; Pavlidis, Dimitrios; Xie, Zhihua International Journal for Numerical Methods in Fluids, Vol. 85, Issue 4 https://doi.org/10.1002/fld.4381	journal	May 2017
An adaptive and efficient greedy procedure for the optimal training of parametric reduced-order models: ADAPTIVE GREEDY PROCEDURE FOR THE OPTIMAL TRAINING OF PARAMETRIC ROMS Paul-Dubois-Taine, A.; Amsallem, D. International Journal for Numerical Methods in Engineering, Vol. 102, Issue 5 https://doi.org/10.1002/nme.4759	journal	September 2014
A POD-selective inverse distance weighting method for fast parametrized shape morphing: A POD-selective IDW method for fast parametrized shape morphing Ballarin, Francesco; D'Amario, Alessandro; Perotto, Simona International Journal for Numerical Methods in Engineering, Vol. 117, Issue 8 https://doi.org/10.1002/nme.5982	journal	November 2018
An autoencoder‐based reduced‐order model for eigenvalue problems with application to neutron diffusion Phillips, Toby R. F.; Heaney, Claire E.; Smith, Paul N. International Journal for Numerical Methods in Engineering, Vol. 122, Issue 15 https://doi.org/10.1002/nme.6681	journal	May 2021
A reduced-order model for groundwater flow equation with random hydraulic conductivity: Application to Monte Carlo methods: Reduced-Order Model for Groundwater Flow Pasetto, Damiano; Putti, Mario; Yeh, William W-G. Water Resources Research, Vol. 49, Issue 6 https://doi.org/10.1002/wrcr.20136	journal	June 2013
Early Stopping - But When? Prechelt, Lutz Lecture Notes in Computer Science https://doi.org/10.1007/3-540-49430-8_3	book	January 1998
Certified Reduced Basis Methods for Parametrized Partial Differential Equations Hesthaven, Jan S.; Rozza, Gianluigi; Stamm, Benjamin SpringerBriefs in Mathematics https://doi.org/10.1007/978-3-319-22470-1	book	January 2016
PorePy: an open-source software for simulation of multiphysics processes in fractured porous media Keilegavlen, Eirik; Berge, Runar; Fumagalli, Alessio Computational Geosciences, Vol. 25, Issue 1 https://doi.org/10.1007/s10596-020-10002-5	journal	October 2020
Prediction of multiphase flows with sharp interfaces using anisotropic mesh optimisation Obeysekara, Asiri; Salinas, Pablo; Heaney, Claire E. Advances in Engineering Software, Vol. 160 https://doi.org/10.1016/j.advengsoft.2021.103044	journal	October 2021
Benchmarks for single-phase flow in fractured porous media Flemisch, Bernd; Berre, Inga; Boon, Wietse Advances in Water Resources, Vol. 111 https://doi.org/10.1016/j.advwatres.2017.10.036	journal	January 2018
Solution of diffusivity equations with local sources/sinks and surrogate modeling using weak form Theory-guided Neural Network Xu, Rui; Wang, Nanzhe; Zhang, Dongxiao Advances in Water Resources, Vol. 153 https://doi.org/10.1016/j.advwatres.2021.103941	journal	July 2021
Non-intrusive reduced order modeling of natural convection in porous media using convolutional autoencoders: Comparison with linear subspace techniques Kadeethum, T.; Ballarin, F.; Choi, Y. Advances in Water Resources, Vol. 160 https://doi.org/10.1016/j.advwatres.2021.104098	journal	February 2022
Modelling saline intrusion using dynamic mesh optimization with parallel processing Hamzehloo, A.; Bahlali, M. L.; Salinas, P. Advances in Water Resources, Vol. 164 https://doi.org/10.1016/j.advwatres.2022.104189	journal	June 2022
Continuous conditional generative adversarial networks for data-driven solutions of poroelasticity with heterogeneous material properties Kadeethum, T.; O’Malley, D.; Choi, Y. Computers & Geosciences, Vol. 167 https://doi.org/10.1016/j.cageo.2022.105212	journal	October 2022
Data-driven reduced order model with temporal convolutional neural network Wu, Pin; Sun, Junwu; Chang, Xuting Computer Methods in Applied Mechanics and Engineering, Vol. 360 https://doi.org/10.1016/j.cma.2019.112766	journal	March 2020
Component-wise reduced order model lattice-type structure design McBane, Sean; Choi, Youngsoo Computer Methods in Applied Mechanics and Engineering, Vol. 381 https://doi.org/10.1016/j.cma.2021.113813	journal	August 2021
Machine learning acceleration for nonlinear solvers applied to multiphase porous media flow Silva, V. L. S.; Salinas, P.; Jackson, M. D. Computer Methods in Applied Mechanics and Engineering, Vol. 384 https://doi.org/10.1016/j.cma.2021.113989	journal	October 2021
Domain-decomposition least-squares Petrov–Galerkin (DD-LSPG) nonlinear model reduction Hoang, Chi; Choi, Youngsoo; Carlberg, Kevin Computer Methods in Applied Mechanics and Engineering, Vol. 384 https://doi.org/10.1016/j.cma.2021.113997	journal	October 2021
Reduced order models for Lagrangian hydrodynamics Copeland, Dylan Matthew; Cheung, Siu Wun; Huynh, Kevin Computer Methods in Applied Mechanics and Engineering, Vol. 388 https://doi.org/10.1016/j.cma.2021.114259	journal	January 2022
Data-driven modeling and learning in science and engineering Montáns, Francisco J.; Chinesta, Francisco; Gómez-Bombarelli, Rafael Comptes Rendus Mécanique, Vol. 347, Issue 11 https://doi.org/10.1016/j.crme.2019.11.009	journal	November 2019
Dynamic mesh optimisation for geothermal reservoir modelling Salinas, P.; Regnier, G.; Jacquemyn, C. Geothermics, Vol. 94 https://doi.org/10.1016/j.geothermics.2021.102089	journal	July 2021
Capacity assessment and cost analysis of geologic storage of hydrogen: A case study in Intermountain-West Region USA Chen, Fangxuan; Ma, Zhiwei; Nasrabadi, Hadi International Journal of Hydrogen Energy, Vol. 48, Issue 24 https://doi.org/10.1016/j.ijhydene.2022.11.292	journal	March 2023
Two- and three-phase horizontal slug flow simulations using an interface-capturing compositional approach Pavlidis, Dimitrios; Xie, Zhihua; Percival, James R. International Journal of Multiphase Flow, Vol. 67 https://doi.org/10.1016/j.ijmultiphaseflow.2014.07.007	journal	December 2014
Non-intrusive reduced order modeling of nonlinear problems using neural networks Hesthaven, J. S.; Ubbiali, S. Journal of Computational Physics, Vol. 363 https://doi.org/10.1016/j.jcp.2018.02.037	journal	June 2018
Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification Zhu, Yinhao; Zabaras, Nicholas Journal of Computational Physics, Vol. 366 https://doi.org/10.1016/j.jcp.2018.04.018	journal	August 2018
Conservative model reduction for finite-volume models Carlberg, Kevin; Choi, Youngsoo; Sargsyan, Syuzanna Journal of Computational Physics, Vol. 371 https://doi.org/10.1016/j.jcp.2018.05.019	journal	October 2018
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations Raissi, M.; Perdikaris, P.; Karniadakis, G. E. Journal of Computational Physics, Vol. 378 https://doi.org/10.1016/j.jcp.2018.10.045	journal	February 2019
Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders Lee, Kookjin; Carlberg, Kevin T. Journal of Computational Physics https://doi.org/10.1016/j.jcp.2019.108973	journal	November 2019
Gradient-based constrained optimization using a database of linear reduced-order models Choi, Youngsoo; Boncoraglio, Gabriele; Anderson, Spenser Journal of Computational Physics, Vol. 423 https://doi.org/10.1016/j.jcp.2020.109787	journal	December 2020
Space–time reduced order model for large-scale linear dynamical systems with application to Boltzmann transport problems Choi, Youngsoo; Brown, Peter; Arrighi, William Journal of Computational Physics, Vol. 424 https://doi.org/10.1016/j.jcp.2020.109845	journal	January 2021
A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder Kim, Youngkyu; Choi, Youngsoo; Widemann, David Journal of Computational Physics, Vol. 451 https://doi.org/10.1016/j.jcp.2021.110841	journal	February 2022
An investigation of hydromechanical effect on well productivity in fractured porous media using full factorial experimental design Kadeethum, T.; Salimzadeh, S.; Nick, H. M. Journal of Petroleum Science and Engineering, Vol. 181 https://doi.org/10.1016/j.petrol.2019.106233	journal	October 2019
Chemical and Hydrodynamic Mechanisms for Long-Term Geological Carbon Storage Altman, Susan J.; Aminzadeh, Behdad; Balhoff, Matthew T. The Journal of Physical Chemistry C, Vol. 118, Issue 28 https://doi.org/10.1021/jp5006764	journal	June 2014
Pore-scale simulation of mixing-induced calcium carbonate precipitation and dissolution in a microfluidic pore network: PORE-SCALE SIMULATION IN A MICROMODEL Yoon, Hongkyu; Valocchi, Albert J.; Werth, Charles J. Water Resources Research, Vol. 48, Issue 2 https://doi.org/10.1029/2011wr011192	journal	February 2012
Review of surrogate modeling in water resources: REVIEW Razavi, Saman; Tolson, Bryan A.; Burn, Donald H. Water Resources Research, Vol. 48, Issue 7 https://doi.org/10.1029/2011wr011527	journal	July 2012
Highly parameterized inverse estimation of hydraulic conductivity and porosity in a three-dimensional, heterogeneous transport experiment: HIGHLY PARAMETERIZED INVERSE ESTIMATION IN 3-D Yoon, Hongkyu; McKenna, Sean A. Water Resources Research, Vol. 48, Issue 10 https://doi.org/10.1029/2012wr012149	journal	October 2012
3-D Modeling of Induced Seismicity Along Multiple Faults: Magnitude, Rate, and Location in a Poroelasticity System Chang, K. W.; Yoon, H. Journal of Geophysical Research: Solid Earth, Vol. 123, Issue 11 https://doi.org/10.1029/2018jb016446	journal	November 2018
A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists Shen, Chaopeng Water Resources Research, Vol. 54, Issue 11 https://doi.org/10.1029/2018wr022643	journal	November 2018
Deep Convolutional Encoder‐Decoder Networks for Uncertainty Quantification of Dynamic Multiphase Flow in Heterogeneous Media Mo, Shaoxing; Zhu, Yinhao; Zabaras, Nicholas Water Resources Research, Vol. 55, Issue 1 https://doi.org/10.1029/2018wr023528	journal	January 2019
Mitigating Injection‐Induced Seismicity Along Basement Faults by Extraction: Application to 2016–2018 Pohang Earthquakes Chang, K. W.; Yoon, H. Journal of Geophysical Research: Solid Earth, Vol. 126, Issue 4 https://doi.org/10.1029/2020jb021486	journal	April 2021
Geological carbon storage Bickle, Mike J. Nature Geoscience, Vol. 2, Issue 12 https://doi.org/10.1038/ngeo687	journal	December 2009
Deep transfer operator learning for partial differential equations under conditional shift Goswami, Somdatta; Kontolati, Katiana; Shields, Michael D. Nature Machine Intelligence, Vol. 4, Issue 12, p. 1155-1164 https://doi.org/10.1038/s42256-022-00569-2	journal	December 2022
Real-time high-resolution CO₂ geological storage prediction using nested Fourier neural operators Wen, Gege; Li, Zongyi; Long, Qirui Energy & Environmental Science, Vol. 16, Issue 4 https://doi.org/10.1039/d2ee04204e	journal	January 2023
An AI-based non-intrusive reduced-order model for extended domains applied to multiphase flow in pipes Heaney, Claire E.; Wolffs, Zef; Tómasson, Jón Atli Physics of Fluids, Vol. 34, Issue 5 https://doi.org/10.1063/5.0088070	journal	May 2022
Non-asymptotic Closed-Loop System Identification using Autoregressive Processes and Hankel Model Reduction Lee, Bruce; Lamperski, Andrew 2020 59th IEEE Conference on Decision and Control (CDC) https://doi.org/10.1109/CDC42340.2020.9304468	conference	December 2020
Data Compression Using Adaptive Coding and Partial String Matching Cleary, J.; Witten, I. IEEE Transactions on Communications, Vol. 32, Issue 4 https://doi.org/10.1109/tcom.1984.1096090	journal	January 1984
Dynamic Mode Decomposition Kutz, J. Nathan; Brunton, Steven L.; Brunton, Bingni W. Society for Industrial and Applied Mathematics https://doi.org/10.1137/1.9781611974508	book	January 2016
Space--Time Least-Squares Petrov--Galerkin Projection for Nonlinear Model Reduction Choi, Youngsoo; Carlberg, Kevin SIAM Journal on Scientific Computing, Vol. 41, Issue 1 https://doi.org/10.1137/17m1120531	journal	January 2019
SNS: A Solution-Based Nonlinear Subspace Method for Time-Dependent Model Order Reduction Choi, Youngsoo; Coombs, Deshawn; Anderson, Robert SIAM Journal on Scientific Computing, Vol. 42, Issue 2 https://doi.org/10.1137/19m1242963	journal	January 2020
An Artificial Compression Reduced Order Model DeCaria, Victor; Iliescu, Traian; Layton, William SIAM Journal on Numerical Analysis, Vol. 58, Issue 1 https://doi.org/10.1137/19m1246444	journal	January 2020
Upscaling two-phase flow in naturally fractured reservoirs Matthäi, Stephan K.; Nick, Hamidreza M. AAPG Bulletin, Vol. 93, Issue 11 https://doi.org/10.1306/08030909085	journal	November 2009
Physics-informed neural networks for solving nonlinear diffusivity and Biot’s equations Kadeethum, Teeratorn; Jørgensen, Thomas M.; Nick, Hamidreza M. PLOS ONE, Vol. 15, Issue 5 https://doi.org/10.1371/journal.pone.0232683	journal	May 2020
Lattice Boltzmann-Based Approaches for Pore-Scale Reactive Transport Yoon, Hongkyu; Kang, Qinjun; Valocchi, Albert J. Reviews in Mineralogy and Geochemistry, Vol. 80, Issue 1 https://doi.org/10.2138/rmg.2015.80.12	journal	January 2015
Balanced Model Reduction via the Proper Orthogonal Decomposition Willcox, K.; Peraire, J. AIAA Journal, Vol. 40, Issue 11 https://doi.org/10.2514/2.1570	journal	November 2002
Supplemental materials for "Boosting Barlow Twins reduced order modeling for machine learning-based surrogate models in multiphase flow problems" Kadeethum, Teeratorn; Silva, Vinicius L. S.; Salinas, Pablo Zenodo https://doi.org/10.5281/zenodo.13756055	dataset	January 2024

Similar Records

Reduced order modeling for flow and transport problems with Barlow Twins self-supervised learning

Journal Article · 2022 · Scientific Reports · OSTI ID:1901445

Kadeethum, Teeratorn; Ballarin, Francesco; O’Malley, Daniel; +3 more

Progressive transfer learning for advancing machine learning-based reduced-order modeling

Journal Article · 2024 · Scientific Reports · OSTI ID:2395972

Kadeethum, Teeratorn; O’Malley, Daniel; Choi, Youngsoo; +2 more

Backflow in twin-screw-type multiphase pump

Journal Article · 1998 · SPE Production and Facilities · OSTI ID:580835

Egashira, Kazuyuki; Shoda, Shinji; Tochikawa, Tetsuro; +1 more

Title: Boosting Barlow Twins Reduced Order Modeling for Machine Learning‐Based Surrogate Models in Multiphase Flow Problems

Citation Formats

References (59)

Similar Records

Related Subjects