DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Linear and nonlinear solvers for simulating multiphase flow within large-scale engineered subsurface systems

Abstract

Multiphase flow simulation is well-known to be computationally demanding, and modeling large-scale engineered subsurface systems entails significant additional numerical challenges. These challenges arise from: (a) the presence of small-scale discrete features like shafts, tunnels, waste packages, and barriers; (b) the need to accurately represent both the waste form processes at the small spatial scale of the repository and the large-scale transport processes throughout heterogeneous geological formations; (c) the strong contrast in material properties such as porosity and permeability, as well as the nonlinear constitutive relations for multiphase flow. Numerical solution entails discretization of the coupled system of nonlinear governing equations and solving a linear system of equations at each Newton–Raphson iteration. Practical problems require a very large number of unknowns that must be solved efficiently using iterative methods in parallel on high-performance computers. The unique challenges noted above can lead to an ill-conditioned Jacobian matrix and non-convergence with Newton’s method due to discontinuous nonlinearity in constitutive models. Moreover, practical applications can require numerous Monte-Carlo simulations to explore uncertainly in material properties, geological heterogeneity, failure scenarios, or other factors; governmental regulatory agencies can mandate these as part of Performance Assessments. Hence there is a need for flexible, robust, and computationally efficientmore » methods for multiphase flow in large-scale engineered subsurface systems. We apply the open-source simulator PFLOTRAN to the practical problem of performance assessment of the US DOE Waste Isolation Pilot Plant (WIPP) site. The simulator employs a finite volume discretization and uses the PETSc parallel framework. We evaluate the performance of several preconditioners for the iterative solution of the linearized Jacobian system; these range from stabilized-biconjugate-gradient with block-Jacobi preconditioning (BCGS) to methods adopted from reservoir modeling, such as the constrained pressure residual (CPR) two-stage preconditioner and flexible generalized residual solver (FGMRES). We also implement within PETSc the general-purpose nonlinear solver, Newton trust-region dogleg Cauchy (NTRDC), which truncates the Newton update or modifies the update with a Cauchy solution that is within the quadratic model trust-region of the objective function. We demonstrate the effectiveness of each method for a series of test problems with increasing difficulty. We find that the NTRDC and FGMRES-CPR-ABF (FCA) preconditioners generally perform best for the test problem having the extreme nonlinear processes, achieving a 50x speed-up compared with BCGS. The most ill-conditioned and extreme nonlinear simulations do not converge with BCGS (as one may expect), but they do complete the simulation with NTRDC and FCA. In conclusion, we also investigate the strong scalability of each method and demonstrate the impact of node-packing upon parallel performance on modern processor architectures.« less

Authors:
ORCiD logo [1]; ORCiD logo [2];  [3];  [4]
  1. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Univ. of Illinois at Urbana-Champaign, IL (United States)
  2. Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
  3. Univ. of Illinois at Urbana-Champaign, IL (United States)
  4. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Publication Date:
Research Org.:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1828787
Report Number(s):
SAND-2021-11975J
Journal ID: ISSN 0309-1708; 700132
Grant/Contract Number:  
NA0003525
Resource Type:
Accepted Manuscript
Journal Name:
Advances in Water Resources
Additional Journal Information:
Journal Volume: 156; Journal ID: ISSN 0309-1708
Publisher:
Elsevier
Country of Publication:
United States
Language:
English
Subject:
42 ENGINEERING; Preconditioner; Nonlinear; Porous media; Multiphase flow; Trust-region; Subsurface system

Citation Formats

Park, Heeho D., Hammond, Glenn E., Valocchi, Albert J., and LaForce, Tara. Linear and nonlinear solvers for simulating multiphase flow within large-scale engineered subsurface systems. United States: N. p., 2021. Web. doi:10.1016/j.advwatres.2021.104029.
Park, Heeho D., Hammond, Glenn E., Valocchi, Albert J., & LaForce, Tara. Linear and nonlinear solvers for simulating multiphase flow within large-scale engineered subsurface systems. United States. https://doi.org/10.1016/j.advwatres.2021.104029
Park, Heeho D., Hammond, Glenn E., Valocchi, Albert J., and LaForce, Tara. Mon . "Linear and nonlinear solvers for simulating multiphase flow within large-scale engineered subsurface systems". United States. https://doi.org/10.1016/j.advwatres.2021.104029. https://www.osti.gov/servlets/purl/1828787.
@article{osti_1828787,
title = {Linear and nonlinear solvers for simulating multiphase flow within large-scale engineered subsurface systems},
author = {Park, Heeho D. and Hammond, Glenn E. and Valocchi, Albert J. and LaForce, Tara},
abstractNote = {Multiphase flow simulation is well-known to be computationally demanding, and modeling large-scale engineered subsurface systems entails significant additional numerical challenges. These challenges arise from: (a) the presence of small-scale discrete features like shafts, tunnels, waste packages, and barriers; (b) the need to accurately represent both the waste form processes at the small spatial scale of the repository and the large-scale transport processes throughout heterogeneous geological formations; (c) the strong contrast in material properties such as porosity and permeability, as well as the nonlinear constitutive relations for multiphase flow. Numerical solution entails discretization of the coupled system of nonlinear governing equations and solving a linear system of equations at each Newton–Raphson iteration. Practical problems require a very large number of unknowns that must be solved efficiently using iterative methods in parallel on high-performance computers. The unique challenges noted above can lead to an ill-conditioned Jacobian matrix and non-convergence with Newton’s method due to discontinuous nonlinearity in constitutive models. Moreover, practical applications can require numerous Monte-Carlo simulations to explore uncertainly in material properties, geological heterogeneity, failure scenarios, or other factors; governmental regulatory agencies can mandate these as part of Performance Assessments. Hence there is a need for flexible, robust, and computationally efficient methods for multiphase flow in large-scale engineered subsurface systems. We apply the open-source simulator PFLOTRAN to the practical problem of performance assessment of the US DOE Waste Isolation Pilot Plant (WIPP) site. The simulator employs a finite volume discretization and uses the PETSc parallel framework. We evaluate the performance of several preconditioners for the iterative solution of the linearized Jacobian system; these range from stabilized-biconjugate-gradient with block-Jacobi preconditioning (BCGS) to methods adopted from reservoir modeling, such as the constrained pressure residual (CPR) two-stage preconditioner and flexible generalized residual solver (FGMRES). We also implement within PETSc the general-purpose nonlinear solver, Newton trust-region dogleg Cauchy (NTRDC), which truncates the Newton update or modifies the update with a Cauchy solution that is within the quadratic model trust-region of the objective function. We demonstrate the effectiveness of each method for a series of test problems with increasing difficulty. We find that the NTRDC and FGMRES-CPR-ABF (FCA) preconditioners generally perform best for the test problem having the extreme nonlinear processes, achieving a 50x speed-up compared with BCGS. The most ill-conditioned and extreme nonlinear simulations do not converge with BCGS (as one may expect), but they do complete the simulation with NTRDC and FCA. In conclusion, we also investigate the strong scalability of each method and demonstrate the impact of node-packing upon parallel performance on modern processor architectures.},
doi = {10.1016/j.advwatres.2021.104029},
journal = {Advances in Water Resources},
number = ,
volume = 156,
place = {United States},
year = {Mon Sep 13 00:00:00 EDT 2021},
month = {Mon Sep 13 00:00:00 EDT 2021}
}

Works referenced in this record:

Evaluating the performance of parallel subsurface simulators: An illustrative example with PFLOTRAN: Evaluating the Parallel Performance of Pflotran
journal, January 2014

  • Hammond, G. E.; Lichtner, P. C.; Mills, R. T.
  • Water Resources Research, Vol. 50, Issue 1
  • DOI: 10.1002/2012WR013483

PFLOTRAN: Reactive Flow & Transport Code for Use on Laptops to Leadership-Class Supercomputers
book, March 2012


Constrained pressure residual multiscale (CPR-MS) method for fully implicit simulation of multiphase flow in porous media
journal, October 2015

  • Cusini, Matteo; Lukyanov, Alexander A.; Natvig, Jostein
  • Journal of Computational Physics, Vol. 299
  • DOI: 10.1016/j.jcp.2015.07.019

Decoupling preconditioners in the implicit parallel accurate reservoir simulator (IPARS)
journal, January 2001

  • Lacroix, S�bastien; Vassilevski, Yuri V.; Wheeler, Mary F.
  • Numerical Linear Algebra with Applications, Vol. 8, Issue 8
  • DOI: 10.1002/nla.264

Valgrind
journal, October 2003


Reducing Complexity in Parallel Algebraic Multigrid Preconditioners
journal, January 2006

  • De Sterck, Hans; Yang, Ulrike Meier; Heys, Jeffrey J.
  • SIAM Journal on Matrix Analysis and Applications, Vol. 27, Issue 4
  • DOI: 10.1137/040615729

A Flexible Inner-Outer Preconditioned GMRES Algorithm
journal, March 1993

  • Saad, Youcef
  • SIAM Journal on Scientific Computing, Vol. 14, Issue 2
  • DOI: 10.1137/0914028

Algebraic Multigrid Preconditioners for Multiphase Flow in Porous Media
journal, January 2017

  • Bui, Quan M.; Elman, Howard C.; Moulton, J. David
  • SIAM Journal on Scientific Computing, Vol. 39, Issue 5
  • DOI: 10.1137/16M1082652

Distance-two interpolation for parallel algebraic multigrid
journal, January 2008

  • De Sterck, Hans; Falgout, Robert D.; Nolting, Joshua W.
  • Numerical Linear Algebra with Applications, Vol. 15, Issue 2-3
  • DOI: 10.1002/nla.559

Performance Assessment for the Waste Isolation Pilot Plant: From Regulation to Calculation for 40 CFR 191.13
journal, April 1997

  • Helton, J. C.; Anderson, D. R.; Marietta, M. G.
  • Operations Research, Vol. 45, Issue 2
  • DOI: 10.1287/opre.45.2.157

Iterative Solution Methods for Modeling Multiphase Flow in Porous Media Fully Implicitly
journal, January 2003

  • Lacroix, S.; Vassilevski, Yu.; Wheeler, J.
  • SIAM Journal on Scientific Computing, Vol. 25, Issue 3
  • DOI: 10.1137/S106482750240443X

The alternate-block-factorization procedure for systems of partial differential equations
journal, December 1989

  • Bank, R. E.; Chan, T. F.; Coughran, W. M.
  • BIT Numerical Mathematics, Vol. 29, Issue 4
  • DOI: 10.1007/BF01932753