skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: A computationally efficient parallel Levenberg-Marquardt algorithm for highly parameterized inverse model analyses

Abstract

Inverse modeling seeks model parameters given a set of observations. However, for practical problems because the number of measurements is often large and the model parameters are also numerous, conventional methods for inverse modeling can be computationally expensive. We have developed a new, computationally-efficient parallel Levenberg-Marquardt method for solving inverse modeling problems with a highly parameterized model space. Levenberg-Marquardt methods require the solution of a linear system of equations which can be prohibitively expensive to compute for moderate to large-scale problems. Our novel method projects the original linear problem down to a Krylov subspace, such that the dimensionality of the problem can be significantly reduced. Furthermore, we store the Krylov subspace computed when using the first damping parameter and recycle the subspace for the subsequent damping parameters. The efficiency of our new inverse modeling algorithm is significantly improved using these computational techniques. We apply this new inverse modeling method to invert for random transmissivity fields in 2D and a random hydraulic conductivity field in 3D. Our algorithm is fast enough to solve for the distributed model parameters (transmissivity) in the model domain. The algorithm is coded in Julia and implemented in the MADS computational framework (http://mads.lanl.gov). By comparing with Levenberg-Marquardtmore » methods using standard linear inversion techniques such as QR or SVD methods, our Levenberg-Marquardt method yields a speed-up ratio on the order of ~10 1 to ~10 2 in a multi-core computational environment. Furthermore, our new inverse modeling method is a powerful tool for characterizing subsurface heterogeneity for moderate- to large-scale problems.« less

Authors:
 [1];  [1];  [1]
  1. Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Publication Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
LANL EP Program; USDOE
OSTI Identifier:
1312574
Report Number(s):
LA-UR-16-22377
Journal ID: ISSN 0043-1397
Grant/Contract Number:  
AC52-06NA25396
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
Water Resources Research
Additional Journal Information:
Journal Name: Water Resources Research; Journal ID: ISSN 0043-1397
Publisher:
American Geophysical Union (AGU)
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Earth Sciences

Citation Formats

Lin, Youzuo, O'Malley, Daniel, and Vesselinov, Velimir V. A computationally efficient parallel Levenberg-Marquardt algorithm for highly parameterized inverse model analyses. United States: N. p., 2016. Web. doi:10.1002/2016WR019028.
Lin, Youzuo, O'Malley, Daniel, & Vesselinov, Velimir V. A computationally efficient parallel Levenberg-Marquardt algorithm for highly parameterized inverse model analyses. United States. doi:10.1002/2016WR019028.
Lin, Youzuo, O'Malley, Daniel, and Vesselinov, Velimir V. Thu . "A computationally efficient parallel Levenberg-Marquardt algorithm for highly parameterized inverse model analyses". United States. doi:10.1002/2016WR019028. https://www.osti.gov/servlets/purl/1312574.
@article{osti_1312574,
title = {A computationally efficient parallel Levenberg-Marquardt algorithm for highly parameterized inverse model analyses},
author = {Lin, Youzuo and O'Malley, Daniel and Vesselinov, Velimir V.},
abstractNote = {Inverse modeling seeks model parameters given a set of observations. However, for practical problems because the number of measurements is often large and the model parameters are also numerous, conventional methods for inverse modeling can be computationally expensive. We have developed a new, computationally-efficient parallel Levenberg-Marquardt method for solving inverse modeling problems with a highly parameterized model space. Levenberg-Marquardt methods require the solution of a linear system of equations which can be prohibitively expensive to compute for moderate to large-scale problems. Our novel method projects the original linear problem down to a Krylov subspace, such that the dimensionality of the problem can be significantly reduced. Furthermore, we store the Krylov subspace computed when using the first damping parameter and recycle the subspace for the subsequent damping parameters. The efficiency of our new inverse modeling algorithm is significantly improved using these computational techniques. We apply this new inverse modeling method to invert for random transmissivity fields in 2D and a random hydraulic conductivity field in 3D. Our algorithm is fast enough to solve for the distributed model parameters (transmissivity) in the model domain. The algorithm is coded in Julia and implemented in the MADS computational framework (http://mads.lanl.gov). By comparing with Levenberg-Marquardt methods using standard linear inversion techniques such as QR or SVD methods, our Levenberg-Marquardt method yields a speed-up ratio on the order of ~101 to ~102 in a multi-core computational environment. Furthermore, our new inverse modeling method is a powerful tool for characterizing subsurface heterogeneity for moderate- to large-scale problems.},
doi = {10.1002/2016WR019028},
journal = {Water Resources Research},
issn = {0043-1397},
number = ,
volume = ,
place = {United States},
year = {2016},
month = {9}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 4 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Solution of Sparse Indefinite Systems of Linear Equations
journal, September 1975

  • Paige, C. C.; Saunders, M. A.
  • SIAM Journal on Numerical Analysis, Vol. 12, Issue 4
  • DOI: 10.1137/0712047

LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares
journal, March 1982

  • Paige, Christopher C.; Saunders, Michael A.
  • ACM Transactions on Mathematical Software, Vol. 8, Issue 1
  • DOI: 10.1145/355984.355989

A modified Levenberg–Marquardt algorithm for quasi-linear geostatistical inversing
journal, July 2004


A method for the solution of certain non-linear problems in least squares
journal, January 1944

  • Levenberg, Kenneth
  • Quarterly of Applied Mathematics, Vol. 2, Issue 2
  • DOI: 10.1090/qam/10666

Active subspaces for sensitivity analysis and dimension reduction of an integrated hydrologic model
journal, October 2015


On level set regularization for highly ill-posed distributed parameter estimation problems
journal, August 2006


Calculating the Singular Values and Pseudo-Inverse of a Matrix
journal, January 1965

  • Golub, G.; Kahan, W.
  • Journal of the Society for Industrial and Applied Mathematics Series B Numerical Analysis, Vol. 2, Issue 2
  • DOI: 10.1137/0702016

Application of inverse methods to contaminant source identification from aquitard diffusion profiles at Dover AFB, Delaware
journal, July 1999

  • Liu, Chongxuan; Ball, William P.
  • Water Resources Research, Vol. 35, Issue 7
  • DOI: 10.1029/1999WR900092

Fast iterative implementation of large-scale nonlinear geostatistical inverse modeling: FAST ITERATIVE NONLINEAR INVERSE MODELING
journal, January 2014

  • Liu, Xiaoyi; Zhou, Quanlin; Kitanidis, Peter K.
  • Water Resources Research, Vol. 50, Issue 1
  • DOI: 10.1002/2012WR013241

A Comparison of Several Methods of Solving Nonlinear Regression Groundwater Flow Problems
journal, October 1985


Principal Component Geostatistical Approach for large-dimensional inverse problems
journal, July 2014


Damping–undamping strategies for the Levenberg–Marquardt nonlinear least-squares method
journal, January 1997


Efficient methods for large-scale linear inversion using a geostatistical approach: EFFICIENT GEOSTATISTICAL APPROACH
journal, May 2012

  • Saibaba, Arvind K.; Kitanidis, Peter K.
  • Water Resources Research, Vol. 48, Issue 5
  • DOI: 10.1029/2011WR011778

Algorithm 583: LSQR: Sparse Linear Equations and Least Squares Problems
journal, June 1982

  • Paige, Christopher C.; Saunders, Michael A.
  • ACM Transactions on Mathematical Software, Vol. 8, Issue 2
  • DOI: 10.1145/355993.356000

Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces
journal, January 2014

  • Constantine, Paul G.; Dow, Eric; Wang, Qiqi
  • SIAM Journal on Scientific Computing, Vol. 36, Issue 4
  • DOI: 10.1137/130916138

Are Models Too Simple? Arguments for Increased Parameterization
journal, May 2007


A truncated Levenberg–Marquardt algorithm for the calibration of highly parameterized nonlinear models
journal, June 2011


Linear functional minimization for inverse modeling: LINEAR FUNCTIONAL MINIMIZATION FOR INVERSE MODELING
journal, June 2015

  • Barajas-Solano, D. A.; Wohlberg, B. E.; Vesselinov, V. V.
  • Water Resources Research, Vol. 51, Issue 6
  • DOI: 10.1002/2014WR016179

Estimation of Aquifer Parameters Under Transient and Steady State Conditions: 1. Maximum Likelihood Method Incorporating Prior Information
journal, February 1986


Using an inverse method to estimate the hydraulic properties of crusted soils from tension-disc infiltrometer data
journal, October 1998


A Parallel Nonlinear Least-Squares Solver: Theoretical Analysis and Numerical Results
journal, May 1992

  • Coleman, Thomas F.; Plassmann, Paul E.
  • SIAM Journal on Scientific and Statistical Computing, Vol. 13, Issue 3
  • DOI: 10.1137/0913046

Geostatistical reduced-order models in underdetermined inverse problems: Groms in Underdetermined Inverse Problems
journal, October 2013

  • Liu, Xiaoyi; Zhou, Quanlin; Birkholzer, Jens
  • Water Resources Research, Vol. 49, Issue 10
  • DOI: 10.1002/wrcr.20489

A hybrid regularized inversion methodology for highly parameterized environmental models: HYBRID REGULARIZATION METHODOLOGY
journal, October 2005

  • Tonkin, Matthew James; Doherty, John
  • Water Resources Research, Vol. 41, Issue 10
  • DOI: 10.1029/2005WR003995