A computationally efficient parallel LevenbergMarquardt algorithm for highly parameterized inverse model analyses
Abstract
Inverse modeling seeks model parameters given a set of observations. However, for practical problems because the number of measurements is often large and the model parameters are also numerous, conventional methods for inverse modeling can be computationally expensive. We have developed a new, computationallyefficient parallel LevenbergMarquardt method for solving inverse modeling problems with a highly parameterized model space. LevenbergMarquardt methods require the solution of a linear system of equations which can be prohibitively expensive to compute for moderate to largescale problems. Our novel method projects the original linear problem down to a Krylov subspace, such that the dimensionality of the problem can be significantly reduced. Furthermore, we store the Krylov subspace computed when using the first damping parameter and recycle the subspace for the subsequent damping parameters. The efficiency of our new inverse modeling algorithm is significantly improved using these computational techniques. We apply this new inverse modeling method to invert for random transmissivity fields in 2D and a random hydraulic conductivity field in 3D. Our algorithm is fast enough to solve for the distributed model parameters (transmissivity) in the model domain. The algorithm is coded in Julia and implemented in the MADS computational framework (http://mads.lanl.gov). By comparing with LevenbergMarquardtmore »
 Authors:

 Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
 Publication Date:
 Research Org.:
 Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
 Sponsoring Org.:
 LANL EP Program; USDOE
 OSTI Identifier:
 1312574
 Report Number(s):
 LAUR1622377
Journal ID: ISSN 00431397
 Grant/Contract Number:
 AC5206NA25396
 Resource Type:
 Journal Article: Accepted Manuscript
 Journal Name:
 Water Resources Research
 Additional Journal Information:
 Journal Name: Water Resources Research; Journal ID: ISSN 00431397
 Publisher:
 American Geophysical Union (AGU)
 Country of Publication:
 United States
 Language:
 English
 Subject:
 97 MATHEMATICS AND COMPUTING; Earth Sciences
Citation Formats
Lin, Youzuo, O'Malley, Daniel, and Vesselinov, Velimir V. A computationally efficient parallel LevenbergMarquardt algorithm for highly parameterized inverse model analyses. United States: N. p., 2016.
Web. doi:10.1002/2016WR019028.
Lin, Youzuo, O'Malley, Daniel, & Vesselinov, Velimir V. A computationally efficient parallel LevenbergMarquardt algorithm for highly parameterized inverse model analyses. United States. doi:10.1002/2016WR019028.
Lin, Youzuo, O'Malley, Daniel, and Vesselinov, Velimir V. Thu .
"A computationally efficient parallel LevenbergMarquardt algorithm for highly parameterized inverse model analyses". United States. doi:10.1002/2016WR019028. https://www.osti.gov/servlets/purl/1312574.
@article{osti_1312574,
title = {A computationally efficient parallel LevenbergMarquardt algorithm for highly parameterized inverse model analyses},
author = {Lin, Youzuo and O'Malley, Daniel and Vesselinov, Velimir V.},
abstractNote = {Inverse modeling seeks model parameters given a set of observations. However, for practical problems because the number of measurements is often large and the model parameters are also numerous, conventional methods for inverse modeling can be computationally expensive. We have developed a new, computationallyefficient parallel LevenbergMarquardt method for solving inverse modeling problems with a highly parameterized model space. LevenbergMarquardt methods require the solution of a linear system of equations which can be prohibitively expensive to compute for moderate to largescale problems. Our novel method projects the original linear problem down to a Krylov subspace, such that the dimensionality of the problem can be significantly reduced. Furthermore, we store the Krylov subspace computed when using the first damping parameter and recycle the subspace for the subsequent damping parameters. The efficiency of our new inverse modeling algorithm is significantly improved using these computational techniques. We apply this new inverse modeling method to invert for random transmissivity fields in 2D and a random hydraulic conductivity field in 3D. Our algorithm is fast enough to solve for the distributed model parameters (transmissivity) in the model domain. The algorithm is coded in Julia and implemented in the MADS computational framework (http://mads.lanl.gov). By comparing with LevenbergMarquardt methods using standard linear inversion techniques such as QR or SVD methods, our LevenbergMarquardt method yields a speedup ratio on the order of ~101 to ~102 in a multicore computational environment. Furthermore, our new inverse modeling method is a powerful tool for characterizing subsurface heterogeneity for moderate to largescale problems.},
doi = {10.1002/2016WR019028},
journal = {Water Resources Research},
issn = {00431397},
number = ,
volume = ,
place = {United States},
year = {2016},
month = {9}
}
Web of Science
Works referenced in this record:
Solution of Sparse Indefinite Systems of Linear Equations
journal, September 1975
 Paige, C. C.; Saunders, M. A.
 SIAM Journal on Numerical Analysis, Vol. 12, Issue 4
LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares
journal, March 1982
 Paige, Christopher C.; Saunders, Michael A.
 ACM Transactions on Mathematical Software, Vol. 8, Issue 1
A modified Levenberg–Marquardt algorithm for quasilinear geostatistical inversing
journal, July 2004
 Nowak, Wolfgang; Cirpka, Olaf A.
 Advances in Water Resources, Vol. 27, Issue 7
Obtaining parsimonious hydraulic conductivity fields using head and transport observations: A Bayesian geostatistical parameter estimation approach: COUPLED FLOW AND PARAMETER ESTIMATION
journal, August 2009
 Fienen, M.; Hunt, R.; Krabbenhoft, D.
 Water Resources Research, Vol. 45, Issue 8
A method for the solution of certain nonlinear problems in least squares
journal, January 1944
 Levenberg, Kenneth
 Quarterly of Applied Mathematics, Vol. 2, Issue 2
Active subspaces for sensitivity analysis and dimension reduction of an integrated hydrologic model
journal, October 2015
 Jefferson, Jennifer L.; Gilbert, James M.; Constantine, Paul G.
 Computers & Geosciences, Vol. 83
On level set regularization for highly illposed distributed parameter estimation problems
journal, August 2006
 van den Doel, K.; Ascher, U. M.
 Journal of Computational Physics, Vol. 216, Issue 2
Calculating the Singular Values and PseudoInverse of a Matrix
journal, January 1965
 Golub, G.; Kahan, W.
 Journal of the Society for Industrial and Applied Mathematics Series B Numerical Analysis, Vol. 2, Issue 2
Application of inverse methods to contaminant source identification from aquitard diffusion profiles at Dover AFB, Delaware
journal, July 1999
 Liu, Chongxuan; Ball, William P.
 Water Resources Research, Vol. 35, Issue 7
Fast iterative implementation of largescale nonlinear geostatistical inverse modeling: FAST ITERATIVE NONLINEAR INVERSE MODELING
journal, January 2014
 Liu, Xiaoyi; Zhou, Quanlin; Kitanidis, Peter K.
 Water Resources Research, Vol. 50, Issue 1
A Comparison of Several Methods of Solving Nonlinear Regression Groundwater Flow Problems
journal, October 1985
 Cooley, Richard L.
 Water Resources Research, Vol. 21, Issue 10
Principal Component Geostatistical Approach for largedimensional inverse problems
journal, July 2014
 Kitanidis, P. K.; Lee, J.
 Water Resources Research, Vol. 50, Issue 7
Damping–undamping strategies for the Levenberg–Marquardt nonlinear leastsquares method
journal, January 1997
 Lampton, Michael
 Computers in Physics, Vol. 11, Issue 1
Efficient methods for largescale linear inversion using a geostatistical approach: EFFICIENT GEOSTATISTICAL APPROACH
journal, May 2012
 Saibaba, Arvind K.; Kitanidis, Peter K.
 Water Resources Research, Vol. 48, Issue 5
Algorithm 583: LSQR: Sparse Linear Equations and Least Squares Problems
journal, June 1982
 Paige, Christopher C.; Saunders, Michael A.
 ACM Transactions on Mathematical Software, Vol. 8, Issue 2
Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces
journal, January 2014
 Constantine, Paul G.; Dow, Eric; Wang, Qiqi
 SIAM Journal on Scientific Computing, Vol. 36, Issue 4
Are Models Too Simple? Arguments for Increased Parameterization
journal, May 2007
 Hunt, Randall J.; Doherty, John; Tonkin, Matthew J.
 Ground Water, Vol. 45, Issue 3
Largescale hydraulic tomography and joint inversion of head and tracer data using the Principal Component Geostatistical Approach (PCGA)
journal, July 2014
 Lee, J.; Kitanidis, P. K.
 Water Resources Research, Vol. 50, Issue 7
A truncated Levenberg–Marquardt algorithm for the calibration of highly parameterized nonlinear models
journal, June 2011
 Finsterle, Stefan; Kowalsky, Michael B.
 Computers & Geosciences, Vol. 37, Issue 6
Linear functional minimization for inverse modeling: LINEAR FUNCTIONAL MINIMIZATION FOR INVERSE MODELING
journal, June 2015
 BarajasSolano, D. A.; Wohlberg, B. E.; Vesselinov, V. V.
 Water Resources Research, Vol. 51, Issue 6
Estimation of Aquifer Parameters Under Transient and Steady State Conditions: 1. Maximum Likelihood Method Incorporating Prior Information
journal, February 1986
 Carrera, Jesus; Neuman, Shlomo P.
 Water Resources Research, Vol. 22, Issue 2
Using an inverse method to estimate the hydraulic properties of crusted soils from tensiondisc infiltrometer data
journal, October 1998
 Šimůnek, Jiřı́; AnguloJaramillo, Rafael; Schaap, Marcel G.
 Geoderma, Vol. 86, Issue 12
Efficient Parallel LevenbergMarquardt Model Fitting towards RealTime Automated Parametric Imaging Microscopy
journal, October 2013
 Zhu, Xiang; Zhang, Dianwen
 PLoS ONE, Vol. 8, Issue 10
A Parallel Nonlinear LeastSquares Solver: Theoretical Analysis and Numerical Results
journal, May 1992
 Coleman, Thomas F.; Plassmann, Paul E.
 SIAM Journal on Scientific and Statistical Computing, Vol. 13, Issue 3
Geostatistical reducedorder models in underdetermined inverse problems: Groms in Underdetermined Inverse Problems
journal, October 2013
 Liu, Xiaoyi; Zhou, Quanlin; Birkholzer, Jens
 Water Resources Research, Vol. 49, Issue 10
A hybrid regularized inversion methodology for highly parameterized environmental models: HYBRID REGULARIZATION METHODOLOGY
journal, October 2005
 Tonkin, Matthew James; Doherty, John
 Water Resources Research, Vol. 41, Issue 10
Works referencing / citing this record:
Quantifying model structural error: Efficient Bayesian calibration of a regional groundwater flow model using surrogates and a datadriven error model: CALIBRATING WITH STRUCTURAL ERROR
journal, May 2017
 Xu, Tianfang; Valocchi, Albert J.; Ye, Ming
 Water Resources Research, Vol. 53, Issue 5
Quantifying model structural error: Efficient Bayesian calibration of a regional groundwater flow model using surrogates and a datadriven error model: CALIBRATING WITH STRUCTURAL ERROR
journal, May 2017
 Xu, Tianfang; Valocchi, Albert J.; Ye, Ming
 Water Resources Research, Vol. 53, Issue 5