Efficient shallow Ritz method for 1D diffusion problems

Cai, Zhiqiang; Doktorova, Anastassia; Falgout, Robert D.; Herrera, César

doi:10.1016/j.camwa.2025.10.020

Efficient shallow Ritz method for 1D diffusion problems

Journal Article · Mon Oct 27 00:00:00 EDT 2025 · Computers and Mathematics with Applications (Oxford)

DOI:https://doi.org/10.1016/j.camwa.2025.10.020· OSTI ID:3011924

^[1]; Doktorova, Anastassia ^[1]; ^[2]; ^[1]

Purdue Univ., West Lafayette, IN (United States)
Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

This paper studies the shallow Ritz method for solving the one-dimensional diffusion problem. It is shown that the shallow Ritz method improves the order of approximation dramatically for non-smooth problems. To realize this optimal or nearly optimal order of the shallow Ritz approximation, we develop a damped block Newton (dBN) method that alternates between updates of the linear and non-linear parameters. Per each iteration, the linear and the non-linear parameters are updated by exact inversion and one step of a modified, damped Newton method applied to a reduced non-linear system, respectively. The computational cost of each dBN iteration is $$\mathcal{O}$$(n). Starting with the non-linear parameters as a uniform partition of the interval, numerical experiments show that the dBN is capable of efficiently moving mesh points to nearly optimal locations. In conclusion, to improve the efficiency of the dBN further, we propose an adaptive damped block Newton (AdBN) method by combining the dBN with the adaptive neuron enhancement (ANE) method [28].

View Accepted Manuscript (DOE)

Research Organization:: Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)

Sponsoring Organization:: National Science Foundation (NSF); USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: AC52-07NA27344

OSTI ID:: 3011924

Report Number(s):: LLNL--JRNL-862433

Journal Information:: Computers and Mathematics with Applications (Oxford), Journal Name: Computers and Mathematics with Applications (Oxford) Journal Issue: 24 Vol. 200; ISSN 0898-1221

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (28)

Neural-network-based approximations for solving partial differential equations Dissanayake, M. W. M. G.; Phan-Thien, N. Communications in Numerical Methods in Engineering, Vol. 10, Issue 3 https://doi.org/10.1002/cnm.1640100303	journal	March 1994
Rayleigh‐Ritz‐Galerkin methods for dirichlet's problem using subspaces without boundary conditions Bramble, James H.; Schatz, Alfred H. Communications on Pure and Applied Mathematics, Vol. 23, Issue 4 https://doi.org/10.1002/cpa.3160230408	journal	July 1970
Nonlinear Approximation and (Deep) $$\mathrm {ReLU}$$ Networks Daubechies, I.; DeVore, R.; Foucart, S. Constructive Approximation, Vol. 55, Issue 1 https://doi.org/10.1007/s00365-021-09548-z	journal	April 2021
The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems E., Weinan; Yu, Bing Communications in Mathematics and Statistics, Vol. 6, Issue 1 https://doi.org/10.1007/s40304-018-0127-z	journal	February 2018
Least-squares neural network (LSNN) method for scalar nonlinear hyperbolic conservation laws: Discrete divergence operator Cai, Zhiqiang; Chen, Jingshuang; Liu, Min Journal of Computational and Applied Mathematics, Vol. 433 https://doi.org/10.1016/j.cam.2023.115298	journal	December 2023
Adaptive two-layer ReLU neural network: I. Best least-squares approximation Liu, Min; Cai, Zhiqiang; Chen, Jingshuang Computers & Mathematics with Applications, Vol. 113 https://doi.org/10.1016/j.camwa.2022.03.005	journal	May 2022
Adaptive two-layer ReLU neural network: II. Ritz approximation to elliptic PDEs Liu, Min; Cai, Zhiqiang Computers & Mathematics with Applications, Vol. 113 https://doi.org/10.1016/j.camwa.2022.03.010	journal	May 2022
DGM: A deep learning algorithm for solving partial differential equations Sirignano, Justin; Spiliopoulos, Konstantinos Journal of Computational Physics, Vol. 375 https://doi.org/10.1016/j.jcp.2018.08.029	journal	December 2018
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations Raissi, M.; Perdikaris, P.; Karniadakis, G. E. Journal of Computational Physics, Vol. 378 https://doi.org/10.1016/j.jcp.2018.10.045	journal	February 2019
Deep least-squares methods: An unsupervised learning-based numerical method for solving elliptic PDEs Cai, Zhiqiang; Chen, Jingshuang; Liu, Min Journal of Computational Physics, Vol. 420 https://doi.org/10.1016/j.jcp.2020.109707	journal	November 2020
Least-squares ReLU neural network (LSNN) method for linear advection-reaction equation Cai, Zhiqiang; Chen, Jingshuang; Liu, Min Journal of Computational Physics, Vol. 443 https://doi.org/10.1016/j.jcp.2021.110514	journal	October 2021
Self-adaptive deep neural network: Numerical approximation to functions and PDEs Cai, Zhiqiang; Chen, Jingshuang; Liu, Min Journal of Computational Physics, Vol. 455 https://doi.org/10.1016/j.jcp.2022.111021	journal	April 2022
A unified deep artificial neural network approach to partial differential equations in complex geometries Berg, Jens; Nyström, Kaj Neurocomputing, Vol. 317 https://doi.org/10.1016/j.neucom.2018.06.056	journal	November 2018
Splines (with optimal knots) are better Burchard †, Hermann G. Applicable Analysis, Vol. 3, Issue 4 https://doi.org/10.1080/00036817408839073	journal	January 1974
Unicity of best mean approximation by second order splines with variable knots Barrow, D. L.; Chui, C. K.; Smith, P. W. Mathematics of Computation, Vol. 32, Issue 144 https://doi.org/10.1090/S0025-5718-1978-0481754-1	journal	January 1978
A method for the solution of certain non-linear problems in least squares Levenberg, Kenneth Quarterly of Applied Mathematics, Vol. 2, Issue 2 https://doi.org/10.1090/qam/10666	journal	January 1944
A Rapidly Convergent Descent Method for Minimization Fletcher, R.; Powell, M. J. D. The Computer Journal, Vol. 6, Issue 2 https://doi.org/10.1093/comjnl/6.2.163	journal	August 1963
On the Best Least Squares Approximation of Continuous Functions using Linear Splines with Free Knots Loach, P. D.; Wathen, A. J. IMA Journal of Numerical Analysis, Vol. 11, Issue 3 https://doi.org/10.1093/imanum/11.3.393	journal	January 1991
Artificial neural networks for solving ordinary and partial differential equations Lagaris, I. E.; Likas, A.; Fotiadis, D. I. IEEE Transactions on Neural Networks, Vol. 9, Issue 5 https://doi.org/10.1109/72.712178	journal	January 1998
Neural-network methods for boundary value problems with irregular boundaries Lagaris, I. E.; Likas, A. C.; Papageorgiou, D. G. IEEE Transactions on Neural Networks, Vol. 11, Issue 5 https://doi.org/10.1109/72.870037	journal	January 2000
An Algorithm for Least-Squares Estimation of Nonlinear Parameters Marquardt, Donald W. Journal of the Society for Industrial and Applied Mathematics, Vol. 11, Issue 2 https://doi.org/10.1137/0111030	journal	June 1963
The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems Whose Variables Separate Golub, G. H.; Pereyra, V. SIAM Journal on Numerical Analysis, Vol. 10, Issue 2 https://doi.org/10.1137/0710036	journal	April 1973
Some Special Nonlinear Least Squares Problems Osborne, M. R. SIAM Journal on Numerical Analysis, Vol. 12, Issue 4 https://doi.org/10.1137/0712044	journal	September 1975
Approximation to Data by Splines with Free Knots Jupp, David L. B. SIAM Journal on Numerical Analysis, Vol. 15, Issue 2 https://doi.org/10.1137/0715022	journal	April 1978
Variable Metric Method for Minimization Davidon, William C. SIAM Journal on Optimization, Vol. 1, Issue 1 https://doi.org/10.1137/0801001	journal	February 1991
Recovery-Based Error Estimator for Interface Problems: Conforming Linear Elements Cai, Zhiqiang; Zhang, Shun SIAM Journal on Numerical Analysis, Vol. 47, Issue 3 https://doi.org/10.1137/080717407	journal	January 2009
Plateau Phenomenon in Gradient Descent Training of RELU Networks: Explanation, Quantification, and Avoidance Ainsworth, Mark; Shin, Yeonjong SIAM Journal on Scientific Computing, Vol. 43, Issue 5 https://doi.org/10.1137/20M1353010	journal	January 2021
Active Neuron Least Squares: A Training Method for Multivariate Rectified Neural Networks Ainsworth, Mark; Shin, Yeonjong SIAM Journal on Scientific Computing, Vol. 44, Issue 4 https://doi.org/10.1137/21M1460764	journal	August 2022

Similar Records

Filtered Rayleigh-Ritz is all you need

Journal Article · Thu Mar 20 20:00:00 EDT 2025 · No journal information · OSTI ID:2549341

Reducing measurement costs by recycling the Hessian in adaptive variational quantum algorithms

Journal Article · Sun Nov 17 19:00:00 EST 2024 · Quantum Science and Technology · OSTI ID:2477881

Related Subjects

Diffusion problems
Elliptic problems
Fast iterative solvers
Neural network
Newton's method
ReLU activation
Ritz formulation

Efficient shallow Ritz method for 1D diffusion problems

Citation Formats

References (28)

Similar Records

Related Subjects