skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Convergence analysis of Anderson-type acceleration of Richardson's iteration

Abstract

We consider here Anderson extrapolation to accelerate the (stationary) Richardson iterative method for sparse linear systems. Using an Anderson mixing at periodic intervals, we assess how this benefits convergence to a prescribed accuracy. The method, named alternating Anderson–Richardson, has appealing properties for high-performance computing, such as the potential to reduce communication and storage in comparison to more conventional linear solvers. We establish sufficient conditions for convergence, and we evaluate the performance of this technique in combination with various preconditioners through numerical examples. Furthermore, we propose an augmented version of this technique.

Authors:
ORCiD logo [1]
  1. Emory Univ., Atlanta, GA (United States). Dept. of Mathematics and Computer Science; Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). National Center for Computational Sciences
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1511931
Alternate Identifier(s):
OSTI ID: 1504993
Grant/Contract Number:  
AC05-00OR22725
Resource Type:
Accepted Manuscript
Journal Name:
Numerical Linear Algebra with Applications
Additional Journal Information:
Journal Volume: 26; Journal Issue: 4; Journal ID: ISSN 1070-5325
Publisher:
Wiley
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Anderson acceleration; fixed-point scheme; projection method; Richardson iteration

Citation Formats

Lupo Pasini, Massimiliano. Convergence analysis of Anderson-type acceleration of Richardson's iteration. United States: N. p., 2019. Web. doi:10.1002/nla.2241.
Lupo Pasini, Massimiliano. Convergence analysis of Anderson-type acceleration of Richardson's iteration. United States. doi:10.1002/nla.2241.
Lupo Pasini, Massimiliano. Wed . "Convergence analysis of Anderson-type acceleration of Richardson's iteration". United States. doi:10.1002/nla.2241. https://www.osti.gov/servlets/purl/1511931.
@article{osti_1511931,
title = {Convergence analysis of Anderson-type acceleration of Richardson's iteration},
author = {Lupo Pasini, Massimiliano},
abstractNote = {We consider here Anderson extrapolation to accelerate the (stationary) Richardson iterative method for sparse linear systems. Using an Anderson mixing at periodic intervals, we assess how this benefits convergence to a prescribed accuracy. The method, named alternating Anderson–Richardson, has appealing properties for high-performance computing, such as the potential to reduce communication and storage in comparison to more conventional linear solvers. We establish sufficient conditions for convergence, and we evaluate the performance of this technique in combination with various preconditioners through numerical examples. Furthermore, we propose an augmented version of this technique.},
doi = {10.1002/nla.2241},
journal = {Numerical Linear Algebra with Applications},
number = 4,
volume = 26,
place = {United States},
year = {2019},
month = {4}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 2 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Reducing the bandwidth of sparse symmetric matrices
conference, January 1969

  • Cuthill, E.; McKee, J.
  • Proceedings of the 1969 24th national conference on -
  • DOI: 10.1145/800195.805928

A characterization of the behavior of the Anderson acceleration on linear problems
journal, February 2013


Chebyshev semi-iterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods: Part II
journal, December 1961

  • Golub, Gene H.; Varga, Richard S.
  • Numerische Mathematik, Vol. 3, Issue 1
  • DOI: 10.1007/BF01386014

Two classes of multisecant methods for nonlinear acceleration
journal, March 2009

  • Fang, Haw-ren; Saad, Yousef
  • Numerical Linear Algebra with Applications, Vol. 16, Issue 3
  • DOI: 10.1002/nla.617

Benchmarking optimization software with performance profiles
journal, January 2002

  • Dolan, Elizabeth D.; Moré, Jorge J.
  • Mathematical Programming, Vol. 91, Issue 2
  • DOI: 10.1007/s101070100263

Numerical Analysis of Fixed Point Algorithms in the Presence of Hardware Faults
journal, January 2015

  • Stoyanov, Miroslav; Webster, Clayton
  • SIAM Journal on Scientific Computing, Vol. 37, Issue 5
  • DOI: 10.1137/140991406

GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems
journal, July 1986

  • Saad, Youcef; Schultz, Martin H.
  • SIAM Journal on Scientific and Statistical Computing, Vol. 7, Issue 3
  • DOI: 10.1137/0907058

Hierarchical Krylov and nested Krylov methods for extreme-scale computing
journal, January 2014


Alternating Anderson–Richardson method: An efficient alternative to preconditioned Krylov methods for large, sparse linear systems
journal, January 2019

  • Suryanarayana, Phanish; Pratapa, Phanisri P.; Pask, John E.
  • Computer Physics Communications, Vol. 234
  • DOI: 10.1016/j.cpc.2018.07.007

Anderson acceleration of the Jacobi iterative method: An efficient alternative to Krylov methods for large, sparse linear systems
journal, February 2016

  • Pratapa, Phanisri P.; Suryanarayana, Phanish; Pask, John E.
  • Journal of Computational Physics, Vol. 306
  • DOI: 10.1016/j.jcp.2015.11.018

Domain Decomposition Preconditioners for Communication-Avoiding Krylov Methods on a Hybrid CPU/GPU Cluster
conference, November 2014

  • Yamazaki, Ichitaro; Rajamanickam, Sivasankaran; Boman, Erik G.
  • SC14: International Conference for High Performance Computing, Networking, Storage and Analysis
  • DOI: 10.1109/SC.2014.81

Convergence Analysis for Anderson Acceleration
journal, January 2015

  • Toth, Alex; Kelley, C. T.
  • SIAM Journal on Numerical Analysis, Vol. 53, Issue 2
  • DOI: 10.1137/130919398

Preconditioning Techniques for Large Linear Systems: A Survey
journal, November 2002


Anderson Acceleration for Fixed-Point Iterations
journal, January 2011

  • Walker, Homer F.; Ni, Peng
  • SIAM Journal on Numerical Analysis, Vol. 49, Issue 4
  • DOI: 10.1137/10078356X

The Tchebychev iteration for nonsymmetric linear systems
journal, September 1977


Iterative Procedures for Nonlinear Integral Equations
journal, October 1965


Chebyshev semi-iterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods: Part I
journal, December 1961

  • Golub, Gene H.; Varga, Richard S.
  • Numerische Mathematik, Vol. 3, Issue 1
  • DOI: 10.1007/bf01386013

    Works referencing / citing this record:

    Benchmarking optimization software with performance profiles
    journal, January 2002

    • Dolan, Elizabeth D.; Moré, Jorge J.
    • Mathematical Programming, Vol. 91, Issue 2
    • DOI: 10.1007/s101070100263

    The Tchebychev iteration for nonsymmetric linear systems
    journal, September 1977


    Two classes of multisecant methods for nonlinear acceleration
    journal, March 2009

    • Fang, Haw-ren; Saad, Yousef
    • Numerical Linear Algebra with Applications, Vol. 16, Issue 3
    • DOI: 10.1002/nla.617

    Anderson Acceleration for Fixed-Point Iterations
    journal, January 2011

    • Walker, Homer F.; Ni, Peng
    • SIAM Journal on Numerical Analysis, Vol. 49, Issue 4
    • DOI: 10.1137/10078356x

    Preconditioning Techniques for Large Linear Systems: A Survey
    journal, November 2002


    Iterative Procedures for Nonlinear Integral Equations
    journal, October 1965


    Hierarchical Krylov and nested Krylov methods for extreme-scale computing
    journal, January 2014


    Convergence Analysis for Anderson Acceleration
    journal, January 2015

    • Toth, Alex; Kelley, C. T.
    • SIAM Journal on Numerical Analysis, Vol. 53, Issue 2
    • DOI: 10.1137/130919398

    Reducing the bandwidth of sparse symmetric matrices
    conference, January 1969

    • Cuthill, E.; McKee, J.
    • Proceedings of the 1969 24th national conference on -
    • DOI: 10.1145/800195.805928

    Chebyshev semi-iterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods: Part II
    journal, December 1961

    • Golub, Gene H.; Varga, Richard S.
    • Numerische Mathematik, Vol. 3, Issue 1
    • DOI: 10.1007/bf01386014

    A characterization of the behavior of the Anderson acceleration on linear problems
    journal, February 2013


    Numerical Analysis of Fixed Point Algorithms in the Presence of Hardware Faults
    journal, January 2015

    • Stoyanov, Miroslav; Webster, Clayton
    • SIAM Journal on Scientific Computing, Vol. 37, Issue 5
    • DOI: 10.1137/140991406

    GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems
    journal, July 1986

    • Saad, Youcef; Schultz, Martin H.
    • SIAM Journal on Scientific and Statistical Computing, Vol. 7, Issue 3
    • DOI: 10.1137/0907058

    Anderson acceleration of the Jacobi iterative method: An efficient alternative to Krylov methods for large, sparse linear systems
    journal, February 2016

    • Pratapa, Phanisri P.; Suryanarayana, Phanish; Pask, John E.
    • Journal of Computational Physics, Vol. 306
    • DOI: 10.1016/j.jcp.2015.11.018

    Alternating Anderson–Richardson method: An efficient alternative to preconditioned Krylov methods for large, sparse linear systems
    journal, January 2019

    • Suryanarayana, Phanish; Pratapa, Phanisri P.; Pask, John E.
    • Computer Physics Communications, Vol. 234
    • DOI: 10.1016/j.cpc.2018.07.007