Convergence of a Constrained Vector Extrapolation Scheme
Abstract
Among extrapolation methods, Anderson acceleration (AA) is a popular technique for speeding up convergence of iterative processes toward their limit points. AA proceeds by extrapolating a better approximation of the limit using a weighted combination of previous iterates. Whereas AA was originally developed in the context of nonlinear integral equations, or to accelerate the convergence of iterative methods for solving linear systems, it is also used to extrapolate the solution of nonlinear systems. Simple additional stabilization strategies can be used in this context to control conditioning issues. In this work, we study a constrained vector extrapolation scheme based on an offline version of AA with fixed window size, for solving nonlinear systems arising in optimization problems, where the stabilization strategy consists in bounding the magnitude of the extrapolation weights. We provide explicit convergence bounds for this method and, as a by-product, upper bounds on a constrained version of the Chebyshev problem on polynomials.