Newton method for stochastic control problems - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles SIAM Journal on Control and Optimization Year : 2022

Newton method for stochastic control problems

Abstract

We develop a new iterative method based on Pontryagin principle to solve stochastic control problems. This method is nothing else than the Newton method extended to the framework of stochastic controls, where the state dynamics is given by an ODE with stochastic coefficients. Each iteration of the method is made of two ingredients: computing the Newton direction, and finding an adapted step length. The Newton direction is obtained by solving an affine-linear Forward-Backward Stochastic Differential Equation (FBSDE) with random coefficients. This is done in the setting of a general filtration. We prove that solving such an FBSDE reduces to solving a Riccati Backward Stochastic Differential Equation (BSDE) and an affine-linear BSDE, as expected in the framework of linear FBSDEs or Linear-Quadratic stochastic control problems. We then establish convergence results for this Newton method. In particular, sufficient regularity of the second-order derivative of the cost functional is required to obtain (local) quadratic convergence. A restriction to the space of essentially bounded stochastic processes is needed to obtain such regularity. To choose an appropriate step length while fitting our choice of space of processes, an adapted backtracking line-search method is developed. We then prove global convergence of the Newton method with the proposed line-search procedure, which occurs at a quadratic rate after finitely many iterations. An implementation with regression techniques to solve BSDEs arising in the computation of the Newton step is developed. We apply it to the control problem of a large number of batteries providing ancillary services to an electricity network.
Fichier principal
Vignette du fichier
article_newton_HAL.pdf (574.25 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03108627 , version 1 (13-01-2021)

Identifiers

Cite

Emmanuel Gobet, Maxime Grangereau. Newton method for stochastic control problems. SIAM Journal on Control and Optimization, 2022, 60 (5), pp.2996-3025. ⟨10.1137/21M1408567⟩. ⟨hal-03108627⟩
177 View
434 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More