H2 linear control with H-infinity robustness guarantee: A game-theoretic approach

Zhang, Xiangyuan

Permalink

https://hdl.handle.net/2142/107233 Copy

Description

Title

H2 linear control with H-infinity robustness guarantee: A game-theoretic approach

Author(s)

Zhang, Xiangyuan

Contributor(s)

Basar, Tamer

Issue Date

2020-05

Keyword(s)

• Reinforcement learning
• Policy optimization
• Robust control

Abstract

In recent years, reinforcement learning (RL) has shown promising developments in solving sequential decision-making problems as well as handling continuous control tasks. Among the success stories, many are related to policy optimization (PO) algorithms, developed in the context of constrained optimization. To address the stability and robustness of the controller as the algorithm iterates, constraints such as the H-infinity norm one need to be enforced on-the-fly. Recently, Zhang et al. (2019) showed the implicit regularization and the global convergence property of PO methods for the mixed H2/H-Infinity design problem, a classic problem in the robust control literature. Despite the non-convex, non-coercive optimization landscape of the problem, iterates of PO methods are guaranteed to preserve the H-infinity norm constraint without explicit encoding, while converging to the global optimizer. In this thesis, we demonstrate that the solution of the mixed H2/H-Infinity design problem can also be obtained through solving the Nash equilibrium (NE) of a sequential zero-sum linear-quadratic (LQ) game via double-loop PO methods. Specifically, we first show that the natural policy gradient algorithm can be applied to solve the inner loop problem with a fixed outer loop control policy. Then, we establish the desired stability and global convergence properties despite the non-coercive nature of the inner loop cost function. Subsequently, the outer loop problem can also be solved using the natural policy gradient algorithm, similar to the techniques presented in Zhang et al. (2019). The connection between the mixed H2/H-Infinity design problem and the zero-sum LQ game provides a path to investigate model-free PO methods for the mixed H2/H-Infinity design problem.

Type of Resource

text

Language

en

Permalink

http://hdl.handle.net/2142/107233

Owning Collections