Files in this item



application/pdfSP20-ECE499-Thesis-Zhang, Xiangyuan.pdf (314kB)
(no description provided)PDF


Title:H2 linear control with H-infinity robustness guarantee: A game-theoretic approach
Author(s):Zhang, Xiangyuan
Contributor(s):Basar, Tamer
Subject(s):Reinforcement learning
Policy optimization
Robust control
Abstract:In recent years, reinforcement learning (RL) has shown promising developments in solving sequential decision-making problems as well as handling continuous control tasks. Among the success stories, many are related to policy optimization (PO) algorithms, developed in the context of constrained optimization. To address the stability and robustness of the controller as the algorithm iterates, constraints such as the H-infinity norm one need to be enforced on-the-fly. Recently, Zhang et al. (2019) showed the implicit regularization and the global convergence property of PO methods for the mixed H2/H-Infinity design problem, a classic problem in the robust control literature. Despite the non-convex, non-coercive optimization landscape of the problem, iterates of PO methods are guaranteed to preserve the H-infinity norm constraint without explicit encoding, while converging to the global optimizer. In this thesis, we demonstrate that the solution of the mixed H2/H-Infinity design problem can also be obtained through solving the Nash equilibrium (NE) of a sequential zero-sum linear-quadratic (LQ) game via double-loop PO methods. Specifically, we first show that the natural policy gradient algorithm can be applied to solve the inner loop problem with a fixed outer loop control policy. Then, we establish the desired stability and global convergence properties despite the non-coercive nature of the inner loop cost function. Subsequently, the outer loop problem can also be solved using the natural policy gradient algorithm, similar to the techniques presented in Zhang et al. (2019). The connection between the mixed H2/H-Infinity design problem and the zero-sum LQ game provides a path to investigate model-free PO methods for the mixed H2/H-Infinity design problem.
Issue Date:2020-05
Date Available in IDEALS:2020-06-11

This item appears in the following Collection(s)

Item Statistics