Files in this item

FilesDescriptionFormat

application/pdf

application/pdfJANSCHPORTO-DISSERTATION-2021.pdf (6MB)Restricted to U of Illinois
(no description provided)PDF

Description

Title:Learning and decentralized control in linear switched systems
Author(s):Jansch Porto, Joao Paulo
Director of Research:Dullerud, Geir E
Doctoral Committee Chair(s):Dullerud, Geir E
Doctoral Committee Member(s):Hu, Bin; Mitra, Sayan; Salapaka, Srinivasa; West, Matthew
Department / Program:Mechanical Sci & Engineering
Discipline:Mechanical Engineering
Degree Granting Institution:University of Illinois at Urbana-Champaign
Degree:Ph.D.
Genre:Dissertation
Subject(s):Nested systems
switched control
decentralized control
MJLS
Switched Systems
Reinforcement Learning
Policy Gradient
Optimal Control
Optimization
Robust Control
Abstract:Switched systems are an important and widely studied area of control theory since their applications can be encountered in many different circumstances. Examples of such systems include robotics, power grids, networked systems, and macroeconomic models. In this dissertation, we focus on the controller synthesis problem for interconnected systems and the application to physical systems. While there are many existing control theory tools to optimize the performance of systems with a centralized structure, we are interested in developing tools that take into consideration the topology of interconnected systems to generate decentralized controllers. In addition, most of the existing literature results consider the controller synthesis problem when the user has full knowledge of the system. With this in mind, we also look into machine learning tools as a way to simplify the synthesis problem. In the first part of the thesis, we develop synthesis methods for decentralized control of switched systems with mode-dependent (more generally, path-dependent) performance specifications. This specification flexibility is important when achievable system performance varies greatly between modes, as a mode-independent specification will lead to designs that do not extract all the system performance available in each mode. More specifically, under these specifications, we derive exact conditions for existence of block lower triangular path-dependent controllers with $\ell_2$-induced norm performance. The synthesis conditions are given in the form of a semidefinite program (SDP) for both uniform and path-dependent performance bounds. Since the given synthesis conditions might become computationally intractable for large-scale systems, we also introduce a basis-based approach that allows computational complexity to be more carefully controlled. We also applied the resulting design methods to a group of miniature quadcopters, illustrating different features of the decentralized control approach, along with some practical engineering considerations. In the second part of this dissertation, we turn to policy gradient approaches with the goal of starting from some sub-optimal controller and using measurement data to optimize the control gains. Recently, policy optimization for control purposes has received renewed attention due to the increasing interest in reinforcement learning. More specifically, there have been studies on the convergence properties of policy gradient methods to learn the optimal quadratic control of linear time-invariant systems. With our overall goal of controlling switched systems, we take the first step of investigating the global convergence of gradient-based policy optimization methods for quadratic optimal control of discrete-time Markovian jump linear systems (MJLS), which are a special case of switched systems. Despite the non-convexity of the resultant problem, we are still able to identify several useful properties such as coercivity, gradient dominance, and almost smoothness. Based on these properties, we show global convergence guarantees of three types of policy optimization methods: the gradient descent method; the Gauss-Newton method; and the natural policy gradient method. We prove that all three methods converge to the optimal state feedback controller for MJLS at a linear rate if initialized at a controller which is mean-square stabilizing. Lastly, we study model-free (data-driven) approaches to the MJLS quadratic control problem. Our simulation results suggest that the data-driven versions of the three studied iterative methods can efficiently learn the optimal controller for MJLS with unknown dynamics. This work brings new insights for understanding the performance of policy gradient methods on the Markovian jump linear quadratic control problem.
Issue Date:2021-01-28
Type:Thesis
URI:http://hdl.handle.net/2142/110628
Rights Information:Copyright 2021 Joao Paulo Jansch Porto
Date Available in IDEALS:2021-09-17
Date Deposited:2021-05


This item appears in the following Collection(s)

Item Statistics