Withdraw
Loading…
Learning-based optimal and robust control: A policy optimization perspective
Keivan Esfahani, Darioush
Loading…
Permalink
https://hdl.handle.net/2142/129357
Description
- Title
- Learning-based optimal and robust control: A policy optimization perspective
- Author(s)
- Keivan Esfahani, Darioush
- Issue Date
- 2024-12-12
- Director of Research (if dissertation) or Advisor (if thesis)
- Dullerud, Geir E
- Hu, Bin
- Seiler, Peter J
- Doctoral Committee Chair(s)
- Dullerud, Geir E
- Hu, Bin
- Committee Member(s)
- West, Mathew
- Salapaka, Srinivasa M
- Department of Study
- Mechanical Sci & Engineering
- Discipline
- Mechanical Engineering
- Degree Granting Institution
- University of Illinois Urbana-Champaign
- Degree Name
- Ph.D.
- Degree Level
- Dissertation
- Keyword(s)
- Control Theory
- Policy Optimization
- Reinforcement Learning
- Abstract
- Reinforcement learning (RL) has achieved remarkable success in domains such as video games, the game of Go, and complex robotic tasks. Central to this success is policy optimization (PO), a subclass of RL where policies—parameterized mappings from observations to actions—are iteratively optimized to enhance system performance. PO’s flexibility, scalability, and data-driven nature make it effective for tackling challenging problems involving nonlinear dynamics, high-dimensional systems, or insufficient parameterization. These qualities also make PO a promising approach for addressing control theory problems, where traditional methods often rely on well-defined models that may not be available. Despite its broad applicability, PO-based control synthesis is inherently nonconvex, posing significant challenges for achieving strong performance guarantees. This dissertation aims to address these challenges in two key areas of control theory: optimal control and robust control. These classical areas provide a solid foundation for evaluating the effectiveness of PO methods relative to traditional approaches and serve as benchmarks for studying the sample complexity of RL algorithms, which learn to control systems through repeated interactions. The first part of this dissertation focuses on two optimal control problems. The first problem addresses the linear quadratic Gaussian (LQG) control problem through a case study on a first-order single-input single-output (SISO) system using PO methods. Despite the inherent nonconvexity of the optimization landscape, we demonstrate that by appropriately parameterizing the policy class and incorporating problem-specific considerations, global convergence can be achieved in this specific scenario. This result provides a foundation for addressing the general LQG problem. Additionally, we investigate the asymptotic stability and region of attraction (ROA) of feedback control systems that utilize large-scale neural network policies and high-dimensional observations. The second part of the dissertation focuses on two cornerstone problems in robust control: structured \(\mathcal{H}_{\infty}\)-synthesis and \(\mu\)-synthesis. Both problems are characterized by nonconvex nonsmooth optimization landscapes, making their solutions particularly challenging. While advanced nonconvex nonsmooth optimization techniques have been developed for the model-based setting, much less is known about their model-free counterparts and the associated sample complexity. In this work, we extend these approaches to the model-free setting, providing theoretical guarantees on sample complexity and convergence performance. We also conduct extensive numerical studies to validate the practical utility of the proposed algorithms. Together, these contributions establish a unified framework that addresses current challenges and paves the way for a comprehensive PO theory for synthesizing dynamical systems with guaranteed stability, robustness, and optimality. Additionally, this work highlights the complexities of solving synthesis problems under partial observations and nonlinearities, offering a vision for scalable and reliable solutions to meet the demands of modern control systems.
- Graduation Semester
- 2025-05
- Type of Resource
- Thesis
- Handle URL
- https://hdl.handle.net/2142/129357
- Copyright and License Information
- Copyright 2025 Darioush Keivan Esfahani
Owning Collections
Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at IllinoisManage Files
Loading…
Edit Collection Membership
Loading…
Edit Metadata
Loading…
Edit Properties
Loading…
Embargoes
Loading…