2026-09-18T19:14:59Z https://www.ideals.illinois.edu/oai-pmh

oai:www.ideals.illinois.edu:2142/121384 2023-12-13 col_2142_5131 col_2142_14787 com_2142_5130 com_2142_9630 com_2142_234

This Thesis was approved for publication on 2023-07-21 at 16:57. DSpace SAF Submission Ingestion Package generated from Vireo submission #19772 on 2023-12-04 at 17:18:54 Salapaka, Srinivasa M 2023-08 application/pdf en text Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2025-08-01 The student, Mustafa Kapadia, accepted the attached license on 2023-07-21 at 15:35. The student, Mustafa Kapadia, submitted this Thesis for approval on 2023-07-21 at 15:37. Maximum entropy on-policy reinforcement learning with monotonic policy improvement Kapadia, Mustafa 2023-07-21 Entropy Maximization Deep Reinforcement Learning Natural Policy Gradient Methods Combinatorial Optimization This thesis focuses on the utilization of the maximum entropy framework to train policies, which are renowned for their superior exploration and robustness, even in the presence of model and estimation errors. Our work encompasses the development of a theoretical foundation and a sample-based on-policy reinforcement learning algorithm based on the Maximum Entropy Principle (MEP). This algorithm ensures a consistent and monotonic improvement of policies across iterations, regardless of the initial policy. Furthermore, our theoretical advancements provide a framework for extending the solution of Paramterized Markov Decision Processes (ParaMDP) to address state and action spaces that were previously considered intractably large. We establish the necessary criteria for a well-posed maximum-entropy reinforcement learning problem in scenarios with an extensive number of states and actions, as well as infinite-horizon MDPs without a cost-free termination state. By incorporating the entropy over state action trajectories (or paths) into the objective function, we derive performance-estimation error bounds under MEP. This analysis involves drawing parallels and extending existing methods for on-policy reinforcement learning to cases where entropy maximization is added to the objective of the underlying optimization problem. We also introduce and analyze an ideal conservative policy iteration algorithm under MEP, and derive a practical sample-based algorithm that guarantees monotonic improvement. To evaluate the learning performance of our proposed algorithm, we conduct experiments on both continuous-control and discrete-control benchmark problems. We observe that resulting algorithms monotonic improvement with iterations and the training curve exhibits an O(1/T ) nature, where T are the number of iterations. Text eng https://hdl.handle.net/2142/121384 Copyright 2023 Mustafa Kapadia M.S. Thesis Mechanical Engineering University of Illinois at Urbana-Champaign Mechanical Sci & Engineering