2026-09-19T06:30:28Z https://www.ideals.illinois.edu/oai-pmh

oai:www.ideals.illinois.edu:2142/46909 2023-07-11 col_2142_5131 col_2142_8888 com_2142_5130 com_2142_8887 com_2142_234

Meyn, Sean P. Meyn, Sean P. Hajek, Bruce Hutchinson, Seth A. Nedich, Angelia Chen, Wei 2014-01-16T18:26:02Z 2014-01-16T18:26:02Z 2016-01-16T11:02:04Z 2013-12 2014-01-16T18:26:02Z 2013-12 Neuro-dynamic programming is a class of powerful techniques for approximating the solution to dynamic programming equations. In their most computationally attractive formulations, these techniques provide the approximate solution only within a prescribed finite-dimensional function class. Thus, the question that always arises is how should the function class be chosen? In this dissertation, we first propose an approach using the solutions to associated fluid and diffusion approximations. In order to evaluate this approach, we establish bounds on the approximation errors. Next, we propose a novel parameterized Q-learning algorithm. Q-learning is a model-free method to compute the Q-function associated with an optimal policy, based on observations of states and actions. If the size of a state or a policy space is too large, Q-learning is often not very practical because there are too many Q-function values to update. One way to address this problem is to approximate the Q-function within a function class. However, such methods often require an explicit model of the system, such as the split sampling method introduced by Borkar. The proposed algorithm is a reinforcement learning (RL) method, in which case the system dynamics are not known. This method is designed based on using approximations of the transition kernel of the Markov decision process (MDP). Lastly, we apply the proposed results of value function approximation techniques to several applications. In the power management model, we focus on the processor speed control problem to balance the performance and energy usage. Then we extend the results to the load balancing and the power management problem of geographically distributed data centers with grid regulation. In the cross-layer wireless control problem, the network utility maximization (NUM) and adaptive modulation (AM) are combined to balance the network performance and transmission power. In these applications, we show how to model the real problems by using the MDP model with reasonable assumptions and necessary approximations. Approximations of the value function are obtained for specific models, and evaluated by getting bounds for the errors. These approximate solutions are then used to construct basis functions for learning algorithms in the simulations. Item withdrawn by Laura Spradlin (lspradl2@illinois.edu) on 2013-12-02T14:33:51Z Item was in collections: University of Illinois Theses & Dissertations (ID: 1) No. of bitstreams: 1 Chen_Wei.pdf: 7488417 bytes, checksum: 8821f75e096c84203444be0e87ffaed1 (MD5) Made available in DSpace on 2014-01-16T18:26:02Z (GMT). No. of bitstreams: 2 Wei_Chen.pdf: 7488417 bytes, checksum: 8821f75e096c84203444be0e87ffaed1 (MD5) license.txt: 4058 bytes, checksum: a1abb7c4bbdb0836afe5bb4e3873ae50 (MD5) Item marked as restricted to the 'Administrator' Group (id=1) by Seth Robbins (robbins.sd@gmail.com) on 2014-01-16T18:27:35Z Item is restricted until 2016-01-16T18:27:27Z Restriction data tranferred 2014-07-01T11:36:47-05:00 Original Data Group with Access Administrator Release Date: 2016-01-16 12:27:27 UTC Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system Limited Restriction Lifted for Item 46928 on 2016-01-16T11:02:04Z. http://hdl.handle.net/2142/46909 en Copyright 2013 Wei Chen Neuro-Dynamic Programming Parametric Q-learning Value Function Approximation Processor Power Management Data Center Power Management Cross-Layer Wireless Control Value function approximation architectures for neuro-dynamic programming text Electrical & Computer Eng 1933 Electrical & Computer Engr 1200 University of Illinois at Urbana-Champaign Dissertation Ph.D. PHD:Electr & Computer Eng-UIUC 10KS1200PHD