2026-09-21T12:00:09Z https://www.ideals.illinois.edu/oai-pmh

oai:www.ideals.illinois.edu:2142/125840 2026-02-03 col_2142_5131 col_2142_8888 com_2142_5130 com_2142_8887 com_2142_234

application/pdf en text Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-08-01 The student, Yicheng Sun, accepted the attached license on 2024-07-19 at 16:45. The student, Yicheng Sun, submitted this Thesis for approval on 2024-07-19 at 16:54. This Thesis was approved for publication on 2024-07-19 at 16:59. DSpace SAF Submission Ingestion Package generated from Vireo submission #21159 on 2025-02-04 at 21:26:02 Bandits in autoregressive Markov models Sun, Yicheng 2024-07-19 Katselis, Dimitrios Bandit Algorithms eng This thesis explores algorithms used in stochastic and Markovian multi-armed bandits, along with their applications in autoregressive models with a graphical structure. It begins by introducing some background of Markov chains, including concentration properties and variations of the Chernoff bounds. The work further elucidates the setup of multi-armed bandits, emphasizing fundamental concepts such as the exploration-exploitation tradeoff and regret minimization. Established algorithms in stochastic bandits, like the Upper Confidence Bound and epsilon-Greedy are analysed as well as their adaptations for Markovian environments. The thesis then introduces binary valued proccesses with a graphical structure, such as the ALARM and the BAR models, and assesses their structural implications for bandit problems. Combining the two topics, it formulates a bandit problem based on the BAR model and applies these algorithms to minimize the regret. A comprehensive analysis of various algorithms is conducted along with experimental validations. These experiments support the theoretical assertions, showing the practical robustness and effectiveness of Markovian bandit algorithms. 2024-08 Thesis https://hdl.handle.net/2142/125840 Copyright 2024 Yicheng Sun Electrical & Computer Eng Electrical & Computer Engr University of Illinois at Urbana-Champaign M.S. Thesis