2026-09-20T09:58:36Z https://www.ideals.illinois.edu/oai-pmh

oai:www.ideals.illinois.edu:2142/121563 2023-12-13 col_2142_5131 col_2142_16359 com_2142_5130 com_2142_16358 com_2142_234

Nagi, Rakesh 2023-08 application/pdf en text Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-12-04 without embargo terms The student, Arkadiy Gertsman, accepted the attached license on 2023-07-20 at 16:47. The student, Arkadiy Gertsman, submitted this Thesis for approval on 2023-07-20 at 16:51. This Thesis was approved for publication on 2023-07-21 at 10:34. DSpace SAF Submission Ingestion Package generated from Vireo submission #19760 on 2023-12-04 at 17:03:37 A faster reinforcement learning approach to efficient job scheduling in Apache Spark Gertsman, Arkadiy 2023-07-21 Reinforcement Learning Graph Neural Networks Job Scheduling Apache Spark Job scheduling problems have been widely studied in theoretical computer science and operations research, and are commonly encountered in applied settings such as computer systems, manufacturing, and construction. There are many variants of job scheduling, but they share a common goal: designat- ing jobs to run on a set of parallel machines at different times, such that the machines are efficiently utilized. This thesis focuses on job scheduling in the context of Apache SparkTM, a popular data analytics engine that harnesses the power of distributed computing. Job scheduling is central to Spark, as each Spark application needs a scheduler to orchestrate its job submissions. The basic scheduling rules provided by Spark work well on lighter workloads, but sophisticated scheduling algorithms can greatly increase cluster efficiency when workloads are heavier. Previous work has introduced such algorithms, some hand-tuned and others learned. This thesis thoroughly documents Dec- ima, the state-of-the-art, reinforcement-learned Spark job scheduler, includ- ing a close look into their simulator and model architectures, and a new SMDP formulation of the problem. This thesis also proposes Decima++, an update to Decima which improves scheduling performance and reduces training time by over 11×. Text eng https://hdl.handle.net/2142/121563 Copyright 2023 Arkadiy Gertsman M.S. Thesis Industrial Engineering University of Illinois at Urbana-Champaign Industrial&Enterprise Sys Eng