HarmonySched: Dynamically scheduling multiple concurrent machine learning models across a heterogeneous system on chip
Pingali, Sanjana
This item's files can only be accessed by the System Administrators group.
Permalink
https://hdl.handle.net/2142/132818
Description
Title
HarmonySched: Dynamically scheduling multiple concurrent machine learning models across a heterogeneous system on chip
Author(s)
Pingali, Sanjana
Issue Date
2025-12-12
Director of Research (if dissertation) or Advisor (if thesis)
Chen, Deming
Department of Study
Electrical & Computer Eng
Discipline
Electrical & Computer Engr
Degree Granting Institution
University of Illinois Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
Accelerator
Heterogeneous
Scheduling
Contention
Machine Learning
Abstract
Increasing interest in Heterogeneous SoCs has led to the need to find more optimal ways to effectively use all the resources provided by such chips. Motivated by various AI applications, modern SoC systems integrate components such as GPUs and NPUs. Very few studies address scheduling multiple concurrent workloads and dynamically arriving workloads on these SoCs.
This study introduces a dual-layer scheduling algorithm that directs workloads to either an iGPU or NPU on the SoC using a novel machine learning algorithm that aims to maximize latency as well as throughput. The accelerator chosen (embedded GPU or NPU) performs its own scheduling. The GPU uses temporal time slicing in a queue to ensure fair resource sharing amongst workloads and the NPU executes workloads sequentially in a queue. Workloads executing on both accelerators are ordered by first priority and then deadline. Unfinished GPU workloads are re-queued to allow for better resource sharing. Experiments show that this approach leads to a 2.7x reduction in tail latencies, 1.5x improvement in throughput, and 2.38x reduction in deadline violations compared to existing schedulers.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
