Files in this item



application/pdfBENSON-THESIS-2018.pdf (551kB)Restricted to U of Illinois
(no description provided)PDF


Title:Improving cache replacement policy using deep reinforcement learning
Author(s):Benson, Christopher Edward
Advisor(s):Peng, Jian
Department / Program:Computer Science
Discipline:Computer Science
Degree Granting Institution:University of Illinois at Urbana-Champaign
Subject(s):Reinforcement Learning
Machine Learning
Deep Learning
Abstract:This thesis explores the use of reinforcement learning approaches to improve replacement policies of caches. In today's internet, caches play a vital role in improving performance of data transfers and load speeds. From video streaming to information retrieval from databases, caches allow applications to function more quickly and efficiently. A cache's replacement policy plays a major role in determining the cache's effectiveness and performance. The replacement policy is an algorithm that chooses which piece of data in the cache should be evicted when the cache becomes full and new elements are requested. In computer systems today, most caches use simple heuristic-based policies. Currently used policies are effective but are still far from optimal. Using more optimal cache replacement policies could dramatically improve internet performance and reduce database costs for many industry-based companies. This research examines learning more optimal replacement policies using reinforcement learning. In reinforcement learning, an agent learns to take optimal actions given information about an environment and a reward signal. In this work, deep reinforcement learning algorithms are trained to learn optimal cache replacement policies using a simulated cache environment and database access traces. This research presents the idea of using index-based cache access histories as input data for the reinforcement learning algorithms instead of content-based input. Several approaches are explored including value-based algorithms and policy gradient algorithms. The work presented here also explores the idea of using imitation learning algorithms to mimic optimal cache replacement policies. The algorithms are tested on several different cache sizes and data access patterns to show that these learned policies can outperform currently used replacement policies in a variety of settings.
Issue Date:2018-12-13
Rights Information:Copyright 2018 Christopher Benson
Date Available in IDEALS:2019-02-07
Date Deposited:2018-12

This item appears in the following Collection(s)

Item Statistics