Files in this item

FilesDescriptionFormat

application/pdf

application/pdfLI-DISSERTATION-2016.pdf (5MB)
(no description provided)PDF

Description

Title:Dynamic resource provisioning for data center workloads with data constraints
Author(s):Li, Shen
Director of Research:Abdelzaher, Tarek F.
Doctoral Committee Chair(s):Abdelzaher, Tarek F.
Doctoral Committee Member(s):Gupta, Indranil; Liu, Jie; Sha, Lui
Department / Program:Computer Science
Discipline:Computer Science
Degree Granting Institution:University of Illinois at Urbana-Champaign
Degree:Ph.D.
Genre:Dissertation
Subject(s):Data Center
Dynamic Provisioning
Big Data
Scheduling
Distributed Systems
Abstract:Dynamic resource provisioning, as an important data center software building block, helps to achieve high resource usage efficiency, leading to enormous monetary benefits. Most existing work for data center dynamic provisioning target on stateless servers, where any request can be routed to any server. However, the assumption of stateless behaviors no longer holds for subsystems that subject to data constraints, as a request may depend on a certain dataset stored on a small subset of servers. Routing a request to a server without the required dataset violates data locality or data availability properties, which may negatively impact on the response times. To solve this problem, this thesis provides an unified framework consisting of two main steps: 1) determining the proper amount of resources to serve the workload by analyzing the schedulability utilization bound; 2) avoiding transition penalties during cluster resizing operations by deliberately design data distribution policies. We apply this framework to both storage and computing subsystems, where the former includes distributed file systems, databases, memory caches, and the latter refers to systems such as Hadoop, Spark, and Storm. Proposed solutions are implemented into MemCached, HBase/HDFS, and Spark, and evaluated using various datasets, including Wikipedia, NYC taxi trace, Twitter traces, etc.
Issue Date:2016-04-07
Type:Thesis
URI:http://hdl.handle.net/2142/90503
Rights Information:Copyright 2016 Shen Li
Date Available in IDEALS:2016-07-07
Date Deposited:2016-05


This item appears in the following Collection(s)

Item Statistics