Files in this item

FilesDescriptionFormat

application/pdf

application/pdfCAI-THESIS-2015.pdf (947kB)Restricted to U of Illinois
(no description provided)PDF

Description

Title:Phurti: application and network-aware flow scheduling for MapReduce
Author(s):Cai, Xiao
Department / Program:Computer Science
Discipline:Computer Science
Degree Granting Institution:University of Illinois at Urbana-Champaign
Degree:M.S.
Genre:Thesis
Subject(s):Network
MapReduce
Abstract:Traffic for a typical MapReduce job in a datacenter consists of multiple network flows. Traditionally, network resources have been allocated to optimize network-level metrics such as flow completion time or throughput. Some recent schemes propose using application-aware scheduling which can reduce the average job completion time. However, most of them treat the core network as a black box with sufficient capacity. Even if only one network link in the core network becomes a bottleneck, it can hurt application performance. We design and implement a centralized flow scheduling framework called Phurti with the goal of decreasing the completion time for Hadoop MapReduce jobs. Phurti communicates both with the Hadoop framework to retrieve job-level network traffic information and the OpenFlow-based switches to learn about network topology. Phurti implements a novel heuristic called Smallest Maximum Sequential-traffic First (SMSF) that uses collected application and network information to perform traffic scheduling for MapReduce jobs. Our evaluation with real Hadoop workloads shows that compared to application and network-agnostic scheduling strategies, Phurti improves job completion time for 95% of the jobs, decreases average job completion time by 20% and tail job completion time by 13%.
Issue Date:2015-04-03
Type:Thesis
URI:http://hdl.handle.net/2142/78595
Rights Information:Copyright 2015 Xiao Cai
Date Available in IDEALS:2015-07-22
Date Deposited:May 2015


This item appears in the following Collection(s)

Item Statistics