Files in this item

FilesDescriptionFormat

application/pdf

application/pdfnewpaper.Ideals.Techrep.Jun7.2013.pdf (749kB)
(no description provided)PDF

Description

Title:Natjam: Eviction Policies For Supporting Priorities and Deadlines in Mapreduce Clusters
Author(s):Gupta, Indranil; Cho, Brian; Rahman, Muntasir Raihan; Chajed, Tej; Abad, Cristina L.; Roberts, Nathan; Lin, Philbert
Subject(s):Cloud Computing
Mapreduce
Scheduling
Abstract:This paper presents Natjam, a system that supports arbitrary job priorities, hard real-time scheduling, and efficient preemption for Mapreduce clusters that are resource-constrained. Our contributions include: i) smart eviction policies for jobs and for tasks, based on resource usage, task runtime, and job deadlines; and ii) a work-conserving task preemption mechanism. We incorporated Natjam into the Hadoop YARN scheduler framework (in Hadoop 0.23). We present experiments from deployments on a test cluster, Emulab and a Yahoo! commercial cluster, using both synthetic traces as well as Hadoop cluster traces we obtained from Yahoo!. Our results reveal that Natjam incurs overheads of under 7%. Under real Hadoop workloads, Natjam performs better than existing techniques.
Issue Date:2013-06-06
Genre:Article
Type:Text
Language:English
URI:http://hdl.handle.net/2142/44871
Publication Status:unpublished
Peer Reviewed:is peer reviewed
Date Available in IDEALS:2013-06-07


This item appears in the following Collection(s)

Item Statistics