University of Illinois Urbana-Champaign

Adaptive batching of streams to enhance throughput and to support dynamic load balancing

Jayakumar, Anirudh

Content Files
JAYAKUMAR-THESIS-2016.pdf

Permalink

https://hdl.handle.net/2142/95402

Description

Title
Adaptive batching of streams to enhance throughput and to support dynamic load balancing
Author(s)
Jayakumar, Anirudh
Issue Date
2016-12-06
Director of Research (if dissertation) or Advisor (if thesis)
Abdelzaher, Tarek F.
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Date of Ingest
2017-03-01T15:49:29Z
Keyword(s)
  • Load-balancing
  • Apache storm
  • Stream processing
Abstract
As data permeates all disciplines, the role of big data becomes increasingly important. Sensors, IoT devices, social networks, and online transactions are all generating data that can be monitored constantly to enable a business to identify opportunity to enhance customer service and increase revenue. This need for real-time processing of big data has led to the development of frameworks for distributed stream processing in clusters. It is important for such frameworks to be resilient against variable operating conditions such as server load variation, changes in data ingestion rates, and workload characteristics. In this thesis, we explore the effects of the batch size on the performance of streaming workloads by developing an adaptive batching framework and building load-balancing algorithms on top of this framework. We explore the idea of using a combination of adaptive batching of tuples and dynamic tuple dispatching to improve the throughput and load-distribution of the workload. We show through experiments that the system is able to be resilient and robust under varying operating conditions.
Graduation Semester
2016-12
Type of Resource
text
Permalink
http://hdl.handle.net/2142/95402
Copyright and License Information
Copyright 2016 Anirudh Jayakumar

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois
Dissertations and Theses - Computer Science
Dissertations and Theses from the Siebel School of Computer Science