Files in this item
Files | Description | Format |
---|---|---|
application/pdf ![]() | (no description provided) |
Description
Title: | Topology-aware distributed graph processing for tightly-coupled clusters |
Author(s): | Bhatt, Mayank |
Advisor(s): | Gupta, Indranil |
Department / Program: | Computer Science |
Discipline: | Computer Science |
Degree Granting Institution: | University of Illinois at Urbana-Champaign |
Degree: | M.S. |
Genre: | Thesis |
Subject(s): | graph processing
supercomputers topology |
Abstract: | Cloud applications have burgeoned over the last few years, but they are typically written for loosely-coupled clusters such as datacenters. In this thesis we investigate how one can run cloud applications in tightly-coupled clusters and network topologies, namely super-computers. Specifically, we look at a class of distributed machine learning systems called distributed graph processing systems, and run them on NCSA Blue Waters. Partitioning the graph is key to achieving performance in distributed graph processing systems. We present new topology-aware partitioning techniques that better exploit the structure of the network topologies in supercomputers. Compared to existing work, our new Restricted Oblivious and Grid Centroid partitioning approaches produce 25-33% improvement in makespan, along with a sizable reduction in network traffic. We also discuss optimizations such as smart network buffers that further amplify the improvement. To help operators select the best graph partitioning technique, we culminate our experimental results into a decision tree. |
Issue Date: | 2018-04-24 |
Type: | Text |
URI: | http://hdl.handle.net/2142/101213 |
Rights Information: | Copyright 2018 Mayank Bhatt |
Date Available in IDEALS: | 2018-09-04 2020-09-05 |
Date Deposited: | 2018-05 |
This item appears in the following Collection(s)
-
Dissertations and Theses - Computer Science
Dissertations and Theses from the Dept. of Computer Science -
Graduate Dissertations and Theses at Illinois
Graduate Theses and Dissertations at Illinois