Files in this item

FilesDescriptionFormat

application/pdf

application/pdfAbhishek_Gupta.pdf (5MB)
(no description provided)PDF

Description

Title:Techniques for efficient high performance computing in the cloud
Author(s):Gupta, Abhishek
Director of Research:Kale, Laxmikant V.
Doctoral Committee Chair(s):Kale, Laxmikant V.
Doctoral Committee Member(s):Garzaran, Maria J.; Gupta, Indranil; Milojicic, Dejan
Department / Program:Computer Science
Discipline:Computer Science
Degree Granting Institution:University of Illinois at Urbana-Champaign
Degree:Ph.D.
Genre:Dissertation
Subject(s):High Performance Computing (HPC)
Cloud
virtualization
scheduling
malleable jobs
evaluation
benchmarking
High performance computing (HPC) in cloud
Abstract:The advantages of pay-as-you-go model, elasticity, and the flexibility and customization offered by virtualization make cloud computing an attractive option for meeting the needs of some High Performance Computing (HPC) users, especially those with emerging or sporadic demands. "Computing or Infrastructure as a service" model in cloud has tremendous potential of spreading the outreach of HPC to wider scientific and industrial community. However, this potential has largely remained unrealized. This thesis makes an attempt to study the reasons for the lack of adoption of cloud computing by HPC community and alleviate them using software techniques. We hypothesize that current clouds are suitable for some HPC applications not all applications, and for those applications, clouds can be more cost-effective compared to typical dedicated HPC platforms using intelligent scheduling of applications to platforms in cloud. Through comprehensive performance evaluation and analysis, we find that there are gaps between the characteristic traits of many HPC applications and existing cloud environments. The poor interconnect and I/O performance in cloud, network virtualization overhead, HPC-agnostic cloud schedulers, and the inherent heterogeneity and multi-tenancy in cloud are some bottlenecks for efficient HPC in cloud. Our philosophy for bridging the divide between HPC and clouds is to a) use a complementary approach of making clouds HPC-aware and HPC cloud-aware, b) consider also the unique opportunities offered by cloud for HPC, such as virtual machine (VM) consolidation and elasticity, besides addressing the challenges posed by clouds, and c) consider views of both, HPC users and cloud providers, who sometimes have conflicting objectives: users must see tangible benefits (in cost or performance) while cloud providers must be able to run a profitable business. With this philosophy, the techniques presented in this thesis, viz. HPC-aware cloud scheduling and VM placement, cloud-aware load balancing for HPC applications, and parallel runtime for enabling dynamically shrinking or expanding parallel jobs, significantly improve HPC performance and cloud resource utilization for HPC in cloud. We believe that our research will help users gain confidence in the capabilities of cloud for HPC, and enable cloud providers to run a more profitable business.
Issue Date:2014-09-16
URI:http://hdl.handle.net/2142/50718
Rights Information:Copyright 2014 Abhishek Gupta
Date Available in IDEALS:2014-09-16
Date Deposited:2014-08


This item appears in the following Collection(s)

Item Statistics