Malleable parallel computing with Ray: a runtime framework for dynamic resource management in iterative solvers

Yuan, Yue

Permalink

https://hdl.handle.net/2142/129273 Copy

Description

Title

Malleable parallel computing with Ray: a runtime framework for dynamic resource management in iterative solvers

Author(s)

Yuan, Yue

Issue Date

2025-04-29

Director of Research (if dissertation) or Advisor (if thesis)

Kindratenko, Volodymyr

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Malleable MPI
• Ray
• Cloud

Abstract

With the increasing demand for scalable parallel computing, frameworks such as MPI have been widely used for scientific research. However, emerging distributed computing frameworks such as Ray provide new opportunities for more flexible and dynamic resource management. This thesis presents an MPI-like framework implemented in Ray which provides one way to migrate the MPI applications to the Ray platform. Our framework features a dynamic process allocation mechanism that empowers process migration by allowing the application’s processes to expand or shrink. This thesis also implements the V-cycle multigrid method, a widely used solver for large-scale linear systems, to evaluate the performance and scalability of our framework. Our experiments demonstrate the trade-offs between static and dynamic resource allocation, highlighting the impact on computational efficiency. We show that while MPI provides strong performance guarantees under fixed workloads, Ray’s flexibility in rank adaptation can significantly improve resource utilization in dynamic environments. Our results provide valuable insights into the feasibility of using Ray as an alternative to MPI for large-scale scientific computing, particularly in scenarios where dynamic load balancing and resource elasticity are crucial.

Graduation Semester

2025-05

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/129273

Copyright and License Information

Copyright 2025 Yue Yuan

Owning Collections