On using ray communication stack and runtime to support traditional HPC applications

Ma, Zicheng

Permalink

https://hdl.handle.net/2142/127126 Copy

Description

Title

On using ray communication stack and runtime to support traditional HPC applications

Author(s)

Ma, Zicheng

Issue Date

2024-07-23

Director of Research (if dissertation) or Advisor (if thesis)

Volodymyr, Kindratenko

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

Ray, MPI, High-performance computing,

Abstract

The increasing demand for computational resources in scientific research highlights the potential for cloud computing to support High-Performance Computing (HPC) applications traditionally designed for Message Passing Interface (MPI). This thesis investigates the feasibility of utilizing the Ray communication stack and runtime to adapt MPI-based applications for cloud environments. Through refactoring MPI functions and integrating them into a thin library compatible with Ray’s architecture, this work attempts to leverage Ray’s dynamic resource management and asynchronous task handling. While the approach shows promise in local testing environments, limitations in the Ray C++ API, particularly concerning remote object references and inter-worker communication, pose significant challenges. This work suggests potential modifications and future work to enhance compatibility and performance, thereby contributing to the development of scalable, cloud-based solutions for HPC applications.

Graduation Semester

2024-12

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/127126

Copyright and License Information

Copyright 2024 Zicheng Ma

Owning Collections