Multimodal content analysis and streaming in multiview environment

Lee, Kuan-Ying

Permalink

https://hdl.handle.net/2142/129862 Copy

Description

Title

Multimodal content analysis and streaming in multiview environment

Author(s)

Lee, Kuan-Ying

Issue Date

2025-07-11

Director of Research (if dissertation) or Advisor (if thesis)

Nahrstedt, Klara

Doctoral Committee Chair(s)

• Nahrstedt, Klara
• Wang, Yu-Xiong

Committee Member(s)

• Do, Minh
• Cesar, Pablo

Department of Study

Siebel School Comp & Data Sci

Discipline

Computer Science

Degree Granting Institution

University of Illinois Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• learning-based methods
• multiview
• multimodal
• streaming

Abstract

A multiview environment where cameras are deployed to capture the same scene is essential in countless applications. However, the complexity of its content analysis and delivery requires intricate design. The complexities could be summarized into two folds: 1) what to deliver at every timestamp, and 2) how to deliver under the constraint of bandwidth. This thesis attempts to leverage learning-based algorithms to analyze the multimodal cues, including visual and audio cues, in the multiview environment to help multiview content delivery that satisfies audience preferences and needs. Based on the nature of multiview content, we divide the research scope into on-demand and real-time content analysis and delivery. We start from on-demand multiview content analysis, study multi-camera view recommendation, and move on to on-demand content delivery of 360◦ videos. Then, considering the capricious characteristics of real-time multiview content, we investigate how continual learning on pre-trained models could aid in its analysis. Finally, we investigate speaking intent detection in 360◦ video streaming, which could be leveraged to distribute limited bandwidth over 360◦ video frames and deliver a more immersive experience in virtual reality meetings. Different aspects investigated in this thesis could provide insights into how various types of multi-view content could be analyzed and delivered to satisfy users’ potential multi-faceted needs.

Graduation Semester

2025-08

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/129862

Copyright and License Information

Copyright 2025 Kuan-Ying Lee

Owning Collections