Dynamic and structured scene representation for robotic manipulation

Wang, Yixuan

Permalink

https://hdl.handle.net/2142/125573 Copy

Description

Title

Dynamic and structured scene representation for robotic manipulation

Author(s)

Wang, Yixuan

Issue Date

2024-07-19

Director of Research (if dissertation) or Advisor (if thesis)

• Li, Yunzhu
• Driggs-Campbell, Katie

Committee Member(s)

Hajek, Bruce

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Robotic Manipulation
• Robot Learning
• Representation Learning

Abstract

Representation is an essential component of typical robotic manipulation frameworks. An ideal representation should be both efficient for computation and sufficient for downstream tasks. However, existing representations typically have fixed dimensions, which may not be optimal for different tasks. Therefore, in the first part of the thesis, we propose a dynamic representation that can dynamically adapt its dimensions to current the observation and the goal. Through various pile manipulation experiments, we demonstrate that dynamic representation can significantly improve the performance of robotic manipulation tasks compared to fixed- size representations. In the second part of the thesis, we propose another novel implicit representation, D$^3$Fields, that is 3D, semantic, and dynamic. Such a representation can be used for zero-shot generalizable rearrangement tasks, where the goal is specified by 2D images. We demonstrate the effectiveness of our D$^3$Fields through a wide range of robotic rearrangement tasks, including organizing shoes, collecting debris, and organizing office desks. Compared to state-of-the-art implicit 3D representations, such as FeatureNeRF and DistilledNeRF [1], [2], our D$^3$Fields is more computationally efficient and effective for novel scenes.

Graduation Semester

2024-08

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/125573

Copyright and License Information

Copyright 2024 Yixuan Wang

Owning Collections