Learning structured models for robotic manipulation of deformable objects

Li, Baoyu

Permalink

https://hdl.handle.net/2142/129326 Copy

Description

Title

Learning structured models for robotic manipulation of deformable objects

Author(s)

Li, Baoyu

Issue Date

2025-05-06

Director of Research (if dissertation) or Advisor (if thesis)

Hauser, Kris

Department of Study

Siebel School Comp & Data Sci

Discipline

Computer Science

Degree Granting Institution

University of Illinois Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Robot Learning
• Robotic Manipulation
• Deformable Object Manipulation
• Graph-Based Neural Dynamics

Abstract

Robotic manipulation of deformable objects poses significant challenges due to their complex, nonlinear behaviors, high degrees of freedom, and variable material properties. Traditional model-based techniques are often inadequate at representing these subtleties in the real world, while many learning-based strategies struggle to generalize across different object shapes, poses, and environmental conditions. This thesis presents two novel and unified frameworks that address these challenges through the structured and adaptive modeling of diverse deformable objects. The first framework, Material-Adaptive Graph-Based Neural Dynamics (AdaptiGraph), uses particle-based representations enriched with material type information and continuous physical property parameters. By incorporating these features into a graph neural network and applying a test-time few-shot adaptation strategy, AdaptiGraph delivers precise dynamics predictions and facilitates online estimation of physical properties across various rigid and soft materials. Second, a Particle-Grid Neural Dynamics framework is proposed to directly learn the dynamics of deformable objects from depth images. This approach leverages the innate geometric and physical structure of particle-grid representations to capture fine-scale features, thereby providing accurate predictions of how deformable objects evolve during robotic interactions while remaining robust even under partial observation conditions. Extensive experiments in real-world robotics settings have demonstrated that these frameworks considerably improve manipulation performance in tasks involving 1D, 2D, and 3D deformable objects, including rope straightening, granular material gathering, cloth manipulation, and plush toy relocation. Together, these contributions advance the development of data-efficient robotic systems capable of operating in unstructured environments with diverse deformable materials, while also opening avenues for further research in adaptive control and deformable object simulation.

Graduation Semester

2025-05

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/129326

Copyright and License Information

Copyright 2025 Baoyu Li

Owning Collections