Representing and manipulating deformable linear objects

Dinkel, Holly M.

Permalink

https://hdl.handle.net/2142/129943 Copy

Description

Title

Representing and manipulating deformable linear objects

Author(s)

Dinkel, Holly M.

Issue Date

2025-07-18

Director of Research (if dissertation) or Advisor (if thesis)

• Bretl, Timothy
• Coltin, Brian
• Smith, Trey

Doctoral Committee Chair(s)

Bretl, Timothy

Committee Member(s)

• Amato, Nancy
• Tran, Huy
• Yim, Justin

Department of Study

Aerospace Engineering

Discipline

Aerospace Engineering

Degree Granting Institution

University of Illinois Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• robotics
• computer vision
• robotic perception
• perception
• robotic manipulation
• manipulation
• object detection
• deformable objects
• deformable object detection
• object tracking
• tracking
• deformable object tracking
• manipulating deformable objects
• prediction
• trajectory prediction

Abstract

This dissertation presents innovations in three areas necessary for robots interacting with Deformable Linear Objects (DLOs): detection, tracking, and manipulation. These skills are essential in tasks such as surgical suturing and cable management in engineering systems. Unlike rigid bodies with fixed geometry and well-studied dynamics, DLOs undergo continuous deformation; have many degrees of freedom; have difficult-to-measure material properties; and frequently experience self-occlusions, self-collisions, environmental occlusions, and environmental collisions. The complex visual and physics characteristics of DLOs complicate their modeling and control. This work distinguishes itself by presenting integrated end-to-end systems for perceiving and manipulating DLOs using cameras and robots, while much of the related work relies on methods developed or validated in simulators. Although this work does not solve the full complexity of suturing or cable management, it tackles fundamental perception and manipulation challenges that arise in these demanding applications, using real robot hardware to make progress toward more capable DLO autonomy. Detecting - COCOpen is an image dataset for instance segmentation of scenes containing both rigid objects and Ethernet cables. It features an augmentation approach that grounds synthetic images in a real world context by automatically labeling wire images that classic computer vision techniques can annotate reliably and combining them with varied background scenes. This strategy provides scalable training data for training Ethernet cable detectors at low acquisition and annotation cost. Tracking - MultiDLO is a real-time algorithm for simultaneously estimating the 3D shapes of multiple entangling DLOs from RGB-D videos, which had previously never been done. It initializes by applying instance segmentation on the first frame, then requires only background segmentation for the remainder of the sequence. TrackDLO is a real-time algorithm for estimating the 3D shape of a single DLO from RGB-D videos during challenging occlusion scenarios: tip occlusion, mid-section occlusion, and self-occlusion. It is the first method to achieve accurate tracking without privileged input from fiducial markers, physics-based simulation, or robot trajectories. DLO-Splatting is an algorithm for estimating the 3D shape of a single DLO from RGB-D videos and a gripper state for tracking DLO states through self-collisions. Manipulating - KnotDLO is a method for one-handed knot tying that is repeatable for varying rope initial planar configurations and requires no human demonstrations or training. Grasp and target waypoints for future DLO states are planned from the current DLO shape. GraphDLO is a method for direct trajectory prediction of a rope that computes the directory only from the current rope state and a planned gripper action sequence. It also includes the first large-scale, real-world labeled rope interaction dataset, collected autonomously over 300 hours using a cloud robotics platform. All code and data for COCOpen, MultiDLO, TrackDLO, DLO-Splatting, KnotDLO, and GraphDLO is publicly available to support future benchmarking in detection, tracking, and manipulation of DLOs. This work helps robots see, follow, and use DLOs, moving toward using them as tools for transforming the world.

Graduation Semester

2025-08

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/129943

Copyright and License Information

Copyright 2025 Holly M. Dinkel

Owning Collections