Data-driven bimanual catching: Integrating model-based and learning-based approaches for dynamic object grasping
Xu, Yiyang
Loading…
Permalink
https://hdl.handle.net/2142/127288
Description
Title
Data-driven bimanual catching: Integrating model-based and learning-based approaches for dynamic object grasping
Author(s)
Xu, Yiyang
Issue Date
2024-12-12
Director of Research (if dissertation) or Advisor (if thesis)
Driggs-Campbell, Katherine
Department of Study
Electrical & Computer Eng
Discipline
Electrical & Computer Engr
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
Bimanual Robotic Arms
Robotic Catching
Language
eng
Abstract
This thesis presents a novel approach to dynamic object catching using a bimanual robotic system, which integrates both model-based and data-driven methods. Traditional unimanual robotic manipulation systems have limitations in handling large, complex, fast-moving, or irregularly shaped objects. Catching with two hands is more intuitive to humans as it provides greater stability and balance compared to using one hand. We aim to have the robotic system mimic human motion patterns, making it capable of handling a broader range of tasks. We propose a teacher-student policy training framework consisting of a model-based teacher policy and a learning-based student policy. Both policies take raw camera input and output the target grippers' poses for catching objects in flight. The model-based approach utilizes a sampling based technique to generate optimal grasp candidates, and it serves as both a baseline and a foundation for data collection used by the data-driven system. The data-driven approach incorporates PointNet++ and a Transformer encoder to process spatial-temporal point cloud data. Additionally, the network is combined with a physics-based ballistic motion model to estimate the object’s landing position. The deep learning model predicts the 6D rotation for both grippers and 3D translation residuals compensated for object-specific dynamic not captured by the physics-based model. The proposed data-driven system is evaluated in a simulated environment, and it demonstrates significant improvements in grasp success rates and generalization across various object types compared to the baseline model-based method. The results highlight the potential for deploying such a system in real-world applications.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
