Towards real-time robotics perception with continual adaptation

Qiu, Rizhao

Permalink

https://hdl.handle.net/2142/120164 Copy

Description

Title

Towards real-time robotics perception with continual adaptation

Author(s)

Qiu, Rizhao

Issue Date

2023-05-03

Director of Research (if dissertation) or Advisor (if thesis)

• Hauser, Kris
• Wang, Yuxiong

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Computer Vision
• Artificial Intelligence
• Robotics
• Segmentation

Abstract

One of the key characteristics of intelligent beings is the ability to perceive and interact with the surrounding environment. Increasingly many learning-based methods have been proposed in attempts to equip robots with such perception power. However, most existing robotics perception algorithms are still inappropriate for practical robotics applications because they 1) do not consider computational efficiency and require a lot of computational resources or 2) assume a fixed and closed set of objects to recognize and hence fail when robots encounter previously unseen objects or repeat the same failure patterns. This thesis aims to address these issues between perception algorithms and robotics systems by developing robotics perception algorithms that are real-time for downstream tasks and can adapt during deployment time for open-world robotics. Specifically, the co-designing of 3D semantic mapping and a downstream planner and continual adaptation from few data for 2D semantic segmentation are discussed. The first work, RA-SLAM (RA-Simultaneous Localization And Mapping), designs a real-time GPU-based volumetric semantic mapping system that understands scenes geometrically and semantically. It operates at 50Hz on a consumer-grade GPU - approximately 5 times faster than CPU-based TSDF semantic reconstruction methods. Two studies are then presented for RA-SLAM: first, it can be combined with a 2D high-touch affordance segmentation model to improve disinfection trajectory planning with human-like semantic awareness. Second, different multi-class semantic representations in voxels are investigated, which leads to a variant of RA-SLAM for multi-class semantic reconstruction that balances computation-memory usage. The second work to be presented, GAPS (Guided copy-And-Paste Synthesis), enables robots to learn continually with few data via training data synthesis. GAPS dramatically boosts the novel IoU of baseline methods on established few-shot continual segmentation benchmarks by up to 80%, and maintains good performance in even more impoverished annotation settings, where only single instances of novel objects are annotated. To investigate its potential for perception in human-in-the-loop robotics application such as tele-operating robotics, a study is then carried out to simulate continual few-shot learning on real robots, which shows that existing algorithms fail dramatically when the limited training budget on robots are considered. The thesis then describes a system approach that balances training-inference resources and allows continual learning algorithms to run on the onboard computers of robots without hindering their perception capability.

Graduation Semester

2023-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Rizhao Qiu

Owning Collections