University of Illinois Urbana-Champaign

Towards real-time robotics perception with continual adaptation

Qiu, Rizhao

Content Files
QIU-THESIS-2023.pdf

Permalink

https://hdl.handle.net/2142/120164

Description

Title
Towards real-time robotics perception with continual adaptation
Author(s)
Qiu, Rizhao
Issue Date
2023-05-03
Director of Research (if dissertation) or Advisor (if thesis)
  • Hauser, Kris
  • Wang, Yuxiong
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
  • Computer Vision
  • Artificial Intelligence
  • Robotics
  • Segmentation
Abstract
One of the key characteristics of intelligent beings is the ability to perceive and interact with the surrounding environment. Increasingly many learning-based methods have been proposed in attempts to equip robots with such perception power. However, most existing robotics perception algorithms are still inappropriate for practical robotics applications because they 1) do not consider computational efficiency and require a lot of computational resources or 2) assume a fixed and closed set of objects to recognize and hence fail when robots encounter previously unseen objects or repeat the same failure patterns. This thesis aims to address these issues between perception algorithms and robotics systems by developing robotics perception algorithms that are real-time for downstream tasks and can adapt during deployment time for open-world robotics. Specifically, the co-designing of 3D semantic mapping and a downstream planner and continual adaptation from few data for 2D semantic segmentation are discussed. The first work, RA-SLAM (RA-Simultaneous Localization And Mapping), designs a real-time GPU-based volumetric semantic mapping system that understands scenes geometrically and semantically. It operates at 50Hz on a consumer-grade GPU - approximately 5 times faster than CPU-based TSDF semantic reconstruction methods. Two studies are then presented for RA-SLAM: first, it can be combined with a 2D high-touch affordance segmentation model to improve disinfection trajectory planning with human-like semantic awareness. Second, different multi-class semantic representations in voxels are investigated, which leads to a variant of RA-SLAM for multi-class semantic reconstruction that balances computation-memory usage. The second work to be presented, GAPS (Guided copy-And-Paste Synthesis), enables robots to learn continually with few data via training data synthesis. GAPS dramatically boosts the novel IoU of baseline methods on established few-shot continual segmentation benchmarks by up to 80%, and maintains good performance in even more impoverished annotation settings, where only single instances of novel objects are annotated. To investigate its potential for perception in human-in-the-loop robotics application such as tele-operating robotics, a study is then carried out to simulate continual few-shot learning on real robots, which shows that existing algorithms fail dramatically when the limited training budget on robots are considered. The thesis then describes a system approach that balances training-inference resources and allows continual learning algorithms to run on the onboard computers of robots without hindering their perception capability.
Graduation Semester
2023-05
Type of Resource
Thesis
Handle URL
https://hdl.handle.net/2142/120164
Copyright and License Information
Copyright 2023 Rizhao Qiu

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois