Combining real-time data and nominal models for machine learning in robotic system control

Tseng, Kuan-Yu

Permalink

https://hdl.handle.net/2142/130050 Copy

Description

Title

Combining real-time data and nominal models for machine learning in robotic system control

Author(s)

Tseng, Kuan-Yu

Issue Date

2025-07-17

Director of Research (if dissertation) or Advisor (if thesis)

Dullerud, Geir E

Doctoral Committee Chair(s)

Dullerud, Geir E

Committee Member(s)

• Shamma, Jeff S
• Hauser, Kris
• West, Matthew

Department of Study

Mechanical Sci & Engineering

Discipline

Mechanical Engineering

Degree Granting Institution

University of Illinois Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• control
• autonomous systems
• machine learning
• iterative learning control
• reinforcement learning
• optimization

Abstract

In this dissertation, we focus on bridging the dynamical gap between nominal models and real systems by proposing a family of learning-based control frameworks that combine nominal model information with real-time data. The thesis presents three distinct contributions. First, we develop a gradient-based iterative learning control algorithm that enables robotic systems to learn control actions stably and efficiently. Second, we integrate a trajectory database learning technique with the algorithm to enhance the sample efficiency of learning. Third, we extend the principles from the first contribution to develop a gradient-based policy optimization algorithm for learning general control policies. In the first part of the thesis, we develop a novel hybrid gradient-based reprogrammable iterative learning control (GRILC) framework. GRILC combines gradients from the nominal model dynamics and trajectory data from the real system to optimize control actions for the real system. Additionally, a reprogrammable learning strategy is introduced, enabling learning actions with a receding horizon and directly incorporating the learned primitives into a library for future motion planning. The proposed method is applied to an autonomous time-trialing task on a one-tenth scaled racing car. The simulation and experimental results illustrate the effectiveness and robustness of the proposed approach. In the second part of this dissertation, we present a novel experience-based technique, Experience-Based Hybrid Gradient Optimization (EHGO), for sample-efficient control of robotic systems in the presence of dynamical modeling errors. EHGO begins with a database seeded with many trajectories optimized under a nominal model of the real system dynamics. When executed on the real system, these trajectories will be suboptimal due to errors in the nominal dynamics. The approach then leverages GRILC to refine the control policy. In past work, GRILC was applied in a restrictive setting in which a robot executes multiple rollouts from identical start states. In this project, we show how to leverage a database to enable GRILC to operate across a wide envelope of possible start states in different iterations. The database balances start state proximity and recentness of experience using a learned distance metric to generate high-quality initial guesses. Numerical experiments on three dynamical systems (pendulum, car, drone) show that the proposed approach adapts quickly to online experience, even when the nominal model has significant errors. In these examples, EHGO generates near-optimal solutions within hundreds of epochs of real execution, which can be orders of magnitude more sample-efficient than reinforcement learning techniques. In the third part of the thesis, we extend the GRILC framework and propose a hybrid gradient-based iterative policy optimization technique (HyGIPO), designed for efficient general control policy learning in robotic systems. We apply HyGIPO to quadcopter target and trajectory tracking problems with the controller parameterized by a deep neural network. We demonstrate HyGIPO effectiveness in both simulation and hardware experiments, and benchmark its performance against model-based and model-free control methods. In simulation, HyGIPO rapidly learns policies within a few thousand samples, showing orders of magnitude higher sample efficiency compared to reinforcement learning methods, while recovering the performance of model-based approaches with more lightweight computation. Real-world experiments further validate that HyGIPO enables robust sim-to-real transfer for real-time control and can be quickly adapted to the physical platform through a few iterative updates.

Graduation Semester

2025-08

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/130050

Copyright and License Information

Copyright 2025 Kuan-Yu Tseng

Owning Collections