University of Illinois Urbana-Champaign

Trustworthy transfer learning

Wu, Jun

Content Files
WU-DISSERTATION-2024.pdf

Permalink

https://hdl.handle.net/2142/125540

Description

Title
Trustworthy transfer learning
Author(s)
Wu, Jun
Issue Date
2024-06-27
Director of Research (if dissertation) or Advisor (if thesis)
He, Jingrui
Doctoral Committee Chair(s)
He, Jingrui
Committee Member(s)
  • Han, Jiawei
  • Zhao, Han
  • Wang, Haixun
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
  • transfer learning
  • distribution shifts
  • trustworthiness
Abstract
Deep transfer learning aims to improve the generalization performance of a learning algorithm on the target domain, by discovering and transferring latent knowledge from relevant source domains. This paradigm is motivated by the human intelligence that people have an inherent ability to apply knowledge already learned from one task to another related one. Despite the exploding interest in deep transfer learning techniques, several fundamental problems in understanding deep transfer learning remain unsolved. This thesis investigates two fundamental research questions for understanding deep transfer learning. (Q1) Knowledge Transferability: How is the source knowledge transferred to the target domain with performance guarantees? The key challenge behind knowledge transfer- ability is the data heterogeneity between source and target domains. Notably, samples within a domain can be either independent and identically distributed (IID) or non-independent and identically distributed (non-IID) in real scenarios. This leads to various underlying factors contributing to data heterogeneity across domains, such as feature distribution shifts in cross-domain image recognition, and graph structure shifts in cross-network mining. (Q2) Knowledge Trustworthiness: Can the transferred source knowledge be reliable and trustworthy for learning the prediction function on the target domain? Specifically, multiple trustworthy concerns exist in the transfer learning process, such as adversarial robustness, privacy, fairness, transparency, etc. Having both research questions in mind, we formulate the problem of trustworthy transfer learning. The goal is to learn a reliable and trustworthy prediction function on the target domain using relevant knowledge from source domains. In this thesis, we systematically develop a suite of theoretical tools and algorithms, in order to understand deep transfer learning from the perspectives of knowledge transferability and trustworthiness. Concisely, we answer the first question (Q1) by theoretically deriving the generalization performance of deep transfer learning and empirically validating its efficacy across various IID and non-IID learning tasks. Furthermore, to answer the second question (Q2), this thesis delves into various trustworthy properties of deep transfer learning, including demonstrating the adversarial robustness of deep transfer learning techniques, developing privacy-preserving transfer learning approaches, and enhancing performance fairness among participant domains. Finally, we present several promising research directions, including (i) unifying knowledge transferability across different data modalities; (ii) uncovering the correlations between data distribution shifts and model transparency, and (iii) unveiling the trade-off between knowledge transferability and trustworthiness.
Graduation Semester
2024-08
Type of Resource
Thesis
Handle URL
https://hdl.handle.net/2142/125540
Copyright and License Information
Copyright 2024 Jun Wu

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois