Improve and certify ML robustness by integrating exogenous information
Yang, Zhuolin
This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.
Permalink
https://hdl.handle.net/2142/130072
Description
Title
Improve and certify ML robustness by integrating exogenous information
Author(s)
Yang, Zhuolin
Issue Date
2025-05-29
Director of Research (if dissertation) or Advisor (if thesis)
Gunter, Carl
Doctoral Committee Chair(s)
Li, Bo
Committee Member(s)
Wang, Gang
Zhang, Huan
Xu, Kaidi
Department of Study
Siebel School Comp & Data Sci
Discipline
Computer Science
Degree Granting Institution
University of Illinois Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
Adversarial attacks
ML robustness
NN verification
Abstract
Several studies have shown that deep neural networks (DNNs) are vulnerable to adversarial examples, which are constructed by adding a carefully crafted noise with small magnitude onto natural inputs thus misleading DNNs. A lot of work has addressed the importance of improving the robustness of DNNs and various robustness enhancing methods have been proposed. However, most of recent works focused on improving robustness of model itself, without exploring exogenous information that are critical for model to make the correct prediction under adversarial scenarios. This thesis presents novel methodologies to improve and certify ML robustness by integrating exogenous information and leveraging various theoretical and empirical techniques on several sub-topics. To import external domain knowledge into model inference, we propose sensing-reasoning pipeline to integrate statistical learning with logical reasoning to cooperate such domain knowledge rules and develop the end-to-end certification algorithm on top of that. We also delve into analyzing the cause of adversarial transferability theoretically under teacher-student setting, providing the sufficient condition of the adversarial example’s existence in student model due to low dimensionality and neuron specialization. On the general cases, among pair of ML models, we first theoretically derive the sufficient and necessary condition on adversarial transferability - gradient similarity and model smoothness, and practically develop Transferability Reduced Smooth (TRS) regularized training algorithm on top of ensemble ML models to improve their robustness. Additionally, on certifiable robustness domain, for ensemble model, we first formally defined the robustness of ensemble w.r.t different ensemble protocol, and analyze the key factors within smoothed ensemble verifiable radius. Beyond that we propose Diversity Regularized Training (DRT) method to improve certified robustness of ensemble ML models by encouraging gradient diversity and enlarging the confidence margin. For single model certifiable robustness, we also explore the internal factors – neuron implications, to accelerate ML verification by reducing redundant branches and improving intermediate bounds in Branch-and-Bound (BaB) procedure. We believe these research works could provide key insights on exploring ML robustness from various perspective – How to gather and integrate exogenous information within models and corresponding tasks could be an interesting direction for later adversarial ML research.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
