Deep feature manipulation for image synthesis and restoration
Rojas Gomez, Renan Alfredo
This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.
Permalink
https://hdl.handle.net/2142/127378
Description
Title
Deep feature manipulation for image synthesis and restoration
Author(s)
Rojas Gomez, Renan Alfredo
Issue Date
2024-12-02
Director of Research (if dissertation) or Advisor (if thesis)
Do, Minh N
Doctoral Committee Chair(s)
Do, Minh N
Committee Member(s)
Boppart, Stephen A
Gupta, Saurabh
Nguyen, Anh M
Zhao, Zhizhen
Department of Study
Electrical & Computer Eng
Discipline
Electrical & Computer Engr
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
Image Synthesis
Image Restoration
Computer Vision
Deep Learning
Signal Processing
Convolutional Neural Networks
Vision Transformers
Multiscale Image Representation
Abstract
Deep learning has revolutionized image generation by capturing the underlying statistical distribution of training data, allowing for the creation of new, realistic images through latent representation manipulation. Deep learning has also significantly advanced image restoration techniques by improving the recovery of degraded images via removing noise, improving contrast, and enhancing details more effectively than traditional techniques, leading to state-of-the-art performance. Nevertheless, deep learning's success in image synthesis and restoration is often hampered by its neglect of data properties, model architecture, and task specifics. For instance, ignoring spatial invariances in tasks like image classification and segmentation can lead to models vulnerable to minor spatial transformations. Latent representations that are not interpretable can limit the model's ability to control salient image attributes, while the lack of feature invertibility can hinder high-quality image generation. To address these issues, this work introduces novel techniques that enhance performance by considering spatial invariances, capturing meaningful features, and improving feature invertibility. We propose (i) a novel down and upsampling method in CNN models to impose perfect shift equivariance towards improved performance on image classification and segmentation, (ii) a data-adaptive framework to build provably shift-equivariant ViT models, improving performance on discriminative tasks across hierarchical architectures, (iii) a neural style transfer-based data-augmentation technique to extract strong image representations through self-supervised learning, (iv) a computationally efficient photorealistic style transfer algorithm that matches multiscale geometric representations, eliminating the need for learned features to produce high-quality results by preserving fine-grained visual details, and (v) the use of adversarially robust features as a perceptual prior to invert contracted representations back into high-quality images, outperforming standard models and unlocking their use in various image synthesis and restoration tasks.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
