University of Illinois Urbana-Champaign

Unsupervised sound separation

Tzinis, Efthymios

Content Files
TZINIS-DISSERTATION-2023.pdf

Permalink

https://hdl.handle.net/2142/120294

Description

Title
Unsupervised sound separation
Author(s)
Tzinis, Efthymios
Issue Date
2023-04-24
Director of Research (if dissertation) or Advisor (if thesis)
Smaragdis, Paris
Doctoral Committee Chair(s)
Smaragdis, Paris
Committee Member(s)
  • Hasegawa-Johnson, Mark
  • Misailovic, Sasa
  • Hershey, John R
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
  • sound separation
  • audio-visual perception
  • unsupervised learning
  • self-supervised learning
  • speech enhancement
  • efficient neural networks
  • federated learning
Abstract
In this thesis, we tackle the problem of training a machine to perceive, disentangle and reconstruct independent source waveforms from a given mixture audio recording without the need of explicit supervision. The problem becomes even more apparent with current state-of-the-art approaches which are largely dependent on the existence of vast amounts of carefully curated data. In essence, this thesis presents a holistic approach on how people can develop sound separation algorithms based on neural networks which are able to scale up to multiple users, modalities and datasets without the need of annotated data. To that end, the contributions of this thesis is threefold. The first part of the thesis describes novel unsupervised and self-supervised algorithms for sound source separation problems under a wide spectrum of environmental setups. The second part aims to expand the applicability of sound separation systems using external condition information (e.g. video, text and other semantic discriminative concepts) which consists of the multi-modal aspect of this work. Finally, the last chapter presents potential obstacles towards the deployment of the aforementioned algorithms (e.g. scarcity of labels, lack of data on the same device during training, limited computational resources, reluctance of the users to share their private data, erroneous predictions, etc.) as well as proposes novel solutions which can be seamlessly integrated into their real-world implementations.
Graduation Semester
2023-05
Type of Resource
Thesis
Handle URL
https://hdl.handle.net/2142/120294
Copyright and License Information
Copyright 2023 Efthymios Tzinis

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois