Unsupervised sound separation

Tzinis, Efthymios

Permalink

https://hdl.handle.net/2142/120294 Copy

Description

Title

Unsupervised sound separation

Author(s)

Tzinis, Efthymios

Issue Date

2023-04-24

Director of Research (if dissertation) or Advisor (if thesis)

Smaragdis, Paris

Doctoral Committee Chair(s)

Smaragdis, Paris

Committee Member(s)

• Hasegawa-Johnson, Mark
• Misailovic, Sasa
• Hershey, John R

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• sound separation
• audio-visual perception
• unsupervised learning
• self-supervised learning
• speech enhancement
• efficient neural networks
• federated learning

Abstract

In this thesis, we tackle the problem of training a machine to perceive, disentangle and reconstruct independent source waveforms from a given mixture audio recording without the need of explicit supervision. The problem becomes even more apparent with current state-of-the-art approaches which are largely dependent on the existence of vast amounts of carefully curated data. In essence, this thesis presents a holistic approach on how people can develop sound separation algorithms based on neural networks which are able to scale up to multiple users, modalities and datasets without the need of annotated data. To that end, the contributions of this thesis is threefold. The first part of the thesis describes novel unsupervised and self-supervised algorithms for sound source separation problems under a wide spectrum of environmental setups. The second part aims to expand the applicability of sound separation systems using external condition information (e.g. video, text and other semantic discriminative concepts) which consists of the multi-modal aspect of this work. Finally, the last chapter presents potential obstacles towards the deployment of the aforementioned algorithms (e.g. scarcity of labels, lack of data on the same device during training, limited computational resources, reluctance of the users to share their private data, erroneous predictions, etc.) as well as proposes novel solutions which can be seamlessly integrated into their real-world implementations.

Graduation Semester

2023-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Efthymios Tzinis

Owning Collections