Enhancing speech technology accessibility for individuals with Parkinson’s

Zheng, Xiuwen

Permalink

https://hdl.handle.net/2142/127266 Copy

Description

Title

Enhancing speech technology accessibility for individuals with Parkinson’s

Author(s)

Zheng, Xiuwen

Issue Date

2024-12-06

Director of Research (if dissertation) or Advisor (if thesis)

Hasegawa-Johnson, Mark Allan

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Accessibility
• Automatic Speech Recognition
• Dysarthria

Language

eng

Abstract

Accessibility is a human right. While automatic speech recognition (ASR) has been widely used in our daily life, it struggles when recognizing dysarthric and dysphonic speech, due to acoustic impairment and lack of available training data. This thesis aims to enhance speech accessibility for people with Parkinson's, by fine-tuning pre-trained ASR systems using the 2023-10-05 data package collected by the Speech Accessibility Project (SAP), which includes speech data from 253 individuals with Parkinson's disease. The proposed method significantly outperforms a baseline model fine-tuned with typical speech, yielding improvements in word error rate of 45.15% to 38.76% compared to models fine-tuned with 100 hours and 960 hours of Librispeech data, respectively. Furthermore, this research explores cluster-dependent fine-tuning and multi-task learning methods that have shown effectiveness in the previous research in addressing speech impairments associated with Cerebral Palsy. The most promising results were obtained using a multi-task learning approach, in which the ASR model is trained to predict the speaker's impairment severity as an auxiliary task.

Graduation Semester

2024-12

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/127266

Copyright and License Information

Copyright 2024 Xiuwen Zheng

Owning Collections