University of Illinois Urbana-Champaign

Using wearable IMUs for multi-modal denoising and tracking

Wei, Yu-Lin

This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.

Permalink

https://hdl.handle.net/2142/125698

Description

Title
Using wearable IMUs for multi-modal denoising and tracking
Author(s)
Wei, Yu-Lin
Issue Date
2024-07-08
Director of Research (if dissertation) or Advisor (if thesis)
Roy Choudhury, Romit
Doctoral Committee Chair(s)
Roy Choudhury, Romit
Committee Member(s)
  • Al-Hassanieh, Haitham
  • Smaragdis, Paris
  • Srikant, Rayadurgam
  • Sabharwal, Ashutosh
Department of Study
Electrical & Computer Eng
Discipline
Electrical & Computer Engr
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
  • Multimodal denoising
  • IMU
  • indoor localization
  • wearable
Abstract
Modern earphones are equipped with microphones and inertial measurement units (IMUs). IMUs are motion sensors used to deduce human activities, such as jogging, falling, and device rotation. These sensors are integrated into many wearable devices, including smartphones, watches, and earphones. While IMUs are well-established sensors, they reveal new potential when utilized on earphone platforms, particularly on the user’s face. Earphone IMUs offer two primary advantages: (a) they can capture jaw vibrations during speech, and (b) they provide smoother and cleaner head motion data compared to lower-body movements. Exploring these opportunities, we propose two applications for earphones. First, by detecting jaw vibrations, IMUs can support the microphone in performing multimodal self-supervised speech denoising. During activities such as phone calls or voice assistant interactions, vibrations from the throat travel through the jawbone and skull, inducing a voltage in the IMU. Although this IMU data is lower resolution and more distorted than microphone recordings, it is unaffected by ambient sounds, providing a unique advantage for multi-modal speech enhancement. Specifically, we explore whether the uninterfered, yet distorted, IMU signal can aid in enhancing speech when the microphone’s signal is compromised by non-stationary ambient noise. Secondly, using the clean acceleration data from the IMU, we propose an infrastructure-free indoor dead reckoning algorithm. By leveraging the inertial sensors in both earphones and smartphones, we can estimate a user’s indoor location and gazing orientation. Additionally, by playing 3D sounds through the earphones and monitoring the user’s responses, we can recalibrate errors in location and orientation estimation. We believe this innovative combination of IMU and acoustics could mark a significant advancement towards indoor Acoustic Augmented Reality (AAR).
Graduation Semester
2024-08
Type of Resource
Thesis
Handle URL
https://hdl.handle.net/2142/125698
Copyright and License Information
Copyright 2024 Yu-Lin Wei

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois