Acoustic event, spoken keyword and emotional outburst detection

Abstract

This thesis presents work in research topics of audio detection. It first describes a system for large-scale multi-label acoustic event detection (AED) in YouTube videos. It explores the potential of the state-of-the-art deep learning classifiers for AED, describes both qualitative and quantitative results (Hit@1 is 47.9%) and presents the pre-trained embedding model as a powerful feature extractor to be adapted to new domains with limited data and improve the detection accuracy (Hit@1 is 58.1%). Second, the thesis focuses on the speech acoustic events and the spoken keyword spotting task for speech. It presents a phonetic keyword spotter as a lightweight alternative to full speech recognition (3x faster, with comparable detection rates and that addresses automatic speech recognition problems). It also explores cross-lingual keyword spotting to support low resource languages and finds that the acoustic model is dominant in determining the cross-lingual keyword search performance. Third, the thesis further presents the emotional outburst detection for infant nonspeech acoustic events. It reports on the efforts to manually code child utterances as being of type “laugh,” “cry,” “fuss,” “babble,” and “hiccup” and to develop the algorithms capable of performing the same task automatically.