Microphone array processing for augmented listening

Abstract

Modern augmented listening technologies, such as hearing aids, smart headphones, and audio augmented reality platforms, perform poorly in noisy environments with many competing sound sources. This work explores the benefits of large microphone arrays, including novel wearable devices and distributed sensor networks, for augmented listening systems. Perceptually transparent space-time remixing filters can apply separate processing to each sound source to modify the auditory scene perceived by a listener. The design parameters and performance tradeoffs of such filters are described, with particular emphasis on the ways in which augmented listening applications differ from machine listening and telecommunication applications. Theoretical tools are developed for interaural cue preservation, delay-constrained array processing, and dynamic range compression of multiple sources. Several implementation issues are considered, including acoustic channel estimation, the design of wearable microphone arrays, the acoustic effects of the body, and models and algorithms for deformable microphone arrays. Finally, the performance of the listening system is improved by cooperative processing among many distributed devices. The proposed system would dramatically improve the performance of listening devices in noisy environments and enable new listening applications that are impossible with current technology.