Improvement of learning-based methods for localization of multiple sound sources

Abstract

Sound source localization is an important topic in humanmachine interacting, teleconferencing, security systems, as well as autonomous driving and robotics. While current state-of-the-art sound source localization methods allow localization of a single or a small number of sound sources in moderately reverberant environments, it is known that their performance deteriorates when the reverberation time is increased. Moreover, the localization of multiple sound sources is an even more difficult task. Learning-based sound source localization methods recently gained interest as they tend to outperform the state-of-the-art methods in multiple source localization cases in reverberant environments. Nevertheless, this branch of sound source localization methods is not yet sufficiently investigated. Therefore, this thesis is aimed to the research of such methods. Both regression-based and classification-based methods for single and multiple sound source localization in two-dimensional and three-dimensional space are investigated. Supervised and semi-supervised training strategies are researched. A dataset of tetrahedral microphone array signals is collected for the evaluation of the performance of sound source localization methods. The dissertation consist of an introduction, three chapters and general conclusions. In the introduction, the dissertation problem is formulated, the object of the research is defined and the aim of the thesis is presented. Next, the objectives of the thesis are formulated. A brief presentation of the research methodology is provided, followed by the outline of the scientific novelty of the thesis and the practical value of the research findings. Finally, the defended statements are formulated. The first chapter reveals the state of the art of sound source localization using microphone arrays and networks. In the section, most important sound source localization methods are outlined, with an emphasis on learning-based source localization methods. In the second chapter presented are the learning-based sound source localization methods suggested by the author. Specifically, the multi-layer perceptron-based method for single sound source localization in two dimensions, the convolutional neural network-based methods for multiple sound source localization in two and three dimensions and the Graph-Regularized Neural Network-based single sound source localization method. In the third chapter, the experimental setups for evaluation of the performance of the original methods, presented in the second chapter, and the results of the experimentation are presented. In the final chapter, the discussion on the experimental results is presented and the conclusions are drawn. The results of the thesis were published in six scientific publications: three papers in the reviewed scientific journals and three papers in other journals. Additionally, the results of the research were presented in five conferences.