基于Kinect的特定说话人跟踪系统的研究与实现

发布时间：2017-12-26 17:34

本文关键词：基于Kinect的特定说话人跟踪系统的研究与实现　出处：《延边大学》2017年硕士论文　论文类型：学位论文

【摘要】：说话人跟踪在视频会议、多媒体系统、智能监控、人机交互、机器人等领域具有重要的研究意义和广泛的应用价值。声源定位技术常被用来作为说话人跟踪的基本手段,然而声源定位技术容易受到混响、噪声的影响,并且当环境中存在多个说话人时会使说话人跟踪产生偏差。Kinect由四个麦克风组成的直线型阵列能够有效抑制噪声和消除回音,本学位论文利用Kinect传感器接收和处理说话人的语音信号,采用基于波束形成算法实现说话人跟踪。多说话人的环境下,加入说话人的声纹特征,对特定说话人的身份进行仔细确认后,再实现实时跟踪。首先,针对多说话人环境,选用基于高斯混合模型-通用背景模型(Gaussian mixture model-Universal Background Model,GMM-UBM)的说话人确认方法对说话人进行身份认证。该方法先提取出模仿人耳听觉特性的声纹特征参数一—梅尔倒谱系数(Mel Frequency Cepstral Coefficients,MFCC),然后将 GMM-UBM模型作为说话人确认的训练模型,测试阶段将测试语音与训练得到的说话人模型进行匹配,匹配得分与预先设置的阈值进行比较,最终给出判断结果。其次,采用自适应波束形成算法实现声源的定位,解决了 Kinect麦克风阵元较少情况下的说话人声源定位问题。最后,设计了特定说话人跟踪系统,主要包括三个功能模块:音频采集与处理模块、说话人确认模块和定位跟踪模块。利用Kinect for Windows SDK v1.8和Open CV以及tsVPR说话人确认引擎等工具包,对这三个模块进行了设计及实现。最终根据获取到的特定说话人的方位角,实现了特定说话人的定位跟踪功能。实验结果表明,本文设计和实现的特定说话人跟踪系统,采用自适应波束形成技术能够准确地定位特定说话人的方位。在理想环境和单一特定说话人情况下平均定位准确率为93.3%,方位角的均方根误差(Rootmean square error,RMSE)为6.4,在理想环境和多人情况下平均定位准确率为89.5%。另外,在环境噪声为30-50dB且伴有30ms和50ms混响的情况下,通过Kinect内部的噪声抑制机制,取得了单人跟踪83.35%的准确率、8.9的RMSE,多人环境跟踪特定说话人81.27%的准确率,均满足本文设定的性能指标,证明了本系统在噪声和混响室内环境下也表现出较强的鲁棒性。
[Abstract]:Speaker tracking has important research significance and wide application value in the fields of video conference, multimedia system, intelligent monitoring, human-machine interaction, robot and so on. Sound source localization technology is often used as a basic means of speaker tracking. However, localization technology is prone to be affected by reverberation and noise, and when there are multiple speakers in the environment, it will cause bias in speaker tracking. Kinect linear array composed of four microphones can effectively suppress noise and eliminate echo. In this thesis, Kinect sensor is used to receive and process speaker's speech signal and realize speaker tracking based on beamforming algorithm. Multi speaker environment, voiceprint speaker added, in particular the identity of the speaker carefully after confirmation, to achieve real-time tracking. First, in view of multi speaker environment, a speaker verification method based on Gauss mixture model Gaussian mixture model-Universal Background Model (GMM-UBM) is used to authenticate the speaker. This method firstly extracts the voiceprint parameters to mimic the human auditory characteristics - Mel cepstral coefficients (Mel Frequency Cepstral Coefficients, MFCC), then the GMM-UBM model is used as a training model of speaker recognition, speaker model testing phase will test the voice and the matching, the matching score and the preset threshold were compared. The final judgement results. Secondly, the adaptive beamforming algorithm is used to locate the sound source, and the problem of the speaker localization is solved in the case of the Kinect microphone array element. Finally, a specific speaker tracking system is designed, which mainly consists of three functional modules: audio acquisition and processing module, speaker recognition module and location tracking module. The three modules are designed and implemented by using Kinect for Windows SDK v1.8 and Open CV and tsVPR speaker confirmation engine toolkit. In the end, the location and tracking function of a specific speaker is realized based on the azimuth of the specific speaker. The experimental results show that the specific speaker tracking system designed and implemented in this paper adopts adaptive beamforming technology to locate the speaker's orientation accurately. Under ideal environment and single speaker, the average location accuracy is 93.3%, the Rootmean square error (RMSE) of azimuth is 6.4, and the average location accuracy is 89.5% under ideal environment and multi person condition. In addition, the environmental noise is 30-50dB with 30ms and 50ms under the condition of reverberation suppression mechanism, through the noise inside the Kinect, made a single tracking accuracy, 83.35% of the 8.9 RMSE, the accuracy rate of more than 81.27% of the speaker tracking environment, meet the set of indicators, it is proved that the system in noise the reverberation and the indoor environment also showed strong robustness.
【学位授予单位】：延边大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TN912.34

【相似文献】