融合面部表情和语音的驾驶员路怒症识别方法研究

发布时间：2018-02-23 20:24

本文关键词： 人脸检测表情识别语音情感识别多模态融合　出处：《江苏大学》2017年硕士论文　论文类型：学位论文

【摘要】：驾驶员路怒症目前已是影响安全驾驶的一个很重要因素,它是由于交通阻塞情况下开车压力与挫折引起的驾驶员愤怒的情绪。“路怒症”驾驶员会袭击他人的汽车,恶意违反交通规则,引发交通事故。路怒症自动检测与预警技术的研究已成为主动安全驾驶技术的重要组成部分。近年来驾驶员路怒症研究得到广泛关注,但大部分的研究主要集中在从心理学、政策、法规方面如何避免怒路症的发生,而针对路怒症自动检测和识别技术的研究还比较少。情感识别领域的研究表明,人的表情和语音是表现情感的两个重要通道。因此,本文在详细分析国内外表情识别和语音情感识别以及驾驶员路怒症检测技术最新进展的基础上,结合Kinect设备所采集的红外、深度信息和语音信息,研究在驾驶条件下驾驶员人脸检测、路怒表情识别、路怒语音情感识别的问题,并提出融合表情和语音的驾驶员路怒症识别方法,最后通过实验进行验证。本文主要工作如下:(1)录制Kinect驾驶员路怒行为数据库。鉴于目前国内外没有基于Kinect较为完备的驾驶员路怒行为数据库,课题组组织并录制了包含驾驶员Infrared-D(红外和深度)信息、驾驶员面部表情Infrared-D信息、驾驶员情感语音数据库。(2)提出融合Infrared-D信息的驾驶员人脸检测方法。该方法首先通过红外和深度信息的融合得到图像中的驾驶员区域;然后,采用卷积网络人脸检测器遍历驾驶员区域图像得到驾驶员人脸的可能位置;进而使用级联的卷积网络人脸检测器进一步缩小驾驶员人脸定位区域;最后,使用NMS(Non-maximum suppression)得到驾驶员人脸最终窗口。该方法和多种现有的方法比较,取得较好的结果,在准确率和召回率平均达到97.3%和84.4%。(3)基于PCANet,提出一种融合面部Infrared-D图像的驾驶员路怒表情识别方法。该方法首先使用驾驶员面部的红外图像和深度图像训练PCANet过滤器,提取面部红外图像和深度图像的特征图,再对得到的特征图分别进行哈希编码,进而对得到的哈希编码图采用叠加操作进行融合,并对融合后的特征图提取直方图特征作为最后的情感特征;最后,采用所提取的情感特征训练SVM,进行驾驶员路怒表情和非路怒表情的识别。该驾驶员路怒表情识别方法的有效性在实验中得到验证,其准确率达到74.6%。(4)提出融合面部表情和语音信号的驾驶员路怒症识别方法。该方法首先采用多任务卷积神经网络从声音信号和说话内容两个方面识别驾驶员语音情感,然后判断驾驶员是否说话,如果不说话则将驾驶员表情识别的结果作为驾驶员路怒症检测的结果;如果说话,则将语音情感识别的结果作为驾驶员路怒症检测的结果;最后,对30s内的驾驶员表情和语音情感识别结果进行投票,投票最多的作为最终驾驶员路怒症的识别结果。
[Abstract]:Driver road rage is now an important factor affecting safe driving. It is a result of driver anger caused by driving stress and frustration in traffic jams. Road rage drivers attack other people's cars. The research on automatic detection and early warning of road rage has become an important part of active safe driving technology. In recent years, the study of driver road rage has received extensive attention. But most of the research focuses on how to avoid road rage in psychology, policy and regulation, but there are few researches on automatic detection and recognition of road rage. Human expression and speech are two important ways to express emotion. Therefore, based on the detailed analysis of the latest development of expression recognition, speech emotion recognition and driver road rage detection technology at home and abroad, this paper combines the infrared data collected by Kinect equipment. In this paper, the problems of driver's face detection, road rage expression recognition and road rage speech emotion recognition under driving condition are studied, and the identification method of driver's road rage disease by combining facial expression and speech is put forward. The main work of this paper is as follows: 1) record Kinect driver's road rage behavior database. In view of the fact that there is no complete driver road rage behavior database based on Kinect at home and abroad, The team organized and recorded the driver infrared-D (infrared and depth) information, the driver's facial expression Infrared-D information, Driver affective voice database. (2) A driver face detection method based on Infrared-D information is proposed. Firstly, the driver region in the image is obtained by the fusion of infrared and depth information. Using convolution network face detector to traverse the driver's region image to get the possible position of driver's face; then using cascaded convolution network face detector to further reduce the driver's face location area; finally, The final window of driver's face is obtained by NMS(Non-maximum expression. Compared with many existing methods, this method has good results. Based on PCANet, a road rage recognition method based on facial Infrared-D images is proposed. Firstly, the infrared and depth images of driver's face are used to train the PCANet filter. The feature images of facial infrared images and depth images are extracted, and the obtained feature images are hashing respectively, and the resulting hash coding images are fused by superposition operation. Finally, the histogram feature is extracted as the final affective feature. SVM was trained with the extracted emotional features to recognize the driver's road anger expression and non-road anger expression, and the effectiveness of the method was verified in the experiment. The method of identification of driver's road rage is put forward, which combines facial expression and speech signal. Firstly, the multi-task convolution neural network is used to recognize the driver's speech emotion from two aspects: sound signal and speech content. Then the driver is judged whether to speak, if he does not speak, the result of driver's facial expression recognition is regarded as the result of driver's road rage detection; if he speaks, the result of speech emotion recognition is regarded as the result of driver's road rage detection; finally, the result of speech emotion recognition is regarded as the result of driver's road rage detection. The results of driver's facial expression and speech emotion recognition were voted in 30s, and the result of the final driver's road rage recognition was the most.
【学位授予单位】：江苏大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TN912.34;TP391.41

【相似文献】