基于特征层融合的多模态情感识别研究

发布时间：2018-05-07 03:21

本文选题：多模态情感识别 + 特征层融合　；参考：《南京邮电大学》2017年硕士论文

【摘要】：情感识别作为情感计算领域中的重要分支,在人工智能、人机交互等许多方面都具有广阔的应用前景。不同模态的情感识别可以优势互补,不断提高识别系统的精度,以促进产生更高质量、更加和谐的人机交互系统,使人类生活变得更加智能便捷。本文主要研究基于面部情感特征、语音情感特征、姿态情感特征以及生理信号相关情感特征的情感识别系统,具体包括:提取不同模态的情感特征、对提取的不同模态的情感特征进行融合以及最后的分类识别。论文的主要工作如下:(1)对数据库中的样本提取不同模态的特征,包括:使用二维Gabor滤波器提取面部表情样本的Gabor特征;使用OPENSMILE对包含情感的语音样本提取其综合的语音情感特征(包括:振幅、基频、共振峰等);使用Harris角点检测姿态情感样本中的兴趣点进而提取其时空特征。(2)提出一种基于遗传算法的特征层融合方法,并将该方法与基于主成分分析的特征层融合方法、基于核典型相关分析的特征层融合方法、基于核矩阵融合的特征层融合方法进行比较,实验结果表明,论文提出的方法具有更好的融合效果。(3)将通过不同方法得到的融合特征输入支持向量机(SVM)进行分类识别,实现多模态情感识别,得到多模态的情感识别结果,将多模态的情感识别结果与单一模态的情感识别结果进行比较,实验结果表明,从识别结果的角度来看,使用适当融合方法的多模态情感识别结果要比单模态情感识别结果有明显提高。
[Abstract]:As an important branch of emotional computing, emotion recognition has broad application prospects in artificial intelligence, human-computer interaction and so on. Emotion recognition of different modes can complement each other, improve the accuracy of recognition system continuously, promote the production of higher quality and more harmonious human-computer interaction system, and make human life more intelligent and convenient. This paper mainly studies the emotion recognition system based on facial emotional feature, speech emotional feature, posture emotional feature and physiological signal related emotional feature. The emotion features of different modes are fused and finally classified. The main work of this paper is as follows: (1) extracting different modal features of the samples from the database, including: using two-dimensional Gabor filter to extract the Gabor features of facial expression samples; OPENSMILE is used to extract the speech samples containing emotion for their comprehensive speech emotional characteristics (including amplitude, fundamental frequency, amplitude, fundamental frequency). Resonance peak etc.; using Harris corner to detect interest points in attitude emotional samples and then extract its spatio-temporal features.) A feature layer fusion method based on genetic algorithm is proposed, and the method is combined with the feature layer fusion method based on principal component analysis (PCA). The feature layer fusion method based on kernel canonical correlation analysis is compared with the feature layer fusion method based on kernel matrix fusion. The experimental results show that, The method proposed in this paper has better fusion effect. It classifies and recognizes the fusion feature input support vector machine (SVM) obtained by different methods to achieve multi-modal emotion recognition and obtain multi-modal emotion recognition results. The result of multi-modal emotion recognition is compared with the result of single-modal emotion recognition. The experimental results show that, from the point of view of recognition result, The result of multi-modal emotion recognition using proper fusion method is better than that of single mode emotion recognition.
【学位授予单位】：南京邮电大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TN912.34

【相似文献】