基于主角和卷积神经网络的视频情感内容分析方法研究

发布时间：2018-03-01 09:53

本文关键词： 情感分析视频内容分析主角信息卷积神经网络光流　出处：《深圳大学》2017年硕士论文　论文类型：学位论文

【摘要】：随着智能设备和科技的高速发展,每天有大量的视频被分享至互联网。为了方便地管理和组织这些海量视频数据,亟需一种自动的视频内容分析方法。传统基于内容的视频分析大多数关注于视频中发生的事件和内容,而很少去分析这些视频内容给观看者带来的心理感受。视频情感内容分析则是从观看者的角度去分析预测视频可能带来的情感。情感识别是视频内容分析中一个重要且具有巨大挑战性的课题。目前已经存在的大多数视频情感分析方法更多的集中于如何有效地提取更多特征用于情感分析。这其中有一些问题值得我们去研究,比如视频中什么信息可以被用于传递情感,同时什么样的信息能够作用于观看者并使之产生对应的情感。目前大多数的方法仅仅是采用视频的空间域信息进行情感分析,而少有人考虑到视频的时间域信息。考虑到上述问题,本文中我们提出一种新颖的基于主角和卷积神经网络的视频情感内容分析方法。本文所完成的研究工作主要包括:(1)典型的情感分析方法仅仅是考虑到了音频等较为低层的特征,而忽略了视频图像这一重要的情感信息载体。本文通过提取视频的关键帧,再通过卷积神经网络提取这些静态关键帧的图像特征并用于最终的视频情感分析过程中。考虑到并非图像所有的部分均可用于情感诱发,因此本文基于SIFT关键点从视频关键帧中提取图像块用于表征图像的情感信息。在提取完图像块特征后,本文还探索了不同的特征融合方法对于情感分析的影响。(2)从观众大多时候关注点集中在视频中的人物中得到启发,尤其是主演出现的场景,因此本文分别提出了基于人脸和主角的视频情感分析方法。具体的在关键帧提取过程中将人脸检测和人脸识别加入到关键帧提取步骤中。基于演员和主角的情感分析对视频内容有一定的要求,因此本文中还建立了一个带情感标注的视频数据库。(3)目前大多数的方法仅仅是在空间域上对视频进行情感内容分析,本文提出将光流这一重要的时间域信息用于情感分析中,具体地我们将提取的光流信息转化为RGB图像,然后采用卷积神经网络从这些光流图像中提取特征加入到情感分析方法中。通过视频光流信息可以在一定程度上获取视频对应的行为信息。而行为信息在一定程度上能够很好的刺激人们产生对应的情感。
[Abstract]:With the rapid development of intelligent equipment and technology, a large number of video sharing to the Internet every day. In order to facilitate the management and organization of these massive video data to an automatic video content analysis method. The traditional content based video analysis most focus on video events and content, and seldom go to the video content analysis bring to the viewer's psychological feelings. The analysis of video content is to analysis and prediction of emotional video may bring emotion from the viewer's perspective. Video content analysis in emotion recognition is an important and great challenge. Most of the existing methods for video emotion more focused on how to effectively extract more features for sentiment analysis. There are some problems worthy of our research, what information such as video can be used to transfer the same emotion. What kind of information to the viewer and produce the corresponding emotion. Most of the methods is just emotional analysis using the spatial domain information of the video, but few people consider the time domain information of the video. Considering the above problems, in this paper we propose a novel method of video affective content the protagonist and the convolutional neural network. Based on the research work done in this thesis mainly includes: (1) sentiment analysis methods of typical is only considering the characteristics of audio relatively low layer, while ignoring the video image as an important information carrier of emotion. This paper video key frame extraction, then through convolution neural network image feature extraction of these static key frames and for the final video emotion analysis process. Taking into account the image is not all the parts can be used for emotion induced, so this paper based on SIFT The key points are extracted from the video key frame image block is used to represent the image. In the end the emotional information extraction of image feature, this paper also explores the different characteristics of fusion methods influence on sentiment analysis. (2) get inspiration from the audience mostly concentrate on the characters in the video, especially in appearance the scene, so this paper put forward analysis method of face and emotion based on the protagonist of the video. Specific key frame extraction in face detection and face recognition process will be added to the key frame extraction step. Actors and characters of the emotion analysis has certain requirements for video based content, so this paper also set up a band emotion tagging video database. (3) most of the methods are only in the spatial domain for video affective content analysis, this paper put forward the important flow time domain information for love The sense of analysis, in particular we will extract the flow of information into the RGB image, then the convolution neural network from the optical flow image feature extraction to sentiment analysis method. The information through the video optical flow information can obtain the video corresponding to a certain extent. And behavior information can stimulate people to generate corresponding the emotion in a certain extent.

【学位授予单位】：深圳大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.41;TP183

【参考文献】