图像情感感知的计算与应用研究

发布时间：2018-06-21 08:04

本文选题：情感计算 + 图像情感　；参考：《哈尔滨工业大学》2016年博士论文

【摘要】：随着计算机科学、多媒体技术以及社交网络的迅速发展,图像、视频等多媒体内容的规模呈指数式爆炸增长,处理和理解这些多媒体内容的需求日益增强。相对于底层视觉特征层,人们只能够感知和理解图像、视频的高层语义层,包括认知层和情感层。以往对图像内容分析的工作主要集中在理解图像的认知层,即描述图像的真实内容,如物体检测与识别。然而,公众对数字摄影技术的广泛使用及对图像情感表达的强烈需求,使得对图像最高语义层—情感层的分析变得越来越迫切。对图像情感层的分析,简称图像情感计算,主要目的是理解观察者看完图像后所引起的情感反应。图像情感计算的发展主要受到两大挑战的制约：一个是情感鸿沟,即“可度量的信号属性即特征与人感知该信号所期望产生的情感之间的不一致性”；另一个是人类情感感知与评估的主观性,即“由于文化背景、教育程度、社交上下文等多种因素的影响,不同观察者对同一幅图像的情感感知是主观的、不同的”。本文针对图像情感计算中的上述问题进行研究,基于艺术学相关理论,期望提取更具有判别力更容易理解的情感特征；利用社交媒体数据进行以用户为中心的个性化情感预测,探索社交媒体中影响情感感知的因素；对图像情感的分布进行建模,预测一幅图像在多位观察者中所诱发情感的分布情况；研究图像情感在计算机视觉、多媒体技术等领域的应用。具体地,本文的研究内容和主要贡献分为以下四个方面：首先,根据艺术理论的相关研究,本文提出了一种基于艺术原理的中层图像情感特征,对以图像为中心的大众化情感进行预测。艺术理论由艺术元素和艺术原理组成：艺术元素是构成艺术作品的基本元素,包括颜色、纹理等；艺术原理是用来对艺术元素进行组织与排列的规则和工具,包括平衡、强调等。现有的工作主要提取基于艺术元素的底层特征对图像的情感进行识别。这些特征容易受到组织规则的影响,并且它们与情感之间的关系很微弱。因此,艺术元素必须通过艺术原理组织排列成有意义的区域与图像,来表达特定的语义与情感。本文系统地学习、表示并实现了基于艺术原理的特征,将量化后的艺术原理串联成情感特征,用来对图像情感进行分类与回归。在Abstract、 ArtPhoto三个数据集上的实验证明了艺术原理特征的有效性。其次,利用社交媒体上的数据,本文提出了一种以用户为中心的个性化情感预测方法,首次对图像情感感知的主观性进行评价。现有的图像情感数据集都是以图像为中心的,以预测图像情感的大众化情感为目的,并且图像数量很少,不能用于个性化的情感分析。本文构造了一个基于Flickr的个性化图像情感感知的大规模数据集,命名为Image-Emotion-Social-Net (IESN),包含100多万张图像和大约8000个用户。社交网络中多种因素可以影响个性化的情感感知：视觉内容、社交上下文、时间演变、地理位置等。本文提出了迭代多任务超图学习方法对这些因素进行联合建模,并且设计了一个学习算法,实现自动优化。实验结果表明,综合考虑多种因素可以有效地提高个性化情感预测的准确率。再次,本文提出了一种以图像为中心的对图像情感的概率分布进行预测的方法,从新的角度对图像情感进行建模。在Abstract以及IESN数据集上的统计发现,尽管图像情感感知呈现出个性化的特点,但整体上也服从一定的分布。基于这一观察,本文提出了基于共享稀疏学习的方法对图像情感的概率分布进行预测,并且使用迭代重加权最小二乘进行优化。对应于离散情感和维度情感两种表示方法,本文对图像情感的离散概率分布和连续概率分布都进行了处理。此外,本文介绍了多种baseline算法。实验结果表明,共享稀疏学习取得了最优的性能。最后,本文实现了图像情感在计算机视觉与多媒体技术领域的多个应用。一个是基于多图学习的情感图像检索,与传统的基于内容的图像检索不同,本文使用多图学习的方法从情感的角度对图像进行检索,并且在3D物体检索上进行了扩充；一个是基于观察者情感分析的视频分类与推荐,提出了使用观察者观看视频时表情的变化来对视频进行分析；一个是基于情感的图像配乐,为输入图像配置表达相似情感的音乐,这可以使图像更加生动,并且带领用户进入图像想要表达的世界。通过上述研究,本文对图像情感计算的各个层面进行了深入的探索,为图像情感计算所面临的关键问题提供了切实有效的解决方案。结果表明：通过引入艺术学等相关学科的研究,可以提取出更具有判别力且容易理解的特征,从而提高图像情感识别的准确率；社交媒体中图像情感的感知是个性化的,并且受到时间演变、社交上下文等多种因素的影响,综合考虑这些因素可以显著提高情感预测的性能；从概率分布的角度对图像情感进行建模,是对个性化情感与大众化情感的折中,更符合实际情况,更具有实际意义。
[Abstract]:With the rapid development of computer science, multimedia technology and social network, the scale of multimedia content, such as image and video, is increasing exponentially. The demand for processing and understanding the multimedia content is increasing. People can only sense and understand the image, the high-level semantic layer of video, including cognition, relative to the underlying visual feature layer. The previous work on the analysis of image content is mainly focused on understanding the cognitive level of the image, that is, to describe the true content of the image, such as the object detection and recognition. However, the widespread use of digital photography and the strong demand for the emotional expression of the image make the analysis of the highest semantic layer of the image more emotional. The more urgent. The analysis of the emotional layer of the image, referred to as the image emotional calculation, is mainly to understand the emotional reaction caused by the viewer after the image. The development of the image emotion calculation is mainly restricted by the two major challenges: one is the emotional gap, that is, "the measurable signal is characteristic and human perception of the desired signal." The other is the subjectivity of human emotion perception and evaluation, that is, "the emotional perception of the same image is subjective and different from the influence of various factors such as cultural background, educational level, social context and so on." this paper studies the above problems in the image emotional calculation. Based on the theory of art, we expect to extract more perceptive emotional features, use social media data to predict the user centered personalized emotion, explore the factors that affect emotional perception in social media, model the distribution of image emotions, and predict the lure of an image in a number of observers. The distribution of emotion, the application of image emotion in computer vision, multimedia technology and other fields. Specifically, the research content and main contributions of this article are divided into four aspects: firstly, according to the related research of art theory, this paper proposes an emotional feature of middle layer image based on the principle of Art, to the image as the medium. Artistic theory consists of artistic elements and artistic principles: the elements of art constitute the basic elements of a work of art, including color, texture, etc.; the principles of art are the rules and tools used to organize and arrange the elements of art, including balance and emphasis. The existing work is mainly based on art. The underlying characteristics of the elements identify the emotions of the images. These features are easily influenced by the rules of the organization, and they have a weak relationship with the emotions. Therefore, the art elements must be organized into meaningful regions and images through the principles of art to express specific semantics and emotions. Based on the characteristics of the principle of art, the artistic principles after quantization are connected into emotional features to classify and regress the emotion of the image. The experiments on the three data sets of Abstract, ArtPhoto prove the validity of the artistic principle characteristics. Secondly, using the data on social media, this paper proposes a user centered one. The method of sexual emotion prediction is the first to evaluate the subjectivity of image emotion perception. The existing image emotional data sets are centered on the image, in order to predict the popular emotion of the image emotion, and the number of images is very small, and can not be used for personalized emotional analysis. A personalized image based on Flickr is constructed in this paper. A large dataset of emotional perception, named Image-Emotion-Social-Net (IESN), contains about 1000000 images and about 8000 users. A variety of factors in social networks can affect personalized emotional perception: visual content, social context, time evolution, geographic location, etc. This paper proposes iterative multitask hypergraph learning methods These factors are jointly modeled and a learning algorithm is designed to achieve automatic optimization. The experimental results show that a comprehensive consideration of a variety of factors can effectively improve the accuracy of personalized emotional prediction. Thirdly, this paper proposes a method for predicting the probability distribution of image emotions with image as the center, from a new angle. The image emotion is modeled. The statistics on the Abstract and the IESN data sets show that although the image emotion perception presents a personalized feature, it also obeys a certain distribution. Based on this observation, this paper proposes a method of sharing sparse learning to predict the probability distribution of image emotion, and use iterative reiteration. Weighted least squares are optimized. Corresponding to two representation methods of discrete emotion and dimension emotion, this paper deals with the discrete probability distribution and continuous probability distribution of image emotion. In addition, this paper introduces a variety of baseline algorithms. Experimental results show that shared sparse learning achieves the best performance. Finally, this paper realizes the graph. One is emotional image retrieval in the field of computer vision and multimedia technology. One is the emotional image retrieval based on multi graph learning, which is different from the traditional content based image retrieval. This paper uses the method of multi graph learning to retrieve the image from the angle of emotion, and extends the 3D physical examination cable; one is based on the view. The video classification and recommendation of the observer's emotional analysis is proposed to analyze the video using the changes of the observer to watch the video. One is the image music based on the emotion, which can express the similar emotion for the input image, which can make the image more vivid and lead the user into the world that the image wants to express. In this study, this paper makes an in-depth exploration of the various levels of image emotional computing, and provides a practical and effective solution for the key problems facing image emotion calculation. The results show that the more discriminative and easy to understand features can be extracted by introducing the related subjects of art and so on, thus improving the image. The accuracy of emotion recognition; the perceptual perception of image in social media is individualized and influenced by many factors such as time evolution, social context and so on. Considering these factors, the performance of emotion prediction can be greatly improved. The modeling of image emotion from the perspective of probability distribution is the individualized emotion and popular feeling. The compromise is more practical and practical.
【学位授予单位】：哈尔滨工业大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TP391.41

【相似文献】