基于相关性分析的跨媒体检索

发布时间：2018-10-31 13:35

【摘要】：为了更加符合人脑对视觉、听觉等不同感官认知和感知外界信息的综合处理模式,使计算机能够模拟人脑对不同类型的多媒体数据进行认知、学习以及推理决策等一些信息加工的过程,于是“跨媒体”技术应运而生。随着互联网上多媒体数据的多样化和复杂化,实现跨媒体数据间准确有效地的相互检索变得愈发重要。跨媒体检索目前主要面临两方面的挑战:一方面,由于不同类型的多媒体数据在其底层特征表示上存在特征维数和属性上的不同,导致彼此之间存在异构性问题,即跨媒体异构鸿沟;另一方面,不同类型多媒体数据的底层特征与其高层语义间可能存在不一致性问题,即跨媒体语义鸿沟。为了解决这两个难题,需要针对跨媒体数据间存在的复杂关联关系进行深度挖掘与分析。本文对深度典型相关性分析进行结构改进,并将其应用在跨媒体数据上构建跨媒体相关性学习模型。深度典型相关性分析(DCCA)是一种将文本和图像对映射到公共潜在子空间进行相似性度量的深度方法。本文采用的方法对传统的DCCA进行结构改进,将网络中第一层隐藏层改造为线性投影损失层。线性投影层的训练是同非线性隐藏层的训练结合在一起的,以确保线性投影能够很好地与非线性处理阶段进行匹配,同时,能够从网络的输出端获取原始输入数据的更加抽象和更加准确的表示。然后通过跨媒体相关性分析挖掘出跨媒体数据间存在的复杂相关关系。在跨媒体检索的潜在空间中,语义上一致的图像和文本应该是彼此互相靠近的,但是CCA及其扩展方法仅仅最大化图像及对应文本之间的相关性并不能满足这个要求。因此,本文提出了一个基于深度相关性分析的跨媒体检索方法(CMSCR)。其通过自动探索语义标签对跨媒体相关性学习所得到的图像与文本最大相关子空间训练语义映射,从而得到图像语义空间及文本语义空间。然后通过相似性度量方法来计算文本和图像之间的相关性,从而实现跨媒体检索。本文最后对跨媒体系统的各个方面分别进行设计与介绍,对跨媒体检索的各种流行算法进行分类和整理。此外设计了一个跨媒体相关性检索引擎并通过示例展示其用户界面及检索结果。
[Abstract]:In order to accord with the integrated processing mode of human brain to different senses, such as vision and hearing, and to perceive the external information, the computer can simulate the human brain's cognition of different types of multimedia data. Learning and reasoning decision-making and other processes of information processing, so "cross-media" technology came into being. With the diversification and complexity of multimedia data on the Internet, it becomes more and more important to realize the accurate and effective cross-media data retrieval. Cross-media retrieval is currently facing two challenges: on the one hand, due to the difference of feature dimension and attribute in the underlying feature representation of different types of multimedia data, there is heterogeneity between them. That is, cross-media heterogeneity gap; On the other hand, there may be inconsistency between the underlying features of different types of multimedia data and their high-level semantics, that is, the cross-media semantic gap. In order to solve these two problems, it is necessary to deeply mine and analyze the complex association relationship between cross-media data. In this paper, the depth canonical correlation analysis is improved and applied to cross-media data to construct cross-media correlation learning model. Depth canonical correlation analysis (DCCA) is a depth method for measuring similarity between text and image pairs mapping to a common potential subspace. The method adopted in this paper improves the structure of the traditional DCCA and transforms the first hidden layer of the network into the linear projection loss layer. The training of the linear projection layer is combined with the training of the nonlinear hidden layer to ensure that the linear projection can be well matched with the nonlinear processing stage, at the same time, A more abstract and accurate representation of the original input data can be obtained from the output of the network. Then, the complex correlation between cross-media data is mined through cross-media correlation analysis. In the potential space of cross-media retrieval, semantically consistent images and texts should be close to each other, but CCA and its extension methods can not satisfy this requirement only by maximizing the correlation between images and corresponding texts. Therefore, a cross-media retrieval method based on depth correlation analysis (CMSCR).) is proposed in this paper. By automatically exploring semantic tags for cross-media relevance learning, the image and text maximal correlation subspace is trained to train the semantic mapping, thus the image semantic space and the text semantic space are obtained. Then the correlation between text and image is calculated by similarity measurement, and cross-media retrieval is realized. In the end, this paper designs and introduces each aspect of cross media system, classifies and arranges the popular algorithms of cross-media retrieval. In addition, a cross-media correlation retrieval engine is designed and its user interface and retrieval results are demonstrated by an example.
【学位授予单位】：中国科学院大学(中国科学院工程管理与信息技术学院)
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.3

【相似文献】