面向用户意图的社会化图像检索

发布时间：2018-02-03 04:49

本文关键词： 图像检索用户意图社交网络用户行为表征学习　出处：《清华大学》2016年博士论文　论文类型：学位论文

【摘要】：图像检索在信息检索领域中扮演着越来越重要的角色。传统的图像检索方法由于很难获取到用户的个人信息,因此更侧重于语义鸿沟(即图像底层特征到高层语义之间的鸿沟),而缺乏对意图鸿沟(即用户内心的搜索意图和表达出来的搜索请求之间的鸿沟)的研究。近些年来兴起的社交网络为研究者获取用户个人信息,分析用户个人兴趣偏好提供了有效的途径。本文通过将用户行为、个人属性、用户关系等社交上下文信息与图像的视觉信息在图像表征、距离度量、结果重排序等图像检索中的三个主要阶段进行融合,借助对用户偏好的建模分析理解用户在图像检索中的意图,从而提出面向用户意图的社会化图像检索方案来解决图像检索中的意图鸿沟问题。本文的主要贡献如下:1.提出社交信息嵌入的图像视觉表征学习方法。本文通过多任务深度学习框架,在图像表征学习阶段将用户行为信息和图像语义信息进行融合,同时考虑了社交行为数据的稀疏性、不可靠性等问题。通过对图像分类任务与兴趣学习任务的联合优化,使得学习到的图像特征可以同时包含用户兴趣和图像语义两方面信息,从而更好地理解图像推荐与检索任务中的用户意图。2.提出知识图谱关系嵌入的图像表征学习方法。该方法针对传统基于标注信息进行图像表征学习的方法中标注信息的准确性及完整性有所欠缺的问题,将知识图谱中概念之间的多种关系信息嵌入到多标签图像表征学习任务中,并利用关系约束的回归卷积神经网络模型对两个问题进行联合优化,使得学习到的图像表征不仅能够反映图像标签信息,还可以包含概念之间的关联关系。3.提出社交信息嵌入的图像度量学习方法。在以用户为中心的应用(如图像检索、推荐等)问题中,传统的基于视觉特征的图像距离度量方式无法有效捕捉用户的实际意图。本文提出社交信息嵌入的图像距离度量学习方法,将图像对应的社交行为相似度嵌入到视觉特征空间中。对图像的视觉特征空间进行变换,使得变换后的相似度与用户的社交行为相似度达到一致。通过这种方法,可以仅利用图像的视觉特征对它们的社交相似度进行预测。同时,本工作验证了学习到的图像度量函数相比于传统的图像度量方法,能够更好地反应用户在搜索、浏览行为中的用户意图。4.提出社交行为与视觉信息融合的个性化图像重排序方法。本方法利用用户所在群组在社交平台中的行为信息,理解用户的兴趣,从而为用户提供个性化的图像搜索结果。面对社交信息的稀疏性,以及社交空间多模态信息的复杂性等问题,本方法利用随机游走模型,在基于社交信息和视觉内容的混合图上对搜索结果进行重排序,从而在给出符合用户个人偏好的搜索结果的同时保证结果的语义相关性。同时,本工作还证明了社交信息在个性化搜索中的有效性和重要性。
[Abstract]:Image retrieval plays an increasingly important role in the field of information retrieval. Traditional image retrieval methods are difficult to obtain personal information of users. Therefore, more emphasis is placed on the semantic gap (that is, the gap between the underlying features of the image and the high-level semantics). However, there is a lack of research on the intention gap (that is, the gap between the user's internal search intention and the expressed search request). In recent years, social networks have emerged to obtain user personal information for researchers. Analysis of users' personal interest preferences provides an effective way. This paper uses social context information such as user behavior, personal attributes, user relations and visual information of images in image representation, distance measurement. Results the three main stages of image retrieval such as reordering were fused to understand the intention of users in image retrieval by modeling and analyzing user preferences. In order to solve the problem of intention gap in image retrieval, a socialized image retrieval scheme oriented to user's intention is proposed. The main contributions of this paper are as follows:. 1. An image representation learning method based on social information embedding is proposed. In the phase of image representation learning, user behavior information and image semantic information are fused, and the sparsity of social behavior data is considered. Through the joint optimization of image classification task and interest learning task, the image features can include both user interest and image semantic information. In order to better understand the user's intention in image recommendation and retrieval task. 2. An image representation learning method based on knowledge map relation embedding is proposed. This method is aimed at the traditional image representation learning method based on annotated information. The accuracy and completeness of tagging information are deficient. The relational information between concepts in knowledge map is embedded into the learning task of multi-label image representation, and the two problems are jointly optimized by using the regression convolution neural network model with relation constraints. So that the image representation can not only reflect the image label information. It can also include the relationship between concepts. 3. An image metric learning method for social information embedding is proposed. In user-centered applications (such as image retrieval, recommendation, etc.). Traditional image distance measurement based on visual features can not effectively capture the actual intention of users. This paper proposes an image distance measurement learning method based on social information embedding. The social behavior similarity of the image is embedded into the visual feature space and the visual feature space of the image is transformed. This method can only use the visual features of images to predict their social similarity. At the same time. This work verifies that the image metric function can better reflect the user in search than the traditional image measurement method. User intention in browsing behavior. 4. A personalized image reordering method based on the fusion of social behavior and visual information is proposed. This method uses the behavior information of the user's group in the social platform to understand the interests of the user. In order to provide users with personalized image search results, this method uses random walk model to solve the problems of sparse social information and complexity of multi-modal information in social space. The search results are reordered on the mixed graph based on social information and visual content, so that the semantic correlation of the results can be guaranteed while giving the search results in accordance with the user's personal preferences. This work also proves the validity and importance of social information in personalized search.
【学位授予单位】：清华大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TP391.41

【相似文献】