基于嵌入结构性信息视觉特征的图像理解模型研究

发布时间：2018-04-25 03:05

本文选题：结构性视觉特征 + 图像理解　；参考：《天津大学》2015年博士论文

【摘要】：随着互联网技术的发展和移动互联网的兴起,大量的多媒体信息特别是图像被上传到互联网上并且数目仍在不断的增加,这些信息将人们带入了图片大数据时代。为了挖掘海量图像数据中的有效信息及其中所蕴含的经济和社会价值,涉及到与图像相关的诸多技术,图像理解是其中较重要的一环。传统的用于图像理解的方法主要是基于词袋模型,即首先提取图像的底层特征表示,然后构建一个编码词典,最后将图像底层特征依据编码词典进行映射得到图像的直方图特征表示。虽然该方法广泛应用于图像理解的相关方向如图像中的物体识别和图像检索,并取得了一定的效果,但是基于词袋模型的表示方法会将图像中蕴含的结构信息丢失,导致图像特征表示的辨别性和鲁棒性存在一定的局限性。区别于基于词袋模型的图像特征表示方法,本论文提出了一种新的构建图像特征表示的方法,该方法将与图像相关的结构信息融合到图像的特征表示中,以此来提高图像特征表示的辨别性和鲁棒性。本论文提出了三种不同形式的结构信息的嵌入视觉特征表示方法,并将其分别应用于图像理解领域中的图像检索,图像分类和图像语义标注中。第一种方法是在基于轮廓图像的图像检索和分类的应用中,不同于传统的方法直接提取轮廓图像的特征点并构建特征描述符,在本论文中,提出了将轮廓图像所描述的物体的本身所具有的结构对称性嵌入到图像的视觉特征表示中,从而构建出包含物体对称结构的图像视觉特征表示。该特征表示能够有效的提高特征描述符的辨别性和鲁棒性。在实验中,将该嵌入对称结构的视觉特征表示应用于轮廓图像的分类和检索上,实验结果表明该方法能够有效的提高轮廓图像的检索和分类的准确度证明了结构信息嵌入特征表示中的有效性。第二种方法是在基于视觉属性的图像检索的应用中,不同于传统的方法仅考虑视觉属性查询词与其他相关视觉属性之间的共存性,在本论文提出的方法中,首先将视觉属性结构之间的互斥和共存性嵌入到视觉属性的特征表达之中。然后利用该嵌入结构信息的图像特征表示,提出了一种基于特征重建的图像检索框架,该框架能够有效的保留图像的结构特征从而能够有效的提高图像检索的稳定性和鲁棒性。实验结果证明该方法能够有效的降低查询关键词的歧义性,提高图像检索的准确度。第三种方法是在基于弱监督的图像标注的应用中,由于图像中包含有多个物体,因此传统的基于词袋的特征表示方法具有歧义性,进而无法表示图像中不同物体之间的结构关系,最后使得图像标注结果的不准确。本论文提出一种将语义标签的结构相关性信息嵌入到图像特征表示之中的方法,从而克服原始特征表示存在的歧义性。实验结果表明该嵌入语义标签的结构性特征表示能够有效的提高图像特征表示的辨别性和泛化能力,进行能够推动图像标注的查全性和准确率的提升。本论文为了验证视觉特征的结构性在图像理解中的作用,考虑到不同的应用场景下图像特征中嵌入不同层次的结构性信息:底层信息中的物体本身的对称结构,中层特征信息中的视觉属性的相关性结构以及高层物体标签信息之间的语义结构。通过不同的实验结果证明本文所提出的嵌入结构性信息的视觉特征表示能够有效的提高特征表示的辨别性和鲁棒性,同时实验结果也表明了嵌入结构性信息的视觉特征的有效性以及能够对计算机视觉中图像理解方向的发展具有一定的推动作用。
[Abstract]:With the development of Internet technology and the rise of mobile Internet, a large number of multimedia information, especially images, are uploaded to the Internet and the number of them is still increasing. These information will bring people into the era of big picture data. In order to excavate the effective information in the massive image data and the economic and social value contained in it, The image understanding is a very important part of the technology related to the image. The traditional method for image understanding is mainly based on the word bag model, that is, first extracting the underlying feature representation of the image, and then constructing a coding dictionary, and then mapping the underlying feature of the image to the histogram of the image to get the histogram. Although this method is widely used in the related direction of image understanding, such as the object recognition and image retrieval in the image, the representation method based on the word bag model will lose the structure information contained in the image, which leads to the limitation of the discrimination and robustness of the image feature representation. In this paper, a new method of image feature representation is proposed in this paper, which combines the structure information associated with the image into the feature representation of the image, in order to improve the discrimination and robustness of the image feature representation. Three different types of structural letters are proposed in this paper. It is applied to image retrieval, image classification and image semantic annotation in the field of image understanding. The first method is to extract feature points of contour images and construct feature descriptors directly from the traditional methods in the application of image retrieval and classification based on contour images. In this paper, the structure symmetry of the object described by the outline image is embedded into the visual feature representation of the image, and the image visual feature containing the symmetrical structure of the object is constructed. The feature representation can effectively improve the discrimination and robustness of the feature descriptors. In the experiment, it is embedded. The visual features of symmetric structures are applied to the classification and retrieval of the contour images. The experimental results show that the accuracy of the method can effectively improve the retrieval and classification of the contour images. The second methods are different in the application of image retrieval based on visual attributes. The traditional method only considers the coexistence between the visual attribute query word and the other related visual attributes. In the method proposed in this paper, the mutual exclusion and coexistence between the visual attribute structures are embedded in the feature expression of the visual attributes. The reconstructed image retrieval framework, which can effectively preserve the structural features of the image, can effectively improve the stability and robustness of the image retrieval. The experimental results show that the proposed method can effectively reduce the ambiguity of the query key words and improve the accuracy of image retrieval. The third method is based on the weak supervised image. In the application of annotation, because there are many objects in the image, the traditional feature representation method based on the word bag is ambiguous, and can not express the structure relationship between different objects in the image. Finally, the result of the image annotation is inaccurate. The experimental results show that the structural features of the embedded semantic label can effectively improve the discriminability and generalization ability of the image feature representation, and improve the recall and accuracy of the image annotation. The role of characteristic structure in image understanding, taking into account the structural information embedded in different layers in different application scenes: the symmetry structure of the object itself in the underlying information, the correlation structure of the visual attributes in the middle feature information and the semantic structure between the label information of the high-level object. The experimental results show that the visual feature of the embedded structural information presented in this paper can effectively improve the discrimination and robustness of the feature representation, and the experimental results also show the effectiveness of the visual features embedded in the structural information and can push the development of the image understanding direction in the computer vision to a certain extent. Use.

【学位授予单位】：天津大学
【学位级别】：博士
【学位授予年份】：2015
【分类号】：TP391.41

【相似文献】