基于内容分析的图像视频编码研究

发布时间：2018-08-30 18:38

【摘要】：数字图像视频的压缩技术已经研究超过二十年,并取得了巨大的成功。然而,多年的开发使得以“预测-变换”为主的传统编码框架越来越接近其性能极限。我们有必要从新的角度分析并理解数字图像视觉内容,开发新的编码压缩方法。最近十年中,计算机视觉领域的快速发展启发我们可以从视觉内容分析出发,发掘图像的视觉相关性,改进图像视频编码性能。在本论文中,我们重点研究视觉内容分析技术与图像视频编码技术的结合,借助视觉内容分析技术对图像间视觉相关性进行分析,消除图像和视频中不同层面的视觉冗余,从而提高图像视频的编码效率。本论文的主要贡献可以总结为三个部分的工作。在第一部分工作中,我们提出了一种基于视觉模式分析的图像编码方法。该方法通过先验视觉模式描述图像低频和高频之间的视觉相关性,并由此自适应地在编码端丢弃图像中的某些高频视觉冗余,改善编码性能。同时在解码端,根据视觉模式包含的先验信息估计并恢复丢失的高频细节,改善图像重建质量。此外,我们进一步地将基于视觉模式的分析技术扩展至可伸缩视频编码应用,提出了一种新的基于视觉模式分析的层间预测方法。该方法借助视觉模式的搜索和映射,同时发掘可伸缩视频序列在时域和空域的视觉相关性,分别生成两个高质量的层间预测信号,改善可伸缩视频编码性能。此外,我们还采用了基于参数分析的预测方法,通过分析基本层已编码的信息(比如HEVC编码中的四叉树信息)来实现复杂度较低的层间预测。我们的方法通过结合多种内容分析机制,同时提供了多环路和单环路的系统实现,在编码性能和复杂度之间取得更好的平衡。在第二部分的工作中,我们提出了一种基于图像特征的高效图像编码方法。该方法通过图像局部特征匹配建立起更紧密的图像局部视觉联系,再配合像素层面的相关性分析,更有效地消除视觉冗余。具体地说,我们利用多尺度小波变换和SIFT特征提取,首先将输入图像分解为全局信息和子带内局部信息,并进行编码压缩。全局信息是对输入图像的基本描述,包含有限的视觉冗余；而子带内局部信息则是从不同的小波子带中提取的SIFT局部特征。在解码端,我们利用解码的SIFT特征从云端图像数据库中,检索出一组视觉相似的图像片。然后,结合基于视觉模式的分析和映射,将这些相似图像片中的信息与解码的全局信息融合起来,重建目标图像。根据子带内SIFT特征建立的视觉联系,我们利用基于视觉模式的映射从最低频的子带开始,由低频至高频,依次将视觉相似图像片内的信息融合进入对应的子带,恢复图像不同频带内的局部细节,直到图像完整重建。我们的方法通过结合局部特征分析技术和视觉模式分析的优点,取得更高效的图像编码性能。在第三部分的工作中,我们提出一种基于图像特征全局相似度分析的图像集整体优化编码方法。根据图像局部特征的整体统计特性,我们定义特征距离来分析图像与图像间的全局相似度。在此基础上,我们将图像集聚类为若干个相关性更强的子集,并将每个子集中图像间的关系描述为一个加权有向图。图中每个节点代表一幅图像,每一条边由特征距离加权。通过寻找该有向图的最小权值生成树,可以得到具有最小预测代价的优化编码结构。为了进一步增强图与图之间的相关性,我们提出了一种全新的基于特征的图像间三步预测方法。首先,我们利用SIFT特征匹配和多模型几何运动估计,消除不同区域的几何形变。其次,我们引入光度变换消除图像间由于光照变化带来的差异。最后,我们利用基于块的运动补偿机制生成局部优化的预测信号。我们提出的基于图像特征的方法充分利用多种内容分析技术的优点。基于特征的全局分析技术有效地确定了优化编码结构；基于局部特征匹配的图像变换增强了图像与图像间的区域相关性；基于像素的运动补偿生成了更精确的预测信号。因此,我们的方法有效地发掘了相关图像间的视觉冗余,提高了图像集整体编码的效率,同时为进一步研究大数据和云存储环境中的大尺度图像视频编码提供了新的思路。
[Abstract]:Digital image and video compression technology has been studied for more than 20 years and has achieved great success. However, many years of development have made the traditional coding framework based on "prediction-transformation" approach its performance limit more and more. The rapid development of computer vision in the last decade has inspired us to explore the visual correlation of images and improve the performance of image video coding based on visual content analysis.
In this paper, we focus on the combination of visual content analysis technology and image video coding technology. We use visual content analysis technology to analyze the visual correlation between images, eliminate visual redundancy at different levels in images and videos, so as to improve the coding efficiency of images and videos. Part of the work.
In the first part of our work, we propose an image coding method based on visual pattern analysis. This method describes the visual correlation between low and high frequencies of an image by a priori visual pattern, and thus adaptively discards some high frequency visual redundancy in the image at the coding end to improve the coding performance. In addition, we extend the analysis technique based on visual pattern to the application of scalable video coding, and propose a novel inter-layer prediction method based on visual pattern analysis. By mapping, the visual correlation of scalable video sequences in time domain and space domain is explored, and two high-quality inter-layer predictive signals are generated to improve the performance of scalable video coding. Our approach achieves a better balance between coding performance and complexity by combining multiple content analysis mechanisms and providing a multi-loop and single-loop system implementation.
In the second part of the work, we propose an efficient image coding method based on image features. This method establishes a closer local visual relationship through image local feature matching, and then combines the correlation analysis of pixel level to eliminate visual redundancy more effectively. Specifically, we use multi-scale wavelet transform. In SIFT feature extraction, the input image is decomposed into global information and local information in sub-band, and then encoded and compressed. SIFT features are used to retrieve a set of image slices with similar vision from cloud image database. Then, combined with the analysis and mapping based on visual pattern, the information in these similar image slices is fused with the decoded global information to reconstruct the target image. This method combines the advantages of local feature analysis and visual pattern analysis to achieve higher efficiency. Image coding performance.
In the third part, we propose an image set global optimization coding method based on global similarity analysis of image features. According to the global statistical characteristics of local features, we define feature distance to analyze the global similarity between images. On this basis, we cluster image sets into several correlations. Each node in the graph represents an image and each edge is weighted by the feature distance. By searching for the minimum weight spanning tree of the directed graph, the optimal coding structure with the minimum prediction cost can be obtained. In order to further enhance the graph and the graph. Firstly, we use SIFT feature matching and multi-model geometric motion estimation to eliminate geometric deformation in different regions. Secondly, we introduce photometric transformation to eliminate the difference between images caused by illumination changes. Motion compensation mechanism generates locally optimized predictive signals. The proposed image feature-based method makes full use of the advantages of various content analysis techniques. Feature-based global analysis techniques effectively determine the optimal coding structure; and image transformation based on local feature matching enhances the regional correlation between images and images. Therefore, our method effectively exploits the visual redundancy between related images, improves the overall coding efficiency of the image set, and provides a new idea for further research on large-scale image video coding in large data and cloud storage environment.
【学位授予单位】：中国科学技术大学
【学位级别】：博士
【学位授予年份】：2014
【分类号】：TN919.81

【相似文献】