面向多视点视频的新视点合成技术研究

发布时间：2019-04-01 14:20

【摘要】：随着人们对于通信、计算机网络、数字多媒体等技术的不断关注和依赖,多视点视频将成为未来数字电视的重要发展方向。与传统的单视点二维视频相比较,多视点视频与人们的视觉需求更加匹配,它能够提供丰富的立体感和沉浸感,并可以根据用户需求自由切换观看角度。多视点视频在给人们带来在全新感官体验的同时,其本身庞大的数据量也对现有多媒体信息处理、通信网络传输等技术带来了巨大的挑战。如何利用有限的参考视点生成高质量的虚拟视点成为多视点视频发展的内在要求。本文从多视点视频系统的基本需求和目标出发,围绕着新视点合成问题进行了深入地研究,提出了适用于该系统的内容表示和视点重构方法。本文的主要工作内容和成果可归纳如下： 1)针对立体匹配问题提出了一种基于多测度融合的匹配代价建立算法。该算法利用基于梯度的Census变换(Gradient-based Census Transform, GCT)、绝对色彩差(Absolute Color Differences, ACD),以及Gabor模式差(Gabor Pattern Differences, GPD)三种测度,从不同特征模式入手,多尺度、全方位地刻画了参考图像和目标图像之间的关联特性,通过全面提取图像区域的边缘信息、纹理特性、色彩差异、方向属性等特征,建立起完整统一的匹配代价计算模型,从而为精确地像素匹配提供了充足的保障。 2)针对传统代价聚合计算复杂度高以及视差优化存在歧义性的问题,提出了一种基于引导滤波器和自适应可信度图的优化算法。该算法采用的引导滤波器不但可以出色地完成传统的代价聚合任务,而且在代价聚合实施过程中,有线性复杂度的计算模型,这使得其运算量不依赖于聚合窗口的大小,而仅仅取决于被聚合图像的总像素点数目,从而明显提高了代价聚合的计算效率。另外,传统的WTA(Winner-Take-All)算法可能由于代价选择的歧义性而引入匹配误差,使用自适应可信度图算法可以有效选择和替换可靠度较低的视差像素点,从而进一步提高了匹配结果的可靠性。 3)针对逐帧生成的深度图序列在时域上不连续进而引发闪烁噪声等失真问题,提出了一种基于自适应时域梯度滤波的空时一致性视差序列估计算法。该算法不仅避免了对全局能量最小化、光流场等复杂问题的求解,而且可以与现有的单帧立体匹配算法无缝兼容,在算法应用范围、可行性、计算复杂度等方面都有独特的优势。 4)针对新视点合成中因视点遮挡而产生的空洞失真等问题,提出了一种基于全局背景图的新视点合成算法。该算法立足于从视频序列的帧间信息中提取空洞区域的真实像素值,并结合图像本身的纹理特征完成对空洞区域的修复。同时,与传统的至少需要使用两路参考视点来绘制中间虚拟视点的方法相比,本文算法仅需一路参考视点,这不仅能够有效降低带宽占用率,而且还可以不再受中间位置的限制,能够提供更广阔的虚拟视点合成范围。总得来说,本文针对多视点视频中的立体匹配、深度图时域连续化、视点合成、空洞失真修复等关键问题进行了较为深入地研究,为面向多视点视频的新视点合成技术的进一步发展做出了有益地探索。
[Abstract]:As people's attention and dependence on communication, computer network, digital multimedia and other technologies, multi-view video will become the important development direction of digital TV in the future. Compared with the traditional single-view two-dimensional video, the multi-view video is more matched with the human visual demand, and can provide rich stereoscopic impression and immersion feeling, and can freely switch the viewing angle according to the user demand. The multi-view video has brought great challenges to the existing multimedia information processing, communication network transmission and other technologies while bringing the new sense experience to people. How to use a limited reference viewpoint to generate a high-quality virtual viewpoint becomes the inner requirement of multi-view video development. In this paper, based on the basic requirements and objectives of the multi-view video system, a thorough study of the new viewpoint synthesis problem is carried out, and the content representation and the viewpoint reconstructor suitable for the system are put forward. The main contents and achievements of this paper can be summarized as follows: (1) Based on the three-dimensional matching problem, a matching cost building based on multi-measure fusion is proposed. The algorithm is based on the gradient-based Census transform (GCT), absolute color difference (ACD), and Gabor pattern difference (GPD). By comprehensively extracting the edge information, texture characteristic, color difference, direction attribute and the like of the image area, the matching cost calculation model is established, so that the accurate pixel matching is provided. (2) Based on the problems of high complexity of the traditional cost aggregation and the ambiguity of the parallax optimization, a guide filter and a self-adaptive reliability map are proposed. The algorithm adopted by the algorithm not only can perform the traditional cost aggregation task well, but also has a linear complexity calculation model in the process of cost aggregation, which makes the calculation amount not depend on the aggregation. the size of the window, but only on the total number of pixels of the aggregated image, significantly increases the cost aggregation In addition, the traditional WTA (Winner-take-All) algorithm can introduce the matching error due to the ambiguity of the cost selection, and the adaptive reliability map algorithm can effectively select and replace the low-reliability parallax pixel points, thereby further improving the matching junction. This paper presents a space-time consistency vision based on adaptive time-domain gradient filtering in order to avoid the distortion problems such as flicker noise in the time domain for the frame-by-frame depth map sequence. The algorithm not only avoids the solution of complex problems such as global energy minimization, light flow field and the like, but also can be seamlessly compatible with the existing single-frame stereo matching algorithm, and can be used in the aspects of the application range, the feasibility, the computational complexity and the like of the algorithm. Has a unique advantage.4) A global background is put forward for the problem of cavity distortion caused by the viewpoint occlusion in the new viewpoint synthesis. The algorithm is based on the extraction of the real pixel value of the cavity region from the inter-frame information of the video sequence, in addition, compared with the traditional method for drawing the middle virtual viewpoint by using at least two reference viewpoints, the algorithm only needs one path of reference viewpoint, which not only can effectively reduce the bandwidth occupation rate, but also can not Rebound by the intermediate position, it can be provided with a wider range of In general, the paper studies the key problems in multi-view video, such as stereo matching, time-domain continuous, point-of-view synthesis, and cavity distortion repair.
【学位授予单位】：北京邮电大学
【学位级别】：博士
【学位授予年份】：2014
【分类号】：TN919.8

【相似文献】