基于空间一致生长的多视图三维重建

发布时间：2018-05-09 18:57

本文选题：多视图三维重建 + 种子点提取　；参考：《华中师范大学》2017年博士论文

【摘要】：多视图三维重建直接从多幅二维图像中恢复场景的三维结构,是计算机视觉一个研究热点,在工业检测、逆向工程、城市规划、文物与遗迹保护和展示等众多领域有重要的应用价值。随着智能手机和高分辨、低成本图像传感器的大规模普及,表现出广阔的应用前景。近二十多年来出现了许多基于多视图的三维重建算法,这些算法可大致分为三类:基于体积的算法、基于深度图的算法和基于特征点生长的算法。要完美处理现实中各种复杂情况,如表面的快速起伏变化、细小结构、微弱纹理特征、遮挡效应等,以达到更高的重建精度和重建完整度,同时保证高的重建效率,现有方法仍需改进提高。本文提出了一种新的基于空间一致生长的多视图三维重建算法,它基于特征点生长,但对传统基于特征点生长的多视图三维重建算法的整体框架进行了拓展,在现有三个环节(稀疏种子点提取、生长和滤波)基础上,增加了一个新的环节:利用已生长完毕的点进行有条件的初始值矫正,同时还对现有三个环节进行了更新改进,取得了明显效果。全文工作和创新点总结如下。在稀疏种子点提取环节,提出了一种新的基于DAISY描述符的稀疏种子点提取方法。传统SFM方法通常提取每幅图像的SIFT特征点,通过特征点匹配提取稀疏种子点,往往由于匹配错误或失败导致种子点质量降低或数目减少。本文对每个特征点采用高性能DAISY描述符进行描述,然后沿对极线搜索与其DAISY特征最相似的点,提高了稀疏种子点的数量和精度,改变了传统的在有限的特征点之间直接进行匹配的方式。为了保证上述方法顺利实施,本算法提出了一系列配套措施,如通过最佳选图和少数特征点的匹配,采用随机抽样一致性方法计算图像对之间的基础矩阵。再如,在多幅图中采用DAISY特征描述符沿对极线进行搜索,利用奇异值分解的方法求解在所有图中均匹配成功的点的对应空间位置,避免了单幅图搜索可能存在的不确定性,同时利用重投影误差滤除大误差的点。最后,根据有条件的双重二次曲面拟合来近似真实物体表面,求取种子点初始方向。以上种子点的提取和后续空间一致生长都是在多层图像金字塔上进行的,进一步提高了算法效率和成功率。在生长环节,提出了一种空间一致生长策略。传统方法需要依赖参考图来寻找下一个生长点,即生长点的初始位置和方向是在一个局部坐标系确定的,随着参考图的更换,局部坐标系也随之发生改变。另外为了避免从质量不高的种子点生长出更低精度乃至错误的点,传统算法对种子点进行排序,优先从最优种子点进行生长,这种串行算法限制了其计算效率。本文提出的算法从所有种子点出发无差别地同步向外扩张生长,非常适用于并行计算;同时在一个固定的世界坐标系内,直接从每个种子点现有空间位置出发,沿其切平面确定生长点的空间初始位置,不需要间接依靠图像寻找生长点,也不需要在每幅图像上时刻记录哪些点已经生长完毕,有利于节省存储空间。进一步通过后续初始值矫正与错误点滤除等措施,本算法有效防止了从质量不高的种子点生长出更多低精度乃至错误的点,保证了每个空间点生长的相对独立性。另外在生长优化之前,从众多视图中挑选最佳主、副图参与优化,在优化过程中及时更新最佳主、副图以提高重建效率、精度和完整度,而传统的方法在生长点优化过程中通常不更换主图。同时,根据物体表面的纹理强弱自适应地调整各种生长参数,如窗口大小和图像金字塔层次等,以提高重建完整度。在新增加的初值矫正环节,提出了有条件的双重二次曲面拟合方法,根据已经生长完毕的点来拟合真实物体表面,进而对初始值进行矫正。从理论上来说,每个点的生长都是独立,即与其他区域是否已经重建完毕无关。但在实际中,生长点的初值来源于邻近种子点,如果初始值离真实表面较远,可能导致无法收敛或收敛到一个局部极小。通过初始值矫正可以有效地提高收敛速度和精度,为此首先判断邻域点是否足够稠密且以生长点位中心,如果条件满足,则进行第一次拟合,删除大误差点后进行第二次拟合,从而保证所拟合的曲面充分接近真实表面。接下来将生长点投影到拟合曲面,即可实现初始值矫正。在滤波环节,设计了三个自适应滤波器,分别根据光滑一致性、深度一致性和方向一致性原理,对误差点进行检测滤波。一方面保证了误差点的有效滤除,另一方面又避免了正确点的无辜删除。在滤波过程中,为了排除局部曲率半径、邻域点密度、遮挡等因素的影响,首先进行条件判断,决定是否滤波;如果进行滤波,则自适应地调整滤波参数。本文对来源于Middlebury标准数据库、DTU标准数据库、VGG多视图三维重建数据库和我们自己拍摄的不同类型实际场景进行了三维重建,均取得了较好的重建结果,证明本算法具有较好的稳定性。与其他多视图三维重建算法相比,本算法重建结果局部瑕疵与缺陷明显减少。定量评估结果表明,无论重建精度与重建完整度,本算法都位列前茅,特别是明显优于同类基于特征点生长的多视图三维重建算法,证明本文对基于特征点生长的多视图三维重建算法整体框架所进行的拓展,和对现有三个环节的改进,成效十分明显。
[Abstract]:The 3D reconstruction of multi view 3D reconstruction directly from multiple two-dimensional images is a hot topic in computer vision. It has important application value in many fields, such as industrial detection, reverse engineering, city planning, cultural relics and relics protection and display. In the past more than 20 years, there have been many 3D reconstruction algorithms based on multi view. These algorithms can be roughly divided into three types: Based on volume based algorithms, algorithms based on depth map and algorithm based on feature point growth. Structure, weak texture feature, occlusion effect and so on, in order to achieve higher reconstruction precision and reconstruction integrity, while ensuring high reconstruction efficiency, the existing methods still need to be improved. In this paper, a new multi view 3D reconstruction algorithm based on spatial uniform growth is proposed. It is based on the feature point growth, but the traditional growth based on feature points is much more. The overall frame of the 3D reconstruction algorithm is expanded. On the basis of three existing links (sparse seed extraction, growth and filtering), a new link is added: using the finished points to correct the conditional initial values, and the existing three rings are updated and improved, and the full text work has been achieved. In sparse seed point extraction, a new sparse seed point extraction method based on DAISY descriptors is proposed. The traditional SFM method usually extracts the SIFT feature points of each image, and extracts sparse seed points through feature point matching, often resulting in the quality reduction or number of seed points due to matching errors or failures. In this paper, a high performance DAISY descriptor is used for each feature point to be described, and then the number and accuracy of the sparse seed point are improved along the polar line search and the most similar point of its DAISY feature, and the traditional matching method between the finite feature points is changed. A series of matching measures, such as the matching of the best selection map and a few feature points, are used to calculate the base matrix between the image pairs by random sampling consistency method. Again, the DAISY feature descriptors are used to search the polar lines in the multiple images, and the method of singular value decomposition is used to solve the corresponding points that have been matched in all graphs. Space position avoids the possible uncertainty of single map search, and uses the re projection error to filter the point of large error. Finally, the initial direction of the seed point is obtained by using the conditional double two order surface fitting to approximate the real object surface. The extraction of the above seed points and the consistent growth of the following space are all in the pyramid of multi-layer images. On the tower, the efficiency and success rate of the algorithm are further improved. In the growth link, a spatial uniform growth strategy is proposed. The traditional method needs to rely on the reference graph to find the next growth point, that is, the initial position and direction of the growth point are determined in a local coordinate system. With the change of the reference map, the local coordinate system is also followed. In addition, in order to avoid the points of lower precision and error from the poor quality seed points, the traditional algorithms sort the seed points and give priority to the growth of the best seed points. This serial algorithm restricts the efficiency of calculation. The algorithm proposed in this paper extends from all kinds of subpoints to expand and grow synchronously from all kinds of subpoints. It is very suitable for parallel computing; at the same time, in a fixed world coordinate system, starting from the existing space position of each seed point, the initial position of the growth point is determined along its tangent plane. It does not need to rely on the image to find the growth point indirectly, and it does not need to record which points have been grown on each image. It is beneficial to the festival. This algorithm effectively prevents more low precision and error points from poor quality seed points, and ensures the relative independence of each space point growth. In addition, before optimization of growth, select the best owner from many views and participate in the sub graph. Optimization, in the process of optimization, the best master is updated in time to improve the efficiency, accuracy and integrity of the reconstruction, while the traditional method usually does not replace the main graph in the process of growth point optimization. At the same time, according to the texture strength of the surface, the growth parameters, such as the size of the window and the Pyramid level of the image, are adjusted to improve the reconstruction. In the newly added initial value correction link, a conditional double two surface fitting method is proposed to fit the surface of the real object based on the points that have been grown, and then correct the initial value. In theory, the growth of each point is independent, that is, it has nothing to do with the reconstruction of other regions. But in practice, The initial value of the growth point is derived from the adjacent seed point. If the initial value is far away from the real surface, it may lead to the failure to converge or converge to a local minimum. The convergence speed and accuracy can be effectively improved by the initial value correction. The first fitting, after the deletion of the large error points, second times of fitting, so as to ensure that the fitted surface is fully close to the true surface. Then the growth point is projected to the fitting surface, and the initial value can be corrected. In the filtering link, three adaptive filters are designed, which are based on smooth consistency, depth consistency and direction consistency, respectively. On the one hand, the error point is effectively filtered and the innocent deletion of the correct point is avoided. In the filtering process, in order to exclude the influence of the local curvature radius, the neighborhood point density, the occlusion and so on, the filter is determined first, and the filter is adjusted adaptively if the filtering is carried out. In this paper, we reconstruct the 3D reconstruction from the Middlebury standard database, the DTU standard database, the VGG multi view 3D reconstruction database and the different types of actual scenes we have taken. The results show that the algorithm has good stability. Compared with other multi view 3D reconstruction algorithms, The results of quantitative evaluation show that both the reconstruction accuracy and the reconstruction integrity are among the best, especially the multi view 3D reconstruction algorithm based on the feature point growth of the same kind, which proves the overall framework of the multi view 3D reconstruction algorithm based on the characteristic point growth. The expansion and the improvement of the existing three links are very effective.

【学位授予单位】：华中师范大学
【学位级别】：博士
【学位授予年份】：2017
【分类号】：TP391.41

【参考文献】