实时视觉特征检测与匹配硬件架构研究

发布时间：2018-12-30 11:22

【摘要】：随着科学技术飞速发展,图像以其具体、直观的特点成为获取信息的主要手段,被广泛应用于数据表示、信息传递和交流通信,在计算机视觉、模式识别、医学图像处理、遥感图像处理、图像信息检索、导航制导、三维重建等众多领域获得广泛应用。这些应用都需要在不同传感器(成像设备)、不同时间、不同条件下获取的两幅或多幅图像间建立对应关系,而且对计算实时性要求非常高。但归咎于问题的复杂性,不使用特殊硬件的普通个人电脑非常难满足计算实时性的需求。本文的研究目标是设计出独立通用、功耗低、体积小、速度快的视觉特征检测与匹配硬件架构。首先,通过大量比较分析发现,SIFT算法是最鲁棒的特征检测与描述方法之一,相对于其他方法的重要优势在于,其对图像位移、尺度变换和旋转变换都具有不变性,同时对光照变化、仿射变换、投影变换以及噪声也具有相当的鲁棒性。然而,它的计算量非常大,通过纯软件方法难以满足计算实时的需求。本文提出一种基于FPGA+DSP的SIFT特征检测与描述向量提取硬件架构,很好的融合了FPGA的并行／流水特性与高性能DSP的灵活性,利用二维高斯核的可拆分性和对称性优化二维高斯滤波器,通过提炼并改造特征检测模块的并行性优化SIFT特征检测模块,通过实验分析确定梯度方向与模值计算模块、SIFT特征检测模块的字长,因此本架构能够在不损失匹配精度的前提下降低FPGA资源占用率。该系统在分辨率为320x256的图像中检测SIFT特征点仅耗时10ms,提取单个SIFT描述向量耗时80us,特征数量不超过200时,其计算速度达到50帧／秒。虽然上述FPGA+DSP硬件架构能实时检测SIFT特征并提取描述向量。然而,它没有实现描述向量匹配步骤,并且需要两个芯片协同工作,必须以单板形式存在。本文进一步提出基于单片FPGA的视觉特征检测与匹配的全并行／流水硬件架构,通过综合分析各模块的像素吞吐率与计算量需求、改进二维高斯滤波器层叠结构、提炼BRIEF描述向量提取与匹配的并行度,从而降低FPGA资源占用率,使整个视觉特征检测与匹配系统能在单片FPGA中实现。它能以60帧／秒的速度在1280x720图像中检测SIFT特征点、提取并匹配BRIEF描述向量,满足大部分应用系统对特征检测与匹配计算实时性的需求。大量实验证明,该系统大大的提升了视觉特征检测与匹配的计算速度,并保持相当的匹配精度。整个系统在单片FPGA中实现,由相机直接驱动,因此可以非常方便的集成到智能相机、视觉导航机器人等嵌入式系统中,提供可靠的图像对应关系。最后,本文通过两个实例阐述实时视觉特征检测与匹配硬件架构的应用。首先利用本文提出的FPGA+DSP硬件架构加速基于SIFT特征的月球着陆器水平测速算法中最耗时、占内存最多的SIFT特征检测及描述向量提取部分,使整个系统能在规定时间内估计着陆器的水平速度,满足着陆器对计算实时性的需求。然后利用本文提出的基于单片FPGA的实时视觉特征检测与匹配模块,加速未标定相机双目视觉视差估计算法中计算量最大、占内存最多的视觉特征提取与匹配部分,消除双目视觉系统对相机内参和几何关系已知的约束,降低对安装工艺精度的要求。本文提出的两种实时视觉特征检测与匹配硬件架构,满足机器视觉应用系统对功耗、体积、计算实时性等方面的约束,在导航制导系统、立体视觉机器人系统、目标跟踪、三维重建等应用领域都有非常重要的理论意义与应用价值,有利于促进机器视觉系统的实际应用。
[Abstract]:With the rapid development of science and technology, the image has become the main means of obtaining information in its concrete and visual characteristics, and is widely used in data representation, information transfer and communication, and is used in computer vision, pattern recognition, medical image processing, remote sensing image processing and image information retrieval. Navigation guidance and three-dimensional reconstruction have been widely used in many fields. these applications need to establish correspondence between two or more images acquired under different sensors (imaging devices), different times, different conditions, and very high in computational real-time requirements. But the complexity of the problem, the general personal computer without special hardware is very difficult to meet the need of real-time computing. The purpose of this paper is to design a visual feature detection and matching hardware architecture which is independent, universal, low in power consumption, small in volume and fast in speed. First, through a large number of comparative analysis, the SIFT algorithm is one of the most important feature detection and description methods, and the important advantage with respect to other methods is that it has invariance to image displacement, scale transformation and rotation transformation, and also changes the illumination and affine transformation. The transformation, projection transformation, and noise also have equivalent Rods Sex. However, its calculation is very large, and it is difficult to meet the need for real-time calculation by the pure software method In this paper, a kind of SIFT feature detection and description vector extraction hardware architecture based on FPGA and DSP is proposed, and the parallel/ flow characteristics of the FPGA and the flexibility of the high performance DSP are well integrated. The two-dimensional Gaussian filter is optimized by the resolution and symmetry of the two-dimensional Gaussian kernel. The invention optimizes the SIFT feature detection module by refining and modifying the parallelism of the feature detection module, determines the length of the gradient direction and the modulus value calculation module and the SIFT characteristic detection module through the experimental analysis, The system detects that the SIFT feature point is only 10ms in the image with the resolution of 320x256, the time of extracting a single SIFT description vector is 80us, the feature quantity is not more than 200, and the calculation speed is up to 50 frames/ seconds. While the above-described FPGA + DSP hardware architecture can detect SIFT features in real time and extract the EGMs the vector. However, it does not implement the description vector matching step and requires two chips to work together and must be in the form of a single board In this paper, the whole-parallel/ running-water hardware architecture based on the visual feature detection and matching of a single-chip FPGA is further put forward, and the structure of the two-dimensional Gaussian filter is improved by comprehensively analyzing the pixel throughput and the calculation amount requirement of each module, and the extraction and matching of the BRIEF description vector is refined. The parallelism is reduced, so that the resource occupation rate of the FPGA is reduced, and the whole visual characteristic detection and matching system can be on a single-chip FPGA. The method can detect the SIFT feature point in the 1280x720 image at a speed of 60 frames per second, extract and match the BRIEF description vector, and meet the real-time performance of the characteristic detection and matching calculation by most of the application systems. A large number of experiments prove that the system greatly improves the calculation speed of the visual feature detection and the matching, The whole system is realized in a single-chip FPGA and is directly driven by a camera, so that a reliable image pair can be provided in an embedded system such as an intelligent camera, a vision navigation robot and the like. In this paper, the real-time visual feature detection and matching hardware are described by two examples. The application of the architecture is the first to use the FPGA + DSP hardware architecture presented in this paper to speed up the time-consuming, up-to-memory SIFT feature detection and description vector extraction part of the SIFT feature-based lunar lander's horizontal velocity measurement algorithm, so that the whole system can estimate the lander within the specified time The horizontal speed of the landing gear is met, and the calculation of the landing gear is met. and then the real-time visual feature detection and matching module based on the single-chip FPGA is used to accelerate the calculation amount of the non-calibrated camera binocular vision parallax estimation algorithm to be the largest, and the maximum amount of the visual feature extraction in the memory is occupied. in combination with that matching part, the constraint that the binocular vision system is known to the parameter and the geometric relation of the camera is eliminated, and the installation process is reduced. The paper presents two real-time visual feature detection and matching hardware architecture, which can meet the constraints of the machine vision application system to power consumption, volume and real-time performance. In the navigation guidance system, the three-dimensional vision robot The application fields of system, target tracking and three-dimensional reconstruction are of great theoretical significance and application value, which is beneficial to the promotion of machine vision.
【学位授予单位】：华中科技大学
【学位级别】：博士
【学位授予年份】：2015
【分类号】：TP391.41

【相似文献】