基于FPGA的实时双目立体视觉系统的设计与实现

发布时间：2019-05-10 20:39

【摘要】：立体视觉技术是计算机视觉领域中非常重要的研究方向,其通过匹配一对不同视角的图像来提取现实场景中的深度信息,该技术在无人驾驶、无人机、虚拟现实、人机交互以及3DTV等领域广泛应用。在过去几十年中,提出了许多算法以及设计平台来提高系统的精确度与实时性,然而由于在获取立体视觉的深度信息时,其运算量巨大且算法复杂度较高,所以实时获得高质量的深度信息仍然具有较大的挑战性。本文主要提出了基于FPGA+ARM架构的实时双目立体视觉的软、硬件系统设计方案。根据系统方案要求,系统主要包括图像传感器视频的高速接收、极线几何校正、Census变换、半全局立体匹配算法、左右一致性校验以及中值滤波等关键模块。整个立体视觉的系统在单一的Xilinx ZC706开发板上实现,硬件平台以XC7Z045芯片为处理器核心,根据对系统软、硬件功能划分,PS(ARM)端主要是实现软件控制与算法的调试,PL(FPGA)端主要实现视觉算法的并行处理的硬件加速。根据系统支持不同的分辨率输出以及算法的配置模式,系统采用了模块化、参数化的设计思想。在系统设计过程中,摄像头采用APTINA的MT9V034,接口数据为高速LVDS串行传输,可以减少噪声等干扰;极线几何校正采用二元回归多项式方程来模拟矩阵相乘,减少两幅图像坐标存储,大大减少了片上硬件逻辑资源。多项式计算采用流水线技术,明显提高系统的工作频率;由于半全局立体匹配算法的复杂度以及对存储器的要求很高,本文采用有效的计算代价函数的方法,并简化为4个方向进行代价聚合,设计并行缓存器,降低对存储器的要求。中值滤波可以滤除误匹配点以及动态规划带来的条纹等问题。实验结果表明,本设计的系统可以达到视频的实时处理要求,图像的分辨率为640×480,最大视差搜索范围为64个像素,帧率60 fps。对于匹配算法模块的工作频率可以达到130 MHz,可以实现1280×1024@72.2 fps的实时视频处理。
[Abstract]:Stereo vision technology is a very important research direction in the field of computer vision. It extracts depth information from real scene by matching a pair of images with different angles of view. This technology is used in unmanned, unmanned aerial vehicles, virtual reality, and so on. Human-computer interaction and 3DTV are widely used. In the past decades, many algorithms and design platforms have been proposed to improve the accuracy and real-time performance of the system. However, because of the huge computational complexity and the high complexity of the algorithm, the depth information of stereo vision is obtained. Therefore, it is still challenging to obtain high quality depth information in real time. In this paper, the software and hardware design scheme of real-time binocular stereo vision based on FPGA ARM architecture is proposed. According to the requirements of the system scheme, the system mainly includes high-speed video reception of image sensor, polar geometric correction, Census transform, semi-global stereo matching algorithm, left and right consistency check and median filtering and other key modules. The whole stereo vision system is implemented on a single Xilinx ZC706 development board. The hardware platform is based on the XC7Z045 chip as the processor core. According to the software and hardware functions of the system, the, PS (ARM) terminal is divided into two parts: software control and algorithm debugging. The PL (FPGA) side mainly realizes the hardware acceleration of parallel processing of visual algorithm. According to the different resolution output and the configuration mode of the algorithm, the system adopts the design idea of modularization and parametrization. In the process of system design, the camera uses APTINA MT9V034, interface data for high-speed LVDS serial transmission, which can reduce noise and other interference. The polar line geometry correction uses the binary regression polynomial equation to simulate the matrix multiplication, reduces the coordinate storage of two images, and greatly reduces the on-chip hardware logic resources. The pipeline technology is used to calculate the multinomial, which obviously improves the working frequency of the system. Because of the complexity of semi-global stereo matching algorithm and the high requirement for memory, this paper adopts an effective method of computing cost function, and simplifies it to four directions for cost aggregation, designs parallel buffer, and reduces the requirement of memory. Median filtering can filter out mismatching points and stripes caused by dynamic programming. The experimental results show that the system can meet the requirements of real-time video processing. The resolution of the image is 640x480, the maximum disparity search range is 64 pixels, and the frame rate is 60 fps.. For the matching algorithm module, the working frequency of the matching algorithm module can reach 130 MHz, and the real-time video processing of 1280 脳 104 鈮，

本文编号：2473972