大视场超高像素相机关键技术研究

发布时间：2019-07-05 11:38

【摘要】：对环境的感知,是人类获取周围环境信息的主要手段。视觉信息占人类从外界获取信息的比例超过八成。图像传感器自诞生以来,给科学研究,工业生产,日常生活带来了极大的影响。随着图像传感器应用的逐渐扩展和深入,对传感器的多方面参数提出了新的要求,传感器自身也在不断发展。大视场,高分辨率和高帧率的成像仪器一直是科学研究与工程技术工作者不断努力的方向。在充分调研大视场,超高像素成像方案的基础上,结合当前快速发展的信息处理技术,本论文采用了多传感器的方案设计了一套大视场成像实验仪器。论文从以下几个方面总结了课题的主要研究工作：1、首先从机械结构、图像传感器的电子驱动与数据采集、图像数据传输三个方面介绍本论文中提出的平面型大视场系统硬件构成。研究球面视场空间与传感器矩形成像面的关系,确定视场分割方法。详细介绍了子相机光轴空间指向和半球空间分割的结果。考虑到视场分割的关键在于光轴指向与矩形视场的大小,保持这两者不变可以将传感器的位置进行局部平移。提出了一种类菲涅耳透镜的平面相机阵列结构。该结构具有灵活、紧凑、便于电气连接的优点。研究了基于FPGA器件实现对多路图像传感器数据采集的方法。在单芯片上实现对图像传感器的曝光控制,数据采集,并实现了Camera Link高速数字图像传输接口。使得最终的大视场相机结构更加紧凑,系统的稳定性得到提升。并对系统进行测试,给出了详细参数。2、一个理想的由镜头和图像传感器组成的摄像系统应该是无畸变、无噪声的空间平移不变线性系统,但实际物理器件会受到各种限制,无法达到这一要求。课题对设计的大视场相机辐射响应特性进行了定标分析。辐射定标是指确定输入相机的辐射亮度与相机输出信号幅值之间的定量关系,即定量相机的辐射响应特性。由多个传感器构成的大视场成像设备,因为各个子相机的光学镜头的误差,传感面的感光敏感度,电子部件的不同,导致相同的场景通过不同的子相机获得的数字图像存在一定的差异。为了定量化这种差异对最终获得的全景图像的影响,我们对大视场相机的各个子相机的辐射响应进行了精确标定。包括传感器的线性和非线性辐射响应,不同曝光状态下的暗电流幅值,和镜头与机械结构引起的渐晕特性。给出了依据标定结果对图像矫正的示例,和利用矫正图进行拼接融合的全景图。在大视场范围成像场景下,整个视场内的光照亮度差异通常会很大。传统的采用广角镜头和单个传感器的形式,会导致较暗的区域和较亮的区域都无法在传感器上得到有效的响应值。通过采用多个子相机覆盖大的视场能够有效缓解这种量化位数不够带来的局限。3、为了得到大视场相机结构中各个子相机的内部和外部参数的精确值,课题中对全部子相机的内部和外部参数进行了标定。基于子相机的畸变参数,对采集到的图像首先进行校正。研究了图像拼接和全景图的合成方法。基于我们的大视场相机实验仪器,结合子相机的内部和外部参数,利用标定获得的内、外参数对图像进行初步配准,并结合图像特征进行多相机联合优化。避免了多相机图像拼接时需要逐个配准,和可能出现配准失效的问题。同时,采用多传感器的大视场方案,能有效避免场景中运动物体给最终全景图合成造成的困难。改进多波段融合方法对配准后的图像进行融合。最终获得可自选投影表面的全景图。多相机内外参数的详细标定,为虚拟现实,三维重建和目标跟踪等应用打下了坚实的基础。
文内图片：

图片说明：图１．邋２单中屯、《尺度ＡＷＡ祀－２型号相机逡逑该系统的核屯、是被命名为ａｗａｒｅ的一种单中也多尺度相机ＷＷＷ，，ａｗａｒｅ－２逡逑
[Abstract]:The perception of the environment is the main means for human to get the information of the surrounding environment. Visual information accounts for more than 80 percent of the world's access to information from the outside world. Since its birth, the image sensor has brought great influence to scientific research, industrial production and daily life. With the gradual expansion and in-depth of the application of the image sensor, new requirements are put forward for many parameters of the sensor, and the sensor itself is developing continuously. Imaging instruments with large field of view, high resolution and high frame rate have been the direction of continuous efforts by scientific research and engineering engineers. Based on the full investigation of the large-field and super-high-pixel imaging scheme, combined with the current rapid development information processing technology, a large-field-of-view imaging experimental instrument is designed in this paper. In this paper, the main research work of the project is summarized from the following aspects:1. Firstly, the hardware structure of the planar large-field-of-view system is introduced from three aspects: the mechanical structure, the electronic drive of the image sensor, the data acquisition and the image data transmission. The relation between the spherical field of view space and the rectangular imaging surface of the sensor is studied, and the method of field-of-view segmentation is determined. The results of the spatial orientation of the optical axis of the sub-camera and the division of the semi-spherical space are described in detail. The position of the sensor can be locally translated in view of the key point of view division at the optical axis pointing to the size of the rectangular field of view. A planar camera array structure of a kind of Fresnel lens is proposed. The structure has the advantages of being flexible, compact and convenient for electrical connection. The method of multi-channel image sensor data acquisition based on FPGA is studied. The invention realizes the exposure control and the data acquisition of the image sensor on a single chip, and realizes the Camera Link high-speed digital image transmission interface. So that the final large-field-of-view camera structure is more compact and the stability of the system is improved. And the system is tested and the detailed parameters are given. An ideal imaging system composed of a lens and an image sensor should be a non-distortion and noise-free space translation invariant linear system, but the actual physical device is subject to various restrictions and cannot meet the requirement. In this paper, the radiation response characteristics of the large-field-of-view camera designed are scaled and analyzed. The radiation calibration is the quantitative relation between the radiation intensity of the input camera and the amplitude of the output signal of the camera, that is, the radiation response characteristic of the quantitative camera. The large-field-of-view imaging device is composed of a plurality of sensors, because the error of the optical lens of each sub-camera, the sensitivity of the sensing surface and the difference of the electronic components cause the same scene to have a certain difference through the digital images obtained by different sub-cameras. In order to quantify the effect of this difference on the final obtained panoramic image, we accurately calibrate the radiation response of each sub-camera of the large field camera. Includes the linear and non-linear radiation response of the sensor, the dark current amplitude in different exposure states, and the vignetting characteristics caused by the lens and the mechanical structure. An example of image correction is given based on the result of the calibration, and a full view of the fusion of the image with the correction diagram is given. In large field-of-view imaging scenarios, the difference in illumination brightness in the entire field of view is generally large. The traditional form of wide-angle lenses and a single sensor results in a darker area and a lighter area that can't get an effective response on the sensor. In order to obtain the accurate values of the internal and external parameters of each sub-camera in the large-field-of-view camera structure, the internal and external parameters of all sub-cameras are calibrated. The acquired image is first corrected based on the distortion parameters of the sub-camera. The synthetic method of image mosaic and panorama is studied. Based on our large field-of-view camera experiment instrument, combined with the internal and external parameters of the sub-camera, the image is preliminarily registered by using the internal and external parameters obtained by the calibration, and the multi-camera joint optimization is carried out in combination with the image characteristics. Which avoids the need for one-by-one registration when the multi-camera image is spliced, and the problem that the registration failure can occur. And meanwhile, a large-field-of-view scheme with a multi-sensor is adopted, so that the difficulty caused by the motion object in the scene to the final panorama synthesis can be effectively avoided. And the improved multi-band fusion method is used for fusing the registered images. And finally, the panorama of the self-selected projection surface is obtained. The detailed calibration of the internal and external parameters of the multi-camera lays a solid foundation for the application of virtual reality, three-dimensional reconstruction and target tracking.
【学位授予单位】：中国科学技术大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TP391.41;TB852.1

【相似文献】