基于深度堆叠卷积神经网络的图像融合

发布时间：2018-02-25 03:13

本文关键词： 图像融合深度学习卷积神经网络堆叠自动编码滤波器　出处：《计算机学报》2017年11期 　论文类型：期刊论文

【摘要】：该文针对多尺度变换融合图像中普遍存在的需要依据先验知识选取滤波器,导致融合效果存在不确定性的问题,提出了基于深度堆叠卷积神经网络的融合方法.首先,分别以高斯拉普拉斯滤波器和高斯滤波器为首层网络的初始卷积核,将源图像分解为高频和低频图像序列;其次,基于He K方法初始化其余层卷积核,获得与源图像尺寸相同的高频和低频重构图像各一幅,并将二者合成源图像的近似图像;再以源图像和近似图像像素值之差的平方和的均值为误差函数,进行反向传播训练形成基本神经单元;之后,将多个基本单元堆叠起来利用end-to-end的方式调整整个网络得到深度堆叠神经网络.然后,利用该堆叠网络分别分解测试图像对,得到各自的高频和低频图像,再基于局部方差取大和区域匹配度合并的规则分别融合高频和低频图像,并将高频融合图像和低频融合图像放回最后一层网络,得到最终的融合图像.实验结果表明:与基于双树复小波变换(Dual-Tree Complex Wavelet Transform,DTCWT)、非下采样轮廓波变换(Non-Subsampled Contourlet Transform,NSCT)和非下采样剪切波变换(Non-Subsampled Shearlet Transform,NSST)的融合结果相比,用高斯拉普拉斯滤波器和高斯滤波器初始化的深度堆叠卷积神经网络融合效果主观效果好,客观指标最优个数为NSCT的3.3倍,运行时间为NSCT的30.3%和NSST的11.6%.
[Abstract]:In this paper, a fusion method based on deep stacked convolution neural network is proposed to solve the problem that filter selection based on prior knowledge is necessary in multiscale image fusion, which leads to uncertainty of fusion effect. Taking Gao Si Laplacian filter and Gao Si filter as initial convolution cores of the first layer network, the source image is decomposed into high frequency and low frequency image sequences respectively. Secondly, the other layer convolution cores are initialized based on He-K method. A high frequency and a low frequency reconstructed image with the same size as the source image are obtained, and the approximate image of the source image is synthesized. Then, the mean value of the square sum of the difference between the pixel value of the source image and the approximate image is taken as the error function. The basic neural unit is formed by backpropagation training. After stacking several basic units and adjusting the whole network by end-to-end, the deep stacking neural network is obtained. Then, the test image pair is decomposed using the stack network. The high frequency and low frequency images are obtained, and then the high frequency and low frequency images are fused based on the rules of local variance and region matching, respectively, and the high frequency fusion images and the low frequency fusion images are put back to the last layer of the network. The experimental results show that the fusion results are compared with those based on Dual-Tree Complex Wavelet transform (DTCWTT), Non-Subsampled Contourlet transform (NSCT) and Non-Subsampled Shearlet transform (NSSTT). The fusion effect of deep stack convolution neural network initialized by Gao Si Laplace filter and Gao Si filter is good, the optimal number of objective indexes is 3.3 times of NSCT, the running time is 30.3% of NSCT and 11.662 of NSST.
【作者单位】：中北大学计算机与控制工程学院;
【基金】：山西省重点研发计划项目(201603D321128) 山西省应用基础研究项目(201701D121062) 中北大学第十三届研究生科技立项(20161354)资助~~
【分类号】：TP183;TP391.41

【相似文献】