基于改进并行回火算法的RBM网络训练研究

发布时间：2018-05-23 14:42

本文选题：深度学习 + 受限玻尔兹曼机　；参考：《自动化学报》2017年05期

【摘要】：目前受限玻尔兹曼机网络训练算法主要是基于采样的算法.当用采样算法进行梯度计算时,得到的采样梯度是真实梯度的近似值,采样梯度和真实梯度之间存在较大的误差,这严重影响了网络的训练效果.针对该问题,本文首先分析了采样梯度和真实梯度之间的数值误差和方向误差,以及它们对网络训练性能的影响,然后从马尔科夫采样的角度对以上问题进行了理论分析,并建立了梯度修正模型,通过修正梯度对采样梯度进行数值和方向的调节,并提出了基于改进并行回火算法的训练算法,即GFPT(Gradient fixing parallel tempering)算法.最后给出GFPT算法与现有算法的对比实验,仿真结果表明,GFPT算法可以极大地减小采样梯度和真实梯度之间的误差,大幅度提升受限玻尔兹曼机网络的训练效果.
[Abstract]:At present, the restricted Boltzmann network training algorithm is mainly based on the sampling algorithm. When using the sampling algorithm to calculate the gradient, the sampling gradient is the approximate value of the real gradient, and there is a large error between the sampling gradient and the real gradient. This seriously affects the training effect of the network. The numerical error and the direction error between the sampling gradient and the real gradient and their influence on the performance of the network training are analyzed. Then the above problems are theoretically analyzed from the angle of Markoff sampling, and the gradient correction model is established, and the improvement of the gradient is adjusted to the value and direction of the sampling gradient, and the improvement is proposed. The training algorithm of parallel backfire algorithm is GFPT (Gradient fixing parallel tempering) algorithm. Finally, the comparison experiment of GFPT algorithm and existing algorithm is given. The simulation results show that the GFPT algorithm can greatly reduce the error between the sampling gradient and the real gradient, and greatly lift the training effect of the limited Boltzmann network.
【作者单位】：西北工业大学电子信息学院;
【基金】：国家自然科学基金(61573285,61305133) 中央高校基本科研业务费专项基金(3102015BJ(Ⅱ)GH01)资助~~
【分类号】：TP18

【相似文献】