基于权值动量的RBM加速学习算法研究

发布时间：2018-08-28 09:38

【摘要】：动量算法理论上可以加速受限玻尔兹曼机(Restricted Boltzmann machine,RBM)网络的训练速度.本文通过对现有动量算法进行仿真研究,发现现有动量算法在受限玻尔兹曼机网络训练中加速效果较差,且在训练后期逐渐失去了加速性能.针对以上问题,本文首先基于Gibbs采样收敛性定理对现有动量算法进行了理论分析,证明了现有动量算法的加速效果是以牺牲网络权值为代价的;然后,本文进一步对网络权值进行研究,发现网络权值中包含大量真实梯度的方向信息,这些方向信息可以用来对网络进行训练;基于此,本文提出了基于网络权值的权值动量算法,最后给出了仿真实验.实验结果表明,本文提出的动量算法具有更好的加速效果,并且在训练后期仍然能够保持较好的加速性能,可以很好地弥补现有动量算法的不足.
[Abstract]:Momentum algorithm can accelerate the training speed of constrained Boltzmann machine (Restricted Boltzmann machine,RBM) network theoretically. Based on the simulation of the existing momentum algorithms, it is found that the current momentum algorithms have poor acceleration effect in the constrained Boltzmann network training, and gradually lose the acceleration performance in the later stage of training. Aiming at the above problems, this paper firstly analyzes the existing momentum algorithm based on the Gibbs sampling convergence theorem, and proves that the acceleration effect of the existing momentum algorithm is at the expense of the network weight. This paper further studies the network weights, and finds that the network weights contain a large number of real gradient directional information, which can be used to train the network. Based on this, this paper proposes a weight momentum algorithm based on network weights. Finally, the simulation experiment is given. The experimental results show that the momentum algorithm proposed in this paper has better acceleration effect, and can still maintain better acceleration performance in the later training period, which can make up for the deficiency of the existing momentum algorithm.
【作者单位】：西北工业大学电子信息学院;
【基金】：国家自然科学基金(61305133,61573285)资助~~
【分类号】：TP181

【相似文献】