互学习神经网络训练方法研究

发布时间：2018-02-13 16:12

本文关键词： 神经网络互学习权值共享 BP算法双向认知分类识别人工智能　出处：《计算机学报》2017年06期 　论文类型：期刊论文

【摘要】：由于BP神经网络具有表达能力强,模型简单等特点,经过近30年的发展,在理论和应用研究上都取得了巨大的进步,然而容易陷入局部最优和泛化能力差等问题却限制了神经网络的发展.同时,大数据的出现和深度学习算法的提出与应用,为神经网络向更类脑的方向发展提出了新的要求.针对上述问题,该文从模拟生物双向认知能力的角度出发,构造了一种新的神经网络模型——互学习神经网络模型,该模型在标准正向神经网络的基础上,引入了与其具有结构对称性的负向神经网络,利用正、负向神经网络分别模拟生物的顺向和逆向认知过程,并在此基础上提出了一种新的神经网络训练方法——互学习神经网络训练方法,该方法通过网络连接权值转置共享,正、负双向交替训练的方式对互学习神经网络模型进行训练,从而实现输入数据和输出标签之间的相互学习,使网络具有双向认知能力.实验表明,互学习神经网络训练方法可以同时训练正、负两个神经网络,并使网络收敛.同时,在此基础上提出了"互学习预训练+标准正向训练"的两阶段学习策略和相应的转换学习方法,这种转换学习方法起到了和"无监督预训练+监督微调"相同的效果,能够使网络训练效果更好,是一种快速、稳定、泛化能力强的新型神经网络学习方法.
[Abstract]:Because BP neural network has the characteristics of strong expression ability and simple model, after nearly 30 years' development, it has made great progress in theory and application research. However, problems such as local optimization and poor generalization ability limit the development of neural networks. At the same time, the emergence of big data and the introduction and application of depth learning algorithm, This paper puts forward new requirements for the development of neural network towards more brain-like. In view of the above problems, a new neural network model-mutual learning neural network model is constructed from the point of view of simulating biologic bidirectional cognitive ability. Based on the standard forward neural network, a negative neural network with structural symmetry is introduced in this model. The forward and negative neural networks are used to simulate the forward and reverse cognitive processes of organisms, respectively. On the basis of this, a new neural network training method-mutual learning neural network training method is proposed. This method trains the model of mutual learning neural network by transposition of network connection weights, positive and negative two-way alternate training. The experiment shows that the training method of mutual learning neural network can train both positive and negative neural networks at the same time, and make the network converge. On the basis of this, the two-stage learning strategy and the corresponding conversion learning method of "mutual learning pre-training standard forward training" are put forward. This conversion learning method has the same effect as "unsupervised pre-training supervision fine-tuning". It is a new learning method of neural network with fast, stable and strong generalization ability.
【作者单位】：辽宁工程技术大学数学与系统科学研究所;辽宁工程技术大学智能工程与数学研究院;辽宁工程技术大学矿业学院;
【基金】：国家自然科学基金(51304114,71371091)资助~~
【分类号】：TP18

【参考文献】