基于交叉熵的随机赋权网络

发布时间：2018-01-07 08:24

本文关键词：基于交叉熵的随机赋权网络　出处：《河北大学》2017年硕士论文　论文类型：学位论文

【摘要】：近年来,随着信息技术与计算机应用技术的不断进步发展,整个社会进入了大数据时代.因此,如何利用当前先进的数据分析技术,从海量的数据中挖掘出所需的信息成为最关键的问题.分类问题作为数据分析中的主要问题也在不断地引起人们的关注.黄广斌提出了一种结构简单的神经网络:极速学习机,它是基于最小均方误差的原则求得矩阵广义逆,具有训练时间短,测试精度较高的优点.但是,因为ELM仅考虑的是训练数据经验误差最小化,容易产生过拟合现象.本文的主要工作是:提出用交叉熵损失函数替代均方误差损失函数,在神经网络产生过拟合情况下,比较两者的测试精度,以此来比较二者的泛化能力.具体地,过拟合现象是机器学习中一种常见的现象,表现为分类器能够100%的正确分类训练样本数据,但对于其他数据则表现较差,其原因是构造的函数过于精细复杂.在ELM中,通过计算隐藏输出矩阵的广义逆,找到具有最小二范数的最优解.但由于隐层输出矩阵的行数远远大于列数,即隐层节点的数量很多,会出现过拟合现象.为了解决这个问题,本文提出一种基于交叉熵的随机赋权网络(CE-RWNNs),用交叉熵最小化原理代替均方误差最小化原理.实验结果证明,提出的CE-RWNNs可以一定程度上克服在具有许多隐层节点的ELM中过拟合的缺点.
[Abstract]:In recent years, with the continuous development of information technology and computer application technology, the whole society has entered the big data era. Therefore, how to use the current advanced data analysis technology. Mining the needed information from massive data has become the most critical issue. Classification, as the main problem in data analysis, has been attracting more and more attention. Huang Guangbin proposed a simple neural network. :. Speed learning machine. It is based on the principle of minimum mean square error to obtain matrix generalized inverse, which has the advantages of short training time and high test accuracy. However, because ELM only considers the minimum empirical error of training data. The main work of this paper is to use the cross-entropy loss function to replace the mean square error loss function. Specifically, over-fitting is a common phenomenon in machine learning, which shows that the classifier can correctly classify the training sample data of 100%. But for other data, the reason is that the constructed function is too fine and complex. In ELM, the generalized inverse of hidden output matrix is calculated. Find the optimal solution with least square norm, but because the number of rows of hidden layer output matrix is far larger than the number of columns, that is, there are many hidden layer nodes, there will be a phenomenon of over-fitting. In order to solve this problem. In this paper, a random weight network based on cross-entropy is proposed. The principle of cross-entropy minimization is used to replace the principle of mean square error minimization. The proposed CE-RWNNs can overcome the shortcoming of overfitting in ELM with many hidden nodes to some extent.
【学位授予单位】：河北大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP18

【参考文献】