极限学习机隐含层节点选择算法研究

发布时间：2018-10-12 10:24

【摘要】：本文主要研究一种新的机器学习方法,即极限学习机(ELM)。作为一种针对单隐层前馈神经网络(SLFNs)的学习算法,ELM具有快速的学习速度和很好的泛化能力。隐含层节点在ELM算法中起着重要的作用,确定隐含层节点的方法有两种:一是剪枝法;二是增量学习方法。本文中我们介绍了两种剪枝方法,最优剪枝ELM(OP-ELM)和Tikhonov正则 OP-ELM(TROP-ELM)。我们的主要工作是ELM的增量学习方法,增量学习即首先初始化一个较小的网络,然后向网络中增加新的节点,直到生成一个我们满意的网络。当新的隐含层节点加入到已经存在的网络中时,重新训练网络往往比较费时,误差最小化极限学习机(EM-ELM)是一种增量的计算输出权值的快速方法。然而,由于过拟合等原因,EM-ELM不能总是得到好的泛化能力。在此,根据结构风险最小化准则,我们提出了一种基于正则化的EM-ELM改进方法,即增量正则化极限学习机(IR-ELM)。当我们逐一的向网络中增加新的隐含层节点时,IR-ELM可以快速的更新输出权值,同时保证网络具有很好的泛化能力,从而避免了上述提到的问题。同时,我们还提出了 IR-ELM的提升方法(EIR-ELM),它可以在一组候选隐含层节点中选出较好的一个加入到网络中,进一步提升了算法的泛化能力,同时生产更紧凑的网络。针对分类和回归问题,我们在基准数据集上与原始ELM算法,OP-ELM和TROP-ELM算法及EM-ELM和EEM-ELM算法进行了对比实验,验证了IR-ELM和EIR-ELM的有效性。
[Abstract]:In this paper, a new machine learning method, called extreme learning machine (ELM).), is studied. As a learning algorithm for single hidden layer feedforward neural network (SLFNs), ELM has fast learning speed and good generalization ability. Hidden layer nodes play an important role in ELM algorithm. There are two methods to determine hidden layer nodes: one is pruning method and the other is incremental learning method. In this paper, we introduce two pruning methods, optimal pruning ELM (OP-ELM) and Tikhonov regular OP-ELM (TROP-ELM). Our main work is the incremental learning method of ELM. Incremental learning is to initialize a small network first and then add new nodes to the network until we generate a satisfied network. When a new hidden layer node is added to an existing network, it is often time-consuming to retrain the network. Error minimization extreme learning machine (EM-ELM) is a fast incremental method for calculating output weights. However, due to over-fitting and other reasons, EM-ELM can not always get good generalization ability. Based on the structural risk minimization criterion, we propose an improved EM-ELM method based on regularization, i.e. incremental regularization extreme learning machine (IR-ELM). When we add new hidden layer nodes to the network one by one, IR-ELM can quickly update the output weights, and at the same time ensure that the network has a good generalization ability, thus avoiding the problem mentioned above. At the same time, we propose a IR-ELM lifting method (EIR-ELM), which can select a better one from a set of candidate hidden layer nodes to join the network, further improve the generalization ability of the algorithm and produce a more compact network. For the classification and regression problems, we compare with the original ELM algorithm, OP-ELM and TROP-ELM algorithm and EM-ELM and EEM-ELM algorithm on the benchmark dataset, and verify the effectiveness of IR-ELM and EIR-ELM.
【学位授予单位】：浙江大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP181

【相似文献】