基于梯度提升模型的负相关学习算法的研究与应用

发布时间：2019-03-24 20:51

【摘要】：论文的第一个工作是关于集成学习的研究。在机器学习领域,我们把具备从经验知识中学习能力的系统或者模型叫做学习器。一般来说训练出一个学习能力较弱的模型比训练出一个学习能力较强的模型所要耗费的代价小得多。集成学习是一类特殊的机器学习方法,其思想是不直接训练一个强学习器,而是通过组合一批弱学习器来得到一个学习能力强的集成学习器。集成学习算法性能好坏主要取决于两个因素：基学习器自身的性能好坏以及基学习器之间的差异性。目前常用的集成学习算法包括Bagging、Boosting等,在提升每个基学习器性能的同时,其实也是在以一种隐性的方式维持了基学习器之间的差异性,从而使得最终的集成学习器的性能达到最佳。负相关学习(Negative correlation learning,NCL)是一种常用于神经网络集成的集成学习算法,它是把基学习器之间的差异性作为一个显性的度量标准引入到神经网络的损失函数中去,进而影响神经网络的训练。通过调整影响因子可以权衡基神经网络之间的性能与多样性,以谋求获得一个性能最优的集成神经网络模型。我们借鉴NCL的思路,提出了一种新的集成学习算法。NCL最早提出是使用神经网络作为基学习器,而且目前有关NCL的研究大多数还是采用神经网络作为基学习模型,主要原因是神经网络具有一个显性的损失函数,而且训练神经网络的BP算法是一种采用梯度下降方法来最小化该损失函数的优化算法。我们比较了神经网络与另外一种常用的学习模型：梯度提升模型(gradient boosting machine, GBM)之间的相似性,提出可以用GBM代替神经网络来实践负相关学习的思想,设计出一种新的集成学习算法：GB-NCL。论文给出了GB-NCL算法的设计思路以及详细的步骤,并且通过实验比较了GB-NCL与原始基于神经网络的NCL算法以及梯度提升算法的分类性能。实验结果表明GB-NCL算法相比于这两种算法,拥有更好的性能。论文的第二个工作是基于GB-NCL算法设计并实现了一种新的用于解决高光谱遥感图像分类问题的分类算法：RCASSL。高光谱遥感图像分类的特点是标记样本少,未标记的样本多,而且人工标记遥感图像的像素点属于什么地物类别的成本比较大。前人的做法主要有两种：第一种,利用主动学习算法,从大量未标记样本中挑选出最值得标记的像素点让人类专家来标记其所属的地物类别。这种方法的特点是新增训练样本的质量高(类标号百分百正确),但是数量少。第二种,利用半监督学习算法,用已训练出来的分类器赋予一些未标记样本类标号,并将其视为真实可用的样本,添加到训练集中,我们称之为“伪标记”样本。这类算法可以大大提高训练样本数量但是无法保证新增的伪标记样本的类标号一定正确。数量多,质量不好,这是半监督学习算法的特点。我们提出不妨将主动学习与半监督学习结合,并且引入一套“伪”标记样本验证的机制,对通过半监督学习引入进来的伪标记样本进行校验,将不合格的伪标记样本剔除出去,从而既能够获得足够多的训练样本,又能够保证训练样本集的质量。拥有了更大更完备的训练集,训练出来的分类器也就自然会有更佳的性能。根据这种想法我们在论文中针对高光谱遥感分类设计了RCASSL算法。RCASSL在训练分类器的时候不仅采用带标记的样本,而且使用半监督学习引入的伪标记样本。我们采用GB-NCL算法校验半监督学习方法引入的伪标记样本,提升伪标记样本集的质量。我们在高光谱遥感数数据集上对比了RCASSL算法、MCLU-ECBD算法以及RCASSL-NoPLV算法。MCLU-ECBD算法是一种常用的主动学习算法。RCASSL-NoPLV算法是去除掉伪标记样验证环节的RCASSL算法。实验的结果表明,在引入相同多的标记样本情况下,RCASSL算法的分类性能最强。RCASSL与MCLU-ECBD的对比结果说明结合半监督学习可以提升主动学习算法的性能,RCASSL与RCASSL-NoPLV的对比结果说明我们采用GB-NCL算法实现的伪标记验证机制的有效性。
[Abstract]:The first work of the paper is to study the integration of learning. In the field of machine learning, we call a system or model that has the ability to learn from empirical knowledge, called a learner. In general, it is much less expensive to train a weaker model than to train a more powerful model. Integrated learning is a kind of special machine learning method, its idea is not to train a strong learner directly, but by combining a group of weak learner to get an integrated learner with strong learning ability. The performance of the integrated learning algorithm depends on two factors: the performance of the base-based learner and the difference between the base-based learning devices. At present, the commonly used integrated learning algorithm includes Bagging, Boosting and the like, while the performance of each base learner is improved, the difference between the base-learning devices is maintained in a recessive way, so that the performance of the final integrated learner is optimized. Negative correlation learning (NCL) is a kind of integrated learning algorithm, which is commonly used in the integration of neural network, which is introduced into the loss function of the neural network as a dominant measure standard, and then influences the training of the neural network. The performance and diversity of the base neural network can be balanced by adjusting the influence factors, so as to obtain an integrated neural network model with optimal performance. Based on the idea of NCL, we put forward a new kind of integrated learning calculation The first point of NCL is to use the neural network as the base learner, and most of the research on the NCL is based on the neural network as the base learning model. The main reason is that the neural network has a dominant loss function. The BP algorithm of training neural network is a kind of optimization calculation using gradient descent method to minimize the loss function. This paper compares the similarity between the neural network and another commonly used learning model: the gradient lifting machine (GBM), and puts forward the idea of using the GBM instead of the neural network to practice the negative correlation study, and designs a new integrated learning algorithm: GB-NC L. The design idea and detailed steps of the GB-NCL algorithm are given in this paper, and the classification of the NCL algorithm and the gradient lifting algorithm based on the neural network are compared by the experiment. The results show that the GB-NCL algorithm has better performance compared with the two algorithms. The second work of the paper is to design and implement a new classification algorithm for high-spectral remote sensing image classification based on the GB-NCL algorithm: RCA The characteristic of high-spectral remote sensing image classification is that the mark sample is small, the unlabeled sample is more, and the pixel point of the remote sensing image of the artificial mark belongs to the cost of the object class. The first one, using the active learning algorithm, selects the most valuable pixel points from a large number of unlabeled samples to let the human expert mark the place to which it belongs. The feature of this method is that the quality of the new training samples is high (the class label is 100% correct), but Second, with a semi-supervised learning algorithm, the trained classifier is used to give some unlabeled sample-like reference numbers, and they are treated as real-available samples, added to the training set, and we call it "trunk>" dummy mark " Samples. This type of algorithm can greatly improve the number of training samples, but cannot guarantee the class label of the newly added pseudo-marker sample. It is correct. The quantity is too large and the quality is not good. This is a semi-supervised learning algorithm. The feature of this paper is to combine the active learning with the semi-supervised learning, and to introduce a set of "pseudo-"-labeled sample verification mechanism to check the pseudo-mark samples introduced in the semi-supervised learning and to use the non-qualified pseudo-marker samples. The method can not only obtain enough training samples, but also guarantee the training sample. The quality of this set. With a more complete set of training, the trained classifiers will naturally Better performance. According to this idea, we designed RCA for hyperspectral remote sensing in the paper The SSL algorithm. RCASSL not only uses the tagged samples while training the classifier, but uses semi-supervised learning to introduce Pseudo-mark samples. We use the GB-NCL algorithm to check the pseudo-marker samples introduced by the semi-supervised learning method to improve the pseudo-mark sample. We compared the RCASSL algorithm, the MCLU-ECBD algorithm and the RCASSL-No on the high-spectral remote sensing data set. The PLV algorithm. The MCLU-ECBD algorithm is a common master The RCASSL-NoPLV algorithm is an RCA to remove the pseudo-marker-like verification link. The results of the experiment show that, in the case of introducing the same number of tag samples, the algorithm of the RCASSL The result of comparison between RCASSL and MCLU-ECBD shows that combining semi-supervised learning can improve the performance of active learning algorithm, and the comparison between RCASSL and RCASSL-NoPLV shows that we use the GB-NCL algorithm to implement the pseudo-mark verification machine.
【学位授予单位】：中国科学技术大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP18;TP751

【共引文献】