基于主动学习的微博情感分析方法研究

发布时间：2018-05-27 00:05

本文选题：微博情感分析 + 主动学习　；参考：《吉林大学》2017年硕士论文

【摘要】：目前,作为文本挖掘重要分支之一的文本情感分析受到学者的广泛关注。随着互联网的飞速发展和社交媒体的普及,网上产生了大量的用户衍生文本,而这些文本主观性很强并带有明显的情感倾向和丰富的情感信息,具有很高的研究价值。主流的情感分类方法广泛采用了机器学习,这种方法的局限在于需要大规模标注语料作为训练集,这需要花费巨大的成本来标注语料。而在实践当中,容易获得的都是未标注的文本语料,因此,如何利用少量的标注语料和大量的未标注语料进行文本情感分类成为了一个重要课题。本文将主动学习方法结合进基于机器学习的文本情感分类方法中,以解决未标记语料的有效利用问题。由于文本特征矩阵的稀疏性,采用支持向量机作为基分类器在准确度上有着较大优势。边缘采样方法是利用支持向量机进行主动学习的经典方法,但该方法同样存在着错误级联、过拟合和冗余迭代等一些准确率和性能上的问题。本文针对这些问题并在同样使用支持向量机作为基分类器的基础上提出了一个新的主动学习方法(Active Learning in Informative Vector Selection-----ALIVS)。主要工作如下:第一,本研究对文本情感分类和主动学习的理论进行了系统研究,分析了文本情感分类的主要任务、研究流派以及主动学习的基本假设和主流方法等基础理论。并对经典的基于边缘的主动学习方法进行了研究和分析,发现其存在的局限。第二,本研究以上文所述的理论研究为起点,提出新的主动学习方法ALIVS,该方法利用未标记样本集的特点提出了信息向量(Informative Vector)的概念并结合支持向量机发展出一个二级分类的学习流程,该流程基于以下想法:采用两级分类器,第一级主分类器负责情感分类;第二级信息向量分类器利用第一级分类器学习到的分类信息从未标记样本中遴选出最具分类信息的信息向量作为候选标记样本,经专家标记后,加入第一级分类器的训练集中,循环迭代,不断增强第一级分类器的分类能力,进而达成利用大量的未标记文本和少量的标记文本进行有效训练的目标。第三,本研究将该方法应用到基于COAE2014评测的任务4的实际场景中,并与广泛应用的边缘采样方法进行对比,设计实验对该方法的准确度和性能进行了测试和分析。实验结果表明,本文提出的ALIVS方法在提高准确率、降低过拟合及错误级联等方面有着良好的表现,从而证明了该方法的可行性。最后本文对该方法在未来的改进和发展进行了展望。
[Abstract]:At present, as one of the important branches of text mining, text emotional analysis has been widely concerned by scholars. With the rapid development of the Internet and the popularity of social media, a large number of user-derived texts have been generated on the Internet, and these texts are highly subjective, with obvious emotional tendency and rich emotional information, which has high research value. Machine learning is widely used in the mainstream affective classification methods. The limitation of this method lies in the need of large-scale tagging corpus as a training set, which requires a huge cost to annotate the corpus. In practice, it is easy to obtain unannotated text corpus, so how to use a small amount of annotated corpus and a large amount of unlabeled corpus to classify text emotion has become an important topic. In this paper, the active learning method is combined with the text emotion classification method based on machine learning to solve the problem of the effective use of unmarked corpus. Because of the sparsity of text feature matrix, support vector machine (SVM) as the basis classifier has a great advantage in accuracy. Edge sampling is a classical method for active learning using support vector machines, but it also has some problems in accuracy and performance, such as error concatenation, overfitting and redundant iteration. In this paper, we propose a new active learning method, active Learning in Informative Vector Selection-ALIVSs, based on the same support vector machine (SVM) as a basis classifier for these problems. The main work is as follows: first, this study systematically studies the theories of text emotion classification and active learning, analyzes the main tasks of text emotion classification, the basic hypothesis and mainstream methods of active learning. The classical edge-based active learning method is studied and analyzed, and its limitations are found. Second, this study starts with the theoretical research mentioned above. A new active learning method, ALIVS, is proposed in this paper. Based on the characteristics of unlabeled sample sets, the concept of information vector Informative vector is proposed and a secondary classification process is developed by combining support vector machine. The process is based on the following ideas: a two-level classifier is used, and the first primary classifier is responsible for emotion classification; The second level information vector classifier uses the information vector of the first level classifier to select the information vector with the most classified information as the candidate marker sample, and adds the training set of the first level classifier after the expert mark. Cyclic iteration enhances the classification ability of the first level classifier and achieves the goal of using a large number of unmarked text and a small amount of marked text for effective training. Thirdly, this method is applied to the actual scenario of task 4 based on COAE2014 evaluation, and compared with the widely used edge sampling method. The accuracy and performance of the method are tested and analyzed by experiments. The experimental results show that the proposed ALIVS method has a good performance in improving the accuracy, reducing over-fitting and error concatenation, which proves the feasibility of this method. Finally, the improvement and development of this method in the future are prospected.
【学位授予单位】：吉林大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.1

【相似文献】