基于神经网络的文本倾向性分类研究

发布时间：2018-01-28 13:01

本文关键词： 倾向性分类文本预处理特征抽取 BP神经网络　出处：《内蒙古大学》2017年硕士论文　论文类型：学位论文

【摘要】：互联网是随着时代发展产生的新型信息交流工具,如今已经进入了我们学习生活中的方方面面。与此同时网络上可交流的平台越来越多,在这些平台中各种各样的评论性文本也呈爆发式增长,对这些文本进行倾向分类,挖掘文本中潜在的价值具有十分重要的意义。各种平台的评论性文本包含着对时事、物品的正负评论等,对此类文本进行倾向性分类主要是以情感、舆论导向为主线进行分类。如今,这项工作越来越被关注同时吸引了大批学者。关于对文本进行倾向性分类主要有两类方法,一是借助于词典、语料库的方法。此方法在文本分类方面具有良好的效果,但是此方法过多依赖词典等外部元素,分类结果的好坏与词典等外部元素有直接关系,同时计算量较大。第二种方法是机器学习方法,该方法结合特征抽取技术,对文本用特殊的方式进行表示进而使用机器学习方法分类。不同的机器学习方法分类效果不同,且容易受到文本好坏的影响。本文以论坛评论性文本和电子商务网站评论性文本为载体,对文本进行倾向性分类。结合不同文本的特点,本文中人工采集了一组含噪声较多的论坛数据同时在网站采集了一组规范的电商评论数据。评论性文本结构复杂、语言风格多样,针对此类现象,首先对文本进行了处理,将文本格式等进行规范化,然后专门构造了分词词典并应用到了分词过程中去。结合文本向量表示方法及X 2统计量特征抽取方法构造了文本的向量矩阵。由于论坛和电商网站评论性文本存在噪声较多且特征多变,结合文本倾向分类的方式,本文选择利用文本预处理结合机器学习进行文本倾向性分类。在众多机器学习方法中,本文选用BP神经网络模型作为文本倾向性分类模型。本文中在文本分类模型训练过程中,构造了一层到两层的神经网络分类模型然后在不同的训练集上进行了比对实验,选出了最优模型进而对测试文本进行了测试实验。通过不同数据集对该模型的实验结果的分析,验证了 BP神经网络分类模型对含有噪声的文本的容错能力,分类效果较好。
[Abstract]:With the development of the times, the Internet is a new information exchange tool, and now it has entered all aspects of our study and life. At the same time, there are more and more communication platforms on the network. In these platforms, a variety of critical text is also explosive growth, these texts tend to classify. It is very important to excavate the potential value in the text. The critical texts of various platforms contain positive and negative comments on current affairs and articles, and the classification of this kind of text is mainly based on emotion. Public opinion is the main line of classification. Nowadays, this work has attracted more and more attention and attracted a large number of scholars. There are two main methods of text classification, one is the use of dictionaries. Corpus method. This method has a good effect in text classification, but this method relies too much on external elements such as dictionaries, and the classification results are directly related to external elements such as dictionaries. The second method is machine learning, which combines feature extraction technology. Text is represented in a special way and then classified by machine learning. Different machine learning methods have different classification effects. And it is easy to be influenced by the text. This paper takes the critical text of forum and the critical text of e-commerce website as the carrier to classify the text tendentiously and combine the characteristics of different texts. In this paper, a group of noisy forum data is collected manually and a set of standard ecommerce review data is collected on the website. The structure of the critical text is complex and the language style is diverse, aiming at this kind of phenomenon. Firstly, the text is processed and the text format is standardized. Then the lexicon of participle is constructed and applied to the process of word segmentation. Combining with the representation of text vector and X. (2) the statistical feature extraction method constructs the vector matrix of the text. Because of the frequent noise and the changeable feature of the critical text in the forum and the e-commerce website. Combined with text tendency classification, this paper chooses text preprocessing combined with machine learning to classify text tendentiousness. In many machine learning methods. In this paper, BP neural network model is selected as the text orientation classification model. In this paper, the text classification model training process. A neural network classification model with one to two layers is constructed and the comparison experiments are carried out on different training sets. The optimal model is selected and the test text is tested. The fault tolerance ability of BP neural network classification model for noisy text is verified by analyzing the experimental results of the model in different data sets. The classification effect is better.
【学位授予单位】：内蒙古大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.1;TP183

【相似文献】