基于Hadoop的中文微博主观情感分类的研究与实现

发布时间：2018-03-23 06:33

本文选题：微博　切入点：情感分类　出处：《华东师范大学》2014年硕士论文　论文类型：学位论文

【摘要】：微博是互联网快速发展的一个产物,以其快速、随意的特点吸引了大量的用户。微博用户热衷于在上面发布自己的状态及观点。随着微博的不断发展,用户量不断增长,截止至2013年,仅新浪微博的用户量便已达到了5.03亿。如此巨大的用户量导致每天微博上新产生的信息不计其数。这些微博信息通常都具有强烈的感情色彩,分析这些信息能够获得很多隐藏的知识、数据,能为生活、工作带来很大的帮助。文本情感分类的研究与应用目前在国内外信息研究与应用领域正在得到大家的普遍关注,成为一个热点的信息研究、挖掘与应用的课题。互联网的发展带动了大数据时代的到来。大数据一般指一个公司或企业创造的大量非结构化和半结构化数据。微博信息便是典型的大数据之一。然而,由于传统的关系型数据库的不足,在它上面分析大数据将会花费过多的时间和金钱。云计算的出现弥补了这些不足,满足了大数据的分析需求。云计算将作业的计算任务分布到了大量的分布式计算机上,拥有强大的计算处理能力。在现今的研究开发过程中,大数据的分析通常都与云计算联系在一起。本文在研究分析了现有的文本情感分类技术的基础上,结合微博文本的特点,提出了一个基于Hadoop的中文微博主观情感分类系统。首先,根据微博信息的特点,建立了适用于中文微博情感分类的情感字典；其次,根据Map/Reduce编程模型,提出了基于Hadoop的分布式朴素贝叶斯分类算法,用以处理大规模的新浪微博数据；然后,针对包含最新网络词汇的信息,辅助以基于情感字典的情感分类法来进行情感分类；最后,选择适当的融合算法来结合本文两个算法的结果,以得到最终的分类结果。本文将该系统应用于新浪微博,实验结果表明,本文提出的方法能够很好地适用于微博情感分类,达到了较理想的分类效果,对处理海量的中文微博信息具有可行性。
[Abstract]:Weibo is a product of the rapid development of the Internet, which has attracted a large number of users with its rapid and casual characteristics. Weibo users are keen to publish their status and views on the Internet. With the continuous development of Weibo, the number of users has been growing. As of 2013, the number of users of Sina Weibo alone had reached 503 million. With such a huge number of users, there are countless new messages generated every day on Weibo, which usually have a strong emotional color. Analysis of this information can obtain a lot of hidden knowledge, data, can bring a lot of help to life and work. The research and application of text emotion classification is getting the universal attention in the field of information research and application at home and abroad. Become a hot topic of information research, mining and application. The development of the Internet has brought about the arrival of big data's era. Big data generally refers to a large number of unstructured and semi-structured data created by a company or enterprise. Because of the shortcomings of the traditional relational database, it will take too much time and money to analyze big data on it. The emergence of cloud computing has made up for these shortcomings. Cloud computing distributes the computing tasks of jobs to a large number of distributed computers, and has powerful computing and processing power. In today's research and development process, Big data's analysis is usually linked to cloud computing. Based on the research and analysis of the existing text emotion classification technology and the characteristics of Weibo's text, this paper puts forward a subjective emotion classification system based on Hadoop. First of all, according to the characteristics of Weibo information, Secondly, according to the Map/Reduce programming model, a distributed naive Bayesian classification algorithm based on Hadoop is proposed to deal with large-scale Sina Weibo data. For the information containing the latest online vocabulary, the emotion classification method based on emotion dictionary is used to classify emotion. Finally, the appropriate fusion algorithm is selected to combine the results of the two algorithms in this paper. In order to obtain the final classification result, this paper applies the system to Sina Weibo. The experimental results show that the method proposed in this paper can be well applied to Weibo emotional classification, and achieve a better classification effect. It is feasible to deal with huge amount of Chinese Weibo information.
【学位授予单位】：华东师范大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.092;TP391.1

【共引文献】