基于微博文本的话题聚类研究与实现

发布时间：2018-03-31 04:27

本文选题：话题聚类　切入点：微博　出处：《河北科技大学》2014年硕士论文

【摘要】：Web2.0技术的出现推动了大数据时代的到来。以微博为代表的社会网络新媒体的兴起，在丰富了大数据的同时，也给数据挖掘和知识发现带来了诸多挑战，因为和传统的文本信息不同，微博信息涉及个人兴趣、娱乐、企业营销、公益宣传等多方面内容，且内容碎片化，数据量庞大。如何有效分析和挖掘其中的信息，成为当前一个重要的研究课题。话题聚类技术是微博研究的一项基础工作。它将海量的微博数据自动归类，使话题内容相近的微博自动归为一类。话题聚类结果可用于深层话题分析与数据挖掘。传统的基于关键字的处理方法可能会返回数以千百计的结果，其内容多是重复性的或关联较小的内容，不适合处理上述问题。话题聚类技术可通过智能处理算法，使内容相近的信息自动聚在一起，，便于了解该类的主要话题；通过主题词提取，可使处理结果更加直观。本文基于微博文本，通过采用智能处理算法，完成了如下工作：首先，获取微博的结构化数据，完成对微博数据的采集以及在聚类前对数据的预处理。第二，完成了针对文本聚类的特征词选择，针对微博短文本的特点，研究如何有效提取特征词以及筛选特征词。第三，设计了有效的聚类算法，针对微博短文本，研究何种算法可使微博文本的聚类结果更好。第四，对聚类结果集的主题词进行提取，该主题词可作为可视化话题的主要表达内容。第五，完成了处理结果的可视化，使结果更加清晰和直观，起到了辅助理解数据与识别隐藏在数据中的规律的作用。实验结果与分析显示了本文方法的有效性。同时，本文也对可能出现的问题及下一步的研究计划进行了说明。
[Abstract]:The emergence of Web2.0 technology promoted the arrival of big data era.The rise of new social network media, represented by Weibo, not only enriches big data, but also brings many challenges to data mining and knowledge discovery. Unlike traditional text information, Weibo information involves personal interests and entertainment.Enterprise marketing, public welfare publicity and other aspects of content, and content fragmentation, huge amount of data.How to effectively analyze and excavate the information has become an important research topic.Topic clustering is a basic work of Weibo.It classifies the massive Weibo data automatically, causes the topic content similar Weibo to fall into one category automatically.Topic clustering results can be used for deep topic analysis and data mining.Traditional keyword-based processing methods may return hundreds of results, most of which are repetitive or less correlated, so it is not suitable to deal with the above problems.The topic clustering technology can make the information of similar content gather together automatically through intelligent processing algorithm, so as to understand the main topic of this kind of topic conveniently, and can make the processing result more intuitionistic by extracting the theme words.Based on Weibo text, this paper completes the following work by adopting intelligent processing algorithm: firstly, we obtain the structured data of Weibo, complete the data collection and pre-processing of the data before clustering.Secondly, the selection of feature words for text clustering is completed. According to the characteristics of Weibo short text, the effective extraction of feature words and the selection of feature words are studied.Thirdly, an effective clustering algorithm is designed.Fourthly, the theme words of clustering result set are extracted, which can be used as the main expression of visual topics.Fifth, the visualization of the processing results is completed, which makes the results more clear and intuitive, and plays a role in understanding the data and recognizing the laws hidden in the data.Experimental results and analysis show the effectiveness of the proposed method.At the same time, this paper also explains the possible problems and the next research plan.
【学位授予单位】：河北科技大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP391.1;TP393.092

【参考文献】