基于最大树划分的微博热点话题检测方法研究

发布时间：2018-06-15 17:55

本文选题：微博 + 热点话题检测　；参考：《重庆大学》2014年硕士论文

【摘要】：随着传统互联网技术和移动互联网技术的快速发展，网络信息的传播速度和传播规模都得到了极大的增长，人们的交流方式也随之发生了改变。微博作为迅速崛起的新兴网络媒体，，越来越多地受到人们的关注。作为消息传播和互动交流的平台微博能够在短时间内产生大量的信息，这使得用户很容易陷入到局部的微博信息中而失去了对整个微博空间最新动态的了解。面对浩瀚的微博信息，如何能够快速准确地获取整个微博社区中的热点话题，已经成为一个重要的研究方向。虽然传统的话题检测技术已经相对比较成熟，能够帮助用户快速地获取隐藏在大量长文本中的话题。但是该类方法在处理海量微博短文本时仍然存在着明显的不足：一是计算复杂度过高，海量微博信息间的文本相似度计算对传统话题检测系统是致命的；二是丢失词语的语义信息，在传统的话题检测模型中，仅仅通过文档间重复词语的多少来判定文档的相似度，忽略了词语之间的语义关联。针对上述问题，本文通过对微博热点话题检测相关理论和算法的学习，分析现有的微博热点话题检测算法的优缺点，结合微博自身的特点，提出了一种基于最大树划分的微博热点话题检测方法。通过在采集到的微博数据集上进行的大量实验，验证了本文方法的有效性。本文所提出方法的主要贡献如下： ①提出了只针对一段时间内的微博数据进行话题检测的思想，这符合实际中微博系统对热点话题检测功能的要求，同时能够很好地去除在话题检测的过程中历史已有话题对新话题检测的影响。 ②改进了特征项权重和微博相似度的计算方法。通过将词语间的语义相似信息结合到现有的计算方法中，达到了降低中文微博由于一词多义和一义多词现象所造成的计算误差的目的，提高了计算的准确性。 ③提出了基于最大树划分的微博热点话题检测方法。通过对模糊相似矩阵进行最大树生成有效地去除了微博彼此间那些似是而非的噪音相似数据，降低了计算规模。同时，采用改进的K-means聚类算法能够自动确定聚类个数，使得聚类结果更加准确。另外，提出了计算微博话题热度的方法，用以对微博话题的热度进行排序，发现热点话题。 ④在整体执行效率、准确率方面相较其他微博话题检测方法有所提高，有效提高了传统话题检测算法在处理大规模数据时存在的效率低下问题。
[Abstract]:With the rapid development of traditional Internet technology and mobile Internet technology, the speed and scale of network information transmission have been greatly increased, and the way people communicate has also changed. Weibo as a rapidly rising network media, more and more people pay attention to. As a platform for message dissemination and interactive communication, Weibo can generate a large amount of information in a short time, which makes it easy for users to fall into the local Weibo information and lose their understanding of the latest developments in the entire Weibo space. In the face of the vast amount of Weibo information, how to quickly and accurately access the hot topics in the whole Weibo community has become an important research direction. Although the traditional topic detection technology is relatively mature, it can help users to quickly obtain topics hidden in a large number of long text. However, this kind of method still has obvious shortcomings in dealing with massive Weibo short text: first, the computational complexity is too high, the text similarity calculation between massive Weibo information is fatal to the traditional topic detection system; The second is the loss of semantic information of words. In the traditional topic detection model the similarity of documents is judged only by the number of repeated words between documents and the semantic association between words is ignored. In view of the above problems, this paper analyzes the advantages and disadvantages of the existing Weibo hot topic detection algorithms by studying the relevant theories and algorithms of Weibo hot topic detection, and combines the characteristics of Weibo itself. A method of Weibo hot topic detection based on maximal tree partition is proposed. The effectiveness of the proposed method is verified by a large number of experiments on the collected Weibo data sets. The main contributions of the proposed method are as follows: 1 the idea of topic detection for Weibo data for a period of time is proposed, which accords with the requirement of Weibo system for hot topic detection in practice. At the same time, it can remove the influence of the historical topic on the new topic detection. 2. The method of calculating the weight of feature item and the similarity of Weibo is improved. By combining the semantic similarity information between words and phrases into the existing calculation methods, the purpose of reducing the calculation error caused by the phenomenon of polysemy and multi-word meaning in Chinese Weibo is achieved. The accuracy of calculation is improved. 3 Weibo hot topic detection method based on maximal tree partition is proposed. By generating the maximum tree of the fuzzy similarity matrix, the specious noise similarity data between Weibo and each other are removed effectively, and the computational scale is reduced. At the same time, the improved K-means clustering algorithm can automatically determine the number of clustering, making the clustering results more accurate. In addition, a method to calculate the heat of Weibo topics is proposed, which is used to sort the heat of Weibo topics and find hot topics. 4 the overall execution efficiency and accuracy are improved compared with other Weibo topic detection methods. It effectively improves the efficiency of traditional topic detection algorithm in dealing with large-scale data.
【学位授予单位】：重庆大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.092;TP391.1

【参考文献】