微博热点话题发现技术研究

发布时间：2018-04-10 04:32

本文选题：微博　切入点：话题发现　出处：《内蒙古科技大学》2013年硕士论文

【摘要】：近年来，微博的迅猛发展，已经成为人们获取和传播信息的最主要的形式，由微博所引发的社会问题和危机事件也日呈频发态势。如何在纷繁复杂的微博信息发现热点话题，已经成为话题发现和跟踪领域研究领域的热点。同时，微博热点话题的发现，为网络舆情的监管，掌握事件发展动态，都提供有利的数据保证，具有十分重要的理论价值和现实意义。本文首先从微博信息收集入手，针对微博平台上API受限或不稳定等问题，自行设计了网络爬虫工具，并实现了信息收集手动和自动两种方式。其次，本文采用lucene对微博文本信息进行预处理，，针对原有向量空间的不足，采用潜在语义分析方法对向量空间进行奇异值分解，达到了降低词袋维度和消除语义噪音等效果。再次，在分析了文本分类的各种算法的优缺点基础上，提出了采用朴素贝叶斯分类算法实现话题发现及跟踪。朴素贝叶斯分类算法是一种非常简单和有效的文本分类算法，计算待分类项出现的条件下，其它已知分类出现的概率，选择概率最大的，将待分类项归为该类。接下来，本文根据微博用户特征，通过lucene关键词的倒排索引，快速找到该关键词所在微博，继而找到该微博用户的关注度，通过词语的关注度，找到微博的关注度，最终计算话题关注度。相对于微博传统计算话题热度的方法，这种算法更多地考虑到微博自身用户特点，所以更有效和准确。最后，基于以上工作，本文实现了微博舆情分析系统，并对实验数据进行了测试，通过对实验结果的分析，找出在研究过程中的有待改进的地方，更加明确了今后研究工作的重点。
[Abstract]:In recent years, with the rapid development of Weibo, it has become the most important form for people to obtain and disseminate information.How to find hot topics in the complicated Weibo information field has become a hot topic in the field of topic discovery and tracking.At the same time, the discovery of Weibo's hot topic, which provides favorable data guarantee for the supervision of network public opinion and the development of events, has very important theoretical value and practical significance.This paper starts with Weibo information collection, aiming at the problem of limited or unstable API on Weibo platform, designs a web crawler tool, and realizes two ways of information collection manually and automatically.Secondly, this paper uses lucene to preprocess Weibo text information, aiming at the deficiency of the original vector space, using latent semantic analysis method to decompose the singular value of vector space, which can reduce the dimension of word bag and eliminate semantic noise and so on.Thirdly, on the basis of analyzing the advantages and disadvantages of various text classification algorithms, a naive Bayesian classification algorithm is proposed to realize topic discovery and tracking.Naive Bayes classification algorithm is a very simple and effective text classification algorithm. When the items to be classified appear, the probability of other known categories is calculated, and the items to be classified are classified as this class if the probability is the greatest.Next, according to Weibo user characteristics, through the inverted index of lucene keyword, this paper quickly finds the user's attention degree of the keyword, then finds out the attention degree of the user, through the attention degree of the words, find the attention degree of Weibo.Finally calculate the topic of concern.Compared with Weibo's traditional method of calculating topic heat, this algorithm is more effective and accurate because of taking into account the user's characteristics.Finally, based on the above work, this paper realizes Weibo public opinion analysis system, and tests the experimental data, through the analysis of the experimental results, find out in the process of research areas to be improved.The emphases of future research work are more clearly defined.
【学位授予单位】：内蒙古科技大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP393.092;TP391.1

【参考文献】