基于中文微博的热门话题提取与追踪

发布时间：2018-08-11 19:55

【摘要】：自微博被推出以来,就凭借其广泛的参与性,改变了人们获取新闻和了解时事的方式。近年来的许多突发新闻和热门话题都是通过微博平台率先发布出来的,其传播速度和范围都是传统媒介所不能比拟的。目前,仅仅是新浪微博每日的发帖数量就达到了数亿条,这些海量的数据涵盖了人们生活的方方面面,包含了很多有价值的话题信息,若是能正确提取这些热点话题,对于了解最新的舆论热点,掌握舆论动向都是有很大意义的。然而,在面对这种量级的数据时,仅凭人工来进行处理是远远不够的,加之微博帖子都是短文本,具有很严重的数据稀疏性,一些传统的话题提取与追踪算法也不能直接用来进行处理。本文针对这个现象,对微博热门话题提取和追踪进行了研究,具体工作主要有:1.本文提出了一个改进的话题提取模型MF-LDA(Microblog Features Latent Dirichlet Allocation)来提取微博中的热门话题,该模型改进了传统的LDA(Latent Dirichlet Allocation)模型,通过结合微博特有的五种特征:赞数、评论数、转发数、发帖时间和用户权威来更好地提取出热门话题。其中,赞数、转发数和评论数用来计算微博的关注度,用户权威用来计算微博的权威值,然后将微博按照发帖时间划分到对应的时间窗口内,再统计出每个时间窗口中的微博帖子的词频。将计算出的特征值表示成特征向量,并加入到MF-LDA模型中进行吉布斯采样,再通过不断地迭代训练,得到模型的最优参数,最后获取到每个词语的概率,概率越高表明这个词语越有可能是个热门话题词。2.本文在对热门话题进行追踪时,主要是从结构和内容两方面来追踪话题的演变过程。对于话题结构上的追踪,本文首先构建了热门话题生命周期模型HTLCM(Hot Topic Life Cycle Model),并将话题的生命周期分为了五个阶段:诞生期、成长期、成熟期、衰退期和消失期,通过计算话题在单位时间内的话题数量、增长速度和新增率等特征来判断话题现在处于的阶段以及是否为一个候选热门话题,从而就可以了解到该话题的整体发展情况。对于话题内容上的追踪,本文将MF-LDA模型和HTLCM模型进行整合,提出了HTT(Hot Topic Tracking Algorithm)算法,该算法通过加入时间窗口,将HTLCM模型标记的候选热门话题按照发布时间分配到对应的时间窗口中,然后将每个时间窗口的数据输入到MF-LDA模型中,从而就可以得到该热门话题在每个时间窗口中与之最相关的关键词,通过分析关键词的变化,就可以追踪到话题在内容上的变化。最后,本文为了验证提出模型和算法的有效性,在真实的数据集上进行了相关实验与分析。实验结果表明MF-LDA模型在同等条件下的Perplexity(困惑度)低于LDA模型,但覆盖率高于LDA模型。本文提出的HTLCM模型和HTT算法不仅能对热门话题进行持续追踪,还能有效的发现潜在的热门话题。最终实验结果显示,本文提出的模型和方法在热门话题提取和追踪方面具备良好的效果和实践意义。
[Abstract]:Since its launch, Weibo has changed the way people get news and get to know current events with its wide participation. In recent years, many breaking news and hot topics have been released through the microblogging platform, and its speed and scope of transmission are incomparable with traditional media. At present, only Sina Weibo is daily. The number of Posts has reached hundreds of millions. These huge amounts of data cover all aspects of people's lives and contain a lot of valuable topic information. If we can extract these hot topics correctly, it is of great significance for us to understand the latest hot topics of public opinion and grasp the trend of public opinion. However, in the face of this magnitude of data, we only rely on people. It is far from enough to process microblog posts. In addition, microblog posts are short texts and have very serious data sparsity. Some traditional topic extraction and tracking algorithms can not be directly used for processing. An improved topic extraction model MF-LDA (Microblog Features Latent Dirichlet Allocation) is proposed to extract hot topics from microblogs. This model improves the traditional LDA (Latent Dirichlet Allocation) model by combining the five unique features of microblogs: praise, comment, forwarding, posting time and user authority. Among them, praise number, forwarding number and comment number are used to calculate the attention of micro-blog, user authority is used to calculate the authoritative value of micro-blog, and then divide the micro-blog into corresponding time windows according to the posting time, and then count the word frequency of micro-blog posts in each time window. The higher the probability, the more likely the word is to be a hot topic. 2. This paper traces the hot topic mainly from the structure and content aspects. To track the topic structure, this paper firstly constructs the Hot Topic Life Cycle Model (HTLCM), and divides the topic life cycle into five stages: birth, growth, maturity, decline and disappearance, by calculating the number of topics in a unit time, the growth rate. This paper integrates MF-LDA model with HTLCM model and proposes an HTT (Hot Topic Tracking Algorithm) algorithm for tracking the topic content. In the time window, the candidate hot topics marked by the HTLCM model are allocated to the corresponding time window according to the publishing time, and then the data of each time window is input into the MF-LDA model, so that the most relevant keywords of the hot topic in each time window can be obtained. By analyzing the changes of the keywords, the key words can be obtained. Finally, in order to verify the validity of the proposed model and algorithm, experiments and analysis are carried out on real data sets. The experimental results show that the Perplexity (perplexity) of MF-LDA model under the same conditions is lower than that of LDA model, but the coverage rate of MF-LDA model is higher than that of LDA model. The algorithm can not only keep track of hot topics, but also find potential hot topics effectively. The experimental results show that the proposed model and method have good effect and practical significance in Hot Topics Extraction and tracking.
【学位授予单位】：西华大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.1;TP393.092

【相似文献】