基于BTM模型的微博话题检测与追踪研究

发布时间：2018-01-28 04:25

本文关键词： 微博话题模型话题检测话题追踪话题词表　出处：《昆明理工大学》2017年硕士论文　论文类型：学位论文

【摘要】：近年来,微博的迅猛发展给人们的沟通交流提供了诸多方便,已经成为社交网络的重要组成部分。由于微博平台具有传播速度快,互动性强等较为显著的特点,用户可以随时利用碎片时间汲取新闻资讯,发布新奇事件,并参与话题。微博平台逐渐呈现出数据量大,文本短小的状态,进而出现信息负载,讯息迷向等问题,这使得微博用户难以有效获取和掌握相关话题的后续信息。对微博进行话题检测与追踪,不仅可以提供给用户感兴趣的话题及其后续报道,还能够有效地引导公众舆论,具有一定的现实意义。微博文本较短,词频较低,缺乏丰富的上下文。传统的话题模型在处理微博短文本方面面临着严重的数据稀疏问题,降低了性能。本文结合了微博的显著特点,进行了基于BTM(bi-term topic model)模型的微博话题检测与追踪研究。首先,本文引入BTM话题模型来处理微博数据这样的短文本。BTM模型是基于词共现建模进行话题学习的,它将所有的bi-term词对构成语料库,并利用语料库的词共现抽取话题分布。BTM能够充分扩展词项所表达的话题,改善LDA模型处理微博文本时面临的数据稀疏问题。其次,本文结合K-means聚类算法进行话题检测。BTM建模后的数据比较集中,并且话题间的差别较为明显,利用K-means聚类算法能够获得区分度较好的话题。在分别结合K-means和层次聚类算法进行实验后,通过分析二者的比对结果,本文采用BTMK-means聚类算法进行微博话题检测。最后,本文在建模的基础上利用BTM模型的话题-话题词表进行话题追踪。研究过程中,本文融入顺序加权方法对相似度计算方法进行改进,经过比较词项间的语义相似度,计算微博话题词表与BTM话题词表之间的相似度,用以改善特征自身携带的语义信息忽视问题。
[Abstract]:In recent years, Weibo's rapid development has provided a lot of convenience for people's communication and communication, and has become an important part of social network. Because Weibo platform has the characteristics of fast transmission, strong interaction and so on. Users can take advantage of the fragment time at any time to absorb news information, publish novel events, and participate in the topic. Weibo platform gradually presents a large amount of data, short text status, and then appear information load. This makes it difficult for Weibo users to effectively obtain and master the follow-up information of relevant topics. The topic detection and tracking of Weibo can not only provide users with topics of interest and follow-up reports. Also can effectively guide public opinion, has certain practical significance. Weibo text is shorter, the word frequency is low. The traditional topic model is faced with serious data sparse problem in dealing with Weibo short text, which reduces the performance. This paper combines the remarkable characteristics of Weibo. The research of Weibo topic detection and tracking based on BTM(bi-term topic model is carried out. First of all. In this paper, we introduce BTM topic model to deal with Weibo data. BTM model is based on co-occurrence model for topic learning. It uses all bi-term word pairs to form a corpus. And using the word co-occurrence in the corpus to extract the topic distribution. BTM can fully expand the topic expressed by the word items, and improve the LDA model to deal with Weibo text data sparse problem. Secondly. This paper combines K-means clustering algorithm for topic detection. BTM modeling data is more concentrated, and the difference between topics is obvious. K-means clustering algorithm can be used to achieve a better classification of topics. After the combination of K-means and hierarchical clustering algorithm experiments, through the analysis of the results of the comparison between the two. In this paper, BTMK-means clustering algorithm is used to detect Weibo topic. Finally, based on the modeling, this paper uses the topic-topic lexicon of BTM model to track the topic. By comparing the semantic similarity between word items, the similarity between Weibo topic lexicon and BTM topic lexicon is calculated. In order to improve the semantic information carried by the feature itself ignore the problem.
【学位授予单位】：昆明理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.1

【参考文献】