微博健康热点话题发现系统的设计与实现

发布时间：2018-12-12 20:21

【摘要】：随着互联网技术的不断发展,微博作为Web3.0新兴起的一类开放互联网社交平台,凭借其使用方便、信息传播速度快、平台开放性以及交互性强等优点,成为备受广大用户喜爱的分享、获取以及传播信息的重要平台。微博上每天都会产生大量的数据,这些数据蕴含着丰富的元数据信息。目前,传统的话题发现模型及文本聚类技术已经被广泛应用于各个领域,并取得了不错的成果。针对微博短文本数据,传统的话题发现方法仍然存在着很大的局限性,给微博热点话题发现带来了新的挑战。因此,如何准确快速的从海量微博数据中获取热点话题信息并将其及时展示给广大用户,是微博话题发现技术亟待解决的问题。基于以上研究背景,本文结合微博的特点,改进传统的话题发现模型以及文本聚类算法,提出了一种基于VSM模型和MLDA模型相结合的微博话题发现算法,在此基础上设计了微博健康话题发现系统。主要工作内容如下:首先,采用微博平台开放API接口和网络爬虫技术,分别对微博用户信息和微博内容两方面数据进行爬取,通过数据去噪、文本分词等操作对微博文本数据进行预处理;其次,利用VSM模型和MLDA模型对微博文本数据进行联合建模,构造微博文本的特征向量,实现微博文本相似度的联合计算。再次,采用改进传统的Single-pass算法和凝聚式层次聚类算法,对微博文数据进行二次聚类,实现微博热点话题的抽取以及话题热度的计算与排序。最后,通过多方面实验验证与分析证明了系统有效性和准确性。
[Abstract]:With the continuous development of Internet technology, Weibo, as a new type of open Internet social platform of Web3.0, has the advantages of convenient use, fast information dissemination, open and interactive platform, etc. It has become an important platform for sharing, obtaining and disseminating information. Weibo produces a lot of data every day, which contains abundant metadata information. At present, traditional topic discovery models and text clustering techniques have been widely used in various fields, and have achieved good results. In view of Weibo's short text, the traditional method of topic discovery still has great limitations, which brings new challenges to Weibo's hot topic discovery. Therefore, how to accurately and quickly obtain hot topic information from mass Weibo data and display it to the majority of users in time is a problem to be solved urgently by Weibo topic discovery technology. Based on the above research background, this paper combines Weibo's characteristics, improves the traditional topic discovery model and text clustering algorithm, and puts forward a topic discovery algorithm based on the combination of VSM model and MLDA model. On this basis, Weibo health topic discovery system is designed. The main work contents are as follows: firstly, by using Weibo platform open API interface and network crawler technology, we crawled the data in two aspects, the user information of Weibo and the content of Weibo, respectively, and removed the noise through the data. Text segmentation and other operations to pre-process Weibo text data; Secondly, using VSM model and MLDA model, the text data of Weibo are modeled jointly, and then the feature vector is constructed to realize the joint calculation of the similarity between the two texts. Thirdly, using the improved traditional Single-pass algorithm and the condensed hierarchical clustering algorithm, the Weibo text data are clustered twice to realize the hot topic extraction and the calculation and sorting of topic heat. Finally, the validity and accuracy of the system are proved by many experiments.
【学位授予单位】：西安理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.1;TP393.092

【参考文献】