网络热点话题实时发现技术研究与实现

发布时间：2018-02-04 15:29

本文关键词： 热点话题发现网络爬虫中文分词极大团挖掘话题展示平台　出处：《北京邮电大学》2014年硕士论文　论文类型：学位论文

【摘要】：随着互联网、社交平台以及移动技术的飞速发展,人们越来越多的和网络接触,并在互联网上和他人分享自己的观点。人们每天所关心的、谈论的内容即是本文提及的热点话题。热点话题可以在政治、经济、文化等领域发挥重要的作用,所以对热点话题实时发现技术的研究具有很高的应用价值。本论文以此为出发点,研究了热点话题实时发现相关技术,并实现了热点话题实时发现系统。本文的主要工作如下：第一,设计并完成了热点话题实时发现系统,可以查看热点话题详情；第二,提出并实现了基于模板的爬虫采集技术,并应用于系统的信息采集模块,解决了对网络新闻和微博数据的爬取问题,能够高效的采集数据；第三,提出并实现了基于词频的中文分词法,并应用于文本预处理模块,改善了分词过程中的歧义问题,能够得到更为准确的分词结果；第四,提出了一种改进的准极大团挖掘方法,并应用于话题提取模块,解决了极大团中相似话题的合并问题,能够得到更为准确的话题。本文论述的系统可以高效的采集新闻、微博数据,并成功进行了文本预处理和话题的提取,最后在前端平台展示结果。该系统具有较高的实际应用价值。
[Abstract]:With the rapid development of the Internet, social platforms and mobile technology, people are more and more in contact with the Internet and share their views with others on the Internet. What we are talking about is a hot topic mentioned in this paper. Hot topics can play an important role in the fields of politics, economy, culture and so on. Therefore, the research on real-time discovery of hot topics has high application value. This paper studies the related technologies of real-time discovery of hot topics, and realizes the real-time discovery system of hot topics. The main work of this paper is as follows:. First, a real-time hot topic discovery system is designed and completed, which can view the details of hot topic. Secondly, the crawler acquisition technology based on template is put forward and implemented, and it is applied to the information collection module of the system, which solves the crawling problem of network news and Weibo data, and can collect data efficiently. Thirdly, the Chinese word segmentation method based on word frequency is put forward and implemented, and it is applied to the text preprocessing module, which improves the ambiguity in the segmentation process and can get more accurate segmentation results. In 4th, an improved quasi-maximal cluster mining method is proposed and applied to the topic extraction module, which solves the problem of merging similar topics in the maximal cluster and can obtain more accurate topics. The system discussed in this paper can collect news and Weibo data efficiently, and successfully carry out text preprocessing and topic extraction, and finally display the results on the front-end platform. The system has high practical application value.
【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP391.1

【参考文献】