推特中的非特定事件检测方法研究

发布时间：2018-04-28 22:27

本文选题：短文本 + 语义相似度　；参考：《电子科技大学》2017年硕士论文

【摘要】：社交网络的兴起和发展带给人们生活极大的便利和变化。社交网络网站一般拥有百万以上的注册用户,全球范围内著名的社交网络网站有脸书(FaceBook)、推特(Twitter)、微博等。社交网络中传播着大量有用信息,而且推特等社交网站对一般用户开放了API以方便获取数据,这些因素使得近年来基于推特的数据挖掘研究迅速增加。事件检测是其中的一个热门研究,可分为特定事件检测和非特定事件检测。本文基于推特平台以英语文本为研究对象,以非特定事件检测为研究课题。重点研究了短文本语义相似度计算方法、非特定事件检测方法和在线非特定事件检测系统实现。本文主要在以下几个方面进行研究和创新:(1)提出基于知识和语料库结合的短文本相似度计算方法。该方法基于改进词语语义相似度计算方法和一般短文本语义相似度计算方法。改进词语相似度方法通过一种选取融合的策略结合两种词语语义相似度,结合两种方法的优点以弥补单个方法的不足,发现更多文本中词语的语义联系,提升相似度的准确性。本文使用大量测试语料集对多种算法进行对比测试分析,改进方法在词语相似度和短文本相似度结果上比其他方法更贴近人工标记结果。(2)提出基于文本增量聚类的非特定事件检测方法。主要改进了事件检测中的增量短文本聚类方法,改进方法是一种包含了贪婪聚类、重聚类、合并类、类删减和可选的语义相似度计算的方法。它针对现有聚类算法缺乏语义相似性计算、增量聚类的顺序影响性和聚合效果差的问题进行改进。此外提出一种基于聚类结果特征的事件识别方法。在真实推文数据测试中改进聚类方法比原方法在聚类效果和顺序影响方面有较大提升,事件检测方法的召回率和准确性也满足应用需求。(3)设计实现了基于推特的非特定事件检测系统。本文对非特定事件检测系统进行了编程实现,主要包括语义相似度计算、非特定事件检测、文本预处理和图形界面。该系统为了处理社交网络的大量数据信息,针对各个模块分别提出了优化方法。在使用真实推文对系统进行的功能测试和优化性能对比测试中,该系统功能正常,优化方法使模块性能有较大提升,使系统能满足在线事件检测的要求。
[Abstract]:The rise and development of social network bring great convenience and change to people's life. Social networking sites typically have more than one million registered users, while world-renowned social networking sites include Facebook Facebook, Twitter, Weibo and others. Social networks spread a lot of useful information, and social networking sites such as Twitter open API to average users for easy access to data. These factors have led to the rapid increase of data mining research based on Twitter in recent years. Event detection is one of the most popular researches, which can be divided into specific event detection and non-specific event detection. Based on Twitter platform, this thesis focuses on English text and non-specific event detection. This paper focuses on the semantic similarity calculation method of short text text, the method of non-specific event detection and the realization of on-line non-specific event detection system. In this paper, we mainly research and innovate in the following aspects: 1) propose a method for calculating the similarity of short text texts based on the combination of knowledge and corpus. This method is based on the improved semantic similarity calculation method and the general text text semantic similarity calculation method. The improved word similarity method combines the semantic similarity of two words with a strategy of selection and fusion, combines the advantages of the two methods to make up for the shortcomings of a single method, finds more semantic links of words in the text, and improves the accuracy of the similarity degree. In this paper, we use a large number of test corpus to compare and analyze many algorithms. The improved method is closer to the result of manual marking than other methods in terms of word similarity and text similarity.) an independent event detection method based on incremental text clustering is proposed. This paper mainly improves the incremental short text clustering method in event detection. The improved method includes greedy clustering, reclustering, merging classes, class deletion and alternative semantic similarity calculation. It improves on the lack of semantic similarity calculation in existing clustering algorithms, the order influence of incremental clustering and the poor aggregation effect. In addition, an event recognition method based on clustering result features is proposed. The improved clustering method is better than the original method in clustering effect and sequence effect in real tweet data testing. The recall rate and accuracy of event detection method also meet the requirement of application. In this paper, the implementation of non-specific event detection system is carried out, including semantic similarity calculation, non-specific event detection, text preprocessing and graphical interface. In order to deal with a lot of data information of social network, the system proposes optimization methods for each module. In the function test and performance contrast test of the system with real tweet, the system has normal function, and the optimization method can improve the performance of the module greatly, so that the system can meet the requirements of online event detection.
【学位授予单位】：电子科技大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP393.09;TP391.1

【相似文献】