中文微博短文本主题挖掘方法研究与原型系统开发

发布时间：2018-02-04 22:58

本文关键词： 微博短文本文本聚类主题挖掘频繁闭项集词对共现　出处：《大连海事大学》2017年硕士论文　论文类型：学位论文

【摘要】：近年来,随着互联网与智能移动设备的快速发展,以Twitter、微博等为代表的社交媒体应用变得越来越受欢迎,短文本交互日益普遍,对海量的微博数据进行分析主题,及时获取人们关注的热点话题以及满足用户在大量产生的信息中查找自己的需求具有重要的现实意义。微博文本内容短小,特征词稀疏并且规模庞大,对于微博这种具有特殊特征的短文本,选取有效的方法进行主题识别,进行细粒度的主题检索,最大程度的满足用户需求是目前需要解决的重要问题。本文主要针对短文本的信息抽取开展相关研究工作,研究重点聚焦于中文微博类短文本的内隐主题提取,在现有对文本聚类和主题模型的研究基础上,针对中文微博短文本语料库,展开了相关研究。主要研究工作和成果包括:(1)对预处理后的微博语料集进行基于top-k频繁闭词集的短文本聚类,聚类过程中对频繁词集挖掘算法进行了改进,避免了 min_support的多次尝试以及频繁词集数据量巨大的问题,并且以频繁词集作为类簇的描述信息,得到微博文本的粗粒度分类。(2)针对类簇内主题不明确并且短文本存在的特征稀疏问题,结合LDA模型和BTM模型提出了一种基于词对共现LDA模型的类簇内潜在主题挖掘方法,对每篇文档的词对进行建模,提高短文本主题特征的性能,得到细粒度的类簇内隐含主题。(3)根据短文本聚类和细粒度类簇内主题挖掘相结合的思想,设计出了微博内隐主题挖掘系统,不仅能够准确的得出微博短文本的划分类簇,而且能够实现在类簇上进一步挖掘主题的目的,最终实现了对微博平台内微博信息的内隐主题的自动化提取与分类存储。
[Abstract]:In recent years, with the rapid development of the Internet and smart mobile devices, social media applications, such as Twitter, Weibo and so on, have become more and more popular. It is very important to analyze the theme of the massive Weibo data, to get the hot topics that people pay attention to in time and to meet the needs of users to find their own in a large number of generated information. Weibo text content is short. Feature words are sparse and large in scale. For Weibo short text with special characteristics, an effective method is selected for topic recognition and fine-grained topic retrieval. To meet the needs of users to the greatest extent is an important issue to be solved. This paper mainly focuses on the information extraction of short text, focusing on the implicit topic extraction of Chinese Weibo short text. Based on the existing research on text clustering and topic models, this paper aims at the Chinese Weibo short text corpus. The main research work and results include: 1) the text clustering of the pretreated Weibo corpus based on the frequent closed words set of top-k. In the process of clustering, the algorithm of mining frequent word sets is improved to avoid the problems of min_support 's multiple attempts and the huge amount of data of frequent word sets. And the frequent word set is used as the description information of the cluster to obtain the coarse-grained classification of Weibo text. Combined with LDA model and BTM model, a method of potential topic mining in clusters based on word pair co-occurrence LDA model is proposed, which can model the word pairs of each document and improve the performance of the topic features of the short text. According to the idea of combination of short text clustering and topic mining in fine-grained cluster, a Weibo implicit topic mining system is designed. Not only can we accurately get the short text of Weibo divided into clusters, but also can achieve the purpose of further mining topics on the cluster. Finally, the automatic extraction and classification of the implicit topics of Weibo information in Weibo platform are realized.
【学位授予单位】：大连海事大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.1

【参考文献】