基于Labeled LDA的微博用户兴趣识别系统的研究与实现

发布时间：2018-05-30 15:10

本文选题：文本分类 + 兴趣识别　；参考：《北京交通大学》2014年硕士论文

【摘要】：微博是一个基于用户关系的信息分享、传播以及获取平台,内容简单、用户之间的交互性强、使用门槛低是微博的特点,近年在我国取得了快速发展。微博作为当下最流行的社会化网络服务媒体,基于微博的用户兴趣挖掘研究也迅速成为一个新兴的研究课题：首先发现感兴趣的微博账户与信息是微博用户最重要的活动,微博平台则需要准确地基于用户的兴趣进行相关信息的推荐；其次用户兴趣识别系统是实现精准广告投放的基础,兴趣挖掘的准确度直接关系到广告投放的效果,关系到微博平台的盈利。本文作者在分析新浪微博的信息特点和用户行为特点的基础上,学习了传统的使用词向量进行文本特征表示的文本分类算法,并基于无监督、无层次结构的主题模型LDA (Latent Dirichlet Allocation,隐含狄利克雷分配),扩展实现了有监督、无层次结构的主题模型Labeled LDA,用于对微博用户兴趣分布的识别。本文主要研究了用户兴趣识别过程中涉及到的关键问题,主要包括以下三个方面的工作：(1)使用Python开发针对新浪微博的定制网络爬虫,绕开微博API的限制,实现微博文本的并发快速获取,为研究工作提供了及其丰富的实验数据；(2)学习文本分类技术,使用有监督、无层次结构的主题模型Labeled LDA,用微博主题账号的微博文本进行模型的训练,用于对其他微博用户兴趣的预测；(3)考虑海量数据的场景,使用Hadoop、Hive等分布式框架,实现海量中文文本数据的分布式分词与预处理。最终,通过用户兴趣识别系统得到的用户兴趣分布的数据在实际中成功的应用于用户个性化词云的产生与展现、搜索结果的调整与优化、广告的个人兴趣化定向投放等方面。
[Abstract]:Weibo is a platform for information sharing, dissemination and acquisition based on user relationship. It is characterized by simple content, strong interaction between users and low threshold of use of Weibo. In recent years, it has achieved rapid development in China. Weibo as the most popular social network service media, the research of user interest mining based on Weibo has quickly become a new research topic: first of all, it is the most important activity of Weibo users to discover the interesting Weibo account and information. The Weibo platform needs to recommend the relevant information accurately based on the user's interest. Secondly, the user interest recognition system is the basis for the implementation of accurate advertising, and the accuracy of interest mining is directly related to the effect of advertising. Related to the Weibo platform profit. On the basis of analyzing the information characteristics and user behavior characteristics of Sina Weibo, the author of this paper studies the traditional text classification algorithm using word vector for text feature representation, and based on unsupervised, The topic model LDA / Latent Dirichlet allocation without hierarchy is extended to implement a supervised and unhierarchical topic model, Labeled LDA. it is used to identify the distribution of interest of Weibo users. This paper mainly studies the key problems involved in the process of user interest identification, including the following three aspects: 1) using Python to develop customized web crawlers for Sina Weibo, circumventing the limitations of Weibo API. To realize the concurrency and fast acquisition of Weibo text, this paper provides an extremely rich experimental data for the research work and studies the text classification technology, which is supervised. The hierarchical topic model Labeled LDAuses the Weibo text of the Weibo theme account to train the model, which is used to predict the interest of other Weibo users. (3) considering the scene of massive data, using the distributed framework such as Hadoop Hive, etc. Distributed word segmentation and preprocessing of massive Chinese text data are realized. Finally, the user interest distribution data obtained through the user interest recognition system has been successfully applied to the generation and presentation of user personalized word cloud, the adjustment and optimization of search results, and the orientation and placement of personal interest in advertising.
【学位授予单位】：北京交通大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.092

【参考文献】