基于改进LDA模型的社交网络用户行为分析

发布时间：2018-05-14 11:24

本文选题：社交网络 + 新浪微博　；参考：《南京邮电大学》2017年硕士论文

【摘要】：随着互联网的快速发展,以微博为代表的社交网络已经成为用户发布和获取实时信息的重要手段。网络信息的爆炸性增长,一方面带来了更多消息来源,另一方面使得获取实用信息的难度加大。通过有效的手段,精准地发现、组织和利用社交网络海量短文本背后隐藏的有价值的信息,分析研究用户的行为特征,不仅具有较高的舆情监控和商业推广价值,还有助于互联网的综合管理。本文以国内主流社交网站新浪微博为研究对象,对微博数据采集和用户行为特点进行深入的研究,主要贡献和创新之处如下:1.微博信息爬虫系统。本文设计并实现了一种基于改进布隆过滤器的新浪微博信息爬虫系统,该系统主要针对布隆过滤器误识率问题进行改进,提供了一种高效的URL去重方案,解决URL去重效率低的问题,同时解决了新浪微博用户数据获取和存储问题。2.异常用户行为分析。根据爬取到的微博用户数据,分析异常用户的行为特点。对LDA模型进行改进,设计适用于处理用户关系数据的UR-LDA模型。利用UR-LDA模型对词化后的微博垃圾用户特征进行降维和聚类。3.用户关注话题分析。由爬虫系统获取的用户数据挖掘出用户关注的话题。首先用中文分词系统ICTCLAS对用户微博数据进行分词和去停用词处理,然后利用UR-LDA模型进行主题挖掘,最后对用户进行聚类分析。在实验过程中,本文将基于改进的布隆过滤器和传统的布隆过滤器过滤器进行分析和比较,结果表明,本文设计的爬虫系统具有更高的URL去重效率,爬取微博数据更加高效。通过LDA模型对和UR-LDA模型对微博主题挖掘效果对比,实验结果表明UR-LDA能更加有效地挖掘出用户关注的话题。
[Abstract]:With the rapid development of the Internet, the social network represented by Weibo has become an important means for users to publish and obtain real-time information. The explosive growth of network information, on the one hand, brings more information sources, on the other hand, makes it more difficult to obtain practical information. Through effective means, we can accurately find, organize and use the valuable information hidden behind the mass short text books of social networks, analyze and study the behavior characteristics of users, which is not only of high value of monitoring and commercial promotion of public opinion. It also contributes to the integrated management of the Internet. This paper takes Sina Weibo, a popular social network in China, as the research object, and makes an in-depth study on the characteristics of Weibo data collection and user behavior. The main contributions and innovations are as follows: 1. Weibo information crawler system. In this paper, we design and implement a kind of Sina Weibo information crawler system based on improved Bron filter. This system mainly improves the error rate of Blunt filter, and provides an efficient URL de-reduplication scheme. To solve the problem of low efficiency of URL, and to solve the problem of user data acquisition and storage of Sina Weibo. 2. 2. Abnormal user behavior analysis. According to the crawling Weibo user data, the behavior characteristics of abnormal users are analyzed. The LDA model is improved and the UR-LDA model is designed for dealing with user relationship data. UR-LDA model is used to reduce and cluster Weibo garbage user features. 3. Users focus on topic analysis. The user data obtained from the crawler system is used to mine the topics concerned by the user. First, the Chinese word segmentation system (ICTCLAS) is used to segment and deactivate the user's Weibo data, then the UR-LDA model is used for topic mining, and finally, the clustering analysis of the user is carried out. In the course of experiment, this paper analyzes and compares the improved Bron filter and the traditional Blunt filter. The results show that the crawler system designed in this paper has higher URL removal efficiency and more efficient crawling Weibo data. By comparing the effect of LDA model pair and UR-LDA model on Weibo topic mining, the experimental results show that UR-LDA can more effectively mine the topics of user concern.
【学位授予单位】：南京邮电大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP393.09;TP391.1

【参考文献】