在线社区中的用户行为分析及热度预测

发布时间：2018-02-03 09:15

本文关键词： 在线社区 Scrapy爬虫回复网络用户分类热贴预测　出处：《浙江大学》2017年硕士论文　论文类型：学位论文

【摘要】：随着Web 2.0的日益普及,各类在线社区也应运而生。在线社区具有现实社区所不具有的超时空性、符号性以及虚拟性,因而吸引了越来越多的互联网用户参与到其中。如何从这些在线社区中产生的海量数据中挖掘出有价值的信息也成为了当下热门的研究课题。本文基于百度贴吧的数据集对贴吧中的用户行为进行了分析,同时也提出了一种热贴预测模型。本文的工作主要包括以下几个方面:(1)设计并实现了基于Scrapy的网络爬虫,对百度贴吧中某贴吧于7、8月间产生的数据进行了采集。经过数据预处理后,数据集包含了约6万个主题贴、249万个回复贴以及22万个用户的信息,通过实验发现主题贴的获回贴数满足幂律分布;(2)基于上文获取的数据集构建了贴吧用户之间的回复网络,验证了该回复网络也具有社交网络的小世界和无标度特性。并且从活跃时间、发贴数、获回贴数以及回复时延四个角度对贴吧用户行为进行了分析。最后,对贴吧用户进行了聚类研究,通过额外引入"平均回复时延"这一用户行为指标,得到了有趣的分类结果并对其进行了详细阐述;(3)提出了一种基于时间阀值T的热贴预测模型,提取了与上文构建的回复网络相关的特征,并结合其他三类特征,对贴子最终能否成为热门贴进行预测,并通过实验验证了该预测模型的有效性。最后,通过对比实验分析了不同的时间阀值T、不同的分类模型以及不同的特征组合对预测效果产生的影响。
[Abstract]:With the increasing popularity of Web 2.0, all kinds of online communities have emerged. Online communities have the transcendental, symbolic and virtual properties that the real community does not have. Therefore, it attracts more and more Internet users to participate in it. How to extract valuable information from the massive data generated from these online communities has also become a hot research topic at present. This paper is based on Baidu Tieba. The user behavior in the post bar is analyzed in the data set of. At the same time, a hot patch prediction model is proposed. The main work of this paper includes the following aspects: 1) designed and implemented the Scrapy based web crawler. The data generated in August was collected. After data preprocessing, the dataset contained about 60, 000 topic posts, 2. 49 million reply posts and 220,000 users' information. It is found by experiments that the number of reposted objects meets the power law distribution. Based on the data set obtained above, a reply network between the users of the post bar is constructed, which verifies that the response network also has the small-world and scale-free characteristics of the social network, and posts from the active time. Finally, the user behavior is analyzed from four aspects: the number of posts returned and the response delay. Finally, the user behavior index of "average reply delay" is introduced by clustering the users. Some interesting classification results are obtained and explained in detail. (3) A hot patch prediction model based on time threshold T is proposed. The features related to the response network constructed above are extracted, and the other three kinds of features are combined to predict whether the post will eventually become a hot sticker. The effectiveness of the prediction model is verified by experiments. Finally, the effects of different time threshold T, different classification models and different feature combinations on the prediction results are analyzed through comparative experiments.
【学位授予单位】：浙江大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP311.13;TP393.09

【相似文献】