基于协同过滤的新闻推荐系统在Hadoop上的研究与实现

发布时间：2018-03-02 06:10

本文关键词： 内容聚合平台推荐系统协同过滤基于用户推荐混合推荐　出处：《郑州大学》2017年硕士论文　论文类型：学位论文

【摘要】：随着信息时代的来临,互联网承载的信息量越来越大,导致了人们在浏览新闻门户网站时,很难找到自己感兴趣的信息。内容聚合平台及推荐系统的出现就是为了解决信息过载的问题。内容聚合平台从各大新闻网站上爬取新闻资讯,储存到本地系统上后,再通过推荐系统推送给平台的用户,为每个用户提供个性化新闻推荐。常规的推荐系统一般基于协同过滤算法,然而,基于协同过滤的热点推荐和基于用户推荐却存在部分瑕疵。常规的热点推荐算法,会将新闻的热度值以固定的衰减系数进行衰减;本文通过测试分析,发现这种算法不能平衡用户流量与热度衰减之间的不均衡性,会导致热点捕捉率很低。而常规的基于用户推荐只使用了协同过滤算法,以近邻用户群为基础对用户进行推荐,在用户兴趣发生改变时会导致推荐结果不准确。本文在建立内容聚合平台的基础上,研究并改进了这两种推荐算法。本文的主要工作和创新点如下:1.建立了一个简单但完善的内容聚合平台,其包括几个子系统:web系统、缓存服务、爬虫、数据库服务以及Hadoop集群,各个子系统间通过相应的协议进行通信;内容聚合平台主要为推荐算法提供服务。2.针对热点推荐算法,本文提出了自适应时间衰减系数的热点新闻推荐,综合考虑了单条新闻流量与系统流量,以此来计算每条新闻的时间衰减系数。测试表明,自适应时间衰减系数的热点推荐算法能有效提高热点捕捉率。最后,对于新入新闻使用了潜在热点挖掘算法以更新热点新闻。3.针对基于用户推荐算法,本文提出了自修正用户模型的基于用户推荐。首先分析出各个用户的近邻用户群,再结合基于项目推荐,为每个用户生成单独的推荐列表;然后,分析用户的浏览历史,利用修正算法定期修正用户模型,以跟踪用户兴趣变化,提供更好的推荐。测试表明,修正算法能在三次迭代内完成对于大多数用户模型的修正,改进后的基于用户推荐算法能提供比常规算法更好的查准率与查全率。最后,本文结合热点新闻推荐,实现了用户个性化推荐,以挖掘用户的潜在兴趣。
[Abstract]:With the advent of the information age, the Internet is carrying more and more information, which leads people to browse news portals. It's hard to find information that you're interested in. Content aggregation platforms and recommendation systems have emerged to solve the problem of information overload. Content aggregation platforms crawl news information from major news websites and store it on local systems. Then push the recommendation system to the users of the platform to provide personalized news recommendation for each user. The conventional recommendation system is usually based on collaborative filtering algorithm, however, The hot spot recommendation based on collaborative filtering and the user recommendation have some defects. The conventional hot spot recommendation algorithm attenuates the calorific value of the news with a fixed attenuation coefficient. It is found that this algorithm can not balance the imbalance between user flow and heat attenuation, which will lead to a very low hot spot capture rate. However, the conventional recommendation based on user recommendation only uses collaborative filtering algorithm, and recommends users on the basis of nearest neighbor user group. Recommendation results can be inaccurate when user interests change. The main work and innovation of this paper are as follows: 1. A simple but perfect content aggregation platform is established, which includes several subsystems: Web system, cache service, crawler, database service and Hadoop cluster. Each subsystem communicates through the corresponding protocols. The content aggregation platform mainly provides services for the recommendation algorithm. 2. Aiming at the hot spot recommendation algorithm, this paper puts forward the adaptive time attenuation coefficient of the hot news recommendation. The time attenuation coefficient of each news is calculated by synthetically considering the single news flow and system traffic. The test results show that the hot spot recommendation algorithm with adaptive time attenuation coefficient can effectively improve the hot spot capture rate. For the new news, we use the latent hotspot mining algorithm to update the hot news. 3. For the user recommendation algorithm, this paper proposes a user recommendation based on self-modified user model. Firstly, the nearest neighbor user groups of each user are analyzed. Combined with project-based recommendations, a separate recommendation list is generated for each user. Then, the browsing history of the user is analyzed, and the user model is regularly modified by the modified algorithm to track the change of user interest and provide better recommendations. The modified algorithm can complete the revision of most user models in three iterations, and the improved user-based recommendation algorithm can provide better recall and recall than the conventional algorithm. Finally, this paper combines the hot news recommendation. The user personalized recommendation is implemented to tap the potential interest of the user.
【学位授予单位】：郑州大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.3

【相似文献】