基于Hadoop的新闻推荐系统关键技术的研究

发布时间：2018-02-25 09:41

本文关键词： 推荐系统大数据聚类算法推荐算法　出处：《天津理工大学》2017年硕士论文　论文类型：学位论文

【摘要】：近年来,互联网技术的迅猛发展对用户造成了信息过载的困扰,导致用户想要获取需要的信息变得愈发困难。搜索引擎虽然能够给用户查询想要的信息,但是仍然缺乏主动性。在这个基础上,出现了个性化推荐系统。个性化推荐系统不需要用户输入信息,便能主动提供用户感兴趣的信息,因此得到了广泛使用。个性化推荐系统通常建立在大数据量分析与挖掘平台上,通过对用户的注册登录信息,历史行为等来分析用户的偏好,找到用户潜在的兴趣,进而进行个性化推荐来改善用户的互联网体验。在新闻媒体领域,每天都会产生海量的数据。新闻数据往往具有鲜明的时效性,归类性,社会化等特点。针对这些特点,已经有一些新兴的新闻媒体将传统的新闻推送方式与个性化推荐系统相结合,极大的增强了用户与新闻介质之间的黏性,也使得新闻媒体更好的融入到互联网的发展中。虽然新闻媒体与个性化推荐系统的结合是主流趋势,但仍然会面临一些推荐系统特有的问题,如冷启动,数据量太大,准确率与召回率过低,时效性等等,这些问题的解决都是研究个性化推荐系统中的关键。本文结合了目前推荐系统中的研究成果,将社交化的用户信息结合到个性化新闻推荐系统中,本文的主要工作成果为:1、将用户的社交信息与用户历史行为捆绑推荐。2、为提高推荐系统效率,提出了一种基于社交网络的最短距离聚类算法(SDCA),力求在推荐算法执行之前,预先对用户进行一次聚类。此算法改变传统的推荐系统的输入方式,使得推荐系统的推荐效率得到了有效提高。3、将个性化新闻推荐系统与大数据相结合,使得此推荐系统能够并行运行,提高了可扩展性,适应了目前海量新闻报道的需求。4、对数据规模与相关参数进行了实验,实现了一个个性化新闻推荐系统,并对准确度,实时性等进行了相关测试。本文首先阐述了当前个性化推荐系统的研究现状,总结了当前主流的推荐算法与推荐系统搭建运行过程中常遇到的问题及解决方案,然后提出了一种基于社交网络的聚类算法,最后给出了基于Hadoop平台实现的新闻推荐系统。
[Abstract]:In recent years, with the rapid development of Internet technology, users are troubled by information overload, which makes it more and more difficult for users to obtain the information they need. However, there is still a lack of initiative. On this basis, personalized recommendation systems have emerged. Personalized recommendation systems can actively provide information of interest to users without requiring users to input information. So it is widely used. Personalized recommendation system is usually built on the platform of large amount of data analysis and mining, through the registration of users login information, historical behavior to analyze the preferences of users, to find out the potential interests of users, In the field of news media, a huge amount of data is produced every day. News data often have distinct characteristics such as timeliness, categorization, socialization and so on. Some new news media have combined the traditional news push mode with the personalized recommendation system, which has greatly enhanced the viscosity between users and news media. It also makes the news media better integrate into the development of the Internet. Although the combination of news media and personalized recommendation system is the mainstream trend, it still faces some unique problems of the recommendation system, such as cold start, too large amount of data. The accuracy rate and recall rate are too low, timeliness and so on. The solution of these problems is the key to the study of personalized recommendation system. In order to improve the efficiency of the recommendation system, the main work result of this paper is: 1, which binds the social information of the user with the historical behavior of the user, and binds the social information of the user to the personalized news recommendation system, so as to improve the efficiency of the recommendation system. In this paper, a shortest distance clustering algorithm based on social network (SNS) is proposed, which tries to cluster users once before the recommendation algorithm is implemented. This algorithm changes the input mode of traditional recommendation system. The recommendation efficiency of the recommendation system is improved effectively. 3. The personalized news recommendation system is combined with big data to make the recommendation system run in parallel and improve the scalability. According to the demand of mass news report. 4, the data scale and related parameters are experimented, a personalized news recommendation system is realized, and the accuracy is analyzed. First of all, this paper describes the current research status of personalized recommendation system, summarizes the current mainstream recommendation algorithm and recommendation system in the process of building and running problems and solutions, Then a clustering algorithm based on social network is proposed, and a news recommendation system based on Hadoop platform is given.
【学位授予单位】：天津理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.3

【相似文献】