基于Hadoop的推荐系统的设计与实现

发布时间：2018-05-24 14:46

本文选题：推荐系统 + 组合引擎　；参考：《华南理工大学》2013年硕士论文

【摘要】：进入二十一世纪之后，人类互联网的大数据时代，我们面临着一个严重的问题就是信息过载。在互联网时代有许多探索解决信息过载的方法，信息分类网站和搜索引擎就已经在解决信息过载问题上取得了成功。通过信息分类来解决信息过载的网站有雅虎和新浪，而谷歌和百度则是通过搜索来解决信息过载的。推荐系统被认为是一种更加优秀的解决方法，相比前两者，，推荐系统更加智能和主动。面对着整个用互联网的时候用户许多时候是不知道自己的需求是什么，而信息分类和搜索引擎是建立在用户通过关键字或者信息所属类目去查找的。推荐引擎是主动发送推荐的信息给用户。它运用集体智慧来帮助用户对海量信息作出选择。集体智慧是是一种共享的或者群体的智能，以及集结众人的意见进而转化为决策的一种过程，许多个体通过合作和竞争所显现出来的智慧。推荐引擎依托海量数据，分析用户的行为、特征以及爱好，并为用户找出符合其兴趣的物品。本论文先阐述研究背景、国内外相关研究，并深入研究了推荐系统的发展，推荐算法及其应用，同时还探讨了大数据处理框架Hadoop的原理。本文通过对推荐系统理论的研究和应用以及对Hadoop的研究，确定了推荐系统的架构，并详细设计了推荐系统，同时还阐述了推荐系统的主要部分的实现。本文的主要贡献有以下几点： 1）设计了一个水平扩展推荐算法的推荐系统框架，可以动态添加和修改推荐引擎，并根据主流的协同重点分析和设计了基于协同过滤的引擎。 2）使用基于用户动态反馈的权值计算方法来综合各个推荐结果，从而组成一个推荐引擎组合，提高了整个推荐系统的测评指标。 3）使用Hadoop大数据平台实现推荐系统来应对推荐系统海量数据的计算，从而提升了计算效率，减少了系统的反应时间。
[Abstract]:After entering the 21 century, the big data era of human Internet, we face a serious problem is information overload. In the Internet era, there are many ways to solve the problem of information overload. Information classification websites and search engines have been successful in solving the problem of information overload. Websites that use information classification to resolve information overload include Yahoo and Sina, while Google and Baidu use search to resolve information overload. Recommendation system is considered to be a better solution, compared with the first two, the recommendation system is more intelligent and active. In the face of the entire use of the Internet, users often do not know what their own needs, and information classification and search engines are built on the user through the keyword or information to the category to find. Recommendation engine is the initiative to send the recommended information to the user. It uses collective wisdom to help users make choices about massive amounts of information. Collective wisdom is a kind of shared or collective intelligence, and a process that gathers the opinions of others and transforms them into decision making. Many individuals are shown to be wise through cooperation and competition. Based on massive data, the recommendation engine analyzes the user's behavior, characteristics and hobbies, and finds out the objects that fit the user's interests. This paper first describes the research background, domestic and foreign related research, and deeply studies the development of recommendation system, recommendation algorithm and its application. At the same time, it also discusses the principle of big data processing framework Hadoop. Through the research and application of recommendation system theory and the research of Hadoop, this paper determines the framework of recommendation system, designs the recommendation system in detail, and expounds the realization of the main part of recommendation system. The main contributions of this paper are as follows: 1) A recommendation system framework of horizontal extended recommendation algorithm is designed, which can dynamically add and modify the recommendation engine, and analyze and design the engine based on collaborative filtering according to the main collaborative emphasis. 2) the weight calculation method based on user dynamic feedback is used to synthesize each recommendation result, thus a recommendation engine combination is formed, and the evaluation index of the whole recommendation system is improved. 3) the Hadoop big data platform is used to realize the recommendation system to deal with the computation of the massive data of the recommendation system, which improves the computing efficiency and reduces the reaction time of the system.
【学位授予单位】：华南理工大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP391.3

【参考文献】