职位信息实时推荐系统的设计与实现

发布时间：2018-05-28 16:38

本文选题：推荐系统 + 实时推荐　；参考：《江苏大学》2017年硕士论文

【摘要】：现有的职位推荐系统主要通过智能检索、简历匹配、用户相似性和定时推送等方法推荐,然而已有职位推荐方法主要基于用户的历史偏好,未考虑用户行为变化对求职偏好的影响且无法根据该变化及时响应其最新的求职动向,导致推荐实时性差。另外,现有基于用户的职位协同推荐主要采用默认值处理职位评分矩阵的缺失值,未考虑用户对职位的偏好差异,降低了相似性计算质量且该方法缺乏考虑用户的特定需求,如对薪资或学历等属性的要求,导致推荐的可靠性不高。针对上述问题,提出了基于用户动态行为变化的职位实时推荐方法。通过监听用户操作事件,判断用户是否发生添加或更新求职意愿信息和点击其他职位的行为变化,及时发现其最新的求职动向并产生推荐,从而解决推荐实时性差的问题。此外,提出了基于职位评分预测的协同推荐算法。在现有UserCF算法基础上,对缺失的职位评分进行预测填充处理,提升相似性计算质量并对结果进行过滤,从而提高推荐可靠性。论文主要工作如下:(1)系统需求分析和总体设计。为能够及时发现用户最新求职动向并推荐可靠的职位,采用了包括数据层、推荐算法层和应用层的总体架构。为获取职位资源和用户偏好信息,数据层需采集职位信息和用户行为数据。为向用户推荐可靠的职位,推荐算法层需利用采集的用户行为数据,训练职位推荐算法,以构建用户求职偏好模型。为向用户展示职位推荐信息,应用层需提供前台交互页面,及时响应用户操作。因此,系统按照功能划分为数据采集模块、职位推荐模块和前台页面交互模块。(2)系统详细设计。针对数据采集模块,采用职位爬虫抓取网络职位数据和利用数据库或消息队列保存用户行为数据。根据不同的推荐逻辑,将职位推荐模块分为职位实时推荐和职位协同推荐两子模块。针对职位实时推荐,采用消息队列保存用户最新的行为事件消息,当用户发生添加或更新求职意愿和点击职位的行为时,通过监听消息队列的事件消息变化,分别触发在线职位匹配推荐和在线职位关联推荐,以提高推荐的实时性。针对职位协同推荐,采用预测值填充职位评分矩阵的缺失值,提升相似性质量,基于此,计算协同推荐结果并对其过滤,以提高推荐的可靠性。前台页面交互模块基于响应式布局的方法设计页面,以提高推荐响应效率。(3)系统实现与测试。系统基于J2EE实现。职位实时推荐通过Storm实现并利用Flume和Kafka收集和缓存事件日志;职位协同推荐通过Mahout实现。由测试结果可知系统能够及时向用户推荐可靠的职位。此外,在对职位评分矩阵的缺失值进行预测填充后,协同推荐在准确率和召回率上平均提升35%和40%。
[Abstract]:The existing job recommendation systems are mainly recommended by intelligent retrieval, resume matching, user similarity and timing push. However, the existing job recommendation methods are mainly based on users' historical preferences. Without considering the influence of user behavior change on job search preference and unable to respond to the latest job search trend according to this change, the recommendation real-time performance is poor. In addition, the default value is mainly used to deal with the missing value of the position score matrix, which does not take into account the difference of the user's preference to the position, which reduces the quality of similarity calculation and does not take into account the specific needs of the users. Such as salary or academic qualifications such as attribute requirements, resulting in recommendation reliability is not high. In order to solve the above problems, a real-time job recommendation method based on user dynamic behavior change is proposed. By monitoring the user's operation events, we can judge whether the user's behavior changes in adding or updating the job seeking intention information and clicking on other positions, and find out the latest job search trend and produce the recommendation in time, so as to solve the problem of poor real-time recommendation. In addition, a collaborative recommendation algorithm based on job score prediction is proposed. Based on the existing UserCF algorithm, the missing job score is predicted and filled, the similarity calculation quality is improved and the results are filtered to improve the reliability of the recommendation. The main work of this paper is as follows: 1) system requirement analysis and overall design. In order to find out the latest job trends of users and recommend reliable positions in time, an overall framework including data layer, recommendation algorithm layer and application layer is adopted. In order to obtain position resource and user preference information, the data layer needs to collect position information and user behavior data. In order to recommend reliable positions to users, the recommendation algorithm layer needs to use the collected user behavior data and train the position recommendation algorithm to build a job preference model. In order to display the position recommendation information to the user, the application layer should provide the front desk interactive page and respond to the user operation in time. Therefore, the system is divided into data acquisition module, position recommendation module and front page interaction module. For the data acquisition module, the position crawler is used to grab the network position data and the database or message queue is used to save the user behavior data. According to the different recommendation logic, the position recommendation module is divided into two sub-modules: real time recommendation and collaborative recommendation. For post recommendation, message queue is used to save the latest behavior event message of user. When the behavior of adding or updating job search will and clicking on position occurs, the event message changes in message queue are monitored. The online position matching recommendation and the online position correlation recommendation are triggered respectively to improve the real-time performance of the recommendation. In order to improve the reliability of the job recommendation, the prediction value is used to fill the missing value of the position score matrix to improve the similarity quality. Based on this, the collaborative recommendation results are calculated and filtered. The front page interaction module designs the page based on the method of response layout to improve the efficiency of recommendation response. The system is implemented based on J2EE. Real time job recommendation is implemented by Storm, event log is collected and cached by Flume and Kafka, and post collaborative recommendation is implemented by Mahout. Test results show that the system can recommend reliable positions to users in time. In addition, after filling in the missing value of the position score matrix, the cooperative recommendation increased the accuracy and recall by an average of 35% and 40%.
【学位授予单位】：江苏大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.3

【参考文献】