基于用户行为特征的E2LSH动态权重混合推荐算法及应用

发布时间：2018-10-26 18:25

【摘要】：近年来,随着互联网服务的迅速崛起致使用户与信息量的激增,而在这些海量的数据中如何精确、快速的检索到用户所需数据,是目前在大数据和数据挖掘领域一项重要的研究方向。为了解决这一问题,推荐系统以其智能寻求用户兴趣资源的特点,颠覆了传统的文本检索方式,提供了更加高质量的用户体验。虽然推荐系统极大的改变了用户对于信息的获取方式,但传统的推荐系统在面对海量高维的稀疏数据时也会遇到"冷启动"和"维度灾难"等问题,这些都对推荐系统的应用提出了巨大的挑战。本文对现阶段主流推荐算法进行归纳,对相似近邻查找、局部敏感性哈希、协同过滤等方面进行讨论,发现目前推荐算法在面对稀疏数据方面计算精度有所下降。同时在面对海量高维数据时,算法的平均用时偏长。为了解决这些问题,本文提出了用户行为特征和动态权重的概念,将E2LSH算法与混合推荐算法相结合,构建了一个准确、高效的推荐系统。本文的主要工作如下:1.针对推荐算法在稀疏数据方面所面临的推荐精度降低问题,本文提出了基于用户行为特征的动态权重混合推荐算法。通过对原始数据集中的数据进行预处理,计算出不同用户对于不同物品的个性化行为特征指数,并将其量化成为用户行为特征向量,将其引入相似度的计算中。依据用户评分数据稀疏性大小的个性化差异计算出动态权重,并依此将基于用户内容的推荐算法和协同过滤推荐算法进行动态混合。实验结果表明,该算法相比于传统混合推荐算法,其MAE平均降低2.26%,尤其是在数据集稀疏性比较极端的情况下,推荐效果的提升更加显著。2.针对海量高维数据对混合推荐算法在推荐效率方面的影响,本文研究了基于E2LSH改进的混合推荐算法。利用E2LSH算法保持数据相似一致性的前提下,在系统离线时构建用户-项目的索引,并在用户需要在线检索近邻时利用离线索将查找的时间复杂度从O(N_2)降低至O(1),在不改变数据相似性的情况下来提高过滤非相似用户的计算效率。从实验结果可以看出,该算法在继续保持了混合推荐算法计算精准性的同时,极大的降低了算法的平均计算时间,大大提高了整体计算效率。3.将本文提出的基于E2LSH改进的混合推荐算法初步应用到国家电网网改云检系统中。在创建任务选择任务对应人员时,系统能够结合人员的历史相关信息,智能的推荐合适的现场作业人员,省去用户手动人工筛选的流程,极大的提高了用户对于系统的使用体验。
[Abstract]:In recent years, with the rapid rise of Internet services, users and the amount of information surge, and in these massive data how to accurately, quickly retrieve the data users need, It is an important research direction in the field of big data and data mining. In order to solve this problem, the recommendation system based on its intelligent search for user interest resources, subverts the traditional text retrieval methods, and provides a higher quality user experience. Although the recommendation system has greatly changed the way users obtain information, the traditional recommendation system will encounter problems such as "cold start" and "dimension disaster" in the face of massive sparse data of high dimension. These all put forward the huge challenge to the application of the recommendation system. In this paper, the current mainstream recommendation algorithms are summarized, and the similar nearest neighbor lookup, local sensitivity hashing and collaborative filtering are discussed. It is found that the accuracy of the recommendation algorithm in the face of sparse data has been reduced. At the same time, in the face of massive high dimensional data, the average time of the algorithm is too long. In order to solve these problems, this paper proposes the concepts of user behavior characteristics and dynamic weights, and combines E2LSH algorithm with hybrid recommendation algorithm to construct an accurate and efficient recommendation system. The main work of this paper is as follows: 1. Aiming at the problem of low recommendation accuracy in sparse data, a dynamic weighted hybrid recommendation algorithm based on user behavior feature is proposed in this paper. By preprocessing the data in the original data set, the personalized behavior feature index of different users for different items is calculated, and quantized into the user behavior feature vector, which is introduced into the calculation of similarity. The dynamic weight is calculated according to the individualized difference of user rating data sparsity, and the dynamic mixing of user content based recommendation algorithm and collaborative filtering recommendation algorithm is carried out. The experimental results show that compared with the traditional hybrid recommendation algorithm, the MAE of the proposed algorithm is 2.26% lower than that of the traditional hybrid recommendation algorithm, especially when the data set sparsity is extreme, the improvement of the recommendation effect is more significant. 2. Aiming at the effect of massive high dimensional data on the efficiency of hybrid recommendation algorithm, this paper studies the improved hybrid recommendation algorithm based on E2LSH. On the premise of keeping the similarity of data using E2LSH algorithm, the index of user-item is constructed when the system is offline. The time complexity of searching is reduced from O (N _ S _ 2) to O (1) when users need to search their nearest neighbors online, so as to improve the computing efficiency of filtering dissimilar users without changing the similarity of data. The experimental results show that the algorithm not only keeps the accuracy of the hybrid recommendation algorithm, but also greatly reduces the average computing time of the algorithm and greatly improves the overall calculation efficiency. The proposed hybrid recommendation algorithm based on E2LSH is applied to the cloud detection system of State Grid. When creating the task selection task counterpart, the system can intelligently recommend the appropriate field operator according to the historical information of the personnel, so as to eliminate the flow of manual screening by the user. It greatly improves the user's experience of using the system.
【学位授予单位】：武汉大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.3

【相似文献】