基于Hadoop的CDN-P2P系统中内容预测机制研究与实现

发布时间：2018-04-22 17:53

本文选题：CDN-P2P + 需求预测　；参考：《北京邮电大学》2013年硕士论文

【摘要】：近十几年来,随着互联网的飞速发展,网络信息量和用户数急剧增长,网络共享和传输的内容也由简单的文字、图片扩展到音频、视频等结构复杂、形式多样的多媒体。为了高效进行网络内容分发,缓解网络拥塞,提升用户体验,CDN和P2P技术作为网络内容分发的主要技术,在众多领域被广泛应用。考虑到CDN和P2P技术在提供服务时与生俱有的互补性,CDN-P2P融合技术也成为新的研究热点。网络规模的不断扩大,共享资源信息的激增,给CDN-P2P网络中节点文件共享以及边缘服务器文件服务的提供,带来了诸多问题。主要表现为：对边缘服务器存储负载能力以及P2P节点文件请求响应时间的要求。CDN-P2P网络需要服务的节点数和提供的文件数量不断增大,需要在边缘服务器和内容源服务器之间,或者边缘服务器之间频繁传送文件,不仅增加节点文件请求的响应时间,而且消耗带宽资源。同时,节点用户也需要花费大量时间在海量资源信息中寻找自己需要的内容。改进CDN-P2P网络中边缘服务器的内容缓存放置策略,快速响应节点文件请求,提高节点用户在海量信息中发现所需求内容以及共享内容的效率,是未来CDN-P2P技术重要的发展方向。本文针对上述问题,通过分析CDN-P2P网络的特点,特别是节点用户能动性参与的影响因素,融合智能推荐、搜索引擎技术,对基于Hadoop的CDN-P2P原型系统进行了改进。本文的研究内容包括以下几个方面： (1)通过分析共享内容的类型属性和节点需求的联系,计算用户偏好因子,然后结合节点用户历史评分相似性和偏好因子,改进协同过滤方法的预测函数,对节点用户需求预测模型进行分析研究。 (2)研究传统CDN技术,并结合现有CDN-P2P系统中节点子网组织的特性以及节点之间的相似性,对目前系统中内容预存策略进行重新设计。 (3)鉴于节点用户对内容共享的需要,为了方便用户查找相关信息,基于Solr设计实现了一个共享内容搜索子系统,用户可以通过输入关键词来查找资源信息。 (4)在CDN-P2P原型系统中对上面提出的节点用户需求预测模型和边缘服务器内容预存策略予以实现。
[Abstract]:In the past decade, with the rapid development of the Internet, the amount of network information and the number of users have increased dramatically. The content of network sharing and transmission has also expanded from simple text, pictures to audio, video and other complex structures, various forms of multimedia. In order to efficiently distribute network content, alleviate network congestion and enhance user experience, CDN and P2P technology are widely used in many fields as the main technology of network content distribution. Considering that CDN and P2P technologies are complementary to each other in providing services, CDN-P2P convergence technology has also become a new research hotspot. With the continuous expansion of network scale and the proliferation of shared resource information, many problems have been brought to the sharing of node files in CDN-P2P network and the provision of edge server file services. It is shown that the number of nodes and the number of files provided by the CDN-P2P network need to be increased, which is between the edge server and the content source server, and the demand for the storage load of the edge server and the response time of the file request of the P2P node. Or the frequent transfer of files between edge servers not only increases the response time of node file requests, but also consumes bandwidth resources. At the same time, node users also need to spend a lot of time searching for their own content in the massive resource information. It is an important development direction of CDN-P2P technology in the future to improve the content cache policy of edge server in CDN-P2P network, respond to the request of node file quickly, and improve the efficiency of node users finding the required content and sharing content in the massive information. In order to solve the above problems, this paper analyzes the characteristics of CDN-P2P network, especially the influence factors of node user's active participation, integrates intelligent recommendation and search engine technology, and improves the CDN-P2P prototype system based on Hadoop. The research content of this paper includes the following aspects: 1) by analyzing the relationship between the type attributes of shared content and node demand, the user preference factor is calculated, and then the prediction function of collaborative filtering method is improved by combining the similarity of node users' history score and preference factor. The node user demand prediction model is analyzed and studied. (2) the traditional CDN technology is studied, and combining with the characteristics of node subnet organization and the similarity between nodes in the existing CDN-P2P system, the content storage strategy in the current system is redesigned. In view of the needs of node users for content sharing, in order to facilitate users to find relevant information, a shared content search subsystem is designed and implemented based on Solr. Users can search resource information by entering keywords. 4) implement the node user demand prediction model and the edge server content storage strategy in the CDN-P2P prototype system.
【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP393.02

【参考文献】