视频分发与缓存协同优化技术研究

发布时间：2018-02-24 19:53

本文关键词： 内容分发网络内容流行度副本替换副本部署云CDN　出处：《中国科学技术大学》2014年博士论文　论文类型：学位论文

【摘要】：在三网融合的大环境下,服务和业务的种类越来越丰富,其中视频内容的播放服务是三网融合的重要服务之一。随着整个视频服务行业的飞速发展,视频内容出现了爆炸式的增长。视频内容通过内容分发网络,分发给电视、PC、手机、PAD等内容终端。视频内容的快速增长及终端用户数量迅猛增加,与此同时基础网络传输带宽发展相对缓慢,为了保证视频服务质量,目前主要通过由内容分发和缓存系统组成的内容分发网络CDN (Content Delivery Network),完成视频内容的分发和缓存。大容量、高并发的视频服务,对视频分发、缓存技术提出了更高要求。视频内容流行度的准确预测、有效的缓存替换算法以及副本的预部署算法对CDN的性能有重要影响。本文围绕三网融合环境下的视频内容的分发与缓存关键技术展开研究工作,重点研究了CDN中视频流行度的预测,分布式缓存系统中移动视频内容缓存的替换,以及基于云存储的CDN中视频内容副本的预部署等策略。本文的主要工作与成果如下： 1)针对基于“推”为主的CDN系统中,视频内容的流行度需要人为估计的问题,本文提出了一种利用贝叶斯网络模型对视频内容的流行度进行预测的方法。在保证用户体验不变的前提下,减少后期系统对视频内容副本的调整,有效减少系统负担,使CDN边缘节点的存储资源能够得到更合理的使用。该方法利用贝叶斯网络通过对视频点播数据进行知识挖掘,实现对需要分发到CDN边缘缓存节点的视频内容流行度的有效预测。实验结果表明,该方法可对视频内容流行度较为准确的预测,电影流行度准确率超过80%。 2)针对分布式协同缓存系统中的内容替换问题,本文提出了一种基于全局信息的内容替换算法。通过理论推导证明了协同缓存优于独立缓存。协同缓存系统中现有的局部贪心算法假定文件长度一致,仅考虑局部的内容信息。考虑这两个不足,本文通过分析移动环境下视频的长度分布与流行度分布情况,建立了视频长度与流行度的分布模型。然后根据分布模型并结合缓存系统的自身特点,针对面向移动视频服务的分布式缓存系统,本文提出了一种基于全局效用值的缓存替换算法,简称VBG算法(Value-Based Global). VBG算法基于全局信息,同时考虑文件长度。实验结果表明,协同缓存系统在三种不同缓存预部署策略下,使用VBG算法的系统传输代价均低于局部贪心算法,在三组实验中,局部贪心算法的传输开销最高约是VBG算法的2.5倍。 3)面向基于云存储的CDN视频内容预部署需求,本文提出了两种离线的副本部署算法：GUCP算法(Greedy User Core Preallocation)和PBP算法(Popularity Based Placement). GUCP算法利用用户请求信息,将副本有针对性的部署到云存储节点上,将用户快速的重定向到拥有所需副本内容的节点,从而解决原有GS (Greedy Site)算法可能引起的用户负载不均问题。实验结果表明,GUCP算法在负载均衡方面的性能优于GS算法很多。当用户数量达1000时,GUCP算法的负载状况值仅为GS算法的1/8。PBP算法则利用内容的流行度计算出副本在系统中所要部署的数量,将用户请求重定向到指定云存储节点,若该节点无法提供副本服务,则从其它节点复制副本。对比其它节点复制副本的平均调整时延和调整费用两种性能指标,PBP算法比随机预部署算法有较大优势。随着用户数量的增加,PBP算法的调整费用近似趋于随机算法的1/2,平均调整时延近似趋于随机算法的3/4。本文的研究工作及成果,依托于国家863重大项目“新一代高可信网络”课题中“新一代业务运行管控协同支撑环境的开发”课题以及华为“分布式分层Cache技术合作”基金项目,已经应用于本文所依托的两个课题(已通过验收)。
[Abstract]:With the development of the whole video service industry , the video content has exploded with the rapid development of the whole video service industry . The content distribution network is distributed to the content terminals such as TV , PC , mobile phone , PAD , etc . The fast growth of video content and the number of end users are increasing rapidly . At the same time , the transmission bandwidth of the base network is relatively slow . In order to guarantee the video quality of service , the content distribution and caching are completed mainly through the content distribution network CDN ( Content Delivery Network ) composed of content distribution and caching system . In this paper , the key technologies of distribution and caching of video content in the three - network integration environment are studied . The prediction of video popularity in CDN , the replacement of mobile video content cache in distributed caching system and the pre - deployment of video content copies in CDN based on cloud storage are studied . The main work and results of this article are as follows : 1 ) In the CDN system based on " push " , the popularity of video content requires man - made estimation . This paper presents a method to predict the popularity of video content using Bayesian network model . Experiments show that the method can predict the popularity of video content accurately , and the accuracy rate of movie popularity is more than 80 % . 2 ) Aiming at the problem of content replacement in distributed cooperative caching system , a content replacement algorithm based on global information is presented in this paper . In this paper , the distribution model of video length and popularity is established by analyzing the distribution of the length and popularity of the video in the mobile environment . Then , according to the distribution model and the characteristics of the caching system , a cache replacement algorithm based on the global utility value is proposed , which is called Value - Based Global . Based on the global information , the VBG algorithm takes into account the file length . The experimental results show that the system transmission cost of the cooperative caching system is lower than that of the local greedy algorithm under three different cache pre - deployment strategies . In the three groups of experiments , the maximum transmission cost of the local greedy algorithm is about 2.5 times that of the VBG algorithm . 3 ) To meet the pre - deployment requirements of CDN video content based on cloud storage , two offline copy deployment algorithms : GUCP algorithm and Popularity Based Placement are proposed in this paper . The results show that the performance of GUCP algorithm is better than GS algorithm when the number of users is 1000 . The results show that the performance of GUCP algorithm is better than that of GS algorithm . When the number of users is 1000 , the load condition value of GUCP algorithm is only 1 / 8 of GS algorithm . When the number of users reaches 1000 , the PBP algorithm is more advantageous than random pre - deployment algorithm . The research work and results of this paper are based on the project of " Next Generation Business Operation Control and Cooperative Support Environment " in the " Next Generation High Trust Network " project of National 863 Major Projects and the project of Huawei " Distributed Tiered Cache Technical Cooperation " , which have been applied to the two topics supported by this paper ( accepted ) .

【学位授予单位】：中国科学技术大学
【学位级别】：博士
【学位授予年份】：2014
【分类号】：TP393.02;TP333

【参考文献】