面向用户体验的文件分发系统调度机制优化研究

发布时间：2019-01-18 08:48

【摘要】：视频直播及点播、视频会议、软件下载、游戏更新等实时和非实时文件分发服务已经成为互联网流量的主要来源,随着高清视频等大容量、高清晰度视频内容的日益丰富,用户对实时和非实时文件分发服务的体验质量要求越来越高。如何优化文件分发系统使其更好地满足用户体验质量要求,是吸引学术界和工业界共同兴趣的热点问题。本文基于大规模实际运营的文件分发系统(PPTV,腾讯旋风下载平台),对无线信道下视频播放、用户需求、缓存配置以及云带宽资源部署等四个方面进行测量分析和理论研究,发现现有的资源分发、配置策略在云端带宽分配方面未充分考虑云带宽对Swarm (即拥有或需要相同资源的用户群体)的影响,用户的个人需求预测尚未得到有效解决,Flash Crowd发生时云端资源消耗过高,无线信道中用户视频播放卡顿现象显著。为此本文分别从用户端优化设计、用户个人需求预测、缓存配置优化以及云端带宽分配这几个角度探索改善用户体验、节约系统资源的有效方法,并通过理论分析及实验仿真验证这些方案的有效性。本文主要工作和创新点如下：1)用户端体验优化：实际测量表明,现有的自适应动态码率实时视频方案,在无线信道条件下卡顿现象显著。目前的研究利用历史知识和当前信道条件调整视频码率切换策略,导致码率频繁切换。本文利用无线信道模型推断信道未来的变化,设计了基于非确定状态决策模型的码率切换算法,能够避免频繁的码率切换,使用户获得最优的视频播放体验,并给出了接近最优算法性能的启发式方案,最后通过仿真实验证实了算法的有效性。2)用户需求和总体流行度预测：基于推荐的方法可以精确预测用户未来需要的资源,但预测结果没有时效性。在电视连续剧发布场景下,依赖剧集间的相关性以及用户的看剧模式并使用机器学习方法,预测用户未来一天需要观看的剧集,该方法具有时效性,能够用于资源部署方案设计。之后针对不同用户类型分别设计了相应的总体需求量预测方法,结果相比ARIMA算法精度提高12%。3)缓存、用户资源利用：热门新文件发布时,大量用户请求文件使云端负荷过高。在文件发布前采用预分发策略提前部署文件给用户,就可以在文件发布后有效利用P2P来缓解云端压力。传统预分发方案只是依据用户历史在线行为以及客户端性能挑选用户,帮助文件扩散,而不考虑用户是否需要云端提供的文件。在电视连续剧发布场景中,大量用户看一两集之后可能弃剧,将文件部署给不需要该文件的用户会浪费宝贵的云端资源。本文基于用户需求预测,协同调度缓存资源和用户资源,设计了最小化云端负载的前摄式缓存算法,在本文的仿真实验中发现它能节约40%的云端流量消耗。4)用户群体(Swarm)间的云端带宽分配：云与P2P协作的系统内,P2P贡献能力不稳定且不同Swarm的P2P贡献能力不尽相同,使用云端带宽作为补充可以保障用户的体验。现有的带宽分配算法主要集中于直播场景或P2P带宽的分配,不涉及云端带宽资源分配对Swarm内用户下载生命周期以及P2P共享能力的影响。在文章中,基于流模型探讨了上述两个问题,得到了云端带宽与用户下载速率的关系,并在云端带宽资源受限的前提下提出了Swarm间的带宽分配算法,优化系统内用户的体验。
[Abstract]:Real-time and non-real-time file distribution services such as video live broadcast and on-demand, video conference, software download and game update have become the main source of internet traffic, and with the high-capacity and high-definition video content of high-definition video, The user experience quality requirements for real-time and non-real-time file distribution services are becoming more and more high. How to optimize the document distribution system makes it better to meet the user's experience quality requirements, and is a hot issue to attract the common interest of the academia and industry. Based on the large-scale practical operation of the file distribution system (PTV, Tencent's cyclone download platform), this paper carries out the measurement and analysis and the theoretical research on the video playing, the user's demand, the cache configuration and the cloud bandwidth resource deployment in the wireless channel, and finds out the existing resources distribution, The configuration policy does not take full account of the influence of the cloud bandwidth on the Swarm (that is, the user group with or needs the same resource) in the cloud bandwidth allocation, and the personal demand forecast of the user has not been effectively solved, and the cloud resource consumption is high in the event of the occurrence of the Flash Crown. the phenomenon of the user video playing card in the wireless channel is significant. In this paper, the effective methods of improving the user experience and saving system resources are explored from the aspects of the client-side optimization design, the user's personal demand forecast, the cache configuration optimization and the cloud bandwidth allocation, and the effectiveness of these schemes is verified through the theoretical analysis and the experimental simulation. The main work and innovation point of this paper are as follows: 1) Client experience optimization: The actual measurement shows that the existing self-adaptive dynamic code rate real-time video scheme is significant under the condition of wireless channel. Current research uses historical knowledge and current channel condition to adjust video rate switching strategy, resulting in frequent switching of code rate. In this paper, the wireless channel model is used to infer the future change of the channel, the rate switching algorithm based on the non-deterministic state decision model is designed, the frequent code rate switching can be avoided, the optimal video playing experience can be obtained by the user, and a heuristic scheme is provided which is close to the optimal algorithm performance, Finally, the validity of the algorithm is confirmed by the simulation experiment. 2) The user's demand and the overall popularity prediction: Based on the recommended method, the resource of the user's future needs can be accurately predicted, but the prediction result is not time-effective. in that case of a television series issue scenario, the dependency on the series and the user's watch play mode are depend on and the machine learning method is used to predict the show that the user needs to watch in the next day, and the method has the timeliness and can be used for resource deployment design. The corresponding overall demand forecasting method is designed for different user types. The result is that the accuracy of ARIMA algorithm is improved by 12%. 3) The cache and user's resource utilization: When the hot new file is released, the large number of user request files make the cloud load too high. The pre-distribution policy is used to pre-deploy the file to the user before the file is released, so that the cloud pressure can be effectively relieved by using the P2P after the file is released. The traditional pre-distribution scheme only selects the users and the help files according to the online behavior of the user and the performance of the client, and does not consider whether the user needs the files provided by the cloud. in a television series issue scenario, a large numb of users may abandon that play after a two-set, deploy the file to users that do not need the file to waste valuable cloud resources. This paper designs a proactive caching algorithm to minimize cloud load based on user demand forecast, collaborative scheduling of cache resources and user resources. In this paper, it is found that it can save 40% of cloud flow consumption. 4) The distribution of cloud bandwidth among user groups: In the system of cloud and P2P collaboration, the P2P contribution capability is unstable and the P2P contribution capability of the different Swarm is different, and the cloud bandwidth is used as the supplement to guarantee the user's experience. The existing bandwidth allocation algorithm is mainly focused on the distribution of the live broadcast scene or the P2P bandwidth, and does not relate to the influence of the cloud bandwidth resource allocation on the user download life cycle and the P2P sharing capability in the Swarm. In this paper, the above two problems are discussed based on the flow model, the relationship between the cloud bandwidth and the user's download rate is obtained, and the bandwidth allocation algorithm between the Swarm is put forward under the premise of the limited bandwidth resource of the cloud, and the user experience in the system is optimized.
【学位授予单位】：北京交通大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TN948.6

【相似文献】