海量视频共享系统中基于溯源的数据减缩方法

发布时间：2018-07-26 06:22

【摘要】：随着视频共享系统和编辑软件的出现以及广泛使用,共享系统中的视频数量和种类快速增加。在这些海量视频中存在大量重复或在内容上高度近似的视频,增加了用户与视频共享系统之间的网络带宽消耗。视频的访问热度分布服从帕勒托分布,少数热门视频占据绝大部分访问量,而大量冷视频的访问量很少或是没人访问。共享系统中不能发生用户数据丢失,导致大量冷数据的长期存储累积开销会非常可观。如何有效地减少近似视频上传带宽消耗,保证视频数据不丢失的前提下实现冷视频数据空间减缩来节省系统整体开销且不对用户的访问造成过多影响成了一个挑战。设计了一种在海量视频共享系统中基于溯源的数据减缩方法—Dpbvd,对系统中已有视频进行编辑时,通过将视频的编辑操作转移到服务器上,避免视频的上传,从而节省带宽。同时又缩减系统中具有溯源信息的冷门近似视频数据来降低服务器整体管理开销。数据减缩方法是指重复数据删除和近似冗余数据压缩。在客户端,用户编辑视频时收集相关的编辑操作信息和视频信息作为新生成视频的应用层溯源信息,并上传至服务器。在服务器端,系统记录每一个阶段内视频的访问次数以及自上传后的累积访问次数,综合考虑视频的访问热度趋势和近期访问次数来动态预测视频的冷热程度;缓存热度高的视频,降低访问时延;对于冷视频,保留不具有溯源信息的冷视频。通过扫描收集的溯源信息,找到具有溯源信息的冷门视频而将其压缩而节省存储开销,具有溯源信息的视频压缩后可根据溯源信息而重新生成,从而系统不会发生数据丢失。试验测试表明,海量视频共享系统中基于溯源的数据减缩方法能够有效节省共享系统视频存储空间,且冷热视频预测精确度高,不会显著影响用户访问视频的速度。
[Abstract]:With the emergence and wide use of video sharing systems and editing software, the number and types of video in sharing systems are increasing rapidly. There are a large number of repetitive or highly similar videos in these massive videos, which increase the network bandwidth consumption between users and video sharing systems. The frequency of video access is distributed from Paleto, with a few popular videos accounting for most of the traffic, while a large number of cold videos have little or no access. There is no loss of user data in the shared system, which leads to a large amount of cold data long-term storage accumulation costs will be very considerable. It is a challenge how to reduce the bandwidth consumption of approximate video uploads effectively and realize the reduction of cold video data space without losing video data so as to save the overall cost of the system and not affect the users' access too much. A traceability based data reduction method-Dpbvd- based on traceability in a mass video sharing system is designed. When the existing video in the system is edited, the editing operation of the video is transferred to the server to avoid the uploading of the video and thus the bandwidth is saved. At the same time, the cold door approximate video data with traceability information in the system is reduced to reduce the overall management overhead of the server. Data reduction refers to repetitive data deletion and approximate redundant data compression. On the client side, the user collects the relevant editing operation information and video information as the application layer information of the newly generated video, and uploads it to the server. On the server side, the system records the number of video access and the cumulative number of visits since uploading, synthetically considering the trend of access heat and the number of recent visits to dynamically predict the cold and hot degree of video. Cache high temperature video, reduce access delay; for cold video, keep cold video without traceability information. By scanning the collected traceability information, we can find the cold-door video with traceability information and compress it to save storage overhead. The video with traceability information can be regenerated according to the traceability information, so that data loss will not occur in the system. The experimental results show that the traceability based data reduction method in mass video sharing system can effectively save the storage space of video in the shared system, and the accuracy of cold and hot video prediction is high, and the speed of users accessing video will not be significantly affected.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP311.52

【相似文献】