网络数据冗余模式分析系统研究与实现

发布时间：2018-04-13 20:03

本文选题：DRE + 冗余模式　；参考：《北京邮电大学》2014年硕士论文

【摘要】：随着互联网的不断发展和普及,互联网所承载的数据流量增长迅速,互联网服务所需的网络带宽也在不断增加。作为互联网的基础支撑服务,CDN网络承载了全球大部分的互联网流量,数据冗余消除(DRE)技术在广域网加速和单网站冗余消除方面已经得到了一定程度的应用,但针对CDN网络所传输数据的数据冗余消除技术目前还没有被实际应用。通过分析CDN网络数据传输中的冗余模式特性,来指导DRE技术在CDN网络中的实际应用,以提高CDN网络中的数据传输效率,是目前亟待解决的问题。本文在实验室项目“数据冗余消除技术在CDN网络流量压缩中的应用研究”的基础上,完成了针对CDN网络流量的冗余模式分析系统各个模块的设计与实现。整个系统分为数据预处理模块、数据基础分析模块、冗余模式分析模块和结果分析与展现模块,输入的网络流量数据经过预处理模块被处理成按时间段划分的数据文件,通过数据基础分析模块提取网络数据包中的各种特性,并决定用以区分不同客户端流量的字段,然后经过冗余模式分析模块完成对流量数据冗余特性的分析,得出各项技术参数和时间周期等对于DRE压缩效果的影响,最后经过分析与展现模块呈现出分析结果。论文首先分析了CDN网络流量中数据冗余消除的特点,对现有冗余消除的应用场景进行了研究。其次对DRE技术的缓存字典与替换算法、字典的冷热模式、动态静态模式等核心部分进行了研究和总结,对DRE技术相关的指纹生成算法中使用的拉宾卡普算法,MAXP算法进行了概括和总结。在此基础上,作者基于对于DRE技术的理解,改进了现有冗余模式分析技术方案的缺点,设计并实现了网络数据冗余模式分析系统。本文从系统设计思想、技术路线、系统流程和系统模块构成等方面对冗余模式分析系统进行全面的介绍。最后本文介绍了借助网络数据冗余模式分析系统对示例网络数据进行分析后得出的详细分析结论与成果。论文基于对CDN网络数据冗余模式分析的需求,将DRE等算法和对HTTP网络数据内容类型等信息的识别方法在数据冗余分析中进行了应用和工程实现,进而开发完成了一个对网络数据冗余模式进行分析的系统。该系统实现了对大量网络数据冗余模式分析的流程,可以用在对特定数据冗余模式分析的研究领域,具有一定的工程应用价值。
[Abstract]:With the continuous development of the Internet and the popularity of the Internet data traffic load increases rapidly, the required Internet service network bandwidth is increasing. The support service as the foundation of the Internet, the CDN network carries most of the global Internet traffic, data redundancy elimination (DRE) technique in Wan acceleration and single site redundancy elimination has been applied to a certain extent, but for the CDN network data transmission data redundancy elimination technology has not yet been applied. Through the analysis of redundancy mode characteristics of CDN network data transmission in practical application, to guide the DRE technology in the CDN network, to improve the efficiency of data transmission in CDN network, is the most important to solve the problem.
Based on the laboratory project "research and application of technology in the CDN network traffic data compression redundancy elimination, the completion of the redundancy mode of CDN network traffic analysis system design and implementation of each module. The whole system is divided into data preprocessing module, data analysis module, module and the redundancy analysis and mode analysis show the input module, the network traffic data after preprocessing module is processed into divided by time data file extraction module and network data packet in various characteristics through the analysis of the data base, and decided to use the field to distinguish different client traffic, and then through the redundancy mode analysis module to complete analysis of the traffic data redundancy, the technical parameters and cycle time for DRE compression effect, finally through the analysis and display module shows the results of the analysis.
This paper first analyzes the characteristics of data redundancy elimination CDN in network traffic, the application scene of existing redundancy elimination are studied. Secondly the DRE technology and dictionary cache replacement algorithm, and model dictionary, dynamic static model of core part is studied and summarized, the Rabin Karp algorithm using fingerprint generation algorithm related to DRE technology, MAXP algorithm is analyzed and summarized. On this basis, the author for the understanding of DRE technology based on improved redundancy mode analysis method has its disadvantages, the design and implementation of network data redundancy mode analysis system. The system design thought, technical route, system process and system module etc. a comprehensive introduction of redundancy mode analysis system. Finally this paper introduces the analysis of redundant data model system for example network data analysis Detailed analysis conclusions and results obtained.
The analysis of CDN network data redundancy mode based on the needs of the DRE algorithm and the HTTP network data content type information recognition method in data redundancy analysis of application and engineering, and has developed a network of data redundancy mode for system analysis. The system realizes the analysis of a large number of redundant data model of the process, can be used in the research field of analysis of the specific data redundancy mode, it has certain engineering application value.

【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.06

【相似文献】