网络大流测量技术研究
发布时间：2018-08-25 17:51
【摘要】：网络流量测量是研究网络行为的一种有效途径，是对互联网进行管理和控制的基础，在高速网络不断发展的今天，流量测量面临着海量数据存储的问题，这就对测量系统的存储容量和存储速率提出了极大的挑战，而基于数据流的测量通过将数据包按照某种分类原则归并为流，大大节省了存储空间，为流量测量开辟了一个全新的途径。 研究表明[6]，网络中流的总数虽大，但是流表现出非常强烈的重尾分布特征，即9%的数据流占据了大约90%的字节流量，因此，了解大流就能很好地掌握网络通信的大致信息，近年来，随着网络规模的不断增加，网速的空前提高，对大流的研究也显得日趋重要，大流测量逐渐成为网络测量的热点，研究高效准确地大流测量算法在当下具有非常重要的意义。 本文分对大流检测相关算法进行了研究，针对哈希技术、抽样技术等关键技术，对已有算法进行了改进，融合出一种新的大流检测算法。具体来说，本文做的研究工作包括： （1）将流量测量分为大流检测和大流存储两个模块。在大流检测模块，对传统的Counting Bloom Filter（计数型布鲁姆过滤器）进行了改进，改进后的Counting BloomFilter采用了多层结构，相较于传统的Counting Bloom Filter节省了大量存储空间，并能有效解决CBF存在的溢出问题；在大流存储模块，使用定长的LRU结构，LRU结构用双向链表来实现，查找效率高，能有效地对检测出的大流进行存储。经理论分析，本文研究的大流测量算法LRU-MCBF占用空间小，时间复杂度低，并通过仿真实验验证了LRU_MCBF在大流测量中漏报率和错报率较低，能实现高速网络环境下大流对象的准确提取。 （2）将抽样算法融合到LRU-MCBF算法中。在高速网络的大流检测中，主机对数据包的处理速率的要求是无上限的，实现更高的处理速率是网络大流检测中不断追求的目标，而基于抽样算法的大流检测就是一种很好的处理方式，抽样算法易于实现，并且能在保证一定检测准确性的前提下，，大大提高主机处理数据包的速率，是一种“低成本，高成效”的测量方式。本文也对抽样算法同非抽样算法进行了比对，通过实验证明了其准确高效的特点，在大流测量中发挥了非常重要的作用。
[Abstract]:Network traffic measurement is an effective way to study network behavior and is the basis of Internet management and control. With the development of high-speed network, traffic measurement is faced with the problem of massive data storage. This poses a great challenge to the storage capacity and storage rate of the measurement system, and the measurement based on the data flow greatly saves the storage space by merging the data packets into streams according to some classification principle. It opens up a new way for flow measurement. The results show that [6] although the total number of flows in the network is large, the flow shows a very strong heavy-tailed distribution, that is, 9% of the data flow accounts for about 90% of the byte traffic, so knowing the large stream can well grasp the general information of the network communication. In recent years, with the continuous increase of network scale and the unprecedented increase of network speed, the research on large flow becomes increasingly important, and the measurement of large flow has gradually become a hot spot in network measurement. It is of great significance to study the efficient and accurate large flow measurement algorithm. In this paper, the related algorithms of large flow detection are studied. The existing algorithms are improved for the key technologies such as hashing and sampling, and a new algorithm for large flow detection is proposed. Specifically, the research work in this paper includes: (1) the flow measurement is divided into two modules: large flow detection and large stream storage. In the large stream detection module, the traditional Counting Bloom Filter (counting Bloom filter is improved. The improved Counting BloomFilter adopts multi-layer structure, which saves a lot of storage space compared with the traditional Counting Bloom Filter. In the large stream storage module, the fixed length LRU structure is used to realize the bidirectional linked list, and the lookup efficiency is high, and the detected large stream can be stored effectively. The theoretical analysis shows that the large flow measurement algorithm LRU-MCBF has small space and low time complexity. The simulation results show that the LRU_MCBF has a low rate of missing and misreporting in the measurement of large flow. It can realize the accurate extraction of large stream objects in high-speed network environment. (2) the sampling algorithm is integrated into LRU-MCBF algorithm. In the large stream detection of high speed network, there is no upper limit for the processing rate of the data packet in the host computer, and the realization of higher processing rate is the goal of the network large flow detection. The large stream detection based on the sampling algorithm is a good processing method. The sampling algorithm is easy to implement, and can greatly improve the speed of the host processing data packets on the premise of ensuring certain detection accuracy, so it is a kind of "low cost." A highly effective way of measuring. This paper also compares the sampling algorithm with the non-sampling algorithm. It is proved by experiments that the algorithm is accurate and efficient and plays a very important role in large flow measurement.
【学位授予单位】：江南大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.06
本文编号：2203645
[Abstract]:Network traffic measurement is an effective way to study network behavior and is the basis of Internet management and control. With the development of high-speed network, traffic measurement is faced with the problem of massive data storage. This poses a great challenge to the storage capacity and storage rate of the measurement system, and the measurement based on the data flow greatly saves the storage space by merging the data packets into streams according to some classification principle. It opens up a new way for flow measurement. The results show that [6] although the total number of flows in the network is large, the flow shows a very strong heavy-tailed distribution, that is, 9% of the data flow accounts for about 90% of the byte traffic, so knowing the large stream can well grasp the general information of the network communication. In recent years, with the continuous increase of network scale and the unprecedented increase of network speed, the research on large flow becomes increasingly important, and the measurement of large flow has gradually become a hot spot in network measurement. It is of great significance to study the efficient and accurate large flow measurement algorithm. In this paper, the related algorithms of large flow detection are studied. The existing algorithms are improved for the key technologies such as hashing and sampling, and a new algorithm for large flow detection is proposed. Specifically, the research work in this paper includes: (1) the flow measurement is divided into two modules: large flow detection and large stream storage. In the large stream detection module, the traditional Counting Bloom Filter (counting Bloom filter is improved. The improved Counting BloomFilter adopts multi-layer structure, which saves a lot of storage space compared with the traditional Counting Bloom Filter. In the large stream storage module, the fixed length LRU structure is used to realize the bidirectional linked list, and the lookup efficiency is high, and the detected large stream can be stored effectively. The theoretical analysis shows that the large flow measurement algorithm LRU-MCBF has small space and low time complexity. The simulation results show that the LRU_MCBF has a low rate of missing and misreporting in the measurement of large flow. It can realize the accurate extraction of large stream objects in high-speed network environment. (2) the sampling algorithm is integrated into LRU-MCBF algorithm. In the large stream detection of high speed network, there is no upper limit for the processing rate of the data packet in the host computer, and the realization of higher processing rate is the goal of the network large flow detection. The large stream detection based on the sampling algorithm is a good processing method. The sampling algorithm is easy to implement, and can greatly improve the speed of the host processing data packets on the premise of ensuring certain detection accuracy, so it is a kind of "low cost." A highly effective way of measuring. This paper also compares the sampling algorithm with the non-sampling algorithm. It is proved by experiments that the algorithm is accurate and efficient and plays a very important role in large flow measurement.
【学位授予单位】：江南大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.06
【参考文献】
相关期刊论文 前5条
1 裴育杰;王洪波;程时端;;基于两级LRU机制的大流检测算法[J];电子学报;2009年04期
2 王洪波;裴育杰;林宇;程时端;金跃辉;;基于LRU的大流检测算法[J];电子与信息学报;2007年10期
3 张连芳,薛飞,舒炎泰;高速网络的自相似业务模型及其性能评价[J];计算机研究与发展;1998年06期
4 刘元珍;刘渊;李小航;;基于DCF的资源可控流抽样[J];计算机工程与应用;2008年19期
5 谢鲲;文吉刚;张大方;谢高岗;;布鲁姆过滤器查询算法[J];软件学报;2009年01期
本文编号：2203645
本文链接：https://www.wllwen.com/guanlilunwen/ydhl/2203645.html
