面向业务感知的流量监控技术研究
发布时间：2018-08-18 10:07
【摘要】： 随着互联网、下一代网络等电信技术的出现,宽带用户量急剧上升,网络应用越来越多样化,运营环境发生了很大的变化。P2P、VoIP、VoD、IPTV、在线游戏、即时通信等各种网络应用的出现,对IP网络提出了越来越高的品质需求和带宽需求。在当前情况下,无监管的VoIP业务、P2P业务、异常流量以及不受控的宽带私接占据了大量的带宽,冲击着各大运营商的运营管理模式,运营商逐渐对网络应用失去掌控能力,同时,由于网络管道化严重,运营商增加投入提高网络容量,但是却难以分享增值业务带来的收入,形成“增量不增收”的尴尬局面。这些环境的变化促使运营商必须对网络流量进行精细化的管理和控制,提供差异化的服务。从业务监控、初级运营到高级阶段,通过详细分析流量和用户行为数据,为流量管理和业务运营提供管理手段,并提供定制化服务,从而改进运营手段。针对这一现状和需求,本文从运营商的角度出发,着重研究了面向业务感知的流量监控技术,在基本网络流量监管的基础上,能够深入发掘用户业务类型和用户行为,减少共享私接带来的业务损失,通过定制化的信息推送提高增值业务收入,从而推动运营商由管道商向服务提供商的转型。 本文是以业务采集分析平台技术为基础,辐射业务识别检测、流量控制、共享接入检测和Web推送等多项业务类型,同时深入研究并解决了这些业务实现的关键技术,能够满足不同方面不同场景下对于网络监管和网络服务的要求。具体来说,研究工作主要包括以下几方面： 1、在业务识别监控方面,研究了从端口识别、特征识别到行为分析的业务检测技术以及基于TCP和UDP的业务阻断干扰技术,重点介绍了基于专家库的流量分类技术和基于行为特征的分类算法,其中在专家库识别领域提出了专家库的五种特征模式,并引入了一种利用多流特征进行识别的技术；在行为特征识别领域,提出了一种基于分形理论进行分类的检测技术；此外,在业务识别的基础上,需要对特定业务进行限制、干扰甚至阻断,本文研究了旁路部署模式下的流量控制技术,在面向无连接的UDP干扰阻断技术方面,提出了四种具体的干扰和控制方法及思路,能够覆盖大多数基于UDP的P2P应用的干扰控制。 2、在Web推送业务方面,分析目前主流的Web推送和广告推送技术实现原理,提出了一种Web推送的方法,同时比较了四种主流Web推送方案的优缺点,并分析了各方案所适用的场景,最后从用户、推送内容、推送类型三个层面对推送业务管理策略提出了建议,使Web推送能更好的具备定制化和针对性,从而促进业务增值的拓展。 3、在共享接入检测方面,鉴于其既属于基础流控平台又属于业务范畴的重要性,通过对当前主流的共享接入主机数目检测技术深入的分析,提出了5种检测算法,包括被动Cookie算法、内网IP算法、主动Cookie算法、系统时间算法和MTU算法,按照主动算法和被动算法进行分类,总结了各个算法的优缺点,指出了当前各算法所具有的场景局限性。在此基础上,提出了一种综合性共享上网主机数目检测的模型和系统,建立了先使用被动算法区分用户类型再使用主动算法进行精确主机数目判断的模型,设计了一种可扩展的检测系统,在提高算法准确度的同时,极大地避免了正常用户受到影响,适应于各种不同的检测场景。 4、引入多轨迹识别的概念,对多轨迹识别问题建立了数学模型,并将一类轨迹——恒变循环轨迹的问题归结为特例的剩余类循环轨迹的问题,基于这类轨迹提出了基于冒泡原理的多轨迹识别算法,通过利用各轨迹之间的交替特性依次使各轨迹浮出,容错性能好,在理论上证明了算法的收敛性,利用IPID变化规律具有恒变循环轨迹特征可用于共享接入检测来进行验证,实验证明该算法大大提高了检测效率,对异常情况更具容忍度。 5、在多轨迹识别理论方面,定义了另一类轨迹——二值异或群循环轨迹,并针对这类轨迹提出了高效的检测算法,通过将求解与基轨迹相异或的元素值的问题转化为求解每一特征位的模式问题,极大地提高了算法的可操作性和准确高效性。利用DNS变化规律具备二值异或群循环轨迹特征进行实验,表明该算法需要较少的样本数据即可进行检测,具有较好的容忍度和准确度。
[Abstract]:With the emergence of Internet, next generation network and other telecommunication technologies, the number of broadband users has risen sharply, network applications are becoming more and more diversified, and the operating environment has changed greatly. Under the circumstances, unregulated VoIP services, P2P services, abnormal traffic and uncontrolled broadband private connection occupy a large amount of bandwidth, which impacts the operation and management mode of major operators. Operators gradually lose control of network applications. At the same time, because of the serious network pipelining, operators increase investment to improve network capacity, but it is difficult to divide them. Enjoying the revenue from value-added services has led to the embarrassing situation of "incremental revenue". These changes in the environment have prompted operators to refine the management and control of network traffic and provide differentiated services. Business operations provide management tools and customized services to improve operational means. In view of this situation and demand, this paper focuses on the study of service-aware traffic monitoring technology from the perspective of operators. On the basis of basic network traffic monitoring, users'business types and behavior can be explored in depth to reduce the total number of users. Enjoy the loss of business brought by private connection, through customized information push to improve value-added business income, thereby promoting the transformation of operators from pipeline providers to service providers.
Based on the technology of service collection and analysis platform, this paper studies and solves the key technologies of these services, which can meet the requirements of network supervision and service in different scenarios. The research work mainly includes the following aspects:
1. In the aspect of service identification and monitoring, the technology of service detection from port identification, feature recognition to behavior analysis and the technology of service blocking jamming based on TCP and UDP are studied. The traffic classification technology based on expert library and the classification algorithm based on behavior feature are introduced in detail. Five kinds of expert library are proposed in the field of expert library identification. In the field of behavior feature recognition, a detection technology based on fractal theory is proposed. In addition, on the basis of service identification, it is necessary to restrict, interfere with and even block specific services. This paper studies the traffic in bypass deployment mode. Control technology, in connection-free UDP interference blocking technology, proposed four specific interference and control methods and ideas, can cover most of the UDP-based P2P applications interference control.
2. In the aspect of Web push service, this paper analyzes the implementation principle of current mainstream Web push and advertising push technology, puts forward a method of Web push, compares the advantages and disadvantages of four mainstream Web push schemes, and analyzes the scenarios suitable for each scheme. Finally, it puts forward the push service management policy from three levels: user, push content and push type. Suggestions are put forward to make Web push more customized and targeted, so as to promote the expansion of business value-added.
3. In view of the importance of shared access detection, which belongs to both basic flow control platform and service category, five detection algorithms are proposed, including passive Cookie algorithm, intranet IP algorithm, active Cookie algorithm, system time algorithm and MTU algorithm, according to the current mainstream shared access host number detection technology. Active algorithms and passive algorithms are classified, the advantages and disadvantages of each algorithm are summarized, and the scenario limitations of the current algorithms are pointed out. Based on this, a model and system for the number detection of shared hosts on the Internet is proposed, and a passive algorithm is used to distinguish user types before the active algorithm is used to precisely determine the number of hosts on the Internet. A scalable detection system is designed to improve the accuracy of the algorithm while avoiding the influence of normal users and adapting to various detection scenarios.
4. By introducing the concept of multi-trajectory identification, the mathematical model of multi-trajectory identification problem is established, and the problem of a class of trajectory-constant variable cycle trajectory is reduced to the problem of residual class trajectory. Based on this kind of trajectory, a multi-trajectory identification algorithm based on bubbling principle is proposed, which makes use of the alternating characteristics of each trajectory in turn. The convergence of the algorithm is proved theoretically. The algorithm can be used to verify the shared access detection by using the characteristic of constant variable cyclic trajectory of IPID. The experimental results show that the algorithm greatly improves the detection efficiency and is more tolerant to abnormal situations.
5. In the theory of multi-trajectory identification, another kind of trajectory-binary XOR Group Cyclic trajectory is defined, and an efficient detection algorithm is proposed for this kind of trajectory. By transforming the problem of solving the value of elements which are different from the basic trajectory into the problem of solving the pattern of each feature bit, the operability and accuracy of the algorithm are greatly improved. Experiments show that the algorithm needs less sample data and has good tolerance and accuracy.
【学位授予单位】：北京邮电大学
【学位级别】：博士
【学位授予年份】：2010
【分类号】：TP393.06
本文编号：2189142
[Abstract]:With the emergence of Internet, next generation network and other telecommunication technologies, the number of broadband users has risen sharply, network applications are becoming more and more diversified, and the operating environment has changed greatly. Under the circumstances, unregulated VoIP services, P2P services, abnormal traffic and uncontrolled broadband private connection occupy a large amount of bandwidth, which impacts the operation and management mode of major operators. Operators gradually lose control of network applications. At the same time, because of the serious network pipelining, operators increase investment to improve network capacity, but it is difficult to divide them. Enjoying the revenue from value-added services has led to the embarrassing situation of "incremental revenue". These changes in the environment have prompted operators to refine the management and control of network traffic and provide differentiated services. Business operations provide management tools and customized services to improve operational means. In view of this situation and demand, this paper focuses on the study of service-aware traffic monitoring technology from the perspective of operators. On the basis of basic network traffic monitoring, users'business types and behavior can be explored in depth to reduce the total number of users. Enjoy the loss of business brought by private connection, through customized information push to improve value-added business income, thereby promoting the transformation of operators from pipeline providers to service providers.
Based on the technology of service collection and analysis platform, this paper studies and solves the key technologies of these services, which can meet the requirements of network supervision and service in different scenarios. The research work mainly includes the following aspects:
1. In the aspect of service identification and monitoring, the technology of service detection from port identification, feature recognition to behavior analysis and the technology of service blocking jamming based on TCP and UDP are studied. The traffic classification technology based on expert library and the classification algorithm based on behavior feature are introduced in detail. Five kinds of expert library are proposed in the field of expert library identification. In the field of behavior feature recognition, a detection technology based on fractal theory is proposed. In addition, on the basis of service identification, it is necessary to restrict, interfere with and even block specific services. This paper studies the traffic in bypass deployment mode. Control technology, in connection-free UDP interference blocking technology, proposed four specific interference and control methods and ideas, can cover most of the UDP-based P2P applications interference control.
2. In the aspect of Web push service, this paper analyzes the implementation principle of current mainstream Web push and advertising push technology, puts forward a method of Web push, compares the advantages and disadvantages of four mainstream Web push schemes, and analyzes the scenarios suitable for each scheme. Finally, it puts forward the push service management policy from three levels: user, push content and push type. Suggestions are put forward to make Web push more customized and targeted, so as to promote the expansion of business value-added.
3. In view of the importance of shared access detection, which belongs to both basic flow control platform and service category, five detection algorithms are proposed, including passive Cookie algorithm, intranet IP algorithm, active Cookie algorithm, system time algorithm and MTU algorithm, according to the current mainstream shared access host number detection technology. Active algorithms and passive algorithms are classified, the advantages and disadvantages of each algorithm are summarized, and the scenario limitations of the current algorithms are pointed out. Based on this, a model and system for the number detection of shared hosts on the Internet is proposed, and a passive algorithm is used to distinguish user types before the active algorithm is used to precisely determine the number of hosts on the Internet. A scalable detection system is designed to improve the accuracy of the algorithm while avoiding the influence of normal users and adapting to various detection scenarios.
4. By introducing the concept of multi-trajectory identification, the mathematical model of multi-trajectory identification problem is established, and the problem of a class of trajectory-constant variable cycle trajectory is reduced to the problem of residual class trajectory. Based on this kind of trajectory, a multi-trajectory identification algorithm based on bubbling principle is proposed, which makes use of the alternating characteristics of each trajectory in turn. The convergence of the algorithm is proved theoretically. The algorithm can be used to verify the shared access detection by using the characteristic of constant variable cyclic trajectory of IPID. The experimental results show that the algorithm greatly improves the detection efficiency and is more tolerant to abnormal situations.
5. In the theory of multi-trajectory identification, another kind of trajectory-binary XOR Group Cyclic trajectory is defined, and an efficient detection algorithm is proposed for this kind of trajectory. By transforming the problem of solving the value of elements which are different from the basic trajectory into the problem of solving the pattern of each feature bit, the operability and accuracy of the algorithm are greatly improved. Experiments show that the algorithm needs less sample data and has good tolerance and accuracy.
【学位授予单位】：北京邮电大学
【学位级别】：博士
【学位授予年份】：2010
【分类号】：TP393.06
【引证文献】
相关硕士学位论文 前1条
1 付薇;电信网络管理趋势监控平台的设计与实现[D];北京邮电大学;2011年
，本文编号：2189142
本文链接：https://www.wllwen.com/wenyilunwen/guanggaoshejilunwen/2189142.html
