基于Django和Netfilter的流量监控系统的设计与实现

发布时间：2018-08-19 10:15

【摘要】：随着软件技术的飞速发展,网络应用和系统结构更加复杂、功能更加强大,因此系统必须以模块化,采用集群技术来处理海量的网络流量,面对庞大的流量冲击,系统必须保持高度的稳定性,因此首先得保证自身的网络安全。防火墙技术是基于对流量的识别和分析,它是解决网络安全的一个重要技术手段。 KV检索系统作为国内某互联网公司的核心检索系统,每天要响应全球数亿次查询请求,系统依然能够疾速、智能的响应查询,让人们最平等、便捷地获取信息,找到所求。检索系统对性能要求极高,其内部每个模块必须经过严格的测试才能上线；线下有一套相对线上规模较小,但完整的检索系统用于线下的模块测试,系统受限于规模瓶颈,必须对系统进行流量控制,以保证系统能够正常运行。因此,本文基于Django Web框架技术和Netfilter防火墙技术设计开发了一套流量监控系统。本文主要研究了Django Web框架的内部结构和实现机制,包括框架结构、请求处理机制、model机制和模板系统等,同时分析了MVC软件设计模式的特点。然后研究了linux2.6Netfilter防火墙技术,深入讨论分析Netfilter的理论基础和内核实现机制。本文以linux内核IP协议栈为入口,研究Netfilter框架对IP数据报的处理流程,选中其中一个HOOK点对为切入口,对数据报进行筛选。接着研究iptables数据报高级管理系统,从用户态的角度分析iptables的内部结构特点,并对iptables的命令结构进行了剖析。最后利用Django框架,采用MVC的Web设计模式,设计开发完成流量监控系统,实现了对线下KV检索系统的流量监控。本系统包括用户管理模块、流量申请模块、任务执行模块和流量控制模块等,测试人员只需提交流量申请,系统便可实现流量的自动化监控。该系统经过严格的测试,符合预期,现已在公司内部提供服务,效果很好。
[Abstract]:With the rapid development of software technology, the network application and system structure are more complex and powerful, so the system must be modularized, cluster technology must be used to deal with massive network traffic, facing the huge traffic impact. The system must maintain a high degree of stability, so first of all to ensure its own network security. Firewall technology is based on the identification and analysis of traffic, it is an important technical means to solve the network security. KV retrieval system is the core retrieval system of a domestic Internet company. To respond to hundreds of millions of query requests every day, the system can still respond to queries quickly and intelligently, so that people can obtain information and find what they are asking for in the most equal and convenient way. The retrieval system requires very high performance, and each module in the retrieval system has to go through strict tests before it can go online; there is a set of relatively small scale offline, but the complete retrieval system is used for offline module testing, so the system is limited by the bottleneck of scale. The flow control of the system must be carried out to ensure the normal operation of the system. Therefore, based on Django Web framework technology and Netfilter firewall technology, a set of traffic monitoring system is designed and developed. This paper mainly studies the internal structure and implementation mechanism of Django Web framework, including the framework structure, request processing mechanism and template system, etc. At the same time, the characteristics of MVC software design pattern are analyzed. Then, the linux2.6Netfilter firewall technology is studied, and the theoretical basis and kernel implementation mechanism of Netfilter are discussed. In this paper, the IP stack of linux kernel is taken as the entry, and the processing flow of IP Datagram in Netfilter framework is studied. One of the HOOK points is selected as the entry to filter the Datagram. Then the advanced management system of iptables Datagram is studied, and the internal structure of iptables is analyzed from the point of view of user, and the command structure of iptables is analyzed. Finally, using Django framework and Web design pattern of MVC, the flow monitoring system is designed and developed, and the flow monitoring of offline KV retrieval system is realized. The system includes user management module, flow request module, task execution module and flow control module. The system has been tested strictly, meets expectations, and has been serving within the company with good results.
【学位授予单位】：昆明理工大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.06

【参考文献】