高速大数据量的网络监视与数据包捕获解析技术研究

发布时间：2018-05-29 23:15

本文选题：WinPcap + 高速文件分片存储　；参考：《华中师范大学》2015年硕士论文

【摘要】：当今互联网技术发展迅猛,与商业交互日益频繁,越来越多的经济活动转移到了线上。网络已经渗透到我们生活的方方面面：我们用网络进行通信、搜索、购物；我们的生活已经离不开网络。在工业生产过程中可以通过网络监视手段,检测网络中的设备及工业生产中各个环节的运行状态,全面掌握并管理追踪所有生产过程和细节。网络数据包的抓取和解析已成为分析管理网络及监视网络设备的重要工具。本文针对特殊的工业应用网络环境,利用WinPcap编程接口在Windows平台下实现一个基本网络监听捕获软件,包括网络数据包捕获、抓包文件存储、网络协议过滤、协议解析、数据查询、流量统计分析等基本网络抓包分析功能。为了满足特殊工业网络环境的工作特性,解决网络中瞬时大量数据的捕获、长时抓包工作的容量和健壮性要求,以及要求网络抓包软件不能丢包的高可靠性问题,本文从以下几个方面展开了特色研究：(1)为了无丢失地捕获网络中突发的大数据量传输时的所有数据包,本文在建立了特殊工业网络应用环境的网络数据传输变化模型的基础上,针对最大的瞬时数据量,利用Endace DAG(?) 9.2x2网络监视采集卡支持大于1Gbps的局域网数据捕获。(2)为了解决在极大网络数据包存储速度大于硬盘读写速度的问题,利用网络长时平均传输数据量不高的特点,通过对缓冲区的动态管理机制,保障数据包的无丢失存储；为解决长时工作中存储数据总量巨大的问题,并方便抓包数据的快速处理和分析调用,本文提出了分片文件管理存储机制,数据包捕获文件能够自动分文件存储或按用户设置的文件大小分别存储。(3)工业网络中不同设备的数据依靠应用层数据类型区分,不同类型的数据解析过程、语义、语法各不相同。利用网络协议过滤和XML可扩展标记语言对设备自定义数据进行解析和预处理,通过数据类型标识灵活调用不同的处理流程,并将预处理结果转录入数据库中,便于后续处理和挖掘。本文还对嗅探程序的工作原理、SQL数据库语言、MFC的消息处理机制及计算资源管理和文件管理进行了讨论和研究。通过对软件使用的网络环境进行特殊需求方分析,建立网络抓包应用的模型,根据需求功能对软件进行模块化的设计,着重对具体的实现方法和满足高性能的特殊改进手段进行了叙述。文章最后对所设计的网络监听程序进行了实例测试和分析讨论,结果表明本系统满足各项功能需求及性能要求。
[Abstract]:With the rapid development of Internet technology and the increasingly frequent interaction with business, more and more economic activities have been transferred to the line. The network has penetrated into all aspects of our life: we use the network for communication, search, shopping; our life has been inseparable from the network. In the process of industrial production, the equipment in the network and the running status of each link in the industrial production can be detected by means of network monitoring, and all production processes and details can be comprehensively grasped and managed and tracked. The capture and resolution of network packets has become an important tool for analyzing, managing and monitoring network devices. Aiming at the special industrial application network environment, using WinPcap programming interface to realize a basic network monitor and capture software under Windows platform, including network packet capture, capture packet file storage, network protocol filtering, protocol analysis, data query, etc. Traffic statistics analysis and other basic network packet analysis function. In order to meet the working characteristics of the special industrial network environment, solve the problems of capturing a large amount of instantaneous data in the network, the capacity and robustness of the long time packet grasping work, and the high reliability problem of requiring the network packet catching software not to lose the packet. In order to capture all the data packets of burst data transmission in the network without loss, this paper establishes the network data transmission variation model of the special industrial network application environment, in order to capture all the data packets in the burst mass data transmission in the network without loss, based on the following several aspects, this paper establishes the change model of the network data transmission in the special industrial network application environment. In order to solve the problem that the storage speed of data packet is faster than that of hard disk, the data capture of LAN larger than 1Gbps is supported by Endace DAGU) 9.2x2 network monitor and data acquisition card in order to solve the problem that the storage speed of data packet is faster than that of hard disk in order to solve the problem. In order to solve the problem of the huge amount of data stored in long working time, we can make use of the feature that the average data quantity is not high in long time, and guarantee the data packet storage without loss through the dynamic management mechanism of buffer. And it is convenient to quickly process and analyze the packet data. In this paper, a file management and storage mechanism is proposed. Packet capture files can be automatically stored in files or stored separately according to the file size set by the user.) the data of different devices in the industrial network can be distinguished by application layer data types, different types of data parsing processes, semantics, The grammar is different. The network protocol filtering and XML extensible markup language are used to parse and preprocess the self-defined data of the device, and the different processing processes are flexibly called through the data type identification, and the preprocessing results are transferred to the database. Easy to follow up processing and mining. This paper also discusses and studies the working principle of sniffer program and the message processing mechanism, computing resource management and file management of SQL Database language (MFC). Through the analysis of the special demand side of the network environment used by the software, the model of the network packet capture application is established, and the modularized design of the software is carried out according to the requirement function. The specific implementation method and special improvement method to meet the high performance are described. At the end of this paper, the network monitor program is tested and analyzed. The results show that the system can meet the requirements of function and performance.
【学位授予单位】：华中师范大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：TP393.08

【参考文献】