基于Spark平台的加密流量应用服务分类

发布时间：2018-10-05 08:57

【摘要】：随着互联网的快速发展,网络规模空前庞大,网络应用的复杂度和多样性都在不断地提升。网络安全形势的日益严峻使得人们对网络信息的保护愈加重视,SSL/TIS协议作为网络中使用最广泛的网络安全协议,既可以保护用户隐私,也可以隐藏非法攻击。然而现有的对SSL/TLS加密流量研究很少涉及到应用层的网络应用服务,SSL/TLS中的网络应用服务复杂多样,对应用服务的分类有助于加密流量的监管和分析。本文基于已有的SSL/TLS流量研究提出一种对加密流量应用服务进行分类的方法,并基于此方法实现了基于Spark平台的流量分类系统。首先,本文针对加密流量的分类提出了一种对SSL/TLS加密流量中应用服务进行标注的方法,该方法通过选取SSL/TLS握手过程中的证书域名、证书链信息和SessionID,并辅助“Client Hello”中的扩展字段,根据与应用服务的映射关系来对加密流量进行标注。其次,本文根据SSL/TLS加密流量中应用服务在网络协议栈中处于应用层较高位置的情况,提出先使用简单的C4.5决策树算法对流量中的HTTPS流量进行二分类过滤识别,然后再使用随机森林对HTTPS上的应用服务进行多分类,并根据网络环境对样本特征值做了相应优化。在实验中该方法对应用服务的整体分类准确率达到了96%。最后基于该方法实现了基于Spark平台流量分类系统,该系统运用Spark Streaming结合Spark MLlib,并整合了 Kafka实现消息管理,通过多个模块实现流量的实时分类,并取得了良好的效果,平均准确率达到90%以上。
[Abstract]:With the rapid development of the Internet, the scale of the network is unprecedented, and the complexity and diversity of network applications are increasing. The increasingly severe situation of network security makes people pay more attention to the protection of network information. As the most widely used network security protocol, SSL / TIS protocol can not only protect the privacy of users, but also hide illegal attacks. However, the existing research on SSL/TLS encrypted traffic rarely involves the application layer of network application services. The classification of application services is helpful to the monitoring and analysis of encrypted traffic. Based on the existing SSL/TLS traffic research, this paper proposes a method to classify encrypted traffic application services, and implements a traffic classification system based on Spark platform. First of all, this paper proposes a method to annotate the application services in the SSL/TLS encrypted traffic according to the classification of the encrypted traffic. This method selects the domain name of the certificate in the process of shaking hands with the SSL/TLS. Certificate chain information and SessionID, and auxiliary extended fields in "Client Hello" are used to annotate encrypted traffic according to the mapping relationship with application services. Secondly, according to the situation that the application service of SSL/TLS encryption traffic is in the higher position of application layer in the network protocol stack, a simple C4.5 decision tree algorithm is proposed to filter and identify the HTTPS traffic in the traffic. Then the application services on HTTPS are classified by random forest, and the sample eigenvalues are optimized according to the network environment. In the experiment, the accuracy of the whole classification of application services is 96%. Finally, the traffic classification system based on Spark platform is implemented based on this method. The system uses Spark Streaming and Spark MLlib, and integrates Kafka to realize message management, and realizes real-time traffic classification through several modules, and achieves good results. The average accuracy was over 90%.
【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP393.08

【相似文献】