多维自适应Web异常检测系统研究与实现

发布时间：2018-11-15 10:23

【摘要】：数据挖掘技术与机器学习算法相结合能从海量的在线数据中挖掘出正常和异常的行为模式,这不仅大大减少了人工分析的工作量,而且可以有效地解决规则匹配对研究人员安全背景的高要求。但是近年来,动态HTTP请求的爆炸式增长对传统针对于Web应用的检测系统提出了极大的挑战,异常检测算法中存在的一些诸如假正率偏高、适应性差、容易过拟合、时间复杂度高等问题逐渐被暴露出来。同时,随着计算机系统和网络环境之间的差异性不断增大,网络攻击切入点和行为模式也在不断地更替和演变,传统的异常检测系统将很难继续满足企业级的检测需求。针对上述问题,本文提供了一种高性能、多维度、自适应的异常检测系统。本文主要的研究工作包括:(1)构建统计模型:提取每个特定域名下的网络访问数据,计算特定域名下指定检测特征的统计特性参数,构建相应的统计模型;(2)构建多维子系统:依据计算得到的统计特征参数,从不同维度检测网络访问数据集中各条记录的行为特性,生成对应的多维特征向量;(3)构建分类模型:基于统计模型生成的多维特征向量和记录的实际类标,采用机器学习中的分类算法,对访问记录中的异常数据进行检测;(4)模型融合:除了挑选一个性能最优的单模型外,为了提升系统的检测性能,需要对不同的分类器进行进一步的模型融合,使融合后的模型相比于之前任何一个单模型来说都具有更优秀的性能指标;(5)结果反馈:需要根据检测结果的统计特性,判断当前的统计模型和分类器是否失效。如果证明统计或检测模型失效,系统需要重新构建统计模型,进行重新学习。本文基于奇虎360公司提供的海量网络访问日志,对所设计系统中涉及到的7种单分类器和融合模型的检测性能分别进行了详细的分析和评估。实验结果证实,基于足够多“好而不同”的单分类器,通过Stacked generalization网络融合后的模型的检测性能与之前任何一个最优单模型相比都具有一定程度的提高。同时,在与6种主流检测模型的性能比较中,本文提出的检测系统在各种检测维度下都具有最优的检测能力。
[Abstract]:The combination of data mining technology and machine learning algorithm can extract normal and abnormal behavior patterns from massive online data, which not only greatly reduces the workload of manual analysis. Moreover, it can effectively solve the high requirement of rule matching to the safety background of researchers. However, in recent years, the explosive growth of dynamic HTTP requests has posed a great challenge to the traditional detection system for Web applications. Some anomaly detection algorithms such as high false positive rate, poor adaptability, easy to over-fit, Problems such as high time complexity are gradually exposed. At the same time, with the increasing differences between computer systems and network environment, network attack entry points and behavior patterns are constantly changing and evolving, the traditional anomaly detection system will be difficult to continue to meet the needs of enterprise-level detection. To solve the above problems, this paper provides a high performance, multi-dimensional, adaptive anomaly detection system. The main research work of this paper is as follows: (1) build the statistical model: extract the network access data under each specific domain name, calculate the statistical characteristic parameters of the specified detection feature under the specific domain name, and construct the corresponding statistical model; (2) constructing multidimensional subsystem: according to the calculated statistical characteristic parameters, the behavior characteristics of each record in the network access data set are detected from different dimensions, and the corresponding multidimensional feature vectors are generated; (3) constructing the classification model: based on the multidimensional feature vector generated by the statistical model and the actual class mark of the record, using the classification algorithm in machine learning to detect the abnormal data in the access record; (4) Model fusion: in addition to selecting a single model with the best performance, in order to improve the detection performance of the system, further model fusion is needed for different classifiers. Compared with any previous single model, the fused model has better performance index. (5) result feedback: it is necessary to judge whether the current statistical model and classifier are invalid according to the statistical characteristics of the detection results. If the statistical or detection model is proved to be ineffective, the system needs to rebuild the statistical model and re-learn. Based on the massive network access log provided by Qihoo 360, this paper analyzes and evaluates in detail the detection performance of seven kinds of single classifiers and fusion models involved in the designed system. The experimental results show that, based on enough "good and different" single classifiers, the detection performance of the model fused by Stacked generalization network is improved to a certain extent compared with any previous optimal single model. At the same time, compared with the six main detection models, the detection system proposed in this paper has the best detection ability under various detection dimensions.
【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP393.08

【相似文献】