基于阈值自适应的异常检测方法研究

发布时间：2018-05-21 01:15

本文选题：异常检测 + 异常分值　；参考：《中国民航大学》2015年硕士论文

【摘要】：近年来,网络攻击事件、网络安全问题备受关注。网络异常检测作为一种主动的安全防御措施,由于它既可以检测到已知攻击,又可以检测到未知的恶意攻击,目前对异常检测技术的研究已经成为热点。本文对三种常用的异常检测模型:最大熵异常检测模型、阈值随机游走异常检测模型、数据包报头异常检测模型进行分析。发现它们计算量大、通用性差而且误报率和漏报率高。本文结合马尔可夫链(Markov Chain)对研究对象进行研究,提出一种计算量小且通用性好的网络异常检测算法,并通过自适应调整异常检测系统的阈值,极大地降低了系统误报率和漏报率。实时网络异常检测在工作时首先将输入数据流转换成有意义的、可量化的异常分值。随后把这些分值与一个固定检测阈值对比,从而分类为正常或异常。然而实时异常检测系统的输入随着时间的推移会发生很大变化,采用固定阈值不能保证良好的异常检测精度。针对这一问题,本文提出一种能够依据网络流量变化自适应调整阈值的方法,论文主要开展了以下工作:首先,本文对异常分值的大量统计特征进行分析,发现异常分值在正常活动期间呈现出一致的相关性衰减结构,而在发生异常行为时,这种依赖性关系会发生明显的变化,据此把时间依赖性作为建模的一个重要指标。然后,根据异常分值存在的相关性衰减的趋势,本文分析并得出由异常分值的马尔可夫性质导致的该趋势。因此本文采用马尔可夫链对异常分值进行建模,在建模基础上进一步提出阈值自适应检测算法,并对算法的步骤和操作流程进行了详细的设计。在该方法中将不同的异常分值作为不同的状态,再对不同状态之间的转移构造状态转移矩阵P,进而使用之前的异常分值预测下一时刻的阈值,从而实现了阈值的自适应调整。最后,本文将提出的阈值自适应方法分别与三种异常检测模型相结合,并通过三种公开的网络流量数据集进行评估实验。结果表明了本论文提出的阈值自适应算法的可行性、有效性,同时证明本文算法在无需人工干预的情况下能够提供较高的检测精度。
[Abstract]:In recent years, the network attack event, the network security question receives the attention. As an active security defense measure, network anomaly detection can detect both known attacks and unknown malicious attacks. At present, anomaly detection technology has become a hot topic. In this paper, three commonly used anomaly detection models: maximum entropy anomaly detection model, threshold random walk anomaly detection model and packet header anomaly detection model are analyzed. It is found that they have a large amount of calculation, poor generality and high false alarm rate and false alarm rate. In this paper, a network anomaly detection algorithm based on Markov chain (Markov chain) is proposed, which has low computational complexity and good generality, and the threshold of anomaly detection system is adjusted by adaptively adjusting the threshold value of anomaly detection system. The false alarm rate and false alarm rate of the system are greatly reduced. Real-time network anomaly detection first converts the input data stream into a meaningful quantifiable anomaly score. These scores are then compared with a fixed detection threshold to classify them as normal or abnormal. However, the input of real-time anomaly detection system will change greatly with the passage of time, and the use of fixed threshold can not guarantee a good accuracy of anomaly detection. In order to solve this problem, this paper proposes a method to adjust the threshold according to the network traffic changes. The main work of this paper is as follows: first, this paper analyzes a large number of statistical characteristics of abnormal scores. It is found that the abnormal score shows a consistent correlation attenuation structure during normal activity, but when abnormal behavior occurs, the dependence relationship will change obviously. Therefore, time dependence is regarded as an important index for modeling. Then, according to the trend of correlation attenuation of abnormal score, this paper analyzes and obtains the trend caused by the Markov property of abnormal score. In this paper, Markov chain is used to model the abnormal value. Based on the modeling, a threshold adaptive detection algorithm is proposed, and the steps and operation flow of the algorithm are designed in detail. In this method, different outliers are taken as different states, then the state transition matrix P is constructed for the transition between different states, and then the threshold of the next moment is predicted by using the previous outlier score, thus the adaptive adjustment of the threshold is realized. Finally, the proposed threshold adaptive method is combined with three kinds of anomaly detection models, and the evaluation experiments are carried out through three kinds of open network traffic data sets. The results show that the threshold adaptive algorithm proposed in this paper is feasible and effective. At the same time, it is proved that the proposed algorithm can provide high detection accuracy without manual intervention.
【学位授予单位】：中国民航大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：TP393.08

【参考文献】