基于Hadoop的全网络流量异常监测算法研究
发布时间：2019-05-07 08:14
【摘要】：网络安全防护水平随着网络规模的不断扩大被提升到一个新的高度。传统的入侵检测均基于单条链路或者单个节点，但是在大规模网络中大多数异常在单条链路或者单个节点的表征不明显，这就导致传统的入侵检测不能兼顾到大规模网络，，因此需要将云计算引入到入侵检测中。Hadoop云计算平台在海量数据处理上具有的高效、高容错、高扩展和高可靠性以及开源的特点均有利于提高海量数据的处理能力，因此将Hadoop云计算平台引入入侵检测势在必行。 本文首先主要研究了Hadoop的两个关键技术：HDFS存储框架和MapReduce计算框架。分析并拆解了多尺度主成分分析(MSPCA)的主要步骤，并且基于MapReduce实现了对MSPCA算法的并行化；其次对并行化的MSPCA算法进行了可扩展性与加速比试验；最后在原型系统中验证并行化MSPCA算法对于异常流量的检测能力。
[Abstract]:With the expansion of network scale, the level of network security protection has been upgraded to a new level. Traditional intrusion detection is based on a single link or a single node, but in a large-scale network, most of the anomalies are not obvious in a single link or a single node, which leads to the traditional intrusion detection can not take into account the large-scale network. Therefore, cloud computing needs to be introduced into intrusion detection. Hadoop cloud computing platform has the characteristics of high efficiency, high fault tolerance, high scalability, high reliability and open source, which are helpful to improve the processing capability of massive data. Therefore, it is imperative to introduce Hadoop cloud computing platform into intrusion detection. Firstly, two key technologies of Hadoop are studied: HDFS storage framework and MapReduce computing framework. The main steps of multi-scale principal component analysis (MSPCA) are analyzed and disassembled, and the parallelization of MSPCA algorithm is realized based on MapReduce. Secondly, the scalability and speedup test of parallel MSPCA algorithm are carried out. Finally, the ability of parallel MSPCA algorithm to detect abnormal traffic is verified in the prototype system.
【学位授予单位】：郑州大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.08
本文编号：2470910
[Abstract]:With the expansion of network scale, the level of network security protection has been upgraded to a new level. Traditional intrusion detection is based on a single link or a single node, but in a large-scale network, most of the anomalies are not obvious in a single link or a single node, which leads to the traditional intrusion detection can not take into account the large-scale network. Therefore, cloud computing needs to be introduced into intrusion detection. Hadoop cloud computing platform has the characteristics of high efficiency, high fault tolerance, high scalability, high reliability and open source, which are helpful to improve the processing capability of massive data. Therefore, it is imperative to introduce Hadoop cloud computing platform into intrusion detection. Firstly, two key technologies of Hadoop are studied: HDFS storage framework and MapReduce computing framework. The main steps of multi-scale principal component analysis (MSPCA) are analyzed and disassembled, and the parallelization of MSPCA algorithm is realized based on MapReduce. Secondly, the scalability and speedup test of parallel MSPCA algorithm are carried out. Finally, the ability of parallel MSPCA algorithm to detect abnormal traffic is verified in the prototype system.
【学位授予单位】：郑州大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.08
【参考文献】
相关期刊论文 前5条
1 王海龙;杨岳湘;李强;;基于子空间方法的大规模网络流量异常检测[J];计算机工程与应用;2007年11期
2 贾冠昕;杨波;陈贞翔;彭立志;;基于NetFlow时间序列的网络异常检测[J];计算机工程与应用;2008年24期
3 肖志新;杨岳湘;杨霖;;基于小波技术的网络异常流量检测与实现[J];计算机科学;2006年10期
4 钱叶魁;陈鸣;叶立新;刘凤荣;朱少卫;张晗;;基于多尺度主成分分析的全网络异常检测方法[J];软件学报;2012年02期
5 胡志刚;梁晓扬;;基于Hadoop的海量网格数据建模[J];计算机系统应用;2010年10期
本文编号：2470910
本文链接：https://www.wllwen.com/guanlilunwen/ydhl/2470910.html
