基于日志分析的AAA服务状态监测系统设计与实现

发布时间：2018-05-11 09:24

本文选题：AAA系统 + 日志数据　；参考：《国防科学技术大学》2014年硕士论文

【摘要】：随着电信运营商网络业务类型(2G\3G\4G、WIFI、固定宽带等)的不断增加和用户数量的迅猛增长,其对AAA系统性能和功能的要求也随之增加,这就导致AAA系统组成设备的种类和规模不断增多,由设备软硬件故障、恶意攻击等引发的系统故障也日益频繁。由于各类设备之间互相影响和依赖,单个设备的故障会引发多个设备多种类型的故障日志,加之日志数据格式互不相同,导致通过分析日志数据定位故障源或攻击源、确认故障影响范围等越来越困难。针对上述问题,本文主要完成了以下4项工作:1.提出了一种日志自动收集和模板提取机制ALCTE(Auto Log Collection and Template Extraction),首先基于Flume实现各类设备日志的自动收集和格式统一转换,然后根据日志组成文本所包含词汇的出现频率将其划分为模板词和数据词,从而将一条日志记录分解为日志模板和数据向量,从而实现不同类型日志数据的自动格式归一化,用于解决因设备类型、软件版本、网络层次等不同带来的日志格式不统一、分析困难的问题;2.设计了一种面向故障事件对格式化日志数据进行聚集的方法Co LDFFE(Cluster of Log Data Facing Fault Event),该方法基于经ALCTE机制处理的格式化日志数据实现,通过日志矩阵分解等手段分析各类故障事件(如数据库宕机等)与日志数据的关系,最终获取与某一事件相关的日志模板和数据向量集合,最终确定故障类型、涉及的设备和影响范围等;3.提出了一种基于TF-IDF算法的攻击源检测机制ASDBT(Attack Source Detection Based on TF-IDF),在对某电信公司近一年AAA认证日志统计分析的基础上,该机制通过重新设定TF-IDF算法的参数,计算待筛选数据源与攻击数据源集(已确认的攻击源)的关联度,通过将计算得出的关联度与计算获取的关联度阈值相比较发现和确定其他攻击源,有效弥补了现有检测机制在有效性和高效性上的不足,可高效全面的发现并确定其他攻击源;4.依据从某电信运营商获取的真实AAA系统组成各类设备近1年的日志数据,综合运用ALCTE机制、Co LDFFE方法和ASDBT机制,设计并实现AAA服务状态监测原型系统,基于物理链路阻断、数据库服务宕机、非法登陆攻击等多个不同的故障场景进行模拟实验,验证了上述方法和机制的有效性。
[Abstract]:With the continuous increase of telecom operators' network service type (2G\ 3G\ 4G WIFI, fixed broadband, etc.) and the rapid growth of the number of users, the requirements for the performance and function of AAA system are also increasing, which leads to the constant increase in the types and scale of the equipment components of the AAA system. System failures caused by hardware and software failures and malicious attacks are becoming more and more frequent. Because of the mutual influence and dependence of all kinds of devices, the failure of a single device will cause many kinds of fault logs of multiple devices, and the log data format is different, which leads to the analysis of log data to locate the fault source or attack source. It is becoming more and more difficult to confirm the extent of failure. In view of the above problems, this paper mainly completed the following four tasks: 1. This paper presents a mechanism of automatic log collection and template extraction, ALCTE(Auto Log Collection and Template Extraction.Firstly, based on Flume, the automatic collection and format conversion of all kinds of device logs are realized. Then, according to the occurrence frequency of the words contained in the log composition text, it is divided into template words and data words, thus a log record is decomposed into log templates and data vectors, and the automatic format of different types of log data is normalized. It is used to solve the problem that the log format is not uniform because of the different device type, software version, network layer and so on. A method of gathering formatted log data, Co LDFFE(Cluster of Log Data Facing Fault event, is designed for fault event oriented. The method is based on formatted log data processed by ALCTE mechanism. Through log matrix decomposition and other means to analyze the relationship between all kinds of fault events (such as database downtime) and log data, finally obtain the log template and data vector set related to a certain event, and finally determine the fault type. The equipment involved and the scope of influence etc. An attack source detection mechanism based on TF-IDF algorithm, ASDBT(Attack Source Detection Based on TF-IDF, is proposed. Based on the statistical analysis of the AAA authentication log of a telecom company for nearly one year, the parameters of the TF-IDF algorithm are reset. The correlation degree between the data source to be filtered and the set of attack data sources (identified attack source) is calculated, and the other attack sources are found and determined by comparing the calculated correlation degree with the calculated correlation degree threshold. It can effectively make up for the shortcomings of the existing detection mechanism in effectiveness and efficiency, and can find and identify other attack sources efficiently and comprehensively. According to the real AAA system obtained from a telecom operator, a prototype system of AAA service condition monitoring is designed and implemented, which is based on physical link blocking, using ALCTE mechanism Co LDFFE method and ASDBT mechanism. Several different fault scenarios, such as database service downtime and illegal landing attack, were simulated to verify the effectiveness of the above methods and mechanisms.
【学位授予单位】：国防科学技术大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TN915.06

【参考文献】