网站异常变化监测系统的研究与实现

发布时间：2018-07-31 09:28

【摘要】：互联网技术进步和大数据时代的到来,网站已经成为政府机关、企事业单位、文化传媒、科研院校以及金融证券机构等信息发布和综合应用的平台。网站的使用量逐年上升,网页内容庞大繁杂,要保障网站信息安全、权威和准确,为大众提供正确的信息和服务是网站拥有者的职责所在。然而,网站面临的安全威胁日趋严峻,非法入侵和篡改网站的行为层出不穷,网站的实时监测及防篡改技术成为当前信息安全领域中一个热点的研究课题。设计研发监测网站异常变化的系统对网站安全问题意义重大。对此,本文提出了一整套网站异常变化监测系统的方案来解决这一问题。首先本文研究了网页异常变化的特征,查询了多种网站内容安全保障系统软件的原理和技术,通过综合的优缺点比对和研究,最终选定了基于Hadoop平台的网站异常变化监测系统。本系统预期实现的主要功能包括网站文件数据的获取、异常变化检测以及监测报警。其中包括爬取大量完整的网站文件数据、对文件数据进行HDFS存储,并进行初步过滤,再检测出网站的具体变化内容,以及变化的合法性判断、异常数据的管理等,利用Hadoop平台提供的文件管理系统HDFS和MapReduce分布式计算模型,对大量的网站文件数据进行处理。系统对爬取到的大量网站文件数据进行HDFS存储,并通过索引存储方式加快数据搜索。系统进行异常变化检测时使用MD5信息摘要算法和改进的基于图论的文本比较算法,结合MapReduce计算模型实现快速准确的异常变化检测,对非法链接的判断采用URL地址转换成IP地址分析,对非法词汇的判断采用匹配过滤、中文分词技术与数据挖掘中朴素贝叶斯分类算法相结合,分类过滤出异常变化信息。通过系统设计、实现和测试,系统在功能和性能方面基本满足监测网站异常变化的需求,系统在使用中也表现出稳定、高效、无差错地运行。
[Abstract]:With the development of Internet technology and the arrival of big data era, website has become a platform for government agencies, enterprises and institutions, cultural media, scientific research institutions, financial and securities institutions and other information dissemination and comprehensive application platform. The usage of websites is increasing year by year, and the content of web pages is huge and complicated. It is the responsibility of website owners to ensure the security, authority and accuracy of website information and to provide correct information and services for the public. However, the security threats to websites are becoming more and more serious, and the behavior of illegal intrusion and tampering is emerging in endlessly. The real-time monitoring and tamper-proof technology of websites has become a hot research topic in the field of information security. The design and development of a system to monitor the abnormal changes of websites is of great significance to the security of websites. In this paper, a set of abnormal change monitoring system is proposed to solve this problem. Firstly, this paper studies the characteristics of the abnormal changes of web pages, inquires the principles and techniques of the software of various kinds of website content security system, and finally selects the monitoring system of the abnormal changes of the website based on Hadoop platform through the comprehensive comparison and research of the advantages and disadvantages. The main functions expected to be realized in this system include the acquisition of website file data, abnormal change detection and monitoring and alarm. These include crawling a large number of complete website file data, storing the file data by HDFS, and carrying out preliminary filtering, and then detecting the specific changing content of the website, as well as the legitimacy judgment of the change, the management of abnormal data, and so on. Using the distributed computing model of HDFS and MapReduce, a file management system provided by Hadoop platform is used to deal with a large number of web site file data. The system carries on the HDFS storage to a large number of website file data which crawls, and speeds up the data search through the index storage way. The MD5 information digest algorithm and the improved text comparison algorithm based on graph theory are used to detect the abnormal change in the system, and the MapReduce computing model is used to realize the fast and accurate anomaly change detection. To judge the illegal link, URL address is translated into IP address, matching filter is used to judge the illegal word, Chinese word segmentation technology is combined with naive Bayes classification algorithm in data mining, and abnormal information is filtered out. Through the system design, implementation and testing, the system basically meets the requirements of monitoring the abnormal changes of the website in function and performance. The system also shows stable, efficient and error-free operation in use.
【学位授予单位】：辽宁大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP393.092

【参考文献】