隐藏型垃圾网页检测技术的研究与实现

发布时间：2018-03-21 12:46

本文选题：垃圾网页　切入点：隐藏型垃圾网页　出处：《西南交通大学》2012年硕士论文　论文类型：学位论文

【摘要】：随着互联网的发展和普及,Web信息出现了爆炸式的增长,搜索引擎已成为人们从这些海量信息中获取所需信息不可或缺的工具。但在搜索结果中,排在前几位的并不是用户真正所需的网页,而是垃圾网页制造者使用黑帽SEO技术制造的与搜索不相关的垃圾网页,甚至是具有欺诈性的重定向垃圾网页。从而浪费了搜索引擎用户的时间,也降低了搜索引擎公司的信誉。在垃圾网页中,隐藏型垃圾网页具有隐蔽性、欺骗性和难以检测等多个特点,已成为垃圾网页检测领域,一个亟待解决的问题。本文综述了目前国内外隐藏型垃圾网页检测技术的研究,详细分析了隐藏型垃圾网页技术特点和类型,特别研究了其中的重定向垃圾网页实现技术。总结归纳了重定向垃圾网页类型,详细分析了这些现象的特征和成因。本文根据已总结的重定向垃圾现象,提出了一种面向搜索结果的重定向垃圾网页检测算法,设计了面向搜索结果的重定向检测系统框架,能对搜索结果的重定向垃圾网页做出有效地检测。本文构建了一个中文垃圾关键词库和中文重定向检测样本数据集,通过实验对提出的重定向检测算法进行了验证,并从混淆矩阵、关键词、伪装网页类型、重定向类型和垃圾网站等多个方面进行了统计分析,获得了很有价值的实验结果。
[Abstract]:With the development and popularization of the Internet and the explosive growth of Web information, search engines have become an indispensable tool for people to obtain the information they need from these huge amounts of information. At the top of the list are not the pages that users really need, but the spam web pages that spammers use black hat SEO technology to create spam pages that are not related to search. Even fraudulent redirection of spam pages wastes the time of search engine users and reduces the reputation of search engine companies. In spam pages, hidden spam pages are hidden. Deceptive and difficult to detect and other characteristics, has become a spam detection field, a problem to be solved. In this paper, the current research on hidden garbage page detection technology at home and abroad is reviewed, and the characteristics and types of hidden garbage page technology are analyzed in detail. Especially, the realization technology of redirected garbage pages is studied, and the types of redirected garbage pages are summarized, and the characteristics and causes of these phenomena are analyzed in detail. In this paper, according to the phenomenon of redirected garbage, a search result oriented redirection garbage detection algorithm is proposed, and the framework of search result oriented redirection detection system is designed. Search results can be redirected spam pages to make an effective detection. In this paper, we construct a Chinese garbage keyword library and Chinese redirect detection sample data set, and verify the proposed redirection detection algorithm through experiments, and from the confusion matrix, keyword, camouflage page type, The statistical analysis of redirection type and spam website is carried out, and valuable experimental results are obtained.
【学位授予单位】：西南交通大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP311.52

【参考文献】