基于内容的垃圾邮件检测方法研究

发布时间：2018-05-16 19:03

本文选题：基于内容 + 垃圾邮件　；参考：《浙江大学》2014年硕士论文

【摘要】：随着互联网的发展和普及,电子邮件已经成为人们一个相互交流、获取资讯的重要渠道。伴随着信息技术的发展垃圾邮件也给人们带来了诸多的危害。在解决垃圾邮件问题众多的途径和思路中,基于内容的垃圾邮件检测是目前最主流的反垃圾邮件方法之一。文本分类与特征降维分别是基于内容的垃圾邮件检测不可或缺的两个部分。文本分类是在已知类别标号的训练集上通过寻找规律或统计学习建立垃圾邮件检测分类模型的方法。基于内容的分类算法不需要考虑语义环境,只需对文本进行相关统计计算,具有自学习,自适应能力和较高的准确率。特征降维是基于内容的垃圾邮件检测中的另一项关键流程。使用向量空间模型来表示邮件文本时,特征向量空间通常呈现高维特性,不利于垃圾邮件检测系统的运行。对原始高维特征空间进行降维处理就成了垃圾邮件检测中的重要步骤。特征降维方法中的特征选择算法具有性能较好,可行性强等优点,是基于内容的垃圾邮件检测中的重要研究对象。本文对不同垃圾邮件检测组合进行了研究,通过它们性能的优劣对比,在此基础上提出一种基于邮件内容特征的结合正交质心特征选择算法(Orthogonal Centroid Feature Selection, OCFS)和支持向量机(Support Vector Machine, SVM)的新型垃圾邮件检测架构OCFSVM,该组合从垃圾邮件内容特征入手,通过OCFS算法成功地从原始高维特征空间中选出足够类别代表能力的特征子集,通过SVM学习方法对降维后的垃圾邮件语料库低维向量空间模型分类运算,并将其作垃圾邮件检测的基础架构。本文的主要工作和创新点如下： (1)通过对垃圾邮件特征降维阶段及分类阶段的分析,在特征降维阶段引入应用较少的OCFS算法,结合文本分类领域的SVM算法,提出了一种结合OCFS和SVM的新型垃圾邮件检测架构OCFSVM。并在matlab、C#、Eclipse平台和怀卡托智能分析环境(Waikato Environment for Knowledge Analysis, Weka)上,实现了垃圾邮件检测架构OCFSVM. (2)在垃圾邮件检测实验平台上,引入PU系列英文语料库、ZH1中文语料库、中英文混杂自选垃圾邮件集,三种特征降维方法,三种文本分类算法,进行了综合有效的OCFSVM检测架构性能的对比实验。 (3)以精准度和F值为评价指标,在对实验结果进行分析的基础上总结出各检测组合的规律,验证了新型垃圾邮件检测架构OCFSVM能够在不同的邮件环境下对垃圾邮件实施有效的检测,且较传统的检测架构性能得到了较大的提高。
[Abstract]:Along with the development and popularization of Internet , e - mail has become an important channel for people to communicate with each other and get information . Along with the development of information technology , spam brings a lot of harm to people . In solving spam problem many ways and ideas , content - based spam detection is one of the most popular anti - spam methods . Text classification and characteristic dimensionality reduction are two indispensable parts of spam detection based on content , respectively .

Text classification is a method for establishing spam detection classification model by searching regular or statistical learning on a training set of a known class label . The content - based classification algorithm does not need to consider the semantic environment , and only needs to carry out relevant statistical calculation on the text , and has the self - learning , self - adaptive ability and high accuracy .

The feature selection algorithm has the advantages of good performance , strong feasibility and the like , and is an important research object in the content - based spam detection .

In this paper , a new spam detection architecture ( OCFSVM ) based on mail content features based on orthogonal centroid feature selection ( OCFS ) and Support Vector Machine ( SVM ) is proposed . Based on the comparison of their performance , a new spam detection architecture ( OCFSVM ) based on mail content features combining orthogonal centroid feature selection ( OCFS ) and Support Vector Machine ( SVM ) is proposed .

The main work and innovation points of this paper are as follows :

( 1 ) Based on the analysis of the feature reduction stage and classification stage of spam , a new OCFSVM combining OCFS and SVM is proposed in this paper , which combines the SVM algorithm in the field of text classification . The OCFSVM is implemented on matlab , C # , Eclipse platform and Waikato Environment for Knowledge Analysis , Weka .

( 2 ) On the spam detection experiment platform , we introduce the PU - series English corpus , the ZH1 Chinese corpus , the Chinese - English hybrid self - selection spam set , the three feature - reducing methods , the three kinds of text classification algorithms , and carry out the comparison experiment of the comprehensive and effective OCFSVM detection architecture performance .

( 3 ) Based on the analysis of the experimental results , the rule of each test combination is summarized based on the precision and the F value . The new spam detection architecture OCFSVM can effectively detect the spam in different mail environments , and the performance of the traditional detection architecture is greatly improved .

【学位授予单位】：浙江大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.098;TP393.08

【参考文献】