基于文本加权KNN算法的中文垃圾短信过滤

发布时间：2018-04-25 20:53

本文选题：垃圾过滤 + 关联规则　；参考：《计算机工程》2017年03期

【摘要】：针对K最近邻(KNN)算法在文本分类决策规则上由于样本重要性相同而导致分类效果不佳的问题,提出一种基于文本加权的KNN文本分类算法,并将其应用于垃圾短信的分类问题。在提取出特征词之后,考虑到特征词在文本中出现的频率对文本重要性的影响,引入第1个加权公式,同时针对垃圾短信数据集,采用关联规则算法挖掘出在垃圾短信中频繁出现的共现词组,并以此引入第2个加权公式,最后将引入的2种文本权重计算公式对每个短信文本进行复合加权处理,以区分各个训练样本对于判定隶属类别的影响程度,从而在分类决策规则上作出改进。实验结果表明,与未经过文本加权的KNN算法相比,该算法对垃圾短信和正常短信在分类准确率、召回率、F1值等指标上都有较大的提升。
[Abstract]:In order to solve the problem that K-nearest neighbor (KNN) algorithm is not effective in text classification because of the same importance of samples, a text-weighted KNN text classification algorithm is proposed and applied to the classification of spam text messages. After extracting the feature words, considering the influence of the frequency of feature words appearing in the text on the importance of the text, the first weighting formula is introduced, and for the spam short message data set, the first weighted formula is introduced. The association rules algorithm is used to mine cooccurrence phrases which frequently appear in spam text messages, and the second weighting formula is introduced. Finally, two kinds of text weight calculation formulas are introduced to deal with each text. In order to distinguish the influence of each training sample on the decision of membership category, the classification decision rules are improved. The experimental results show that compared with the untext-weighted KNN algorithm, the algorithm can improve the classification accuracy, recall rate and F1 value of spam SMS and normal SMS.
【作者单位】：桂林电子科技大学广西可信软件重点实验室;桂林电子科技大学计算机与信息安全学院;
【基金】：广西可信软件重点实验室研究课题(kx201106) 桂林电子科技大学研究生教育创新计划项目(2016YJCX64)
【分类号】：TP391.1

【参考文献】