基于主题模型的垃圾邮件过滤系统的设计与实现

发布时间：2018-05-26 23:19

本文选题：文本分类 + 垃圾邮件　；参考：《电信科学》2017年11期

【摘要】：垃圾邮件过滤技术在保证信息安全、提高资源利用、分拣信息数据等方面都发挥着重要作用。然而,垃圾邮件的出现影响了用户的体验,并且会造成不必要的经济与时间损失。针对现有的垃圾邮件过滤技术的不足,基于多个主题词理论,构建了基于朴素贝叶斯的垃圾邮件分类方法。在邮件主题获取中,采用主题模型LDA得到邮件的相关主题及主题词;并进一步采用Word2Vec寻找主题词的同义词和关联词,扩展主题词集合。在邮件分类中,对训练数据集进行统计学习得到词语的先验概率;基于扩展的主题词集合及其概率,通过贝叶斯公式推导得到某个主题和某封邮件的联合概率,以此作为垃圾邮件判定的依据。同时,基于主题模型的垃圾邮件过滤系统具有简洁易应用的特点。通过与其他典型垃圾邮件过滤方法的对比实验,证明基于主题模型的垃圾邮件分类方法及基于Word2Vec的改进方法均能有效提高垃圾邮件过滤的准确度。
[Abstract]:Spam filtering technology plays an important role in ensuring information security, improving resource utilization and sorting information data. However, the emergence of spam affects the user's experience and causes unnecessary economic and time losses. Aiming at the deficiency of the existing spam filtering technology, a new method of spam classification based on naive Bayes is constructed based on the theory of multiple subject words. The topic model LDA is used to obtain the topic and the theme words, and the Word2Vec is used to find the synonyms and related words of the subject words to expand the collection of the theme words. In mail classification, the priori probability of words is obtained by statistical learning of training data set, and the joint probability of a topic and a message is derived by Bayesian formula based on the extended subject word set and its probability. This is the basis for judging spam. At the same time, the spam filtering system based on topic model is simple and easy to use. By comparing with other typical spam filtering methods, it is proved that both the method based on topic model and the improved method based on Word2Vec can effectively improve the accuracy of spam filtering.
【作者单位】：华东理工大学信息科学与工程学院;
【分类号】：TP393.098

【相似文献】