垃圾短信过滤系统的设计与实现

发布时间：2018-06-24 02:31

本文选题：垃圾短信 + 贝叶斯分类　；参考：《电子科技大学》2009年硕士论文

【摘要】： 近年来,随着中国手机用户数的增长,手机短消息业务得到了迅猛发展,但同时垃圾短信也以非常快的速度在增长。目前,垃圾短信过滤技术主要有黑名单过滤、关键词过滤和基于文本分类的内容过滤。黑名单过滤和关键词过滤能快速的过滤垃圾短信,但其过滤的准确率不高;基于文本分类的短信过滤的准确率主要依赖于其训练样本的数量及质量,由于日常短信涉及个人的私隐,短信中心从不对外公布,现有的短信样本很少,因此单纯依靠文本分类的短信过滤其准确率也不高。同时,现有的过滤技术都采用逐条检测的方式,对于每天有大量短信息的短信中心来说,其运算量是很大的,这样会造成短信服务中心网络堵塞。因此,从现有的技术上来说,垃圾短信的过滤在准确率和效率方面仍然不能满足现实需要。本文将针对现有短信过滤技术的不足,研究有效的解决或改进的方法,同时引入新的技术与原有的过滤技术相结合,使垃圾短信过滤系统能够具有更好的过滤性能。本文所做的主要工作如下: 1.研究了应用在垃圾短信过滤系统中的关键词查找算法,该算法采用WM算法的多模式匹配思想,通过预处理建立的哈希表来加快匹配速度,并且根据垃圾短信的特征,提出了用压缩存储的TRIE树来组织模式串的方法,加快了查找速度。 2.分析了主要的文本分类技术,重点叙述基于最小风险的贝叶斯分类在短信过滤中的应用。 3.提出用日志分析的方法来分析已经过滤的短信,提取有用的数据,更新关键词库及分类训练样本,实现系统的自我优化。 4.将贝叶斯分类与新的过滤方法(包括流量检测、抽样检测、日志分析)结合,在保证短信过滤准确率的情况下,提高短信过滤效率。最后,给出整个垃圾短信过滤系统的设计与实现。
[Abstract]:In recent years, with the increase of the number of mobile phone users in China, the short message service of mobile phone has been developing rapidly, but at the same time, the spam message is also growing at a very fast speed. At present, spam filtering technology mainly includes black list filtering, keyword filtering and text classification based internal volume filtering. Blacklist filtering and keyword filtering can be fast. Spam messages are filtered, but the accuracy of filtering is not high; the accuracy of text filtering based on text categorization is mainly dependent on the number and quality of the training samples. As the daily short message involves personal privacy, the SMS center is never published and the existing SMS samples are very few, so the text simply relies on text categorization to filter its accuracy. The rate is not high. At the same time, the existing filtering technology uses one by one detection method. For short message centers with a large number of short messages every day, the amount of operation is very large, which will cause the congestion of the SMS service center network. Therefore, from the existing technology, the filtering of spam messages is still not satisfied with the accuracy and efficiency. You need it.
This article will aim at the shortage of the existing SMS filtering technology, and study the effective solution or improvement method, and combine the new technology with the original filtering technology, so that the spam message filtering system can have better filtering performance. The main work done in this paper is as follows:
1. study the keyword search algorithm used in the spam message filtering system. The algorithm uses the multi pattern matching idea of WM algorithm to speed up the matching speed by preprocessing the hash table. And according to the characteristics of the spam message, a method of organizing the pattern string with the compressed storage TRIE tree is proposed to speed up the search speed.
2., the main text categorization techniques are analyzed, and the application of Bias classification based on minimum risk in short message filtering is emphasized.
3. the method of log analysis is used to analyze the filtered SMS, extract useful data, update the key word library and classify training samples, and realize the self optimization of the system.
4. combining the Bias classification with the new filtering methods (including flow detection, sampling detection and log analysis), the efficiency of SMS filtering is improved under the condition of ensuring the accuracy of SMS filtering. Finally, the design and implementation of the whole spam message filtering system are given.
【学位授予单位】：电子科技大学
【学位级别】：硕士
【学位授予年份】：2009
【分类号】：TP311.52

【相似文献】