基于在线排序逻辑回归的垃圾邮件过滤

发布时间：2018-09-14 15:08

【摘要】：垃圾邮件过滤是网络信息处理中的重要问题,基于机器学习方法的垃圾邮件过滤技术是目前的研究热点。现有研究一般将过滤问题视为二值分类问题进行解决,存在着模型优化目标和性能评价指标1-AUC不一致的问题,导致模型优化结果产生偏差,过滤性能受到很大影响。该文通过直接优化评价指标1-AUC来提升过滤器性能,将垃圾邮件过滤问题转化成排序问题进行建模,提出了在线排序逻辑回归学习算法,解决了在线学习中的邮件得分偏移问题;综合应用TONE算法和重采样技术,提出参数权重更新算法,解决模型学习中在线调整模型参数时的处理速度问题,满足垃圾邮件实时过滤的要求。在垃圾邮件过滤公开评测数据集上的实验结果表明,基于在线排序逻辑回归模型的过滤结果全面优于在线逻辑回归模型的过滤结果。
[Abstract]:Spam filtering is an important problem in network information processing. The current research generally regards the filtering problem as a binary classification problem, and there exists the problem that the model optimization goal is inconsistent with the performance evaluation index (1-AUC), which results in the deviation of the model optimization results and greatly affects the filtering performance. In this paper, the filter performance is improved by directly optimizing the evaluation index 1-AUC, and the spam filtering problem is transformed into a sort problem for modeling. An online sorting logical regression learning algorithm is proposed to solve the problem of mail score offset in online learning. By using TONE algorithm and resampling technology, a parameter weight updating algorithm is proposed to solve the problem of processing speed when model parameters are adjusted online in model learning, and meet the requirements of spam real-time filtering. The experimental results on the public evaluation data set of spam filtering show that the filtering result based on the online sorting logical regression model is better than that of the online logical regression model.
【作者单位】：哈尔滨理工大学计算机科学与技术学院;黑龙江工程学院计算机科学与技术学院;
【基金】：国家自然科学基金资助项目(60903083) 黑龙江省新世纪人才项目(1155-ncet-008) 教育部博士点新教师基金资助项目(20092303120005)
【分类号】：TP393.098

【相似文献】