三网融合下实时文本新闻的选择与分类技术研究

发布时间：2019-01-12 13:06

【摘要】：近年来随着科技与社会的发展,三网融合成为大势所趋。然而,三网融合在带来更全面更快捷更广泛服务的同时,也带来了巨大的安全隐患。互联网的特点是实时便捷全面,然而信息量大信息质量参差不齐。因此,三网融合后对网上信息内容的监管成为三网融合需要解决的重要问题。新闻是网络上信息传递的主要载体,对网络新闻的监管是内容监管的重要组成部分。文本分类与聚类技术是文本挖掘领域的重要技术,分类后的新闻易于管理与鉴别。分类在一定程度上解决了信息杂乱的问题,是信息过滤目标营销性能预测和医疗诊断等领域的基础,因此对文本分类的研究具有重要意义。HUSTRIM系统是三网融合环境下的内容监管系统。通过对内容的安全监控和管理为全网内容可监可管提供保证,其新闻选择分类为三网融合下新闻内容的监管提供支持。HUSTRIM采用网络爬虫新闻链接分析基于行块分布函数的正文抽取等技术,获取有效的新闻正文信息;采用朴素bayes与k-means相结合的方法对新闻进行先分类再聚类。HUSTRIM新闻选择分类包括新闻获取模块正文提取模块朴素bayes分类模块和k-means聚类模块。从互联网爬取了近700个网页新闻信息,借助搜狗实验室文本分类语料库对这700个网页新闻进行了分类和聚类。通过实验得到了正文抽取过程的最佳阀值朴素bayes分类的最佳特征属性维数和k-means聚类的最佳k值。
[Abstract]:In recent years, with the development of science and technology and society, the integration of three networks has become the trend of the times. However, triangulation is leading to a more comprehensive approach? Faster? More extensive service at the same time, but also brought huge security risks. The Internet is characterized by real-time? Convenient? Comprehensive, but large amount of information? The quality of information is uneven. Therefore, the supervision of the online information content becomes an important problem to be solved after the integration of three networks. News is the main carrier of information transmission on the network, and the supervision of network news is an important part of content supervision. Text classification and clustering technology is an important technology in the field of text mining, the classified news is easy to manage and identify. Classification solves the problem of information clutter to a certain extent, is information filtering? Target marketing? Because of the foundation of performance prediction and medical diagnosis, the research on text classification is of great significance. HUSTRIM system is a content monitoring system under the environment of three-network convergence. Through the security monitoring and management of the content for the entire web content can be monitored and managed to provide assurance, its news selection classification for the three-network integration of the supervision of news content to provide support. HUSTRIM using web crawlers? News link analysis? Based on line block distribution function of text extraction technology to obtain effective news text information; using the combination of simple bayes and k-means to classify news first and then clustering. HUSTRIM news selection classification includes news acquisition module. Text extraction module? Naive bayes classification module and k-means clustering module. Nearly 700 web pages were crawled from the Internet and classified and clustered with Sogou's laboratory text classification corpus. The optimal threshold of the text extraction process is obtained by experiments. The best feature attribute dimension of naive bayes classification and the best k value of k-means clustering.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：TP391.1

【相似文献】