云环境下的舆情监测关键技术研究

发布时间：2018-06-07 19:02

本文选题：舆情监测 + 分布式爬虫　；参考：《西南科技大学》2017年硕士论文

【摘要】：为满足高校、医院等特殊行业领域单位的网络舆情需求,解决网络舆情工作中针对性不足、成本高、实时性低的问题,本文设计实现一套针对特殊行业领域的云环境下的舆情监测系统。主要工作包括:通过分析监测源的信息排列特征,利用分布式网络爬虫技术实现了舆情信息的快速采集系统,能够快速的爬取指定监测源的相关页面,能在5分钟的延时范围内采集到特定的网页信息,提高了数据的采集速率。利用基于HTML模版的文本抽取方法,实现页面文本数据的精确快速抽取。利用全文检索技术构建了全文检索引擎服务,并在此基础上实现了10分钟内发现舆情,平均准确率达到95%以上,平均召回率到70%以上的舆情发现效果。根据用户的实际需求,设计实现了云环境下的舆情监测系统,提供了舆情监测与管理功能。本文集成数据采集系统、舆情发现系统、舆情监测管理系统三个子系统形成云环境下的网络舆情监测系统,为高校、医院等非政府机构提供云环境下的在线舆情监测服务,实时性强。
[Abstract]:In order to meet the demand of network public opinion in universities, hospitals and other special industry units, solve the problems of lack of pertinence, high cost and low real-time performance in the work of network public opinion. In this paper, we design and implement a monitoring system of public opinion in cloud environment. The main work includes: by analyzing the information arrangement characteristics of the monitoring source and using distributed web crawler technology to realize the rapid collection system of public opinion information can quickly crawl the relevant pages of the designated monitoring source. It can collect the specific web page information in 5 minutes delay range, and improve the data acquisition rate. The accurate and fast extraction of page text data is realized by using HTML template based text extraction method. The full-text search engine service is constructed by using full-text retrieval technology, and the public opinion discovery effect is realized within 10 minutes. The average accuracy is more than 95%, and the average recall rate is more than 70%. According to the actual needs of users, the monitoring system of public opinion under cloud environment is designed and implemented, and the function of monitoring and managing public opinion is provided. This paper integrates the three subsystems of data acquisition system, public opinion discovery system and public opinion monitoring management system to form a network public opinion monitoring system in cloud environment, which provides online public opinion monitoring services for universities, hospitals and other non-governmental organizations in the cloud environment. Real-time ability.
【学位授予单位】：西南科技大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.1

【相似文献】