智能新闻采集处理系统的设计与实现

发布时间：2018-04-01 20:38

本文选题：新闻采集　切入点：数据抓取　出处：《山东大学》2017年硕士论文

【摘要】：近年来,快速发展的移动互联网技术为新闻传播提供了更加丰富多彩的传播环境。大量的传统媒体开始逐渐的向基于互联网的新媒体转变,将其新闻传播方式改成网络新闻传播。随着传播方式的改变,原有的新闻采集方式以及不能满足网络新闻传播的需求,为此新闻采集方式也需要随新闻传播方式一块更新换代,变为从互联网信息源中的大量新闻信息提取出新闻线索重新加以编辑加工。基于以上背景,本文设计并实现了一个智能新闻采集处理系统,以协助新闻工作者在互联网信息源中提取有效的新闻信息。该系统根据不同新闻网站的更新规律智能调节采集频率,不断地采集新闻工作者指定的网站、微博、贴吧中的信息,并将信息加以分析以及分类,然后将信息回馈给不同的新闻工作者当做新闻素材,让新闻工作者根据这些信息编辑加工出新闻文章发布到网络中。论文综合智能新闻采集处理系统的的开发特点,围绕软件工程的核心理念,采MVC模式对系统进行设计。本论文从需求定位、概要分析、详细设计和实现、系统测试等方面对智能新闻采集处理系统的做了全面叙述。系统采用B/S架构格式,开发平台选用MyEclipse。在数据存储过程中,服务器端的采用oracle数据库作为数据存储工具。本文的智能新闻采集处理系统包含前台MIS部分和后台服务部分两个部分。前台MIS部分的负责对要采集的新闻、微博、贴吧信息进行管理以及采集结果查询并且支持用户自定义新闻标签以及标签的匹配规则。后台服务部分负责根据配置的采集规则进行数据采集,并对采集到的新闻进行自动分类,并根据采集数量智能调节采集频率。
[Abstract]:In recent years, the rapid development of mobile Internet technology has provided a more colorful communication environment for news dissemination. A large number of traditional media began to gradually change to new media based on the Internet. With the change of communication mode, the original news collection method and the need of network news dissemination cannot be satisfied. Therefore, news collection mode also needs to be updated with the news communication mode. This paper designs and implements an intelligent news collection and processing system based on the above background. In order to assist journalists to extract effective news information from Internet information sources, the system intelligently adjusts the collection frequency according to the updating rules of different news websites, continuously collects information from websites designated by journalists, Weibo, and posts. And analyze and sort the information, and then give it back to different journalists as news material, According to these information, journalists can edit and process news articles to be published on the network. The paper synthesizes the development characteristics of intelligent news collection and processing system, and revolves around the core idea of software engineering. This paper describes the intelligent news collection and processing system from the aspects of requirement orientation, summary analysis, detailed design and implementation, system test and so on. The system adopts B / S structure format. The development platform is MyEclipse. In the data storage process, The oracle database is used as the data storage tool in the server. The intelligent news collection and processing system in this paper includes two parts: the front MIS part and the background service part. The front MIS part is responsible for the news to be collected, Weibo, Post information management and collection results query and support for user-defined news labels and label matching rules. The background service is responsible for data collection according to the configuration of the collection rules. The collected news is classified automatically and the frequency is adjusted intelligently according to the quantity collected.
【学位授予单位】：山东大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP311.52

【相似文献】