政策信息垂直搜索引擎研究与实现

发布时间：2018-03-03 22:11

本文选题：垂直搜索引擎　切入点：政策信息　出处：《河北经贸大学》2017年硕士论文　论文类型：学位论文

【摘要】：随着信息时代的深入人心与互联网技术的飞跃式发展,人们已经把从网络信息的海洋中搜寻出对自己有用的信息作为最主要的信息获取途径,互联网中的搜索服务在人们日常工作和生活中发挥着无可取代的作用。人们利用百度、谷歌、雅虎、必应等搜索引擎在互联网中寻找信息,这类通用搜索引擎已经极大地缩短了我们定位有用信息的时间,但对于搜索特定主题的需求来说,通用搜索引擎返回的结果往往无序、分散。而垂直搜索引擎旨在为用户提供特定主题的相关服务与信息,满足用户特定需求。为了制定造福社会和服务大众的政策文件,决策者不仅要把握社会需求,而且需要遍览当前已出台文件为制定新政策提供依据。因此,面向政策信息的垂直搜索引擎能够够满足政策制定者的需要。本文以我国人社领域各级政府部门的官方网站作为实验数据,在对垂直搜索引擎关键技术进行深入研究的基础上,结合用户需求,完成了一个面向政策信息的垂直搜索引擎。本文详细阐述了政策信息垂直搜索引擎的相关理论基础和研究过程。首先,本文介绍了完成该垂直搜索引擎所利用的关键技术即主题爬虫技术、网页信息抽取技术、语句相似度比较、全文检索技术等;然后在开源网络爬虫框架WebMagic的基础上,针对某一类网站集合设计了专用的解析器;本文还提出了基于语义相似度的政策树生长模型;最后,本文详细阐述了政策信息垂直搜索引擎的实现,并进行了系统测试。
[Abstract]:With the leap forward development of the information age and the popularity of Internet technology, people have put the network information from the sea to search for useful information on their own as the main access to information, Internet search services in people's daily work and life plays an irreplaceable role. People use Baidu, Google, YAHOO, Bing and other search engines to find information on the Internet, the general search engine has greatly shortened the time we locate the useful information, but for searching topic specific needs, return to the general search engine often results in disorder, scattered. The vertical search engine to provide specific topics for user information and related services and meet the specific needs of the user. In order to make the benefit of the community and public service policy documents, policy makers should not only grasp the needs of society, but also need to see The current has issued a document to provide the basis for the development of the new policy. Therefore, the vertical search engine oriented policy information to meet the needs of policy makers. Based on China's social field of government departments at all levels on the official website as the experimental data, based on the in-depth study of key technology of vertical search engine, combined with the needs of users, completed a policy oriented information vertical search engine. This paper describes the policy information vertical search engine related theory and research process. Firstly, this paper introduces the vertical search engine is the key technology that the theme crawler technology, web information extraction technology, similarity comparison, full-text retrieval technology; then based on the open source web crawler framework WebMagic, according to a website set is designed with a special parser; this paper also put forward the base of Yu Yuyi In the end, the implementation of the policy information vertical search engine is described in detail, and the system test is carried out.

【学位授予单位】：河北经贸大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.3

【相似文献】