一个垂直搜索系统的设计与实现

发布时间：2018-04-25 20:30

本文选题：垂直搜索 + 主题爬虫　；参考：《中山大学》2012年硕士论文

【摘要】：当前互联网中传统的通用搜索引擎服务商提供给用户搜索水平的海量信息，它的优点就是能搜索到比较全面的信息，但是由于范围过广，很难兼顾搜索的准确度，而且当用户需要某一领域行业信息时，通用搜索引擎更不能较好地满足要求，这时，就可以借助面向领域的垂直搜索引擎，该类搜索引擎基于某一特定领域或者行业，，对其中的信息进行深度加工，提供给用户更加准确的信息。本文以人们对当前流行的平板电脑搜索需求为背景，研究和分析了垂直搜索引擎的关键技术，设计并实现一个平板电脑领域的垂直搜索系统。文章首先分析了垂直搜索引擎中的主题爬虫、信息抽取以及全文检索等核心技术，尤其是索引技术中的倒排索引以及Lucene开源全文检索工具包。然后重点分析中文分词这另一关键技术，包括中文分词的常用方法和算法。基于字符串匹配分词这一方法，在建立了平板电脑领域的基本词典之后，采用基于前缀的逐字最大匹配算法，最终设计和实现了适合平板电脑领域的中文自动分词组件，并实现了Lucene分词器接口。将其与其他一些开源的分词系统相比较的结果表明，在该领域内，该中文分词组件的切分词准确度较好。基于这些关键理论和技术，本文先对待实现的系统进行了总体设计，包括功能模块划分、采用的架构、开发技术和环境。最后是系统的详细设计和实现，采用UML设计分析技术以及J2EE三层架构思想，较为详细地论述了利用Lucene来构建垂直搜索系统的整个设计和实现过程。通过本系统与传统搜索引擎在平板电脑产品上的搜索进行比较，可以看出本系统在搜索结果的查准度上具有一定的直观优势。
[Abstract]:At present, the traditional universal search engine service provider in the Internet provides users with a large amount of information at the level of search. Its advantage is that it can search more comprehensive information, but because of its wide scope, it is difficult to take into account the accuracy of the search. And when users need industry information in a certain field, the general search engine can not meet the requirements better. In this case, we can use the vertical search engine of the domain, which is based on a specific field or industry. The information is further processed to provide users with more accurate information. In this paper, the key technology of vertical search engine is studied and analyzed, and a vertical search system in the field of tablet computer is designed and implemented under the background of people's demand for popular tablet computer search. This paper first analyzes the core technologies of vertical search engine, such as topic crawler, information extraction and full-text retrieval, especially inverted index and Lucene open source full-text retrieval toolkit. Then it analyzes the other key technology of Chinese word segmentation, including the common methods and algorithms of Chinese word segmentation. Based on the method of string matching word segmentation, after establishing the basic dictionary of tablet computer field, the Chinese automatic word segmentation component suitable for tablet computer domain is designed and implemented by using the word for word maximum matching algorithm based on prefix. The interface of Lucene word Segmentation is realized. Compared with other open source word segmentation systems, the results show that the segmentation accuracy of the Chinese word segmentation component is good in this field. Based on these key theories and technologies, this paper first introduces the overall design of the system, including functional module partition, architecture, development technology and environment. Finally, the system is designed and implemented in detail. The whole design and implementation process of vertical search system based on Lucene is discussed in detail by using UML design and analysis technology and J2EE three-tier architecture. Through the comparison between this system and the traditional search engine on the tablet computer products, we can see that this system has some intuitive advantages in the search result checking degree.
【学位授予单位】：中山大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP391.3

【参考文献】