基于Nutch的图情博客搜索引擎的设计与实现

发布时间：2018-05-12 15:59

本文选题：Lucene + Nutch　；参考：《郑州大学》2011年硕士论文

【摘要】：随着Web2.0理念和技术的兴起,全球互联网用户享受到了丰富多样的交互性信息服务,博客正是这种交互性信息服务的典型代表。在这种时代背景下,图书情报学领域的学生、研究人员等纷纷开设博客进行信息交流。然而,图情博客分散、博文质量参差不齐的现状给界内人士的使用带来了不便,虽然Google博客搜索、百度博客搜索等相关的主题搜索引擎已经解决了一些问题,但是仍旧不能满足图情界用户的需求。本文就是针对这一问题尝试构建图情博客搜索引擎,以满足图情用户的需求。本文首先对搜索引擎相关技术和图情博客进行分析,其次引入Nutch这一开源搜索引擎并基于Nutch制定了图情博客搜索引擎的设计方案,再次依据该设计方案开发出相应的主题搜索引擎,最后以实验的方法对该主题搜索引擎的性能进行评估。本文各章节的主要内容如下： 1绪论。本章介绍了论文的选题背景、选题意义、国内外研究现状、所采用的研究方法和创新之处。 2搜索引擎相关技术及图情博客分析。本章首先分析了搜索引擎、主题搜索引擎的运行原理,指出二者的主要区别在于信息采集模块和网页内容解析模块两部分,主题搜索引擎改进了网络爬虫模块和并在网页内容解析模块增加了主题词库进行信息过滤。其次从博客站点结构、博客页面内容、博客之间链接结构三方面对图情博客进行分析,以增加对图情博客的全面认识。 3 Nutch简介及Nutch系统的运行环境配置与运行。本章首先介绍了开源搜索引擎Nutch的基本情况和框架结构,对Nutch有个初步的认识。然后配置Nutch系统的运行环境并详细阐释其运行流程,对Nutch的运行原理和详细结构有进一步的认识。 4基于Nutch的图情博客搜索引擎的设计。本章依据软件工程的思想,首先分析搜索引擎系统要实现的目标、要解决的问题以及可行性,然后通过用例图(Use Case Diagram)和序列图(Sequence Diagram)对系统的用户需求进行阐述,最后给出了系统的总体设计方案和详细设计方案。 5基于Nutch的图情博客搜索引擎的核心模块实现。本章对详细设计方案中的三个核心模块进行实现。首先是借助图书情报学的信息检索理论和实践对主题资源发现模块进行实现,其次是通过软件分析对爬虫模块的采集策略进行实现,最后是根据用户需求对检索模块进行改进。 6实验测试分析与结论。本章首先设定了一系列参数并依据这些参数进行了六轮实验测试,然后对测试结果进行了分析。最后,笔者总结了图情博客搜索引擎的特点及不足之处,并对以后的改进工作做出了展望。
[Abstract]:With the rise of Web2.0 concept and technology, Internet users worldwide enjoy a variety of interactive information services, blog is the typical representative of this interactive information service. Against this background, students and researchers in the field of library and information science have started blogs to exchange information. However, the scattered picture blog, the uneven quality of blog has brought inconvenience to the use of people in the field, although Google blog search, Baidu blog search and other related theme search engines have solved some problems. But still can not meet the needs of users. This paper attempts to build a blog search engine to meet the needs of users. In this paper, we first analyze the related technologies of search engine and map blog, then introduce Nutch, an open source search engine, and work out the design scheme of map blog search engine based on Nutch. At last, the performance of the theme search engine is evaluated by the experimental method. The main contents of each chapter are as follows: 1 introduction. This chapter introduces the background, significance, domestic and international research status, research methods and innovations. 2 search engine related technology and map blog analysis. This chapter first analyzes the operation principle of search engine and theme search engine, and points out that the main difference between them lies in two parts: information collection module and web page content analysis module. The topic search engine improves the web crawler module and adds the topic thesaurus to the web content parsing module for information filtering. Secondly, from the blog site structure, blog page content, blog links between the three aspects of the blog analysis, in order to increase the overall understanding of the picture blog. Introduction of Nutch and configuration and operation of Nutch system. This chapter first introduces the basic situation and framework of open source search engine Nutch, and has a preliminary understanding of Nutch. Then configure the running environment of Nutch system and explain its running flow in detail, and have a further understanding of the operation principle and detailed structure of Nutch. 4 the design of blog search engine based on Nutch. According to the idea of software engineering, this chapter first analyzes the goal, the problem and the feasibility of the search engine system, and then through use Case Diagrams and sequence Diagrams, the user needs of the system are expounded. Finally, the overall design scheme and detailed design scheme of the system are given. 5 the core module of blog search engine based on Nutch. In this chapter, three core modules in the detailed design scheme are implemented. Firstly, it implements the topic resource discovery module with the help of the information retrieval theory and practice of library and information science; secondly, it implements the crawler module's acquisition strategy through software analysis; finally, it improves the retrieval module according to the user's demand. 6 Experimental analysis and conclusion. In this chapter, a series of parameters are set up and six rounds of experimental tests are carried out according to these parameters, and then the test results are analyzed. Finally, the author summarizes the features and shortcomings of blog search engine, and makes a prospect for future improvement.
【学位授予单位】：郑州大学
【学位级别】：硕士
【学位授予年份】：2011
【分类号】：G250.73

【参考文献】