基于Nutch的移动WEB搜索系统的研究与实现

发布时间：2018-04-01 02:06

本文选题：移动WEB　切入点：搜索引擎　出处：《北京邮电大学》2013年硕士论文

【摘要】：随着3G时代的到来,移动电话,便携计算机等移动设备的普及,越来越多的用户使用移动终端就能够便捷的访问网络。这样用户对于个性化和智能化搜索引擎的需求更加明显。现有的移动终端的搜索引擎,大都是直接把本地搜索引擎转移到移动终端。这些移动搜索引擎仅仅利用纯粹的文本相关度进行搜索,甚至把用户输入的位置信息也当做普通的文本关键字,并没有很好的和用户地理位置等移动空间信息结合起来,而人们在使用移动设备搜索时大多数需求都与空间位置密切相关。移动用户进行搜索查询时,一般希望搜索引擎不仅可以提供与查询内容密切相关的网页,而且可以提供与用户所在位置空间距离相近的网页。因此,现有的移动搜索引擎很难使用户获得理想的查询结果。本文针对移动搜索引擎所面临的问题入手,研究同时基于文本相关性搜索和地理位置相近性搜索的解决方案,提出了一个基于Nutch的移动WEB搜索系统的实现方案,搭建了一个基于位置和关键字双重搜索的移动WEB搜索系统,实现了位置相关的空间搜索。根据网页所描述内容的地理位置信息对网页进行地理标记,该方案可以搜索到与用户所在位置相关的网页,可以用于解决移动用户搜索附近相关性结果的难题。通过使用Lucene和R-tree的混合索引,系统实现了对搜索排序结果的有效优化,验证了混合索引结构能够更快速的为用户提供综合文本相关和距离相近性的结果。本文阐述了整套系统的整体框架结构设计和各个主要模块的实现细节,详细介绍了网页预处理模块,索引建立模块和搜索模块的各个关键技术,包括对网页进行地理标记,基于文本聚类的混合索引插入算法,以及节点优先队列的搜索算法。最后,在功能方面和性能方面对系统进行验证测试。测试结果表明,移动WEB搜索系统具备了综合地理位置和文本信息的双重搜索功能,并具备较好的性能。
[Abstract]:With the arrival of the 3G era, the popularity of mobile phones, portable computers and other mobile devices, more and more users use mobile terminal can easily access the network. So the user for intelligent and personalized search engine needs more obvious. The mobile terminal of the existing search engines, are directly transferred from the local search engine mobile terminal. These mobile search engines use only pure text of the search, and even the location information of the user input as a normal text keyword, and not a very good geographical position and user mobile spatial information together, and people in the use of mobile devices to search the most requirements are closely related to the spatial position. Mobile users search query, search engines can not only provide general hope is closely related with the query content, and can provide It is very difficult for the existing mobile search engine to get the desired result of the query by the existing mobile search engine.
According to the mobile search engine facing problems, at the same time based on text search and Study on correlation between geographical proximity search solutions, proposed a mobile WEB based on Nutch search system, set up a mobile search system WEB position and double keyword search based implementation of space position the search. According to Geotag Web Geographic Information Page describes the content of the program can search and user location of related web pages can be used to solve the problem of mobile user search results. By mixing near correlation index using Lucene and R-tree, the system realizes the effective optimization of search results. Verify the hybrid index structure can more quickly provide users with comprehensive text and distance similar results.
This paper describes the implementation details of main module structure of the overall framework of the whole system design and each of the details of the web page preprocessing module, each key indexing module and search module, including Geotag ", mixed text clustering index insertion algorithm based on search algorithm and node priority queue. Finally, to test the system in function and performance. The test results show that the WEB mobile search system with the dual search function comprehensive geographic location and text information, and have a better performance.

【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP391.3;TN929.5

【参考文献】