基于用户日志分析的搜索引擎排序算法的设计与实现

发布时间：2018-11-09 18:21

【摘要】：互联网的快速发展使得如何从海量信息中查找有效的数据变得越来越重要,搜索引擎通过对网络中的信息进行爬取和组织,为用户提供了优质的查询服务接口,它的出现使得目标信息的获取变的更为便捷。搜索引擎已经成为互联网用户获取网络资源所必不可少的工具,但由于互联网信息量巨大,搜索引擎也无法每次都返回让人满意的查询结果：首先,当用户输入一个查询,搜索引擎会返回数量众多的相关结果,而用户最为关心的结果却没能显示在最前面或最显眼的位置；其次,由于用户的对搜索引擎了解程度不同,大部分用户不能通过检索请求十分准确的表达检索想法,从而导致搜索结果的不准确性。因此,通过用户的搜索行为了解用户的查询意图对于改善搜索引擎结果排序的准确性具有重要意义。本文通过对搜索引擎查询日志进行统计分析,由大量用户的行为找出用户访问的一般性规律,进而优化网页排序算法,引导最终的结果排名,提高搜索引擎的结果排序的准确性。论文工作主要包括两个方面： (1)分析搜索引擎用户查询日志。研究搜索日志的各个特征及其相互之间的联系,总结中文搜索引擎用户的一些基本行为规律,并根据对不同时期搜索日志的分析,发现中文搜索引擎用户的搜索行为变化趋势,为以后的搜索引擎用户行为分析提供了基础。 (2)对Lucene的原始排序算法的进行优化。原始算法是基于向量空间模型的TF-IDF算法,针对该算法只重视关键词词频和文档的匹配度,而不考虑网页自身的特点,设计了一种基于词频匹配度和网页自身特点相结合的网页排名算法。根据大量用户查询行为日志研究出的用户搜索行为趋势,在原有的排序算法基础上增加用户认可度排序因子,可以根据搜索引擎的需要调整该因子的权重系数使网页排序最优化。这样可以在既保证了搜索结果的相关性和匹配度的前提下,同时使搜索返回结果的排序更加符合用户的搜索需求。本文设计的搜索引擎系统通过boost因子对排序算法做出改进,通过对原始排序算法以及结合用户反馈信息的优化后排序算法的返回结果进行了实验对比分析,验证了优化后排序算法对于查询返回结果的排列顺序的改善效果,为未来在搜索引擎用户查询意图方面的研究提供了参考。
[Abstract]:With the rapid development of the Internet, how to find effective data from mass information becomes more and more important. Search engine provides users with high-quality query service interface by crawling and organizing the information in the network. Its appearance makes the acquisition of target information more convenient. Search engine has become an indispensable tool for Internet users to access network resources, but because of the huge amount of information on the Internet, search engines can not return satisfactory results every time: first, when users enter a query, The search engine will return a large number of related results, while the results most concerned by the user are not displayed in the front or most prominent position; Secondly, because the users have different understanding of the search engine, most users can not express the retrieval idea accurately through the retrieval request, which leads to the inaccuracy of the search results. Therefore, it is important to understand the user's intention through the search behavior to improve the accuracy of search engine results ranking. Based on the statistical analysis of search engine query log, this paper finds out the general rules of user access by the behavior of a large number of users, and then optimizes the sorting algorithm of web pages to guide the final result ranking. Improve the accuracy of search engine results sorting. This paper mainly includes two aspects: (1) analyzing search engine user query log. This paper studies the characteristics of search logs and their relationships, summarizes some basic behavior rules of Chinese search engine users, and finds out the changing trend of search behavior of Chinese search engine users according to the analysis of search logs in different periods. It provides the foundation for user behavior analysis of search engine in the future. (2) optimize the original sorting algorithm of Lucene. The original algorithm is a TF-IDF algorithm based on vector space model. The algorithm only pays attention to the frequency of keywords and the matching degree of documents, and does not consider the characteristics of web pages. A web page ranking algorithm based on word frequency matching and web page characteristics is designed. According to a large number of user query behavior logs, the user search behavior trend is studied, and the sorting factor of user recognition is added to the original sorting algorithm. According to the need of search engine, the weight coefficient of this factor can be adjusted to optimize the ranking of web pages. This can not only guarantee the correlation and matching degree of search results, but also make the ranking of the returned results more in line with the users' needs. The search engine system designed in this paper improves the sorting algorithm by boost factor, and makes a comparative analysis of the results of the original sorting algorithm and the optimized post-sorting algorithm combined with user feedback information. The results show that the optimized post-sorting algorithm can improve the order of query return results and provide a reference for future research on search engine users' query intention.
【学位授予单位】：武汉理工大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP391.3

【参考文献】