基于查询日志的查询扩展算法研究与实现

发布时间：2018-05-30 15:35

本文选题：搜索引擎 + 查询扩展　；参考：《华东理工大学》2013年硕士论文

【摘要】：随着互联网技术的快速发展,Internet上的信息也呈现爆炸式的增长；与此同时,网民数也在不断的增长,如何让用户在海量的信息中找到自己所需要的信息已成为信息检索领域一个非常重要的研究课题。一方面,中文查询词语的平均长度相比英文查询词更短,从而中文搜索引擎从用户获得的信息就更少；另一方面,中文中的同义词、多义词导致了查询中的词语和文档中词语之间的差异,使大多数基于关键词的中文搜索引擎的查询结果出现差错。因而,目前的很多中文搜索引擎不能很好地满足用户的需求。基于这些问题,查询扩展技术应运而生。本文提出了一种基于查询日志的局部共现查询扩展算法(LCQEBQL:Local Co-occu-rrence Based on Query Log),该算法首先在获取相关用户文档集合时,使用了改进的编辑距离向量算法和用户行为信息,使该集合相关性更高；其次,在对文档或相关查询词集合进行分词时,加入了命名实体库,避免了将命名实体词拆分成多个无意义的词语,使分词更精确；再者,在过滤相关用户文档集合时,同时考虑了三个方面的因素(空链接、导航页、文档和查询词集合的相似度),排除了不相关的用户文档,提高了算法的性能；然后,在计算用户文档词项与相关查询词集合中词项的相似度时,使用了局部共现的分析方法,同时考虑了查询日志中URL链接的权重和HTML文档结构的位置信息,提高了算法的性能；最后,在重新计算扩展词的权值时,加入了相关查询词信息,使得扩展词权值更精确。本文实验部分利用搜狗日志中的URL链接提取了1000篇不同领域的网页,过滤后保存作为实验的测试集,并设计了原型系统对本文的方法和其他的方法进行了实验评估。实验表明,LCQEBQL目比其他算法更有效,检索结果更相关。
[Abstract]:With the rapid development of Internet technology and the explosive growth of information on the Internet, at the same time, the number of Internet users is also growing. How to make users find the information they need in mass information has become a very important research topic in the field of information retrieval. On the one hand, the average length of Chinese query words is shorter than that of English query words, thus the Chinese search engine gets less information from users; on the other hand, Chinese synonyms, Polysemous words lead to the difference between the words in the query and the words in the document, and make the query results of most keyword based Chinese search engines go wrong. Therefore, many Chinese search engines can not meet the needs of users. Based on these problems, query expansion technology emerged as the times require. In this paper, a local co-occurrence query expansion algorithm based on query log is proposed. In this algorithm, the improved Editing distance Vector algorithm and user behavior information are used to obtain the collection of relevant user documents. It makes the set more relevant. Secondly, the named entity library is added to the document or related query word set to avoid splitting the named entity word into several meaningless words, which makes the segmentation more accurate. When filtering related user document sets, three factors (empty link, navigation page, document and query word set) are considered simultaneously, which eliminate irrelevant user documents and improve the performance of the algorithm. In order to improve the performance of the algorithm, a local co-occurrence analysis method is used to calculate the similarity between the terms in the user document and the related query words. The weight of the URL link in the query log and the location information of the HTML document structure are taken into account at the same time. Finally, when we recalculate the weights of extended words, we add the information of query words to make the weights of extended words more accurate. In the experiment part, 1000 web pages in different fields are extracted by using URL links in Sogou logs, and then filtered and saved as the test set of experiments. A prototype system is designed to evaluate the methods of this paper and other methods. Experiments show that LCQEBQL is more effective than other algorithms and the retrieval results are more relevant.
【学位授予单位】：华东理工大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP391.3

【参考文献】