查询理解与正负双向相关反馈技术研究

发布时间：2018-03-08 14:23

本文选题：信息检索　切入点：查询理解　出处：《大连理工大学》2016年博士论文　论文类型：学位论文

【摘要】：广泛迅捷的分享和交换信息是互联网最重要的优点之一,然而随着互联网中承载的数据量和信息量呈指数级爆炸式增长,导致人们必须面对日益严重的信息过载问题。在该背景下信息检索技术应运而生并随着互联网一起迅猛的发展,成为当前解决信息过载问题最直接和有效的手段。同时查询理解和相关反馈技术是近几年被广泛验证的用于改善信息检索性能的有效途径,尽管现有研究已经取得了一些重要进展,但是仍然有很多关键的问题没有得到很好的解决。针对现有研究中存在的种种不足,本文在分析当前各查询理解和相关反馈算法基础上,重点对查询理解技术以及以其为基础的相关反馈技术进行了深入的研究。本文的主要研究工作和贡献如下：1.针对查询理解技术中的词项权重预测问题,本文将其转换为序列标注问题,提出了一种新的基于循环神经网络的查询词项权重学习模型。该模型通过综合考虑查询中各词项的统计、语法、语义以及词项间关系信息构造查询词项特征向量,同时利用遗传算法结合真实文档相关性标注得到最优词项权重值,最后利用双向循环神经网络对查询词项序列与相应最优权重序列之间的关系进行有监督学习建模,实现了对查询词项权重自动、合理和有效地预测。实验结果表明,通过该方法得到的查询词项权重能够明显地提升检索效果,并且在多个数据集和检索结果准确率评价指标上均显著地优于各对比方法。2.针对查询理解技术中现有查询意图分类方法普遍存在的严重依赖人工标注数据和面对类别体系变化不灵活的问题,本文将其转换为由一个序列分类问题和一个经典分类问题组成的两阶段分类问题,并根据该分类问题的特点,提出了一种新的基于级联深度学习的查询意图分类方法。该方法首先从提高分类灵活度和效率的角度出发,提出了一种级联的深度神经网络,构造了一个两阶段查询意图分类器；然后从降低对人工标注依赖的角度出发,通过隐式相关反馈技术挖掘源于真实用户的标注行为,实现了查询分类训练数据的自动构造。实验结果表明,该方法能够有效的对查询按主题意图进行分类,且分类效果显著的优于各对比方法。3.针对现有基于查询扩展技术的相关反馈方法对检索系统查询日志及其中查询词项间关系挖掘不足的问题,本文从充分利用检索系统查询日志进行查询扩展的角度出发,提出了一种新的基于两阶段SimRank算法和查询扩展技术的相关反馈方法。该方法通过引入权重关系改进了基于图结构的相似度算法SimRank,并使用改进算法在由查询点击图经多次转换得到的词项关系图上全面深入地挖掘词项间相似度及语义关联,从而筛选出高质量的查询扩展词项。通过在公开标准数据集上的实验表明该方法可以有效地选择高质量相关扩展词项,使得检索效果有显著的提升。4.针对现有基于语言模型的相关反馈方法未能同时充分利用正向和负向相关信息的问题,本文从充分利用隐式反馈和同时挖掘正负两种相关信息的角度出发,提出了一种新的基于语言模型的正负双向相关反馈方法。该方法通过分析疑难查询场景下隐式反馈的正负双向相关文档,基于语言模型检索框架,同时构造正向和负向相关语言模型,并利用正向模型进一步优化负向模型,最大化地提高相关文档排名并尽量过滤非相关文档,从而改善反馈检索的效果。通过基于多个TREC标准数据集的实验验证了该相关反馈方法的有效性,且效果显著优于仅使用正向或负向相关信息的相关反馈方法,使得反馈检索效果有显著的提升。通过以上四个方面的研究,能够得到一个利用查询理解和相关反馈技术改善信息检索整体过程的解决方案,帮助信息检索系统提升检索效果并改善用户体验。
[Abstract]:Wide quick exchanging and sharing information is one of the most important advantages of the Internet, but with the Internet in the carrying amount of data and information of the exponentially explosive growth, leading people must face the increasingly serious problem of information overload. And with the development of the Internet with rapid and in the context of information retrieval technology should be. To solve the problem of information overload is the most direct and effective means. At the same time query understanding and relevance feedback technology in recent years has been widely used to validate the effective way to improve the performance of information retrieval, although the existing research has made some important progress, but there are still many key problems have been solved very well. Aiming at the shortcomings of in the existing studies, based on the analysis of the current understanding of the query and relevance feedback algorithm based on query technology and focus on the understanding of its base Relevant feedback technology foundation are studied. The main research work and contributions are as follows: 1. for the query understanding in lexical entry weight prediction problem, this paper converts it into sequence labeling problems, put forward a new query lexical entry weight learning model based on the model of recurrent neural network. According to the statistics, the considering the lexical entry in the query syntax, semantic and lexical entry information to construct the relationship between lexical entry query feature vector, using genetic algorithm to get the optimal combination of real document relevance marking lexical entry weight value, finally using double to recurrent neural network supervised learning modeling on the relationship between the query sequence and the corresponding lexical entry sequence to achieve the optimal weights. The lexical entry query weights automatically, reasonable and effective prediction. Experimental results show that the query obtained by this method can obviously raise the weight of lexical entry L search results, and in multiple data sets and retrieval accuracy evaluation indicators were significantly better than the contrast method for.2. query understanding relies heavily on common classification methods in existing query intention of labeled data and the flexibility of system changes the face of the category, the two stage is converted to a sequence the classification problem and a classical classification problem consisting of classification problems, and according to the characteristics of the classification problem, proposed a new cascade deep learning based query intent classification method. This method firstly improve the flexibility and efficiency of classification point of view, put forward the depth of the neural network in a cascaded structure, a a two stage query intent classifier; then from the perspective of reducing manual annotation angle of annotation by implicit feedback from real users of mining technology For the realization of the automatic classification, structure of the training data. The experimental results show that this method can classify the query according to the theme of intention, and the classification results were significantly better than the existing.3. comparison method based on query expansion technology problems of relevance feedback retrieval system and the query log query between lexical entry in this paper, mining, query expansion from the perspective of making full use of the retrieval system query log, proposes a relevance feedback method extended two phase SimRank algorithm and query based on new technology. The method by introducing weights of SimRank similarity algorithm based on the graph structure is improved, and the improved algorithm in the lexical entry diagram by a query click through many transformations on the comprehensive and in-depth mining word similarity and semantic association between, in order to find out the query expansion of high quality Lexical entry through open standard data set on the experiment shows that this method can effectively select high quality related extended lexical entry, the retrieval results have significantly improved the existing.4. language model based on relevance feedback method also failed to make full use of positive and negative related information, the paper will make full use of implicit feedback and at the same time, two kinds of mining positive and negative information point of view, put forward a new language model and bidirectional feedback method based on this method. Through the analysis of positive and negative related documents difficult query scenarios implicit feedback retrieval model based on language, framework, and construct the positive and negative correlation model to the language, and the use of positive the model to further optimize the negative model, to maximize the relevant document ranking and try to filter the non relevant documents, so as to improve the effect of feedback retrieval based on multi pass. TREC standard data sets the experimental results verify the effectiveness of the relevant feedback method, and the effect was better than using only the feedback method of positive or negative information, the feedback effect is improved. Through the study of the above four aspects, can get a query using relevance feedback technology to improve understanding and solving for the whole process of information retrieval and information retrieval system to help improve the retrieval effectiveness and improve the user experience.

【学位授予单位】：大连理工大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TP183

【相似文献】