网页学习排序算法研究

发布时间：2018-08-05 13:50

【摘要】：高端先进的信息技术，将我们带入了一个浩如烟海的数字时代。大量数据的涌入使得搜索引擎变得越来越重要，如何从海量数据中快速定位所需信息显得异常关键。搜索引擎包含多个组成部分，其中网页排序是搜索引擎设计的核心问题，它决定着搜索引擎排序结果，直接影响着搜索引擎的性能和用户体验。信息检索领域有许多网页排序算法，大致可归类为样本点级别方法模型、样本对级别方法模型以及样本列表级别方法模型。研究者们在这三类方法中运用多种算法做了很多贡献，然而对网页学习排序算法的研究仍处于白热化阶段。针对网页学习排序问题，本文首先SVM(Support Vector Machine)分别从样本点级别和样本对级别建立了基于SVM的网页学习排序模型。求解部分运用交叉检验的思想选择SVM模型中的参数，同时进行了核函数选择分析；在数据分析与处理时，选取了部分特征数据进行可视化以及数据分维分析，为消除数据量纲影响在预处理部分做了归一化处理。在样本对方法下的SVM网页排序模型求解中，本文采用随机排序配对方法得到训练样本。接着，运用启发式方法建立了遗传算法优化BP(BackPropagation)神经网络学习排序模型。该模型利用遗传算法的寻优能力，得到较好的BP网络初始权值和阈值，以提高BP网络的性能。为减轻BP网络训练复杂度，求解部分运用主成分分析法将训练数据进行了压缩，，在保证较高的数据保真度时使压缩后的数据维度降至使BP网络结构合适的程度。最后，基于Boosting思想建立了基于Boosting算法的网页排序模型，旨在研究强学习排序器在弱学习机基础上的提升能力。本文在LETOR数据子集OHSUMED上进行实验。实验结果表明样本对级别模型的学习排序能力稍高于样本点级别；遗传算法优化BP网络权值与阈值能提高模型的排序精度，但时间开销很大；Boosting方法中的强学习排序器能取得较好效果。
[Abstract]:High-end advanced information technology, will bring us into a vast digital age. With the influx of data, search engines become more and more important. How to locate the information quickly from the massive data is very important. Search engine consists of many components, among which web page sorting is the core problem of search engine design. It determines the search engine ranking results and directly affects the performance and user experience of search engine. There are many sorting algorithms in the field of information retrieval, which can be classified as sample point level method model, sample to level method model and sample list level method model. Researchers have made a lot of contributions to these three methods, but the research of web page learning sorting algorithm is still in the hot stage. Aiming at the problem of web page learning ranking, this paper first establishes a web page learning sorting model based on SVM from sample point level and sample pair level, respectively. The solution uses the idea of cross test to select the parameters in the SVM model, at the same time, the kernel function selection analysis is carried out, and some characteristic data are selected for visualization and fractal dimension analysis in data analysis and processing. In order to eliminate the influence of data dimension, the preprocessing part is normalized. In this paper, the training samples are obtained by random sorting pairing method in the solution of SVM web page sorting model based on sample pair method. Then, the genetic algorithm optimization BP (BackPropagation) neural network learning scheduling model is established by heuristic method. The model uses the optimization ability of genetic algorithm to get better initial weight and threshold value of BP network, so as to improve the performance of BP network. In order to reduce the training complexity of BP network, the principal component analysis (PCA) is used to compress the training data, which can reduce the dimension of compressed data to the appropriate degree of BP network structure. Finally, a web page sorting model based on Boosting algorithm is established based on the idea of Boosting. The purpose of this model is to study the ability of the strong learning sorter on the basis of weak learning machine. The experiment is carried out on OHSUMED, a subset of LETOR data. The experimental results show that the learning ability of the sample to the hierarchical model is slightly higher than that of the sample point, and the genetic algorithm can improve the sorting accuracy of the model by optimizing the weights and thresholds of BP network, but the time cost is very large. The strong learning sorter in Boosting method can achieve good results.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP391.3

【相似文献】