基于标签词抽取的搜索结果聚类研究

发布时间：2018-09-03 12:29

【摘要】：当前人们正处于一个“信息爆炸”的时代,因此各种各样的搜索引擎应运而生。但是由于网上的信息都是半结构化或者非结构化的,尽管采用了各种方法来提高检索结果的精度,但是检索结果中仍然包含了与用户查询不相关的页面。虽然可以采取相关度排序等方法,仍不能很方便的为用户展示结果。为了方便用户查看自己感兴趣的网页,将搜索引擎返回的结果进行聚类,使用户可以按照主题类别浏览网页,减轻用户浏览网页的负担。本文在研究中文文本聚类现状的基础上,对中文文本聚类的关键技术进行了总结,其中,包括文本预处理、文本表示模型、特征抽取、特征降维、文本相似度计算以及现有的聚类算法,并对现有的聚类算法作了分析比较。然后,论文分析并研究了文本的相似度计算,包括文档的相似度计算和相异度计算,以及簇和簇之间的邻近度度量。并且分析了支持向量回归理论和其技术上的实现。本文提出了一种基于标签词抽取的文本聚类方法,该方法的实现目标是对搜索引擎返回的搜索结果进行聚类,然后论文实现了文本聚类系统。首先从搜索结果返回的网页文档进行预处理,包括去噪、分词、去除停用词。然后从中抽取3元模型词作为标签词,提出了基于监督模型的打分方法,并对标签词做一些相似词替换、词串整合等后期处理。最后根据标签词进行语料聚类,采用了层次聚类的方法,最终完成聚类。论文设计了聚类系统,并对其进行实验,实验内容包括标签词的抽取,支持向量的回归统计,标签词的聚类实验。通过实验证明,算法在对搜索结果进行聚类时有着较好的效果,能够将类别相似的文档聚到同一个类别中。
[Abstract]:At present, people are in an era of "information explosion", so various search engines emerge as the times require. However, because the information on the Internet is semi-structured or unstructured, although a variety of methods are used to improve the accuracy of the retrieval results, the retrieval results still contain pages that are not related to the user query. Although we can use correlation ranking and other methods, it is still not convenient for users to display the results. In order to facilitate users to view their interested web pages, the search engine returns the results of clustering, so that users can browse the web according to the subject category, reducing the burden of users browsing web pages. On the basis of studying the present situation of Chinese text clustering, this paper summarizes the key technologies of Chinese text clustering, including text preprocessing, text representation model, feature extraction, feature dimensionality reduction, etc. Text similarity calculation and existing clustering algorithms are analyzed and compared. Then, the paper analyzes and studies the text similarity calculation, including document similarity calculation and dissimilarity calculation, as well as the proximity measure between clusters. The support vector regression theory and its technical realization are analyzed. In this paper, a text clustering method based on tag word extraction is proposed. The goal of this method is to cluster the search results returned by search engines, and then the text clustering system is implemented in this paper. First, we preprocess the web pages returned from the search results, including de-noising, participle, and deactivation. Then the three-element model words are extracted as label words, and a scoring method based on supervised model is put forward, and some similar word substitution and string integration are made for label words. Finally, according to the label word clustering, hierarchical clustering method is used to complete the clustering. This paper designs a cluster system and carries on the experiment to it, the experiment content includes the tag word extraction, the support vector regression statistics, the label word clustering experiment. The experimental results show that the algorithm is effective in clustering search results and can cluster similar documents into the same category.
【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP391.1

【参考文献】