个性化新闻搜索引擎的研究与设计

发布时间：2018-06-12 00:14

本文选题：垂直搜索 + 个性化　；参考：《电子科技大学》2012年硕士论文

【摘要】：随着计算机各项技术的不断发展，信息化时代已经到来。如何让用户在海量的数据中快速并且准确的获取自己所需要的信息，是互联网行业一个亟待解决的问题。为此，搜索引擎应运而生。垂直搜索是搜索引擎中一个重要的组成部分，它能帮助用户在自己的专业领域快速的获取信息，，可以获得比通用搜索更高的搜索效率和更准确的搜索结果。另外，个性化技术能针对不同的用户给出不同的检索方案，它对提高用户的检索满意度有很大帮助。目前大多数的搜索引擎是通过匹配索引关键词进行数据检索，并没有考虑到各个用户所关心的信息领域，所以搜索引擎检索出的相当一部分结果与用户的需求并不相关。垂直搜索引擎的个性化研究是解决这种检索结果相关性瓶颈的一种有效方法。本文从垂直搜索引擎的基本概念，基本原理，组成结构以及工作流程入手，研究了网络爬虫模块，索引模块及关键词检索模块等，并结合用户兴趣模型，设计了一种适合于垂直搜索引擎的个性化搜索解决方案，最后实现了个性化垂直搜索引擎的一个实例。网络爬虫模块从各大门户网站中进行数据爬取，并结合主题相关度进行URL筛选；索引模块引入了文本分类技术，在保证索引效率的基础上对不同类别的文本在构建索引时区别考虑；检索模块结合了用户兴趣模型和文本分类技术，提升了检索结果与用户意图之间的相关性。本文的研究内容和创新点主要包括三个方面。第一，提出了一种个性化垂直搜索中可行的专业爬虫解决方案。目前的垂直搜索引擎并没有充分的考虑URL主题相关性过滤，这将导致大量噪声网页产生。本文把URL相关性过滤机制引入到专业爬虫中，并对网络爬行策略进行了改进，提高了专业爬虫的信息采集效率。第二，提出了一种适合于垂直搜索的文本分类方法。目前的垂直搜索引擎主要是通过栏目标题等进行分类，它们存在人工干预强，分类过程不灵活等问题。本文将通用搜索的特征选择以及文本分类算法引入到垂直搜索中，并对其改进和优化，使其更适合垂直搜索引擎。第三，本文引入了适合垂直搜索的个性化建模方法，并通过相关反馈技术完善兴趣模型，让搜索引擎具有更强的个性化能力。
[Abstract]:With the continuous development of computer technology, the information age has come. How to make users get the information they need quickly and accurately in the massive data is an urgent problem in the Internet industry. Therefore, the search engine emerges as the times require. Vertical search is an important part of search engine, it can help users to obtain information quickly in their own professional field, and can obtain higher search efficiency and more accurate search results than general search. In addition, personalized technology can provide different retrieval schemes for different users, which is of great help to improve users' search satisfaction. At present, most search engines do data retrieval by matching index keywords, and do not consider the information field concerned by each user, so a considerable part of the search engine retrieval results are not related to the needs of users. The personalized research of vertical search engine is an effective method to solve the bottleneck of relevance of retrieval results. This paper starts with the basic concept, basic principle, composition structure and workflow of vertical search engine, and studies the web crawler module. Index module and keyword retrieval module, combined with user interest model, designed a personalized search solution suitable for vertical search engine. Finally, an example of personalized vertical search engine was implemented. The web crawler module crawls the data from the major web portals, and combines the topic relevance to the URL filtering, and the index module introduces the text classification technology. On the basis of ensuring the efficiency of index, different types of text are considered when constructing index, and the retrieval module combines user interest model and text classification technology. The research content and innovation of this paper mainly include three aspects. First, a professional crawler solution in personalized vertical search is proposed. The current vertical search engine does not fully consider URL topic correlation filtering, which will lead to a large number of noisy pages. In this paper, the URL correlation filtering mechanism is introduced into the professional crawler, and the network crawling strategy is improved to improve the information collection efficiency of the professional crawler. Secondly, a text classification method suitable for vertical search is proposed. At present, vertical search engines are mainly classified by column titles, which have some problems such as strong manual intervention and inflexible classification process. In this paper, the feature selection and text classification algorithm of general search is introduced into vertical search, and it is improved and optimized to make it more suitable for vertical search engine. Thirdly, this paper introduces the personalized modeling method suitable for vertical search, and improves the interest model through the relevant feedback technology, so that the search engine has stronger individuation ability.
【学位授予单位】：电子科技大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP391.3

【参考文献】