基于时效性的垂直搜索引擎的设计与实现

发布时间：2018-02-13 21:00

本文关键词： 信息检索搜索引擎时间信息未登录词识别向量空间模型　出处：《北京建筑工程学院》2012年硕士论文　论文类型：学位论文

【摘要】：近年来，互联网的飞速发展使信息呈现爆炸式增长，这让用户准确及时地获取所需信息变得愈加困难，搜索引擎的出现在一定程度上缓解了这种危机。后来面向特定领域的垂直搜索引擎出现，它提供给用户面向特定领域的个性化信息搜索服务，弥补了通用搜索引擎主题较宽泛的缺点，使搜索结果更有针对性，同时也提高了用户对查询结果的满意度。时间信息在自然语言中有着非常重要作用，它可以定位事件发生的先后顺序。据统计时间信息占全部文本信息的27%，仅次于31%的专有名词。因此本文对如何在垂直搜索引擎中引入时间信息因素进行了研究。本文首先对网页文本中时间表达式的识别和规范化进行了深入研究，然后提出一种以文档中的时间为分量的文档时间向量表达方式和以时间为索引项的倒排索引结构，最后在以上方法的基础上完成了一个能够根据网页文本内容中的时间信息进行查询的垂直搜索引擎。本文的主要工作如下： 1．对搜索引擎的发展历程及现状进行了阐述，介绍了搜索引擎的工作原理、基本架构和搜集器、预处理器及检索器等核心技术。 2．分析了垂直搜索引擎产生的必然性及其与通用搜索在效果和技术实现上的区别。 3．参考TIMEX2规范对时间表达式进行分类，并给出了规则模板和时间词典相结合的时间表达式识别方法，然后探讨了把已识别出的时间表达式规范化的方法。 4．本系统是面向旅游领域的垂直搜索，故提出了一种基于规则的识别未登录中文地名实体的方法，，经实验其召回率达90%以上。 5．基于空间向量模型(VSM)提出了以时间为分量的文档时间向量表示方法和以时间为索引项的倒排索引的构建方法，还对时间向量相似性的计算方法进行了定义，最后给出了根据时间向量相似性对文档进行排序的算法和查询规则。 6．在以上理论及算法的基础上，设计并实现了一个基于时效性的垂直搜索引擎原型，并对该系统的系统架构及各功能模块的具体实现进行了详细介绍。
[Abstract]:In recent years, the rapid development of the Internet has caused the explosive growth of information, which makes it more difficult for users to accurately and timely access to the information they need. The emergence of search engines alleviated the crisis to some extent. Later, vertical search engines for specific areas emerged, providing users with personalized information search services for specific areas. It makes up for the shortcomings of the general search engine, which makes the search results more targeted and improves the users' satisfaction with the query results. Time information plays a very important role in natural language. It can locate the sequence of events. According to statistics, time information accounts for 27% of all text information, second only to 31% proper nouns. Therefore, this paper studies how to introduce time information factors into vertical search engines. In this paper, the recognition and normalization of time expressions in web pages are studied in depth, and then a time vector representation and inverted index structure with time as index items are proposed. Finally, a vertical search engine is developed based on the above methods. The main work of this paper is as follows:. 1. The development course and present situation of search engine are described, and the working principle, basic structure, collector, preprocessor and retrieval technology of search engine are introduced. 2. The inevitability of vertical search engine and the difference between vertical search engine and general search engine are analyzed. 3. According to the TIMEX2 specification, this paper classifies the time expression, and gives the method of recognizing the time expression combining the rule template and the time dictionary, and then discusses the method of normalizing the recognized time expression. 4. This system is oriented to the vertical search in the tourism field. Therefore, a rule-based method for identifying unrecorded Chinese geographical names entities is proposed. The recall rate of the system is more than 90%. 5. Based on the spatial vector model (VSM), a document time vector representation method based on time component and a method of constructing inverted index based on time are proposed. The method of calculating the similarity of time vector is also defined. Finally, an algorithm and query rules for sorting documents according to the similarity of time vectors are given. 6. Based on the above theories and algorithms, a vertical search engine prototype based on timeliness is designed and implemented, and the system architecture and the implementation of each functional module are introduced in detail.
【学位授予单位】：北京建筑工程学院
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP391.3

【参考文献】