基于Hadoop的物联网（WoT）搜索引擎设计与实现

发布时间：2018-03-14 06:00

本文选题：规则推理　切入点：查询扩展　出处：《北京邮电大学》2017年硕士论文　论文类型：学位论文

【摘要】：感知设备的搜索是物联网发展道路上的关键一环,是物联网应用的前提,目前尚未有成熟的解决方案,而面向互联网的传统搜索引擎技术却已相当成熟,重用这些技术,对发展物联网搜索无疑是一种助力。另外,开源分布式计算平台Hadoop凭借易扩展、海量数据存储和超强计算能力被产业界和学术界视为大数据处理最重要的工具,除此之外,Hadoop可以被广泛地部署于廉价PC之上,能够降低生产成本。在此背景之下,本课题设计并实现了一个“基于Hadoop的物联网(WoT)搜索引擎”,它含有查询扩展模块和分布式倒排索引系统两个核心组成部分。本课题研究工作的主要内容如下:为了克服传统搜索引擎技术基于关键词进行机械式地符号匹配所带来的查全率低下的问题,查询扩展模块中引用一个已有的物联网领域本体,并对其进行基于规则的语义推理,获取本体中的隐含信息,完成对本体的扩充,依据推理之后的本体进行查询扩展。着重研究了本体相关理论、基于规则的本体推理以及Jena推理机,构建了用于本体推理的规则库,给出了一种基于本体的查询扩展算法。分布式倒排索引系统采用的是按文档划分的局部倒排索引组织策略。着重研究了面向互联网的传统搜索引擎的工作原理、HDFS、MapReduce、开源核心搜索库Lucene以及分布式倒排索引的两种组织策略。为了体现物联网搜索应该具有的空间特性,课题增加了一个基于距离的搜索结果排序模块。基于本体的查询扩展模块、分布式倒排索引系统、基于距离的搜索结果排序模块以及用户查询接口共同构成了完整的WoT搜索引擎。最后,通过测试证明了课题方案的可行性及有效性。
[Abstract]:Perception device search is a key link in the development of the Internet of things, is the premise of the Internet of things application, there is no mature solution, but the traditional search engine technology for the Internet has been quite mature, reuse these technologies, In addition, Hadoop, an open source distributed computing platform, is regarded by industry and academia as the most important tool to deal with by big data because of its easy expansion, massive data storage and super computing power. In addition, Hadoop can be widely deployed on cheap PCs and reduce production costs. This paper designs and implements a "Internet of things (IoT) search engine based on Hadoop", which contains two core components: query extension module and distributed inverted index system. The main contents of this research work are as follows:. To overcome the problem of low recall caused by traditional search engine technology based on keywords for mechanical symbol matching, In the query extension module, an existing ontology of the Internet of things is referenced, and the semantic reasoning based on rules is carried out to obtain the implicit information in the ontology to complete the extension of the ontology. Based on the query extension of ontology after reasoning, this paper focuses on the ontology theory, rule-based ontology reasoning and Jena inference engine, and constructs a rule base for ontology reasoning. This paper presents a query extension algorithm based on ontology. The distributed inverted index system adopts the local inverted index organization strategy according to documents. The working principle of traditional search engine for Internet is studied emphatically. Open source core search library Lucene and two organizational strategies for distributed inverted indexes. In this paper, a distance based search result sorting module is added. The query extension module based on ontology and the distributed inverted index system are introduced. The search result sorting module based on distance and the user query interface constitute the complete WoT search engine. Finally, the feasibility and effectiveness of the project are proved by testing.
【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.3;TP391.44;TN929.5

【参考文献】