海量RDF数据存储与查询技术的研究与实现

发布时间：2018-03-25 09:32

本文选题：语义网　切入点：RDF　出处：《北京工业大学》2013年硕士论文

【摘要】：随着互联网上数据规模的日益增长，人们准确、快速、全面获取信息变得越来越困难，语义网(Semantic Web)通过对互联网增加语义支持，使机器能够理解数据的含义，帮助人们快速获取信息资源。RDF(Resource Description Framework,资源描述框架)是语义网中数据交换的标准，它以主语,谓词,宾语三元组的形式描述语义信息。随着语义网技术的不断完善与广泛应用，RDF数据规模也在急剧增加，海量RDF数据的涌现给RDF的管理带来巨大挑战，构建可扩展的RDF存储和查询系统成为语义网领域的研究热点。 MapReduce是近年来新兴的海量数据处理技术，Hadoop是MapReduce开源实现工具，基于Hadoop平台研究海量数据处理问题被广泛关注；HBase(HadoopDatabase)，是一个高可靠性、高性能、面向列、可伸缩的分布式存储系统，和Hadoop MapReduce有着完善的集成机制。通过研究RDF存储和查询的原理以及HBase的特点，分析HBase存储RDF的优势，给出了一个基于HBase的RDF存储系统设计方案，并实现了RDF存储原型系统，主要包括以下内容： 1.根据RDF存储和查询的特点，，结合HBase自身的索引机制以及行健(HBase RowKey)按字典排序的特性，设计了RDF在HBase上的存储模式。 2.针对海量RDF数据的加载问题，给出了一种基于MapReduce的并行加载算法，实现将RDF数据快速加载到HBase的功能。 3.根据RDF在HBase上的存储模式，设计了三元组模式查询响应策略；采用MapReduce迭代实现基本图模式查询的并行处理，给出了一种以HBase Region为数据源的MapReduce并行连接(join)方法。通过构建基于HBase的RDF存储原型系统，采用基准测试工具对系统的加载性能和查询性能进行了测试和分析，试验表明基于HBase实现海量RDF数据的存储和查询系统是一种有效的可选方案。
[Abstract]:With the increasing scale of data on the Internet, it is becoming more and more difficult for people to obtain information accurately, quickly and comprehensively. The semantic Web adds semantic support to the Internet to enable machines to understand the meaning of data. RDF Resource Description Framework (Resource description Framework) is the standard of data exchange in the semantic Web. With the continuous improvement and wide application of semantic Web technology, the scale of RDF data is increasing rapidly, and the emergence of massive RDF data brings great challenges to the management of RDF. The construction of extensible RDF storage and query systems has become a research hotspot in the semantic web. MapReduce is a newly emerging mass data processing technology in recent years. Hadoop is an open source MapReduce implementation tool. The research of mass data processing based on Hadoop platform has been widely concerned. It is a high reliability, high performance, column oriented, scalable distributed storage system. It has perfect integration mechanism with Hadoop MapReduce. By studying the principle of RDF storage and query and the characteristics of HBase, this paper analyzes the advantages of HBase storage RDF, presents a RDF storage system design scheme based on HBase, and implements the RDF storage prototype system. It mainly includes the following:. 1. According to the characteristics of RDF storage and query, combined with the index mechanism of HBase and the dictionary-sorting property of HBase, the storage mode of RDF on HBase is designed. 2. Aiming at the loading problem of massive RDF data, a parallel loading algorithm based on MapReduce is presented, which can load RDF data into HBase quickly. 3. According to the storage mode of RDF on HBase, the query response strategy of triple schema is designed, the parallel processing of basic graph schema query is realized by MapReduce iteration, and a MapReduce parallel join method based on HBase Region is presented. By constructing the RDF storage prototype system based on HBase, the loading performance and query performance of the system are tested and analyzed by using the benchmark tool. The experiment shows that the storage and query system of massive RDF data based on HBase is an effective alternative.
【学位授予单位】：北京工业大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【相似文献】