Goldfish:基于矩阵分解的大规模RDF数据存储与查询系统

发布时间：2018-08-05 18:21

【摘要】：随着互联网应用的迅猛发展和语义网技术研究的深入,语义数据呈现出爆炸性增长趋势.一方面,对于语义数据实现高效存储和查询是语义网应用的重要基础,越来越多的语义应用可以依赖于此以提供更好的服务;另一方面,语义数据的爆炸性增长,对大数据环境下的语义数据的存储与查询技术提出了新的挑战.传统的基于关系型数据库的语义数据与查询系统已难以满足大规模语义数据的存储与查询需求.该文针对大规模RDF数据的存储与查询问题,以OpenRDF Sesame框架为基础,采用分布式分层式存储架构,提出并实现了属性表存储结构来进行语义数据的存储.在此基础上,针对布尔矩阵分解算法在对大规模语义数据构造属性表较慢的问题,基于Spark分布式计算框架提出并实现了并行化频繁项集挖掘算法求解大规模矩阵分解,以加速属性表的构造过程.并且,在查询层增加了基于哈希转换等查询优化.最后,基于该文所提出的索引结构和优化方法设计实现了原型系统Goldfish,并在大规模合成和真实数据集上进行了实验对比.结果表明,Goldfish原型系统比Rainbow系统查询性能平均提升约6倍,比Jena-HBase查询性能平均提升约500倍,比基于MapReduce的RDF查询系统SHARD性能平均提升约1200倍.
[Abstract]:With the rapid development of Internet applications and the development of semantic Web technology, semantic data show an explosive growth trend. On the one hand, the efficient storage and query of semantic data is an important foundation of semantic web applications, more and more semantic applications can rely on it to provide better services; on the other hand, the explosive growth of semantic data, This paper presents a new challenge to the storage and query of semantic data in big data environment. The traditional semantic data and query system based on relational database can not meet the requirement of large-scale semantic data storage and query. Aiming at the problem of storing and querying large scale RDF data, based on the OpenRDF Sesame framework, this paper proposes and implements the attribute sheet storage structure to store semantic data by using the distributed hierarchical storage architecture. On this basis, aiming at the problem that Boolean matrix decomposition algorithm is slow to construct attribute tables for large-scale semantic data, a parallel frequent itemset mining algorithm based on Spark distributed computing framework is proposed and implemented to solve large-scale matrix decomposition. To accelerate the construction of a property sheet. Moreover, query optimization based on hash conversion is added in the query layer. Finally, based on the index structure and optimization method proposed in this paper, the prototype system Goldfish is designed and implemented, and the experimental results are compared on large-scale composite and real data sets. The results show that the query performance of Goldfish prototype system is about 6 times higher than that of Rainbow system, 500 times higher than that of Jena-HBase query system, and 1200 times higher than that of RDF query system based on MapReduce.
【作者单位】：南京大学计算机软件新技术国家重点实验室;江苏省软件新技术与产业化协同创新中心;
【基金】：国家自然科学基金专项基金(61223003);国家自然科学基金(61370019) 江苏省科技支撑计划项目(BE2014131)资助~~
【分类号】：TP311.13

【相似文献】