分布式环境下海量空间数据的存储和并行查询技术研究

发布时间：2018-04-23 18:47

本文选题：海量空间数据 + 分布式存储　；参考：《江西理工大学》2012年硕士论文

【摘要】：随着GIS在各行业的广泛应用，加上人们对空间数据的精度要求也不断在增加，，于是GIS中需要管理的空间数据越来越多，数据量也越来越大近年来，分布式存储和并行计算的不断发展为海量空间数据的存储和处理提供了一个新的方向然而，目前在分布式环境下对于海量空间数据的存储和处理大部分是基于传统关系数据库，对于二三维的海量空间数据存储效果不理想，其独有的关系模型束缚对海量空间数据的快速访问和处理能力因此，分布式环境下如何利用非关系数据库和并行处理技术实现海量空间数据的高效存储和快速处理具有重要的研究意义首先比较分析目前几类典型非关系数据库的特点，提出了非关系数据库MongoDB的对于海量数据存储优势在总结空间数据的特点基础上，设计了海量空间数据在MongoDB中的存储方式为了验证MongoDB对于海量数据存储的可行性，与关系型数据库SQLSERVER进行了数据插入查询实验比较接着根据上述的存储方式，结合了并行处理框架Hadoop的MapReduce实现海量空间数据的并行处理最后设计实现了基于Hadoop+MongoDB的HMGIS数据库管理器，并在多台Linux服务器上搭建了HMGIS的分布式环境，通过利用HMGIS进行海量空间数据导入和多种空间查询等并行处理实验证明HMGIS的可行性和高效性主要取得的研究成果： τ1υ利用MongoDB来对非结构化的海量空间数据存储是可行的； τ2υ利用Hadoop的MapReduce结合MongoDB数据库实现的HMGIS数据库管理器在海量空间数据的多种查询方面比传统关系数据库具有更高的访问和查询效率； τ3υHMGIS设计实现可以进一步为海量空间数据的分析和复杂的地理计算提供软件支持
[Abstract]:With the wide application of GIS in various industries and the increasing demand for spatial data accuracy, more and more spatial data need to be managed in GIS, and the amount of data is becoming larger and larger in recent years. The continuous development of distributed storage and parallel computing provides a new direction for the storage and processing of massive space data. At present, most of the storage and processing of massive spatial data in distributed environment is based on the traditional relational database, which is not ideal for two or three dimensional mass spatial data storage, and its unique relational model restraints the rapid access and processing power of massive spatial data, and how to use non relation under the distributed environment. Database and parallel processing technology are of great significance for efficient storage and rapid processing of massive spatial data.
First, the characteristics of several typical non relational databases are compared and analyzed. On the basis of the characteristics of massive data storage in the non relational database MongoDB, the storage mode of massive spatial data in MongoDB is designed to verify the feasibility of MongoDB for mass data storage and the number of relational types. According to the storage mode of data insertion and query according to library SQLSERVER, the parallel processing of the parallel processing framework Hadoop MapReduce is combined to implement the parallel processing of massive spatial data. The HMGIS database manager based on Hadoop+MongoDB is designed and implemented, and a distributed loop of HMGIS is built on a number of Linux servers. The feasibility and efficiency of HMGIS are proved by parallel processing experiments such as massive spatial data import and multiple spatial queries using HMGIS.
The main achievements of the research are:
It is feasible for MongoDB 1 to store unstructured massive spatial data.
The HMGIS database manager, implemented by Hadoop MapReduce and MongoDB database, has higher access and query efficiency than traditional relational database in many queries of massive spatial data.
The design and implementation of HMGIS 3 can further provide software support for massive spatial data analysis and complex geographic computation.

【学位授予单位】：江西理工大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：P208;TP333

【参考文献】