基于Redis的矢量数据组织研究

发布时间：2018-04-17 12:13

  本文选题：Redis + 矢量数据　； 参考：《南京师范大学》2013年硕士论文

【摘要】：目前空间信息应用遭遇到严重的瓶颈,一方面已经积累了巨量空间数据,并且还在继续投巨资生产数据,另一方面,大量的空间数据无法被综合处理和有效利用。为了对数据进行高效存储和管理,关系数据库技术应运而生。但在web2.0时代,面对大规模和高并发的应用,关系数据库疲态尽显,究其原因,主要是关系数据库考虑了过多的特性,如事务、范式约束等,极大地影响了数据库的性能和扩展性。NoSQL数据库摒弃了关系数据库的事务一致性和范式的约束,在很大程度上解决了关系数据库面临的诸多挑战。 本文研究基于Redis的矢量数据组织。作为一种key-value型NoSQL数据库,Redis在高并发时依然具有很高的性能,同时具有丰富的数据结构,适合于对复杂的矢量数据进行组织管理。在矢量数据库管理方面,本文按照矢量数据库、空间数据集、图层、要素的四级结构对矢量数据进行层次组织,针对性地设计了各层的元数据结构,并选用高效的Redis数据结构对元数据信息和矢量数据信息(几何和属性)进行存储。在空间索引方面,本文深入研究了基于Redis的空间索引组织和查询机制,并以网格索引为例,研究了基于Redis的网格索引的表达方式、存储结构和查询方法；针对传统网格索引的缺点,提出了一种基于小角编码的网格索引：首先对矢量数据进行小角编码,然后利用Redis的有序set数据结构存储网格单元索引信息表,在此基础上,提出一种改进的网格索引方法——小角编码算法进行快速索引查询。利用小角编码思想构建的网格单元索引信息表,自动分成重复区和非重复区两部分,查询时只需要针对重复区索引信息使用简单的比较操作就能在0(N)时间内完成查重。由于查重时无需维护查找树之类的数据结构,从而降低了空间复杂度,提高了矢量数据网格索引的效率。 最后,论文设计和实现了基于Redis的矢量数据引擎原型系统,并利用大批量的矢量数据进行对比实验和性能分析,实验结果验证了本文的基本思想。
[Abstract]:At present, the application of spatial information has encountered a serious bottleneck. On the one hand, it has accumulated a huge amount of spatial data, and continues to invest a large amount of production data. On the other hand, a large number of spatial data can not be comprehensively processed and effectively utilized.In order to store and manage data efficiently, relational database technology emerges as the times require.But in the era of web2.0, in the face of large scale and high concurrency application, the fatigue of relational database is obvious. The main reason is that the relational database takes too many characteristics into account, such as transaction, normal form constraint, etc.It greatly affects the performance and extensibility of database. NoSQL database abandons the constraints of transaction consistency and paradigm of relational database and solves many challenges faced by relational database to a great extent.This paper studies vector data organization based on Redis.As a kind of key-value NoSQL database, Redis still has high performance and rich data structure in high concurrency, so it is suitable for organizing and managing complex vector data.In the aspect of vector database management, according to the four-level structure of vector database, spatial data set, map layer and element, this paper organizes vector data hierarchically, and designs the metadata structure of each layer.An efficient Redis data structure is used to store metadata and vector data (geometry and attributes).In the aspect of spatial index, this paper deeply studies the spatial index organization and query mechanism based on Redis. Taking grid index as an example, the representation, storage structure and query method of grid index based on Redis are studied.Aiming at the shortcoming of traditional grid index, a grid index based on small angle coding is proposed. Firstly, vector data is encoded in small angle, then the grid cell index information table is stored by using Redis's ordered set data structure.In this paper, an improved mesh indexing method, small angle coding algorithm, is proposed for fast index query.The grid cell index information table which is constructed by using small angle coding idea is automatically divided into two parts: duplicate area and non-repetition area. When querying, we only need to use a simple comparison operation for the index information of repeat area to complete the search in 0 ~ N) time.Because the data structure such as lookup tree does not need to be maintained, the spatial complexity is reduced and the efficiency of vector data grid index is improved.Finally, the prototype system of vector data engine based on Redis is designed and implemented, and the comparison experiment and performance analysis are carried out by using large quantities of vector data. The experimental results verify the basic idea of this paper.
【学位授予单位】：南京师范大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：P208
，

本文编号：1763557