一种大数据存储模型的研究与应用

发布时间：2018-02-04 17:17

本文关键词： 数据库大数据关系数据存储模型面向对象　出处：《北京邮电大学》2013年硕士论文　论文类型：学位论文

【摘要】：在大数据环境下,数据存储出现了许多新的需求,传统基于关系数据库的数据存储方式不能满足这些需求,许多应用系统逐渐倾向于使用NoSQL解决大数据存储问题。然而,NoSQL放弃了对关系操作的支持,使得部分已有应用系统难以使用简单的方式进行移植。该论文参考现有大数据存储的一些典型方案,提出一种兼容关系存储模型的大数据存储方案,该方案不仅能够满足大数据存储的需求,还能够支持大多数关系操作,从而使原有基于关系数据库的系统能够简单方便地移植到新的存储方案中。为实现上述目标,论文进行了下述研究工作：(1)对几种典型的NoSQL存储模型以及一些基于关系数据库的分布式存储方案进行深入研究,分析其工作原理与设计思想,明确论文方案的技术难点；(2)针对论文方案的各项技术难点,进行重点攻破,论文引入面向对象设计思想,设计三维关系模型,解决原有关系模型扩展性问题,并在三维关系模型基础上,设计基于类型的数据切分方案以及分布式并行计算方案,满足海量数据实时处理的需求；(3)将论文方案应用到具体的集群环境中,与原有方案进行对比分析,并进行改进与完善。该论文选题来源于国家“十一五”科技重点支撑计划的免费孕前优生健康检查项目,并最终用于解决该项目数据存储的结构复杂、数据量大、查询缓慢等问题,目前已取得一定成效。该论文提出的方案可以解决关系数据模型的扩展性问题,提高关系数据库数据查询效率,在海量关系型数据无法进行NoSQL移植时,可作为存储优化方案,解决数据存储问题。
[Abstract]:In big data environment, there are many new requirements for data storage, which can not be met by traditional data storage methods based on relational database. Many applications tend to use NoSQL to solve big data storage problems. However, NoSQL gives up support for relational operations. Some existing application systems are difficult to transplant in a simple way. Referring to some typical schemes of big data storage, this paper proposes a big data storage scheme for compatible relational storage model, which can not only meet big data storage requirements. It can also support most relational operations, so that the existing relational database system can be easily transplanted to the new storage scheme. In order to achieve the above goal, this paper carries out the following research work: 1) deeply study several typical NoSQL storage models and some distributed storage schemes based on relational database. The working principle and design idea are analyzed, and the technical difficulties of the thesis scheme are clarified. 2) aiming at the technical difficulties of the thesis scheme, the paper introduces the object-oriented design idea, designs the three-dimensional relational model, and solves the expansibility problem of the original relational model. On the basis of the 3D relational model, the data segmentation scheme based on type and the distributed parallel computing scheme are designed to meet the demand of real-time processing of massive data. 3) apply the thesis scheme to the concrete cluster environment, compare and analyze with the original scheme, and improve and perfect it. The topic of this paper comes from the national "11th Five-Year" science and technology key support plan of the free pre-pregnancy health check project, and ultimately used to solve the project data storage structure complex, data volume. Some problems, such as slow query, have been achieved at present. The scheme proposed in this paper can solve the expansibility of relational data model and improve the efficiency of query in relational database. When the massive relational data can not be transplanted into NoSQL, it can be used as a storage optimization scheme to solve the data storage problem.
【学位授予单位】：北京邮电大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】