针对非关系型数据库的海量数据的事务处理

发布时间：2018-06-30 04:20

本文选题：非关系型数据库 + 事务支持　；参考：《上海交通大学》2014年硕士论文

【摘要】：随着信息社会的发展，越来越多的信息被数据化，尤其是伴随着互联网的发展，数据呈爆炸式地增长，海量数据的存储和处理问题随之出现。在面对海量数据的高效率存储和高效率访问、数据库的高并发读写以及数据库的高可扩展性等需求时，传统的关系数据库暴露了很多问题。非关系数据库（NoSQL）应运而生，与传统的关系型数据库相比，非关系型数据库在数据库事务一致性方面进行了妥协。为了解决非关系型数据库在数据库事务一致性方面的问题，本文以HBase为例并基于两阶段提交协议提出了针对非关系型数据库的海量数据的事务支持机制。依靠HBase自身的持久化、高吞吐量、低延迟等特点，，本文使用HBase的单独的一列实现了锁服务，同时依靠两阶段提交协议实现了多行多表之间的同步，并对分布式系统中经常出现的宕机问题进行了考虑，设计了异常情况处理机制。同时本文针对HBase实现了read uncommitted、read committed和repreatable read三种隔离级别以满足不同类型的应用对事务隔离性的不同要求。在此基础上，本文又设计和实现了“期货交易仿真系统”，用于验证该事务支持机制。该平台使用HBase作为后台数据库，并支持数百甚至上千用户的并发访问请求。通过该系统，本文对针对非关系型数据库的海量数据的事务支持机制的可用性和效率进行了验证。验证结果表明在非关系型数据库上应用该事务支持机制之后，能够满足大规模并发访问对事务隔离性与一致性的需求，虽然在效率上有所下降，但是基本在可以接受的范围之内。
[Abstract]:With the development of the information society, more and more information is digitized, especially with the development of the Internet, the data increases explosively, and the storage and processing of massive data appear. In the face of the requirements of efficient storage and efficient access of massive data, high concurrent reading and writing of databases, and high scalability of databases, traditional relational databases have exposed many problems. Non-relational database (NoSQL) emerged as the times require. Compared with the traditional relational database, the non-relational database compromise on the consistency of database transactions. In order to solve the problem of database transaction consistency in non-relational database, this paper takes HBase as an example and proposes a transaction support mechanism for massive data of non-relational database based on two-stage commit protocol. Depending on the persistence, high throughput and low latency of HBASE, this paper realizes the lock service by using a single column of HBASE, and realizes the synchronization between multiple rows and tables by the two-phase commit protocol. The outage problem in distributed system is considered, and the exception handling mechanism is designed. At the same time, three isolation levels of read uncommitted read committed and repreatable read are implemented for HBASE to meet the different requirements of different types of applications for transaction isolation. On this basis, this paper designs and implements a futures trading simulation system, which is used to verify the transaction support mechanism. The platform uses HBASE as a backend database and supports concurrent access requests from hundreds or even thousands of users. Through this system, this paper verifies the availability and efficiency of the transaction support mechanism for the massive data of non-relational database. The verification results show that the application of the transaction support mechanism on non-relational databases can meet the requirements of transaction isolation and consistency for large-scale concurrent access, although the efficiency is somewhat reduced. But basically within acceptable limits.
【学位授予单位】：上海交通大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP311.13

【参考文献】