分布式存储系统可用性与一致性研究

发布时间：2018-01-15 06:28

本文关键词：分布式存储系统可用性与一致性研究　出处：《华中科技大学》2013年硕士论文　论文类型：学位论文

【摘要】：分布式系统中的一致性，可用性，分区容忍性，可扩展性等问题一直是困扰分布式存储系统设计实现的几个难题，很多分布式文件系统和存储系统的设计目标都是如何提升系统的性能，，可扩展性等。系统研究如何在廉价的普通设备上，构建高可用的分布式存储服务。系统为一分布式KV（key value）存储系统，使用外部高可用性系统部件Zookeeper进行Leader选举，使得在Leader节点失效时能够自动重新选举一新的Leader继续提供服务。采用类Paxos多数派协议实现分布式副本，实现多副本之间的强一致性，在三个副本的情况下，确保由于各种异常比如网络，磁盘，进程故障等导致一个副本不可用情况下系统继续正常运行。Leader（主）节点与Follower（从）节点能够在宕机启动后自动恢复，并且确保恢复后系统写入成功数据不会丢失。系统支持每份数据三副本，支持对数据进行哈希分区提高系统扩展性，分区后系统能够实现很好的线性扩展性。系统向客户提供可选的一致性服务，其中一种为强一致性服务，写入成功后即可立即被所有客户端读取到。另一种为较实用的单调一致性，保证不会读到比之前读到数据更老的数据。测试表明系统使用的分布式副本协议开销较小，并且在通过将网络与磁盘操作并行化，多请求执行之间并行化以及将所有的写操作首先转化为对日志文件的顺序操作等优化措施之后大大提高了系统的性能，写性能基本已经达到了磁盘所能提供性能的极限。类Paxos多数派分布式协议确实是分布式环境中提供高可用性服务的可行性较强的协议方案。能提供较好的一致性，可用性，并且有较好的分区容忍性。
[Abstract]:The problems of consistency, availability, partition tolerance and extensibility in distributed system are always difficult problems in the design and implementation of distributed storage system. Many distributed file systems and storage systems are designed to improve system performance, scalability, and so on. Build a highly available distributed storage service. The system is a distributed KV(key value) storage system. Use the external high availability system component Zookeeper for the Leader election. This makes it possible to automatically re-elect a new Leader to continue to provide services when the Leader node fails. Distributed replicas are implemented using the Paxos like majority protocol. Achieve strong consistency between multiple replicas, in the case of three replicas, ensure that due to various exceptions such as network, disk. Process failures, etc., cause the system to continue running normally when a copy is unavailable. The primary and follower nodes can automatically recover after a downtime startup. The system supports three copies of each data, and supports hash partitioning of the data to improve the scalability of the system. The system can achieve good linear expansibility after partitioning. The system provides the customer with optional consistency service, one of which is strong consistency service. The other is a more practical monotone consistency that guarantees that you won't read older data than you read before. The test results show that the distributed replica protocol used by the system is less expensive and is parallelizing network and disk operations. Optimization measures such as parallelizing the execution of multiple requests and converting all write operations first into sequential operations on log files greatly improve the performance of the system. Write performance has basically reached the limits of the disk can provide performance. Paxos majority distributed protocol is indeed a distributed environment to provide high availability service is more feasible protocol scheme. Consistency. Availability and good partition tolerance.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【引证文献】