DOA下分布式数据注册中心高可用性研究与设计

发布时间：2018-01-02 11:41

本文关键词：DOA下分布式数据注册中心高可用性研究与设计　出处：《成都理工大学》2015年硕士论文　论文类型：学位论文

【摘要】：大数据时代,使得信息技术(IT,Information Technology)时代来到了数据技术(DT,Data Technology)时代,对数据的存储和管理已经成为数据技术的核心问题。面向数据的体系结构——DOA(Data Oriented Architect)应运而生,它为传统软件设计面对海量数据的挑战提供了一个很好的解决思路。数据注册中心是DOA的核心组件,它对各种类型的数据和广义数据进行登记注册,形成逻辑的数据资源池,方便应用对数据的访问。高频率的数据访问和高速的数据传输,使得数据注册中心在可靠性方面面临着严峻的挑战。针对分布式系统高可用性,目前业界有多种方案。它们的核心思想都是采用备机的方式来预防中心服务器的失效。这样的方案虽然可以在系统发生单点失效时,用备机为客户提供读写服务。然而往往需要人工的干预,及存在数据丢失的可能。因此,需要一个全新的思路来解决分布式系统中的单点故障问题。本文设计的基于一致性选举的双中心服务器数据注册中心机制,有效的满足系统的高可用性。基于Raft算法的多机一致性,解决多机状况下选举问题,并将数据保持同步。针对如何将数据注册中心的请求分发和宕机恢复,设计了三机实现双中心服务器数据注册中心的方法,有效的解决单点障碍和多机状态下的数据一致性问题。具体来讲,本文的主要工作如下:(1)从可靠性和可维护性两方面分析了影响分布式系统高可用性的因素。并按照此思路,调研了目前比较成熟的高可用性解决方案,并通过比较它们各自的优缺点,提出了针对分布式数据注册中心的高可用性方案。(2)研究了目前的分布式一致性算法,最终选用容易结合到实际中的Raft一致性算法,使用Raft一致性算法解决了三机NameNode的选主问题。(3)为了满足数据注册中心的高可用性指标,本文设计了一种双中心服务器的分布式数据注册中心模型,并为此模型设计了相关的数据结构和接口,最后改造了基于单NameNode的通信协议,以适用于三机NameNode的设计。(4)通过对选举方案和高可用性两方面的测试,验证了基于双中心服务器的数据注册中心的高可用性方案的可行性和有效性。本文为分布式数据注册中心的高可用性提供了新的思路和参考方法,同时基于Raft算法的多机选举和数据一致性同步方法有一定的理论和现实意义。主要的成果与创新点如下:(1)提出了一种基于Raft算法的多机一致性。Raft算法解决了多机状态下的选举问题,通过设计一种数据注册中心的数据一致性同步方法,有效的解决了多机状态下的数据一致性问题。(2)提出了一种三机实现双中心服务器数据注册中心的方法。双中心服务器数据注册中心主要解决了单点故障的瓶颈,选举后的Leader使用分布式数据注册中心数据一致性同步方法使每个Name Node上的数据保持一致。
[Abstract]:The big data era brought information technology to the data technology (DT) era. The Data Technology era. Data storage and management has become the core problem of data technology. The data oriented architecture (load data Oriented architect) came into being. It provides a good solution for traditional software design to face the challenge of massive data. Data registration center is the core component of DOA. It registers various types of data and generalized data. A logical data resource pool is formed to facilitate the application of data access, high-frequency data access and high-speed data transmission. It makes the data registration center face the severe challenge in the aspect of reliability, aiming at the high availability of the distributed system. At present, there are many kinds of schemes in the industry. Their core idea is to prevent the failure of the central server by the way of backup machine, although such a scheme can be used in the event of a single point of failure of the system. Using the standby machine to provide reading and writing services for customers. However, it often requires human intervention, and there is the possibility of data loss. A new way of thinking is needed to solve the single point fault problem in distributed system. This paper designs a data registration center mechanism based on consistency election. Effectively meet the high availability of the system. Based on the Raft algorithm of multi-machine consistency, to solve the multi-machine election problem. Aiming at how to distribute the request of the data registry and restore the data down, we design a method of realizing the data register center of the dual-center server on three computers. Effectively solve the problem of data consistency in single-point obstacle and multi-machine state. The main work of this paper is as follows: (1) the factors that affect the high availability of distributed systems are analyzed from the aspects of reliability and maintainability. By comparing their respective advantages and disadvantages, this paper proposes a high availability scheme for distributed data registry. Finally, the Raft consistency algorithm, which is easy to be combined into practice, is chosen. In order to meet the high availability index of the data registry, the Raft consistency algorithm is used to solve the master selection problem of the three-machine NameNode. In this paper, we design a distributed data registry model of dual central server, and design the related data structure and interface for this model. Finally, we transform the communication protocol based on single NameNode. To apply to the three-machine NameNode design. 4) to pass the election program and high availability of the two aspects of the test. Verify the feasibility and effectiveness of the high availability scheme of the data registry based on the dual central server. This paper provides a new way of thinking and reference for the high availability of the distributed data registry. At the same time, the method of multi-machine election and data consistency synchronization based on Raft algorithm has certain theoretical and practical significance. The main achievements and innovations are as follows: 1). A multi-machine consistency. Raft algorithm based on Raft algorithm is proposed to solve the election problem in multi-machine state. By designing a method of data consistency synchronization in the data registry. Effectively solve the problem of data consistency in multi-machine state. This paper presents a method to realize the data registration center of dual central server with three computers. The data register center of double center server mainly solves the bottleneck of single point fault. The elected Leader uses the distributed data registry data consistency synchronization method to keep the data on each Name Node consistent.
【学位授予单位】：成都理工大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：TP308

【参考文献】