协同内核KVSmith的设计与实现

发布时间：2018-05-11 20:06

本文选题：协同内核 + 数据一致性　；参考：《南京大学》2013年硕士论文

【摘要】：计算与通信技术的高速发展加速了互联网应用在人们日常生活中的普及,而在数据中心内构建高效可靠的数据中心应用系统是支持高质量互联网应用的重要保障。数据中心中主要包括三类软件实体：数据中心应用、协同服务、协同内核。多个数据中心应用共同为用户提供服务,它们之间必须进行有效的合作。协同服务为数据中心应用的协调与合作提供了有效支持。协同服务可以有很多种形式,而协同内核的主要目标是用来支撑这些不同形式的协同服务的构建。协同内核作为一个通用模块,提供了一组用于构建协同服务的接口。基于协同内核,我们可以高效地构建各种不同的协同服务。协同内核是构建数据中心应用的一个关键元素,我们希望设计出一个同时满足数据一致性,系统可用性,以及容忍网络分隔的协同内核。但是理论分析表明上述三个性质不能同时满足。对于协同内核而言,为了确保协同的正确完成,数据的一致性必须被满足。而同时网络的分隔难以避免,系统必须对网络分隔具有充分的容忍度。因此对协同内核而言,在必要的时候牺牲系统的可用性是一个必然的选择。这样,构建协同内核的主要挑战在于如何在同时满足数据一致性和网络分隔容忍性的前提下,尽量保证协同内核的可用性。针对上述问题,本文着重研究协同内核构建技术。具体而言,本文的主要贡献包括：首先,本文给出协同内核KVSmith的设计与优化。KVSmith的外在表现形式是一个键值存储服务,它的主要原理是基于多台机器之间的容错协商,保证存储数据的一致性和整个系统的网络分隔容忍性,由此完成协同内核的功能。在上述设计的基础上,KVSmith采用三个优化技术提升系统的性能：并行协商优化、选主优化和本地读优化。这些优化技术是提升KVSmith性能的关键。其次,协同内核的主要目标是支撑高效构建不同的协同服务。本文选择了两个典型的协同服务-锁服务和选主服务-作为应用案例,讨论了如何基于KVSmith协同内核来构建具体的协同服务。基于KVSmith提供的键值数据模型的接口,本文给出了锁服务和选主服务的具体实现。最后,本文实现了KVSmith原型系统,并且对KVSmith的性能进行了全面的实验评估。实验验证的结果表明,在不同类型不同数量的系统负载下,KVSmith均具有较好的性能,并能较快地从节点失效中恢复。
[Abstract]:The rapid development of computing and communication technology accelerates the popularization of Internet application in people's daily life, and the construction of efficient and reliable data center application system in data center is an important guarantee to support high-quality Internet application. The data center mainly includes three kinds of software entities: data center application, cooperative service and cooperative kernel. Multiple data center applications provide services to users together, and they must cooperate effectively. Collaborative services provide effective support for the coordination and cooperation of data center applications. There can be many forms of collaborative services, and the main goal of the cooperative kernel is to support the construction of these different forms of collaborative services. As a common module, the cooperative kernel provides a set of interfaces for building collaborative services. Based on the cooperative kernel, we can build various cooperative services efficiently. The cooperative kernel is a key element in building data center applications. We hope to design a cooperative kernel that can satisfy data consistency, system availability, and tolerate network separation at the same time. But the theoretical analysis shows that the above three properties can not be satisfied at the same time. For the cooperative kernel, in order to ensure the correct completion of the collaboration, the consistency of the data must be satisfied. At the same time, the separation of network is inevitable, the system must have sufficient tolerance to network separation. Therefore, it is an inevitable choice for cooperative kernel to sacrifice the availability of the system when necessary. In this way, the main challenge of constructing cooperative kernel is how to ensure the availability of cooperative kernel while satisfying both data consistency and network separation tolerance. In view of the above problems, this paper focuses on the cooperative kernel construction technology. In particular, the main contributions of this paper are as follows: first, this paper presents the design and optimization of the collaborative kernel KVSmith. The external representation of .KV Smith is a key-value storage service, whose main principle is based on fault-tolerant negotiation between multiple machines. To ensure the consistency of storage data and the network separation tolerance of the whole system, the function of cooperative kernel is completed. Based on the above design, KV Smith uses three optimization techniques to improve the performance of the system: parallel negotiation optimization, selection master optimization and local read optimization. These optimization techniques are the key to improve KVSmith performance. Secondly, the main goal of cooperative kernel is to support efficient construction of different cooperative services. In this paper, two typical collaborative services, lock service and master service, are selected as application cases, and how to construct concrete cooperative service based on KVSmith cooperative kernel is discussed. Based on the interface of key data model provided by KVSmith, this paper presents the implementation of locking service and selecting master service. Finally, the KVSmith prototype system is implemented and the performance of KVSmith is evaluated. The experimental results show that KV Smith has better performance under different types of system loads and can recover from node failure quickly.
【学位授予单位】：南京大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP308

【相似文献】