分布式数据缓存机制的研究和设计

发布时间：2018-08-18 13:36

【摘要】：随着大数据时代的来临，分布式应用系统成为了发展的主流。作为提升应用性能的一种重要手段，分布式数据缓存已经成为当今分布式应用开发中不可或缺的一个环节，越来越受到企业的广泛关注和研究。当前出现的很多开源缓存产品在数据冗余备份、失败转移以及扩展性方面都存在一定的不足，而商用缓存系统则费用高且不透明。从而导致了开源的产品需要进行扩展才能符合应用需求，而收费的产品也不能以最有效率的方式大量使用。因此研究并设计出一种能解决这些问题的分布式数据缓存框架变得非常迫切。本文首先对现有分布式数据缓存技术进行了研究，重点分析了数据冗余备份、失败转移以及扩展的实现原理和现有技术特点，并指出了现有技术的不足及其造成这些不足的内在因素。在此基础上，提出了一种通用的、低成本的、可伸缩的分布式数据缓存框架。该框架基于已有的服务组件来进行设计，集成了各个服务组件的自身优势，也保证了整个框架的可扩展性和可用性，能够实现数据的冗余备份、故障转移和负载均衡等功能特性，同时还扩展了系统的缓存替换策略、分布式事务提交机制和网络I/O模型等额外特性。为了保证系统的可用性，框架的各个组件都实现了冗余机制，，具有较强的容错能力。系统的并发能力可以通过水平扩展得到不断地提升。最后搭建了模拟的测试环境，验证了其各项功能机制均能自动正确地运行，性能和并发能力都达到了设计目标，完全能满足实际应用的需求，具有一定的实用价值和工程意义。
[Abstract]:With the coming of big data era, distributed application system has become the mainstream of development. As an important means to improve application performance, distributed data caching has become an indispensable part of distributed application development, and has been paid more and more attention and research by enterprises. At present, many open source cache products have some shortcomings in data redundant backup, failure transfer and expansibility, while commercial cache systems are expensive and opaque. As a result, open source products need to be extended to meet application requirements, and charging products cannot be used in the most efficient way. So it is urgent to study and design a distributed data caching framework to solve these problems. In this paper, the existing distributed data cache technology is studied, and the implementation principle and existing technical characteristics of data redundancy backup, failure transfer and extension are analyzed. And pointed out the shortcomings of the existing technology and the internal factors that caused these deficiencies. On this basis, a universal, low-cost, scalable distributed data caching framework is proposed. This framework is designed based on the existing service components, integrates the advantages of each service component, and ensures the extensibility and availability of the whole framework. The functions of failover and load balancing are also extended such as cache replacement policy distributed transaction commit mechanism and network I / O model. In order to ensure the availability of the system, each component of the framework implements redundancy mechanism and has strong fault-tolerant ability. The concurrency ability of the system can be continuously improved through horizontal expansion. Finally, a simulated test environment is set up, which verifies that each functional mechanism can run automatically and correctly, the performance and concurrency ability reach the design goal, and it can completely meet the needs of practical application. It has certain practical value and engineering significance.
【学位授予单位】：湖南大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】