分布式缓存技术在插入密集型系统中应用的研究与实现

发布时间：2018-06-26 00:07

本文选题：缓存 + 分布式　；参考：《电子科技大学》2012年硕士论文

【摘要】：随着国内外互联网的快速发展，分布式缓存技术也得到了广泛的应用。几乎所有的互联网公司都会使用到分布式缓存技术来提高系统的吞吐率，降低对数据库的访问频率，提高系统的快速响应能力。互联网中的应用系统都是属于查询密集型的，也就是数据的查询操作数远远大于其插入的操作数。所以分布式缓存技术在互联网中的应用基本上都是用于提高系统的查询效率的。但是虽然分布式缓存技术在查询密集型系统中得到了广泛的应用，但是并不意味着分布式缓存技术只能应用在查询密集型的系统中。该论文的研究方向就是把分布式缓存技术应用到插入密集型的系统中，以提高系统的最大吞吐率。本文依托于国家信息产业部电子发展基金项目“高性能、高可靠入侵防御系统研发与产业化”。针对项目中对系统吞吐量的需求，在该系统中引入分布式缓存技术。并重点研究了在插入密集型的系统中引入分布式缓存技术后的系统架构问题、分布式缓存的管理以及分布式缓存下实现数据冗余的方法，同时使用该论文中的研究成果最终实现了一个分布式缓存系统。本文的主要研究成果包括： 1)通过对插入密集型系统进行分析和研究，完成了在该系统中引入分布式缓存技术后的系统架构的设计。 2)在研究和分析一致性hash算法和虚拟节点技术在分布式缓存系统中的应用以后，提出了基于一致性hash的数据冗余技术。将数据按照一定的算法保存在不同的缓存服务上，，降低因系统故障造成数据丢失的风险。 3)在研究和分析常用的缓存分布式管理技术以后，提出了集中管理、本地询问的缓存管理技术。并详细的描述了管理数据初始化、管理数据修改的流程。 4)通过在系统的实现中使用非阻塞通信技术、基于JSON的序列化以及concurrent同步技术，实现了一个高速、高并发的分布式缓存系统。
[Abstract]:With the rapid development of the Internet at home and abroad, distributed caching technology has been widely used. Almost all Internet companies will use distributed cache technology to improve the throughput of the system, reduce the frequency of access to the database, and improve the system's ability to respond quickly. The application systems in the Internet are query intensive, that is, the query operands of data are much larger than the operands inserted. So the application of distributed cache technology in the Internet is basically used to improve the query efficiency of the system. But although distributed cache technology has been widely used in query intensive systems, it does not mean that distributed cache technology can only be applied in query intensive systems. The research direction of this paper is to apply distributed cache technology to insert intensive system to improve the maximum throughput of the system. This paper is based on the National Ministry of Information Industry Electronic Development Fund project "High performance, high reliability intrusion prevention system R & D and industrialization." To meet the requirement of system throughput in the project, distributed cache technology is introduced into the system. This paper also focuses on the system architecture after introducing the distributed cache technology into the inserted intensive system, the management of the distributed cache and the method of realizing the data redundancy under the distributed cache. At the same time, a distributed cache system is implemented using the research results in this paper. The main research results of this paper are as follows: 1) through the analysis and research of the insert intensive system, The design of the system architecture after introducing the distributed cache technology into the system is completed. 2) after the research and analysis of the application of the consistent hash algorithm and the virtual node technology in the distributed cache system, Data redundancy technology based on consistent hash is proposed. In order to reduce the risk of data loss caused by system failure, the data is stored on different cache services according to certain algorithms. 3) after researching and analyzing the commonly used distributed cache management technology, centralized management is proposed. Local query cache management technology. And described in detail the management data initialization, management data modification process. 4) through the use of non-blocking communication technology in the implementation of the system, JSON-based serialization and concurrent synchronization technology, the realization of a high-speed, High concurrency distributed cache system.
【学位授予单位】：电子科技大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP333

【引证文献】