分布式缓存系统的内存利用率和并发性能优化机制

发布时间：2018-02-09 06:47

本文关键词： 分布式缓存系统内存利用率并发性能布谷鸟哈希乐观并发控制　出处：《华中科技大学》2013年硕士论文　论文类型：学位论文

【摘要】：Memcached是一个高性能的分布式内存对象缓存系统，它通过在内存中缓存对象来减少读取数据库的次数，提高数据库读写速度。随着web应用的高速发展，Memcached在缓存领域发挥越来越重要的作用，对Memcached内存利用率和并发性能的要求也越来越高，寻求同时满足应用服务和性能需求的优化机制，是目前亟待解决的问题。结合对Memcached工作负载特征分析，和不同系统内存利用率和并发机制的分析对比，，以偏重小对象键值存储和读操作为主的工作负载，难以兼顾内存利用率和高并发请求。据此提出了相应的解决方案：优化布谷鸟哈希算法，用其取代系统中默认的开链哈希算法，实现哈希表高占用，从而提高Memcached内存利用率；优化哈希表查询插入算法，基于标记值的查询和插入保持哈希表操作的缓存友好性，实现对哈希表的多读单写并发访问；基于计数器的乐观并发控制采用版本计数器来替换原系统全局锁，通过版本号的改变发现并发的替换操作，减小内部线程同步，确保数据一致性，允许大量并发访问，降低插入查询开销。基于Linux操作平台和cpu密集型计算，实现了Memcached的内存利用率和并发性能优化，在维持了内存高效率的前提下，提高了Memcached多读单写的高并发访问。系统测试采用符合zipf分布的测试数据集，存储相同记录，优化后的系统内存占用比原系统内存占用率减小20%；客户端请求读写高并发时，优化后的系统相较原系统，吞吐量至少提高2倍。
[Abstract]:Memcached is a high-performance distributed memory object caching system that reduces the number of database reads by caching objects in memory. With the rapid development of web applications, Memcached plays a more and more important role in the field of cache, and demands more and more high memory utilization and concurrent performance of Memcached. It is an urgent problem to seek optimization mechanism to meet both application service and performance requirements. Combining the characteristic analysis of Memcached workload, comparing with the memory utilization and concurrency mechanism of different systems, the workload, which is mainly focused on small object key value storage and read operation, is analyzed and compared with different system memory utilization and concurrency mechanism. It is difficult to take account of memory utilization and high concurrent request. Based on this, the corresponding solutions are put forward: optimizing the cuckoo hashing algorithm, replacing the default open-chain hash algorithm in the system to achieve high hash table occupancy, thus improving the Memcached memory utilization rate; The algorithm of hash table query insertion is optimized, and the cache friendliness of hash table operation is maintained based on the query and insert of tag value, and the multiple read single write concurrent access to hash table is realized. Optimistic concurrency control based on counter uses version counter to replace the global lock of original system, finds concurrent replacement operation by changing version number, reduces internal thread synchronization, ensures data consistency, and allows a large number of concurrent access. Reduce insert query overhead. Based on Linux operating platform and cpu intensive computing, the memory utilization and concurrency performance of Memcached are optimized, and the high memory efficiency is maintained. The system test adopts the test data set that accords with the zipf distribution, stores the same record, the optimized system memory occupation reduces 20% compared with the original system memory occupation, when the client requests the reading and writing high concurrency, the system test adopts the test data set which conforms to the zipf distribution, and stores the same record, the optimized system memory occupation reduces 20% compared with the original system memory occupation. The throughput of the optimized system is at least 2 times higher than that of the original system.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333.1

【参考文献】