Memcached访存性能优化

发布时间：2018-07-25 15:19

【摘要】：近些年来,社交网络、电子商务等交互网站迎来了巨大的增长。磁盘一直作为存储的主要介质,其I/O性能成为制约服务器性能的瓶颈。Memcached有效地解决了基于磁盘的数据库中CPU和磁盘I/O之间的主要矛盾,并且作为一个解决访问距离和提高源服务器能力瓶颈的有效方法得到了越来越广泛的应用。由于Memcached的重要作用以及其简单和开源架构,很多研究集中于Memcached的性能优化方面。本论文首先剖析了Memcached的软件架构并进行完整源码分析,整理Memcached的工作原理和流程,为后续分析和研究打下基础。在网络模型方面,Memcached基于C/S服务器模型,使用了Reactor事件处理模式和半同步/半异步并发模式;在内存管理方面,Memcached使用了slab内存池和LRU(Least Recently Used)实现内存的高效管理;哈希表被用于Memcached进行对象的快速查找;在命令分析和处理方面,Memcached使用有限状态机。其次本论文从Memcached架构和流程的角度,设计一个适合Memcached的数据压缩方案。从压缩率以及通用的角度,选择了Zlib数据压缩库中的DEFLATE算法进行压缩功能的实现。实验从对象数量、对象分布情况、命中率、系统延迟、延迟提速比等方面来说明数据压缩能够明显提高Memcached的访存性能。在使用数据压缩功能的情况下,Memcached只需70%的内存空间就可以达到原先的命中率。接着,本论文分析了Memcached的钙化现象及其产生原因,进一步讨论了新增的数据压缩功能对钙化的影响情况。在请求的分布情况随时间变化的情况下,钙化现象明显降低Memcached的命中率。本文在Linux的slab分配器思想的基础上提出了调整Memcached的slab分配机制;在Memcached采用数据压缩功能后而数据特征变化的情况下,提出最小影响因子选择算法来帮助Memcached实现内存回收过程中的slab选取问题;两者相结合,可以有效地解决压缩情况下的钙化问题。通过实验测试验证这种方案,并且与其他的解决方法和策略的效果进行比较,效果明显。这种方案有效降低压缩情况下钙化问题对Memcached的命中率的影响,提升了10%的命中率。综上,本文从使用数据压缩提高内存存储密度和解决钙化问题提高内存使用率的角度,实现对Memcached访存性能进行优化的目标。
[Abstract]:In recent years, social networks, e-commerce and other interactive sites ushered in a huge growth. Disk has always been the main medium for storage. Its I / O performance has become the bottleneck of server performance. Memcached has effectively solved the main contradiction between CPU and disk I / O in disk-based database. And as an effective method to solve the bottleneck of access distance and improve the capability of source server, it has been applied more and more widely. Due to the importance of Memcached and its simple and open source architecture, many studies have focused on the performance optimization of Memcached. In this paper, the software architecture of Memcached is analyzed firstly, and the whole source code is analyzed, and the working principle and flow of Memcached are sorted out, which lays a foundation for further analysis and research. In network model, Reactor event processing mode and semi-synchronous / semi-asynchronous concurrent mode are used based on C / S server model, and slab memory pool and LRU (Least Recently Used) are used to manage memory efficiently in memory management. Hash tables are used for fast object lookup in Memcached, and finite state machines are used for command analysis and processing. Secondly, from the point of view of Memcached architecture and flow, this paper designs a data compression scheme suitable for Memcached. From the point of view of compression ratio and general purpose, the DEFLATE algorithm in Zlib data compression library is selected to realize the compression function. From the aspects of object number, object distribution, hit ratio, system delay, delay speed ratio and so on, the experiment shows that data compression can obviously improve the memory access performance of Memcached. With data compression, it takes only 70% of memory space to hit the original hit rate. Then, this paper analyzes the calcification phenomenon of Memcached and its causes, and further discusses the effect of the new data compression function on calcification. When the distribution of requests varies with time, calcification can significantly reduce the hit ratio of Memcached. Based on the idea of slab allocator of Linux, this paper puts forward the slab allocation mechanism of adjusting Memcached, and when the data compression function is adopted in Memcached, the data characteristics change. A minimum influence factor selection algorithm is proposed to help Memcached realize the slab selection problem in the process of memory recovery, and the calcification problem under compression can be effectively solved by combining the two methods. The experimental results show that the scheme is effective compared with other solutions and strategies. This scheme effectively reduces the impact of calcification on the hit ratio of Memcached under compression, and increases the hit ratio by 10%. In summary, this paper aims to optimize the memory access performance of Memcached from the point of using data compression to improve memory density and to solve the problem of calcification to improve memory utilization.
【学位授予单位】：深圳大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP311.13

【相似文献】