基于Redis的结构化数据缓存系统的设计与实现

发布时间：2018-09-08 20:29

【摘要】：随着数据规模的不断扩大和用户数量的急剧增加,传统的结构化数据库访问面临的压力越来越大,提高MySQL等结构化数据库的读写性能成了一个亟待解决的问题。Redis内存数据库将所有数据放于内存中进行管理,大大提高了数据访问的性能,适用于数据缓存管理。在Redis中设计缓存策略实现结构化数据的缓存对减轻MySQL数据库访问压力和提高读写性能具有重要的理论意义和实用价值。在对结构化数据的数据特征、存取过程深入分析的基础上,使用结构化数据库和代理服务器的架构,并在代理服务器上部署了Redis内存数据库,在Redis上设计并实现了针对结构化数据的缓存系统。针对部分查询结果集过于庞大的情况,设计了一种基于用户查询频率、读写比例等行为的前缀缓存方法,缓存块中仅仅缓存前缀部分数据。当用户实际需求增加时,即时扩大缓存块,补充用户需求的数据;当用户对该块数据需求减少时,则采用惰性更新方案设置待更新标志,等待内存不足时统一进行内存空间的压缩。该方法不仅可以更好满足用户需求,还可以在内存不足时释放大量内存空间,降低缓存替换的概率。当缓存空间不足时,系统还提出了针对查询结果集类型的自适应缓存替换算法,将Hybrid替换算法进行改进并和LFU替换算法相结合,以更好满足结构化数据缓存的实际需求。该系统基于用户的实际查询需求,充分利用Redis内存数据库良好的读写性能,更高效地使用内存空间,从而提高了缓存系统的性能。通过设计实验对缓存系统的各个功能模块进行测试,结果表明,该系统通过前缀缓存策略和缓存块压缩方案,提升了缓存块个数,从而提高了缓存命中率和字节命中率,具有一定的应用价值。
[Abstract]:With the continuous expansion of the data scale and the sharp increase of the number of users, the traditional structured database access is facing more and more pressure. Improving the reading and writing performance of structured databases such as MySQL becomes a problem to be solved urgently. Redis memory database puts all data in memory for management, which greatly improves the performance of data access and is suitable for data cache management. It is of great theoretical significance and practical value to design a cache strategy in Redis to realize the cache of structured data in order to reduce the access pressure of MySQL database and improve the performance of reading and writing. Based on the deep analysis of the data characteristics and access process of structured data, the structure of structured database and proxy server is used, and the Redis memory database is deployed on the proxy server. A cache system for structured data is designed and implemented on Redis. In order to solve the problem that part of the query result set is too large, a prefix caching method based on the behavior of user query frequency, read / write ratio and so on is designed, in which only the prefix part of data is cached in the cache block. When the actual demand of the user increases, the cache block is expanded immediately to supplement the data required by the user; when the user needs the block data to be reduced, the inert update scheme is adopted to set the mark to be updated. When waiting for memory shortage, compression of memory space is unified. This method can not only better meet the needs of users, but also release a large amount of memory space when memory is out of capacity, thus reducing the probability of cache replacement. When the cache space is insufficient, the system also proposes an adaptive cache replacement algorithm for query result set type. The Hybrid replacement algorithm is improved and combined with the LFU replacement algorithm to better meet the actual needs of structured data cache. Based on the actual query requirements of users, the system makes full use of the good read and write performance of Redis memory database, and makes more efficient use of memory space, thus improving the performance of the cache system. The experimental results show that the system improves the number of cache blocks by prefix cache strategy and cache block compression scheme, thus improving the hit rate of cache and the hit rate of bytes. It has certain application value.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP311.13

【参考文献】