基于Chunk Folding的多租户云数据存储缓存管理机制

发布时间：2019-06-06 20:59

【摘要】：随着网络技术的发展,以及外包计算和存储的大量涌现,一种新的计算模式---云计算,正在逐渐兴起。所谓云计算,是指通过网络方便按需地访问可配置的共享计算资源,如网络、存储、内存、应用等,其计算资源的供给和释放不需要或只需很少的人工参与。就目前而言,SaaS(Software as a Service)是公认的云计算的最好的实现形式。在SaaS模式下,服务提供商需要存储成千上万租户的数据,而单独为每一个租户分配一个数据库实例需要大量的资源,而事实上任一数据库实例在大多数情况下的使用率是极低的,这就导致了大量的资源浪费。针对此问题,共享数据库共享存储模式被提出来用以解决具有相似存储模式的数据库实例的资源浪费,如Universal Table, Pivot Table, Chunk Folding等,而为了减少那些不具备相似存储模式的数据库实例的资源浪费,数据库合并(Database Consolidation)被提出来用于进一步减少数据库实例的数量,从而获取规模经济效益。共享存储模式和数据库合并可以极大程度的减少数据库实例的数量,从而降低资源浪费,但是基于传统数据库搭建的多租户数据库的缓存管理机制存在着以下不足或挑战： (1)数据块缓存替换单元的局限性。传统数据库缓存机制以数据块作为缓存单元,而多租户共享存储架构下,任一数据块均包含了大量其他租户的无关数据,采用数据块作为缓存单元导致大量缓存资源的浪费。 (2)租户间缓存资源分配的盲目性。传统数据库缓存机制缺乏多租户的概念,对于来自租户的请求,传统缓存机制会从提高数据库整体性能的角度进行缓存管理,这就会导致租户间资源分配的极为不合理,如高频访问租户抢占低频访问租户的资源,使得低频访问租户的SLA响应时间需求得不到保障,这与云计算环境下的弹性以及按需分配特性相违背。 (3)云缓存资源分配缺乏有效的分配机制。云计算环境下,为获取良好的伸缩性以及达到负载均衡,租户数据被划分到多个数据节点进行存储,如何确定各节点缓存内容使得：(a)租户的SLA响应时间得到满足,(b)云缓存效益(I/O减少次数)尽可能高,云缓存资源消耗尽可能少,(c)各子节点的I/O负载均衡。本文针对云计算环境下多租户数据库缓存管理机制的上述问题和挑战,结合Chunk Folding共享存储模式的特性,从缓存替换单元,多租户特性及云缓存资源协调分配等几方面着手,提出了适应负载的动态缓存单元生成机制,缓存单元I／0估价模型,多租户云缓存资源分配机制。本文的主要工作及成果包括： (1)提出了基于Chunk Folding的动态缓存替换单元的生成机制。该机制以租户的请求负载和租户后台Chunk Folding共享存储架构的物理存储模式作为输入,输出一系列列(集)缓存替换单元,将该缓存替换单元取代传统数据库的数据块缓存替换单元可以大幅度降低缓存消耗,提升缓存利用率。 (2)给出缓存单元的I/0效益估价模型。该模型结合数据库(MySQL)查询优化器的执行计划以及Chunk Folding的特性,得出每一缓存替换单元的I／0效益,并用此效益值与缓存替换单元所占缓存空间的比值作为该缓存替换单元的I/0效益率(半命中率),此外为实现云数据节点I/O负载均衡,结合当前节点的I/0负载状况,对每一缓存替换单元的I/0效益率进行加权修正,并将其作为衡量是否缓存该替换单元的标准,取代传统的命中率作为缓存与否衡量标准,使得缓存整体效益得到提升。 (3)给出两种缓存分配策略,租户级和系统级缓存分配策略。采用贪婪算法迭代选择缓存单元进驻内存,并对相关缓存单元的效益指标进行修正。通过租户级缓存分配策略实现租户缓存分配的在线动态调整,通过系统级缓存分配策略实现系统缓存松弛操作从而降低系统整体缓存消耗。针对基于传统数据库搭建的多租户数据库在缓存管理机制存在的不足和新挑战,本文给出了相应的解决机制---多租户云数据存储缓存管理机制(Multi-tenant Memory Management for Cloud data storage, M3C),该机制依据租户SLA目标按需为多租户分配缓存,降低云缓存资源消耗的同时并达到负载均衡。
[Abstract]:With the development of network technology and the emergence of outsourcing computing and storage, a new computing model--cloud computing, is emerging. The so-called cloud computing means that the configurable shared computing resources, such as network, storage, memory, application, etc., are conveniently accessed through the network, and the supply and release of computing resources are not required or require little manual participation. As far as is concerned, SaaS (Software as a Service) is the best form of implementation of recognized cloud computing. In SaaS mode, service providers need to store data for thousands of tenants, while allocating a single database instance for each tenant requires a large amount of resources, and virtually any database instance is very low in most cases And this leads to a lot of resource waves. Fee. For this issue, a shared database shared storage mode is proposed to address resource waste such as Universal Table, Pivot Table, Chunk Foling, and so on for database instances with similar storage patterns, while in order to reduce resource waves for database instances that do not have similar storage patterns The Database Consolidation is proposed to further reduce the number of instances of the database to get economies of scale The shared storage mode and the database combination can greatly reduce the number of database instances, thus reducing the resource waste, but the cache management mechanism of the multi-tenant database built on the basis of the traditional database has the following defects or disadvantages war: (1) data block cache replacement unit The traditional database caching mechanism uses the data block as a cache unit, and under the multi-tenant shared storage architecture, any data block contains irrelevant data of a large number of other tenants, and the data block is used as a cache unit to lead to a large amount of cache resources. waste. (2) Inter-tenant cache resource points The traditional database caching mechanism lacks the concept of multi-tenancy, and for the request from the tenant, the traditional caching mechanism can cache management from the point of improving the overall performance of the database, which can lead to the resource allocation among the tenants. Extremely unreasonable, such as high-frequency access to the tenant's resources to seize the low-frequency access to the tenant, so that the SLA response time requirements of the low-frequency access tenant are not guaranteed, which is in contrast to the flexibility in the cloud computing environment and on demand characteristic phase violation. (3) Lack of cloud cache resource allocation Effective distribution mechanism. In the cloud computing environment, to get good scalability and load balance, the tenant data is divided into a plurality of data nodes for storage, and how to determine the cache contents of each node makes it possible to: (a) the SLA of the tenant The time should be met, (b) the cloud cache efficiency (the number of I/ Os) is as high as possible, the cloud cache resources consume as little as possible, and (c) each sub-node I/ O load balancing. Based on the above-mentioned problems and challenges of the multi-tenant database cache management mechanism in the cloud computing environment, combined with the characteristics of the Chunk Foling shared storage mode, from the cache replacement unit, the multi-tenant feature and the cloud cache resource association In this paper, an adaptive load dynamic cache unit generation mechanism, a cache unit I/0 valuation model and a multi-tenant are proposed. The cloud cache resource allocation mechanism. This article The main work and achievements include: (1) proposed a dynamic based on Chunk Foling The mechanism uses the physical storage mode of the tenant's request load and the tenant background Chunk Foling to share the physical storage mode of the storage structure as input, outputs a series of column (set) cache replacement units, and replaces the cache replacement unit with the data block cache replacement unit of the traditional database. can greatly reduce the delay Save and improve cache utilization. (2) give a slow The I/ O benefit valuation model of the storage unit. The model is used to query the execution plan of the optimizer and the characteristics of the Chunk Foling to obtain the I/0 benefit of each cache replacement unit, and the ratio of the benefit value to the cache space occupied by the cache replacement unit is used as the cache replacement unit. The I/0 benefit rate (half-hit rate) of each cache replacement unit is weighted according to the I/ O load condition of the current node, and the I/0 benefit rate of each cache replacement unit is weighted and corrected as a standard for measuring whether to cache the replacement unit, instead of the traditional I/ O load condition of the current node, hit rate as a measure of cache or not So that the overall benefit of the cache is improved. (3) Two types of cache allocation are given. Slightly, tenant-level and system-level cache allocation policies. The performance index of the relevant cache unit is modified. The on-line dynamic adjustment of the tenant cache allocation is realized through the tenant-level cache allocation strategy, and the system cache is realized by the system-level cache allocation policy. In order to reduce the overall cache consumption of the system, this paper presents the corresponding solution mechanism--multi-tenant cloud data storage and cache management mechanism (Multi-Tenant Memory Management for Clou) for multi-tenant databases built on the basis of the traditional database. d data storage, M3C), which is based on the tenant's SLA target to allocate cache for multi-tenancy, lowe
【学位授予单位】：山东大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP333

【相似文献】