容错分布式存储系统扩容机制研究

发布时间：2018-11-19 09:06

【摘要】：当今大规模分布式存储系统采用冗余存储的方式来维持数据的可用性。冗余信息产生方式有复制和纠删码。纠删码相对于复制,因提供相同的容错能力所需的存储开销大大降低,而被越来越多的存储系统所采用。另一方面,数据的快速增长以及用户对系统容量和性能需求的不断提高导致当前构建存储系统经常出现存储能力和带宽资源不足的情况。当应用需求超出系统能力,需要增加存储资源,并将部分数据迁移到新的存储设备上来缓解压力,这一操作被称作存储系统扩容。因此,研究基于纠删码的分布式存储系统扩容机制,对云存储以及数据中心背景下的数据存储具有重要意义。本文从设计纠删码存储系统的扩容算法、调度在线扩容过程中的用户I/O请求与系统I/O请求、优化扩容后的用户访问性能三个维度出发,研究分布式存储系统的扩容机制,主要研究内容与贡献如下：(1) Cauchy Reed-Solomon (CRS)扩容问题研究随着当前存储系统对容错要求的逐渐提高,考虑容任意错的CRS编码的扩容问题愈发重要。CRS编码主要适用于由众多存储节点以及互联网络组成的分布式存储系统(例,CleverSafe, OceanStore)。扩容过程需要迁移部分数据到新的存储设备,同时需要更新校验。数据迁移与校验更新带来的存储I/O与网络传输带宽开销直接影响扩容过程中的系统性能。本文研究了基于CRS编码的分布式存储系统的扩容问题,通过第一步设计扩容后的编码矩阵,第二步设计扩容过程中的数据迁移方案,第三步利用校验解码部分数据的思想进一步优化数据迁移过程,为CRS系统扩容设计了一个三阶段优化扩容算法。理论分析表明,本文的三阶段优化扩容算法相对于基本扩容算法,能有效逐步地减少CRS系统扩容过程中的系统I/O与网络传输带宽。通过在实际的分布式文件系统中部署CRS三阶段优化扩容算法,并与基本扩容算法进行广泛实验对比,本文证实了算法在单线程以及多线程架构下的有效性与实用性。(2)在线扩容问题研究在实际存储系统中,大多数上层用户级应用都要求系统提供7x24小时的在线服务。因此,当存储系统进行在线扩容的时候,用户的I/O请求和迁移的I/O请求相互竞争,势必影响扩容过程中的用户和迁移的响应时间性能。·然而,已有的扩容算法在设计之时都很少考虑用户I/O请求,在线扩容过程中的用户和迁移的响应时间性能势必降级。本文基于此问题,为已有众多的扩容算法设计了一个在线扩容优化机制Popularity-based Online Scaling (POS)。本文的在线扩容优化机制POS结合实际系统中用户访问的两个特征,即：数据热度和数据局部性,通过将原有存储空间划分为多个区域,并记录每个区域的热度(主要以访问频度为指标),从而改变扩容顺序,优先迁移热度高的区域,进一步利用数据局部性来更好地响应用户的读、写请求,同时可以减少用户访问对迁移性能的影响。POS可以看作一个插件,垂直地应用在已有众多的扩容算法之上,提高在线扩容性能。通过在实际的磁盘模拟器中部署POS,并与已有的RAID-0扩容算法FastScale开展广泛实验对比,本文证实了POS相对于传统扩容算法能显著提高在线扩容过程中的用户以及迁移的响应时间性能。(3)扩容后读、写性能优化研究存储系统扩容必须兼顾扩容过程中性能与扩容结束后用户读、写操作性能。一方面,扩容过程中的系统I/O开销越大,扩容时间窗口越长,对于扩容过程中的迁移与用户的响应时间性能影响越大：另一方面,扩容结束后,必须服务正常的用户读、写操作,扩容后的用户访问性能亦为重要。然而,已有的扩容算法主要考虑最小化扩容过程中的数据迁移量,并未考虑优化扩容后的用户读、写操作性能。由于扩容过程改变了系统的数据布局,所以,扩容过程直接影响扩容结束后正常的用户访问性能。因此,本文从扩容过程出发,考虑设计好的数据迁移方法。本文以RAID-0扩容为例,设计一种新的扩容算法PostScale。 PostScale实现了扩容过程中的最小化数据迁移量,在此约束条件下,保证了扩容结束后的连续数据块的最大化分散放置。通过如此设计,扩容时间窗口得以缩小,同时扩容结束后的用户读、写请求能利用存储系统最大的并发访问性能。模拟实验表明,PostScale相对于传统的两种RAID-0扩容算法round-robin、 FastScale皆有优势,PostScale能大大缩小round-robin的扩容时间窗口,亦能有效提高FastScale的扩容结束后用户读、写响应时间性能。本文的PostScale可以进一步延伸应用于RAID-5系统扩容、基于Reed-Solomon编码的分布式存储系统扩容,改进扩容后的用户访问性能。
[Abstract]:Today's large-scale distributed storage systems use redundant storage to maintain data availability. The redundant information generation mode has the copy and deletion codes. the storage overhead required for providing the same fault-tolerant capability is greatly reduced with respect to the replication, and is used by an increasing number of storage systems. On the other hand, the rapid growth of data, as well as the user's increasing system capacity and performance requirements, often result in the current build-up of storage systems with low storage capacity and insufficient bandwidth resources. When application requirements exceed system capabilities, the storage resource needs to be increased and some of the data is migrated to the new storage device to relieve the pressure, which is known as the storage-system expansion. Therefore, it is of great significance to study the capacity expansion mechanism of the distributed storage system based on the erasure code, and it is of great significance to the cloud storage and the data storage in the background of the data center. This paper studies the expansion mechanism of the distributed storage system from the three dimensions of the system I/ O request and the system I/ O request and the user's access performance after the expansion, and the main research contents and contributions are as follows: (1) The research of the capacity expansion of the Cauchy Reed-Solomon (CRS) is becoming more and more important as the current storage system is improving the fault tolerance. CRS encoding is mainly applicable to a distributed storage system (e.g., CleverSafe, OceanStore) consisting of a number of storage nodes and the Internet. The expansion process requires the migration of part of the data to the new storage device, while the check needs to be updated. The storage I/ O and network transmission bandwidth overhead brought by the data migration and check update directly influence the system performance in the expansion process. In this paper, the expansion of the distributed storage system based on CRS is studied, the first step is to design the expanded coding matrix, the second step is to design the data migration scheme in the expansion process, and the third step further optimizes the data migration process by using the idea of the data of the check and decoding part. In this paper, a three-stage optimization and expansion algorithm is designed for the expansion of CRS system. The theoretical analysis shows that the three-stage optimization expansion algorithm in this paper can effectively reduce the system I/ O and network transmission bandwidth in the expansion process of the CRS system with respect to the basic capacity expansion algorithm. In this paper, the validity and practicability of the algorithm under the single thread and multi-thread architecture are verified by deploying the CRS three-stage optimization expansion algorithm in the actual distributed file system and comparing with the basic capacity expansion algorithm. (2) On-line capacity expansion is studied in the actual storage system. Most upper-level user-level applications require the system to provide an online service of 7x24 hours. Therefore, when the storage system is expanded online, the I/ O request and the migration I/ O request of the user compete with each other, and the response time performance of the user and the migration in the expansion process is bound to be affected. However, the existing capacity expansion algorithm seldom takes into account the user I/ O request at the time of design, and the response time performance of the user and the migration in the on-line expansion process is bound to be degraded. In this paper, an on-line capacity expansion optimization mechanism, Popularity-based Online Scaling (POS), is designed for a number of expansion algorithms. The on-line capacity expansion optimization mechanism (POS) of this paper is based on two characteristics of user access in the actual system, namely, data heat and data locality, by dividing the original storage space into a plurality of areas, and recording the heat of each area (mainly taking the access frequency as an index), and the influence of user access on the migration performance can be reduced. The POS can be regarded as a plug-in, which can be applied vertically to a large number of expansion algorithms, so as to improve the on-line capacity expansion performance. By deploying the POS in the actual disk simulator, and carrying out extensive experimental comparison with the existing RAID-0 expansion algorithm FastScale, this paper proves that the performance of the response time of the user and the migration in the on-line expansion process can be improved significantly with respect to the traditional expansion algorithm. (3) After capacity expansion, read and write performance optimization study storage system expansion must take account of the performance of the expansion process and the user's reading and writing operation performance after the end of expansion. On the one hand, the greater the system I/ O overhead in the expansion process, the longer the expansion time window, the greater the impact on the migration and the user's response time performance during the expansion: on the other hand, after the expansion is over, the normal user read and write operation must be served, The user access performance after the expansion is also important. However, the existing capacity expansion algorithm is mainly concerned with minimizing the amount of data migration in the expansion process, and does not consider optimizing the user's reading and writing operation performance after the expansion. Because the expansion process changes the data layout of the system, the expansion process directly influences the normal user access performance after the expansion. Therefore, this paper, from the process of expansion, considers the design of the data migration method. In this paper, a new expansion algorithm, PostScale, is designed based on the expansion of RAID-0. PostScale realizes the minimum data migration in the expansion process, and under the constraint condition, the maximum dispersion and placement of the continuous data blocks after the expansion end is guaranteed. With such a design, the expansion time window is reduced, and the user read and write requests after the expansion end can utilize the maximum concurrent access performance of the storage system. The simulation results show that the PostScale has the advantages of both the traditional two RAID-0 expansion algorithms, round-robin and FastScale, and PostScale can greatly reduce the expansion time window of the round-robin, and can effectively improve the time performance of user read and write response after the expansion of the FastScale. PostScale in this paper can further extend to the expansion of the RAID-5 system, expand the distributed storage system based on Reed-Solomon coding, and improve the user access performance after the expansion.
【学位授予单位】：中国科学技术大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TP333

【相似文献】