纠删码集群存储的数据访问优化技术研究

发布时间：2019-04-21 07:08

【摘要】：近年来，随着信息化的高速发展，数据量呈爆炸式增长，分布式存储方式被广泛应用，同时数据可用性也得到了极大的重视。在此情况下，作为一种重要的冗余机制，纠删码被广泛应用于分布式存储系统以获得高可用性。但是，纠删码在读、写和失效恢复方面的代价较高，因此，为纠删码系统设计新的读、写和重构方案以提高系统性能具有重要的研究意义和应用前景。在基于纠删码的集群存储环境中，，分别针对读、写和重构提出了三种优化方案，分别称之为基于最小负载的大读优化方案、局部式小写更新优化方案和基于重定向的在线重构方案。在基于最小负载的大读优化方案中，首先结合纠删码集群系统的特点，定义了负载衡量基准，根据该基准，将集群系统中高负载节点的读请求转移到其他负载较低的节点上，最后解码出所需的数据，使得在平衡节点负载的同时，降低用户的访问响应时间。在局部式小写更新优化方案（PUS）中，充分利用存储节点的计算能力，将部分更新工作从更新节点转移至存储节点，减少由于更新所带来数据读、写和传输开销，有效缩短更新操作流程，不仅优化用户响应时间，而且减轻更新节点压力。实验结果表明，相比于传统更新方案，PUS能有效降低至少42%的小写更新时间。在基于重定向的在线重构方案（ROW-R）中，按照最小化用户I/O对重构I/O干扰的策略，将面向失效节点的全部写请求和部分读请求重新定位至其他存活节点，从而在物理上将用户工作流与重构工作流在一定程度上分开，通过充分利用磁盘在连续写时的高性能特性，加速重构进程。实验表明，ROW-R能有效优化用户响应时间达52%，并能够加速系统重构速度约6%。
[Abstract]:In recent years, with the rapid development of information technology, the amount of data has exploded, distributed storage has been widely used, at the same time, data availability has been paid great attention. In this case, as an important redundancy mechanism, erasure codes are widely used in distributed storage systems to obtain high availability. However, the cost of erasure codes in reading, writing and failure recovery is high. Therefore, designing new read, write and reconstruct schemes for erasure code systems has important research significance and application prospects in order to improve the performance of erasure codes. In the cluster storage environment based on erasure codes, three optimization schemes are proposed for reading, writing and refactoring respectively, which are called large read optimization schemes based on minimum load, respectively. Local lowercase update optimization scheme and on-line reconfiguration scheme based on redirection. In the large-reading optimization scheme based on minimum load, firstly, considering the characteristics of erasure code cluster system, the load measurement benchmark is defined. According to this benchmark, the read requests of high-load nodes in cluster system are transferred to other nodes with lower load. Finally, the desired data is decoded to reduce the user's access response time while balancing the node load. In the local lowercase update optimization scheme (PUS), taking full advantage of the computing power of the storage node, part of the update work is transferred from the update node to the storage node, thus reducing the data reading, writing and transmission overhead caused by the update. Effectively shorten the update operation process, not only optimize user response time, but also reduce the pressure of the update node. The experimental results show that PUS can effectively reduce the lowercase update time by at least 42% compared with the traditional updating scheme. In the redirected online refactoring scheme (ROW-R), all write requests and partial read requests for failed nodes are repositioned to other surviving nodes in accordance with the strategy of minimizing user I / O interference to the refactoring Ido. Thus, the user workflow is separated from the refactoring workflow to a certain extent, and the refactoring process is accelerated by making full use of the high performance characteristics of disk in continuous writing. Experiments show that ROW-R can effectively optimize the response time of users up to 52%, and can accelerate the reconfiguration speed of the system by about 6%.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】