分布式存储系统中数据再生策略研究

发布时间：2018-03-16 21:06

本文选题：分布式存储系统　切入点：数据再生　出处：《复旦大学》2012年硕士论文　论文类型：学位论文

【摘要】：分布式存储系统通过利用大量存储节点提供海量数据存储服务。分布式存储系统需保存冗余数据,从而在有存储节点失效的情况下维持数据的完整性。为补偿存储节点失效造成的数据损失,当有存储节点失效时,应在系统中另一节点中再生其保存的数据。与保存副本数据相比,由MDS编码生成的冗余数据能够提供更高的对节点失效的抵抗能力,但典型的MDS编码在再生过程中的传输开销远高于副本数据。在MDS编码中,再生码能够达到存储开销和再生时网络带宽开销间的最优折衷曲线。目前对于存储编码及相应再生开销的研究集中于最小化再生时的网络带宽开销,但未考虑到实际再生中的时间及参与节点等开销。本文在不牺牲数据完整性的前提下,通过理论分析和基于真实数据的仿真,研究了提高再生过程性能的优化设计方案。我们首先回顾了冗余数据及其维持方面的工作进展。接下来,我们通过利用网络中不同链路上的带宽异质性,提出一种树形拓扑再生过程,显著降低了再生时间。进一步的,在对可用带宽非对称链路进行建模的基础上,本文提出了基于多棵树并行传输的再生过程。另一方面,基于再生过程中参与节点数量对再生效率的影响,我们通过将多个节点的再生过程进行流水线化处理,从而降低每轮再生过程中的参与节点数量。通过分析可知,流水线再生过程能够降低再生过程中所需时间及网络带宽开销,同时不牺牲数据完整性,且其要求的额外存储开销是低且实际的。我们的工作表明,流水线再生过程能够支持随机线性编码以及再生码,并支持立即修复和惰性修复。
[Abstract]:Distributed storage systems provide massive data storage services by using a large number of storage nodes. In order to compensate for the loss of data caused by the failure of the storage node, when there is a failure of the storage node, the integrity of the data is maintained. The stored data should be regenerated in another node in the system. The redundant data generated by the MDS code can provide a higher resistance to node failure than to save replica data. But typical MDS codes have much more transmission overhead than replica data during reproduction. In MDS coding, The regenerative code can achieve the optimal tradeoff curve between the storage overhead and the network bandwidth overhead when regenerated. At present, the research on the storage coding and the corresponding reproducing overhead is focused on minimizing the network bandwidth overhead when regenerated. However, it does not take into account the time of actual regeneration and the cost of participating nodes. In this paper, without sacrificing the integrity of data, through theoretical analysis and simulation based on real data, The optimal design scheme for improving the performance of regeneration process is studied. We first review the progress of redundant data and its maintenance. Then, we utilize the bandwidth heterogeneity of different links in the network. A tree topology regeneration process is proposed, which reduces the regeneration time significantly. Furthermore, based on the modeling of asymmetric links with available bandwidth, this paper proposes a regeneration process based on parallel transmission of multiple trees. Based on the effect of the number of participating nodes on regeneration efficiency during regeneration, we reduce the number of participating nodes in each round of regeneration by treating the regeneration process of multiple nodes with pipeline. Pipeline regeneration process can reduce the time and network bandwidth cost of the regeneration process without sacrificing data integrity, and the additional storage cost required is low and practical. Our work shows that, Pipeline regeneration process can support random linear coding and regenerative code, and support immediate repair and inert repair.
【学位授予单位】：复旦大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP333

【相似文献】