减少重建数据量的冗余编码技术研究

发布时间：2018-05-18 03:25

  本文选题：冗余编码 + 数据重建　； 参考：《计算机科学》2017年S1期

【摘要】：为防止硬件故障或机器宕机导致的数据丢失,冗余编码技术被广泛应用于分布式存储系统中来保证数据的可靠性。然而,传统的冗余编码技术,如里德-所罗门码,存在着重建数据量大的问题。副本技术在重建丢失数据时只需要读取和传输丢失的数据,而冗余编码需要读取和传输更大的数据量,从而消耗更多的磁盘I/O带宽和网络带宽。因此,基于冗余编码的分布式存储系统在重建数据时将消耗更长的时间,从而将整个系统长时间暴露在一种降级的模式下,进而增加了发生永久性数据丢失的风险。为解决这个问题,减少重建数据量的冗余编码技术不断被提出,然而只有这些冗余编码与传统的里德-所罗门码的比较,缺少它们在存储系统的综合比较。系统地从减少重建数据量等几个重要方面研究了这些减少重建数据量的冗余编码技术,从而为实际系统中采用合适的编码提供重要参考和依据。
[Abstract]:In order to prevent data loss caused by hardware failure or machine outage, redundant coding technology is widely used in distributed storage systems to ensure the reliability of data. However, the traditional redundant coding techniques, such as Reid-Solomon code, have the problem of large amount of reconstruction data. The replica technique only needs to read and transmit the lost data while redundant coding needs to read and transmit more data so as to consume more disk I / O bandwidth and network bandwidth. Therefore, distributed storage system based on redundant coding will consume longer time when reconstructing data, thus exposing the whole system to a degradation mode for a long time, thus increasing the risk of permanent data loss. In order to solve this problem, redundant coding techniques to reduce the amount of reconstructed data have been proposed. However, only these redundant codes are compared with the traditional Reed-Solomon codes, but they lack a comprehensive comparison in the storage system. This paper systematically studies these redundant coding techniques for reducing the amount of reconstructed data from several important aspects such as reducing the amount of reconstructed data, thus providing an important reference and basis for the adoption of appropriate codes in practical systems.
【作者单位】： 海军工程大学电子工程学院;华中科技大学计算机科学与技术学院;
【分类号】：TP333
，

本文编号：1904162