分布式存储中的纠删码容错技术研究

发布时间：2018-03-22 15:49

本文选题：分布式存储　切入点：纠删码　出处：《计算机学报》2017年01期 　论文类型：期刊论文

【摘要】：大数据规模上体量大和增长速度快的特点对存储系统的性能和可扩展性提出了严峻挑战.使用普通商用服务器构建的分布式存储系统服务能力强、成本低廉且极易扩展,在大数据的存储管理中得到了极为广泛的应用.分布式存储系统庞大的节点数量导致节点失效情况频发,必须采用一定的容错技术来保证数据可靠性.常用的容错技术主要包括多副本技术和纠删码技术两种.与多副本容错技术相比,纠删码容错技术能够以低得多的存储开销提供相同甚至更高的数据可靠性.随着近年来数据规模的爆炸式增长,纠删码容错技术受到了业界的广泛关注.该文综述了分布式存储中纠删码容错技术的研究现状.首先,介绍了纠删码容错技术的基本原理和概念,指出了纠删码容错技术在大规模分布式存储中面临的主要技术挑战;然后,从编码实现、纠删码设计、数据修复和数据更新等方面阐述了分布式存储中纠删码容错技术的研究进展,重点研究分析了各项关键技术的特点和局限性,并依据主要评价指标对现有纠删码的编码性能和修复性能进行了对比和分析;最后,基于最新研究动态指出了分布式存储中纠删容错技术未来的研究方向,包括同步编码实现技术、低冗余再生码设计和数据失效预测技术等.
[Abstract]:Big data's characteristics of large size and fast growth have posed a severe challenge to the performance and scalability of storage system. The distributed storage system built with ordinary commercial servers has strong service capability, low cost and easy to expand. Big data has been widely used in storage management. The large number of nodes in distributed storage system leads to frequent node failure. It is necessary to adopt certain fault-tolerant technology to ensure the reliability of data. The commonly used fault-tolerant techniques mainly include multi-replica technique and erasure code technology. Erasure code fault-tolerant technology can provide the same or higher data reliability with much lower storage overhead. Erasure code fault-tolerant technology has attracted wide attention in the industry. This paper summarizes the research status of erasure code fault-tolerant technology in distributed storage. Firstly, the basic principle and concept of erasure code fault-tolerant technology are introduced. The main technical challenges of erasure code fault-tolerant technology in large-scale distributed storage are pointed out, and then, from the coding implementation, erasure code design, In this paper, the research progress of erasure code fault-tolerant technology in distributed storage is described, and the characteristics and limitations of the key technologies are analyzed. The coding performance and repair performance of existing erasure codes are compared and analyzed according to the main evaluation indexes. Finally, the future research direction of erasure fault-tolerant technology in distributed storage is pointed out based on the latest research trends. It includes synchronous coding technology, low redundancy regenerative code design and data failure prediction technology.
【作者单位】：国防科学技术大学并行与分布处理国家重点实验室;国防科学技术大学计算机学院;
【基金】：国家自然科学基金(61379052) 国家重点研发计划项目(2016YFB1000101) 国家“八六三”高技术研究发展计划项目(2013AA01A213) 湖南省自然科学杰出青年基金项目(14JJ1026) 高等学校博士学科点专项科研基金资助课题(20124307110015)资助~~
【分类号】：TP333

【相似文献】