一种面向纠删码技术的云存储可靠性机制

发布时间：2018-04-01 04:22

本文选题：云存储系统　切入点：可靠性　出处：《计算机应用与软件》2014年02期

【摘要】：在云存储中,数据可以分为三种类型:读频繁、写频繁以及冷数据。对不同类型的数据设置不同的编码机制:对读频繁数据采用多副本技术,频繁写的数据采用单文件编码方案,冷数据采用跨文件编码方案。在云存储系统中提出了一种新的纠删码技术的可靠性机制—魔方码,该编码易于实现,具备高的容错性并且相比多副本机制可以减少资源的消耗。以HDFS上的一个RAID包为基础,实现魔方码的原型,并添加BlaumRoth、RDP以对比测试。虽然各种编码各有优缺点,通过单机和集群测试RAID-5、BlaumRoth、RDP和魔方码的编译码速度,魔方码译码速度比现有最优RAID-6编码RDP码提高了一个数量级。同时,针对大规模分布式系统提出了一种三维纠删码,可以容任意7个错误。总之,测试结果显示魔方码相比三份副本技术,在减少一半资源消耗的同时可以保证高的可用性和可靠性。
[Abstract]:In cloud storage, data can be divided into three types: frequent reading, frequent writing and cold data. Different encoding mechanisms are set for different types of data. This paper presents a new reliability mechanism of erasure correcting code, which is the magic square code, which is easy to realize in cloud storage system. It has high fault tolerance and can reduce resource consumption compared with multi-copy mechanism. Based on a RAID package on HDFS, it implements the prototype of the magic code and adds Blaum Roth RDP to compare the test. Although each coding has its own advantages and disadvantages, By testing the encoding and decoding speed of RAID-5 Blaum Rothian RDP and magic cube code on single computer and cluster, the decoding speed of magic square code is improved by an order of magnitude compared with the existing optimal RAID-6 coding RDP code. At the same time, a three dimensional erasure correcting code is proposed for large-scale distributed system. All in all, the test results show that compared with three replicas, the code can reduce resource consumption by half while ensuring high availability and reliability.
【作者单位】：武汉体育学院体育工程与信息技术系;
【分类号】：TP333

【参考文献】