基于DHT的数据冗余存储技术研究

发布时间：2018-09-19 13:00

【摘要】：随着网络与应用的发展，信息呈爆炸性增长趋势，数据达到前所未有的规模，大规模数据中心的存储压力正迅速增长。基于分布式哈希表(DHT)的对等存储系统由于其良好的扩展性以及快速的数据存取能力，已成为当前大规模数据存储研究的热点。基于DHT的数据存储系统具有动态性和异构性，导致数据随时可能丢失。因此，，如何保证DHT系统的高可用性成为研究DHT存储系统的关键问题。本文对基于DHT的存储系统的冗余机制和基于柯西矩阵的纠删码技术进行了研究，通过将文件分割后编码与单副本结合，给出了一种1N混合的冗余存储策略。本文的主要工作概括如下： 1.概述了分布式哈希表的设计和几种常见的DHT协议，详细分析了传统的副本和纠删码两种数据冗余机制，阐述了运用在DHT存储系统中的RS纠删码。 2.详细研究了基于柯西矩阵的RS纠删码，通过线性变换的思想给出了一种改进型柯西矩阵的构造算法。实验结果表明基于改进型柯西矩阵RS码比传统RS码的编码和解码效率都提高了10%左右。 3.给出了一种混合的冗余数据存储策略，该策略将数据文件进行分割后使用纠删码同时结合单副本，从而在相同的冗余度下有效地增加了系统的可用性，并在提高访问服务效率和减少了路由访问节点数的同时，降低了网络维护开销。仿真结果表明本文所给出冗余数据存储策略的有效性。
[Abstract]:With the development of network and application, the information is increasing explosively, the data has reached an unprecedented scale, and the storage pressure of large-scale data center is increasing rapidly. Peer-to-peer storage system based on distributed hash table (DHT) has become a hot spot in the research of large-scale data storage due to its good scalability and fast data access ability. The data storage system based on DHT is dynamic and heterogeneous, resulting in data loss at any time. Therefore, how to ensure the high availability of DHT system becomes a key problem in the research of DHT storage system. In this paper, the redundancy mechanism of storage system based on DHT and erasure code technology based on Cauchy matrix are studied. A 1N hybrid redundant storage strategy is proposed by combining file segmentation coding with single copy. The main work of this paper is summarized as follows: 1. This paper summarizes the design of distributed hash table and several common DHT protocols, analyzes in detail two kinds of data redundancy mechanisms of traditional replica and erasure code, and expounds the RS erasure code used in DHT storage system. 2. The RS erasure code based on Cauchy matrix is studied in detail, and an improved Cauchy matrix construction algorithm is given by the idea of linear transformation. The experimental results show that the coding and decoding efficiency of the modified Cauchy matrix RS code is about 10% higher than that of the traditional RS code. In this paper, a mixed redundant data storage strategy is presented, which divides the data file and combines the erasure code with a single copy, which increases the availability of the system effectively under the same degree of redundancy. At the same time, the efficiency of access service and the number of routing access nodes are reduced, and the network maintenance overhead is reduced. Simulation results show the effectiveness of the proposed redundant data storage strategy.
【学位授予单位】：西安电子科技大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP333

【相似文献】