智能网络存储系统中故障检测与数据迁移技术研究

发布时间：2018-12-14 16:23

【摘要】：随着计算机和通信技术的发展，，数据产生和共享的速率呈指数级增长，存储技术逐渐成为IT基础设施的核心之一。传统的集中式存储系统已经无法满足大规模存储应用的需要，逐渐被分布式存储系统所取代。分布式存储系统在容量、I/O吞吐量和可扩展性方面体现出了优越的性能，但它的廉价组成部件的可靠性不高，容易引发数据丢失造成灾难性后果。因此，设计一个高效的容错机制对分布式存储系统具有重要的意义。本文基于智能网络存储系统（INSS）提出了以故障检测和数据迁移为核心的容错机制，旨在快速准确地检测出节点的运行状态，并提前迁移存储在故障节点上的数据，以保证数据的可靠性，提高存储系统的业务连续性。本文的主要任务是设计一种新型的累积式故障检测器，提高检测的精度，缩短检测时间，满足不同应用的QoS要求；并根据检测器的输出启动数据迁移模块，将疑似故障节点中的数据分配到其他正常节点中，实现迁移后的存储负载均衡。目前所做的研究工作和取得的研究成果包括： 1)建立节点容错组，结合基于层次的检测方法，制定出了一种灵活的可扩展的故障传播模型，缓解了leader节点的瓶颈问题。 2)根据累积式故障检测器的定义，设计出了一种新的故障检测器D-FD，从理论上证明了D-FD符合强完整性和最终强准确性。 3)建立了数据估值模型，便于找出热点数据并优先迁移。 4)建立了故障检测和数据迁移的关联，并设计了在节点/链路疑似故障时启动数据迁移模块的时机选择算法。 5)建立了基于负载的数据迁移模型，并设计了一种数据分配算法以保证迁移后各节点的负载均衡。
[Abstract]:With the development of computer and communication technology, the rate of data generation and sharing is increasing exponentially. Storage technology is becoming one of the core of IT infrastructure. Traditional centralized storage system has been unable to meet the needs of large-scale storage applications, and has been gradually replaced by distributed storage systems. Distributed storage systems exhibit superior performance in terms of capacity, I / O throughput and scalability, but the reliability of its cheap components is not high, which can easily lead to data loss. Therefore, it is very important to design an efficient fault-tolerant mechanism for distributed storage systems. Based on the intelligent network storage system (INSS), a fault tolerant mechanism based on fault detection and data migration is proposed in this paper, which aims to quickly and accurately detect the running state of the node and migrate the data stored on the fault node in advance. In order to ensure the reliability of data, improve the business continuity of storage system. The main task of this paper is to design a new cumulative fault detector to improve the accuracy of detection, shorten the detection time, and meet the requirements of different applications of QoS. According to the output of the detector, the data migration module is started, and the data in the suspected fault node is distributed to other normal nodes to achieve storage load balancing after migration. The current research work and research results are as follows: 1) establishing node fault-tolerant group, combining with hierarchical detection method, a flexible extensible fault propagation model is developed to alleviate the bottleneck problem of leader nodes. 2) according to the definition of cumulative fault detector, a new fault detector D-FDD is designed. It is proved theoretically that D-FD accords with strong integrity and final strong accuracy. 3) the data valuation model is established, which is convenient to find hot data and migrate preferentially. 4) the association between fault detection and data migration is established, and the timing selection algorithm is designed to start the data migration module when the node / link is suspected to be fault. 5) A data migration model based on load is established, and a data allocation algorithm is designed to ensure the load balance of each node after migration.
【学位授予单位】：华南理工大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP333

【相似文献】