海量硬盘数据快速复制技术研究与实现

发布时间：2018-06-30 17:57

本文选题：快速复制 + 磁盘解析　；参考：《南京航空航天大学》2012年硕士论文

【摘要】：信息技术日新月异，必然要求信息传输速度更快更强，随着数字化地球的日趋形成，数字化信息存储量呈现爆炸式发展，海量磁盘随之出现，这对海量硬盘数据的快速复制提出了新的要求。本文在深入了解国内外有关数据快速复制技术的基础上，就目前数据快速复制应用场景与模式进行了深入的分析，针对当前国内外有关硬盘数据复制系统存在的弱点以及复制速率已经达到极限等问题，提出一种基于多块位多接口并发输出的数据复制策略，并就此展开研究。论文提出基于主机多个数据输出接口进行并发数据输出的数据复制机制，并围绕该目标展开相关研究并加以实现。主要做的工作如下：（1）构建了磁盘文件系统快速解析模型。通过对硬盘的物理结构特性、数据结构特性、磁盘文件系统、硬盘数据存储原理进行了深入分析，提出了磁盘文件系统快速识别技术、文件系统驱动技术，实现了磁盘文件系统的快速解析。（2）提出了“单模接口数据并发输出”与“多模接口数据并发输出”两种数据输出策略，并完成了数据输出的整体架构建设。设计了数据输出实行块位输出的策略模式，通过测试与实验得出最优块位划分策略表，并设计出进行多线数据传输、块位数据流向以及数据输出整体架构方案。（3）研究了块位划分算法、块位派发与接收算法，并对其进行测试，验证该策略与算法的可行性与有效性。提出数据散列存储与重组方案策略，并通过自定义数据元结构协议，对数据元散列存储进行分析，设计出数据元散列式存储方案，并对其进行了实现，，然后依据数据元散列存储协议，对基于散列式存储数据元的重组进行研究，设计并实现“快速重组”与“全息重组”两种散列数据重组模式。（4）在技术研究基础上实现了原型系统的开发，并进行了性能测试，达到了预期的目标。在对相关技术与应用策略深入研究的基础上，根据课题研究目标，对课题提出的海量硬盘数据快速复制系统的应用需求进行分析，进而对系统的硬件设计与软件集成进行了实现，最后对研制出的系统原型系统进行功能测试与性能分析，测试结果表明了系统的有效性，达到了预期的研究目标。
[Abstract]:With the rapid development of information technology, the speed of information transmission is bound to be faster and stronger. With the formation of digital earth, the amount of digital information storage presents explosive development, and mass disks appear. This puts forward new requirements for the fast copy of massive hard disk data. On the basis of deep understanding of domestic and foreign rapid data replication technology, this paper makes a deep analysis of the current data rapid replication application scenarios and patterns. Aiming at the weakness of hard disk data replication system at home and abroad and the limitation of replication rate, a data replication strategy based on multi-bit and multi-interface concurrent output is proposed and studied. This paper proposes a data replication mechanism for concurrent data output based on multiple host data output interfaces, and carries out related research and implementation around this goal. The main work is as follows: (1) A fast parsing model of disk file system is built. By analyzing the physical structure characteristic, data structure characteristic, disk file system and hard disk data storage principle of hard disk, the paper puts forward the fast recognition technology of disk file system, file system driving technology, The fast parsing of disk file system is realized. (2) two kinds of data output strategies, "single mode interface data concurrent output" and "multimode interface data concurrent output", are proposed, and the whole data output architecture is completed. The strategy mode of block bit output for data output is designed. The optimal block bit partition strategy table is obtained by testing and experiment, and the multi-line data transmission is designed. The whole architecture of block bit data flow and data output. (3) the block bit partition algorithm, block bit dispatch and receiving algorithm are studied and tested to verify the feasibility and effectiveness of the strategy and algorithm. The strategy of data hash storage and recombination is put forward, and the scheme of data element hash storage is designed and implemented by analyzing the data element hash storage through the self-defined data element structure protocol. Then, according to the hash storage protocol of data element, the recombination of data element based on hash storage is studied. Two kinds of hash data recombination modes are designed and implemented. (4) the prototype system is developed on the basis of technical research, and the performance test is carried out, and the expected goal is achieved. On the basis of deep research on related technology and application strategy, according to the research goal, the paper analyzes the application requirements of the mass hard disk data fast copy system put forward by the subject. Then the hardware design and software integration of the system are realized. Finally, the function test and performance analysis of the developed prototype system are carried out. The test results show that the system is effective and achieves the expected research goal.
【学位授予单位】：南京航空航天大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP333

【参考文献】