一种高速数据存储方法的研究

发布时间：2018-10-17 12:23

【摘要】：信息技术的高速发展推动人类社会全面进入数字时代,突出表现在信息总量和交换量的迅猛增长,不断出现新兴的应用领域。传输、处理和存储如此庞大的数据量使存储系统面临前所未有的机遇和挑战。数据传输和数据存储是存储系统中的最重要的两个环节,本文以地震油气勘探仪器系统应用为背景,探索利用现有的普适设备最大性能的发挥网络带宽和磁盘驱动器吞吐量的方式。当前通用计算机的组织架构在处理网络通信和数据存储的过程中至少需要经历从网络到内存,再从内存到磁盘的两次数据拷贝,加之频繁的中断处理使其难以执行高速、持续、可靠的数据传输和存储。因此,本文在对当前的数据存储系统进行研究的基础上,设计了一种高速数据传输、存储的原型。该原型充分利用FPGA在并行事件处理上的优势,降低处理器在传输、存储协议上的负担和中断处理频率,实现了低功耗、小成本的数据存储系统；同时,也使处理器可以将更多的资源用于算法处理与数据运算。本论文共分为以下五章：第一章作为绪论,从存储需求出发,介绍了存储设备和存储系统的评价标准。在分析当前仪器对数据传输、处理和存储的基本需求以及现有通用计算机平台的吞吐瓶颈的基础上,通过不同方案的对比,提出基于硬件实现的存储原型设计。第二章从基本存储设备出发,介绍了扩展存储系统容量、提升存储系统性能、增强存储系统可靠性的方法。为了进一步理解存储系统的运行,讨论了存储通道和接口的实现方法。最后针对存储系统的设计方法,简要分析了数据存储优化的途径。第三章详细叙述了研制存储原型过程中硬件实现的技术路线与细节。分别从时钟、复位、接口与总线以及电源等多个方面阐明系统设计。重点介绍了在数据传输、存储过程中扮演重要角色的基于千兆以太网的可靠数据传输和Serial ATA接口的原理、分析和实现。第四章给出了已完成的初步测试结果,包括数字时钟同步所实现的同步精度,基于千兆以太网的可靠数据传输的吞吐量,Serial ATA接口与磁盘驱动器的链路初始化结果以及PCI Express的吞吐量,并对测试结果进行了讨论。最后是整篇论文的总结与展望,以本原型系统为平台,为继续开展数据传输、处理和存储的深入研究提出建议。
[Abstract]:The rapid development of information technology promotes the human society to enter the digital age in an all-round way, which is highlighted by the rapid growth of the total amount of information and the amount of exchange, and the emergence of new application fields. Transmission, processing and storage of such a large amount of data makes storage system facing unprecedented opportunities and challenges. Data transmission and data storage are the two most important links in the storage system. The background of this paper is the application of seismic oil and gas exploration instrument system. Explore ways to maximize network bandwidth and disk drive throughput using existing pervasive devices. In the process of processing network communication and data storage, the current organizational structure of a general-purpose computer needs at least two copies of data from network to memory, and then from memory to disk, together with frequent interrupt processing, which makes it difficult to carry out high-speed and continuous operations. Reliable data transmission and storage. Therefore, based on the research of current data storage system, this paper designs a prototype of high speed data transmission and storage. The prototype makes full use of the advantage of FPGA in parallel event processing, reduces the burden of processor in transmission, storage protocol and interrupt processing frequency, and realizes a low power and low cost data storage system. It also allows the processor to devote more resources to algorithm processing and data operations. This paper is divided into the following five chapters: the first chapter as an introduction, from the storage requirements, introduced the storage equipment and storage system evaluation criteria. On the basis of analyzing the basic requirements of data transmission, processing and storage of the current instrument and the bottleneck of the current general computer platform, a storage prototype design based on hardware implementation is put forward through the comparison of different schemes. In the second chapter, the methods of expanding storage system capacity, improving storage system performance and enhancing storage system reliability are introduced. In order to further understand the operation of storage system, the implementation method of storage channel and interface is discussed. Finally, aiming at the design method of storage system, the paper briefly analyzes the way of data storage optimization. In the third chapter, the technical route and details of the hardware implementation in the process of developing the storage prototype are described in detail. The system design is explained from clock, reset, interface and bus, power supply and so on. The principle, analysis and implementation of reliable data transmission based on gigabit Ethernet and Serial ATA interface which play an important role in data transmission and storage process are introduced. In chapter 4, the preliminary test results are given, including the synchronization accuracy of digital clock synchronization. The throughput, Serial ATA interface of reliable data transmission based on Gigabit Ethernet, the link initialization result of disk drive and the throughput of PCI Express are discussed, and the test results are discussed. Finally, the conclusion and prospect of the whole paper are presented. Based on this prototype system, some suggestions are put forward for further research on data transmission, processing and storage.
【学位授予单位】：中国科学技术大学
【学位级别】：博士
【学位授予年份】：2013
【分类号】：TP333

【共引文献】