用FPGA实现文件系统及其应用

发布时间：2019-01-24 09:46

【摘要】：当今社会,各种数据量的急剧膨胀使人们进入到大数据时代。大数据是指单个寻址单元可以容纳不确定的或任意尺寸的数据。规模非常巨大和复杂的数据集,对于动辄达到PB、EB或ZB级别的数据量,传统的数据存储管理机制处理起来将面临诸多新的难题。通过对大数据底层存储研究,寻求一种行之有效的方式来满足大数据高并发读写、海量数据的高效率存储与访问需求、存储容量可扩展和高可用性的需求,是本研究的主要课题。本研究针对大数据所具有的特性,从数据的底层介质存储、文件系统管理及界面访问形式等层面一一剖析,提出利用FPGA实现文件系统的方案,通过具体实例介绍其实现过程,仿真并调试。.具体如下：一、底层ATA主机控制器的设计及优化。针对实验室现有的ATA控制器进行重新设计与优化,完善Ultra DMA读写功能,归纳DOS、Windows与Linux操作系统中对底层硬盘读写机制,总结出底层硬盘接口的主要参数并对接口进行重新封装,使其易与上层文件系统接口。二、FPGA实现Ext2硬件文件系统。(1)在深入理解FAT 32及Ext 2文件系统的存储原理的基础上,抽取文件系统共性,归纳总结文件基本操作的流程：创建文件、读文件、写文件、删除文件、更新文件等；(2)硬件文件系统与底层ATA主机控制器直接接口,用VHDL描述Ext 2文件系统的各个功能模块及主要操作的状态机描述,代码通过调试与仿真；(3)给出两种方式实现文件系统的具体方案：状态机与定序器引擎,针对各自所具备的特点进行了详细的阐述。三、多模式远程访问的客户端实现。(1)B/S模式下通过WEB浏览器进行远程访问：设计前端HTML访问页面、编写CGI程序、在FPGA中的PetaLinux操作系统中添加Boa服务器、编译文件系统库等；(2)C/S模式下通过编写Socket应用程序,实现文件的远程存取过程。本研究在利用FPGA实现文件系统的具体实现方案中,提出采用RAM实现文件系统的基本数据结构的方法；将字符查找功能应用到硬件电路中,实现文件名的匹配；硬盘命令控制模块合理地控制发往硬盘的命令,提升硬盘的工作效率。用FPGA实现文件系统可在不通过CPU专门调度指挥下完成对数据及文件的读写操作,实现用户数据缓存与硬盘的直接交互,节省资源,缩短文件处理时间,将文件系统与硬盘控制器作为独立的资源节点可极大的增加系统的可扩展性,可为满足大数据需求、实现资源与服务的高效调配提供参考。
[Abstract]:Nowadays, the rapid expansion of all kinds of data makes people enter the era of big data. Big data refers to a single addressable unit that can accommodate data of uncertain or arbitrary size. Because the data set is very large and complex, the traditional data storage management mechanism will face many new problems when it comes to the amount of data which can easily reach the level of PB,EB or ZB. Through the research on big data's underlying storage, we seek an effective way to meet the needs of big data's high concurrent reading and writing, the efficient storage and access of massive data, the scalability of storage capacity and the high availability. Is the main subject of this study. According to the characteristics of big data, this paper analyzes the underlying media storage, file system management and interface access form of data, and puts forward a scheme to realize file system by using FPGA, and introduces its implementation process through concrete examples. Emulate and debug. The details are as follows: first, the design and optimization of the underlying ATA host controller. This paper redesigns and optimizes the existing ATA controller in laboratory, consummates the function of Ultra DMA reading and writing, and induces the reading and writing mechanism of the underlying hard disk in DOS,Windows and Linux operating system. The main parameters of the underlying hard disk interface are summarized and the interface is re-encapsulated to make it easy to interface with the upper file system. Second, FPGA implements Ext2 hardware file system. (1) on the basis of deeply understanding the storage principle of FAT 32 and Ext 2 file system, extract the common character of file system, sum up the flow of file basic operation: create file, read file, write file, Delete files, update files, etc. (2) the hardware file system is directly interfaced with the underlying ATA host controller, each function module of the Ext 2 file system and the state machine description of the main operation are described by VHDL. The code is debugged and simulated; (3) give two ways to realize the file system: state machine and sequencer engine. Third, the client implementation of multi-mode remote access. (1) remote access through WEB browser in B / S mode: design front-end HTML access page, write CGI program, add Boa server to PetaLinux operating system in FPGA, Compile file system library and so on; (2) in C / S mode, the remote access process of files is realized by writing Socket application program. In this paper, in the realization of file system with FPGA, a method of realizing the basic data structure of file system with RAM is put forward, and the function of character searching is applied to the hardware circuit to realize the matching of file name. The hard disk command control module reasonably controls the commands sent to the hard disk and improves the working efficiency of the hard disk. The file system with FPGA can read and write the data and files without the command of CPU, realize the direct interaction between the user's data cache and the hard disk, save the resources and shorten the processing time of the file. Taking the file system and the hard disk controller as independent resource nodes can greatly increase the scalability of the system and provide a reference for meeting the needs of big data and realizing the efficient allocation of resources and services.
【学位授予单位】：太原理工大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】