分级存储管理系统MLFS的研究与实现

发布时间：2018-02-05 02:30

本文关键词： 分级管理存储虚拟化 FV算法　出处：《国防科学技术大学》2013年硕士论文　论文类型：学位论文

【摘要】：随着互联网和移动通信技术的高速发展，人们对储存系统的容量和性能的要求也越来越高，尤其随着云计算的发展，大容量、高性能的存储管理系统成为云计算平台的重要组成，云存储技术正在逐步走向普及，这就导致了高性能存储设备高昂的成本与海量的数据存储容量之间的矛盾也随之日益突出；复杂的存储维护及扩容更进一步增加了存储系统的管理成本，分级存储管理一直以来就是存储管理的研究重点，可以有效的解决高性能与大容量之间的矛盾。本文对分级存储管理系统进行研究，并最终实现了分级存储管理系统(Multi-Level Files System，MLFS)原型，通过分级存储和统一视图虚拟化达到了合理分配系统资源，提高系统性能的目的。首先，提出一个分级存储管理模型，不仅将存储设备划分为高性能和海量存储两级，而且针对不同的分布式文件系统特点，分别使用了Lustre、GlusterFS等文件系统来管理，以提高性能和海量存储资源，并在上述分布式文件系统之上，开发了具有统一存储资源视图的虚拟化文件系统MLFS，并实现分级管理功能。其次，，分析了现有的数据迁移算法，对数据要素考虑较为全面、数据价值判定较为准确的FV算法进行深入研究分析，并在MLFS系统中实现该算法。最后，使用Disksim模拟器对FV算法和其他算法进行了比较，并在搭建的真实环境中实测了MLFS系统的单线程和多线程I/O性能，同时对分级管理的有效性进行了验证。
[Abstract]:With the rapid development of Internet and mobile communication technology, the storage system capacity and performance requirements are increasingly high, especially with the development of cloud computing, large capacity. High performance storage management system has become an important component of cloud computing platform, cloud storage technology is gradually becoming popular. This leads to the contradiction between the high cost of the high performance storage device and the massive data storage capacity. The complex storage maintenance and expansion further increase the management cost of storage system. Hierarchical storage management has been the research focus of storage management. It can effectively solve the contradiction between high performance and large capacity. This paper studies the hierarchical storage management system, and finally implements the prototype of hierarchical storage management system named Multi-Level Files system. Through hierarchical storage and unified view virtualization, the system resources are allocated reasonably and the system performance is improved. Firstly, a hierarchical storage management model is proposed, which not only divides storage devices into two levels: high performance and mass storage, but also uses Lustre according to the characteristics of different distributed file systems. File systems such as GlusterFS are managed to improve performance and mass storage resources on top of the above distributed file systems. A virtual file system MLFS with unified storage resource view is developed and hierarchical management is implemented. Secondly, the existing data migration algorithm is analyzed, and the FV algorithm, which takes the data elements into account and the data value is accurate, is studied and analyzed, and the algorithm is implemented in MLFS system. Finally, the Disksim simulator is used to compare FV algorithm with other algorithms, and the single-thread and multi-thread I / O performance of MLFS system is measured in the real environment. At the same time, the effectiveness of hierarchical management is verified.
【学位授予单位】：国防科学技术大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】