元数据存储单元设计与实现
发布时间：2018-07-18 16:27
【摘要】：随着信息的大发展数据呈现爆炸式增长，用于存储飞速增长的数据的分布式文件系统应运而生。目前主流的分布式文件系统多采用元数据与数据分离的管理机制，即将文件元数据和命名空间保存到元数据存储节点上，而向系统上传的数据则保存到数据节点上，以便高效、高扩展、安全、快速地存取数据。因此，，在整个分布式文件系统中，元数据的存储占据举足轻重的地位。 本文主要研究分布式文件系统中的元数据存储单元。通过对元数据高可用性技术的深入研究，在前人研究成果基础上，设计实现了Cstore分布式文件系统元数据存储单元，并对实现的存储单元进行了部分性能测试。主要工作如下： 研究了分布式文件系统中目前主流的文件系统以及这些系统的设计理念与设计原则，并对上述主流的文件系统的元数据管理的方式方法进行了重点研究，同时对元数据职责和元数据管理的方式进行了详细的阐述。 研究了分布式文件系统常用的系统组织模式，并详细的介绍了两大类设计模式：active/active模式和active/standby模式。对这两大类设计模式介绍的同时分析了这两大模式下的不同子模式的优点与缺点。同时介绍了常用的服务器编程I/O模型。 对Cstore文件系统进行了概括性的介绍，并对整个系统中所涉及到的关键技术进行了介绍与阐述，其中对元数据存储单元提供服务所面临到的问题提出了解决方案。如对数据安全和稳定服务方面提出的主从互备机制，面对如何确保用户上传的元数据快速定位到用户上传元数据定位元数据服务器，上传的文件如何快速定位到需要上传数据的数据服务器等技术。同时介绍了元数据规则服务器用到的一些关键技术。 总体介绍上述元数据存储单元的模块划分，然后详细给出各子模块的设计类图和数据流程图，并对图中所涉及到的类的作用加以详细说明和介绍。 对实现后的元数据存储单元集群进行了测试。结果表明该元数据存储单元集群能够高效稳定地对外提供元数据访问，性能达到了设计要求。
[Abstract]:With the rapid development of information, the distributed file system which is used to store the rapidly increasing data comes into being. At present, the mainstream distributed file system uses the management mechanism of separating metadata from data, that is, file metadata and namespace are saved to metadata storage node, and data uploaded to the system is saved to data node, so as to be efficient. High extension, secure, fast access to data. Therefore, metadata storage plays an important role in the whole distributed file system. This paper focuses on metadata storage units in distributed file systems. Based on the previous research results, the Cstore distributed file system metadata storage unit is designed and implemented, and the performance of the implemented memory unit is tested. The main work is as follows: the main file systems in the distributed file systems are studied, and the design concepts and principles of these systems are also discussed. The methods and methods of metadata management of the main file system mentioned above are studied in detail. At the same time, the responsibility of metadata and the way of metadata management are expounded in detail. This paper studies the common system organization patterns of distributed file systems, and introduces in detail two kinds of design patterns: active / active mode and active/standby mode. In this paper, the advantages and disadvantages of different sub-patterns in these two kinds of design patterns are analyzed. At the same time, I / O model of server programming is introduced. This paper gives a general introduction to Cstore file system, introduces and expounds the key technologies involved in the whole system, and puts forward solutions to the problems faced by metadata storage units in providing services. For example, the master-slave mutual backup mechanism for data security and stability services, how to ensure that the metadata uploaded by the user can be quickly located to the metadata server, Upload files how to quickly locate the need to upload data server and other technologies. At the same time, some key technologies used in metadata rule server are introduced. The module partition of the above metadata storage unit is introduced in general, and then the design class diagram and data flow chart of each sub-module are given in detail, and the functions of the classes involved in the diagram are explained and introduced in detail. The implementation of the metadata storage unit cluster was tested. The results show that the cluster of metadata storage units can provide metadata access efficiently and stably, and the performance meets the design requirements.
【学位授予单位】：电子科技大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333
本文编号：2132486
[Abstract]:With the rapid development of information, the distributed file system which is used to store the rapidly increasing data comes into being. At present, the mainstream distributed file system uses the management mechanism of separating metadata from data, that is, file metadata and namespace are saved to metadata storage node, and data uploaded to the system is saved to data node, so as to be efficient. High extension, secure, fast access to data. Therefore, metadata storage plays an important role in the whole distributed file system. This paper focuses on metadata storage units in distributed file systems. Based on the previous research results, the Cstore distributed file system metadata storage unit is designed and implemented, and the performance of the implemented memory unit is tested. The main work is as follows: the main file systems in the distributed file systems are studied, and the design concepts and principles of these systems are also discussed. The methods and methods of metadata management of the main file system mentioned above are studied in detail. At the same time, the responsibility of metadata and the way of metadata management are expounded in detail. This paper studies the common system organization patterns of distributed file systems, and introduces in detail two kinds of design patterns: active / active mode and active/standby mode. In this paper, the advantages and disadvantages of different sub-patterns in these two kinds of design patterns are analyzed. At the same time, I / O model of server programming is introduced. This paper gives a general introduction to Cstore file system, introduces and expounds the key technologies involved in the whole system, and puts forward solutions to the problems faced by metadata storage units in providing services. For example, the master-slave mutual backup mechanism for data security and stability services, how to ensure that the metadata uploaded by the user can be quickly located to the metadata server, Upload files how to quickly locate the need to upload data server and other technologies. At the same time, some key technologies used in metadata rule server are introduced. The module partition of the above metadata storage unit is introduced in general, and then the design class diagram and data flow chart of each sub-module are given in detail, and the functions of the classes involved in the diagram are explained and introduced in detail. The implementation of the metadata storage unit cluster was tested. The results show that the cluster of metadata storage units can provide metadata access efficiently and stably, and the performance meets the design requirements.
【学位授予单位】：电子科技大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333
【参考文献】
相关期刊论文 前3条
1 雷江涛;白照高;;网络存储技术的发展、现状及应用[J];工业仪表与自动化装置;2007年01期
2 段翰聪,卢显良,宋杰;基于EPOLL的单进程事件驱动通信服务器设计与分析[J];计算机应用;2004年10期
3 段翰聪;杨帆;李林;;基于二维表的对象存储元数据管理[J];实验科学与技术;2012年06期
本文编号：2132486
本文链接：https://www.wllwen.com/kejilunwen/jisuanjikexuelunwen/2132486.html
