分布式文件系统元数据服务器高可用设计与实现

发布时间：2018-04-27 03:41

本文选题：分布式文件系统 + 元数据　；参考：《上海交通大学》2012年硕士论文

【摘要】：伴随全球数据量的飞速增长和数据密集型应用的不断增加,分布式文件系统得到越来越广泛的应用。目前主流的分布式文件系统均采用将元数据与数据分离管理的机制,将命名空间和文件元数据保存在元数据节点,而数据保存在数据节点。很显然,元数据节点的高可用是决定分布式存储系统的可用性的关键因素。本文对分布式文件系统中的元数据高可用性技术进行了全面。深入研究,并在此基础上设计和实现了BlueOcean分布式文件系统元数据服务器高可用方案。本文的主要工作如下: (1)研究了分布式文件系统中元数据管理的关键问题,包括文件系统元数据职责。分布式元数据管理方式。命名空间分布策略等; (2)研究了可用性相关知识及常用高可用系统组织模式,将不同的高可用组织模式对系统性能及恢复时间的影响进行了比较; (3)研究了提高分布式文件系统元数据服务器可用性的关键技术,设计并实现了碧海分布式文件系统元数据服务器高可用方案,在设计过程中对元数据复制。服务器状态监测。失败确认及接管。节点恢复后的处理等关键问题进行了详细分析; (4)对实现后的高可用系统设计了实验方案进行了测试。结果表明,该高可用设计方案对元数据操作性能和系统整体性能影响都比较小,且在进行大文件访问时性能影响可以忽略;系统可在元数据服务器异常停机后的一分钟时间里完成系统服务的恢复。
[Abstract]:With the rapid growth of global data volume and the increasing of data-intensive applications, distributed file systems have been more and more widely used. At present, the mainstream distributed file systems adopt the mechanism of separating metadata from data, and keep namespace and file metadata in metadata nodes, while data are stored in data nodes. Obviously, the high availability of metadata nodes is the key factor to determine the availability of distributed storage systems. In this paper, the technology of high availability of metadata in distributed file system is discussed. The high availability scheme of BlueOcean distributed file system metadata server is designed and implemented. The main work of this paper is as follows: 1) the key issues of metadata management in distributed file system, including the metadata responsibility of file system, are studied. Distributed metadata management. Namespace distribution strategy; Secondly, the knowledge of usability and the organization pattern of commonly used high availability system are studied, and the effects of different high availability organization modes on system performance and recovery time are compared. The key technology of improving the availability of distributed file system metadata server is studied, and the high availability scheme of Bihai distributed file system metadata server is designed and implemented. Server status monitoring. Fail to confirm and take over. The key issues such as node recovery are analyzed in detail. Finally, the experimental scheme of the high availability system is designed and tested. The results show that the high availability design scheme has little effect on the performance of metadata and the whole system, and it can be neglected when accessing large files. The system can complete the recovery of the system service in a minute after the metadata server outages.
【学位授予单位】：上海交通大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP368.5

【引证文献】