集群NAS系统CLNASFS的关键技术研究
发布时间：2018-10-12 10:54
【摘要】：随着信息时代数据量的急剧膨胀,传统存储系统如存储局域网(SAN)和网络附加存储(NAS)采用的均为纵向拓展的架构,在性能和扩展性上已经难以满足海量数据的应用需求。集群NAS系统技术采用了横向拓展的架构,拥有高性能及易拓展的两大优点,已成为当前构建大型数据存储系统的一种有效解决方式。本文在对当前的集群NAS技术进行研究的基础上,选择了基于集群文件系统的方式,结合现有的成熟稳定的开源软件,设计并实现了一个集群NAS系统CLNASFS。CLNASFS具有高性能、高可拓展能力、高可用、存储空间利用率高等特点,适用于视频编辑、科学技术、视频监控、共享文件存储等对存储系统的性能、容量及可用性要求高的领域。CLNASFS基于集群文件系统Gluster FS、开源CIFS服务器Samba、高可用组件CTDB构建,可对外提供包括CIFS、NFS、HTTP等多种访问协议。该系统采用了无元数据服务器模型,通过文件和目录的拓展文件属性来管理系统的元数据,避免了单点故障问题。本系统所设计实现的核心技术包括:基于类型的数据布局策略、高可用监视模块、服务器端基于硬件的实时文件压缩系统等。其中本文提出的基于类型的数据布局策略还可以被应用于其他的Key/Value存储系统中,具有较大参考价值。同时服务器端基于硬件的实时压缩文件系统既可以在CLNASFS中作为服务器端的压缩模块使用,也可以单独使用配合i SCSI协议对外作为提供压缩功能的块设备使用。最后,通过对比测试,验证了CLNASFS系统在性能、可拓展性、可用性及压缩功能上的优势,证明了该系统的稳定性及其内部各种技术的有效性。
[Abstract]:With the rapid expansion of data in the information age, the traditional storage systems, such as (SAN) and (NAS), are both vertically extended, so it is difficult to meet the application requirements of mass data in performance and expansibility. Cluster NAS system technology adopts the architecture of horizontal expansion and has the advantages of high performance and easy to expand. It has become an effective solution to construct large data storage system. Based on the research of the current cluster NAS technology, this paper chooses the method of cluster file system, and combines the existing mature and stable open source software, designs and implements a cluster NAS system CLNASFS.CLNASFS with high performance. High scalability, high availability, high storage space utilization and other characteristics, suitable for video editing, science and technology, video surveillance, shared file storage and other storage system performance, CLNASFS is based on the cluster file system Gluster FS, open source CIFS server Samba, high availability component CTDB which can provide multiple access protocols including CIFS,NFS,HTTP. The system adopts the model of no metadata server, and manages the metadata of the system by extending the file attributes of files and directories, thus avoiding the problem of single point of failure. The core technologies of this system include: data layout strategy based on type, high availability monitoring module, real-time file compression system based on hardware on the server side and so on. The data layout strategy based on type proposed in this paper can also be applied to other Key/Value storage systems, which is of great reference value. At the same time, the hardware-based real-time compressed file system can be used not only in CLNASFS as the compression module on the server side, but also as a block device with I SCSI protocol. Finally, the advantages of CLNASFS system in performance, expansibility, availability and compression function are verified by comparing tests. The stability of the system and the effectiveness of various internal technologies are proved.
【学位授予单位】：国防科学技术大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333;TP311.52
，
本文编号：2265852
[Abstract]:With the rapid expansion of data in the information age, the traditional storage systems, such as (SAN) and (NAS), are both vertically extended, so it is difficult to meet the application requirements of mass data in performance and expansibility. Cluster NAS system technology adopts the architecture of horizontal expansion and has the advantages of high performance and easy to expand. It has become an effective solution to construct large data storage system. Based on the research of the current cluster NAS technology, this paper chooses the method of cluster file system, and combines the existing mature and stable open source software, designs and implements a cluster NAS system CLNASFS.CLNASFS with high performance. High scalability, high availability, high storage space utilization and other characteristics, suitable for video editing, science and technology, video surveillance, shared file storage and other storage system performance, CLNASFS is based on the cluster file system Gluster FS, open source CIFS server Samba, high availability component CTDB which can provide multiple access protocols including CIFS,NFS,HTTP. The system adopts the model of no metadata server, and manages the metadata of the system by extending the file attributes of files and directories, thus avoiding the problem of single point of failure. The core technologies of this system include: data layout strategy based on type, high availability monitoring module, real-time file compression system based on hardware on the server side and so on. The data layout strategy based on type proposed in this paper can also be applied to other Key/Value storage systems, which is of great reference value. At the same time, the hardware-based real-time compressed file system can be used not only in CLNASFS as the compression module on the server side, but also as a block device with I SCSI protocol. Finally, the advantages of CLNASFS system in performance, expansibility, availability and compression function are verified by comparing tests. The stability of the system and the effectiveness of various internal technologies are proved.
【学位授予单位】：国防科学技术大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333;TP311.52
，
本文编号：2265852
本文链接：https://www.wllwen.com/kejilunwen/jisuanjikexuelunwen/2265852.html
