分布式块级别存储系统的设计与实现

发布时间：2018-06-28 19:02

本文选题：块存储 + 分布式系统　；参考：《哈尔滨工业大学》2013年硕士论文

【摘要】：随着人们对存储需求的不断增长以及新型存储技术的快速发展，存储系统的发展日新月异。存储系统向着大容量、高速度、高可靠性以及易于管理的趋势发展。对存储软件系统和存储设备不断提出更高的要求，而存储设备的发展受到物理硬件本身的制约，存储容量和稳定性都存在一定的限制和瓶颈。特别需要指出的是存储系统通常与存储硬件绑定，造成硬件与软件不能分离，，存储硬件的更新与替换对存储系统的发展与应用带来成本和安全性上的损失。本文设计并实现了块级别分布式数据存储系统，使用高可用集群将廉价的普通PC管理起来，提供一块虚拟的巨大的逻辑存储空间，在该空间内可以任意创建虚拟磁盘，每个虚拟磁盘分成固定大小的数据片分布在不同的物理存储节点上。根据NWR策略创建三份副本分别保存在不同的存储设备中，只有一部分设备损坏不会影响系统的正常使用。本系统支持异构的存储设备，存储能力大的设备负载能力更强，系统能够自动根据不同设备的负载能力，按照一定的比例关系使用每台存储设备的存储资源，做到负载均衡分布，以此优化系统设备的利用率与提高性能。为了保证数据的安全性，针对虚拟磁盘设计了快照功能，使用写时拷贝技术实现的快照技术可以快速的对虚拟磁盘数据进行差异备份，在不间断系统运行的情况下快速的创建数据备份，创建过快照的虚拟磁盘可以随时恢复。传统的存储技术发展到今天，存储区域网络(SAN)技术仍然是存储技术的主要行业标准，iSCSI协议是SAN的主要实现协议之一，本系统提供块级别访问接口，可作为iSCSI Target后端存储。经验证本分布式存储系统解决了存储系统的单点故障问题，以去中心化多副本技术保证了数据安全，并且能够提供PB级别的存储能力。论文最后对系统进行了功能性的测试与集成测试，并且对iSCSI的集成也进行了功能和性能上的测试。
[Abstract]:With the increasing demand for storage and the rapid development of new storage technology, storage system is developing with each passing day. Storage systems tend to be large capacity, high speed, high reliability and easy to manage. More and more demands are put forward to storage software system and storage device. The development of storage device is restricted by physical hardware itself. There are some limitations and bottlenecks in storage capacity and stability. It should be pointed out that the storage system is usually bound to the storage hardware, resulting in the hardware and software can not be separated, storage hardware update and replacement of the storage system development and application of the cost and security losses. In this paper, a block level distributed data storage system is designed and implemented. The low cost ordinary PC is managed by using a high availability cluster, which provides a large virtual logical storage space in which virtual disks can be created arbitrarily. Each virtual disk is divided into fixed size pieces of data distributed on different physical storage nodes. According to the NWR policy, three copies are created and stored in different storage devices. Only one part of the device damage will not affect the normal use of the system. The system supports heterogeneous storage devices, and the storage capacity of the devices with large storage capacity is stronger. The system can automatically use the storage resources of each storage device according to the load capacity of different devices and according to a certain proportion. Load balance distribution is achieved to optimize the utilization of system equipment and improve performance. In order to ensure the security of data, the snapshot function is designed for virtual disk. The snapshot technology realized by using write-time copy technology can backup the data of virtual disk quickly. Create a backup of data quickly when the system is running continuously. The virtual disk that has created the snapshot can be recovered at any time. With the development of traditional storage technology, storage area network (San) is still the main industry standard of storage technology, iSCSI protocol is one of the main implementation protocols of San. This system provides block level access interface and can be used as iSCSI Target back-end storage. It is proved that the distributed storage system solves the single point failure problem of the storage system, and the data security is ensured by the decentralized multi-copy technology, and the storage capacity of PB level is provided. Finally, the functional and integration tests of the system are carried out, and the integration of iSCSI is also tested in function and performance.
【学位授予单位】：哈尔滨工业大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】