私有云存储系统的设计与实现

发布时间：2018-01-04 18:43

本文关键词：私有云存储系统的设计与实现　出处：《电子科技大学》2014年硕士论文　论文类型：学位论文

【摘要】：随着信息技术的不断发展和普及,数据量迅猛增长,数据之间的联系也越来越紧密,这使得高能性能计算[23,24]和大数据处理[25-27]日益成为人们关注的焦点。如今,云存储技术因其高效的管理方式,高可靠性,高安全性等特点成为企业存储数据的首选解决方案。而在云存储系统设计之中,其高效性、可扩展性和安全性成为设计的难点和重点。在云计算业界,云存储系统的元数据管理单元和数据存储单元的设计和实现一直是整个难点问题。由于文章篇幅和时间的限制,本文重点研究数据存储单元的设计和实现。本文根据当前的存储系统和分布式文件系统的特点和应用,并结合企业存储服务的实际需求,提出并实现了一种具有高可扩展性和高可用性的云存储系统C-Store。本文先介绍已有的几种分布式存储系统和与云存储相关技术理论基础,接着介绍本云存储系统的总体设计思路和方法,重点介绍数据存储系统的设计和实现,最后通过对系统的测试来验证之前的设计思路。云存储系统C-Store的总体架构采用分布式构架,元数据部分和数据部分的数据采用分别存储方式,并分别用不同的、各自的资源管理单元(桶)来存储,这样有利于元数据和数据分别管理和进行性能优化,避免单个节点失效带来的风险;同时采用负载均衡技术使之各个节点负载均衡,使用数据恢复策略等来提高系统的可靠性和可扩展。本文实现了数据存储单元中的设计思路和设计要点。整个数据存储子系统实现是在Linux环境下使用C++来实现的;实现之后系统,用户可以通过客服端来上传、下载和删除数据;实现数据以键值对存储方式进行存储,据按照基于规则的资源定位策略(多层映射),定位到多个不同的数据存储单元;通过桶的扩展和迁移来实现系统的负载均衡机制;用多副本机制来实现数据的恢复和重建;总的来讲实现了一个具有高性能、可用性[22]和高可靠性的云存储系统。
[Abstract]:With the continuous development of information technology and the popularity of the rapid growth of data volume, data between the increasingly close ties, which makes the high performance computing and data processing of [23,24] [25-27] has become the focus of attention. Nowadays, the cloud storage technology because of its efficient management, high reliability, high security has become the preferred solution scheme of enterprise data storage. In the design of cloud storage system, its efficiency, scalability and security has become the key point of the calculation. The industry in the cloud, cloud storage system design and implementation of metadata management unit and a data storage unit is always the difficult problem. Because of the limited time and space this paper focuses on the research, design and implementation of a data storage unit. According to the characteristics and application of the current storage system and distributed file system, combined with the enterprise storage service The actual service demand, proposes and implements a scalability and high availability of distributed storage system has been first introduced by the cloud storage system C-Store. and the related technology of cloud storage has high theoretical basis, and then introduces the overall design ideas and methods of the cloud storage system, introduces the design and implementation of data storage system finally, through the testing of the system to verify the previous design. The overall architecture of cloud storage system C-Store using distributed architecture, metadata part and data part of the data are stored by way, and don't use different resources, their management unit (barrels) to store, which is beneficial to the metadata and data management and to optimize performance, avoid the risk of failure of a single node; at the same time using the load balancing technology so that each node load balancing, data recovery strategy To improve the system reliability and scalability. This paper realizes the design ideas and design points of data storage unit. The data storage subsystem is C++ in the Linux environment to achieve the realization of the system; after, the user can through the customer terminal to upload, download and delete data; data storage in pairs for storage, according to the strategy of resource location based on rules (multi-level mapping), located in a number of different data storage unit; through the expansion and relocation of barrel to realize the load balancing mechanism system; multi copy mechanism to achieve data recovery and reconstruction; in general, to achieve a high performance, the availability of [22] and cloud storage system with high reliability.

【学位授予单位】：电子科技大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP333

【相似文献】