面向终端应用的云存储系统研究与设计

发布时间：2018-02-15 00:00

本文关键词： 云存储副本策略负载均衡系统架构粒子群优化算法　出处：《广东工业大学》2013年硕士论文　论文类型：学位论文

【摘要】：随着信息技术产业的发展和逐渐成熟,它的各种应用已经渗透到社会的各行各业,使社会生产力水平得到了极大的提高,更重要的是为人们的生活、工作、学习带来了前所未有的便利和实惠。在这背后,信息技术带来的数据爆炸式增长,急需廉价、快捷并且安全的数据存储方式。近几年云计算技术的快速发展,为云存储从概念到实际应用奠定了基础。同时由于终端技术的发展、移动网络的快速普及,云存储的应用范围也逐渐扩大。云终端的模式更加贴近人们的生活,正改变着人们的日常习惯。本文对云存储技术进行研究,提出了针对终端应用下的云存储系统方案。对于小文件类型采用两种不同的副本存储方式,一种是MySQL关系数据库方式,另一种是MongoDB非关系数据库方式；对于大文件类型采用动态一致性哈希算法方式来存储。同时还引入了粒子群优化算法,用于云存储中数据最优存储节点选择L。通过小规模的实际运行系统以及CloudSim仿真的形式测试系统的性能。下面是本论文主要进行的工作： (1)通过查阅期刊文献、电子资源,研究了HDFS、FastDFS等开源分布式文件系统源代码。在分析系统的功能性需求以及非功能性需求的基础之上,设计了云存储系统架构原型。 (2)对云存储系统的客户端和服务端程序的主要功能模块进行详细设计,并在普通PC机上编程搭建了一个小型的云存储系统。通过该云存储系统验证后文提出的副本策略以及粒子群优化算法在数据最优存储节点选择中的应用效果。设计系统过程中,考虑移动终端的特点,在地域上考虑采用最近邻的云存储节点对其服务,提高系统的存储效率。 (3)针对云存储系统的副本策略进行研究,本文提出了对文件分类存储的方式,对于小文件类型采用数据库的方式,而大文件类型采用动态一致性哈希算法的方式。并且数据库的方式也采用了两种方案进行对比分析设计,一种是采用MySQL关系型数据库方案,另一种是采用MongoDB非关系型数据库方案。大文件类型的处理方案设计成去中心化结构,消除大并发处理过程中的性能瓶颈,并对所提出的方案进行了有效性和优越性的验证。 (4)在数据最优存储节点选择过程中引入粒子群优化算法,保证系统的负载均衡和存储效率。节点选择是处在离散空间的,粒子群优化算法重新在离散问题空间中进行定义,并且定义了问题解的适应度评价函数,评价候选解的优劣,同时为了得到更优的解,引入了启发信息的方式引导算法得出更优的解；最后给出了仿真比较的结果验证方法的有效性。
[Abstract]:With the development and maturity of the information technology industry, its various applications have penetrated into all kinds of industries of the society, so that the level of social productivity has been greatly improved, more importantly, for the people's life, work, Learning has brought unprecedented convenience and benefits. Behind this, information technology has brought explosive growth in data, and is in urgent need of cheap, fast, and secure data storage. The rapid development of cloud computing technology in recent years, It lays a foundation for cloud storage from concept to practical application. At the same time, with the development of terminal technology and the rapid popularization of mobile network, the application scope of cloud storage is gradually expanded. The model of cloud terminal is closer to people's life. Is changing people's daily habits. In this paper, the cloud storage technology is studied, and a cloud storage system scheme for terminal applications is proposed. For small file types, two different replica storage methods are adopted, one is MySQL relational database, the other is MySQL relational database. The other is the MongoDB non-relational database, the dynamic consistency hash algorithm is used to store large file types, and the particle swarm optimization (PSO) algorithm is also introduced. The performance of the system is tested in the form of small scale actual running system and CloudSim simulation. The following is the main work of this paper:. This paper studies the source code of open source distributed file system such as HDFS FastDFS. Based on the analysis of the functional and non-functional requirements of the system, the architecture prototype of cloud storage system is designed. The main function modules of client and server programs of cloud storage system are designed in detail. A small cloud storage system is built by programming on ordinary PC computer. The replica strategy and particle swarm optimization (PSO) algorithm are used to verify the application effect of the proposed replica strategy and particle swarm optimization algorithm in the selection of data optimal storage nodes through the cloud storage system. In the design process of the system, Considering the characteristics of the mobile terminal, the nearest neighbor cloud storage node is considered to serve the mobile terminal in order to improve the storage efficiency of the system. 3) aiming at the research of replica strategy of cloud storage system, this paper puts forward the method of file classification storage, and adopts the method of database for small file type. The large file type uses the dynamic consistency hash algorithm. And the database also adopts two schemes to carry on the comparative analysis design, one is to adopt the MySQL relational database scheme, one is to adopt the MySQL relational database scheme, one is to adopt the MySQL relational database scheme, one is to adopt the MySQL relational database scheme. The other is to adopt the MongoDB non-relational database scheme. The processing scheme of large file type is designed as a decentralized structure to eliminate the performance bottleneck in the process of large concurrent processing. The validity and superiority of the proposed scheme are verified. In order to ensure the load balance and storage efficiency of the system, the particle swarm optimization algorithm is introduced in the data optimal storage node selection process. The node selection is in the discrete space, and the particle swarm optimization algorithm is redefined in the discrete problem space. The fitness evaluation function of the problem solution is defined to evaluate the advantages and disadvantages of the candidate solution. In order to obtain a better solution, the heuristic information is introduced to guide the algorithm to get a better solution. Finally, the effectiveness of the simulation results is given.
【学位授予单位】：广东工业大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】