基于粒子群调度器的云存储系统针对交互密集型任务的缓存模型研究

发布时间：2018-01-06 05:34

本文关键词：基于粒子群调度器的云存储系统针对交互密集型任务的缓存模型研究　出处：《华东师范大学》2012年硕士论文　论文类型：学位论文

【摘要】：随着计算机和网络技术日益先进,以及当今社会对对计算资源的需求日益旺盛,许多公司厂商和研究机构开始探求通过互联网络向外部用户租售输出存储和计算能力的可能性。这种以经济需求为导向的外向型计算模式在今天被称作云计算模式。而云计算模式实现的基础,则是大型的计算和存储集群。在另一个方面,云计算的创新性也给其相关技术提出了严峻的挑战和崭新的问题。比如,其巨大的规模已经使计算集群在很多方面产生了亟待突破的性能瓶颈,而本文所研究的云存储系统在处理交互密集型任务时会产生巨大的性能损益这一问题便是其中之一。本文的研究依托于Hadoop Distribute File System分布式文件系统所构建的分布式存储平台,提出了一种基于粒子群调度分配算法的主从名字节点缓冲架构,旨在解决原有系统在面对具有频繁写入读出特性的一类用户应用程序时系统吞吐量剧烈降低这一问题。该架构在结构上对原分布式文件系统做出了两项修改。首先,将原分布式文件系统的单名字节点结构改变为双层主从名字节点结构。然后再在每个从名字节点上实现基于tmpfs文件系统的数据块缓存机制。同时,该架构还采用了基于粒子群算法的文件块调度措施,用来将到来的文件块优化分配到合适的数据节点或缓存中。由于单纯的粒子群算法在本文所提系统的环境中极易陷入局部最优解而对整个系统的性能造成较大的负面印象,所以本文对粒子群算法进行了修改,使之具有进化程度评估和参数自适应调整的能力。本文所提出的架构在一个130个节点的松耦合分布式环境中进行了实际地部署和验证,并针对交互密集型应用的特点进行了一系列测试。实验结果证明,本文所采用的具有进化程度评估和参数自适应调整能力的粒子群优化调度有效地提高了算法的效率并降低了调度方案陷入局部最优解的问题。同时,针对多项指标的综合测试实验证明,本文提出的基于粒子群调度的两层缓冲模型能够切实地解决分布式文件系统面对交互密集型应用所带来的系统吞吐量下降的问题。
[Abstract]:With the increasing sophistication of computer and networking technologies , and the growing demand for computing resources in today ' s society , many companies and research institutes have begun to explore the possibility of renting out output storage and computing power through the Internet to external users . Such export - oriented computing models guided by economic demand are today called cloud computing models . The foundation for cloud computing models is large computing and storage clusters . In another aspect , the innovative nature of cloud computing poses a serious challenge and a new problem to its related technologies . For example , its massive scale has made computing clusters a critical performance bottleneck in many ways , and the problem of cloud storage systems , as discussed herein , results in significant performance gains and losses when dealing with interactive intensive tasks . Based on the distributed storage platform built by the distributed file system of Hadoop distributed file system , this paper proposes a master - slave name node buffer architecture based on the particle swarm scheduling algorithm . It is designed to solve the problem of drastic reduction in throughput of the original distributed file system . At the same time , the structure of the single - name node of the original distributed file system is changed to double - layer master - slave name node structure . In this paper , a series of tests are carried out in a loosely coupled distributed environment of 130 nodes , and a series of tests are carried out for the characteristics of interaction - intensive applications . The experimental results show that the particle swarm optimization scheduling with the evolutionary degree evaluation and the adaptive adjustment capability of the parameters effectively improves the efficiency of the algorithm and reduces the problem that the scheduling scheme is trapped in the local optimal solution .

【学位授予单位】：华东师范大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP333

【参考文献】