基于P2P-云平台的作业管理机制的研究

发布时间：2018-06-18 02:53

本文选题：云计算 + 对等网络　；参考：《南京邮电大学》2014年硕士论文

【摘要】：云计算的概念自提出以来，即在全世界范围内得到了广泛应用。云计算凭借其高度并行性、虚拟化等特点大幅简化了程序员的工作，提高了作业执行速率。MapReduce作为云计算中的核心编程模型，为程序员自动完成了容错、死锁等至今仍无通用解决方案的问题。如今，越来越多的企业都已经将自己的业务部署到了云计算平台中。然而，传统云计算平台中的MapReduce编程模型是基于集中式的C/S框架，模型中由一个控制节点管理多个工作节点，且用户提交的作业全部都由唯一的控制节点进行分配管理，控制节点一旦失效，就会造成无法挽回的后果。不但如此，单一控制节点负载有限，随着集群规模的不断增大，其势必会成为整个系统的瓶颈，严重限制了集群的扩展。因此，为了解决传统云计算平台中单控制节点失效和集群扩展性不足两大问题，本文提出了基于P2P-云平台的作业管理机制。该机制首先对传统云计算平台中基于单控制节点的主从结构进行改进，提出基于Chord协议的多控制节点协同控制多工作节点的体系结构ChordMR。其次，针对控制节点失效所带来的作业丢失问题，提出作业动态管理机制，该机制从作业分配、作业备份、作业恢复以及控制节点加入时的作业迁移等各方面对云作业进行有效管理，以加快云作业执行速率、提高云作业执行成功率。通过仿真实验，验证了该作业管理机制在可行性和有效性等方面的优势。最后，本文编程实现了基于P2P-云平台的MapReduce作业处理框架。理论分析和实验结果表明，，传统云平台与P2P技术的有效结合不但解决了传统云平台中存在的单点失效问题，而且增强了集群的扩展性，有效解决了系统的瓶颈问题。
[Abstract]:The concept of cloud computing has been widely used in the world since it was put forward. Because of its high parallelism, virtualization and other characteristics, cloud computing greatly simplifies the work of programmers, improves the job execution rate. MapReduce as the core programming model in cloud computing, automatically completes fault tolerance for programmers. Deadlock is still not a common solution to the problem. Nowadays, more and more enterprises have deployed their business to cloud computing platform. However, the MapReduce programming model in the traditional cloud computing platform is based on the centralized C / S framework. In the model, multiple work nodes are managed by one control node, and all the jobs submitted by the user are allocated and managed by the unique control node. Once the control node fails, there will be irreparable consequences. Moreover, the load of single control node is limited. With the increasing of cluster scale, it will become the bottleneck of the whole system, which seriously limits the expansion of the cluster. Therefore, in order to solve the problems of failure of single control node and insufficient scalability of cluster in traditional cloud computing platform, a job management mechanism based on P2P- cloud platform is proposed in this paper. The mechanism firstly improves the master-slave structure based on single control node in traditional cloud computing platform, and proposes a chord protocol based architecture for multi-control nodes to control multi-work nodes. Secondly, aiming at the problem of job loss caused by the failure of control nodes, a dynamic job management mechanism is proposed, which includes job assignment, job backup, and so on. In order to speed up the cloud job execution rate and improve the success rate of cloud job execution, the job recovery and job migration when the control nodes join are effectively managed in order to speed up the cloud job execution rate. The feasibility and effectiveness of the job management mechanism are verified by simulation experiments. Finally, a MapReduce job processing framework based on P2P-cloud platform is implemented. The theoretical analysis and experimental results show that the combination of traditional cloud platform and P2P technology not only solves the single point failure problem in traditional cloud platform, but also enhances the expansibility of cluster and effectively solves the bottleneck problem of the system.
【学位授予单位】：南京邮电大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP393.02;TP311.1

【参考文献】