面向工作流任务的云计算资源多目标与容错调度研究

发布时间：2018-06-04 12:32

本文选题：云计算 + 资源调度　；参考：《东华大学》2016年博士论文

【摘要】：近年来,云计算由于其高可扩展性、较强的通用性和按需提供服务等特点,已经得到了政府和行业越来越多的认可和使用。随着云计算应用的普及,云数据中心所承担的工作势必将越来越繁重。面对愈加繁重的任务,如何对大量异构资源进行合理的调度以保证云数据中心高效、稳定的运行就显得尤为重要,特别是当处理的任务是各任务间具有依赖关系的工作流任务时,因为它直接关系到云计算的服务质量和效益以及用户的满意程度。本论文针对任务间具有依赖关系的工作流任务,分别研究了多目标和容错要求下的云计算资源调度问题,主要内容包括如下几个方面:(1)针对多目标要求,设计了一种内分泌-多种群协作进化启发的多目标优化算法(ECMSMOO),并将其应用到面向工作流任务的云计算资源多目标调度之中。ECMSMOO采用多个种群实现多目标优化,每个种群以一个优化目标为主进行优化。同时,ECMSMOO还设计了一种种群间的竞争与合作机制来实现种群之间的信息交流与合作,从而使其能够搜索到更好的前沿面。为了避免粒子陷入局部最优解,受内分泌系统中促激素和释放激素工作机制的启发,ECMSMOO还嵌入了一个受内分泌启发的进化策略,从而更好地利用外部档案集中的粒子。最后通过对比验证,证明了ECMSMOO在云计算资源多目标调度中的有效性。(2)针对容错的要求,受免疫系统工作机理的启发,设计了一种面向工作流的云计算资源重调度算法(IRW)。IRW模拟免疫系统的工作机制,由监测模块、应答模块、学习模块和记忆模块等四个模块相互协同工作,从而为工作流执行过程中因资源故障而不能顺利执行的任务提供相应的重调度策略,特别适用于相同故障再次发生的情况。为了提高搜索速度,IRW采用K-means算法将资源池中的可用资源进行聚类,从而缩小抗体查找的范围。同时,针对资源池中资源不能满足重调度服务质量要求的情况,IRW还利用虚拟化技术创建新的虚拟机来满足任务对服务质量的要求。最后,通过实验验证了IRW的可行性和有效性。(3)针对容错的要求,基于Primary-Backup(PB)模式设计了一种面向工作流任务的云计算资源弹性容错调度算法(FTESW)。算法首先分析了云计算环境中任务之间的依赖关系对基于PB模式的容错调度的约束,然后分析了容错调度对虚拟机迁移的约束。基于这些工作,在保证容错的需求下,算法设计了云计算环境中面向工作流任务的弹性资源提供策略,包括资源扩展策略和资源整合策略,然后设计了相应的任务主本弹性容错调度策略和任务副本弹性容错调度策略。最后通过仿真实验验证了FTESW能够在实现容错的前提下,可有效的提高系统的资源利用率。(4)同样针对容错要求,提出了一种面向不平衡工作流的云计算资源容错策略的自适应选择算法(IMWSFW)。该算法将复制和重调度两个容错策略有效地结合起来,以发挥它们各自在容错中的优势,并尽量满足工作的软期限。IMWSFW首先对工作流的软期限进行了划分,使得每个任务均有自身的子期限。然后基于任务子期限的不平衡性和云计算资源性能的异构性,为每个任务从复制和重调度中选择相应的容错策略。然后,结合云计算系统资源按需提供的特点,IMWSFW为每个任务选择相应的虚拟机并预定相应的服务时间间隔。对于选择重调度作为其容错策略的任务,当其在首次执行过程中遇到故障时,IMWSFW通过在线重调度为其提供相应的容错策略。为了充分利用任务间的时间间隔,在任务执行过程中,IMWSFW还通过在线预留调整机制来扩展部分未执行任务的子截止期。最后通过仿真实验验证了所提的IMWSFW算法能够提供较好的容错调度策略,且比重调度容错策略具有更高的任务完成率;同时比副本复制容错调度策略具有更高的资源利用率。最后,对全论文的研究工作进行了总结,指出了其中的不足,并对下一步的研究工作和方向进行了展望。
[Abstract]:In recent years, cloud computing has gained more and more recognition and use by the government and industry because of its high scalability, strong generality and service on demand. With the popularization of cloud computing applications, the work potential of the cloud data center will become more and more heavy. Reasonable scheduling is particularly important to ensure the efficient and stable operation of the cloud data center, especially when the task is a workflow task with dependency between various tasks, because it is directly related to the service quality and efficiency of the cloud computing and the satisfaction of the users. This paper is dependent on the dependency relationship between the tasks. The task of workflow task is to study the problem of cloud computing resource scheduling under multi-objective and fault-tolerant requirements. The main contents include the following aspects: (1) a multi-objective optimization algorithm (ECMSMOO) is designed for multi-objective requirements, and it is applied to the cloud computing resources for workflow tasks. In multi-objective scheduling,.ECMSMOO uses multiple populations to achieve multi-objective optimization, and each population is optimized by an optimization goal. At the same time, ECMSMOO also designs a variety of groups of competition and cooperation mechanisms to achieve information exchange and cooperation among populations, so that they can search for better frontiers. In order to avoid particle subsidence. In the local optimal solution, inspired by the mechanism of hormone and releasing hormone in the endocrine system, ECMSMOO also embeds an evolutionary strategy inspired by endocrinology to make better use of the particles in the external archives. Finally, the effectiveness of ECMSMOO in multi-objective scheduling of cloud computing resources is proved by comparison and verification. (2) fault tolerance is aimed at fault tolerance. Inspired by the working mechanism of the immune system, the work mechanism of a workflow based cloud computing resource rescheduling algorithm (IRW).IRW simulation immune system is designed, which are worked together by four modules, such as monitoring module, response module, learning module and memory module. In order to improve the search speed, IRW uses the K-means algorithm to cluster the available resources in the resource pool to reduce the range of antibody search. At the same time, the resources in the resource pool can not meet the quality requirements of the rescheduling service. In addition, IRW also uses virtualization technology to create new virtual machines to meet the requirements of service quality. Finally, the feasibility and effectiveness of IRW are verified by experiments. (3) based on the requirement of fault tolerance, a flexible fault tolerant scheduling algorithm for cloud computing resources for workflow tasks (FTESW) is designed based on the Primary-Backup (PB) model. The method first analyzes the constraints of the dependencies between tasks in the cloud computing environment to the PB based fault-tolerant scheduling, and then analyzes the constraints of the fault-tolerant scheduling on the migration of the virtual machine. Based on these tasks, the algorithm designs a flexible resource provision strategy for the work flow task in the cloud computing environment under the requirement of fault tolerance. Source extension strategy and resource integration strategy, then the corresponding task master resilient fault-tolerant scheduling strategy and task replica resilient fault-tolerant scheduling strategy are designed. Finally, the simulation experiments show that FTESW can effectively improve the resource utilization of the system under the premise of fault tolerance. (4) a kind of face is also proposed for the fault tolerance requirements. The adaptive selection algorithm (IMWSFW) for the fault-tolerant strategy of cloud computing resources to unbalanced workflow. This algorithm combines duplication and rescheduling two fault-tolerant strategies effectively to give full play to their respective advantages in fault tolerance, and to meet the soft deadline of work as far as possible..IMWSFW is first divided into the soft deadline of the workflow, which makes every appointment. It has its own submission. Then, based on the imbalances of the task submission and the heterogeneity of the performance of the cloud computing resources, the corresponding fault tolerant strategies for each task are selected for each task from replication and rescheduling. Then, combining the features provided by the cloud computing system resources on demand, IMWSFW selects the corresponding virtual machine for each task and preorders the corresponding service. Time interval. For the task of selecting rescheduling as its fault tolerant strategy, IMWSFW provides a corresponding fault tolerance strategy by on-line rescheduling when it has a fault in the first execution process. In order to make full use of the interval between tasks, IMWSFW also extends the part by online reservation adjustment mechanism during the task execution process. The simulation experiment shows that the proposed IMWSFW algorithm can provide a better fault-tolerant scheduling strategy, and the specific gravity scheduling fault tolerant strategy has a higher task completion rate. At the same time, it has a higher resource utilization ratio than the replica replication fault-tolerant scheduling strategy. Finally, the research work of the whole paper is summarized. It pointed out the shortcomings and prospected the next research work and direction.
【学位授予单位】：东华大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TP393.09;TP301.6

【相似文献】