云计算应用中的任务调度与资源分配的协同优化

发布时间：2018-06-19 12:41

本文选题：云计算 + 任务调度　；参考：《中国科学技术大学》2017年博士论文

【摘要】：随着业务的发展,越来越多的企业或组织开始采用基于多数据中心的云环境甚至多云环境为用户提供服务。多数据中心或多云可以为不同地理位置的用户提供时延更短、带宽更高、成本更低的服务,同时多数据中心间可构建互为容灾备份机制,从而提供高可靠的服务。多数据中心在提供更好的服务性能的同时,也增加了云应用管理的复杂性和挑战性。一方面,多数据中心环境将出现大规模的用户到数据中心(U2D)和数据中心到数据中心(D2D)的任务,这些任务通常包含大量待处理数据,有较高的计算、存储、网络等多维资源的需求。另一方面,多数据中心或多云环境存在大量不同类型、不同粒度的异构资源,这些资源的配置、性能、费用模型都不尽相同。随着基础设施地资源规模和异构性地增加,越来越多的企业将基础设施的管理和应用开发分离,使得应用开发人员可以敏捷开发,从而快速地应对用户需求和市场变化。这必将导致任务调度和资源分配分离,会带来一定程度的资源浪费和应用性能下降。现有的调度策略也未能充分考虑多数据中心环境的资源异构性以及云任务的特性,难以做到针对性的优化。因此,研究多数据中心云环境下的任务调度和资源分配的协同优化具有重要的理论和现实意义。针对多数据中心环境下云任务调度的问题,本文从U2D和D2D云任务的特征出发,围绕其在多数据中心环境下的调度机制和资源分配方式展开研究。本文的主要工作包括以下三个方面:·首先,本文研究了多数据中心环境下的U2D任务调度问题。为了更好地应对资源异构性,这里选取了较为复杂的混合云环境进行研究。首先针对不同区域的用户请求和多数据中心环境进行建模,设计了基于混合云的U2D任务服务模型和效益最大化优化模型。然后,分别针对静态和动态场景做具体分析,并设计相应的分布式调度算法实现各区域的任务调度、私有云和公有云数据中心上的任务调度和资源分配。该算法分层分布式地实现了混合云中的任务调度和异构资源分配,可以应对大规模的U2D任务请求。仿真结果表明,该算法能够保证任务时延需求,相对于基于提前预知的T-slot算法可以实现较高的服务效益,并显著降低算法运行时间。·其次,本文研究了多数据中心环境下的D2D任务调度问题。数据中心间网络(Inter-DC)存在大量的数据同步或备份任务,这些任务通常具有任播特性,并且可以通过存储-转发机制实现数据传输。本研究首先结合上述这两个特性,提出GlobalAMny传输机制,动态地全局选择任播目的数据中心。然后,本文设计了效益最大化模型,并利用BackPressure算法提出了逐跳的GlobalAny传输算法。为了降低传输时延,本文还综合逐跳传输和直接传输的特点,设计了数据传输的加速算法GlobalAny_Ext。仿真结果表明,GlobalAny算法提高了平均服务效益,降低了算法运行时间,而传输加速方案GlobalAny_Ext则显著降低了任务的时延。·最后,本文研究了多层Inter-DC光网络(ML-IDCON)中的DoT任务调度问题。区别于普通IP网络,光网络拥有较大带宽,但需要提前建立光路,并且资源分配粒度较大,因此ML-IDCON成为大型云服务商的选择。针对ML-IDCON中的DoT调度问题,本研究首先尝试利用IP层剩余的带宽资源实现数据传输,即通过时间扩展网络(TEN)方法将多数据中心环境中动态的网络、计算、存储资源分配问题转化为静态的网络流问题,并采用最小费用最大流算法来降低任务调度成本。若IP层资源不足,本研究将利用宽度优先搜索方法,迭代地在传输路径上找到成本最低的光路建立方案,并实现DoT调度和处理。仿真结果表明,对于DoT调度,灵活栅格ML-IDCON在阻塞概率,收发器的能量消耗和DC存储使用均优于固定栅格ML-IDCON。该算法在略微增加能耗的基础上,显著地降低了请求阻塞率、数据存储需求以及算法运行时间。
[Abstract]:With the development of business, more and more enterprises or organizations have begun to provide services to users based on multi data center cloud environment or even multi cloud environment. Multi data center or multi cloud can provide shorter time delay, higher bandwidth and lower cost service for different geographic users, and multiple data centers can be built and compatible with each other. Multiple data centers, while providing better service performance, also increase the complexity and challenge of cloud application management. On the one hand, the multi data center environment will have large-scale users to data center (U2D) and data center to data center (D2D). These tasks usually contain large numbers of tasks. On the other hand, there are a large number of different types, different granularity and heterogeneous resources, and the allocation, performance and cost model of these resources are not the same. With the increase of infrastructure resources and heterogeneity, more and more The enterprise separations the management of infrastructure and application and development so that the application developers can be developed quickly, which can quickly respond to user needs and market changes. This will lead to the separation of task scheduling and resource allocation, which will bring a certain degree of waste of resources and lower application performance. The existing scheduling strategy also fails to consider the majority of the majority. According to the resource heterogeneity of the central environment and the characteristics of the cloud task, it is difficult to optimize the target. Therefore, it is of great theoretical and practical significance to study the cooperative optimization of task scheduling and resource allocation under the cloud environment of multi data center. In view of the problem of cloud task scheduling in the multi data center environment, this paper from the U2D and D2D cloud tasks The main work of this paper includes the following three aspects: first, this paper studies the U2D task scheduling problem in the multi data center environment. In order to better cope with the heterogeneity of resources, a more complex mixed cloud environment is selected. Firstly, the U2D task service model and the benefit maximization optimization model based on mixed cloud are designed to model the user requests and multi data center environment in different regions. Then, the static and dynamic scenarios are analyzed, and the corresponding distributed scheduling algorithms are designed to implement the task scheduling in each region. Task scheduling and resource allocation on the cloud and public cloud data center. This algorithm realizes the task scheduling and heterogeneous resource allocation in the mixed cloud, and can deal with large-scale U2D task requests. The simulation results show that the algorithm can guarantee the task delay requirement, and the phase based T-slot algorithm based on pre prediction can be implemented. Higher service benefits and significantly lower algorithm running time. Secondly, this paper studies the D2D task scheduling problem under the multi data center environment. The data center network (Inter-DC) has a large number of data synchronization or backup tasks. These tasks usually have the character of the broadcast, and the data transmission can be realized through the storage and forwarding mechanism. First, the GlobalAMny transmission mechanism is proposed to dynamically select the data center of the destination. Then, the benefit maximization model is designed, and the hop by hop GlobalAny transmission algorithm is proposed by using the BackPressure algorithm. In order to reduce the transmission time delay, this paper also combines the hop by hop transmission and direct transmission. GlobalAny_Ext. simulation results of the acceleration algorithm of data transmission have been designed to show that the GlobalAny algorithm improves the average service efficiency and reduces the running time of the algorithm, while the transmission acceleration scheme GlobalAny_Ext reduces the task delay significantly. Finally, this paper studies the scheduling problem of DoT task in the multi-layer Inter-DC optical network (ML-IDCON). Unlike ordinary IP networks, optical networks have large bandwidth, but need to establish optical paths ahead of time, and the granularity of resource allocation is large. So ML-IDCON becomes the choice of large cloud service providers. In view of the DoT scheduling problem in ML-IDCON, this study first attempts to use the remaining bandwidth resources of IP layer to realize data transmission, that is, through the time extended network (TEN). Methods the dynamic network, computing, storage resource allocation problem in the multi data center environment is transformed into a static network flow problem, and the minimum cost maximum flow algorithm is used to reduce the cost of task scheduling. If the IP layer is insufficient, this research will use the width first search method to find the lowest cost path on the transmission path. Establish and implement the DoT plan, scheduling and processing. The simulation results show that for DoT scheduling, flexible grid ML-IDCON in blocking probability, energy consumption and DC storage were better than the fixed grid transceiver using the algorithm ML-IDCON. based on the increase in energy consumption slightly, significantly reduce the request blocking rate, data storage requirements and running time of the algorithm.
【学位授予单位】：中国科学技术大学
【学位级别】：博士
【学位授予年份】：2017
【分类号】：TP3

【相似文献】