面向IaaS的云计算自适应资源管理机制

发布时间：2018-05-01 10:16

本文选题：云计算 + 自适应机制　；参考：《西北工业大学》2015年博士论文

【摘要】：云计算(Cloud Computing)能通过虚拟化的方式,将系统内部多样化的异构资源封装成统一的计算资源池,并按需为用户提供所需的计算资源,是一种面向服务的计算模式。其中,基础设施即服务(Infrastructure as a Service, IaaS)作为云计算服务中的基础服务,主要是为用户提供按需的底层基础设施资源(计算资源、存储空间和网络带宽等)。而作为IaaS研究核心的资源管理和调度策略,对系统的稳定性和服务的可靠性起着非常重要的作用。本文着重对IaaS中的虚拟资源管理机制,以及其在高性能应用集群、Web服务集群中的自适应资源分配及调度机制等方面进行了深入研究。首先通过现有流行的集群、虚拟化技术、云计算平台等相关研究,针对其资源的特点及现有资源管理策略中的不足,结合面向市场(Marketing-Oriented)的资源调度策略、资源协商(Negotiating)机制、资源预留(Advance Reservation)机制及自适应机制,侧重研究IaaS自适应资源管理机制；其次,针对现有集群资源管理机制在网格应用集群、高性能计算集群、电子商务应用中大量使用的Web集群,对于自适应资源协调机制所存在的不足,研究了基于云计算IaaS服务的资源分配及调度自适应弹性管理机制。最后,由于系统识别部分是自适应控制的关键,鉴于最小成分分析算法在自适应系统识别领域有显著的成效,因此本文结合目前最小成分分析算法在系统识别方面的特点及存在问题,研究IaaS服务的资源自适应识别算法。本文所论述的研究内容,都是本文作者在Platform公司参与网格和云计算方面的合作课题中所取得的研究成果,这些成果不仅纳入市场上某些主流的商业项目,而且被成功、广泛应用到欧洲原子能(CERN)、美国国家航空航天局(NASA)等研究中心,可以保证研究具有高效性及高可用性。主要研究工作及创新点如下：1)为了有效地管理虚拟资源,使资源使用率最大化,并能保证用户对资源使用的有效性,本文通过对虚拟资源的划分、预留及调度策略等方面的研究,提出了一种面向虚拟资源的IaaS资源管理机制。该机制实现了资源的按需分配和调度,从而为用户提供有效的IaaS服务。仿真实验结果表明,该方法能够提高虚拟资源的使用率及保证用户对资源使用的有效性。2)提出了一种面向网格化Web集群的按需IaaS资源管理机制及其分配调度策略。Web集群使用一组作为后端节点的系统资源,能够有效地解决传统Web服务器的负载均衡及故障恢复问题,但是到目前为止,如何使Web集群更有效地利用系统资源尚未见到有效的解决方案。针对这一问题,本章采取一种网格化Web集群的思想,并针对网格化的Web集群后端节点制定了一种按需IaaS资源管理机制及其分配调度策略,此技术路线能够有效地提高Web集群对资源的使用效率3)提出了一种面向高性能应用集群的自适应IaaS资源弹性管理机制。高性能计算(High Performance Computing,HPC)集群通过整合并利用一组计算节点的计算能力来处理大量提交到高性能应用系统中的计算作业。吞吐量是高性能计算机群的一个重要考察指标,然而不恰当的容量规划及其相关的资源管理机制,将会严重影响高性能计算集群的吞吐量,从而降低高性能应用集群的效率,同时还会大大浪费投入的成本。为了克服这些问题,本文所提出的弹性管理机制通过动态按需调节高性能应用环境中异构资源的特性,使得高性能应用环境能够针对等待作业进行异构资源的按需切换,从而减少了等待作业以提高系统的吞吐量、提高性能。4)电子商务应用中,Web集群能够作为一个强大的Web服务器来处理巨大的并发请求。然而,传统的Web集群在对固有的基础设施进行预部署和预配置时,由于不适当的预先计划,可能会导致过多的成本投入或资源冗余、浪费,尤其是面对无法预计的峰值问题。因此本文首先在云计算系统上通过管理虚拟机来部署Web集群,以替代在固定的基础设施上部署Web集群的方法,然后进一步通过基于借入-借出(Lend-Brow)策略的资源预留机制,提出一种自适应IaaS资源管理机制和高低水位线(low-high Water Line) I阈值检测负载均衡算法,使Web集群能够按需对集群负载进行计算节点的自适应调节,从而提高了资源的利用率,且有效地减少了投入成本和能耗。5)提出了两种基于递归最小成分分析的IaaS自适应系统识别算法。由于云计算系统资源的动态性,及自适应控制研究对象的不确定性,常规反馈调节机制的效果往往难以令人满意。针对自适应调节的不确定性所造成的系统波动,需要一个高效、精确的系统识别算法,从而为动态变化的系统负载提供预测的计算资源。近来最小成分分析法被广泛用于系统识别方面,因此本章对其进行分析并优化,进而在此基础上构建了a-RMCA和f-RMCA两种算法。其中a-RMCA算法具有高精度的优点,但是计算速度有待改进；而f-RMCA算法运算速度较快,但精度稍低。最后本文从数学的角度证明了算法的收敛性,并通过数学建模仿真和真实环境测试,证明了算法的有效性。
[Abstract]:Cloud Computing (Cloud Computing) can encapsulate the heterogeneous heterogeneous resources in the system into a unified pool of computing resources by virtualization, and provide users with the required computing resources on demand. It is a service oriented computing model. Among them, the infrastructure (Infrastructure as a Service, IaaS) is used as a cloud computing service. The basic service is to provide users with the underlying infrastructure resources (computing resources, storage space and network bandwidth). As the core of the IaaS research, the resource management and scheduling strategy plays a very important role in the stability of the system and the reliability of the service. This paper focuses on the virtual resource management mechanism in IaaS. And it has studied the adaptive resource allocation and scheduling mechanism in Web service cluster. Firstly, through the popular cluster, virtualization technology, cloud computing platform and other related research, the characteristics of its resources and the shortage of existing resource management strategies are combined with the market (Marketing-Orient). ED) resource scheduling strategy, resource negotiation (Negotiating) mechanism, resource reservation (Advance Reservation) mechanism and adaptive mechanism, focusing on the study of IaaS adaptive resource management mechanism. Secondly, the existing cluster resource management mechanism in the grid application cluster, high performance computing cluster, a large number of Web clusters used in e-commerce applications, In the shortcomings of adaptive resource coordination mechanism, this paper studies the adaptive resilient management mechanism of resource allocation and scheduling based on cloud computing IaaS services. Finally, because the system recognition part is the key to adaptive control, the minimum component analysis algorithm has remarkable achievements in the field of adaptive system recognition, so this paper combines the present situation. The characteristics and existing problems of the minimum component analysis algorithm in system identification and its existing problems are studied. The research content of this paper is the research results obtained by the author in Platform company's cooperation in grid and cloud computing. These results are not only included in some market owners of IaaS. The flow of commercial projects, and has been successfully applied to the European Atomic Energy Energy (CERN), the National Aeronautics and Space Administration (NASA) and other research centers, can ensure the efficiency and high availability of the research. The main research work and innovation are as follows: 1) in order to effectively manage virtual resources, maximize the utilization of resources, and ensure the user's capital The effectiveness of source use is based on the research of virtual resource division, reservation and scheduling strategy. A virtual resource oriented IaaS resource management mechanism is proposed. This mechanism realizes the allocation and scheduling of resources on demand and provides effective IaaS services for users. The simulation experiment results show that this method can improve the virtual resource. The utilization rate of quasi resource and the effectiveness of ensuring the user's use of resources.2) proposed a IaaS resource management mechanism for grid oriented Web cluster and its allocation and scheduling policy.Web cluster to use a set of system resources as backend nodes, which can effectively solve the load balancing and failure recovery of the traditional Web server. So far, how to make Web clusters more efficient use of system resources has not yet seen effective solutions. In this chapter, a grid based Web cluster is adopted in this chapter, and a kind of IaaS resource management mechanism and allocation scheduling strategy for the grid based Web cluster back end nodes are formulated. Effectively improving the efficiency of Web cluster for resource use 3) an adaptive IaaS resource flexible management mechanism for high performance application clusters is proposed. High Performance Computing (High Performance Computing, HPC) clusters deal with a large number of computations submitted to high performance application systems by integrating and utilizing a set of computing nodes' computing power. Throughput is an important indicator of high performance computer groups. However, inappropriate capacity planning and related resource management mechanisms will seriously affect the throughput of high performance computing clusters, thus reducing the efficiency of high performance application clusters and the cost of big wave costs. In order to overcome these problems, this paper The proposed flexible management mechanism regulates the characteristics of heterogeneous resources in the high performance application environment by dynamic demand, making the high performance application environment switching on the demand for waiting jobs for heterogeneous resources, thus reducing the waiting job to improve the throughput of the system and improving the performance.4). In the e-commerce application, the Web cluster can be used as a one. A powerful Web server handles huge concurrent requests. However, when the traditional Web cluster is predeployed and preconfigured for inherent infrastructure, due to improper pre planning, it may lead to excessive cost input or resource redundancy and waste, especially in the face of unanticipated peak problems. The Web cluster is deployed by managing virtual machines to replace the method of deploying Web clusters on a fixed infrastructure, and then an adaptive IaaS resource management mechanism and a low-high Water Line (low-high Water Line) I threshold detection load are proposed in order to replace the method of deploying a cluster on a fixed infrastructure. The algorithm makes the Web cluster adaptively adjust the computing nodes of the cluster load on demand, thus improving the utilization of resources, reducing the input cost and energy consumption.5 effectively. Two kinds of IaaS adaptive system recognition algorithm based on recursive least component analysis are proposed. In order to control the uncertainty of the research object, the effect of the conventional feedback regulation mechanism is often unsatisfactory. For the system fluctuation caused by the uncertainty of adaptive adjustment, a high efficient and accurate system recognition algorithm is needed to provide the pre measured computing resources for the dynamic system load. It is widely used for system recognition, so this chapter analyses and optimizes it. On this basis, two algorithms of a-RMCA and f-RMCA are constructed. Among them, the a-RMCA algorithm has the advantages of high precision, but the computing speed needs to be improved, while the f-RMCA algorithm has a fast operation speed, but the precision is a little low. Finally, this paper proves the algorithm from the mathematical point of view. The validity of the algorithm is proved by mathematical modeling and simulation and real environment testing.

【学位授予单位】：西北工业大学
【学位级别】：博士
【学位授予年份】：2015
【分类号】：TP393.09

【相似文献】