云计算资源调度方法改进与应用设计

发布时间：2018-08-06 13:34

【摘要】：论文云计算资源调度方法改进与应用设计,针对国内外云计算的商业特性及服务模式,对云计算服务中高并发、高实时性的在线服务模式提出了更高的要求,而传统的资源调度策略下,仅仅对虚拟化计算、内存、存储和网络资源进行分配,导致LC(Latency Critical)负载服务时延过长、平台整体服务质量较差的现实,研究了在此背景下新兴的CAT(Cache Allocation Technology)技术,创新性地提出了使用PID(Proportion Integration Differentiation)算法作为资源调度的控制器,利用WCP主板作为多芯片处理器(CMPs)的平台,搭配Intel?Xeon?E5系列的处理器,实现在Mesos资源调度系统的基础上,提出针对多核、多芯片环境下缓存资源的调度方法,弥补了业界在缓存资源控制方面的缺陷,其选题具有较强的针对性和实用性。为此研究了在Linux操作系统下,利用Mesos资源调度系统和Marathon框架,搭建并模拟了一个master节点、两个slave节点的Mesos集群环境,测试了Marathon框架的负载启动过程,资源分配方式及状态监控功能。在进一步的研究中,利用WCP主板和Intel?Xeon?E5系列的处理器作为硬件服务平台,结合Mesos、Marathon和Docker容器技术,通过使用C语言在Linux系统中完成了PCM数据监控模块、PID控制模块、前端参数读取模块、后端数据写入模块的设计,并利用SQL语言在Linux系统下完成了Mysql数据库的设计,包括前端参数存储和后端数据存储的表结构,在此基础上利用HTML、JavaScript语言完成了后端数据可视化系统的设计,进行了四种后端数据动态展示、PID控制开关、PID算法参数配置、负载添加及扩展功能的设计。该项目的研究对促进云计算资源调度技术的发展具有一定的参考价值。论文的研究工作主要体现在如下几个方面:其一,研究了国内外云计算资源调度方法的现状和最新成果,结合传统资源分配方式存在的局限性,进行了CAT缓存分配技术、PID自动控制算法、云计算资源调度技术、Docker容器技术的分析,重点分析了云计算资源调度方法中的自动控制模块的工作原理及协议标准。其二,研究了Mesos资源调度系统和Marathon框架的相关技术,在对相关平台和软件进行对比和选择基础上,在三台Linux主机中利用Linux Shell命令分别安装上一个Mesos master节点,两个Mesos slave节点,并在每个节点上安装PID控制器,搭建了一个小型的Mesos集群,利用Docker容器技术对负载进行容器化封装,进行了PID控制器的功能测试和性能测试,以此掌握整个Mesos集群中各个主节点、控制节点间的通信过程。其三,进行了PID控制器的设计,使负载具备通过PID控制项来自动调整性能数据的功能,并可由PCM数据监控项来将实时数据发送至控制器内部,作为真实数据与控制器中的目标数据求取差值,通过PID控制算法的数学公式求出控制输出值,并根据此输出值控制缓存资源的分配。此外还进行了Mysql数据库的设计,使后端数据及前端参数实现数据的持久化,能够存储来自前后端程序的数据。其四,进行了后端数据可视化系统的设计,研究了Web服务器的工作原理及工作模式,通过对PID控制器可视化需求的分析,选择合适的Web服务器并利用HTML和JavaScript语言开发了包括数据动态展示、控制按钮、参数调整栏框、负载添加栏框的设计和功能测试。其五,将整个系统软硬件连接在一起做整体的功能测试和性能测试,包括启动负载,设置缓存资源的大小、后端数据存储,前端数据读取,数据动态展示等测试,选取多种LC负载与其他负载进行性能测试。测试结果表明,系统整体达到预定设计的要求。
[Abstract]:The improvement and application design of cloud computing resource scheduling method, aiming at the business characteristics and service mode of cloud computing at home and abroad, put forward higher requirements for high concurrency and high real-time online service mode in cloud computing service, and the traditional resource scheduling strategy only distributes the virtual computing, memory, storage and network resources. The LC (Latency Critical) load service delay is too long and the overall service quality of the platform is poor. Under this background, the new CAT (Cache Allocation Technology) technology is studied. The PID (Proportion Integration Differentiation) algorithm is used as the controller of the resource scheduling, which uses the WCP main board as the multi core. The platform of chip processor (CMPs), with the processor of Intel? Xeon? E5 series, is implemented on the basis of the Mesos resource scheduling system, and proposes a scheduling method for the caching resources in the multi core and multi chip environment, which makes up for the defects in the field of caching resources control, and its selection has a strong pertinence and practicability. Therefore, a study on Linux is made in this paper. Under the operating system, the Mesos resource scheduling system and the Marathon framework are used to build and simulate the Mesos cluster environment of a master node and two slave nodes. The load starting process, the resource allocation mode and the state monitoring function of the Marathon framework are tested. In further research, the WCP master and Intel Xeon E5 series processor are used in further research. As a hardware service platform, combined with Mesos, Marathon and Docker container technology, the PCM data monitoring module, PID control module, the front end parameter reading module, the back end data write module are designed in the Linux system using the C language, and the Mysql database is designed with the SQL language in the Linux system, including the front end parameter memory. The table structure of storage and back end data storage is built on this basis by using HTML and JavaScript language to design the back end data visualization system. Four kinds of back end data dynamic display, PID control switch, PID algorithm parameter configuration, load addition and extension function are designed. The research of this project promotes the technology of cloud computing resource scheduling. Development has a certain reference value. The research work of this paper is mainly embodied in the following aspects: first, it studies the current situation and latest achievements of cloud computing resource scheduling methods at home and abroad, and combines the limitations of traditional resource allocation methods, and carries out CAT cache allocation technology, PID automatic control algorithm, cloud computing resource scheduling technology, Doc The analysis of Ker container technology focuses on the principle and protocol standard of the automatic control module in the cloud computing resource scheduling method. Secondly, the related technologies of the Mesos resource scheduling system and the Marathon framework are studied. On the basis of the comparison and selection of the related platforms and software, the Linux Shell command is used in three Linux hosts. Install a Mesos master node, two Mesos slave nodes respectively, and install PID controller on each node, build a small Mesos cluster, use the Docker container technology to container the load, carry out the function test and performance test of the PID controller, so as to master the main nodes in the whole Mesos cluster and control the control. Thirdly, the design of the PID controller is carried out. The load has the function of automatically adjusting the performance data through the PID control item, and the real-time data can be sent to the controller by the PCM data monitoring item, and the difference between the real data and the target data in the controller is obtained, and the mathematics is controlled by the PID control algorithm. The output value is obtained by the formula, and the allocation of caching resources is controlled according to the output value. In addition, the design of the Mysql database is also carried out, which makes the back end data and the front end parameters to be persistent and can store the data from the front and back programs. Fourthly, the design of the back end data visualization system is carried out and the work of the Web server is studied. By analyzing the visual requirements of the PID controller, selecting the appropriate Web server and using HTML and JavaScript language to develop the dynamic display of data, the control button, the parameter adjustment hurdles, the design of the load adding bar frame and the functional test. Functional testing and performance testing, including starting load, setting up the size of caching resources, back end data storage, front end data reading, dynamic display of data and other tests, selected a variety of LC loads and other loads to perform performance testing. Test results show that the system as a whole meets the requirements of the predetermined design.
【学位授予单位】：成都理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP3

【参考文献】