基于Docker的资源预调度策略构建弹性集群的研究

发布时间：2018-01-02 18:39

本文关键词：基于Docker的资源预调度策略构建弹性集群的研究　出处：《浙江理工大学》2017年硕士论文　论文类型：学位论文

【摘要】：在实际环境中,服务器会经常出现不可预知的负载需求,特别是像天猫双11等活动每年都在刷新交易记录,大量的订单交易背后是突增的巨大用户访问量,此时需要快速增加集群中服务器的数量来应对额外负载,同时在流量下降时,需要停止空闲服务器以降低成本。因此构建弹性集群,实现集群服务器数量的自动伸缩显得尤为重要。弹性伸缩是云计算中的一个重要功能,而虚拟化技术是支撑云计算的重要技术基石,虚拟化的程度不同决定了云平台的架构,Docker容器技术作为操作系统级的虚拟化为云平台的架构提出了新思路,它是基于进程级隔离的,相较于目前大多数基于虚拟机技术的云平台而言,移除了客户操作系统层,能够提供更快的启停速度和更优的资源利用率。本文研究使用Docker容器技术作为底层云平台虚拟化技术,通过对集群性能的实时监控及资源的合理调度,构建弹性集群,应对不可预知的负载需求,保证云服务平台低成本、高效、稳定的运营,大大提高云平台的资源利用率,更加有效的应对网络峰值,提供高吞吐率和低延迟的服务。为此,本文实现了以下内容:(1)解决构建容器集群的两个主要问题:跨主机容器通信和文件共享。针对前者提出了三种解决方案:OVS+GRE、Flannel、直接路由,针对后者提出了构建镜像仓库解决镜像共享问题,结合DRBD构建多节点NFS网络文件系统解决数据持久化存储问题,为弹性集群提供实践环境。(2)实现了Docker弹性集群性能监控。基于OpenTSDB和Docker容器技术实现了对容器及集群节点的性能数据的采集、存储、分组聚合及展示,也为构建弹性集群中弹性伸缩组的触发机制提供了判定条件。(3)构建弹性伸缩组。使用Haproxy提供负载均衡服务、Consul提供弹性伸缩组的服务自动发现机制、Confd定期获取Consul中的服务信息并动态生成Haproxy配置文件。(4)研究并实现了多种资源调度策略。首先是构建弹性集群过程中的基于资源使用量的资源调度策略,然后为解决集群资源使用量不均匀问题实现了宿主机调度策略,最后实现了基于LMS算法的负载预测调度策略。(5)构建Docker弹性集群并对系统进行了测试,验证了弹性集群的有效性和可行性。
[Abstract]:In the actual environment, the server will often load demand unpredictable, especially as Tmall double 11 every year to refresh the record of the transaction, the transaction is huge orders a lot behind the sudden increase in user traffic, the rapid increase in the number of servers in the cluster to cope with the additional load, while the flow down the need to stop the server idle, to reduce the cost. So the construction of automatic telescopic elastic cluster, cluster server number is very important. Flexibility is an important function in cloud computing, and virtualization technology is an important technical support for the cornerstone of cloud computing, virtualization degree determines the cloud platform architecture, Docker container as the operating system level virtualization to cloud platform architecture and put forward new ideas, it is based on the isolation process level, compared to the most current based on virtual machine technology The cloud platform, remove the client operating system layer, can provide a faster start stop speed and better resource utilization. This paper use the Docker container virtualization platform as cloud bottom, through reasonable scheduling of real-time monitoring of the cluster performance and resources, construction of flexible cluster, cope with the unpredictable load demand the guarantee of cloud service platform with low cost, high efficiency, stable operation, greatly improve the utilization rate of resources in a cloud, with peak network more effectively, provide high throughput and low delay services. Therefore, this paper is as follows: (1) to solve two major problems in construction of container cluster: cross host container communication and file sharing. For the former put forward three kinds of solutions: OVS+GRE, Flannel, direct route, for the latter is proposed to solve the problem of mirror image share warehouse, combined with the construction of DRBD multi node NFS The network file system to solve data storage problems, provide a practical environment for elastic clusters. (2) the Docker elastic cluster performance monitoring. OpenTSDB and Docker container storage technology to achieve the performance data of the container and the cluster nodes acquisition based on packet aggregation and display, but also provides conditions for construction of flexible trigger mechanism telescopic group of elastic in the cluster. (3) to construct flexible group. The use of Haproxy to provide load balancing services, Consul provides a flexible set of automatic service discovery mechanism, Confd regular Consul to obtain the service information and dynamically generate the Haproxy configuration file. (4) the research and implementation of multi resource scheduling strategy. The first is to build a resource the use of resource scheduling strategy based on the amount of flexibility in the process of cluster, and then to solve the problem of cluster resources usage is not uniform implementation of the host scheduling strategy, finally realized The load forecasting and scheduling strategy based on LMS algorithm. (5) constructing Docker elastic cluster and testing the system to verify the effectiveness and feasibility of the elastic cluster.

【学位授予单位】：浙江理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP393.09;TP368.5

【相似文献】