高能物理云平台中的弹性计算资源管理机制

发布时间：2018-12-13 02:29

【摘要】：虚拟化技术作为一种新的资源管理技术,正在高能物理领域得到越来越广泛的应用。静态虚拟机集群方式已经逐渐不能满足多作业队列对于计算资源动态的需求。为此,实现了一种云计算环境下面向多作业队列的弹性计算资源管理系统。系统通过高吞吐量计算系统HTCondor运行计算作业,使用开源的云计算平台Openstack管理虚拟计算节点,给出了一种结合虚拟资源配额服务,基于双阈值的弹性资源管理算法,实现资源池整体伸缩,同时设计了二级缓冲池以提高伸缩效率。目前系统已部署在高能所公共服务云IHEPCloud上,实际运行结果表明,当计算资源需求变化时系统能够动态调整各队列虚拟计算节点数量,同时计算资源的CPU利用率相比传统的资源管理方式有显著的提高。
[Abstract]:As a new resource management technology, virtualization technology is more and more widely used in the field of high energy physics. The static virtual machine cluster mode has been unable to meet the demand of multi-job queue for computing resource dynamics. Therefore, a flexible computing resource management system for multiple job queues in cloud computing environment is implemented. The system runs computing jobs through high throughput computing system (HTCondor) and uses open source cloud computing platform (Openstack) to manage virtual computing nodes. An elastic resource management algorithm based on double threshold is presented, which combines virtual resource quota service. The overall expansion of the resource pool is realized, and a two-stage buffer pool is designed to improve the expansion efficiency. At present, the system has been deployed on the public service cloud IHEPCloud of the High Energy Institute. The actual operation results show that the system can dynamically adjust the number of virtual computing nodes in each queue when the demand for computing resources changes. At the same time, the CPU utilization rate of computing resources is significantly improved compared with the traditional resource management method.
【作者单位】：中国科学院高能物理研究所;中国科学院大学;
【基金】：国家自然科学基金(No.11575223,No.11605223,No.11305192,No.11605224)
【分类号】：TP393.09

【参考文献】