虚拟化集群的磁盘IO QoS控制

发布时间：2018-08-31 10:24

【摘要】：在计算机科学领域，云计算目前被视为客户机/服务器架构变革后的另一大技术革新。为了满足云用户的不同服务需求，云计算系统必须统一管理一个巨大的资源集群。因此，云资源管理成为当前研究人员所面临的一个巨大挑战。云计算资源管理的主要任务包括数据存储和资源管理两方面，二者都涉及到磁盘IO资源。磁盘IO资源是云计算环境下所有资源中重要的一个，一个高效的磁盘IO QoS系统将会为云计算服务供应商、云用户和终端用户带来很大的便利。然而，在设计和实现一个磁盘IO QoS调度系统时需要考虑许多因素，诸如系统高性能、管理灵活性、兼容性等等。如何实现虚拟化存储的磁盘IO资源调度策略并且确保磁盘IO的服务质量成为具有实践意义的重要研究课题。本论文的研究工作来源于国家863计划课题项目。该项目期望能够通过现有的虚拟化技术和流量控制技术实现一种有效的磁盘流量的控制方案，，以实现对特定单元进行流量控制，保证系统资源的高可用性，确保云计算服务提供商为用户提供优质、稳定的数据流量方案。针对以上需求，本文提出一个基于虚拟化技术和QoS控制技术的磁盘IO资源管理方案，对磁盘IO流量进行有效管理和控制。这一磁盘IO管理方案具有提供多种服务质量的高性能和管理的灵活性，并在已有的系统测试环境下对模块进行测试优化，为线上运行的虚拟化集群环境提供高可靠保证的实验验证。在本论文中，具体研究工作及创新包括以下几方面： 1.设计和实现磁盘IO QoS控制模块。利用漏桶算法和加权公平队列算法的结合模型，实现两种对应的控制策略，用于限制系统资源的throttle策略和用于针对不同请求分配不同优先级的weight策略。利用device mapper技术将模块实现为独立的虚拟化块设备层，降低引入模块对系统性能的影响。 2.实现QoS管理控制接口。修改虚拟化平台的相关接口脚本，加入QoS相关设计，实现虚拟设备QoS参数的动态配置，方便系统整合和参数管理。模块整合后，评估并分析其对现有系统的影响。 3.分析磁盘IO优化因素。对系统磁盘IO进行测试分析，结果表明，通过加入对IO请求大小和并发IO数量等因素的控制可以提高磁盘IO效率，优化系统磁盘性能。本论文通过对虚拟化环境下的磁盘资源管理技术现状进行分析和研究，提出基于虚拟化技术的磁盘IO QoS控制模块，选择合理的设计方案并利用device mapper技术进行实现。为了使该模块更易于系统管理，文中也提供了磁盘QoS管理的设计方案和实现细节。经验证，该模块具备完整的磁盘IO QoS功能，能够提供多种IO调度策略，为用户提供不同的服务质量，确保系统效率。其策略能够限制磁盘IO资源在某一值附近，波动不超过整体的3%，防止突发流量的情况，并且在满足功能需求的基础上，将模块整合带给系统的影响控制在2%以内。
[Abstract]:In the field of computer science, cloud computing is now seen as another major technological innovation following the change in client / server architecture. In order to meet the different needs of cloud users, cloud computing systems must manage a huge cluster of resources. Therefore, cloud resource management has become a great challenge for researchers. The main tasks of cloud computing resource management include data storage and resource management, both of which involve disk IO resources. Disk IO resource is one of the most important resources in cloud computing environment. An efficient disk IO QoS system will bring great convenience to cloud computing service providers, cloud users and end users. However, many factors need to be taken into account when designing and implementing a disk IO QoS scheduling system, such as system high performance, management flexibility, compatibility and so on. How to implement the disk IO resource scheduling strategy of virtualized storage and ensure the quality of service of disk IO has become an important research topic with practical significance. The research work of this thesis comes from the national 863 project. The project hopes to realize an effective disk traffic control scheme through the existing virtualization technology and flow control technology, in order to achieve flow control of specific units and ensure the high availability of system resources. Ensure that cloud computing service providers provide users with quality, stable data flow solutions. According to the above requirements, this paper proposes a disk IO resource management scheme based on virtualization technology and QoS control technology, which can effectively manage and control disk IO traffic. The disk IO management scheme has the advantages of high performance and flexibility to provide multiple quality of service, and the modules are tested and optimized under the existing system test environment. It provides high reliable experimental verification for the virtualized cluster environment running on the line. In this paper, the specific research work and innovation includes the following aspects: 1. Design and implement disk IO QoS control module. Using the combination model of leaky bucket algorithm and weighted fair queue algorithm, two corresponding control strategies are implemented, which are used to restrict the system resource throttle policy and to assign different priority weight policy to different requests. The module is realized as an independent virtualized block device layer by using device mapper technology to reduce the impact of the introduced module on the system performance. 2. Implement QoS management control interface. Modify the interface script of virtualization platform, add QoS related design, realize the dynamic configuration of QoS parameters of virtual device, facilitate system integration and parameter management. After module integration, evaluate and analyze its impact on existing systems. Analyze disk IO optimization factors. The system disk IO is tested and analyzed. The results show that the system disk performance can be optimized by adding the control factors such as the size of IO request and the number of concurrent IO to improve the disk IO efficiency. Based on the analysis and research on the current situation of disk resource management technology in virtualization environment, this paper proposes a disk IO QoS control module based on virtualization technology, selects a reasonable design scheme and implements it by using device mapper technology. In order to make the module easier to manage, the paper also provides the design and implementation details of disk QoS management. It has been proved that the module has complete disk IO QoS function and can provide a variety of IO scheduling strategies to provide users with different quality of service and ensure the efficiency of the system. The strategy can limit disk IO resources near a certain value, fluctuate less than 3 percent of the whole, prevent sudden traffic, and control the impact of module integration to less than 2 percent on the basis of meeting the functional requirements.
【学位授予单位】：上海交通大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333.35

【参考文献】