基于HDFS的虚拟机镜像存储机制研究
发布时间：2019-03-01 17:44
【摘要】：随着信息技术的迅速发展及互联网数据的急剧膨胀,互联网应用的动态资源需求快速增长,增加了服务器的投资需求,给互联网企业带来了巨大的经济负担。然而,服务器增加易造成资源浪费,特别在系统负载较低时,大量的服务器将处于闲置状态。云计算的资源高效利用特性为企业降低运营成本、资源的按需分配和自动部署提供了可行途径。作为云计算的核心支撑技术,虚拟化能对服务器进行有效整合,支持虚拟机动态迁移,实现服务器的动态封装和运行时的负载均衡。近年来,多种虚拟化产品(如Xen, KVM)被开发出来,有效支撑了云计算的发展。 在典型的云计算环境中,虚拟机的部署需要高效存储设备的支持,而目前Xen等虚拟化方式采用NAS、SAN等作为存储模式,难以满足实际应用需要：一方面高速存储设备价格高昂、设备能耗较高且存储空间利用率较低；另一方面单一设备的低容错率易造成数据丢失或错误。针对云计算虚拟机环境中的存储问题,业界也提出了多种云存储方案,通过将众多异构、廉价的服务器和存储设备集合起来协同工作,提供数据存储和访问服务。其中的Hadoop分布式文件系统HDFS实现了海量数据的存储管理,并具有高可用性、高可靠性、高稳定性和低成本等优势,在许多企业和科研机构中得到了广泛应用。 但是现有的HDFS实现也存在数据访问延迟较大、不支持随机写等问题,限制了对虚拟机存储支持的应用。本文针对云计算环境下的Xen虚拟机存储问题,基于HDFS设计并实现了一个虚拟化平台解决方案,将虚拟机镜像存储扩展到云存储平台HDFS。论文主要工作包括以下几个方面： (1)深入分析了虚拟机动态迁移过程的性能瓶颈问题,提出了一种基于HDFS体系结构的虚拟机镜像存储框架,建立了支持虚拟机镜像动态运行的分布容错云存储机制。 (2)以HDFS系统为基础,通过引入日志结构文件系统(LFS)的思想,设计了一种支持虚拟机镜像存储的块级存储系统HBISS,解决了IDFS无法支持随机写的缺陷,为镜像数据的动态访问提供了支持。 (3)结合Xen虚拟机访问镜像文件的特性,通过对Xen内带的blktap工具扩展,建立了从虚拟机I/O到分布式云存储文件的访问机制,实现了虚拟机对HBISS镜像的访问。
[Abstract]:With the rapid development of information technology and the rapid expansion of Internet data, the dynamic resource demand for Internet applications is growing rapidly, which increases the investment demand of servers and brings a huge economic burden to Internet enterprises. However, the increase of servers is easy to waste resources, especially when the system load is low, a large number of servers will be idle. The efficient resource utilization of cloud computing provides a feasible way for enterprises to reduce operating costs, allocate resources on demand and deploy them automatically. As the core supporting technology of cloud computing, virtualization can effectively integrate servers, support dynamic migration of virtual machines, and realize dynamic encapsulation of servers and load balancing at runtime. In recent years, a variety of virtualization products (such as Xen, KVM) have been developed to effectively support the development of cloud computing. In a typical cloud computing environment, the deployment of virtual machines needs the support of efficient storage devices, but at present, Xen and other virtualization methods adopt NAS,SAN and other storage modes, which are difficult to meet the needs of practical applications: on the one hand, high-speed storage devices are expensive, The energy consumption of the equipment is high and the utilization of storage space is low. On the other hand, the low fault tolerance rate of a single device is easy to cause data loss or error. In order to solve the storage problem in cloud computing virtual machine environment, many cloud storage schemes are proposed, which can provide data storage and access services by integrating many heterogeneous and cheap servers and storage devices together to provide data storage and access services. The Hadoop distributed file system (HDFS), which implements the storage management of massive data, has the advantages of high availability, high reliability, high stability and low cost, and has been widely used in many enterprises and scientific research institutions. However, the existing implementation of HDFS also has some problems such as large delay of data access, not supporting random writing and so on, which limits the application of storage support for virtual machines. Aiming at the storage problem of Xen virtual machine in cloud computing environment, this paper designs and implements a virtual platform solution based on HDFS, which extends the virtual machine mirror storage to the cloud storage platform HDFS.. The main work of this paper is as follows: (1) the performance bottleneck of virtual machine dynamic migration is analyzed deeply, and a virtual machine mirror storage framework based on HDFS architecture is proposed. A distributed fault-tolerant cloud storage mechanism which supports the dynamic running of virtual machine mirror is established. (2) based on HDFS system, a block-level storage system HBISS, which supports virtual machine mirror storage is designed by introducing the idea of log structure file system (LFS), which solves the defect that IDFS can't support random writing. Provides support for dynamic access to mirrored data. (3) according to the characteristic of Xen virtual machine accessing image file, the access mechanism from virtual machine I / o to distributed cloud storage file is established by extending the blktap tool in Xen, and the access mechanism of virtual machine to HBISS image is realized.
【学位授予单位】：南京大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP302;TP333
本文编号：2432658
[Abstract]:With the rapid development of information technology and the rapid expansion of Internet data, the dynamic resource demand for Internet applications is growing rapidly, which increases the investment demand of servers and brings a huge economic burden to Internet enterprises. However, the increase of servers is easy to waste resources, especially when the system load is low, a large number of servers will be idle. The efficient resource utilization of cloud computing provides a feasible way for enterprises to reduce operating costs, allocate resources on demand and deploy them automatically. As the core supporting technology of cloud computing, virtualization can effectively integrate servers, support dynamic migration of virtual machines, and realize dynamic encapsulation of servers and load balancing at runtime. In recent years, a variety of virtualization products (such as Xen, KVM) have been developed to effectively support the development of cloud computing. In a typical cloud computing environment, the deployment of virtual machines needs the support of efficient storage devices, but at present, Xen and other virtualization methods adopt NAS,SAN and other storage modes, which are difficult to meet the needs of practical applications: on the one hand, high-speed storage devices are expensive, The energy consumption of the equipment is high and the utilization of storage space is low. On the other hand, the low fault tolerance rate of a single device is easy to cause data loss or error. In order to solve the storage problem in cloud computing virtual machine environment, many cloud storage schemes are proposed, which can provide data storage and access services by integrating many heterogeneous and cheap servers and storage devices together to provide data storage and access services. The Hadoop distributed file system (HDFS), which implements the storage management of massive data, has the advantages of high availability, high reliability, high stability and low cost, and has been widely used in many enterprises and scientific research institutions. However, the existing implementation of HDFS also has some problems such as large delay of data access, not supporting random writing and so on, which limits the application of storage support for virtual machines. Aiming at the storage problem of Xen virtual machine in cloud computing environment, this paper designs and implements a virtual platform solution based on HDFS, which extends the virtual machine mirror storage to the cloud storage platform HDFS.. The main work of this paper is as follows: (1) the performance bottleneck of virtual machine dynamic migration is analyzed deeply, and a virtual machine mirror storage framework based on HDFS architecture is proposed. A distributed fault-tolerant cloud storage mechanism which supports the dynamic running of virtual machine mirror is established. (2) based on HDFS system, a block-level storage system HBISS, which supports virtual machine mirror storage is designed by introducing the idea of log structure file system (LFS), which solves the defect that IDFS can't support random writing. Provides support for dynamic access to mirrored data. (3) according to the characteristic of Xen virtual machine accessing image file, the access mechanism from virtual machine I / o to distributed cloud storage file is established by extending the blktap tool in Xen, and the access mechanism of virtual machine to HBISS image is realized.
【学位授予单位】：南京大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP302;TP333
【参考文献】
相关期刊论文 前2条
1 胡冷非;李小勇;;基于Xen的I/O准虚拟化驱动研究[J];计算机工程;2009年23期
2 陈康;郑纬民;;云计算:系统实例与研究现状[J];软件学报;2009年05期
，本文编号：2432658
本文链接：https://www.wllwen.com/kejilunwen/jisuanjikexuelunwen/2432658.html
