基于UKSM的高效内存合并技术的研究和实现

发布时间：2018-03-22 17:27

本文选题：操作系统　切入点：内存文件系统　出处：《南京大学》2017年硕士论文　论文类型：学位论文

【摘要】：内存去冗余化一直是计算机高性能领域研究的一个热点,在服务器上特别是云计算中,各类的服务或者虚拟机对内存的需求很高,在传统的虚拟机技术中,每个虚拟机(VM)都需要有一块独立的内存,不能的虚拟机内存之间存在着相同页面;而在轻量级的容器技术中,不同容器之间,容器与主机之间会存在着不少的文件和内存数据是重复的;在内存计算中,对内存的需求尤其大。其他包括个人计算机和终端设备如手机也存在着内存紧张的问题。因此不仅有必要,也有需求提高内存利用率。因而,探究更好的去除内存冗余度的方法将有很大的应用前景。内存去冗余就是消除内存中的相似或是相同的内存区域,把存储多份冗余的内存变为只存储一份的一种方法。系统中可能会存在大量的冗余内存,当前主要有内存压缩和内存去冗余两种方法,并且都已经成功应用在Linux内核中。内存去冗余的代表作是UKSM,内存压缩的代表作是Zswap。然而,现在Linux内核中的这两种机制是由不同的团队设计并且实现的,它们去除内存冗余的速度和方式不相同,因而不能相互替代,在内存紧张时,同步Zswap会先压缩内存,会导致UKSM无法扫描到被压缩的页面。这时需要一种协同工作机制能够协调这两种方法运行。文件系统的文件缓存存在于内存中,当前没有方法能够消除文件系统中文件缓存的冗余数据。主要原因是文件缓存存在于内存中,涉及与I/O交互的块设备,设计一种方法统一消除所有文件系统中文件缓存的冗余是个很困难的任务。但是如果可以消除内存中文件缓存的冗余数据,就可以有效提高系统内存利用率。本文对上述两个问题进行研究,针对第一个问题提出了内存压缩和内存去冗协同工作机制,协调两种方法的运行,具体设计了一种同步机制,在进行内存压缩前首先进行内存去冗余扫描,确保所有相同的页面都被合并。同时考虑了不对整个系统的性能造成过大影响。针对第二个问题,结合UKSM设计了种包含文件数据切片,页面合并,页面拆分的文件系统去冗余方法。本文的主要成果如下:1.同机制,考虑了内存压缩和内存去冗余这两种方法的特性,也考虑了协同机制对系统性能的影响,并且在Linux内核中实现了这种机制。2.设计对比实验测试了协同运行机制,丰富的实验结果证明了这个机制的有效性,并且在高CPU负荷和高内存需求时都表现良好。在给系统带来快速合并内存页面的性能优势时,对CPU占用的增加并不明显。3.针对特定的内存文件系统,本文设计了文件缓存的去冗余机制。结合UKSM的扫描设计,根据虚拟文件系统的设计(VFS),对文件(inode)切片,分割成适合采样、扫描的小块。设计了中断安全的页面比较合并机制和页面拆分的写时拷贝技术。在Linux内核中实现。4.根据内存文件系统文件缓存去冗余机制,设计对比实验,在实验中对文件缓存页面的去冗余速度高达2.28MB/s。
[Abstract]:To memory redundancy is a hot research area of high performance computer, especially on the server in the cloud, all kinds of services or virtual machine memory demand is very high, in the traditional virtual machine technology, each virtual machine (VM) are required to have a separate memory, there are the same the page between the virtual machine memory cannot; and in the lightweight container technology in different containers, between the container and the host will have a lot of documents and data memory is repeated; in memory computing, the memory requirement is particularly large. Others include personal computer and terminal devices such as mobile phone also has the memory stress the problem is not only necessary. Therefore, there is also a demand to improve memory utilization. Therefore, the removal method of memory redundancy into better will have great application prospect. To eliminate redundant memory is in memory or similar The same area of memory, to store multiple copies of the redundant memory becomes a method of a stores only. There may be a large number of redundant memory systems, currently there are two methods to redundant memory compression and memory, and have been successfully applied in the Linux kernel. On behalf of the redundant memory to UKSM that representative memory compression is Zswap., however, now the two mechanisms in the Linux kernel is implemented and designed by a team of different speed and the way they remove redundant memory are not the same, so it can not replace each other in memory, tense, synchronous Zswap will be compressed memory, will cause UKSM to be unable to scan compression of the page. Then we need a collaborative work mechanism to coordinate operation of the two methods. The file system cache file exists in memory, currently there is no way to eliminate redundant files in a file system cache The main reason is the data. The file cache exists in memory, to interact with the I/O block device, design a unified method to eliminate redundant files of all the file system cache is a very difficult task. But if you can eliminate the redundant data in the file cache memory, can effectively improve the system memory utilization in this paper. Research on these two problems, put forward to solve the first problem of memory compression and memory redundancy to collaborative work mechanism, coordination of two methods of operation, the specific design of a synchronization mechanism in memory compression before the first memory to redundant scanning, ensure all the same pages are merged. Considering the performance the whole system does not cause too much influence. Aiming at the second problems, combined with the UKSM file containing the data slice, the design of the page to merge the file system page split redundancy method. The main results are as follows: 1. the same mechanism, considering the characteristics of these two kinds of methods to redundant memory compression and memory, also consider the influence of cooperative mechanism on the performance of the system, and in the Linux kernel implements the mechanism design of.2. experiment mechanism test, abundant experimental results prove the validity of this the mechanism, and CPU in high load and high memory requirements are in good performance. Bring the performance advantages of fast merge memory page, the increase of CPU is not obvious for.3. memory occupy a particular file system, this paper designed to redundancy mechanism of the file cache. Combined with UKSM scan design, based on Virtual design the file system (VFS), the file (inode) were divided into small pieces for sampling, scanning is designed. Compared with the Security page interrupt mechanism and page split copy on write technology. In the Linux kernel,.4. is implemented according to the memory redundancy mechanism of memory file system file, and a contrast experiment is designed. In experiment, the redundancy speed of file cache page is as high as 2.28MB/s..

【学位授予单位】：南京大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP333.1

【相似文献】