闪存数据库系统存储管理及可裁剪性研究

发布时间：2018-06-27 18:30

本文选题：闪存 + 自适应存储管理　；参考：《中国科学技术大学》2012年博士论文

【摘要】：闪存是20世纪80年代由因特尔公司和东芝公司发明的一种新型存储介质,相比传统的磁盘,它在I/O性能,抗震性,功耗,体积等诸多方面都具有明显的优势。所以近年来,闪存有逐渐替代传统磁盘的趋势,开始在各种移动终端,手持设备甚至大型数据中心得到广泛的应用。另一方面,传统的数据库系统性能出现了瓶颈,这主要体现在作为其存储介质的磁盘发展速度已经远远落后于CPU,内存等其他硬件的发展。由于闪存的出现,一种直观上有效的解决办法即使用闪存作为数据库系统的二级存储介质。但是有研究表明,直接将现有的数据库系统迁移至闪存,其带来的性能加速比远低于闪存相对磁盘的性能加速比。这是因为现有的数据库系统都是针对磁盘进行设计,而闪存与磁盘的物理特性有很大的不同,导致数据库系统中针对磁盘设计的一系列算法在闪存上表现并不令人满意,所以要充分利用闪存的性能,我们必须根据其特性重新设计数据库系统中的各核心算法。本论文主要针对数据库系统的存储管理,缓冲区管理以及事务恢复等方面进行了深入研究,同时由于闪存应用场景的广泛性,不同的应用场景对于数据库功能的需求也各不相同,所以本论文在如何实现数据库系统功能模块快速裁剪和定制方面也进行了一些试探性的研究。论文首先介绍了闪存的发展历史,然后针对两种不同的闪存类型：NOR型闪存和NAND型闪存,分析对比了它们在性能,应用场合等各方面的差异。存储管理是组成数据库系统性能的重要模块,本论文在现有研究的基础上提出了一种适应随机小粒度更新的闪存存储管理模式,该模式通过检测闪存块中数据页的更新模式,动态地分配日志区大小。能够有效地减少闪存的写入和擦除次数,延长整个存储系统的使用寿命。缓冲区是影响数据库系统性能的重要因素。由于闪存独特的物理特性,命中率一般不能再作为衡量缓冲区算法好坏的标准。在此观察上,论文提出了一种针对日志型闪存存储管理模式的缓冲区算法,它通过检测缓冲页面的冷热,延时写出热页从而有效地减少了闪存的写入和擦除次数,同时该算法中实现了两种日志打包算法,可以明显提高闪存日志区的空间利用率,进一步减少块擦除次数,提高了闪存的使用寿命。论文还为日志型闪存存储模型提供了一种简单快速的事务支持和恢复算法,该算法通过将提交日志和未提交日志分开存储,可以避免闪存块合并操作导致某些事务无法回滚的问题,同时由于不需要写全局事务状态日志,可以大量减少闪存写次数同时提高闪存空间利用率。最后在闪存数据库系统的快速定制方面,论文利用面向方面的编程模型,提出了一套基于横切代码位置的抽取算法,并给出了代码从OOP向AOP迁移的技术路线,实验证明该算法可以有效的实现横切代码分离,从而实现数据库系统的定制。本论文的主要贡献在以下几个方面： (1)提出了一种适应小粒度随机更新的闪存存储管理模型,通过检测闪存块的更新模式,自适应地调整日志区大小,有效地减少了闪存写入和擦除次数。 (2)提出了一种针对日志型闪存存储模型的缓冲区算法,该算法通过对缓冲区的页面进行冷热分类,延时置换热页,有效地减少了闪存写入次数,同时结合两种日志打包算法,提高了闪存日志区的空间利用率,有效地缓和了日志区的碎片问题,提高了闪存的使用寿命。 (3)为(1)中的存储模型提供了一种用于支持快速事务恢复的算法,通过将提交日志和未提交日志进行分离,可以有效解决块合并操作导致的某些事务不可恢复的问题。同时不再需要全局事务状态日志,节省了大量的闪存写操作。 (4)利用面向方面的编程,定义了基于横切代码位置的抽取算法,给出了用于实现数据库系统可裁剪和定制的技术路线,并通过实验证明了该算法的可行性。
[Abstract]:Flash memory is a new type of storage medium invented by Intel Corp and Toshiba Co in 1980s. Compared with traditional disk, it has obvious advantages in I/O performance, seismic resistance, power consumption, volume and so on. So in recent years, flash memory has gradually replaced traditional disk, and began to be in various mobile terminals, handheld devices even. Large data centers are widely used.
On the other hand, the performance of the traditional database system has a bottleneck, which is mainly reflected in the development speed of the disk as its storage medium is far behind the development of other hardware such as CPU, memory and other hardware. Because of the flash memory, a visual and effective solution is to use flash memory as the two level storage medium of the database system. Studies have shown that the performance acceleration of the existing database system directly to flash memory is far lower than the performance acceleration ratio far below the flash relative disk. This is because the existing database systems are designed for the disk, and the physical characteristics of the flash memory and the disk are very different, causing the disk design in the database system. A series of algorithms are not satisfactory in the flash memory, so in order to make full use of the performance of the flash memory, we must redesign the core algorithms in the database system according to their characteristics.
This thesis focuses on the storage management of the database system, the buffer management and the transaction recovery. At the same time, because of the wide application scene of the flash memory, the requirements of the different application scenes are different to the database function. Some tentative studies have also been carried out in the area of customization.
This paper first introduces the development history of flash memory, and then compares two different flash memory types: NOR flash memory and NAND flash memory, and compares their differences in performance, application and other aspects.
Storage management is an important module to make up the performance of the database system. On the basis of the existing research, this paper presents a flash memory management model which adapts to the random particle size update. By detecting the update mode of the data page in the flash memory block, this model dynamically distributs the log area. It can effectively reduce the writing and erasure of the flash memory. The number of times, prolong the life of the whole storage system.
The buffer area is an important factor affecting the performance of the database system. Due to the unique physical characteristics of the flash memory, the hit rate can not be used as a standard to measure the buffer algorithm. In this observation, a buffer algorithm for the log storage management mode is proposed in this paper. By detecting the cold heat of the buffer page, the time delay is written. It can effectively reduce the number of writing and erasure of flash memory. At the same time, two kinds of log packing algorithms are implemented in the algorithm, which can obviously improve the space utilization rate of the flash log area, reduce the number of erasure and improve the service life of the flash memory.
The paper also provides a simple and fast transaction support and recovery algorithm for the log flash memory storage model. This algorithm can avoid the problem that some transactions can not be rolled back by the merge operation of the flash memory block by separating the submission log from the uncommitted log, and it can be greatly reduced because the global transaction status log is not required. The number of flash writes improves the utilization of flash space simultaneously.
Finally, in the fast customization of flash memory database system, the thesis uses aspect oriented programming model to propose a set of extraction algorithm based on cross cut code location, and gives the technical route from OOP to AOP. The experiment proves that the algorithm can effectively separate the crosscutting code, thus realizing the customization of the database system.
The main contributions of this paper are in the following aspects:
(1) a flash memory management model which adapts to the random particle size updates is proposed. By detecting the update mode of the flash memory block, the log area size is adjusted adaptively, and the number of flash memory writes and erasure can be reduced effectively.
(2) a buffer algorithm for log flash memory storage model is proposed. This algorithm can effectively reduce the number of flash memory by using cold and heat classification of the page of the buffer zone, effectively reducing the number of flash writing, and combining two log packing algorithms to improve the spatial utilization of the flash log area and effectively mitigate the fragmentation of the log area. The problem is to improve the life of the flash memory.
(3) the storage model in (1) provides an algorithm to support fast transaction recovery. By separating the submission log from the uncommitted log, it can effectively solve the problem of the unrecoverable transaction caused by the block merge operation. At the same time, the global transaction log is no longer needed, and a large number of flash writing operations are saved.
(4) using aspect oriented programming, a decimation algorithm based on the crosscutting code location is defined, and a technical route for the database system to be tailored and customized is given, and the feasibility of the algorithm is proved by experiments.
【学位授予单位】：中国科学技术大学
【学位级别】：博士
【学位授予年份】：2012
【分类号】：TP333

【相似文献】