数据去重过程中的垃圾回收机制研究

发布时间：2018-04-21 02:20

本文选题：垃圾回收 + 数据去重　；参考：《华中科技大学》2014年硕士论文

【摘要】：随着数字信息的爆炸式增长，备份系统中的数据随之增大，数据的存储管理越加复杂。数据去重技术的采用，优化了存储效率，却带来了数据删除过程中产生的垃圾问题。如何清理这些存储垃圾是备份系统中亟需解决的重要问题。在研究已有的网络容灾备份系统HUSTBackup系统的基础上，结合存储服务器端的数据组织和存储管理，分析了存储服务器在数据去重过程垃圾产生的原因。详细分析了三种经典垃圾回收算法，并结合系统进行了可行性分析，设计并实现了延迟引用计数法垃圾回收机制。延迟引用计数法垃圾回收的主要思想是对每一个存储的数据块设置一个引用标记，通过该引用标记的值判断其是否为数据垃圾，如果是，，则将其加入回收池中。动态更新回收池中的指纹块信息，直至垃圾回收进程被触发时才回收回收池中对应的数据块。用哈希表作回收池，提高了查询，插入和删除的速度；锁机制保证了存储服务器端的稳定性。该算法具有稳定的开销，系统负荷小，垃圾回收速度快的优点。测试数据表明，延迟引用计数法垃圾回收技术不仅减少了内存消耗，增大了存储空间，而且还优化系统的存储效率，提高系统备份性能。
[Abstract]:With the explosive growth of digital information, the data in backup system increases, and the storage and management of data becomes more and more complicated. The adoption of data removal technology optimizes storage efficiency, but it brings garbage problem in data deletion process. How to clean up these storage garbage is an important problem in backup system. Based on the research of the existing network disaster recovery and backup system (HUSTBackup), combined with the data organization and storage management of the storage server, the causes of garbage generation in the data removal process of the storage server are analyzed. In this paper, three classical garbage collection algorithms are analyzed in detail, and the feasibility of the system is analyzed, and the garbage collection mechanism of delayed reference counting method is designed and implemented. The main idea of garbage collection by delayed reference counting is to set up a reference mark for each stored data block, and to determine whether it is data garbage or not by the value of the reference tag, and if so, to add it to the collection pool. The fingerprint block information in the recovery pool is dynamically updated until the garbage collection process is triggered before the corresponding data block in the recovery pool is recovered. The hash table is used as the recovery pool to improve the speed of query, insert and delete, and the lock mechanism ensures the stability of the storage server. The algorithm has the advantages of stable overhead, low system load and fast garbage collection speed. The test data show that the garbage collection technology not only reduces the memory consumption and increases the storage space, but also optimizes the storage efficiency of the system and improves the system backup performance.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP333

【相似文献】