分级存储中数据迁移机制的实现

发布时间：2018-05-10 11:56

本文选题：数据迁移 + 分级存储　；参考：《华中科技大学》2013年硕士论文

【摘要】：分级存储的设计初衷是将数据按访问热度的不同存储于不同性能的存储设备上，最大化存储利用率。当数据的热度发生变化时，便将数据迁移至更符合其当前热度值的存储设备上。因此，对于分级存储系统而言，设计对前台应用透明、高效的数据迁移机制具有重要的研究价值。迁移过程主要分为三个模块来设计：热度统计模块、策略匹配模块以及数据迁移模块。热度统计模块在通用块层实现访问频度计数，支持基于访问频度与时间权重值的热度评价。策略匹配模块建立可定制的策略规则和通用规则，设计了基于物理卷（PV）占用率的阈值动态调整算法和基于存储节点负载均衡的PV申请算法。数据迁移模块实现三种能够断点续传的数据迁移算法：（1）将读取的源卷数据块写入目的卷对应数据块的简单数据迁移算法（SCP）；（2）基于Device-mapper中内核复制线程（kcopy）并实现数据一致性的数据迁移算法（I-kcopy）；（3）基于SCP迁移并实现读写并行的数据迁移算法（RWCP）。另外，为保障迁移后数据的一致性，设计并分析了基于记录-写策略的一致性方案，以及基于分段处理的一致性方案。具体实现了数据迁移机制的原型HSDM_COPY，并对其数据迁移速度及数据迁移过程中对用户性能的影响两个方面进行测试。实验数据表明，在高负载强度的情况下，数据迁移使磁盘吞吐率下降10%~25%，，在正常情况下，数据迁移基本不影响用户对系统的正常访问。
[Abstract]:Hierarchical storage is designed to maximize storage utilization by storing data on storage devices with different performance according to access heat. When the heat of the data changes, the data is migrated to a storage device that is more consistent with its current heat value. Therefore, for hierarchical storage system, the design has important research value for transparent and efficient data migration mechanism of foreground application. The migration process is mainly divided into three modules: heat statistics module, policy matching module and data migration module. The thermal statistics module realizes the access frequency count in the general block layer and supports the heat evaluation based on the access frequency and time weight. The policy matching module establishes customizable policy rules and general rules, and designs a threshold dynamic adjustment algorithm based on PVR) occupancy and a PV application algorithm based on storage node load balancing. The data migration module implements three data migration algorithms: 1) which can continuously transfer data from breakpoints) writes the read source volume data block into the corresponding data block of the destination volume and a simple data migration algorithm based on the kernel copy thread in Device-mapper. Based on SCP, a parallel data migration algorithm based on SCP is implemented. In addition, in order to ensure the consistency of the migrated data, the consistency scheme based on the record-write strategy and the consistency scheme based on segmentation are designed and analyzed. The prototype of data migration mechanism, HSDM COPYY, is implemented, and its data migration speed and the impact on user performance in the process of data migration are tested. The experimental data show that under the condition of high load intensity, data migration reduces disk throughput by 10% and 25%. Under normal conditions, data migration basically does not affect users' normal access to the system.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】