面向海量存储系统的分层存储技术研究
发布时间：2018-12-21 09:35
【摘要】：随着数据信息量的快速增长，一方面迫切需要有大容量的海量存储系统来存储这些海量数据信息，但是另一方面，由于在海量存储系统中往往会出现只有小部分活跃的数据文件经常被访问，而有相当大比例的数据文件被访问的次数比较少的情况，因此必须对海量存储系统中的数据文件进行分层存储管理。 分层存储的主要技术思想是：将活跃数据文件存放在高性能且价格较昂贵的存储设备上，而将不活跃的文件存放在容量大但价格低廉的存储设备中。但是目前的分层存储技术还存在一些问题，如：典型的文件价值判定方法仅仅考虑数据本身的固有属性以及历史访问频率，很少考虑文件的访问模式，也没有考虑将文件存放在存储系统的哪个层次来加快存储系统的访问速度；另外由于文件的活跃度也经常是随着业务的变化而变化，所以也亟待提出更好的动态迁移策略来优化存储系统的整体性能。 针对以上问题，本文对海量存储系统中数据文件的分层管理技术进行了较深入的分析和研究，所做的主要研究工作和取得的研究成果如下： 1、分析和研究了海量存储系统中文件访问模式和不同存储设备性能差异，提出了一种自动分层存储系统（ATSS：Automated Tiered Storage System）的结构。 2、提出了一种基于访问模式与存储性能相匹配的文件价值判定方法（FVPSP：FileValuation based on access Pattern and Storage Performance）。该方法的思想是将访问频率较高或者随机读写比例较高的数据文件放在固态硬盘SSD上，而将访问频率较低或者顺序读写比例较高的数据文件存放在机械硬盘上。 3、针对文件活跃程度不断变化的存储系统，，提出了一种基于文件活跃度自适应调整时间间隔的动态迁移策略，该策略减少了迁移过程中迁移模块对正常业务的影响。 本文对提出的文件价值判定方法和动态迁移策略进行了性能测试。实验结果表明在顺序读写比例较高的工作负载中，FVPSP文件价值判定方法能够取得较好的优化效果；另外，自适应调整时间间隔的动态迁移策略与固定时间间隔的迁移策略相比，能够在不影响性能的条件下有效地减少文件的迁移次数。因此本文的研究结果对海量存储系统中数据文件的分层管理技术研究有一定的参考价值。
[Abstract]:With the rapid growth of data information, on the one hand, there is an urgent need for massive storage system to store these massive data information, but on the other hand, Because in mass storage systems, only a small number of active data files are frequently accessed, while a large proportion of data files are accessed less frequently. Therefore, data files in mass storage system must be managed by hierarchical storage. The main technical idea of hierarchical storage is to store active data files on high performance and expensive storage devices while inactive files are stored in large but inexpensive storage devices. However, there are still some problems in the current hierarchical storage technology, such as: the typical method for determining the value of a file only considers the inherent attributes of the data and the historical access frequency, and rarely considers the access mode of the file. It also does not consider which level of storage system to store files in order to speed up the access speed of storage system; In addition, because the activity of files often changes with the change of business, it is urgent to propose a better dynamic migration strategy to optimize the overall performance of storage system. Aiming at the above problems, this paper makes a deep analysis and research on the hierarchical management technology of data files in mass storage system. The main research work and research results are as follows: 1. This paper analyzes and studies the file access mode and the performance differences of different storage devices in mass storage system, and proposes an automatic hierarchical storage system (ATSS:Automated Tiered Storage System) structure. 2. A file value determination method (FVPSP:FileValuation based on access Pattern and Storage Performance).) based on the matching of access mode and storage performance is proposed. The idea of this method is that the data files with high access frequency or high random reading / writing ratio are placed on the solid state hard disk (SSD), while the data files with low access frequency or high proportion of sequential reading and writing are stored on the mechanical hard disk. 3. For the storage system with changing file activity, a dynamic migration strategy based on adaptive adjusting time interval of file activity is proposed, which reduces the influence of migration module on normal business during migration. This paper tests the performance of the proposed document value determination method and dynamic migration strategy. The experimental results show that the value determination method of FVPSP file can achieve better optimization effect in the workload with high proportion of sequential reading and writing. In addition, the dynamic migration strategy with adaptive time interval can effectively reduce the number of file migrations without affecting the performance compared with that of the fixed time interval migration strategy. Therefore, the results of this paper have a certain reference value for the research of hierarchical management of data files in mass storage system.
【学位授予单位】：华南理工大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333
本文编号：2388732
[Abstract]:With the rapid growth of data information, on the one hand, there is an urgent need for massive storage system to store these massive data information, but on the other hand, Because in mass storage systems, only a small number of active data files are frequently accessed, while a large proportion of data files are accessed less frequently. Therefore, data files in mass storage system must be managed by hierarchical storage. The main technical idea of hierarchical storage is to store active data files on high performance and expensive storage devices while inactive files are stored in large but inexpensive storage devices. However, there are still some problems in the current hierarchical storage technology, such as: the typical method for determining the value of a file only considers the inherent attributes of the data and the historical access frequency, and rarely considers the access mode of the file. It also does not consider which level of storage system to store files in order to speed up the access speed of storage system; In addition, because the activity of files often changes with the change of business, it is urgent to propose a better dynamic migration strategy to optimize the overall performance of storage system. Aiming at the above problems, this paper makes a deep analysis and research on the hierarchical management technology of data files in mass storage system. The main research work and research results are as follows: 1. This paper analyzes and studies the file access mode and the performance differences of different storage devices in mass storage system, and proposes an automatic hierarchical storage system (ATSS:Automated Tiered Storage System) structure. 2. A file value determination method (FVPSP:FileValuation based on access Pattern and Storage Performance).) based on the matching of access mode and storage performance is proposed. The idea of this method is that the data files with high access frequency or high random reading / writing ratio are placed on the solid state hard disk (SSD), while the data files with low access frequency or high proportion of sequential reading and writing are stored on the mechanical hard disk. 3. For the storage system with changing file activity, a dynamic migration strategy based on adaptive adjusting time interval of file activity is proposed, which reduces the influence of migration module on normal business during migration. This paper tests the performance of the proposed document value determination method and dynamic migration strategy. The experimental results show that the value determination method of FVPSP file can achieve better optimization effect in the workload with high proportion of sequential reading and writing. In addition, the dynamic migration strategy with adaptive time interval can effectively reduce the number of file migrations without affecting the performance compared with that of the fixed time interval migration strategy. Therefore, the results of this paper have a certain reference value for the research of hierarchical management of data files in mass storage system.
【学位授予单位】：华南理工大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333
【参考文献】
相关期刊论文 前1条
1 王印红;郭相坤;;信息生命周期理论在企业数据中心的应用研究[J];中国管理信息化(综合版);2007年03期
相关硕士学位论文 前1条
1 宋丽娜;海量存储系统中数据分级存储的关键技术研究[D];国防科学技术大学;2011年
本文编号：2388732
本文链接：https://www.wllwen.com/kejilunwen/jisuanjikexuelunwen/2388732.html
