一种面向分布式文件系统的文件预取模型的设计与实现
发布时间：2019-02-13 08:51
【摘要】：如何为上层应用和计算提供稳定高效的文件I/O性能,是分布式文件系统性能研究的热点。文中分析分布式文件系统在设计机理上的共同特征,基于此提出一种通用型的启发式文件预取模型,并选取HDFS平台进行系统实现。启发式文件预取对上层应用透明,采用在文件系统内部建立预取线程池的方法,以组成文件块的数据存储文件为预取单位,在分布式文件系统内部实现。这种设计思路具有一定的普适性,适合推广应用于多种分布式文件系统。实验结果表明,所述的启发式文件预取,能够有效提升分布式文件系统的I/O性能。
[Abstract]:How to provide stable and efficient file I / O performance for upper layer application and computing is a hot topic in the research of distributed file system performance. This paper analyzes the common features of the design mechanism of distributed file system, and proposes a general heuristic file prefetching model based on this, and selects HDFS platform to implement the system. The heuristic file prefetching is transparent to the upper application. The prefetching thread pool is set up in the file system. The data storage file that consists of the file block is used as the prefetching unit, and it is implemented in the distributed file system. This design idea has certain universality and is suitable for popularizing and applying to many distributed file systems. Experimental results show that the heuristic prefetching can effectively improve the I / O performance of distributed file systems.
【作者单位】： 北京航天飞行控制中心;航天飞行动力学技术重点实验室;北京航空航天大学计算机学院中德联合软件研究所;
【基金】：国家“十二五”863计划信息技术领域重大项目“云计算关键技术与系统”课题:以公众汉语服务为主的搜索引擎研制(2011AA01A205)资助
【分类号】：TP333;TP316.4
[Abstract]:How to provide stable and efficient file I / O performance for upper layer application and computing is a hot topic in the research of distributed file system performance. This paper analyzes the common features of the design mechanism of distributed file system, and proposes a general heuristic file prefetching model based on this, and selects HDFS platform to implement the system. The heuristic file prefetching is transparent to the upper application. The prefetching thread pool is set up in the file system. The data storage file that consists of the file block is used as the prefetching unit, and it is implemented in the distributed file system. This design idea has certain universality and is suitable for popularizing and applying to many distributed file systems. Experimental results show that the heuristic prefetching can effectively improve the I / O performance of distributed file systems.
【作者单位】： 北京航天飞行控制中心;航天飞行动力学技术重点实验室;北京航空航天大学计算机学院中德联合软件研究所;
【基金】：国家“十二五”863计划信息技术领域重大项目“云计算关键技术与系统”课题:以公众汉语服务为主的搜索引擎研制(2011AA01A205)资助
【分类号】：TP333;TP316.4
【相似文献】
相关期刊论文 前10条
1 刘伟;刘露;陈荦;钟志农;;海量遥感影像数据存储技术研究[J];计算机工程;2009年05期
2 何公明;张元涛;;面向数字媒体的高性能分布式存储系统的研究与应用[J];广播电视信息;2009年10期
3 许春聪;黄小猛;吴诺;孙宁伟;杨广文;;分布式文件系统存储介质评测与分析[J];计算机学报;2010年10期
4 吴英;谢广军;刘t，
本文编号：2421374
本文链接：https://www.wllwen.com/kejilunwen/jisuanjikexuelunwen/2421374.html
