网络计算环境分布式COW盘设计及优化方法

发布时间：2018-05-13 15:02

本文选题：网络计算环境 + 分布式COW盘　；参考：《湖南大学》2013年硕士论文

【摘要】：人类社会进入信息时代以来，对计算性能的要求不断提高，网络计算模式能充分的整合计算资源，满足人们对计算性能的高要求，成为当前研究的热点。网络计算模式将计算任务分配给各种分布的计算资源（各客户主机），而将数据（系统数据和用户数据）部署在相应的网络存储设备中，实现了存储与计算的分离，即存储在存储设备的系统数据不再属于某一特定的主体，而是被网络计算环境中所有主机共享。在这种模式下，网络计算环境中存储结构及存储优化变得极为关键。本文主要针对大规模网络计算环境的分布式计算和数据集中存储的特点，结合COW(copy-on-write)写时复制技术,提出了一种分布式COW网络盘的存储体系结构。在这种存储体系下,客户主机的共享数据以只读的方式集中存储在共享源盘中,各客户主机对共享数据的改写块被保存在其相应的分布式COW盘中，在实现存储与计算分离的基础上进一步优化了对共享数据的读写操作。本文的主要研究工作如下：首先，基于COW技术提出了分布式COW网络盘的存储模型,同时结合磁盘记录改写块的稀疏和突发特性，详细描述了分布式COW网络盘体系结构及读写模式，将服务器处理所有客户主机相应COW盘的聚合开销分摊到各个客户主机自身，以加速网络计算环境系统的构建速度。其次，设计了一种改进的64位位图压缩算法以有效减少COW盘位图文件大小，节省服务器磁盘空间和降低分布式COW盘网络传输开销；提出一种适合COW盘改写块突发特性的预取算法，以提高分布式COW盘Cache的命中率。最后，通过实验验证，，采用COW网络盘成功的实现了网络计算环境下多主机数据的共享访问，同时实验证明基于改进的COW盘位图压缩和预取算法实现的分布式COW盘降低了多客户主机网络计算环境的构建延迟。
[Abstract]:Since the human society has entered the information age, the demand for computing performance has been continuously improved. The network computing model can fully integrate computing resources to meet the high demand for computing performance, which has become the focus of current research. The network computing model allocates computing tasks to all kinds of distributed computing resources (each client host computer), while the data (system data and user data) are deployed in the corresponding network storage device, which realizes the separation of storage and calculation. That is, the system data stored in the storage device no longer belongs to a particular subject, but is shared by all hosts in the network computing environment. In this mode, storage structure and storage optimization in network computing environment become critical. In this paper, according to the characteristics of distributed computing and data centralized storage in large-scale network computing environment, a storage architecture of distributed COW network disk is proposed in combination with COWcopy-on-write copy technology. In this storage system, the shared data of the client host is stored in the shared source disk in a read-only manner, and the rewriting blocks of the shared data of each client host are stored in the corresponding distributed COW disk. Based on the separation of storage and computation, the read and write operation of shared data is further optimized. The main work of this paper is as follows: Firstly, based on COW technology, the storage model of distributed COW network disk is proposed. At the same time, combining the sparse and burst characteristics of disk record rewriting block, the architecture of distributed COW network disk and the mode of reading and writing are described in detail. In order to accelerate the construction speed of the network computing environment, the aggregation overhead of the server processing the corresponding COW disk of all the client hosts is distributed to each client host itself. Secondly, an improved 64-bit map compression algorithm is designed to effectively reduce the size of COW disk bitmap file, save server disk space and reduce the transmission overhead of distributed COW disk network. A prefetching algorithm suitable for burst characteristics of COW rewriting block is proposed to improve the hit rate of distributed COW disk Cache. Finally, through the experiment verification, the COW network disk has successfully realized the multi-host data sharing access under the network computing environment. At the same time, it is proved that the distributed COW disk based on the improved COW disk bitmap compression and prefetching algorithm can reduce the construction delay of multi-client host network computing environment.
【学位授予单位】：湖南大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】