基于PaaS云存储服务的文件系统设计与实现

发布时间：2018-02-03 11:51

本文关键词： PaaS云存储文件系统数据块关系缓存策略费用优化　出处：《湖南大学》2013年硕士论文　论文类型：学位论文

【摘要】：随着计算机的普及和移动互联网大潮的到来，用户在不同场景不同环境中都可以享受电子设备带来的便利与无缝接入互联网的体验，但同时不得不面临管理分散于各种设备中数据碎片的问题。此外，许多创新型应用如协同办公也需要数据同步与共享作为基础。目前市面上大量出现的网盘类应用也侧面印证了此类需求普遍存在，无论个人还是企业都在寻求改善多设备数据管理的方式。本文的主要工作是探索一种更彻底的数据共享模式，即利用PaaS（Platform asa Service）云存储服务为存储后台构建文件系统。本文首先概述PaaS类云存储服务的基本概念、使用方式与计费模型，进而分析了常见文件系统数据块操作方式对于使用费用的影响。针对云存储的特点，确定通过优化数据组织方式与缓存策略减少远程访问的基本原则，以便适应跨越公网的数据访问延时和服务商收费问题。具体来说，选定三种指标用于描述文件系统中数据块间关系，文件间关系与访问热度，用于确定存储对象的内部结构，，以及基于数据块关联性的缓存置换策略。然后基于此核心，设计了一整套支撑文件系统运行的配套部件，包括增强的用户空间文件系统作为用户操作接口；围绕内嵌数据库的元信息管理；伴随下载过程的垃圾清理器；和远端服务交互前的压缩加密预处理等。然后，遵循设计实现了一个原型系统。该系统采用分层结构，面向接口编程，即每层真实执行模块都可替换。除了基于KD树实现本文设计的数据组织和缓存策略以外，还将已有较为典型的数据块管理策略一并实现作为对比。此外，在通过一些技巧屏蔽不同云服务商的数据一致性保证的前提下，引入数据块级别去冗余、快照与数据恢复功能，增强系统性能与可用性。最后，在跨越大洲的真实测评环境中，通过模拟典型应用负载和真实使用的多种测试用例下，本系统在运行花费、执行时间、垃圾清理和缓存命中率等指标中都比已有方案体现出明显的优化结果。
[Abstract]:With the popularity of computers and the advent of mobile Internet, users can enjoy the convenience and seamless Internet access experience brought by electronic devices in different scenarios and different environments. But at the same time it has to face the problem of managing data fragments scattered in various devices. Many innovative applications such as collaborative office also need data synchronization and sharing as the basis. Both individuals and businesses are looking for ways to improve data management for multiple devices. The main work of this paper is to explore a more thorough data sharing model. That is, using PaaS(Platform asa Service to build file system for storage background. This paper first summarizes the basic concept of PaaS cloud storage service, usage and billing model, and then analyzes the common file system data block operation mode on the cost of use, aiming at the characteristics of cloud storage. To determine the basic principles of reducing remote access by optimizing data organization and caching policies in order to adapt to the problem of data access latency and service provider charges across public networks. Three indicators are selected to describe the relationship between data blocks in the file system, the relationship between files and access heat, and to determine the internal structure of storage objects. And cache replacement strategy based on data block correlation. Based on this core, a set of supporting components are designed to support the file system, including enhanced user space file system as the user operation interface; Management of meta-information around embedded database; Garbage cleaners that accompany the download process; The compression encryption preprocessing before interacting with the remote service. Then, a prototype system is implemented in accordance with the design. The system adopts hierarchical structure and interface oriented programming. In addition to the data organization and cache strategy based on KD tree, the typical data block management strategy has been implemented as a comparison. On the premise of shielding the data consistency of different cloud service providers through some techniques, this paper introduces data block level de-redundancy, snapshot and data recovery function to enhance system performance and availability. In the real testing environment across continents, the system takes time to run and execute by simulating the typical application load and the real use of a variety of test cases. Garbage cleaning and cache hit ratio are better than existing schemes.
【学位授予单位】：湖南大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【共引文献】