面向个人云存储的缓存机制研究

发布时间：2018-03-24 18:26

本文选题：云存储　切入点：个人云存储　出处：《浙江大学》2015年硕士论文

【摘要】：云存储服务作为云计算的重要内容,是解决海量用户访问海量数据的关键技术。个人云存储作为云存储的典型应用之一,不仅为用户提供了基本的数据存储功能,还提供了文件多端同步、工作协同、多应用汇聚等功能,已经成为一种较为流行的数据服务,但仍面临着用户更高服务质量要求的挑战。论文针对云存储后端分布式文件系统的缓存机制开展研究,为海量用户在海量存储的背景下提供更高效的个人云存储服务。通过分析真实的个人用户访问数据集,综合考虑个人云存储的应用场景,论文发现传统缓存机制中存在低读写比场景下缓存存储空间的利用率问题,以及海量缓存文件场景下缓存替换的执行效率问题,这些问题关系到个人云存储服务的成本与服务质量。针对低读写比场景下缓存存储空间的利用率问题,论文提出了基于动态决策的缓存文件选择策略。该策略通过对频繁访问文件的大小和类型进行在线分析,动态判断请求文件是否应当放入缓存,避免将“只一次访问”的文件放入缓存,并通过文件访问次数和文件访问时间间隔对不满足文件大小、类型的频繁文件进行弥补,降低项误判的可能性。实验证明,基于动态决策的缓存文件选择策略可以显著提高缓存空间的利用率,在维持缓存容量的低成本条件下,提升缓存命中率,且决策时间较低。针对海量缓存文件场景下缓存替换的执行效率问题,论文提出了基于分代技术的缓存文件更新策略。该策略被设计实现为一种功能框架,可应用不同的缓存替换算法,在保持原有缓存替换策略特性的同时,为缓存对象增加生命周期管理功能,并通过更新缓存对象的生命周期,缩小缓存替换执行的范围。实验证明,基于分代技术的缓存文件更新策略可以有效提高缓存替换效率,解决“缓存污染”问题,且灵活可配,适合用户使用场景多样化的个人云存储系统。最后,论文在为开源分布式文件系统HDFS实现缓存模块的基础上,将以上两种缓存策略应用到HDFS中,从而优化个人云存储系统使用HDFS作为后端存储时的系统性能,并在实际运行中取得了较好的效果。
[Abstract]:As an important part of cloud computing, cloud storage service is the key technology to solve the problem of mass users accessing mass data. As one of the typical applications of cloud storage, personal cloud storage not only provides users with basic data storage functions, It also provides functions such as file multi-terminal synchronization, work cooperation, multi-application convergence, etc., and has become a more popular data service. However, it is still facing the challenge of higher quality of service (QoS) of users. This paper focuses on the cache mechanism of cloud storage back-end distributed file system. To provide a more efficient personal cloud storage service for mass users in the context of mass storage. By analyzing the real personal user access data set, considering the application scenarios of personal cloud storage, In this paper, we find that there are some problems in the traditional cache mechanism, such as the utilization of cache storage space in the low read-write ratio scenario, and the efficiency of cache replacement in the massive cache file scenario. These problems relate to the cost and quality of service of personal cloud storage. This paper proposes a cache file selection strategy based on dynamic decision, which dynamically determines whether the request file should be put into the cache by analyzing the size and type of frequently accessed files online. Avoid "only once access" files into the cache, and through the number of file visits and file access time interval to not meet the file size, type of frequent files to make up for, reduce the possibility of misjudgment of items. The cache file selection strategy based on dynamic decision can significantly improve the utilization of cache space and improve the cache hit ratio under the condition of maintaining cache capacity at low cost. And the decision time is low. Aiming at the efficiency of cache replacement in the massive cache file scenario, this paper proposes a cache file update strategy based on generational technology, which is designed and implemented as a functional framework. Different cache replacement algorithms can be applied to maintain the characteristics of the original cache replacement strategy, while adding life cycle management function to the cache object, and by updating the life cycle of the cache object, Experiments show that the cache file update strategy based on generational technology can effectively improve the efficiency of cache replacement, solve the problem of "cache pollution", and can be configured flexibly. Personal cloud storage system suitable for users to use a variety of scenarios. Finally, based on the implementation of cache module for open source distributed file system (HDFS), the above two caching strategies are applied to HDFS. Therefore, the performance of personal cloud storage system with HDFS as back-end storage is optimized, and good results are obtained in actual operation.
【学位授予单位】：浙江大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：TP333

【参考文献】