网络硬盘存储优化方法的设计与实现

发布时间：2018-03-28 04:23

本文选题：同步更新网络硬盘　切入点：海量数据　出处：《电子科技大学》2012年硕士论文

【摘要】：网络硬盘是网络公司向用户提供的一种在线存储服务，提供文件的存储、访问、备份、共享等功能。同步更新网络硬盘通过相应的文件备份技术，能够实现文件同步更新功能因而被广泛应用。但是，当用户访问和操作的文件较大时，由于网络带宽的限制以及网络传输速度与文件更新速度的矛盾，造成了网络硬盘同步更新的性能瓶颈，影响了同步更新网络硬盘的推广和使用。针对这一问题，本文通过采用一种适用于满足大文件同步更新的网络硬盘的海量数据高速缓存方法，旨在缓解Internet的传输速度与文件更新数据生成速度之间的矛盾。具体的讲，本文的主要工作有： 1、设计了一种内存磁盘二级缓存方法，通过采用此方法，，既保证了用户同步更新数据的有效缓存，又保证了海量数据高速缓存有较高的响应速度，并对用户自身服务的影响较小，提高了同步更新网络硬盘的整体性能。 2、采用了一种多任务共享缓存和冗余备份记录合并策略，在满足数据同步更新需求的基础上，节约了缓存空间并提高了存储空间的利用率，降低了单位数据同步更新所产生的网络流量，降低了同步更新网络硬盘对网络的压力，同时也降低了同步更新网盘远程存储中心的写重放压力。 3、通过采用一套数据一致性控制方法，保证了缓存过程中同步更新数据的正确性，同时也提高了同步更新网盘系统的可靠性。 4、设计了一种适用于海量数据高速缓存的内存管理方法，向海量数据高速缓存提供大容量内存资源的分配与回收功能，通过提高海量数据高速缓存在内核空间申请内存资源的成功率，保证了海量数据高速缓存的性能。 5、采用了一种基于proc文件系统的海量数据高速缓存配置管理方法，实现了同步更新网盘系统及网盘用户对海量数据高速缓存的实时控制，提高了海量数据高速缓存的可控性和易用性。综上所述，本文设计并实现了一种适用于同步更新网盘系统的海量数据高速缓存方法。经实验分析，该方法可以满足同步更新网盘系统的需求，解决了同步更新时数据生成速度与网络传输速度之间的矛盾，具有良好的经济和社会效益。
[Abstract]:Network hard disk is a kind of online storage service provided by network company to users. It provides file storage, access, backup, sharing and other functions. Because of the limitation of network bandwidth and the contradiction between the speed of network transmission and the speed of file updating, the function of synchronizing file updating is widely used, however, when the file is accessed and operated by the user, the network bandwidth is limited and the network transmission speed is in contradiction with the file update speed. It causes the bottleneck of network hard disk synchronous update and affects the promotion and use of synchronous updating network hard disk. Aiming at this problem, this paper adopts a method of mass data cache suitable for the network hard disk which can satisfy the large file synchronous update, aiming at alleviating the contradiction between the transmission speed of Internet and the speed of generating file update data. The main work of this paper is as follows:. 1. A kind of memory disk two-level cache method is designed. By adopting this method, the user can update the data efficiently and the mass data cache has a high response speed. It has little influence on user's own service, and improves the overall performance of synchronous updating network hard disk. 2. A multi-task shared cache and redundant backup record merging strategy is adopted, which saves cache space and improves the utilization of storage space on the basis of satisfying the requirement of data synchronization update. It reduces the network traffic caused by synchronous updating of unit data, reduces the pressure of synchronous updating network hard disk on the network, and also reduces the pressure of writing and replaying in the remote storage center of synchronous updating network disk. 3. By adopting a set of data consistency control method, the correctness of synchronous updating data is ensured, and the reliability of synchronous updating network disk system is also improved. 4. A memory management method suitable for mass data cache is designed, which can provide large capacity memory resource allocation and recovery function to mass data cache. The performance of mass data cache is guaranteed by improving the success rate of applying for memory resources in kernel space. 5. A configuration management method of mass data cache based on proc file system is adopted to realize the synchronous updating of network disk system and real-time control of mass data cache by users of network disk. It improves the controllability and ease of use of mass data cache. To sum up, this paper designs and implements a massive data cache method suitable for synchronous update disk system. The experimental analysis shows that this method can meet the needs of synchronous updating disk system. It solves the contradiction between the speed of data generation and the speed of network transmission, and has good economic and social benefits.
【学位授予单位】：电子科技大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：TP333.35

【参考文献】