面向移动终端的图片存取机制的研究与实现

发布时间：2018-04-09 21:23

本文选题：分布式文件系统　切入点：图片服务器　出处：《北方工业大学》2017年硕士论文

【摘要】：随着互联网技术的蓬勃发展,一些大型的社交网站如Facebook、腾讯,大型的电商平台如淘宝、京东,对于图片服务的需求也越来越大,这些大型的网站需要对千亿级别数量的图片进行存储和管理。同时移动终端深入人们的生活,移动社交App得到了广泛使用。在移动社交APP中往往也会使用大量的图片资源,如微信朋友圈、Instagram的图片分享等。如何支持如此大规模的图片存储和处理成了一个亟待解决的问题。在App中浏览图片会消耗较多的网络流量,影响加载速度,因此,根据移动端的特点,大部分App采用首先显示缩略图,根据用户需求再加载原图的策略。在图片服务器设计和实现过程中采用缓存技术来加快缩略图的生成速度,减少磁盘I/O。但是当前的缓存机制更多关注的是缓存的访问频率、最近访问时间等因素,并没有过多关注数据生成用户之间的社交关系,也没有考虑移动用户对缩略图和原图的不同访问模式。因此,本文把社交网络用户中的影响度同缓存对象紧密联系,同时把原图的缓存对象和缩略图的缓存对象相关联。论文的主要工作包括以下四个方面:第一,本文首先通过对缩略图和原图的分析,把缓存区划分为两个部分:缩略图缓存区和原图缓存区。然后分析了缓存区中的数据特性,定义了不同的缓存对象模型。第二,对于不同的缓存区定义了不同的缓存价值的公式,并对缓存价值的计算公式进行详细介绍。在定义缓存价值公式的同时,也定义了社交网络中用户影响度的计算方法、LRU价值的计算方法。结合缓存对象模型提出了基于社交关系的图片缓存替换算法。在传统缓存替换算法的基础上增加用户的社交关系以及缩略图和原图的关联关系,通过计算图片的缓存价值进行缓存替换。第三,结合项目背景,设计并实现了图片服务器。本文设计的图片服务器向客户端提供了简单易用的接口,能够很方便的应用到网站的图片存储系统中。第四,在本文的最后通过设计合理的模拟性实验加以验证,并对实验的结果进行分析和总结。实验表明,本文提出的缓存替换替换策略可以提高命中率。
[Abstract]:With the rapid development of Internet technology, some large social networking sites such as Facebook, Tencent, large e-commerce platforms such as Taobao, JingDong, the demand for photo services is also growing.These large websites need to store and manage hundreds of billions of images.At the same time, mobile terminals deeply into people's lives, mobile social App has been widely used.Mobile social APP also often uses a large number of photo resources, such as the WeChat Moments and Instagram photo sharing.How to support such a large-scale picture storage and processing has become a problem to be solved.Browsing pictures in App consumes more network traffic and affects loading speed. Therefore, according to the characteristics of mobile side, most of App adopts the strategy of displaying thumbnail first and loading original image according to user demand.In the process of design and implementation of picture server, cache technology is used to speed up the generation of thumbnail and reduce disk I / O.However, the current caching mechanism pays more attention to the frequency of cache access, recent access time and other factors, and does not pay much attention to the social relationship between data generation users.The different access modes of thumbnails and original images by mobile users are not taken into account.Therefore, the influence degree of social network users is closely related to the cache object, and the cache object of the original image and the cache object of the thumbnail are associated with each other.The main work of this paper includes the following four aspects: firstly, through the analysis of thumbnail and original image, the buffer is divided into two parts: thumbnail cache and original image cache.Then, the data characteristics in cache are analyzed, and different cache object models are defined.Secondly, different cache value formulas are defined for different cache areas, and the calculation formulas of cache value are introduced in detail.In addition to defining the cache value formula, this paper also defines the calculation method of user influence degree in social network and the calculation method of LRU value.A new image cache replacement algorithm based on social relation is proposed based on cache object model.Based on the traditional cache replacement algorithm, the social relationship of users and the correlation between thumbnail and original image are added, and the cache replacement is carried out by calculating the cache value of images.Thirdly, the picture server is designed and implemented according to the background of the project.The picture server designed in this paper provides the client with a simple and easy to use interface, which can be easily applied to the image storage system of the website.At the end of this paper, we design a reasonable simulation experiment to verify it, and analyze and summarize the results of the experiment.Experiments show that the cache replacement strategy proposed in this paper can improve the hit rate.
【学位授予单位】：北方工业大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP333

【参考文献】