高性能分布式图片服务器研究与实现

发布时间：2018-07-31 15:24

【摘要】：互联网进入web2.0时代之后,互联网内容从由网站产生转变为由用户主导产生。随着互联网用户基数不断增加,互联网计算和储存规模也日益扩大,单台的服务器早已不能够胜任高吞吐率、高并发、高可靠的要求,分布式计算应运而生。分布式系统采用分而治之的思想,将总存储和计算量分布到多台服务器上,使得整个系统能够获得更高的性能。如今,具备一定规模的公司都在研究并实施分布式方案,以求在保证高性能的前提下,节约成本。分布式的图片服务伴随着互联网的发展而出现。大型的社交网络如Facebook、腾讯,大型的电商平台如淘宝、京东,以及其他大型的在线相册服务等都对图片服务有着极大的依赖。这些网站后台储存的图片数量规模都达到了千亿级别,如何支持如此大规模的图片储存和处理成了一个亟待解决的问题。一方面,互联网图片具有小而多的特点,类似GFS的传统分布式文件系统不适合这样的海量小文件的存储;另一方面,图片应用常常需要同一图片的不同尺寸版本,如何使得图片服务能够支持提供定制尺寸的图片是一个具有现实意义的课题。所以,本课题的目的在于研究和实现一个高性能的分布式图片服务器系统-WIMG。WIMG借鉴传统分布式Master/Slave的架构,采用分布式的存储和分布式的处理,采用缓存和保存新图片的策略,能够提供高性能的图片服务,整体架构稳定可靠、可扩展。WIMG提供了简单易用的客户端接口,使得WIMG可以很方便的作为个人应用或者网站系统的图片存储后端。
[Abstract]:After the Internet entered the web2.0 era, the content of the Internet changed from website to user-oriented. With the increasing number of Internet users and the increasing scale of Internet computing and storage, a single server has been unable to meet the requirements of high throughput, high concurrency and high reliability, so distributed computing came into being. The distributed system uses the idea of divide-and-conquer, distributes the total storage and computation to multiple servers, so that the whole system can achieve higher performance. Today, companies of a certain size are researching and implementing distributed solutions to save costs while ensuring high performance. Distributed picture service appears along with the development of Internet. Large social networks such as Facebook, Tencent, large ecommerce platforms such as Taobao, JingDong, and other large online photo album services are heavily dependent on photo services. The number of images stored in the background of these websites has reached hundreds of billions of levels, so how to support the storage and processing of images on such a large scale has become an urgent problem. On the one hand, the Internet images have the characteristics of small and many, the traditional distributed file system like GFS is not suitable for the storage of such a large number of small files; on the other hand, the application of images often requires different versions of the same picture. How to enable the picture service to support the customized size of the picture is a practical issue. Therefore, the purpose of this paper is to study and implement a high performance distributed picture server system -WIMG.WIMG, which uses the traditional distributed Master/Slave architecture, uses distributed storage and distributed processing, and adopts the strategy of caching and saving new pictures. Can provide high performance picture services, the overall framework is stable and reliable, extensible. WIMG provides a simple and easy to use client interface, so that WIMG can be very convenient as a personal application or website system picture storage back end.
【学位授予单位】：上海交通大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：TP393.05

【相似文献】