基于HDFS和加密检索的云存储系统设计与实现

发布时间：2018-12-13 19:58

【摘要】：为了将硬件和软件等计算机资源也能像水和电一样,只需要支付少许“资金”就可以源源不断地使用,云计算适时而生。云计算改变了人们对软硬件的传统使用模式,规避了应用程序安装和硬件设施维护等繁琐过程,做到了真正意义上的计算机资源服务直接使用。它满足了大数据时代下,对数据计算和存储的高要求。云存储由云计算发展而来,其核心点是对数据的存储。相比传统存储,云存储拥有非常好的容错性及可扩展性,能够解决数据的海量存储,只要有网络,用户访问云中的数据可不分时间和地点。然而,由于云存储中对于分布在不同地理位置的云服务器具有分布式的特点,导致数据存储位置不确定性,会给用户云端数据安全带来存储隐患。另外,HDFS因大数据存储而生,对大量小文件的存储不可避免会造成空间上的浪费。针对上述存在问题,本文主要做了如下工作:(1)本文提出一种加密检索方法,对用户敏感数据进行加密存储,利用MD5加密文件名生成的信息摘要构成索引,提供信息摘要检索密文文件,明文数据利用改进的DES算法进行本地加解密。解决了用户敏感数据安全存储隐忧,密文文件检索困难等问题。(2)本文利用HDFS作为数据存储端,以浏览器/服务器模式实现云存储系统。系统提供了用户注册、账户管理、空间和文件的管理、文件上传与下载、文件搜索、数据备份等系列功能。云存储用户本身需要存储许多小文件,为了不影响用户的使用体验,又能够尽量降低存储空间上的浪费,增加小文件合并上传模式,在小文件问题上做一个折中,用户可以选择性的将某些不常用的小文件进行合并上传。本文设计和实现了基于HDFS和加密检索的云存储系统。能有效满足用户对数据存储的基本需求,结合加密检索方法,实现用户敏感数据的安全存储和有效检索。并且用户可以选择性地对小文件合并上传,提高存储空间利用率。
[Abstract]:In order to make computer resources such as hardware and software as well as water and electricity, only a small amount of "money" can be paid to continue to use, cloud computing is timely. Cloud computing has changed the traditional usage mode of software and hardware, avoided the tedious process of application installation and hardware facilities maintenance, and achieved the direct use of computer resource services in the real sense. It meets the big data era, the high demand for data computing and storage. Cloud storage comes from cloud computing, and its core point is the storage of data. Compared with traditional storage, cloud storage has a very good fault tolerance and scalability, can solve the massive storage of data, as long as there is a network, users can access the data in the cloud regardless of time and place. However, due to the distributed characteristics of cloud servers distributed in different geographical locations in cloud storage, the uncertainty of data storage location leads to storage hidden trouble for users' cloud data security. In addition, HDFS due to big data storage, the storage of a large number of small files will inevitably cause a waste of space. In view of the above problems, the main work of this paper is as follows: (1) this paper proposes an encryption retrieval method, which encrypts and stores sensitive user data, and uses MD5 to encrypt the information digest generated by the filename to form an index. Provides the information summary retrieval ciphertext file, plaintext data using the improved DES algorithm for local encryption and decryption. It solves the problems of security storage of user sensitive data and difficulty of searching ciphertext files. (2) this paper uses HDFS as data storage terminal and implements cloud storage system in browser / server mode. The system provides user registration, account management, space and file management, file upload and download, file search, data backup and other functions. Cloud storage users themselves need to store many small files, in order not to affect the user's use experience, but also to minimize the waste of storage space, increase the small file merge upload mode, make a compromise on the small file problem, Users can optionally merge and upload small, infrequent files. This paper designs and implements a cloud storage system based on HDFS and encryption retrieval. It can effectively meet the basic needs of users for data storage, combined with encryption retrieval method, to achieve the safe storage and effective retrieval of user sensitive data. And users can selectively merge and upload small files to improve storage space utilization.
【学位授予单位】：东华理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP333;TP309

【参考文献】