基于Hadoop的分布式对称加密数据存储的分析与研究

发布时间：2018-07-27 11:00

【摘要】：随着互联网技术和分布式数据存储的高速发展,企业对云平台使用越来越普及。与此同时,人们对于自身存储在云平台的数据安全的关心程度也前所未有,对云平台上存储的数据安全有了更高的要求。Hadoop能够让用户轻松架构和使用分布式平台,然后通过该平台对大量数据进行分布式处理。但是基于Hadoop的分布式数据存储目前存在着很大的安全隐患,缺少对静态数据的加密保障,在安全方面只关注了数据的验证和备份等部分,并没有注重数据本身的安全性,因此频频爆发安全问题。本文研究的主要内容就是将加密技术应用到Hadoop云平台中,能够实现对存储在Hadoop云平台的数据进行分布式的对称加密和解密。首先,本论文在介绍了云计算和分布式数据存储的国内外研究现状和发展趋势后,对Hadoop开源平台的两大关键技术:分布式文件系统HDFS(Hadoop Distributed Filesystem)和用于处理海量数据的编程模型MapReduce做了详细的介绍。其次,通过对现有加密算法的对比,分析传统分布式数据存储和Hadoop分布式数据存储存在的问题,本文结合Hadoop分布式云计算框架中的的MapReduce模型,总结得出了基于Hadoop的分布式对称加密数据存储技术,并进行了简要的分析。然后,从整体上对数据的分布式加密和解密各个模块进行了设计,并详细介绍了用MapReduce实现对称加密AES(Advanced Encryption Standard)算法过程,特别是着重介绍和设计了能够实现该算法的Map函数和Reduce函数。最后,在实验室内搭建Hadoop集群,并且通过该云平台,来实现加密和解密过程,并对分布式对称加密算法进行功能测试、扩展性测试和效率测试,验证整个流程。通过比较测试数据给出仿真性能分析,提出优化问题,并对工作进行了总结和展望。
[Abstract]:With the rapid development of Internet technology and distributed data storage, the use of cloud platform is becoming more and more popular. At the same time, people pay more attention to the security of data stored on cloud platform than ever before. Hadoop can make it easy for users to construct and use distributed platform. Then, a large number of data are distributed through the platform. However, the distributed data storage based on Hadoop has great security hidden danger at present, it lacks the encryption guarantee of static data, pays attention only to the verification and backup of data in the security aspect, and does not pay attention to the security of the data itself. As a result, security issues frequently erupt. The main content of this paper is to apply encryption technology to Hadoop cloud platform, which can realize distributed symmetric encryption and decryption of data stored in Hadoop cloud platform. First of all, this paper introduces the research status and development trend of cloud computing and distributed data storage at home and abroad. This paper introduces two key technologies of Hadoop open source platform: distributed file system (HDFS (Hadoop Distributed Filesystem) and programming model (MapReduce) which is used to deal with massive data. Secondly, by comparing the existing encryption algorithms, this paper analyzes the problems existing in traditional distributed data storage and Hadoop distributed data storage, and combines the MapReduce model of Hadoop distributed cloud computing framework. The distributed symmetric encryption data storage technology based on Hadoop is summarized and analyzed briefly. Then, the distributed encryption and decryption modules of data are designed, and the process of using MapReduce to realize symmetric encryption AES (Advanced Encryption Standard) algorithm is introduced in detail. Especially, the Map function and Reduce function which can realize the algorithm are introduced and designed. Finally, the Hadoop cluster is built in the laboratory, and the encryption and decryption process is realized through the cloud platform. The distributed symmetric encryption algorithm is tested by function, extensibility test and efficiency test, and the whole process is verified. The simulation performance analysis is given by comparing the test data, the optimization problem is put forward, and the work is summarized and prospected.
【学位授予单位】：西安科技大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：TP309.7;TP333

【相似文献】