云计算环境下分布存储与并行计算的协同性研究

发布时间：2018-07-23 20:35

【摘要】：云计算是搭建在同构廉价普通节点上的分布式计算环境，在节点不可信的特点下，分布式存储成为了云计算的必然选择，存储与计算的整合趋势也日益明显。本文搭建了一个简单的对等架构的云计算环境，并从分布式存储与并行计算两个方面对二者可能的协同进行了研究，提出了一套分布式存储策略和面向数据驱动的调度策略。本文的云计算环境的物理层搭建在机柜式环境上，网络拓扑层采用对等式架构，整个环境基于Spring框架和各Java嵌入式组件搭建，并采用了多种流行通信框架，可以实现环境的快速搭建、高可扩展和快速部署。本文将传统的MPI环境与分布式存储进行了整合，探究了一种云计算环境下不采用MapReduce编程模型实现并行计算的方法。在协同性的分布式存储方面，本文提出了一套云计算环境下适用于地震资料的分布式存储策略，该策略在充分研究地震资料及其处理特点的基础上，对地震资料的数据划分，副本的部署和定位进行了研究。在协同性的并行计算方面，本文除了对分布式存储和MPI环境进行了整合，还设计了一种面向数据驱动的对等式调度策略，，该策略充分利用了分布式存储的优势，不仅摊销了分布式存储的成本，而且节约了数据准备的时间。
[Abstract]:Cloud computing is a distributed computing environment built on the isomorphic low cost ordinary nodes. With the characteristics of the nodes being unreliable, distributed storage has become the inevitable choice of cloud computing, and the integration of storage and computing is becoming more and more obvious. In this paper, a simple peer-to-peer cloud computing environment is built, and the possible collaboration between distributed storage and parallel computing is studied. A set of distributed storage strategy and data-driven scheduling strategy are proposed. In this paper, the physical layer of cloud computing environment is built in the cabinet environment, the network topology layer adopts the peer-to-peer architecture, the whole environment is based on the Spring framework and each Java embedded component, and adopts a variety of popular communication frameworks. Environment can be quickly built, high scalability and rapid deployment. In this paper, the traditional MPI environment is integrated with distributed storage, and a method of parallel computing without using MapReduce programming model in cloud computing environment is explored. In the aspect of collaborative distributed storage, this paper proposes a distributed storage strategy for seismic data in cloud computing environment. Based on the full study of the characteristics of seismic data and its processing, this strategy divides the seismic data. The deployment and location of copies were studied. In the aspect of collaborative parallel computing, this paper not only integrates distributed storage and MPI environment, but also designs a data-driven peer-to-peer scheduling strategy, which makes full use of the advantages of distributed storage. It not only amortizes the cost of distributed storage, but also saves the time of data preparation.
【学位授予单位】：中国石油大学（华东）
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】