基于Hadoop的智能调度云数据中心关键技术研究

发布时间：2018-03-21 09:35

本文选题：智能电网调度　切入点：云计算　出处：《华北电力大学》2013年硕士论文　论文类型：学位论文

【摘要】：电网调度系统是整个电力系统的中枢。我国电网互联程度的不断提高以及智能电网建设过程的逐步推进，给现有的电网调度系统带来了极大的挑战。其主要挑战就来自于现有的电力计算平台，其计算与存储能力已经无法满足电网发展的需求，，考虑到电网发展的计算平台计算及存储能力不断增长的趋势，建设具有海量数据计算和存储的智能电网调度系统已迫在眉睫。首先，本文在结合我国现有电网调度体系的基础上，提出了基于云计算的智能电网调度系统新架构以及云计算数据中心的体系结构，从宏观上对整个智能电网调度体系结构进行了规划。其次，本文针对智能电网调度云数据中心的三个关键问题进行了重点研究。第一、云数据中心内部网络结构。在分析对比传统网络结构算法以及最新提出的BCube、DCell数据中心网络结构算法的前提下，提出了更具有扩展性、容错性的PCube网络结构算法；第二、Hadoop云平台的副本存储策略。针对Hadoop现有副本存放策略的不足，本文研究目前学者提出HDFS改进存储策略——CoHadoop副本存储策略，并在此基础上对其智能性加以改进，依靠FP数组挖掘算法对Locator的相关性进行定义，完善了CoHadoop算法的性能，使其更好的适合智能电网调度系统中事务数据集大而稀疏的特点；第三、Hadoop云平台的任务调度策略。本文改进了现有Hadoop平台的公平调度算法，基于模拟退火策略对任务选取进行优化以适合智能电网调度系统任务多、优先级区分明显等特点。最后，为验证相关算法的有效性，随着研究过程的进行，本文分别对网络结构算法进行模拟实验，验证了其扩展性、容错性等关键特性；之后为验证Hadoop副本存储策略和任务调度算法的性能，本文借助一个已有云计算平台搭建了Hadoop集群，对相关实验环境进行了模拟并运行了相关算法实例，验证了算法的有效性。
[Abstract]:The power grid dispatching system is the center of the whole power system. The main challenge comes from the existing electric power computing platform, whose computing and storage capacity can no longer meet the needs of the development of the power grid. Considering the increasing computing and storage capacity of power grid, it is urgent to build a smart grid dispatching system with massive data computing and storage. First of all, this paper proposes a new architecture of smart grid dispatching system based on cloud computing and the architecture of cloud computing data center based on the existing power grid dispatching system in China. The whole dispatching architecture of smart grid is planned macroscopically. Secondly, this paper focuses on three key issues of smart grid dispatching cloud data center. On the premise of analyzing the network structure of cloud data center, a more extensible and fault-tolerant PCube network structure algorithm is proposed, which is more scalable and more fault-tolerant than the traditional network structure algorithm and the latest BCube DCell data center network structure algorithm. Second, the replica storage strategy of Hadoop cloud platform. Aiming at the deficiency of the existing replica storage strategy of Hadoop, this paper proposes an improved storage strategy for HDFS, which is CoHadoop replica storage strategy, and then improves its intelligence. The correlation of Locator is defined by FP array mining algorithm, which improves the performance of CoHadoop algorithm and makes it more suitable for the characteristics of large and sparse transaction data set in smart grid dispatching system. The third is the task scheduling strategy of Hadoop cloud platform. This paper improves the fair scheduling algorithm of the existing Hadoop platform and optimizes the task selection based on simulated annealing strategy to suit for the smart grid scheduling system with many tasks and distinct priorities. Finally, in order to verify the effectiveness of the algorithm, with the progress of the research process, the network structure algorithm is simulated to verify its scalability, fault tolerance and other key characteristics. Then in order to verify the performance of Hadoop replica storage strategy and task scheduling algorithm this paper builds a Hadoop cluster with the help of an existing cloud computing platform and simulates the related experimental environment and runs an example of relevant algorithms to verify the effectiveness of the algorithm.
【学位授予单位】：华北电力大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP308;TP311.13

【参考文献】