智能电网大数据在线分析与决策系统研究

发布时间：2018-11-03 09:05

【摘要】：随着全球能源互联网的建设以及智能电网建设的快速推进,大量的物联网信息采集设备终端将会接入电网,这些终端将会产生海量的采集数据——智能电网大数据。为应对这些海量数据分析的需求,本文研究了智能电网大数据的流处理和批处理引擎的构建,并在此基础上完成了智能电网大数据在线分析与决策系统的设计。本文在国内外的研究的基础上研究了智能电网大数据的来源和分类,对智能电网大数据分析的主要需求进行了分析。本文对大数据相关的分布式计算理论进行了介绍,其中主要介绍了分布式计算框架MapReduce、分布式文件系统GFS和HDFS、分布式应用程序协调服务Chubby和ZooKeeper、分布式资源管理框架YARN和Mesos的原理和架构,同时也介绍了Map Reduce迭代计算模型、BSP计算模型、SSP计算模型这三种分布式数据分算法基础模型;随后研究了智能电网大数据流处理的任务需求,对流处理的概念进行了介绍,同时研究了智能电网大数据流处理系统的需求特征,着重研究了Strom、Spark Streaming、Samza这三种流处理引擎以及其应用场景,根据其特点和智能电网大数据流处理分析的特性需求本文选择Strom作为构建智能大数据在线分析与决策系统的流处理引擎,并通过基于Storm的VFDT算法在重要电力客户供用电安全实时分析中的应用展示了Strom在电网数据实时分析上的有效性,通过机器的扩展和模拟数据流的增加进行压力测试证明了Strom流处理引擎的在智能电网大数据分析场景中的可扩展性;随后,对智能电网大数据批处理的任务需求进行了研究,提出了利用Spark建设智能电网大数据批处理分析引擎的方案,并通过基于Spark的随机森林算法在海量用电负荷数据分析中的应用验证了该解决方案的有效性、可扩展性;最后,在以上研究的基础上对智能电网大数据在线分析与决策系统进行了详细的需求分析,并设计了系统的整体架构和每个模块的功能,这一设计方案可为后续的软件开发提供直接的参考依据。
[Abstract]:With the construction of the global energy Internet and the rapid development of the smart grid, a large number of Internet of things information acquisition equipment terminals will be connected to the grid, these terminals will produce a huge amount of data acquisition-big data smart grid. In order to meet the demand of these massive data analysis, this paper studies the construction of big data's flow processing and batch processing engine of smart grid, and on this basis completes the design of online analysis and decision making system for big data in smart grid. On the basis of domestic and international research, this paper studies the source and classification of big data in smart grid, and analyzes the main requirements of the analysis of the intelligent power grid big data. This paper introduces the distributed computing theory related to big data, including the distributed computing framework MapReduce, distributed file system GFS and HDFS, distributed application coordination service Chubby and ZooKeeper,. The principle and architecture of distributed resource management framework (YARN and Mesos) are introduced, and three basic models of distributed data division algorithm, namely, Map Reduce iterative computing model, BSP computing model and SSP computing model, are also introduced. Then, the task requirements of large data flow processing in smart grid and the concept of convection processing are introduced. At the same time, the demand characteristics of large data flow processing system in smart grid are studied, and the Strom,Spark Streaming, is emphatically studied. According to the characteristics of Samza and its application scenarios, this paper chooses Strom as the flow processing engine to construct the intelligent big data online analysis and decision system according to its characteristics and the characteristics of large data flow processing and analysis of smart grid. The application of VFDT algorithm based on Storm in real-time analysis of power supply and power security of important power customers shows the effectiveness of Strom in real-time analysis of power network data. The expansibility of Strom flow processing engine in the analysis scene of smart grid big data is proved by the expansion of the machine and the increase of simulated data flow. Then, the task requirement of big data batch processing in smart grid is studied, and the scheme of building a batch processing engine based on Spark is put forward. The application of stochastic forest algorithm based on Spark in the analysis of massive power load data proves the validity and expansibility of the solution. Finally, on the basis of the above research, a detailed requirement analysis of big data online analysis and decision-making system of smart grid is carried out, and the overall structure of the system and the function of each module are designed. This design can provide a direct reference for the subsequent software development.
【学位授予单位】：华北电力大学(北京)
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP311.13;F426.61

【参考文献】