ODPS平台下的电力设备监测大数据存储与并行处理方法

发布时间：2018-03-17 12:04

本文选题：电力大数据　切入点：公有云　出处：《电工技术学报》2017年09期 　论文类型：期刊论文

【摘要】：计算性能是制约电力大数据应用(基于大数据的故障诊断、预测等)的关键问题。利用分布式存储、并行计算加速此类数据密集型应用是目前较有效的手段。尝试利用阿里云开放数据处理服务(ODPS)存储并加速电力设备监测大数据分析过程。以变压器局部放电(PD)数据相位图谱分析(PRPD)为例,提出了适合高采样率、时序性强的局部放电信号数据存储方法。采用ODPS扩展MapReduce模型(MR2)设计了"Map-Reduce-Reduce"方式的PD信号宏观特征提取方法,提出了并行化PRPD分析算法(ODPS-PRPD),实现了大量PD信号的并行基本参数提取、统计特征计算与放电类型识别。在实验室中构造了4种放电模型并采集了大量PD信号,分别在ODPS平台上和实验室自建的Hadoop平台上进行了性能评估和成本分析。实验分析和结果表明,ODPS-PRPD将大量的中间过程数据(PD谱图数据等)一直保存在内存中,相比自建Hadoop MapReduce平台性能明显提升,并在数据可靠性、服务可用性以及成本方面具有明显优势。
[Abstract]:Computing performance is a key problem that restricts the application of power big data (the fault diagnosis, prediction, etc.) based on big data. Parallel computing is an effective method to accelerate this kind of data intensive application. This paper attempts to use Ali Cloud Open data processing Service (ODPSs) to store and accelerate the analysis process of power equipment monitoring big data. According to the phase map analysis (PRPD) as an example, A data storage method for PD signals with high sampling rate and strong timing is proposed. A "Map-Reduce-Reduce" method for extracting macroscopic features of PD signals is designed by using ODPS extended MapReduce model (MR2). A parallel PRPD analysis algorithm, ODPS-PRPDD, is proposed. The parallel basic parameters of a large number of PD signals are extracted, the statistical characteristics are calculated and the discharge types are identified. Four discharge models are constructed and a large number of PD signals are collected in the laboratory. The performance evaluation and cost analysis were carried out on the ODPS platform and on the Hadoop platform built by the laboratory, respectively. The experimental results show that ODPS-PRPD keeps a large amount of intermediate process data and PD spectrum data in memory. Compared with the self-built Hadoop MapReduce platform, the performance of the platform is significantly improved, and it has obvious advantages in data reliability, service availability and cost.
【作者单位】：华北电力大学控制与计算机工程学院;
【基金】：国家自然科学基金项目(51677072) 河北省自然科学基金项目(F2014502069) 中央高校基本科研业务费专项资金(2016MS116,2016MS117)资助
【分类号】：TM769

【相似文献】