采用Map-Reduce模型的海量电能质量数据交换格式文件快速解析方案

发布时间：2018-03-08 04:16

本文选题：电能质量　切入点：PQDIF文件　出处：《电网技术》2014年06期 　论文类型：期刊论文

【摘要】：针对现有方案对电能质量监测系统中海量PQDIF文件解析效率低、解析时间长等问题,研究实现了一种基于集群计算架构的海量PQDIF文件快速解析方案。通过Map-Reduce模型来实现海量PQDIF文件的快速并行解析。Map-Reduce模型将数据集的大规模操作任务拆分成若干子任务分配给网络上的每一个计算节点(如PC机),实现多计算节点共同协调处理任务,达到提高运行效率的目的。方案采用Java和C++编写,在异构平台间具有较强的可移植性,并且可以通过增加计算节点来提高解析效率,具有较强的可扩展性。通过对海量PQDIF文件进行解析测试,结果表明该方案可以显著提高文件解析效率。
[Abstract]:In order to solve the problems of low efficiency and long parsing time of massive PQDIF files in power quality monitoring system, In this paper, a fast parse scheme of massive PQDIF files based on cluster computing architecture is proposed. The fast parallel parse of massive PQDIF files is realized by using Map-Reduce model. Map-Reduce model splits the large-scale operation tasks of data set into several sub-tasks. The service is assigned to every computing node on the network (such as PC computer) to coordinate the processing tasks of multiple computing nodes. The scheme is written by Java and C, which has strong portability among heterogeneous platforms, and can improve the parsing efficiency by adding computing nodes. By analyzing the massive PQDIF files, the results show that this scheme can significantly improve the efficiency of file parsing.
【作者单位】：四川大学电气信息学院;国网福建省电力有限公司电力科学研究院;
【分类号】：TM76

【参考文献】