基于Hadoop的电力设备状态监测数据存储与访问研究

发布时间：2018-07-04 19:02

本文选题：智能电网 + 状态监测　；参考：《华北电力大学》2013年硕士论文

【摘要】：进入新世纪后，智能电网一直是电力行业研究和应用的热点，受到了世界各国的广泛关注和推广。随着智能电网的逐步开展，其环境下的电力设备状态监测数据量将剧增，传统的数据存储和处理方法将遇到很大困难；如何安全高效地存储这些数据，并对其进行快速访问已成为新的研究课题。Google关于云计算的论文发表之后，云计算平台Hadoop异军突起，具有海量存储与运算、高扩展性和高可靠性等优势，，成为了解决上述电网问题的新策略。本文首先详细归纳和总结了电力设备状态监测系统的研究现状，重点分析当前技术在数据存储和处理方面所面临的问题；随后对云计算技术尤其是Hadoop平台的研究和使用进行汇总。明确当前具体的应用需求是海量状态监测数据高效存储与快速处理；亟需解决的问题是不断到来的高采样率状态监测数据快速插入、以时序数据为主的多源异构数据可靠性存储，以及在海量状态监测数据上的快速查询问题；根据Hadoop平台的特点，确定将Hadoop平台应用到电力设备状态监测系统，以实现海量状态监测数据的可靠存储与高效查询。结合国内电网公司的应用需求和软硬件实力，以及Hadoop平台的相关技术，针对智能电网对电力设备状态监测的高要求，提出了一种基于虚拟化技术的Hadoop数据存储查询模型，并从总体架构设计、各个存储子模块设计等方面介绍了该模型的实现方法，解决了逻辑存储结构设计、查询算法并行化等相关问题。文中详细地介绍了虚拟化技术下的Hadoop集群搭建，以及集群性能的基准测试，确定所搭建的集群具有海量数据存储能力；对各个子模块进行了性能测试，尤其是通过修改第三方软件YCSB的内核，对设计的HBase数据库进行测试，证明了在电力设备状态监测各类应用环境中相关设计的正确性和有效性。本课题为下一代电力设备状态监测数据的存储与查询研究提供了新的思路；是采用Hadoop平台服务于智能电网领域的一次有益尝试。
[Abstract]:After entering the new century, smart grid has been the research and application hotspot of power industry, which has been widely concerned and popularized all over the world. With the gradual development of smart grid, the data of power equipment condition monitoring will increase dramatically in its environment, the traditional data storage and processing methods will meet great difficulties, how to store these data safely and efficiently, After Google's paper on cloud computing was published, Hadoop, a cloud computing platform, has many advantages, such as massive storage and computation, high scalability and high reliability. It has become a new strategy to solve the above problems. In this paper, the status quo of power equipment condition monitoring system is summarized in detail, and the problems in data storage and processing are analyzed. Then the research and use of cloud computing technology, especially Hadoop platform, are summarized. It is clear that the current specific application requirements are efficient storage and rapid processing of mass state monitoring data, and the urgent problem to be solved is the rapid insertion of continuous high sampling rate status monitoring data. According to the characteristics of Hadoop platform, it is determined to apply Hadoop platform to power equipment condition monitoring system. In order to realize the reliable storage and efficient query of massive state monitoring data. According to the application requirements, hardware and software strength of domestic grid companies, and the related technologies of Hadoop platform, a new Hadoop data storage query model based on virtualization technology is proposed to meet the high requirements of smart grid for power equipment state monitoring. The implementation method of the model is introduced from the aspects of the overall architecture design and the design of each storage sub-module, which solves the problems of logical storage structure design, query algorithm parallelization and so on. In this paper, the Hadoop cluster building under virtualization technology and the benchmark test of cluster performance are introduced in detail. It is determined that the cluster has the capacity of massive data storage, and the performance of each sub-module is tested. In particular, by modifying the core of the third-party software YCSB and testing the designed HBase database, the correctness and effectiveness of the related designs in various application environments of power equipment condition monitoring are proved. This paper provides a new idea for the storage and query of the state monitoring data of the next generation power equipment, and is a useful attempt to use Hadoop platform to serve the smart grid field.
【学位授予单位】：华北电力大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】