基于Hadoop的智能变电站数据管理系统设计

发布时间：2018-03-05 05:02

本文选题：Hadoop　切入点：智能变电站　出处：《吉林大学》2017年硕士论文　论文类型：学位论文

【摘要】：伴随我国智能变电站和相应的监测体系不断成熟和完善,产生的电力数据在数量方面大大提高,海量的数据存储和复杂的数据类型使得传统的数据管理模式很难满足电力领域对于数据存储和管理的要求。为使智能变电站高效、稳定的工作,同时实现站内外各类数据的统一管理,需要构建智能变电站信息一体化平台来提高站内各个部门交流与合作,并且基于大量监控数据的存储和管理来为领导层辅助决策。因此,构建新型的智能变电站数据管理系统,以满足企业在海量状态信息存储与查询分析的需要具有重要的意义。同时,Hadoop、No SQL等大数据相关技术的快速发展,为支持海量数据存储、快速检索的智能变电站数据管理系统的研发奠定了坚实基础。本文的具体研究内容如下:(1)针对智能变电站数据特点将数据整理为结构化、半结构化、非结构化三种类型,分别对不同类型的数据设计管理模式,研究变电站在线监测数据的故障诊断框架和方案;(2)设计基于Hadoop的智能变电站数据管理系统总体框架、数据流转过程、存储和检索功能,基于信息融合对变电站故障设备进行精确分析,基于关键词提取设计变电站文献关联检索;(3)在信息一体化平台的基础上开发基于Hadoop框架的智能变电站数据管理系统,采用分布式文件系统(HDFS)和HBase数据库对数据进行分布式存储和管理,并行计算框架Map Reduce作为海量数据查询分析的计算模式,利用Java语言实现B/S模式的智能变电站数据管理系统;(4)使用真实的变电站运行监测数据进行系统测试,分析系统的存储性能、查询性能、读写延迟对比、分布式索引对比。基于Hadoop的智能变电站数据管理系统可以实现变电站各类信息的存储和管理,包括变电站运行数据、在线监测数据、设备管理信息、故障处理报告等。数据的录入既可以手动输入也支持批量导入,管理的数据类型包括结构化数据、半结构化数据和非结构化数据,文档管理中可以自动生成关键词进行存储,便于操作人员快速的检索和读取。利用Hadoop集群对不同规模数据集进行集群I/O性能测试、多维数据查询与分析,测试结果表明该系统能够合理的发挥出Hadoop在分布式存储和计算方面的优势,完成对智能变电站各类业务信息的稳定保存和快速检索等管理功能,不仅极大的降低了运维成本,而且在系统稳定性、扩展性等方面具有明显优势,能够满足大规模智能变电站数据存储查询的需要,同时也为电力系统信息化的建设进行了有益探索。
[Abstract]:With the maturity and perfection of intelligent substation and corresponding monitoring system in our country, the quantity of electric power data produced has been greatly improved. The massive data storage and complex data types make it difficult for the traditional data management mode to meet the requirements of data storage and management in the electric power field. At the same time, to realize the unified management of all kinds of data inside and outside the station, it is necessary to construct the intelligent substation information integration platform to improve the communication and cooperation among the various departments in the station, and based on the storage and management of a large amount of monitoring data to assist the leadership decision-making. It is of great significance to construct a new intelligent substation data management system to meet the needs of mass state information storage and query analysis in enterprises. At the same time, with the rapid development of big data technology, such as Hadoop No SQL, to support mass data storage, The research and development of intelligent substation data management system with fast retrieval has laid a solid foundation. The specific research contents of this paper are as follows: 1) according to the characteristics of intelligent substation data, the data are arranged into three types: structured, semi-structured and unstructured. For different types of data design and management mode, this paper studies the fault diagnosis framework and scheme of substation on-line monitoring data. (2) Design the general framework, data flow process, storage and retrieval function of intelligent substation data management system based on Hadoop. Based on the accurate analysis of substation fault equipment based on information fusion and the design of substation literature association retrieval based on keyword extraction, an intelligent substation data management system based on Hadoop framework is developed on the basis of information integration platform. The distributed file system (HDFS) and the HBase database are used to store and manage the data. The parallel computing framework, Map Reduce, is used as the computing model for the query and analysis of massive data. The intelligent substation data management system based on B / S mode is implemented by Java language. Real substation operation monitoring data are used to test the system, and the storage performance, query performance, read and write delay of the system are analyzed and compared. Distributed index contrast. Intelligent substation data management system based on Hadoop can store and manage all kinds of substation information, including substation operation data, on-line monitoring data, equipment management information, etc. The data input can be manually input and can be imported in batches. The data types managed include structured data, semi-structured data and unstructured data, and can automatically generate keywords for storage in document management. Using Hadoop cluster to test the I / O performance of different size data sets, query and analyze multi-dimensional data. The test results show that the system can reasonably exert the advantages of Hadoop in distributed storage and computing, and complete the management functions of stable storage and fast retrieval of all kinds of service information in intelligent substation, which not only greatly reduces the cost of operation and maintenance, but also reduces the cost of operation and maintenance. Moreover, it has obvious advantages in system stability and expansibility, which can meet the needs of large scale intelligent substation data storage and query. At the same time, it also makes a beneficial exploration for the construction of power system informatization.
【学位授予单位】：吉林大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TM63;TM76;TP311.52

【参考文献】