基于海洋平台监测的流数据管理研究
发布时间：2018-07-16 19:13
【摘要】：海洋平台在远离陆地、恶劣环境、极端天气、网络条件差等诸多苛刻的不利条件下实施监测,监测生成的流数据来源各异、数量巨大、快速实时、结构多样、瞬时变化,管理技术要求越来越高,许多传统管理方法和模式不能满足需求,如何科学有效地管理海洋平台监测流数据研究成果和实践经验相对较少,也正是当前亟待解决的问题。本文总结了海洋平台监测流数据的特点和相关特性的变化,国内外流数据远程传输、规范化处理、分布式存储和管理系统的研究现状以及存在的主要问题,提出了基于反馈机制和MD5码检测的流数据远程传输技术,通过发送端传输的数据文件的MD5码和接收端生成的MD5码一致性检验,保证了流数据传输的完整性和一致性；建立了一个包含所有可能情况的数据规范处理方式的监测流数据修复函数库和数据格式信息表,实现了对监测流数据错误、丢失、无序、噪声等问题的处理和对格式、范围、标准不一致统一规范；基于Hadoop搭建了监测流数据分布式存储平台,为高效查询分析数据提供服务。针对现实应用的实际需求,设计了海洋平台监测流数据管理系统,对用户管理权限、流数据远程传输、数据规范化处理、分布式数据存储、数据查询等功能进行了实现。通过对海洋平台监测流数据相关管理问题的研究,为海洋平台监测流数据的管理应用提供了思路,对确保海洋平台安全作业、推动海洋重大工程监测技术的发展和应用具有重大意义。
[Abstract]:Offshore platforms are monitored under harsh adverse conditions, such as far from land, harsh environment, extreme weather, poor network conditions, and so on. The current data generated by the monitoring are of various sources, large quantities, fast and real-time, diverse structures, and instantaneous changes. The requirement of management technology is getting higher and higher, and many traditional management methods and models can not meet the demand. How to manage the monitoring flow data of offshore platform scientifically and effectively has less research results and practical experience, which is also the problem to be solved urgently. In this paper, the characteristics and related characteristics of monitoring stream data on offshore platform are summarized. The research status and main problems of remote transmission, standardized processing, distributed storage and management system of stream data at home and abroad are summarized. Based on the feedback mechanism and MD5 code detection, the streaming data remote transmission technology is proposed. The integrity and consistency of stream data transmission are ensured by checking the MD5 code of the data file transmitted by the sender and the MD5 code generated by the receiver. A monitor stream data repair function library and data format information table including all possible data canonical processing methods are established, which can deal with the problems of monitoring stream data error, loss, disorder, noise and so on. Based on Hadoop, a distributed storage platform for monitoring stream data is built, which provides a service for efficient query and analysis of data. In order to meet the practical requirements of practical applications, a data management system for offshore platform monitoring stream is designed. The functions of user management authority, stream data remote transmission, data normalization, distributed data storage, data query and so on are implemented. Through the research on the management of the monitoring current data of the offshore platform, this paper provides some ideas for the management and application of the monitoring current data of the offshore platform, and ensures the safe operation of the offshore platform. It is of great significance to promote the development and application of marine engineering monitoring technology.
【学位授予单位】：大连理工大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：P715
本文编号：2127403
[Abstract]:Offshore platforms are monitored under harsh adverse conditions, such as far from land, harsh environment, extreme weather, poor network conditions, and so on. The current data generated by the monitoring are of various sources, large quantities, fast and real-time, diverse structures, and instantaneous changes. The requirement of management technology is getting higher and higher, and many traditional management methods and models can not meet the demand. How to manage the monitoring flow data of offshore platform scientifically and effectively has less research results and practical experience, which is also the problem to be solved urgently. In this paper, the characteristics and related characteristics of monitoring stream data on offshore platform are summarized. The research status and main problems of remote transmission, standardized processing, distributed storage and management system of stream data at home and abroad are summarized. Based on the feedback mechanism and MD5 code detection, the streaming data remote transmission technology is proposed. The integrity and consistency of stream data transmission are ensured by checking the MD5 code of the data file transmitted by the sender and the MD5 code generated by the receiver. A monitor stream data repair function library and data format information table including all possible data canonical processing methods are established, which can deal with the problems of monitoring stream data error, loss, disorder, noise and so on. Based on Hadoop, a distributed storage platform for monitoring stream data is built, which provides a service for efficient query and analysis of data. In order to meet the practical requirements of practical applications, a data management system for offshore platform monitoring stream is designed. The functions of user management authority, stream data remote transmission, data normalization, distributed data storage, data query and so on are implemented. Through the research on the management of the monitoring current data of the offshore platform, this paper provides some ideas for the management and application of the monitoring current data of the offshore platform, and ensures the safe operation of the offshore platform. It is of great significance to promote the development and application of marine engineering monitoring technology.
【学位授予单位】：大连理工大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：P715
【参考文献】
相关期刊论文 前1条
1 彭高辉;王志良;;数据挖掘中的数据预处理方法[J];华北水利水电学院学报;2008年06期
相关硕士学位论文 前2条
1 刘少伟;基于云计算的科学工作流数据存储策略研究[D];国防科学技术大学;2011年
2 苗润华;基于聚类和孤立点检测的数据预处理方法的研究[D];北京交通大学;2012年
，本文编号：2127403
本文链接：https://www.wllwen.com/kejilunwen/haiyang/2127403.html
