面向油田科技项目管理的大数据查询优化研究

发布时间：2018-01-08 07:32

  本文关键词：面向油田科技项目管理的大数据查询优化研究　出处：《东北石油大学》2017年硕士论文　论文类型：学位论文

  更多相关文章： 科技项目管理 大数据 ETL设计 查询优化

【摘要】：2009年大庆油田科技管理平台开始上线运行,平台实现了对科技项目审核的全生命周期管理。随着时代的变迁,有关油田科技项目的数据量呈逐年增长态势,庞大的数据量给科技项目管理增加了诸多庞杂的工作,为科技项目的高效管理和避免重复立项等情况的发生,如何能够快速和正确的判断别相似的海量数据信息是要解决的首要问题。因此,本课题拟引入大数据的相关概念,通过搭建大数据运行环境,应用大数据的相关技术对油田科技项目的查询进行优化研究,以此来提高油田项目管理的效率。本文基于课题研究的背景及实际需求,首先,构建了Hadoop大数据运行环境,针对油田科技项目的数据类型和数据源,展开了存储结构的研究和搭建,提供了对异地数据源的数据进行加载存储的策略,以及针对多种类型数据中的结构化数据展开研究,设计支持向存储后的数据进行ETL处理的NoSQL数据库。其次,在此基础上建立基于Hadoop平台的ETL体系结构,实现更为合理的数据ETL工作流程,为下一步的查询优化提供保证。最后,针对油田科技项目管理的数据查询,结合当前大数据查询方法的弊端,分析并研究其底层运行机制和流程,展开优化设计,从而提高整个大数据查询系统的效率,较好的解决了油田科技项目在面向海量数据时的管理效率问题。本课题针对油田科技项目管理平台的实际发展及需要,将基于大数据环境下的存储结构,查询方法融合于油田科技项目管理上,并开发了相应的系统进行实验,验证其可行性,为课题的实现提供了理论支撑和实例验证,在一定程度上对数据量日益增长的油田科技项目管理具有一定的理论意义与参考价值。
[Abstract]:In 2009, Daqing oilfield science and technology management platform began to run online, the platform realized the full life cycle management of science and technology project audit. The volume of scientific and technological projects in oilfields is increasing year by year, and the huge amount of data adds a lot of complicated work to the management of scientific and technological projects, which is the occurrence of the efficient management of scientific and technological projects and the avoidance of duplicate projects. How to quickly and correctly judge the similar mass of data information is the first problem to be solved. Therefore, this paper intends to introduce the relevant concepts of big data, through the establishment of big data operating environment. In order to improve the efficiency of oil field project management, big data's related technology is used to optimize the query of oilfield science and technology project. Firstly, based on the background of the research and the actual needs. The Hadoop big data environment is constructed, and the storage structure is studied and built according to the data types and data sources of oilfield science and technology projects. The strategy of loading and storing data from remote data sources and the research of structured data in various types of data are provided. Design the NoSQL database which supports the ETL processing to the stored data. Secondly, build the ETL architecture based on the Hadoop platform. To achieve a more reasonable data ETL workflow, for the next step of query optimization to provide assurance. Finally, for oilfield science and technology project management data query, combined with the current shortcomings of big data query method. Analysis and study of its underlying operation mechanism and process, the development of optimization design, so as to improve the efficiency of the whole big data query system. Better solve the problem of management efficiency of oilfield science and technology projects in the face of massive data. This subject will be based on the storage structure of big data environment in view of the actual development and needs of oilfield science and technology project management platform. The query method is integrated into the oil field science and technology project management, and the corresponding system is developed to carry on the experiment, to verify its feasibility, provides the theory support and the example verification for the realization of the subject. To a certain extent, it has certain theoretical significance and reference value for the management of oil field science and technology project.
【学位授予单位】：东北石油大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TE4;TP311.13

【相似文献】

中国期刊全文数据库 前10条

1 岳耀书，蒋诞一，向新柱;确定科技项目的基本模式研究[J];武汉纺织工学院学报;1994年S1期

2 赵明;在科技项目中的应用加强项目管理[J];石油规划设计;2002年02期

3 李家庆;孙宴增;卢明;关新;吴兵;;科技项目管理更要科学化[J];中国石油企业;2006年11期

4 胡克松;;石油企业科技项目管理存在的问题及对策研究[J];经济研究导刊;2012年12期

5 帅训波;李家庆;葛兰;霍思，

本文编号：1396196