空间科学大数据存储模型SP-HDF及应用研究

发布时间：2018-03-08 17:04

本文选题：空间天气科学数据　切入点：存储模型　出处：《中国地质大学（北京）》2013年博士论文　论文类型：学位论文

【摘要】：空间科学是研究发生在日地空间、行星际空间及至整个宇宙空间的物理、天文、化学及生命等自然现象及其规律的科学。空间科学观测和研究过程中将产生数量巨大、种类繁多、数据结构复杂、计算密集型的科学数据，有效地存储、管理、共享这些科学数据已经成为空间科学应用研究迫切需要解决的一个重要问题。本文充分分析了海量空间科学数据的特征，深入研究了HDF（Hierarchy Data Format）、CDF（Common DataForm）等新的存储技术。在此基础上，，提出一种新的空间科学大数据自描述存储模型SP-HDF（Space-HDF）。考虑到目前标准化和非标准化数据并存的情况，作者提出了一种基于元数据描述的虚拟观测站点共享服务模型。本文的研究成果及创新点包括： (1)提出一种针对空间科学大数据存储管理和集成共享的自描述存储模型SP-HDF。针对空间天气和子午工程空间科学数据的特点和逻辑结构，设计了相应的逻辑存储结构，在此基础上，为用户设计了简单、易用、友好的数据访问接口。SP-HDF存储模型的优势在于屏蔽了用户直接访问底层HDF和CDF文件的细节，因而降低了数据访问的复杂性。另一方面逻辑存储结构直接面向领域应用，提升了数据访问粒度和数据抽象级别，在实际应用中更加易于理解。 (2)针对目前大量空间科学数据在短期内无法完全标准化的现状，提出一种基于虚拟观测站点的共享服务模型。该模型通过共享服务元数据模型和抽象数据服务接口，实现多元、异构科学数据的集成和共享服务。 (3)设计并研发了一个基于SP-HDF存储模型的空间天气科学数据逻辑结构及各种大数据服务接口的原型系统。通过对核心数据接口的访问，原型系统实现了与数值模拟、可视化等多个系统的集成应用。验证了基于虚拟观测站点的共享服务模型的子午工程数据服务接口原型，实现了数据检索、数据下载、数据可视化等多个功能。 (4)设计并实现了基于XML半形式化描述和数据处理模板技术的子午工程科学数据标准化处理模型，解决了多元、异构、海量空间科学数据的高效处理和灵活定制的困难问题。
[Abstract]:Space science is research in physics, sun earth space, interplanetary space and the whole space astronomy, chemistry and life and other natural phenomena and laws of science. Space science research and observation in the process of large quantity, variety, complex data structure, scientific data intensive computing, efficient storage, management. The scientific data sharing has become an important issue in the research of space science applications need to be solved urgently. This paper analyses the characteristics of massive space science data, in-depth study of the HDF (Hierarchy Data Format), CDF (Common DataForm) and other new storage technology. On this basis, put forward a new kind of space science data self description storage model of SP-HDF (Space-HDF). Considering the current standardized and non standardized data coexist, the author proposed a method according to the virtual observation station based on quaternion description The research results and innovation points of this paper are as follows:
(1) is proposed for space science data storage management and self characteristics and logical structure description storage model of SP-HDF. for space weather and meridian project space science data integration and sharing, design the logical storage structure, on this basis, for the user to design a simple, easy to use, friendly interface of data access.SP-HDF the storage model has the advantage of shielding users direct access to the underlying HDF and CDF files in detail, thus reducing the complexity of data access. On the other hand the logical storage structure directly facing the field of application, enhance the level of abstraction and granularity of data access, more easy to understand in the practical application.
(2) according to the current situation of large space science data in the short term can not completely standardized, proposed a shared service model for virtual stations. Based on the model to realize multiple through shared service metadata model and abstract data service interface, service, integration and sharing of heterogeneous scientific data.
(3) the design and development of a SP-HDF storage model based on space weather science data logical structure and all kinds of large data service interface prototype system. Based on the core data access interface, prototype system has realized the numerical simulation and visualization, integration of multiple systems should be used. To verify the data service interface prototype of meridian project shared service model of virtual observation stations based on the implementation of data retrieval, data download, data visualization and other functions.
(4) a standardized processing model of meridian engineering data based on XML semi formal description and data processing template technology is designed and implemented, which solves the difficult problems of high efficiency and flexible customization of multivariate, heterogeneous and massive spatial scientific data.

【学位授予单位】：中国地质大学（北京）
【学位级别】：博士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】