地震资料统一存取中索引机制的研究

发布时间：2018-01-10 12:13

本文关键词：地震资料统一存取中索引机制的研究　出处：《东北石油大学》2013年硕士论文　论文类型：学位论文

【摘要】：地震资料是典型的大规模静态数据集,即由海量特定的记录或数据块组成的数据集合,因规模庞大一般存储于分布式文件系统中,这些数据一旦写入便很少再修改。它有多种不同的存储格式,应用程序若为每一种格式都应开发一套接口会有很多不便,不仅自身的代码量会大大增加,其灵活性、可移植性、可扩展性都会受到很大的限制。为解决上述问题,人们研究了多种解决方案,其中“多格式海量数据统一存取”方案提出为数据建立统一的逻辑描述方式,通过对逻辑层标准化接口的操作实现对物理数据的统一访问。这种设计在具有良好的灵活性的同时也由于在物理层和应用层间添加了逻辑层,增加了操作步骤,存在着效率降低的弊端。本文依据地震资料的特点,优化了地震数据统一存取(Unified access of the seismic data,UASD)模型中逻辑层的索引结构,提出了基于UASD的分布式索引框架。同时提出了基于HDFS的一种的可适用于B-树和R-树及其变种的层次索引结构,并给出了索引结构的并行构建方式。研究分析地震资料的统一存取模型,对模型中逻辑层的索引结构进行优化,提取出可快速定位的索引公式。提出基于UASD和HDFS的分布式索引架构,以提高UASD的索引性能。提出基于地震道头关键字的分级建索策略,避免非活跃关键字索引占用空间及维护的开销。研究分析海量数据中常用索引机制,提出基于HDFS的一种可适用于B-树和R-树及其变种的层次索引结构,改变键-值存储结构在非主键索引过程中的劣势。提出层次索引的并行建立方法,加快索引构建速度。最后综合本文的研究内容,介绍UASD模式在地震资料并行处理平台中的应用,以此来验证本文提出的相关技术和算法的可行性和有效性。实验表明,该系列方法减少了查询响应时间、索引空间占用和数据传输开销,有效的提高了UASD的索引性能。
[Abstract]:Seismic data is a typical large-scale static data set, that is, a data set consisting of a large number of specific records or data blocks, because of its large scale, it is generally stored in distributed file systems. Once written, this data is rarely modified. It has many different storage formats, and it would be inconvenient for an application to develop an interface for each format, not just its own code. Its flexibility, portability, scalability will be greatly limited. In order to solve the above problems, people have studied a variety of solutions. One of the "multi-format massive data unified access" scheme proposed for the establishment of a unified logical description of the data. The uniform access to physical data is realized through the operation of the logical layer standardized interface. This design has good flexibility but also adds the logical layer between the physical layer and the application layer and adds the operation steps. There is a drawback of reduced efficiency. According to the characteristics of seismic data, the unified access of the seismic data is optimized. The index structure of the logical layer in the UASD model. A distributed index framework based on UASD and a hierarchical index structure based on HDFS for B- tree, R- tree and its varieties are proposed. The parallel construction method of index structure is given. The unified access model of seismic data is analyzed and the index structure of logic layer in the model is optimized. The index formula which can be located quickly is extracted. The distributed index architecture based on UASD and HDFS is proposed to improve the index performance of UASD. A hierarchical cable construction strategy based on seismic trace head keyword is proposed. To avoid the overhead of using inactive keyword index and maintaining it, the common indexing mechanism in mass data is studied and analyzed. This paper presents a hierarchical index structure based on HDFS, which can be applied to B- tree, R- tree and its varieties, and changes the disadvantage of key-value storage structure in the process of non-primary key index. Finally, the application of UASD model in the parallel processing platform of seismic data is introduced. In order to verify the feasibility and effectiveness of the related techniques and algorithms proposed in this paper. The experiments show that this series of methods reduce query response time, index space consumption and data transfer overhead, and effectively improve the index performance of UASD.
【学位授予单位】：东北石油大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP333

【参考文献】