地震数据存储系统研究

发布时间：2018-03-24 11:30

本文选题：SDFS　切入点：地震数据　出处：《中国科学技术大学》2015年硕士论文

【摘要】：近年来,得益于人类社会对石油的需求,石油勘探技术也在飞速发展。地震勘探是进行石油勘探的主要的方法,随着地震勘探技术在地震数据采集、资料处理和资料解释方面不断创新,地震勘探所产生的数据也在海量增长。目前,人们针对这些海量的数据计算发展了高性能计算,使得地震数据处理速度越来越快,但是地震数据处理过程中数据读取的I0瓶颈严重阻碍了数据的处理速度。因此,如何提高地震数据的读取速度成了一个亟待解决的问题。目前针对地震数据10瓶颈的问题,做法多是采用分布式文件系统作为存储系统。每种分布式文件系统的存储特点是不一样的,如HDFS适合大文件的存储,并且对大文件读取进行了优化,又如Fast DFS适合小文件的存储。而地震数据的读取特点是块小量大,即读取的粒度小但读取量大。目前的分布式系统并没有与地震数据读取特点相吻合的读取模式,因而地震数据读取的10瓶颈问题仍存在。本文通过研究地震数据处理的主要数据格式SEGY的格式特点以及一般地震数据的处理流程,分析出目前地震数据处理遇到10瓶颈的原因,实现了一种基于Fast DFS的地震数据存储系统即SDFS(Seismic Data File System)系统。该系统实现了解决地震数据读取10瓶颈的方案。该方案为:首先,通过将地震数据文件以道为单位的分块存储,提高了数据的读取效率;其次,通过将文件宏块再分块,并使用并行磁盘读取技术来提高了地震数据的读取速度;最后还提出了针对系统数据储可靠性的研究,即通过完整备份的方式增加系统可靠性。为了证明该方案的有效性,本文实现了 SDFS系统并对系统的主要功能进行了设计,然后进行了两组地震数据读取的实验。实验一测试了并行磁盘读取技术的有效性,实验结果显示,当每个节点硬盘数据量增加时系统的读取性能增加,速度几乎与磁盘数量成正比;当数据块大小增加时,对顺序读有利而减小则对随机读有利。实验二则用对比实验的方式进行了系统的读取性能测试,对比的系统为Fast DFS,测试结果为SDFS系统的顺序读和随机读性能都要明显优于Fast DFS。综合两次实验结果可知,本文提出的SDFS系统能够帮助改善地震数据读取的10性能。
[Abstract]:In recent years, petroleum exploration technology has been developing rapidly because of the demand for oil in human society. Seismic exploration is the main method of petroleum exploration, and with seismic exploration technology in seismic data acquisition, Data processing and data interpretation are constantly innovating, and the data generated by seismic exploration are also increasing. At present, people have developed high-performance computing for these massive data calculations, which has made seismic data processing faster and faster. However, in the process of seismic data processing, the I _ 0 bottleneck of data reading seriously hinders the speed of data processing. Therefore, how to improve the speed of seismic data reading has become a problem to be solved. Distributed file system is used as storage system. The storage characteristics of each distributed file system are different. For example, HDFS is suitable for large file storage, and the large file reading is optimized. For example, Fast DFS is suitable for the storage of small files. However, the reading of seismic data is characterized by a small number of blocks, that is, small granularity but large amount of reading. The current distributed system does not have a reading mode consistent with the characteristics of seismic data reading. Therefore, the 10-bottleneck problem of seismic data reading still exists. By studying the format characteristics of SEGY, the main data format of seismic data processing and the processing flow of general seismic data, this paper analyzes the causes of the 10 bottleneck in seismic data processing. A seismic data storage system based on Fast DFS, that is, SDFS(Seismic Data File system, is implemented in this system, which solves the 10 bottleneck problem of seismic data reading. The scheme is as follows: firstly, the seismic data file is stored in blocks with trace as the unit. The efficiency of data reading is improved. Secondly, the read speed of seismic data is improved by dividing the macro blocks of files into blocks and using parallel disk reading technology. Finally, the research on the reliability of data storage in the system is put forward. In order to prove the validity of the scheme, the SDFS system is realized and the main functions of the system are designed. Then two groups of seismic data reading experiments are carried out. Experiment 1 tests the effectiveness of the parallel disk reading technology. The experimental results show that the reading performance of the system increases with the increase of hard disk data at each node. The speed is almost proportional to the number of disks; when the size of the data block increases, it is advantageous to read at random while it is beneficial to sequential reading. The compared system is Fast DFS.The test results show that the sequential and random reading performance of SDFS system is obviously better than that of Fast DFS.The results of two experiments show that the SDFS system presented in this paper can help to improve the 10 performance of seismic data reading.
【学位授予单位】：中国科学技术大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：P631.43;TP333

【相似文献】