基于Hadoop平台的地震波形数据处理方法研究

发布时间：2018-06-25 20:26

本文选题：Hadoop + 并行解压　；参考：《河北师范大学》2015年硕士论文

【摘要】：近年来国家对地震领域的发展颇为关注,地震数据分析成为地震领域的热门话题。随着科学技术的不断发展,地震数据采集仪器的精密度有了很大提升。分散在国内的地震监测台站数量不断增加,这使得数据采集量大增。国家台网中心每天汇集约40GB的地震波形数据,如此海量的数据给数据存储和分析造成了很大的挑战。为了方便传输,通常将波形数据保存为SEED格式。在进行地震数据分析前,需要首先将SEED格式文件解压缩,得到原始样本序列。现有方法是利用串行思想,每次只能够解压缩单个文件,无法完成批量文件的解压缩操作。由于地震数据在采集过程中会收集到一些干扰信号,这些信号会影响地震分析的准确性,因此在解压缩得到原始样本序列后,需要先对序列进行滤波处理,保证地震数据分析的质量。面对大量地震波形数据时,串行滤波方法存在计算速度慢,处理效率较低的问题。Hadoop是开源的分布式计算框架,也是目前使用较广泛的云计算技术之一。其核心组件分布式文件系统可以实现海量数据的可靠存储,Map Reduce编程模型可以通过并行化方式处理大规模数据,缩短数据处理时间,因此可以利用Hadoop平台来解决上述问题。本文从如下方面进行了研究:(1)基于地震波形数据文件的格式特征,对在Hadoop平台上并行处理该类数据文件进行可行性分析。(2)利用Map Reduce并行编程模型思想,设计了与地震波形数据文件相匹配的输入格式,提出并实现了批量地震波形数据文件的并行解压缩算法。其中采用二次排序方法确定通道内数据记录的排列顺序,保证解压后数据拼接的正确性。(3)针对解压后得到的数据,通过分析滤波操作原理,结合Map Reduce并行编程模型特点,提出了一种并行滤波处理算法,实现了多通道数据的并行滤波操作。(4)搭建小型Hadoop集群环境,对提出的两种算法进行实验测试,分析实验结果及算法性能。
[Abstract]:In recent years, the country pays close attention to the development of the seismic field, and seismic data analysis has become a hot topic in the seismic field. With the development of science and technology, the precision of seismic data acquisition instrument has been greatly improved. The number of seismic monitoring stations scattered in China is increasing, which makes the data acquisition increase greatly. The National Network Center collects about 40 GB of seismic waveform data every day, which poses a great challenge to data storage and analysis. In order to facilitate transmission, waveform data is usually saved as seed format. Before seismic data analysis, the seed format file should be decompressed to obtain the original sample sequence. The existing method is to decompress a single file at a time by using the idea of serial, and can not complete the decompression operation of batch file. Because some interference signals will be collected in the process of seismic data acquisition, these signals will affect the accuracy of seismic analysis. Therefore, after decompressing and obtaining the original sample sequence, we need to filter the sequence first. Ensure the quality of seismic data analysis. In the face of a large number of seismic waveform data, the serial filtering method has the problem of slow computation speed and low processing efficiency. Hadoop is an open source distributed computing framework and one of the most widely used cloud computing technologies. Its core component, distributed file system, can realize the reliable storage map reduce programming model of massive data. It can process large scale data by parallelization and shorten data processing time. Therefore, Hadoop platform can be used to solve the above problems. In this paper, the following aspects are studied: (1) based on the format features of seismic waveform data file, the feasibility of parallel processing of this kind of data file on Hadoop platform is analyzed. (2) the idea of parallel programming model based on Map reduce is used to analyze the feasibility of parallel processing of this kind of data file on Hadoop platform. The input format matching the seismic waveform data file is designed and the parallel decompression algorithm of batch seismic waveform data file is proposed and implemented. Among them, the secondary sorting method is used to determine the arrangement order of the data records in the channel to ensure the correctness of the data splicing after decompression. (3) according to the decompressed data, the filtering operation principle is analyzed and the characteristics of the Map reduce parallel programming model are combined. A parallel filtering algorithm is proposed to realize the parallel filtering operation of multi-channel data. (4) A small Hadoop cluster environment is built and the two algorithms are tested and the experimental results and algorithm performance are analyzed.
【学位授予单位】：河北师范大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：P315.63;TP311.13

【参考文献】