基于自定义RDD的海量遥感图像并行镶嵌方法

发布时间：2018-04-05 06:26

本文选题：遥感图像　切入点：并行镶嵌　出处：《地球信息科学学报》2017年10期

【摘要】：图像镶嵌是遥感图像处理中的重要内容,在跨区域遥感图像分析中发挥重要作用。为了解决传统遥感图像并行算法中存在的计算节点利用率低、频繁数据I/O等问题,本文根据Spark分布式内存计算框架,充分利用Spark利于迭代数据处理的优势,提出了一种基于Spark自定义RDD(弹性分布式数据集)的并行镶嵌方法。该方法首先在集群的多个节点上通过相位相关法执行图像重叠区域估计操作,从而提高了图像重叠区域估计的多节点并行计算;然后,通过重写Spark中RDD的compute和get Partitions方法,自定义针对遥感图像处理的RDD,并将图像镶嵌中的重叠区域估计、图像配准和图像融合3个关键步骤作为自定义RDD的Transformation类型的操作算子;最后,通过隐式转换创建自定义RDD,并调用自定义RDD的操作算子实现图像镶嵌的并行处理。实验结果表明,与传统基于MPI的并行镶嵌算法相比,该方法在保证图像镶嵌效果的基础上,能够有效提高大数据量的图像镶嵌效率。
[Abstract]:Image mosaic is an important part of remote sensing image processing and plays an important role in cross-region remote sensing image analysis.In order to solve the problems of low utilization of computing nodes and frequent data I / O in the traditional parallel algorithm of remote sensing images, this paper makes full use of the advantage of Spark in iterative data processing according to the Spark distributed memory computing framework.A parallel mosaic method based on Spark custom RDD (resilient distributed dataset) is proposed.The method firstly performs the image overlap region estimation operation on multiple nodes of the cluster by phase correlation method, which improves the multi-node parallel computation of the image overlap area estimation, and then rewrites the compute and get Partitions methods of RDD in Spark.The three key steps of image mosaic estimation, image registration and image fusion are used as the Transformation type operators of custom RDD.The custom RDD is created by implicit transformation, and the operation operator of custom RDD is called to realize the parallel processing of image mosaic.The experimental results show that compared with the traditional parallel mosaic algorithm based on MPI, this method can effectively improve the image mosaic efficiency of large amount of data on the basis of guaranteeing the image mosaic effect.
【作者单位】：东北林业大学信息与计算机工程学院;
【基金】：黑龙江省自然基金重点项目(ZD201403)
【分类号】：TP751

【相似文献】