基于分布式并行聚类的SAR图像变化检测算法研究

发布时间：2018-02-09 19:40

本文关键词： SAR图像变化检测核模糊C均值 Hadoop Spark　出处：《西安电子科技大学》2014年硕士论文　论文类型：学位论文

【摘要】：SAR(Synthetic Aperture Radar)图像变化检测可以应用在自然环境监测、灾害评估、军事打击效果评估和城市规划等众多领域,使得SAR图像变化检测成为遥感领域的研究热点。近年来随着雷达成像能力的提升,SAR图像的数据规模变得越来越大,传统串行变化检测算法已经越来越难以应对数据量不断增长的SAR图像。本文将核模糊C均值聚类算法(Kernel Fuzzy C-Means,KFCM)与分布式并行计算框架相结合,提出了两种基于分布式并行聚类的SAR图像变化检测方法,分布式并行计算框架能够充分利用集群中众多节点的计算能力和存储能力,可以有效应对大规模的SAR图像变化检测问题,加快变化检测的速度。1.提出了一种基于H-KFCM(Hadoop based KFCM)的分布式并行SAR图像变化检测方法。在变化检测过程中,利用聚类算法对差异图进行聚类分析是计算复杂度最高,计算时间最长的部分。Hadoop分布式集群计算平台可以利用分布式文件系统HDFS(Hadoop Distributed File System)和Map Reduce计算模型分布式地存储和处理大规模数据。本文提出的基于H-KFCM的SAR图像变化检测方法,将KFCM算法与Hadoop相结合,将KFCM算法中隶属度迭代计算映射到Map Reduce中的Map阶段,通过集群多节点分布式执行Map任务实现隶属度矩阵的并行计算;将KFCM的聚类中心更新过程映射到Reduce阶段,Reduce收集不同节点的Map任务计算结果,规约求和更新聚类中心,实现了对SAR图像变化检测差异图的分布式并行聚类。通过设计相关实验,证明了本文提出的H-KFCM分布式并行变化检测方法是有效的,并且在Hadoop集群节点数增加时,对相同SAR图像的变化检测时间明显减少,证明了该算法具有良好的并行加速性能。2.提出了一种基于S-KFCM(Spark based KFCM)的SAR图像变化检测方法。Spark是一种针对迭代作业和交互分析等领域而设计的基于内存计算的分布式并行平台,相比于Hadoop的Map Reduce框架,Spark在运行相同的作业时可以取得几十倍甚至百倍加速性能提升。该算法将利用KFCM算法分析差异图的过程迁移到Spark内存计算框架上实现,将变化检测过程中最耗时的聚类分析过程并行的完成。在Spark中,KFCM算法的计算过程由一系列有依赖关系的弹性分布式数据集(Resilient Distributed Datasets,RDD)实现,Spark能够在集群多个节点上对RDD进行并行计算。KFCM迭代过程需要重复地利用输入数据集,RDD能够将这些复用数据集驻留在内存中,避免频繁的硬盘I/O,提高了计算效率。通过在不同规模SAR图像数据集上的实验,结果证明基于S-KFCM的变化检测方法是有效的,且相比基于Hadoop实现的变化检测方法,该方法能达到17倍左右的加速性能,证明了该方法能够支持大规模SAR图像数据的聚类分析,快速有效的完成SAR图像变化检测。
[Abstract]:SAR(Synthetic Aperture Radar-based image change detection can be used in many fields, such as natural environment monitoring, disaster assessment, military strike effect evaluation and urban planning, etc. In recent years, with the improvement of radar imaging ability, the data scale of SAR image becomes larger and larger. The traditional serial change detection algorithm has become more and more difficult to cope with the growing data volume of SAR images. In this paper, Kernel Fuzzy C-Meansn KFCM-based clustering algorithm is combined with distributed parallel computing framework. Two SAR image change detection methods based on distributed parallel clustering are proposed. The distributed parallel computing framework can make full use of the computing power and storage capacity of many nodes in the cluster. It can effectively deal with the problem of large-scale SAR image change detection, and accelerate the speed of change detection. 1. A distributed parallel SAR image change detection method based on H-KFCM-Hadoop based KFCM is proposed. Using clustering algorithm to cluster analysis of difference map is the highest computational complexity. The longest computing time. Hadoop distributed cluster computing platform can use distributed file system HDFS(Hadoop Distributed File system) and Map Reduce computing model to store and process large-scale data distributed. This paper proposes a SAR image change detection method based on H-KFCM. The KFCM algorithm is combined with Hadoop, and the iterative calculation of membership degree in KFCM algorithm is mapped to the Map stage in Map Reduce. The parallel computation of membership matrix is implemented by cluster multi-node distributed execution of Map task. The update process of KFCM cluster center is mapped to the Map task calculation result of different nodes collected by reduce in Reduce stage, and the clustering center is updated by protocol summation. The distributed parallel clustering of SAR image change detection difference graph is realized. It is proved that the H-KFCM distributed parallel change detection method proposed in this paper is effective, and when the number of Hadoop cluster nodes increases, the change detection time for the same SAR image is obviously reduced. It is proved that the algorithm has good parallel acceleration performance. 2. A SAR image change detection method based on S-KFCM-Spark based KFCM is proposed. Spark is a distributed parallel platform based on memory computing, which is designed for iterative operation and interactive analysis. Compared with Hadoop's Map Reduce framework, Spark can achieve a performance improvement of tens or even a hundredfold when running the same job. The algorithm migrates the process of analyzing difference graph using KFCM algorithm to Spark memory computing framework. Parallel completion of the most time-consuming clustering analysis process in the change detection process. In Spark, the KFCM algorithm is computed by a series of resilient Distributed data sets resilience data sets (RDDD), which can be implemented on multiple node pairs in a cluster. The RDD parallel computation. KFCM iteration process requires repeated use of input data sets to enable these multiplexed datasets to reside in memory, Through experiments on different scale SAR image data sets, the results show that the change detection method based on S-KFCM is effective and compared with the change detection method based on Hadoop. This method can achieve about 17 times the acceleration performance. It is proved that this method can support the clustering analysis of large scale SAR image data and can quickly and effectively complete the change detection of SAR images.
【学位授予单位】：西安电子科技大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TN957.52

【相似文献】