遥感数据快速处理编程支撑工具研究

发布时间：2018-04-08 08:50

本文选题：遥感数据处理　切入点：并行计算　出处：《中国科学院大学(中国科学院遥感与数字地球研究所)》2017年博士论文

【摘要】：传感器和计算机技术的发展,正逐渐改进着遥感数据的收集、管理和分析的方式。传感器的分辨率在光谱维度和空间维度不断增长,遥感数据越来越呈现出“超大规模数据量”的特征,同时随着地理空间分析和建模复杂程度的提高,遥感图像处理与信息产品生产的计算越来越呈现出“密集型计算”的特征,另外,应急遥感等应用通常具有很高的时效性要求,以上海量数据处理、算法复杂以及实时性的要求,为遥感数据的信息提取带来了挑战。并行计算为解决上述问题提供了良好契机,但是利用低级的并行编程语言对遥感处理算法进行并行化实现是复杂繁琐的工作,遥感专家在应对海量的多维遥感图像数据和复杂遥感应用算法的同时,还必须具备充分的并行系统结构知识。因此,需要研究遥感数据并行处理快速编程工具,来简化并行计算编程并提高代码的复用性。另外,遥感数据处理模式多样且复杂,其中数值问题并行计算以及DAG类遥感应用处理算法的并行实现难度比较大。并行计算在遥感数据处理中的应用已经发展了很多年,但是存在以下不足之处:第一,遥感数据处理并行解决方案与具体算法之间过于耦合,代码复用性差;第二,遥感数据具有多波段和元数据复杂的特点,将多维遥感图像映射到通用并行编程骨架中具有一定难度;第三,直接基于低级的编程语言实现的遥感数据并行处理快速编程工具,代码的稳定性和可靠性存在一定风险。第四,已有遥感并行编程模版通常只提供简单数据并行模式,一般只有点运算、邻域计算和全局计算等。缺乏面向数值计算和DAG类运算的遥感并行编程快速实现工具。针对以上问题,本文在分析遥感数据和处理流程的基础上,探讨遥感数据快速处理编程支撑工具的关键问题,构建和实现了各类典型遥感并行算法骨架。本论文的主要贡献和创新性主要体现在以下几个方面:(1)提出了基于通用并行算法骨架构建遥感数据快速处理编程工具的思路,从三个渐进层次构建和实现了各类典型的遥感数据并行编程模版。避免直接利用低级编程语言实现的不稳定性的风险,提高并行编程模版构建的抽象层次。(2)提出遥感数据抽象模型,并以泛型编程的形式,构建遥感数据模版类。分别针对单波段和多波段遥感数据,制定多类划分方式,按照规则映射到Muesli分布式矩阵空间内,构建分布式遥感数据模版类。(3)对典型遥感图像处理和信息产品生产算法,提炼共性的并行处理流程。在Muesli任务并行骨架之上构建遥感数据并行编程模版RSParallel,为遥感领域并行模型的发展提供了示范性思路。(4)提出遥感应用领域共性数值计算问题进行更高层次抽象的思想,构建针对非线性方程组求解的遥感数据并行编程算法骨架。融合遥感数据并行编程模版和已有数值计算库,构建更高层次的算法骨架RSPallelNLE,为其他遥感应用数值问题并行化快速实现提供了参考。(5)构建基于many-task计算的遥感数据处理DAG并行编程模版,针对具有前后数据依赖关系的DAG类遥感应用算法,基于多任务计算,定义了务表示模型,设计了基于关键路径和深度的DAG动态调度策略,构建的遥感数据处理DAG并行编程模版,并在大尺度生态水文模拟计算中实现二十万量级的任务的并行计算。选取典型遥感应用算法,对以上各类并行编程模版进行并行性能测试,取得了良好的效果。理论和实验表明,提炼遥感数据处理共性模式,形成相应的并行编程模版是遥感数据并行计算快速实现的有效途径。
[Abstract]:The development of sensors and computer technology, is gradually improved with remote sensing data collection, management and analysis. The resolution of the sensor in the spectral dimension and space dimension is growing, more and more remote sensing data showing the characteristics of large scale data ", with spatial analysis and modeling complexity of remote sensing image processing the production of information products and the calculation shows more and more" intensive computing "features, in addition, high requirement of timeliness usually have emergency remote sensing applications, the amount of data processing in Shanghai, complex algorithm and real-time requirements, a challenge for the information extraction of remote sensing data parallel computing provides a good opportunity. In order to solve the problem, but the use of parallel programming language on the remote sensing processing algorithm parallel implementation is a complicated work of low-level, remote sensing experts in response to the sea Multi dimensional remote sensing image data and the complexity of the application of remote sensing algorithms at the same time, also must have sufficient knowledge of the parallel system structure. Therefore, fast programming tools for parallel processing of remote sensing data, to simplify parallel programming and improve the reusability of the code. In addition, the remote sensing data processing mode is diverse and complex, the problem of parallel computing and numerical DAG parallel processing algorithm for remote sensing applications more difficult to achieve. Parallel computing in remote sensing data processing have been developed for many years, but has the following shortcomings: first, remote sensing data processing to solve the parallel between the program and specific algorithm is coupled, code reuse is poor; second, has the characteristics of multi band remote sensing data and metadata the complex, multidimensional remote sensing image is mapped to a general-purpose parallel programming framework has a certain degree of difficulty; third, directly based on low level programming language Fast programming tools for parallel processing of remote sensing data is implemented, the stability and reliability of the code there is a certain risk. Fourth, the existing remote sensing parallel programming template usually provide only simple data parallel mode, generally only point operations, neighborhood calculation and global computing. For the lack of numerical calculation and DAG calculation of remote sensing parallel programming tools. For fast implementation the above problems, based on the analysis of remote sensing data and processes on the key issues of rapid processing programming tools to support remote sensing data, constructs and implements various typical remote sensing parallel algorithm framework. The main contributions and innovations of this paper are mainly reflected in the following aspects: (1) put forward the general framework construction fast parallel algorithm remote sensing data processing programming tool based on the ideas from the three progressive level of construction and implementation of the remote sensing data of all kinds of typical parallel programming Template. Avoid instability risk directly using low-level programming language, improve parallel programming template construction level. (2) proposed model of remote sensing data, and generic programming in the form of construction of remote sensing data template class. For single band and multi band remote sensing data, making multi class classification, in accordance with the rules distributed matrix is mapped to the Muesli space, the construction of distributed remote sensing data template class. (3) of typical remote sensing image processing and information production algorithm, parallel processing processes of extracting common. Parallel frame based parallel programming template RSParallel remote sensing data in the Muesli task, provides a demonstration of the way of thinking for the development of remote sensing parallel model. (4) the common field of remote sensing application problems of numerical calculation of higher level abstract thought, construction of remote sensing data for solving the nonlinear equations of parallel series The process algorithm skeleton. Fusion numerical library programming template and the existing remote sensing data parallel algorithm for constructing RSPallelNLE skeleton of a higher level, the application of remote sensing for other numerical problems parallel fast reference. (5) the construction calculation of many-task remote sensing data processing based on DAG parallel programming template, for a class DAG application of remote sensing data and algorithm the dependence, based on Calculation of multi task, the definition of service model, designed the DAG dynamic scheduling strategy based on the critical path and depth, the construction of remote sensing data processing DAG parallel programming template, and calculate the parallel implementation of the order of two hundred thousand tasks in simulation of eco hydrological scale. The typical application of remote sensing algorithm. All kinds of parallel programming templates for parallel performance test, and achieved good results. The theoretical and experimental results show that the total pattern extraction of remote sensing data, The formation of corresponding parallel programming templates is an effective way for the rapid realization of remote sensing data parallel computing.

【学位授予单位】：中国科学院大学(中国科学院遥感与数字地球研究所)
【学位级别】：博士
【学位授予年份】：2017
【分类号】：TP75

【相似文献】