海量遥感影像共性产品生产任务调度模型研究及应用

发布时间：2018-02-05 04:49

本文关键词： 任务调度模型负载均衡并行计算高性能遥感产品生产　出处：《河南大学》2014年硕士论文　论文类型：学位论文

【摘要】：随着社会对信息数据的巨大需求，遥感影像数据量呈爆炸式增长，同时，遥感应用对数据处理的巨量需求，推动了遥感影像处理技术的不断发展。高性能集群计算是实现海量高分辨率遥感卫星数据快速处理的重要技术之一，能有效缓解高分遥感应用的“瓶颈”。其中，任务调度模型是提升集群系统效率的关键。作者所参与研发的国家重大专项课题“高分辨率对地观测应用系统”中海量遥感影像共性产品生产系统，具有任务性质单一、子任务量轻、资源文件大且相对独立等特点，而传统的任务调度模型还存在针对性差、实用性不强、调度时间过长等问题，不能满足该系统的特点要求。为解决上述问题，本文采用多指标负载均衡技术以及分层式任务调度方法，对系统中原有的调度模型进行改进，提出了一种与负载均衡相结合的两级任务调度模型。主要工作和贡献如下： 1.在对海量遥感影像共性产品生产系统以及系统原有的任务调度策略分析的基础上，针对原有任务调度策略所存在的吞吐量低、可靠性低以及容错性差等问题，提出了考虑负载均衡的两级任务调度模型。 2.在提出该模型的基础上，首先通过分析负载均衡技术，并根据海量遥感产品生产系统的特点，设计负载均衡器，以及任务管理器与负载均衡器之间的接口；然后，根据系统的任务特点，完成任务调度模型的整体流程设计；最后，完成了任务调度模型中的一级任务调度和二级任务调度的设计。 3.将改进的任务调度模型应用于海量遥感影像共性产品生产系统中，并通过大量的实验数据表明，相比原有的任务调度模型，在系统吞吐量、系统自适应性、系统的可靠性以及系统的容错性等方面，都得到了大大提高。利用改进的任务调度策略，，某些特定卫星切片数据的处理时间甚至缩短了一半以上。
[Abstract]:With the huge demand of society for information data, the amount of remote sensing image data increases explosively. At the same time, there is a huge demand for data processing in remote sensing applications. High performance cluster computing is one of the most important technologies to realize the fast processing of massive high resolution remote sensing satellite data. The task scheduling model is the key to improve the efficiency of cluster system. The production system of mass remote sensing image generality product in the national special project "High Resolution Earth observation Application system", which the author participated in, has a single task and a light sub-task. Resource files are large and relatively independent, but the traditional task scheduling model also has some problems, such as poor pertinence, weak practicability and long scheduling time, which can not meet the characteristics of the system. This paper adopts multi-index load balancing technology and hierarchical task scheduling method to improve the original scheduling model in the system. A two-level task scheduling model combined with load balancing is proposed. The main work and contributions are as follows: 1. On the basis of analyzing the common product production system of mass remote sensing image and the original task scheduling strategy of the system, the throughput of the original task scheduling strategy is low. Due to the low reliability and poor fault tolerance, a two-level task scheduling model considering load balancing is proposed. 2. On the basis of this model, the load balancer is designed by analyzing the load balancing technology and according to the characteristics of mass remote sensing product production system. And the interface between task manager and load balancer; Then, according to the task characteristics of the system, the overall process design of the task scheduling model is completed. Finally, the primary task scheduling and the second level task scheduling are designed in the task scheduling model. 3. The improved task scheduling model is applied to the mass remote sensing image generic product production system, and a large number of experimental data show that compared with the original task scheduling model, the system throughput. The adaptability of the system, the reliability of the system and the fault tolerance of the system have been greatly improved, using the improved task scheduling strategy. Processing time for certain satellite slicing data is even reduced by more than half.
【学位授予单位】：河南大学
【学位级别】：硕士
【学位授予年份】：2014
【分类号】：TP79

【参考文献】