云环境下分层的中间数据容错方法

发布时间：2019-02-23 19:09

【摘要】：通常在云计算框架的处理过程中会产生大量的、短暂的,同时又非常重要的中间数据。一旦有服务器失效,将会导致中间数据失效,进而影响整个任务的计算。现有的数据容错处理方法仅仅采用简单的复制策略,没有考虑中间数据的特点,会带来庞大的网络开销。因此,提出了一种有效的分层中间数据容错方法,即IDF_Support(intermediate data fault-tolerance_support)方法。通过将计算任务划分为不同类别,IDF_Support方法能够有效地处理中间数据失效。提出了分层的中间数据容错算法,分别是用于解决一个任务内部容错的中间数据容错算法(Inner_Task IDF)和用于解决任务间容错的中间数据容错算法(Outer_Task IDF)。实验结果表明,这些算法在机器出现故障的情况下提高了作业响应时间,保证了系统的可靠性。
[Abstract]:In the process of cloud computing framework, a large amount of intermediate data is generated, which is short and important at the same time. Once the server fails, the intermediate data will fail, which will affect the calculation of the whole task. The existing data fault-tolerant processing methods only adopt simple replication strategy and do not consider the characteristics of intermediate data, which will bring huge network overhead. Therefore, an effective fault-tolerant method for layered intermediate data, called IDF_Support (intermediate data fault-tolerance_support, is proposed. By dividing computing tasks into different classes, the IDF_Support method can effectively deal with intermediate data failures. In this paper, a hierarchical intermediate data fault-tolerant algorithm is proposed, which is used to solve an intra-task fault-tolerant intermediate data fault-tolerant algorithm (Inner_Task IDF) and an inter-task fault-tolerant intermediate data fault-tolerant algorithm (Outer_Task IDF).), respectively. The experimental results show that these algorithms can improve the response time and ensure the reliability of the system in the event of machine failure.
【作者单位】：辽宁大学信息学院;
【基金】：国家自然科学基金Nos.60873068,61472169 辽宁省教育厅优秀人才项目No.LR201017 辽宁大学青年科研基金No.LDQN201438~~
【分类号】：TP302.8

【共引文献】