一种能效优化的MapReduce资源比模型

发布时间：2018-03-01 06:00

本文关键词： 云计算能效资源比 MapReduce　出处：《计算机学报》2015年01期 　论文类型：期刊论文

【摘要】：随着云计算的快速发展,IT资源规模的不断扩大导致能耗问题日益凸显.为降低MapReduce编程模型带来的高能耗,文中研究Map/Reduce任务的资源消费特征及该特征与能效的关系,旨在寻找一种能够指导资源分配和任务调度的资源模型,进而实现能效优化.文中提出任务的能效与任务被分配的资源量无关,而与其被分配的各种资源的资源量比例相关,且存在一个"最佳资源比"使得能效达到最高.基于此,文中首先提出了普适的资源和能效模型,从模型层面证明最佳资源比和能效之间的关系,量化空闲资源量和空闲能耗;随后分析MapReduce编程模型,将普适资源比模型变换到MapReduce下.通过抽象的数据的"生产者-消费者"模式,求解Map/Reduce任务的最佳资源比;最后,通过实验从任务能效和空闲能耗两个角度证明了最佳资源比的存在,并根据实验结果,对MapReduce执行过程进行划分,给出了部分Map/Reduce任务的最佳资源比.最佳资源比的提出和求解将有利于基于该最佳资源比的任务调度和资源分配算法的研究,进而实现Map/Reduce任务能效的提高.
[Abstract]:With the rapid development of cloud computing, the problem of energy consumption is becoming more and more serious due to the expansion of IT resources. In order to reduce the high energy consumption brought by MapReduce programming model, this paper studies the resource consumption characteristics of Map/Reduce task and its relationship with energy efficiency. The purpose of this paper is to find a resource model that can guide resource allocation and task scheduling so as to optimize energy efficiency. In this paper, it is proposed that the energy efficiency of a task is independent of the amount of resources assigned to the task, but is related to the proportion of resources allocated to various resources. And there is a "best resource ratio" to achieve the highest energy efficiency. Based on this, a universal resource and energy efficiency model is proposed in this paper. The relationship between the optimal resource ratio and energy efficiency is proved from the model level, and the amount of free resources and free energy consumption are quantified. Then, the MapReduce programming model is analyzed, and the universal resource ratio model is transformed to MapReduce. Through the "producer-consumer" model of abstract data, the optimal resource ratio of the Map/Reduce task is solved. The existence of optimal resource ratio is proved from two aspects of task energy efficiency and idle energy consumption through experiments. According to the experimental results, the execution process of MapReduce is divided. In this paper, the optimal resource ratio of some Map/Reduce tasks is given, which will benefit the research of task scheduling and resource allocation algorithm based on the optimal resource ratio, and then improve the efficiency of Map/Reduce tasks.
【作者单位】：东北大学软件学院;东北大学信息科学与工程学院;
【基金】：国家自然科学青年基金(61202088) 中国博士后科学基金面上项目(2013M540232) 中央高校基本科研业务费专项资金(N120817001) 教育部博士点基金(20120042110028)资助~~
【分类号】：TP3

【参考文献】