通信密集环境下基于内存利用率的预计算方法

发布时间：2018-06-30 18:37

  本文选题：通信密集型图计算 + 内存利用率　； 参考：《西安交通大学学报》2017年10期

【摘要】：针对通信密集型图计算环境下原静态最大消息数阈值方法由于内存不足导致的频繁低效I/O问题,提出了一种基于内存利用率的预计算方法。该方法利用了图应用的计算满足交换律和结合律的特点,根据当前进程的内存利用率判断是否将本轮超步通信过程中的部分消息进行预计算,同时在预计算过程中使用细粒度锁以增大预计算线程的并发度;在下轮超步的正常计算时合并上轮的预计算结果,实现了通信和计算的重叠,达到减少作业响应时间和磁盘I/O开销的目的。实验结果表明,在通信密集场景下,该方法在性能和I/O开销上均优于已有的MMT方法,作业响应时间减少了5.9%~79.0%,同时计算过程中的磁盘开销减少了9.99%~79.87%。
[Abstract]:Aiming at the frequent and inefficient I / O problem caused by the original static maximum message number threshold method in communication intensive graph computing environment, a precalculation method based on memory utilization is proposed. This method utilizes the calculation of graph application to satisfy the characteristics of switching law and combination law. According to the memory utilization ratio of current process, it determines whether part of the messages in the course of this super-step communication can be pre-calculated. At the same time, the fine granularity lock is used to increase the concurrency degree of the prediction thread in the process of pre-calculation, and when the next wheel overstep is calculated normally, the pre-calculated results of the upper wheel are combined to realize the overlap of communication and calculation. To reduce job response time and disk I / O overhead. The experimental results show that the proposed method is superior to the existing MMT method in performance and I / O overhead in the communication intensive scenario. The response time of the job is reduced by 5.9% 79.0 and the disk overhead in the calculation process is reduced by 9.99% 79.87.
【作者单位】： 西安交通大学电子与信息工程学院;深圳信息职业技术学院软件学院;
【基金】：国家重点研发计划资助项目(2016YFB0201402,2016YFB0201800) 国家自然科学基金资助项目(61572394) 深圳市科技计划资助项目(JSGG20140519141854753)
【分类号】：TP333
，

本文编号：2086460