基于Hadoop-GPU的RBM云计算实现
发布时间：2018-07-23 12:07
【摘要】：针对受限波尔兹曼机处理大数据时存在的训练缓慢问题,在Hadoop云计算平台和GPU并行加速的基础上设计了基于Hadoop-GPU框架的的RBM加速计算实现方法.通过对MapReduce机制和RBM训练过程的分析,将RBM训练分割为采用Map端实现吉布斯采样,Reduce端实现参数更新,并通过GPU实现运算并行加速的方法组合.最后通过MNIST手写数字识别集实验证明,在大规模数据下,Hadoop-GPU平台对RBM的训练具有良好的可行性,加速比达到20以上,并且随着数据规模的增加,加速比呈现更为显著的增长.
[Abstract]:Aiming at the problem of slow training of constrained Boltzmann machine in big data processing, a RBM accelerated computing implementation method based on Hadoop-GPU framework is designed on the basis of Hadoop cloud computing platform and GPU parallel acceleration. Based on the analysis of MapReduce mechanism and RBM training process, the RBM training is divided into two parts: the Gibbs sampling and reduce end is used to realize the parameter updating and the GPU is used to realize the method combination of parallel computation acceleration. Finally, the experiment of MNIST handwritten numeral recognition set shows that the Hadoop-GPU platform is feasible for RBM training under large scale data, and the speedup ratio is more than 20, and the speedup increases more significantly with the increase of data scale.
【作者单位】： 海军航空工程学院信息融合研究所;山西美佳矿业装备有限公司;
【基金】：国家自然科学基金(61032001)
【分类号】：TP332
[Abstract]:Aiming at the problem of slow training of constrained Boltzmann machine in big data processing, a RBM accelerated computing implementation method based on Hadoop-GPU framework is designed on the basis of Hadoop cloud computing platform and GPU parallel acceleration. Based on the analysis of MapReduce mechanism and RBM training process, the RBM training is divided into two parts: the Gibbs sampling and reduce end is used to realize the parameter updating and the GPU is used to realize the method combination of parallel computation acceleration. Finally, the experiment of MNIST handwritten numeral recognition set shows that the Hadoop-GPU platform is feasible for RBM training under large scale data, and the speedup ratio is more than 20, and the speedup increases more significantly with the increase of data scale.
【作者单位】： 海军航空工程学院信息融合研究所;山西美佳矿业装备有限公司;
【基金】：国家自然科学基金(61032001)
【分类号】：TP332
【参考文献】
相关期刊论文 前2条
1 江务学;张t，
本文编号：2139363
本文链接：https://www.wllwen.com/kejilunwen/jisuanjikexuelunwen/2139363.html
