基于分布式内存计算的深度学习方法

发布时间：2018-04-02 22:27

本文选题：人工智能　切入点：并行深度学习　出处：《吉林大学学报(工学版)》2015年03期

【摘要】：为了提升深度学习技术并行化学习效率,设计了一种面向计算机集群的分布式内存计算方法。构建分布式内存环境,建立数据分片处理和多任务调度机制,使模型参数和神经元节点的计算和存储并行运行于该环境中,避免了磁盘I/O对训练速率的影响;采用深度信念网络模型,以多个副本异步并行计算的方式进行模型训练,并使用dropout方法防止模型训练过拟合。对CIFAR-10图像数据集进行分类训练,试验结果表明,该方法可以明显提高深度神经网络的训练效率,并具备良好的可扩展性。
[Abstract]:In order to improve the parallel learning efficiency of deep learning technology, a distributed memory computing method for computer cluster is designed.The distributed memory environment is constructed, and the mechanism of data fragmentation and multitask scheduling is established, which makes the calculation and storage of model parameters and neuron nodes run in parallel in this environment, thus avoiding the influence of disk I / O on the training rate.The deep belief network model is used to train the model by asynchronous parallel computing with multiple replicas, and the dropout method is used to prevent the model from overfitting.The classification training of CIFAR-10 image data sets is carried out. The experimental results show that this method can obviously improve the training efficiency of the deep neural network and has good scalability.
【作者单位】：吉林大学仪器科学与电气工程学院;国家标准化管理委员会标准信息中心;
【基金】：科技部创新方法工作专项项目(2011IM010400)
【分类号】：TP333;TP18

【相似文献】