以LDA为例的大规模分布式机器学习系统分析

发布时间：2018-01-09 20:12

本文关键词：以LDA为例的大规模分布式机器学习系统分析　出处：《计算机应用》2017年03期 　论文类型：期刊论文

【摘要】：针对构建大规模机器学习系统在可扩展性、算法收敛性能、运行效率等方面面临的问题,分析了大规模样本、模型和网络通信给机器学习系统带来的挑战和现有系统的应对方案。以隐含狄利克雷分布(LDA)模型为例,通过对比三款开源分布式LDA系统——Spark LDA、PLDA+和Light LDA,在系统资源消耗、算法收敛性能和可扩展性等方面的表现,分析各系统在设计、实现和性能上的差异。实验结果表明:面对小规模的样本集和模型,Light LDA与PLDA+的内存使用量约为Spark LDA的一半,系统收敛速度为Spark LDA的4至5倍;面对较大规模的样本集和模型,Light LDA的网络通信总量与系统收敛时间远小于PLDA+与Spark LDA,展现出良好的可扩展性。"数据并行+模型并行"的体系结构能有效应对大规模样本和模型的挑战;参数弱同步策略(SSP)、模型本地缓存机制和参数稀疏存储能有效降低网络开销,提升系统运行效率。
[Abstract]:Aiming at the problems in scalability, convergence performance and running efficiency of constructing a large-scale machine learning system, large scale samples are analyzed. The challenge to machine learning system caused by model and network communication and the solution of existing system. Take the implicit Delikley distributed LDA-model as an example. By comparing three open source distributed LDA systems, Spark LDA-PLDA and Light LDAs, the performance of system resource consumption, algorithm convergence performance and extensibility are compared. The differences in design, implementation and performance of each system are analyzed. The experimental results show that: facing the small-scale sample set and model. The memory usage of Light LDA and PLDA is about half of that of Spark LDA, and the convergence speed of the system is 4 to 5 times that of Spark LDA. The total amount of network communication and system convergence time of light LDA are much smaller than that of PLDA and Spark LDA. The architecture of "data parallel model parallelism" can effectively meet the challenges of large-scale samples and models. Parameter weak synchronization strategy (SSPN), model local cache mechanism and parameter sparse storage can effectively reduce network overhead and improve system efficiency.
【作者单位】：并行与分布处理国家重点实验室(国防科学技术大学);国防科学技术大学计算机学院;
【基金】：国家自然科学基金资助项目(61222205)~~
【分类号】：TP181
【正文快照】： 0引言在大数据时代,数据已经成为重要的资源。面对海量的数据,如何实现有效的处理和分析非常重要。机器学习理论旨在设计一些让计算机自动“学习”的算法,使得计算机能够从数据中自动获得规律,并利用规律对未知数据进行预测。常见的机器学习算法可分为有监督学习、无监督学习

【相似文献】