基于C-MCMC和MapReduce的并行贝叶斯网络分类器研究

发布时间：2018-05-03 12:11

本文选题：贝叶斯网络 + 结构学习　；参考：《太原理工大学》2017年硕士论文

【摘要】：贝叶斯网络分类器是具有很强的学习和推理能力,是数据处理领域研究热点之一。虽然贝叶斯网络分类器表现出了良好的分类预测性能,但是仍存在先验只是利用率不高、实用性差而导致学习不能得到最优网络结构,从而影响了分类器的性能。如何更好的实现现有贝叶斯网络分类器的并行化仍然是亟待解决的问题之一。为了解决上述问题,本文开展了并行贝叶斯网络分类器相关的研究,设计并实现了新型的并行贝叶斯网络分类器,主要包括以下内容:(1)本文在马氏链蒙特卡洛算法(Markov Chain Monte Carlo,MCMC)的基础上引入存在、缺失和PD/CPD三种先验知识,提出了一种新的贝叶斯网络结构学习算法C-MCMC(Constrained-MCMC),运用以及先验知识对MCMC贝叶斯网络结构学习算法的影响,并通过一系列的实验验证了算法的有效性,从而学习得到更加优良的贝叶斯网络;(2)将C-MCMC贝叶斯网络结构学习算法应用在传统的增广朴素贝叶斯分类器(BAN)和通用贝叶斯网络分类器(GBN)中,并进行相应的参数估计,从而设计了C-MCMC BAN分类器和C-MCMC GBN分类器;借助开源平台Hadoop的并行编程模型MapReduce,设计了相应的Map函数与Reduce函数,对C-MCMC贝叶斯网络分类器使用MapReduce并行编程框架进行了并行化,给出了具体的编程实现过程,并通过搭建Hadoop平台验证了算法并行化对算法效率的改进和提高。实验结果表明,本文所设计的贝叶斯网络分类器的性能优于传统的贝叶斯网络分类器,有着较高的分类准确率和效率,且适用于大数据处理的场合,可以被应用于多个场合,具有广阔的市场应用前景。
[Abstract]:Bayesian network classifier has strong learning and reasoning ability, and it is one of the research hotspots in data processing field. Although Bayesian network classifier has shown good classification and prediction performance, there is still a priori only low utilization ratio and poor practicability, which leads to the failure of learning to obtain the optimal network structure, thus affecting the performance of classifier. How to better realize the parallelization of existing Bayesian network classifiers is still one of the problems to be solved. In order to solve the above problems, this paper develops the research of parallel Bayesian network classifier, designs and implements a new parallel Bayesian network classifier. The main contents are as follows: 1) this paper introduces three kinds of prior knowledge of existence, missing and PD/CPD on the basis of Markov Chain Monte Monte MCMCs of Markov chain Monte Carlo algorithm. In this paper, a new Bayesian network structure learning algorithm, C-MCMC- Constrained-MCMC-, is proposed. The effect of using and prior knowledge on the learning algorithm of MCMC Bayesian network structure is proved by a series of experiments, and the effectiveness of the algorithm is verified by a series of experiments. Thus, a better Bayesian network is obtained. The C-MCMC Bayesian network structure learning algorithm is applied to the traditional augmented naive Bayesian classifier (Ann) and the general Bayesian network classifier (GBN), and the corresponding parameters are estimated. In this paper, C-MCMC BAN classifier and C-MCMC GBN classifier are designed, the corresponding Map function and Reduce function are designed with the help of Hadoop parallel programming model of open source platform, and C-MCMC Bayesian network classifier is parallelized using MapReduce parallel programming framework. The implementation process of the algorithm is given, and the improvement and improvement of the algorithm efficiency are verified by building the Hadoop platform. The experimental results show that the proposed Bayesian network classifier is superior to the traditional Bayesian network classifier and has high classification accuracy and efficiency. It is suitable for big data processing and can be applied to many occasions. Has broad market application prospect.
【学位授予单位】：太原理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP18

【参考文献】