基于高斯过程混合的分类模型

发布时间：2018-06-23 02:47

本文选题：高斯过程 + 混合模型　；参考：《华东师范大学》2017年硕士论文

【摘要】：在贝叶斯非参数框架中,高斯过程是一种常用且功能强大的建模工具,被广泛应用于分类和回归问题中。为了提高高斯过程的数据建模能力和降低模型训练的计算复杂度,基于高斯过程混合的模型被相继提出。然而,这些已有的模型都是基于高斯过程回归模型,通常用来处理回归问题。在本文中,我们提出了一个新的基于无限高斯过程混合的分类模型(infinite mixtures of Gaussian processes based classification model,MGPC)。新的模型采用 logistic 函数作为似然,使得输出变量能够表示数据点属于正负类别的概率。因此,MGPC更适用于分类问题。混合权重服从狄利克雷过程(Dirichlet process,DP),模型具有建模无限多个混合成分的能力。MGPC采用完全贝叶斯的建模方式,而logistic似然函数以及DP的引入导致模型的后验分布难以求解。我们在变分推理的框架下,推导了所有潜变量的近似后验分布的更新算法。为了算法的可行性,我们采用线性高斯过程模型建模每个混合成分,它是高斯过程的一种等价的参数化表达,输出变量之间具有条件独立性。我们使用变分期望最大算法优化模型的超参数。在多个真实世界数据集上的实验结果显示,在与五种常用模型的比较中,MGPC取得了更好的分类性能。我们从多个角度出发分析实验结果,得到了不同于以往的结论。在单一高斯过程的分类和回归模型的比较中,两者性能没有明显差距。而MGPC的性能则显著地优于基于高斯过程混合的回归模型。最后,我们讨论了近似后验分布的截断水平对模型性能的影响,并通过实验验证了我们的分析。同时,实验结果也表明了 MGPC取得更好分类性能的潜力。
[Abstract]:In Bayesian nonparametric framework, Gao Si process is a common and powerful modeling tool, which is widely used in classification and regression problems. In order to improve the data modeling ability of Gao Si process and reduce the computational complexity of model training, the hybrid model based on Gao Si process has been proposed one after another. However, these existing models are based on Gao Si process regression models, which are usually used to deal with regression problems. In this paper, we propose a new classification model (infinite mixtures of Gaussian processes based classification model based on infinite Gao Si process. In the new model, the logistic function is used as the likelihood function, which enables the output variables to represent the probability that the data points belong to the positive and negative categories. Therefore, MGPC is more suitable for classification. The mixed weight is derived from the Dirichlet process DP. The model has the ability to model infinite mixed components. MGPC adopts a complete Bayesian modeling method. However, the introduction of logistic likelihood function and DP makes it difficult to solve the posteriori distribution of the model. In the framework of variational reasoning, we derive an algorithm for updating the approximate posterior distribution of all latent variables. For the feasibility of the algorithm, we use the linear Gao Si process model to model each mixed component, which is an equivalent parameterized expression of the Gao Si process, and the output variables are conditional independent. We use the variational expectation maximum algorithm to optimize the superparameters of the model. Experimental results on several real-world datasets show that MGPC achieves better classification performance compared with five commonly used models. We analyze the experimental results from many angles and get different conclusions. There is no significant difference between the classification and regression models of a single Gao Si process. The performance of MGPC is significantly better than that of mixed regression model based on Gao Si process. Finally, we discuss the effect of the truncation level of the approximate posterior distribution on the performance of the model, and verify our analysis by experiments. At the same time, the experimental results show the potential of MGPC to achieve better classification performance.
【学位授予单位】：华东师范大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP18

【相似文献】