贝叶斯惩罚回归中正则化参数的选择

发布时间：2018-01-29 04:51

本文关键词： 贝叶斯惩罚回归正则化参数岭回归 lasso回归变量选择　出处：《西南交通大学》2017年硕士论文　论文类型：学位论文

【摘要】：惩罚回归在统计学中是一种至关重要的回归系数估计和变量选择方法,而正则化参数在惩罚回归中起着平衡损失函数和正则项的作用。因此,在惩罚回归的拟合过程中,选择一个合适的正则化参数显得极其重要。在深入学习贝叶斯理论知识、研究惩罚回归的基础上,从贝叶斯分析角度出发,通过寻找惩罚回归中各个组成部分与贝叶斯模型中先验函数、似然函数之间的对应关系,相应地,给出惩罚回归中所选择的正则化参数的贝叶斯估计表达式。具体地,文章在惩罚回归中的响应变量服从正态分布,回归参数服从指数族分布族中的某一个成员假设条件下,将惩罚回归中的损失函数表示成贝叶斯模型中的似然函数,正则项可表示成贝叶斯模型中的先验函数,通过贝叶斯公式,将损失函数和正则项以贝叶斯模型的形式结合起来,于是形成了一个关于回归系数的后验分布的贝叶斯模型。这个过程在找出惩罚回归和贝叶斯模型之间各个部分之间的对应关系的同时,相应地,得到惩罚回归中的正则化参数的贝叶斯估计表达式。将其推广到一般的惩罚回归(岭回归和lasso回归),得到它们具体的正则化参数估计表达式。从贝叶斯角度得到的正则化参数的估计表达式包含有响应变量和回归系数分布中参数,因此文章又重点探讨了关于分布中未知参数(位置参数和刻度参数)的贝叶斯估计值。最后,通过实例分析,在数据集满足一定的条件下,和现有的岭参数选择方法(广义交叉验证法、岭迹法)分析比较,文中所探讨的方法,和广义交叉验证法相比降低了计算的复杂度,和岭迹法相比较,在一定意义上给出了贝叶斯意义下的统计解释。
[Abstract]:Penalty regression is a very important method of regression coefficient estimation and variable selection in statistics, and regularization parameters play a role of balance loss function and regular term in penalty regression. In the fitting process of penalty regression, it is very important to choose a suitable regularization parameter. On the basis of studying the Bayesian theory and punishment regression, we proceed from the perspective of Bayesian analysis. By looking for the corresponding relation between each component of penalty regression and the prior function and likelihood function in Bayesian model, the corresponding relation is obtained. The Bayesian estimation expression of the regularized parameters selected in the penalty regression is given. Specifically, the response variables in the penalty regression are obeyed from the normal distribution. Under the assumption of a member of exponential family distribution, the loss function in penalty regression is expressed as the likelihood function in Bayesian model, and the regular term is expressed as a prior function in Bayesian model. By using Bayesian formula, the loss function and the regular term are combined in the form of Bayesian model. A Bayesian model of a posterior distribution of regression coefficients is formed, which finds out the corresponding relationship between penalty regression and each part of Bayesian model, and at the same time, the corresponding relationship between each part of the penalty regression and Bayesian model is found. The Bayesian estimation expression of regularization parameters in penalty regression is obtained and generalized to general penalty regression (Ridge regression and lasso regression). Their specific regularization parameter estimation expressions are obtained. The estimated expressions of regularization parameters obtained from Bayesian perspective contain parameters in the distribution of response variables and regression coefficients. Therefore, the Bayesian estimation of unknown parameters (position parameter and scale parameter) in the distribution is discussed. Finally, through an example analysis, the data set satisfies certain conditions. Compared with the existing ridge parameter selection methods (generalized cross validation method, ridge trace method), the method discussed in this paper reduces the computational complexity compared with the generalized cross validation method and is compared with the ridge trace method. The statistical explanation in Bayesian sense is given in a certain sense.
【学位授予单位】：西南交通大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：O212.8

【参考文献】