广义熵的Bayes估计方法

发布时间：2018-04-03 05:16

本文选题：广义熵　切入点：Bayes估计　出处：《华中科技大学》2015年硕士论文

【摘要】：1948年,C.E.Shannon将热力学系统中熵的概念引入到信息论中,标志着现代信息论的诞生。随后,Shannon熵的估计被应用在工程学、生物医药学和生物化学等方面。学者们为了解决生活中的实际问题,又提出了其他不同形式的广义熵,如Tsallis熵和Renyi熵。在统计分析中,广义熵的应用研究越来越受到学者们的关注。本文主要讨论离散型随机变量的广义熵,将熵的估计由Shannon熵推广到Tsallis熵和Renyi熵。在Shannon熵极大似然估计方法的基础上,得到Shannon熵的Bayes估计方法,并将其与极大似然估计方法进行比较。进一步,本文推导出广义熵中具有代表性的Renyi熵和Tsallis熵的Bayes估计方法。运用Matlab软件,以概率(0.1,0.4,0.5)产生样本容量为N的随机数。用每个数的频率代替概率,得到传统熵估计的极大似然估计方法。将每个数出现的相应次数,代入到Bayes估计式中。通过Matlab运算出样本容量为N的Bayes估计值。改变N的大小,经过多次试验,得到一组极大似然估计值与Bayes估计值,并将两种方法进行比较。由误差平方和可以看出Shannon熵的Bayes估计效果更好一些,然而Tsallis熵的Bayes估计并没有显现出更好的效果。可以考虑改变q的值以及进行大量的随机试验,再将Tsallis熵的Bayes估计方法与极大似然估计方法进行对比。虽然广义熵的Bayes估计方法并不一定是最优的,但从广义熵的应用角度来看,仍具有一定的实际意义。最后,本文结合次序统计量,进一步讨论了连续型随机变量的Bayes估计方法,推导出连续型随机变量Shannon熵和Renyi熵的Bayes估计量。
[Abstract]:In 1948, C.E. Shannon introduced the concept of entropy into information theory, which marked the birth of modern information theory.Then Shannon entropy estimation was applied in engineering, biomedical and biochemistry.In order to solve the practical problems in life, scholars put forward other forms of generalized entropy, such as Tsallis entropy and Renyi entropy.In statistical analysis, more and more scholars pay attention to the application of generalized entropy.In this paper, the generalized entropy of discrete random variables is discussed. The estimation of entropy is extended from Shannon entropy to Tsallis entropy and Renyi entropy.Based on the Shannon entropy maximum likelihood estimation method, the Bayes estimation method of Shannon entropy is obtained and compared with the maximum likelihood estimation method.Furthermore, the Bayes estimation method for the representative Renyi entropy and Tsallis entropy in generalized entropy is derived.By using Matlab software, the random number with sample size of N is generated by probability 0. 1 / 0. 4 ~ 0. 5).The maximum likelihood estimation method of traditional entropy estimation is obtained by replacing the probability with the frequency of each number.The corresponding number of occurrences of each number is substituted into the Bayes estimator.The Bayes estimate of sample size N is calculated by Matlab.By changing the size of N, a set of maximum likelihood estimators and Bayes estimators are obtained through many experiments, and the two methods are compared.It can be seen from the sum of squared errors that the Bayes estimation of Shannon entropy is better, but the Bayes estimation of Tsallis entropy does not show better effect.We can consider changing the value of Q and carrying out a lot of random experiments, and then compare the Bayes estimation method of Tsallis entropy with the maximum likelihood estimation method.Although the Bayes estimation method of generalized entropy is not necessarily optimal, it still has some practical significance from the point of view of the application of generalized entropy.Finally, the Bayes estimation method for continuous random variables is discussed with order statistics, and the Bayes estimators of Shannon entropy and Renyi entropy of continuous random variables are derived.
【学位授予单位】：华中科技大学
【学位级别】：硕士
【学位授予年份】：2015
【分类号】：O212.8

【相似文献】