基于回溯式搜索算法的随机神经网络优化及应用

发布时间：2019-06-14 18:07

【摘要】：在神经网络的发展历史中,BP算法(Error Back Propagation, BP误差反向传播)一直作为神经网络权值优化的主流方法。但其收敛速度慢、易陷入局部极小值等缺点降低了神经网络的性能。随机神经网络采用单隐含层结构,隐含层(输入节点到隐含节点)参数随机产生,输出层(隐含节点到输出节点)参数通过计算得到。相比于采用BP算法的神经网络而言,随机神经网络的学习速度得到了上百倍的提高,同时也提高了网络模型的准确率和泛化能力。隐含层参数的随机性策略提升了网络性能,但这种机制导致隐含层需要过多的节点。这就造成了网络结构过于臃肿,降低了测试速度。许多学者对此进行研究,旨在简化网络结构,用进化算法优化随机神经网络的隐含层参数便是其中一种。进化算法是一种基于自然选择和生物遗传等生物进化机制而出现的一种启发式搜索算法。进化算法包括遗传算法[2],遗传编码,进化策略,进化编程四部分。进化算法拥有强大的全局搜索能力。因此,本文尝试通过回溯式搜索算法(进化算法的一种)优化随机神经网络的参数,以提高随机神经网络的效率、简化神经网络结构。回溯式搜索算法求解过程是一个贪婪的过程,在用回溯式搜索算法迭代优化随机神经网络时,会造成模型趋向于拟合验证集,但在测试集上的性能甚至可能出现下降的现象。因此,本文中提出了一种双项约束的损失函数,通过数据约束,很大程度上减轻了模型趋向于拟合验证集的问题。在对网络模型评价时,泛化能力是一个重要的指标。在本文中,提出了一种新的泛化能力评价准则,能更直观的表现出模型的泛化能力。许多疾病如糖尿病、青光眼等早期症状都表现在视网膜上。通过视网膜分析可以对这些疾病做早期的预防和治疗。视网膜血管分割是视网膜分析的基础。在血管分割中,血管的弯曲、分支等分割的是否准确,直接影响到视网膜分析的准确性。本文把回溯式搜索算法改进的随机神经网络模型用于视网膜血管分割,取得了令人满意的效果。在UCI数据集和视网膜血管分割数据集上,基于回溯式搜索算法改进的随机神经网络模型获得了令人满意的效果。本文对基于回溯式搜索算法优化随机神经网络的模型做了较广泛的探究,但仍存在一些问题以待进一步的实验验证和理论分析。
[Abstract]:In the development history of neural network, BP algorithm (Error Back Propagation, BP error back propagation has been used as the mainstream method of neural network weight optimization. However, its convergence speed is slow and it is easy to fall into local minima, which reduces the performance of neural network. The stochastic neural network adopts a single implicit layer structure, the parameters of the hidden layer (from the input node to the hidden node) are randomly generated, and the parameters of the output layer (from the hidden node to the output node) are obtained by calculation. Compared with the neural network with BP algorithm, the learning speed of stochastic neural network is improved by hundreds of times, and the accuracy and generalization ability of the network model are also improved. The randomness strategy of hidden layer parameters improves the network performance, but this mechanism leads to the need of too many nodes in the hidden layer. This causes the network structure to be too bloated and reduces the test speed. Many scholars have studied this in order to simplify the network structure, and using evolutionary algorithm to optimize the hidden layer parameters of stochastic neural networks is one of them. Evolutionary algorithm is a heuristic search algorithm based on natural selection and biological heredity and other biological evolution mechanisms. Evolutionary algorithm includes four parts: genetic algorithm [2], genetic coding, evolutionary strategy and evolutionary programming. Evolutionary algorithm has strong global search ability. Therefore, this paper attempts to optimize the parameters of stochastic neural network by retrospective search algorithm (one of evolutionary algorithm) in order to improve the efficiency of stochastic neural network and simplify the neural network structure. The solution process of traceability search algorithm is a greedy process. When the backtracking search algorithm is used to optimize the stochastic neural network iteratively, the model tends to fit the verification set, but the performance on the test set may even decline. Therefore, in this paper, a loss function with binomial constraints is proposed, which greatly reduces the problem that the model tends to fit the verification set through the data constraints. Generalization ability is an important index in the evaluation of network model. In this paper, a new evaluation criterion of generalization ability is proposed, which can show the generalization ability of the model more intuitively. Many diseases such as diabetes, glaucoma and other early symptoms are manifested in the retinal. Retinal analysis can be used for early prevention and treatment of these diseases. Retinal vascular segmentation is the basis of retinal analysis. In vascular segmentation, the accuracy of retinal analysis is directly affected by the accuracy of vascular bending and branch segmentation. In this paper, the improved stochastic neural network model of retrospective search algorithm is applied to retinal vascular segmentation, and satisfactory results are obtained. On UCI dataset and retinal vascular segmentation data set, the improved stochastic neural network model based on traceability search algorithm has achieved satisfactory results. In this paper, the model of optimizing stochastic neural network based on traceability search algorithm is widely explored, but there are still some problems to be further verified by experiments and theoretical analysis.
【学位授予单位】：山东大学
【学位级别】：硕士
【学位授予年份】：2016
【分类号】：TP183

【相似文献】