基于深度联想记忆网络和多维权值的多模态识别模型的研究

发布时间：2018-01-26 18:35

本文关键词： 深信度网络特征提取联想记忆多模态识别　出处：《吉林大学》2017年硕士论文　论文类型：学位论文

【摘要】：现实世界中的信息往往同时以多种不同的形式出现,认知科学的研究表明生物在认识世界的过程中,是结合多个感官获得的信息做出综合判断来获取一个总体的认识,源于不同感官的信息对最终的认知结果来说是相辅相成,相互配合的。联想记忆是生物大脑中的一个重要功能,对一个新事物的认知,会通过感官将多种模态信息获取后,在脑神经细胞形成记忆存储,下次再遇到该事物的某些信息时即能在脑中联想出相应的其他信息。随着对大脑神经系统研究的不断深入的,人们意识到大脑对新事物的学习是一个不断提取抽象概念的过程,是将信息从原始状态抽象到一种更高层的表示,然后将多种高层抽象表示联系到一起形成联想记忆。文中为了模拟生物利用多种模态信息进行学习并形成联想记忆的能力,基于深信度网络、双向联想记忆网络建立了深度联想记忆网络以及在该网络的基础上的多维权值的多模态识别模型。其中深度联想记忆网络能够利用深信度网络提取出各个模态信息的特征,然后利用双向联想记忆网络建立多个特征间的联系,这种联系的建立加上深信度网络本身的双向生成功能使得模型能够完成缺失信息的寻回任务,同时基于该深度联想记忆网络进行的多维权值的多模态识别,不仅在分类准确性上比以往的模型有较大提高,而且在有信息缺失的情况下仍能够较好进行多模态分类。本文在实验中提供图片信号与语音信号这两种模态的信息,让网络进行多模态学习。实验结果验证了深度联想记忆网络重新生成缺失信号的能力;通过跟多种传统模型进行比较,我们验证了基于多维权值的多模态识别模型的有效性,以及由于权值的多维性而非单一性使得对不同类别进行识别具有较强的稳定性。
[Abstract]:The information in the real world often in many different forms, cognitive science research shows that biological understanding of the world in the process, is a combination of multiple sensory information available to make a comprehensive judgment to get a general understanding from different sensory information on the final result of cognitive are complementary with each other,. Associative memory is an important biological function in the brain, a new cognition, through a variety of sensory information acquisition mode, the formation of memory in brain nerve cells, the next time you have some information of the things which can associate other relevant information in the brain. With the study of the brain the deepening of the nervous system, people are aware of the brain to learn new things is a continuous process of abstracting the concept, is the information from the original state to a higher level of abstraction table Then, a variety of high-level abstraction together to form associative memory. In this paper, in order to simulate the biological information using a variety of modal learning and ability to form associative memory, that network based on multi modal identification model of multi dimension value of the bidirectional associative memory network is established and the depth of associative memory neural network based on the network. In depth. Associative memory network can utilize network to extract the features of each convinced modal information, and then establish multiple features between bidirectional associative memory network with bidirectional generation function to establish contact with the deep belief network itself makes the model to complete the missing information retrieve task, and multimodal recognition the rights the depth of associative memory network based on value, not only in classification accuracy than the previous model is greatly improved, and in a letter Still can carry out multimodal classification information. The absence of the picture signal and sound signal of the two modes in the experiment information, let the network for multimodal learning. The experimental results verify the ability to re generate the missing signal depth of associative memory networks; through with a variety of traditional model comparison, we verified multi modal identification model based on the value of human rights, as well as multidimensional weights rather than single sex makes recognition has the strong stability of different categories.

【学位授予单位】：吉林大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：R318;TP18

【相似文献】