基于循环神经网络的蒙古文语言模型研究

发布时间：2018-01-18 13:36

本文关键词：基于循环神经网络的蒙古文语言模型研究　出处：《内蒙古大学》2017年硕士论文　论文类型：学位论文

【摘要】：语言模型是自然语言处理任务中的重要组成部分。其中,N-gram语言模型是目前应用最为广泛的统计语言模型。近年来,随着深度学习技术的不断发展,深度神经网络模型逐渐被应用于语音识别中,它为研究者带来新一轮的研究热潮。神经网络语言模型是其中比较重要的研究方向之一。蒙古文语言模型对于蒙古语语音识别、蒙古文信息检索和蒙古文机器翻译等蒙古文信息处理技术的研究起着至关重要的作用。现阶段,神经网络语言模型已被广泛应用于英文和汉文中,但是神经网络语言模型在蒙古文中的使用还比较少。本文主要针对蒙古文神经网络语言模型进行研究。蒙古文是一种在国际上有广泛影响的语言文字。然而,在蒙古文文本语料中,存在大量的显现形式相同但编码不同的单词,这给蒙古文单词的统计和检索等带来了很大困难。本文着重解决显现形式相同但编码不同蒙古文单词的统计和检索问题,从而提高蒙古文语言模型的性能。首先,提出了采用中间字符对蒙古文显现形式相同但编码不同的字母进行合并表示的方法;接着,分别建立了基于拉丁字符的N-gram语言模型与基于中间字符的N-gram语言模型,以及基于拉丁字符的快速循环神经网络语言模型(Faster Recurrent Neural Network Language Model,FRNNLM)与基于中间字符的FRNNLM;然后,实现了 N-gram语言模型和FRNNLM融合的方法,得到了性能更好的语言模型;最后,用困惑度评价了蒙古文语言模型的性能,并将其应用到蒙古语语音识别中进行词错误率(Word ErrorRate,WER)的比较。实验结果表明,基于中间字符的蒙古文文本语料的词汇量比基于拉丁字符的语料平均减少了 41%;基于中间字符的语言模型(3-gram、FRNNLM)比相应基于拉丁字符的语言模型在困惑度方面下降了近40%,提高了蒙古文语言模型的性能。并且在蒙古语语音识别中,基于中间字符的语言模型(3-gram、FRNNLM、3-gram+FRNNLM)比相应基于拉丁字符的语言模型在WER方面下降了近20%;3-gram+FRNNLM(基于拉丁字符、基于中间字符)比3-gram、FRNNLM在WER方面下降得更加明显,有效提升了蒙古语语音识别的准确率。
[Abstract]:Language model is an important part of natural language processing. N-gram language model is the most widely used statistical language model. In recent years, with the development of in-depth learning technology. Depth neural network model is gradually used in speech recognition. It brings a new wave of research for researchers. Neural network language model is one of the more important research directions. Mongolian language model for Mongolian speech recognition. The research of Mongolian information processing technology such as Mongolian information retrieval and Mongolian machine translation plays an important role. At present, neural network language model has been widely used in English and Chinese. However, the use of neural network language model in Mongolian language is relatively small. This paper mainly focuses on the Mongolian neural network language model. Mongolian language is a kind of language which has a wide influence in the world. However. In the Mongolian text corpus, there are a large number of words with the same manifestation but different codes. This brings great difficulties to the statistics and retrieval of Mongolian words. This paper focuses on solving the problem of statistics and retrieval of Mongolian words with the same manifestation but different codes. In order to improve the performance of Mongolian language model. Firstly, the method of combining middle characters to represent Mongolian characters with the same form but different encoding is put forward. Then, the N-gram language model based on Latin character and N-gram language model based on intermediate character are established respectively. And the fast loop neural network language model based on Latin characters, Recurrent Neural Network Language Model. FRNNLM) and FRNNLM based on intermediate characters; Then, the N-gram language model and the FRNNLM fusion method are implemented, and a better performance language model is obtained. Finally, the performance of the Mongolian language model is evaluated with the degree of confusion, and the word ErrorRate is applied to the Mongolian speech recognition. The experimental results show that the vocabulary of Mongolian text corpus based on intermediate characters is 41% less than that based on Latin characters; The language model based on intermediate characters / FRNNLM) is nearly 40% less confusing than the corresponding language model based on Latin characters. The performance of Mongolian language model is improved, and in Mongolian speech recognition, the language model based on intermediate character is 3-gram/ FRNLM. 3-gram FRNNLM) is nearly 20% lower in WER than the corresponding Latin character-based language model; 3-gram FRNNLM (based on Latin characters, based on intermediate characters) is significantly lower in WER than 3-gram FRNNLM. It effectively improves the accuracy of Mongolian speech recognition.
【学位授予单位】：内蒙古大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.1;TP183

【参考文献】