基于注意力神经网络的蒙汉机器翻译系统的研究

发布时间：2019-05-28 02:36

【摘要】：近年来,深度学习成为众多领域研究的热点。对于自然语言处理领域中的机器翻译任务,序列到序列的神经网络翻译系统的出现打破了传统机器翻译多模块协调组合的局面,一体化的结构,令人满意的翻译结果使其一开始就备受学者瞩目。后来基于注意力(Attention-based)的神经网络进一步改进了模型,使得翻译效果一定程度上超过了传统的统计机器翻译系统,成为主流的翻译系统之一。本文以注意力神经网络为研究背景,结合最近的相关科研成果,从以下三方面展开了对基于注意力神经网络的蒙汉机器翻译系统的研究:(1)蒙古文词向量的预训练:词向量是直接参与模型训练的词语的表示形式,其训练的质量直接关系到最后训练的翻译模型的质量,因此我们探索了三种蒙古文词向量的预训练方法来提升翻译的质量;(2)基于字典的蒙古文词切分:蒙古文的构词特点会造成训练语料出现严重的数据稀疏问题,我们基于字典,对蒙古文单词进行了词缀、词干及格的附加成分等不同粒度的词切分,以此来缓解数据稀疏问题;(3)蒙古文特征提取:对于蒙古文来说,词干、词缀和格的附加成分是其构词的语言特色,我们同样基于字典将这些语言特色作为特征提取出来参与到系统的训练中,以此来提高神经网络翻译系统的翻译效果。最后,我们构建并改进了一个完整的基于注意力神经网络的蒙汉翻译系统。实验表明,基于我们的方法,该系统能比基线系统BLEU得分最多提高了 2.47个百分点,最优的模型BLEU值达到了 30.19。
[Abstract]:In recent years, in-depth learning has become a hot topic in many fields. For machine translation tasks in the field of natural language processing, the emergence of sequence-to-sequence neural network translation system breaks the situation of multi-module coordination and combination of traditional machine translation, and the integrated structure. Satisfactory translation results have attracted the attention of scholars from the beginning. Later, the neural network based on attention (Attention-based) further improved the model, which made the translation effect exceed the traditional statistical machine translation system to a certain extent and become one of the mainstream translation systems. In this paper, based on the research background of attention neural network, combined with the recent scientific research results, The research on Mongolian-Chinese machine translation system based on attention neural network is carried out from the following three aspects: (1) the pre-training of Mongolian word vector: word vector is the representation of words directly involved in model training. The quality of the training is directly related to the quality of the translation model of the final training, so we explore three pre-training methods of Mongolian word vectors to improve the quality of translation. (2) Mongolian word segmentation based on dictionary: the character of Mongolian word formation will cause serious data sparse problem in training corpus. Based on dictionary, we affix Mongolian words. In order to alleviate the problem of sparse data, word segmentation with different granularity, such as the additional components of word stem passing, can be used to alleviate the problem of sparse data. (3) Mongolian feature extraction: for Mongolian, the additional components of stem, affix and case are the language features of word formation. We also extract these language features as features to participate in the systematic training based on dictionaries. In order to improve the translation effect of neural network translation system. Finally, we construct and improve a complete Mongolian-Chinese translation system based on attention neural network. The experimental results show that, based on our method, the BLEU score of the system is up to 2.47 percentage points higher than that of the baseline system, and the optimal model BLEU value is 30.19%.
【学位授予单位】：内蒙古大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.2;TP183

【相似文献】