基于汉越双语平行语料库的词对齐方法研究

发布时间：2019-02-23 20:41

【摘要】：近年来,机器翻译正在逐渐成为克服人们之间进行交流时所面临的语言障碍的重要手段。双语词对齐研究是自动获取翻译知识的基础环节,尤其在机器翻译领域,经过词语对齐的语料是极具有价值的翻译知识源。它为后期汉越词典编制、机器翻译、语音识别、信息检索、语义消歧以及双语句子对齐系统等自然语言处理领域研究提供重要支撑,这使得人们越来越意识到获取双语词对齐语料的重要性。研究如何在前人基础上提高汉越双语词语对齐质量,构建大规模的汉-越双语词对齐语料库在学术上具有一定的研究价值。目前,针对汉-英,法-英等大语种的双语词对齐都取得了很好的效果,但针对汉语与越南语之间的词对齐研究还很少见。本文深入探究影响汉语-越南语双语词语对齐质量的原因并分析对齐过程中存在的问题,同时在结合越南语的语言特征以及现有研究工作的基础上,主要完成以下特色研究工作:(1)提出基于组块的汉-越双语词对齐方法。为提高汉-越双语词对齐准确率以及缓解汉-越双语词对齐过程中存在的非对称问题,构建了一定规模的汉越双语组块对齐语料库,在组块对齐语料的基础上,结合汉越双语的语言特点,利用CRFs模型实现组块内部的词对齐。(2)提出融合语义信息的汉越双语词语对齐算法。由于对齐过程存在低频词对齐错误率高的问题,考虑构建词汇相似性模型。在单语的语料库中利用神经网络模型训练出词语相似性模型,利用词语的相似性模型来扩展IBM词对齐模型,最后用融合词汇相似性模型的GIZA++实现汉语与越南语间词汇的对齐。(3)基于集成学习的思想,提出结合语义信息、word2vec词对齐模型以及基于组块的三个词对齐模型,把它们看作独立的对齐分类器,利用简单投票和加权投票的策略对多个词对齐模型进行融合,以进一步改善词对齐的质量,实现对三个不同的词对齐方法进行评估研究。
[Abstract]:In recent years, machine translation is becoming an important means to overcome the language barriers that people face in communication. The study of double word alignment is the basic link of automatic acquisition of translation knowledge, especially in the field of machine translation, word alignment is a valuable source of translation knowledge. It provides important support for the research of natural language processing such as Chinese-Vietnamese dictionary compilation, machine translation, speech recognition, information retrieval, semantic disambiguation and bilingual sentence alignment system. This makes people more and more aware of the importance of acquiring bilingual word alignment data. The research on how to improve the quality of Chinese-Vietnamese bilingual word alignment on the basis of predecessors and to construct a large-scale Chinese-Vietnamese bilingual word alignment corpus has certain academic value. At present, Chinese-English, French-English and other major languages have achieved good results in word alignment, but word alignment between Chinese and Vietnamese is rare. This paper probes into the reasons that affect the quality of Chinese-Vietnamese bilingual word alignment and analyzes the problems existing in the alignment process. At the same time, on the basis of combining the linguistic characteristics of the Vietnamese language and the existing research work, The main works are as follows: (1) A Chinese-Vietnamese bilingual word alignment method based on chunks is proposed. In order to improve the accuracy of Chinese-Vietnamese bilingual word alignment and to alleviate the asymmetric problem in the process of Chinese-Vietnamese bilingual word alignment, a Chinese-Vietnamese bilingual block alignment corpus is constructed, which is based on the block alignment corpus. According to the characteristics of Chinese and Vietnamese bilingualism, CRFs model is used to realize word alignment within blocks. (2) A Chinese-Vietnamese bilingual word alignment algorithm is proposed, which combines semantic information. Due to the problem of high error rate of low frequency word alignment in the alignment process, a lexical similarity model is proposed. In the monolingual corpus, we use neural network model to train word similarity model, and extend IBM word alignment model by word similarity model. Finally, the lexical alignment between Chinese and Vietnamese is realized by using GIZA which combines lexical similarity model. (3) combining semantic information, word2vec word alignment model and three word alignment models based on chunks are proposed based on the idea of integrated learning. They are regarded as independent alignment classifiers, and the strategies of simple voting and weighted voting are used to fuse multiple word alignment models to further improve the quality of word alignment and to evaluate and study three different word alignment methods.
【学位授予单位】：昆明理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.1

【参考文献】