越南语浅层句法分析方法的研究

发布时间：2018-01-01 02:08

本文关键词：越南语浅层句法分析方法的研究　出处：《昆明理工大学》2017年硕士论文　论文类型：学位论文

【摘要】：随着两国政治、经济和文化等方面日益频繁接触和深入合作,语言交流显得尤为重要。由于两国语言相差较大,造成了沟通上的障碍,进而成为两国发展的绊脚石;同时越南语自然语言处理在人工智能中起到核心作用,同样底层的浅层句法分析是自然语言处理的基础与前提,关系到后续工作的开展,且为上层应用服务。为了两国更好的发展,解决语言问题势在必行;针对上述问题,汉越机器翻译工作越来越重要。本文开展了越南语浅层句法分析的研究,主要完成了以下几个方面的研究工作:1.收集、整理和预处理越南语兼类词、实体和组块相关语料。语料是自然语言处理过程中基础性的课题,故构建语料库显得尤为重要,主要构建越南语兼类词、实体、实体库和组块等语料库,语料主要来源于已公开的少量语料和人工标记校对。2.提出了一种基于条件随机场的越南语兼类词方法。首先通过分析越南语兼类词特点进行分析,选取有效的兼类词消歧特征,制定相应的特征模版;其次使用条件随机场进行统计建模,得到基于条件随机场的越南语兼类词消歧模型。兼类词问题的解决有利于提高词性标注的准确率,提高词性语料库质量,尽可能防止错误向后累积传递,为越南语命名实体识别提供了基础和支撑。3.提出了一种融合实体特性的越南语命名实体识别的混合方法。首先根据越南语语言和实体特点进行分析,选取全局特征和局部特征作为本文的有效特征,构建基于最大熵模型的越南语实体识别模型;其次利用以上的特点进行制定越南语实体识别的规则集合;最后将最大熵模型和规则集合相结合进行实体识别。实体可以作为组块中有效的特征,同时有利于后续工作的开展。4.提出来了一种条件随机场和错误驱动学习的越南语组块分析方法。首先根据越南语组块和语言特点,选取基本特征和实体特征作为本文的有效特征,使用条件随机场统计方法进行建模,得到组块统计分析模型;其次利用转换学习方法进行获取候选转换规则集合,利用评价函数进行筛选,得到转换规则集合;最后将统计模型和转换规则相结合进行组块分析标记。组块作为实体识别的有效特征有利于实体识别正确率的提高。
[Abstract]:As the political, economic and cultural aspects of the two countries increasingly frequent contact and in-depth cooperation, language exchange is particularly important. Then become the stumbling block of the development of the two countries; At the same time, Vietnamese natural language processing plays a central role in artificial intelligence, the same low-level shallow syntax analysis is the basis and premise of natural language processing, related to the development of follow-up work. For the better development of the two countries, it is imperative to solve the language problem; In order to solve the above problems, Sino-Vietnamese machine translation is becoming more and more important. In this paper, the research on the shallow syntactic analysis of Vietnamese has been carried out, and the following research work has been completed: 1. Collating and preprocessing Vietnamese concomitant words, entities and chunks related corpus. Corpus is a basic subject in the process of natural language processing, so it is particularly important to construct corpus, mainly to construct Vietnamese concomitant words and entities. A corpus of entities and blocks. The corpus mainly comes from a few published corpus and manual marker proofreading. 2. A conditional random field based method of Vietnamese conjunctive words is proposed. Firstly, the characteristics of Vietnamese conjunctive words are analyzed. The effective disambiguation feature of the compound word is selected and the corresponding feature template is established. Secondly, the conditional random field is used for statistical modeling to obtain a conditional random field based Vietnamese word disambiguation model. The solution of concurrent word problem is helpful to improve the accuracy of part of speech tagging and improve the quality of part of speech corpus. As far as possible, prevent the accumulation of errors from being passed back. This paper provides the basis and support for Vietnamese named entity recognition. 3. A hybrid method of Vietnamese named entity recognition is proposed. Firstly, it is analyzed according to the characteristics of Vietnamese language and entity. The global feature and local feature are selected as the effective features in this paper, and a Vietnamese entity recognition model based on the maximum entropy model is constructed. Secondly, make use of the above characteristics to make the Vietnamese language entity recognition rules set; Finally, the maximum entropy model and the rule set are combined to identify the entity. The entity can be used as an effective feature in the block. At the same time, it is helpful to carry out the following work. 4. A conditional random field and error-driven learning method of Vietnamese language block analysis is proposed. Firstly, according to the Vietnamese language block and language characteristics. The basic features and entity features are selected as the effective features in this paper. The conditional random field statistical method is used to model the block statistical analysis model. Secondly, the candidate transformation rule set is obtained by using the transformation learning method, and the set of transformation rules is obtained by the selection of the evaluation function. Finally, the statistical model and the transformation rule are combined to carry out block analysis marking. As an effective feature of entity recognition, block is beneficial to improve the accuracy of entity recognition.
【学位授予单位】：昆明理工大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.1

【相似文献】