哈萨克语基本动词短语自动识别研究

发布时间：2018-03-24 13:33

本文选题：哈萨克语基本动词短语　切入点：短语分析　出处：《新疆大学》2013年硕士论文

【摘要】：哈萨克语自然语言信息处理技术在文字处理、词法分析、文本校对等阶段均取得了一定的成果，目前可以考虑句子的处理阶段,即如何自动分析短语结构、短语定界、短语内部句法关系、结构成分之间的语义关系的不同等。面对如此丰富的网上信息,越来越多的人们需要对自然语言进行深入分析,例如机器翻译、搜索引擎、文本分类、信息提取等方面。本研究首先明确提出了哈萨克语基本动词短语的定义、性质、分类、结构，并且确立了哈萨克语基本动词短语的句法功能分类框架，初步描述现代哈萨克语短语结构所需要的句法体系和比较完整短语功能分类体系。其二，对哈萨克语基本动词短语的结构进行了统计与分析；接着确定哈萨克语基本动词短语的定界确定规则，最终识别哈萨克语基本动词短语。基于规则的方法中存在一些没有被考虑到的规则，而且基本动词短语跟其他短语之间存在一些歧义，所以得到的准确率并不高。其三，使用基于最大熵的方法对哈萨克语基本动词短语识别，其利用哈萨克语的单词、词性、词缀等上下文信息来设计最大熵模型的特征模板，通过GIS算法来对特征集合进行参数估计，最终输出最优的动词短语识别结果。基于统计的方法在在封闭测试环境下可以得到较高的准确率，，在开放测试环境下却无法得到很好的结果，这种方法要求的训练语料库规模较大。其四，细致分析了哈萨克语基本动词短语结构歧义类型与消除策略分析，对哈萨克语基本动词短语结构中无歧义格式使用规则的方法识别方法基础上，针对一些典型的歧义格式使用统计的方法。本系统对实验室现有的“新疆日报”语料中抽取30天语料（规模为20MB）中进行哈萨克语基本动词短语（KzBaseVP）识别。从实验结果可以得出：以上三种方法对哈萨克语基本动词短语识别是可行的，并且基本动词短语搭配规则和特征模板的选择是正确，在封闭和开发测试条件下可以得到令人满意的效果。
[Abstract]:Kazakh natural language information processing technology has achieved some results in the stages of word processing, lexical analysis, text proofreading, etc. At present, we can consider the processing stage of sentences, that is, how to analyze phrase structure automatically, and how to delimit phrase. In the face of so much information on the Internet, more and more people need to conduct in-depth analysis of natural language, such as machine translation, search engine, text classification. Information extraction and other aspects. In this study, the definition, nature, classification and structure of basic verb phrases in Kazakh language are proposed, and the syntactic functional classification framework of basic verb phrases in Kazakh language is established. A preliminary description of the syntactic system and the relatively complete functional classification system of the phrase structure of the Kazakh language is given. Secondly, the structure of the basic verb phrase in the Kazakh language is statistically analyzed. Then the basic verb phrases of Kazakh language are determined and the basic verb phrases are identified. There are some rules that have not been considered in the rule-based approach. Moreover, there are some ambiguities between the basic verb phrases and other phrases, so the accuracy is not high. Thirdly, the basic verb phrases in Kazakh are recognized by the method based on maximum entropy, which uses the Kazakh words and parts of speech. The feature template of the maximum entropy model is designed with the context information such as affix, and the parameters of the feature set are estimated by GIS algorithm. The method based on statistics can get high accuracy in closed test environment, but it can not get good result in open test environment. The training corpus required by this method is large. Fourthly, the paper analyzes the types of structural ambiguity of basic verb phrases in Kazakh language and the strategies of eliminating them. On the basis of the method of recognizing the rules of the use of the unambiguous format in the basic verb phrase structure of Kazakh, the statistical method is used for some typical ambiguous forms. In this system, the basic verb phrase KzBaseVPP of Kazakh language is identified from 30 days' data (20 MBs) extracted from the existing data of Xinjiang Daily in our laboratory. From the experimental results, it can be concluded that the above three methods can be used to identify Kazakh bases. This verb phrase recognition is feasible, Moreover, the selection of basic verb phrase collocation rules and feature templates is correct, and satisfactory results can be obtained under closed and developed test conditions.
【学位授予单位】：新疆大学
【学位级别】：硕士
【学位授予年份】：2013
【分类号】：TP391.1

【参考文献】