统计中文问句分类研究

发布时间：2018-03-12 11:43

本文选题：问答系统　切入点：中文问句分类　出处：《昆明理工大学》2012年硕士论文　论文类型：学位论文

【摘要】：问答系统(Question Answer System)能为人们提供自然语言的问句方式的提问,直接返回答案,而不是大量网页。相对于传统搜索引擎来说,问答系统能够更好地表达用户的需求,适应用户的习惯,回答信息也更准确,更快捷,更高效,其克服传统搜索引擎存在的缺陷,是当前研究的热点问题。问句分类是问答系统的一个重要组成部分,它能为问答系统的答案抽取环节提供答案的选取策略,所以分类的准确性直接影响问答系统的性能。本文对问句分类中特征选取与降维、问句属性核函数等方面开展了系列的研究和探讨。主要成果如下： 1、针对问句分类过程中词袋方式选取特征所面临的特征空间维数过高以及数据稀疏的问题,本文提出了一种结合词语相关性与流形学习的特征提取方法,该方法首先选取训练语料库中文档频率(DF)值高的词作为分类特征的属性维,其次以词汇语义相似度方法获取问句特征空间特征值,再次使用有监督局部线性嵌入算法对特征空间进行非线性降维,从而获得问句分类特征向量,最后使用支持向量机建立问句分类模型,在旅游领域7000多中文文句上的实验结果表明。本文所提方法能够有效解决特征空间维数过高与数据稀疏问题。 2、在使用支持向量机的标准核函数进行问句分类过程中,问句的内在结构常常被忽略。针对以上问题,本文提出了一种结合问句依存关系与词性的属性核函数方法,该方法首先提取问句中的词、词性、核心词依存关系、疑问词依存关系等特征,其次通过问句中的词的依存关系,词性以及共有的依存路径进行计算核函数的值,最后采用SMO算法优化求解。在旅游领域中文问句进行了不同核函数的中文问句分类对实验,结果表明提出的核函数能够有效利用问句内在依存结构,提高模型的训练速率以及分类准确率。 3、采用本文中所提的算法,分别设计并实现了结合流形学习的问句分类系统,基于问句属性核函数的问句分类系统。
[Abstract]:Question Answer system can provide people with questions in natural language and return answers directly instead of a lot of web pages. Compared with traditional search engines, question answering system can better express the needs of users. Adapting to the habits of users, answering information is more accurate, faster and more efficient. It overcomes the shortcomings of traditional search engines and is a hot issue in current research. Question classification is an important part of question answering system. It can provide the answer selection strategy for the question answering system, so the accuracy of the classification directly affects the performance of the question answering system. A series of researches and discussions have been carried out on the attribute kernel function of question sentence. The main results are as follows:. 1. In order to solve the problem of high dimension of feature space and sparse data, a feature extraction method combining word correlation and manifold learning is proposed in this paper. The method firstly selects the words with high document frequency and DFV value in the training corpus as the attribute dimension of the classification feature, and then obtains the feature space feature value of question sentence by the method of lexical semantic similarity. Thirdly, the linear embedding algorithm is used to reduce the dimension of the feature space, and then the feature vector of question sentence classification is obtained. Finally, the question sentence classification model is established by using support vector machine (SVM). The experimental results on more than 7000 Chinese sentences in the tourism field show that the proposed method can effectively solve the problem of high dimension of feature space and sparse data. 2. In the process of classifying question sentences with support vector machine (SVM) standard kernel function, the internal structure of question sentences is often ignored. In view of the above problems, a method of attribute kernel function combining question dependency and part of speech is proposed in this paper. The method firstly extracts the features of words, parts of speech, core words and interrogative words in question sentences, and then calculates the values of kernel functions through the dependency of words, parts of speech and common dependency paths in question sentences. Finally, the SMO algorithm is used to optimize the solution. The Chinese question classification experiments with different kernel functions are carried out in the tourism field. The results show that the proposed kernel function can effectively utilize the internal dependency structure of the question. The training rate and classification accuracy of the model are improved. 3. Using the algorithm proposed in this paper, the question sentence classification system combined with manifold learning and the question sentence classification system based on question attribute kernel function are designed and implemented respectively.
【学位授予单位】：昆明理工大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：H146.3

【相似文献】