基于神经网络的自然语言语义表达及推理方法研究

发布时间：2018-03-03 11:06

本文选题：认知智能　切入点：自然语言理解　出处：《中国科学技术大学》2017年博士论文　论文类型：学位论文

【摘要】：认知智能旨在实现机器具备像人一样的理解、联想、推理等能力,是人工智能的重要领域。自然语言语义表达与常识推理是认知智能研究的关键任务。自然语言语义表达指的是将自然语言转换成机器可以处理的语义表征形式,是实现自然语言理解的基础。现阶段的语义表达方法主要基于统计分布假设,利用海量文本与统计建模,将自然语言中的语义信息表征为高维稀疏或低维稠密的向量形式。如何提高语义表征向量的精度仍然是当前语义表达研究的关键问题。常识推理关注机器对常识知识的运用以及推理能力。现阶段的常识推理方法仍以马尔科夫逻辑网络、贝叶斯网络等传统概率逻辑推理方法为代表。这些方法往往存在模型结构复杂、先验信息依赖性强、效率低、扩展性差等问题。本文围绕基于神经网络的自然语言语义表达及推理方法,在词语语义表达、面向常识推理的神经网络模型、常识知识库构建方法和自然语言推理系统等方面开展研究工作,具体包括:首先,研究了融合多源信息与神经网络建模的词语语义表达方法。现有词语语义表达方法单一依赖基于海量文本的统计分布假设,受文本噪声和歧义等影响,语义表达的精度不够理想。因此本文提出了融合海量文本与词汇语义知识的语义词向量构建方法和词性信息监督下的词性增强词向量方法,通过在神经网络训练过程中合理利用语义知识库、词性序列等多源信息,提高了单词语义表达精度,取得了多个自然语言理解任务上的性能提升。其次,研究了面向常识推理的神经网络建模方法。针对传统推理方法中存在的事件表达稀疏性与推广性问题,本文将连续语义空间表达引入常识推理,提出了神经联想模型。该模型将大量自然事件映射到连续语义空间中,利用深层人工神经网络实现对事件间联想关系的统一建模,最终完成基于事件联想的常识推理。在多个自然语言理解及推理任务上的实验结果表明,神经联想模型取得了优于现有模型的性能,并且具有良好的知识迁移学习能力。再次,研究了基于海量文本的常识知识库自动构建方法。针对常识知识库稀缺且人工构建代价高的问题,本文提出了基于海量文本的因果知识获取方法。该方法首先定义常用词语词典用于约束常识知识库的构建空间,然后在海量文本上进行核心句抽取与自动分析操作,最终得到大量具有因果关系的短语对作为常识知识库。基于以上方法,本文完成了包含五十余万条因果短语对的常识知识库的构建,为后续的自然语言推理系统构建提供了数据支撑。最后,设计实现了面向认知智能评测的自然语言推理系统。在上述语义表达、常识推理模型、常识知识库构建等研究工作的基础上,构建了面向Winograd Schema Challenge(WSC)评测任务的自然语言推理系统。针对常识推理子任务,设计实现了基于常识知识库和神经联想模型的因果推理系统,首次完成了 WSC因果子集上的自动常识推理;针对指代消解子任务,提出了基于知识增强语义模型的推理方法,采用语义词向量技术将常识知识融入词向量构建过程,实现了缺少任务相关训练数据情况下无监督的语义特征提取与推理,使用该方法构建的系统在2016年的WSC评测中取得了最优的性能表现。
[Abstract]:Cognitive intelligence aims to realize the machine with human like Lenovo, understanding, reasoning ability, is an important field of artificial intelligence. The natural language semantic expression and commonsense reasoning is the key task of cognitive intelligence research. Natural language semantic expression refers to the semantic representation of natural language processing can be converted into machine, is based on natural language understanding. At this stage the main semantic expression method based on the assumption of statistical distribution, and use the massive statistical modeling, the semantic information representation in natural language for high dimensional sparse or dense low dimensional vector form. How to improve the accuracy of semantic representation of vector semantic expression is still the key problems in the research. The use of common sense reasoning pay attention to machine to commonsense knowledge and reasoning ability. Knowledge reasoning method is using Markov logic network, traditional probabilistic network Bias The method of logical reasoning as a representative. These methods are complex model structure, a priori information dependence, low efficiency, poor scalability problems. This paper focuses on natural language semantic expression and reasoning method based on neural network, the expression in terms of semantics, a neural network model for commonsense reasoning, common sense knowledge base construction method and natural language the reasoning system and other aspects of the research work, including: first, the research on semantic integration of multi-source information and neural network modeling method. The expression of existing semantic expression method of statistical dependence on a single massive text distribution based on the assumption by the text noise and ambiguity, semantic expression accuracy is not ideal. So this paper presents a fusion mass the text semantic and lexical knowledge semantic vector construction method and part of speech information under the supervision of the part of speech enhancement method in word vector, by God The rational use of the semantic knowledge base of network training process, part of speech sequences of multi-source data, improve the accuracy of word semantic expression, made a number of natural language understanding and improve the performance of the task. Secondly, study the neural network modeling method for knowledge reasoning. In view of the existing traditional reasoning method in sparse representation and generalization of events in this problem, continuous semantic space expression into commonsense reasoning, proposes a neural associative model. This model will be a large number of natural events are mapped to continuous semantic space, realize the unified modeling association event between the use of deep artificial neural network, the final completion of the event. Based on commonsense reasoning Lenovo in a number of natural language understanding and reasoning the task of experimental results show that the neural associative model has made the performance of the existing model is superior, and has good learning ability of knowledge transfer again, Study on the automatic method to construct a common knowledge base based on the massive text. According to the common knowledge base of scarce and artificial construction costs, this paper proposes a method for massive text based on causal knowledge. This method firstly defines common word dictionary for construction of space constraints common knowledge base, and then in the massive text on core sentence extraction and automatic analysis of operation, finally obtained with a large number of causal phrases for commonsense knowledge base. Based on the above method, this paper completed the construction of the 50 million causal phrase common knowledge base contains, provides data support for the subsequent construction of natural language reasoning system. Finally, the design and implementation of natural language reasoning system for cognitive intelligence the evaluation. In the above expression semantics, general knowledge reasoning model, knowledge base construction of knowledge base and research work on the construction of the surface to Win Ograd Schema Challenge (WSC) natural language inference system evaluation task. For commonsense reasoning task, the design and implementation of causal reasoning system common knowledge base and neural associative model based on the completion of the first WSC for automatic commonsense reasoning on the set of fruit; to refer to the digestion of sub tasks, and puts forward a knowledge enhancement method of semantic reasoning based on the model, using the semantic vector technology will build into the process of word vector commonsense knowledge, the lack of semantic feature extraction and reasoning tasks related to unsupervised training data under the condition of the construction of the system, using the method of performance achieved optimal performance evaluation in WSC in 2016.

【学位授予单位】：中国科学技术大学
【学位级别】：博士
【学位授予年份】：2017
【分类号】：TP391.1;TP18

【相似文献】