基于LSTM的汉语语义角色标注研究

发布时间：2018-01-21 17:15

本文关键词： 深度学习 LSTM 反向传播算法语义角色标注　出处：《西藏大学》2017年硕士论文　论文类型：学位论文

【摘要】：随着计算机网络技术和通信技术的迅速发展和普及,自然语言处理技术的应用需求急剧增加,人们迫切需要实用的自然语言处理技术来帮助打破语言屏障,为人际之间、人机之间的信息交流提供便捷、有效的人性化服务。中文信息处理作为自然语言处理中的一个分支,近些年来得到了快速发展,无论是在基础理论研究方面,还是在技术开发和产业化发展方面都取得了显著成绩。语义角色标注是浅层语义分析的一种实现方式,近几年颇受研究人员的关注。深度学习是一种让计算机自动进行特征学习的技术,随着该技术在图像识别、语音识别等领域取得了巨大成功,研究人员逐渐开始将这一技术应用在自然语言处理领域,成为目前该领域的一大研究热点。在当前多个深度学习模型中,基于长短期记忆单元(Long Short-term memory,LSTM)的递归神经网络(recurrent neural network,RNN)模型因为能有效利用序列数据中长距离的依赖信息,被认为特别适合文本序列数据的处理。因此,本文提出一种基于LSTM的汉语语义角色标注模型,该方法避免了复杂的特征提取和选择工作,摆脱了语义角色标注对句法分析的依赖,其最好标注结果的F值为70.34%。本文的主要工作如下:(1)确定了实验所用的语料和标记集:在中文宾州树库(Chinese Proposition Bank,CPB)标注语料的基础上,确定了本文使用的19类语义角色。结合模型特点,选择使用IOBES序列标注规则,并由此形成77个标签。实验采用OntoNote 5.0包含的文件chtb_0001.onf chtb_0399.onf按3:1的比例分别为训练和测试语料。(2)构建并训练了基于LSTM的语义角色标注模型:本文以词为基本标注单元,利用Word2Vec训练得到的词向量为输入,以LSTM标准单元为神经元构建网络层用于学习语义角色相关特征表达,并将得到的特征向量经过softmax函数计算和后处理后得到词对应的语义角色标签。采用反向传播算法进行模型训练,并对模型各参数进行实验分析。(3)采用LSTM模型训练得到词性向量并与词向量结合进行语义角色标注:首先构建LSTM网络层用于学习得到词性向量的表达,再将得到的词性向量与词向量结合,构建并训练LSTM网络层得到每个词相应的语义角色标签,最后对模型各参数进行实验,将其结果与前述模型进行对比分析。实验表明,词性信息有助于语义角色的识别和分类,且该模型可以有效地进行自动语义角色标注。尽管本文所构建的模型还不能与目前基于人工提取特征的最好结果相媲美,但已经取得了良好的效果,显示出了LSTM在语义角色标注任务中的强大能力。
[Abstract]:With the rapid development and popularization of computer network technology and communication technology, the application demand of natural language processing technology increases rapidly. People urgently need practical natural language processing technology to help break the language barrier. Chinese information processing, as a branch of natural language processing, has developed rapidly in recent years. Both in basic theory research and in the development of technology development and industrialization have made remarkable achievements. Semantic role annotation is a shallow semantic analysis of the way to achieve. In recent years, researchers have paid close attention to it. Deep learning is a technology that allows computers to learn features automatically. With the development of this technology, it has achieved great success in image recognition, speech recognition and other fields. Researchers have gradually begun to apply this technology in the field of natural language processing, which has become a major research hotspot in this field. Long Short-term memory based on long and short term memory unit. The recurrent neural Network (RNN) model of LSTM can effectively utilize the long distance dependency information in the sequence data. Therefore, this paper proposes a Chinese semantic role annotation model based on LSTM, which avoids complex feature extraction and selection. It gets rid of the dependence of semantic role annotation on syntactic parsing. The F value of the best tagged result is 70.34. The main work of this paper is as follows: (1) the corpus and marker set used in the experiment are determined. Chinese Proposition Bank. Based on the IOBES tagging corpus, 19 kinds of semantic roles used in this paper are determined. Combined with the characteristics of the model, we choose to use IOBES sequence annotation rules. From this, 77 tags were formed. The experiment used the file chtb_0001.onf chtb_0399.onf included in OntoNote 5.0 to press 3:. The proportion of 1 is training and testing corpus. 2) construct and train semantic role annotation model based on LSTM: this paper takes words as the basic annotation unit. The word vector trained by Word2Vec is used as input and LSTM standard unit is used as neuron to construct network layer for learning semantic role-related feature expression. The corresponding semantic role labels are obtained by softmax function calculation and post-processing, and the model is trained by back-propagation algorithm. The parameters of the model are analyzed experimentally. The part of speech vector is trained by LSTM model and combined with word vector for semantic role tagging. Firstly, the LSTM network layer is constructed for learning to get the expression of part of speech vector. Then the word vector and the word vector are combined to construct and train the LSTM network layer to get the corresponding semantic role label of each word. Finally, the model parameters are experimented. The experimental results show that the part of speech information is helpful to the recognition and classification of semantic roles. And this model can effectively automate semantic role annotation. Although the model constructed in this paper is not comparable to the best results based on artificial feature extraction, it has achieved good results. It shows the powerful ability of LSTM in semantic role tagging task.
【学位授予单位】：西藏大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP391.1

【相似文献】