基于深度学习的答案选择

发布时间：2018-03-20 12:12

本文选题：答案选择　切入点：深度学习　出处：《哈尔滨工业大学》2017年硕士论文　论文类型：学位论文

【摘要】：答案选择是给定一个问题和该问题的候选答案列表,根据问题和答案的相关度对候选列表中的答案进行重新排序。在答案选择任务中,大多数问题和答案之间单词的重合率和相似度并不高,且语义也不相似,很难使用单词或文本相似来解决,这给传统的特征工程方法带来了一些困难。近年来,深度学习已经在很多自然语言处理领域取得了不错的成绩,如文本蕴含、机器翻译和问答系统等。利用深度学习技术处理答案选择任务,不是单纯的提取单词或单词组合特征,而是从语义层面上对句子进行理解,得到问题和答案的相关度信息。在本文中,将句子单词向量化后,利用深度学习技术建立句子编码模型获取问题和答案的句子向量,后使用相关度计算方法计算问题和答案相关度,对候选答案进行排序。本文的主要研究方向从以下三个方向进行:⑴基于卷积神经网络的答案选择模型。实现了基于卷积神经网络的答案选择模型,使用卷积神经网络对问题和答案进行编码,提取句子中的语义特征,最终通过相关性矩阵计算得到问题向量和答案向量的相关度。⑵基于长短期记忆网络和注意力机制的答案选择模型。循环神经网络擅长处理序列信息,可以存储历史信息,并且可以捕捉到单词位置不同带来的语义变化。本文实现了一种基于长短期记忆网络和注意力机制的句子编码模型,并提出了一种自动学习非文本特征的方法,与基于Attention-LSTM的答案选择模型相结合后,比Attention-LSTM模型效果有所提高。⑶基于双向长短期记忆网络和自动编码器的答案选择模型。Bi LSTM模型相对于LSTM模型,在对句子编码时同时考虑上下文信息,最终得到的句子编码更加完整。本文实现了基于Bi LSTM的答案选择模型,后期为了使句子编码模型参数训练的更加充分,实现了一种基于Seq2Seq的自动编码模型,对基于Bi LSTM的句子编码模型参数进行预训练。实验结果表明,经过预训练的基于Bi LSTM的答案选择模型性能更优,与非文本特征结合后,整体模型在测试集上的结果已经高于了基线系统的最优结果。
[Abstract]:The answer selection is to reorder the answers in the candidate list based on the relevance of the question and the answer given a question and a list of candidates for that question. The coincidence rate and similarity between most questions and answers are not high, and the semantics are not similar, so it is difficult to solve the problem by using the word or text similarity, which brings some difficulties to the traditional feature engineering methods in recent years. Deep learning has achieved good results in many fields of natural language processing, such as text implication, machine translation and question answering system. In this paper, after the sentence words are vectorized, the sentence encoding model is established to obtain the sentence vector of the question and the answer. Then using the correlation calculation method to calculate the correlation between the question and the answer, The main research direction of this paper is to carry out the answer selection model based on convolution neural network in the following three directions: 1. The answer selection model based on convolutional neural network is realized. Using convolutional neural network to encode the questions and answers, the semantic features of sentences are extracted. Finally, the correlation between the question vector and the answer vector is obtained by calculating the correlation matrix. 2. The model of choice of answer is based on the long-term and short-term memory network and attention mechanism. The cyclic neural network is good at processing sequence information and can store historical information. In this paper, a sentence coding model based on short and long term memory network and attention mechanism is implemented, and an automatic learning method of non-text features is proposed. Combined with the answer selection model based on Attention-LSTM, the effect of Attention-LSTM model is better than that of Attention-LSTM model. 3. The answer selection model. Bi LSTM model based on bidirectional long and short memory network and automatic encoder is relative to LSTM model. In this paper, the answer selection model based on Bi LSTM is implemented, in order to train the parameters of the sentence coding model more fully. An automatic coding model based on Seq2Seq is implemented, and the parameters of sentence coding model based on Bi LSTM are pre-trained. The experimental results show that the pre-trained answer selection model based on Bi LSTM has better performance and is combined with non-text features. The results of the whole model on the test set are higher than the optimal results of the baseline system.
【学位授予单位】：哈尔滨工业大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP18

【参考文献】