基于强化学习的开放领域聊天机器人对话生成算法

发布时间：2018-04-26 11:07

本文选题：聊天机器人 + 对话生成　；参考：《哈尔滨工业大学》2017年硕士论文

【摘要】：近年来,大规模深度学习算法的发展使得聊天机器人不再仅仅依靠模板匹配与检索等方式,生成式对话算法逐步成为可能。与检索式聊天机器人相比,生成式的聊天机器人的迁移性与泛化性更佳,拥有着更广阔的应用场景。本文主要关注开放领域的聊天机器人,区别于特定任务的聊天机器人,开放领域下的聊天机器人其目标不是为了尽快完成用户指定的任务,而是吸引并留住用户令其有兴趣与机器人继续交谈。本文首先对生成式聊天机器人领域主流的基于RNN的Encoder-Decoder算法模型进行实验,同时对比了LSTM和GRU等不同的神经网络单元对结果的影响,随后加入attention注意力模型机制提高了生成效果,而后又采用了beam search方法提升生成效果的多样性。由于目前seq2seq的生成式算法在输出时采用最大似然估计方法,极易产生大量无实际意义的安全回复。为解决这个问题,本文采用强化学习的方法评估对话的未来奖励,不再是针对当前输入选择对应的最大可能回复,而是评估其对未来对话的奖励,以促进聊天持续发展。同时本文认为一个更为智能的机器人应具有情感色彩,故本文在强化学习的奖励函数上,除对生成的语句流畅性等方面进行检测并将其作为奖励外,还加入了对候选句的情感检测,以期令聊天机器人尽可能选择对未来对话情感指数有提升的候选句,有效地延续对话。本文还将情感信息作为监督信号加入到生成过程中,从语料中学习聊天过程的情感转移分布,并将该分布作为监督信号指导聊天机器人回复,使其产生蕴含情感的语句。最后,通过自动评价结合人工评价的方式,本文就上述几种方法进行了对比,验证了所提出模型方法的有效性。
[Abstract]:In recent years, with the development of large-scale in-depth learning algorithms, chat robots no longer rely solely on template matching and retrieval, so it is gradually possible to generate dialogue algorithms. Compared with the retrieval chat robot, the generative chat robot has better mobility and generalization, and has a wider application scenario. This paper focuses on the open domain chat robot, which is different from the specific task chat robot. The goal of the open domain chat robot is not to complete the user assigned task as soon as possible. Instead, attract and retain users to be interested in continuing to talk to the robot. In this paper, the main Encoder-Decoder algorithm model based on RNN in the field of generative chat robot is experimented at first. At the same time, the influence of different neural network units, such as LSTM and GRU, on the result is compared, and then the attention model mechanism is added to improve the effect of the algorithm. Then the beam search method is used to improve the diversity of the generation effect. Because the seq2seq generation algorithm adopts the maximum likelihood estimation method in output, it is easy to produce a large number of meaningless security responses. In order to solve this problem, this paper uses the reinforcement learning method to evaluate the future reward of the dialogue, instead of selecting the maximum possible reply corresponding to the current input, but to evaluate the reward for the future dialogue so as to promote the sustainable development of chat. At the same time, this paper holds that a more intelligent robot should have emotional color, so in the reward function of reinforcement learning, besides detecting the fluency of the generated sentence and taking it as a reward, In order to make the chat robot choose the candidate sentence which can improve the emotion index of the future dialogue as much as possible, it can effectively continue the dialogue. In this paper, emotional information is added to the generation process as a supervisory signal to learn the emotional transfer distribution of the chat process from the corpus, and the distribution is used as a supervisory signal to guide the chat robot to reply, so that it can produce emotional sentences. Finally, through the way of automatic evaluation combined with manual evaluation, this paper compares the methods mentioned above, and verifies the validity of the proposed model method.
【学位授予单位】：哈尔滨工业大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TP242

【相似文献】