语音驱动虚拟说话人研究

发布时间：2018-09-05 18:22

【摘要】：语音驱动虚拟说话人技术指的是通过输入语音信息生成虚拟人面部动画。不仅提高用户对语音的理解度,而且提供一种真实、友好的人机交互方式。随着该技术的发展进步,势必为我们带来更多新的人机交互体验,极大丰富我们的日常生活。本论文采用两种方案研究语音驱动虚拟说话人动画合成,并对其进行分析对比。第一种方案,基于深度神经网络的语音驱动发音器官运动合成。第二种方案,基于MPEG-4的语音驱动虚拟说话人动画合成。这两种方案均需要找到相应的语料库,并对其进行提取构建出适合本论文研究问题的声视觉数据。第一种方案:语音的产生与声道发音器官的运动直接相关,如唇部、舌头和软腭的位置与移动。通过深度神经网络学习语音特征参数与发音器官位置信息两者之间的映射关系,系统根据输入的语音数据估计出发音器官的运动轨迹,并将其体现在一个三维的虚拟人上面。首先,在一系列参数下对比传统神经网络(Artificial Neural Network,ANN)和深度神经网络(Deep Neural Network,DNN)的实验结果,得到最优网络;其次,设置不同上下文语音特征参数长度并调整隐藏层单元数,获取最佳的上下文长度;最后,选取最优网络结构,由最优网络输出的发音器官运动轨迹信息控制发音器官运动合成,实现虚拟人动画合成。第二种方案:基于MPEG-4的语音驱动虚拟说话人动画合成的方法是一种数据驱动方法。首先,本论文从LIPS2008数据库中提取构建出适合本论文的声视觉语料库。然后,使用BP(Back Propagation)神经网络的方法学习语音特征参数与虚拟人人脸动画参数(Facial Animation Parameters,FAP)两者之间的映射关系。最后,系统根据预测得到的FAP序列控制虚拟人面部模型合成虚拟人口型动画。本论文分别对两种方案合成的动画进行主客观评价,均证明两种方案的有效性,并且动画效果自然逼真。对比两种动画合成方案,第一种方案需要一个与之相适应的唇部模型,虽然其精准度较高,但通用性不强,且其语料库不易获得。第二种方案符合MPEG-4标准,使用FAP序列驱动的虚拟人面部模型合成动画,其通用性更强,更便于广泛应用。
[Abstract]:Speech driven virtual speaker technology refers to the generation of virtual human facial animation by input of speech information. It not only improves the user's understanding of speech, but also provides a real and friendly way of human-computer interaction. With the development of the technology, it will bring us more new human-computer interaction experience and enrich our daily life. In this paper, two schemes are used to study speech driven virtual speaker animation synthesis, and to analyze and compare them. The first scheme is speech driven speech motion synthesis based on deep neural network. The second scheme is speech driven virtual speaker animation synthesis based on MPEG-4. Both schemes need to find the corresponding corpus and construct sound vision data suitable for the research of this paper. The first scheme: speech production is directly related to the movement of vocal organs, such as the lip, tongue and soft palate position and movement. The mapping relationship between speech feature parameters and speech organ location information is studied by deep neural network. The system estimates the movement track of the speech organ according to the input speech data and embodies it on a 3D virtual human. First, the optimal network is obtained by comparing the experimental results of traditional neural network (Artificial Neural Network,ANN) and depth neural network (Deep Neural Network,DNN) with a series of parameters. Secondly, the length of speech feature parameters of different contexts is set and the number of hidden layer cells is adjusted. The optimal context length is obtained. Finally, the optimal network structure is selected, and the motion path information of the speech organ output by the optimal network is used to control the speech organ motion synthesis, and the virtual human animation synthesis is realized. The second scheme: speech driven virtual speaker animation synthesis method based on MPEG-4 is a data-driven method. Firstly, the sound visual corpus is constructed from LIPS2008 database. Then, BP (Back Propagation) neural network is used to study the mapping relationship between speech feature parameters and virtual human face animation parameters (Facial Animation Parameters,FAP). Finally, according to the predicted FAP sequence, the virtual human facial model is controlled to synthesize virtual population animation. In this paper, the animations synthesized by the two schemes are evaluated subjectively and objectively, and the validity of the two schemes is proved, and the animation effect is natural and lifelike. Compared with the two animation synthesis schemes, the first one needs a lip model which is suitable for it. Although its accuracy is high, it is not universal enough, and its corpus is not easy to obtain. The second scheme conforms to MPEG-4 standard and uses FAP sequence driven virtual human facial model to synthesize animation, which is more versatile and more convenient for wide application.
【学位授予单位】：西南交通大学
【学位级别】：硕士
【学位授予年份】：2017
【分类号】：TN912.3

【参考文献】