面向情感语音合成的言语情感建模研究

发布时间：2018-05-01 15:14

本文选题：情感语音合成 + 情感生成　；参考：《北京交通大学》2016年博士论文

【摘要】：语音作为人类重要的交际工具之一,除传递字面信息,还通过语气的变化传递情感。当前情感语音研究,主要集中于某些特定情感状态与语音信号关联关系的探寻,虽然观察到一些情感与声学参数变化相关联的指向性线索,但由于情感表现的多样性和复杂性,导致情感声学参数的数值分布多呈现较大的离散特性。情感语音合成时,情感信息来源于人工指定或特定数据库分析,结合文本内容与场景因素的情感预测研究仍处于起步阶段。论文研究重点是基于文本分析实现言语情感预测,解决情感语音合成中文本-情感信息分析问题。研究主要涉及以下问题：1)相关的理论需要有所升华,尤其是要解决情感的准确刻画和动态衍化过程的描述；2)建模技术需要有所突破,考虑到影响情感因素及情感生成过程的复杂性,所需处理的特征参数可能会来自多个层面,模型应能支持多尺度特征处理及动态衍化过程刻画。针对第一个问题,论文在心理学、朗读学、播音学与语音学等相关理论和实践指导下,采用心理语言学、感知语音学实验和数据分析相结合的方法,探索汉语朗读或播音等创作型有声语言活动中情感表达与言语特征间的关联关系,进而对言语情感生成及衍化机制进行归纳。以此为基础,提出多视角情感描述方案,分别从认知评价、心理感受、生理反应和发音描述四种视角描述言语情感的不同侧面,各视角互为补充,共同构成言语情感的分布式表达。各视角之间依据言语情感生成过程形成直接或间接的衍化关系。发音描述作为言语情感生成过程的最终输出结果,形成连接情感描述与声学参数的接口,有助于发现二者之间更为显性的映射关系。基于该描述方案,构建了一个新闻言语情感数据库,通过言语情感标注的实施以及后续预测模型的建立验证了言语情感生成过程及描述方案的合理性。针对第二个问题,采用深度神经网络构建文本-情感预测模型；一方面由于深度网络的多层非线性映射结构与多视角描述模型的多层分布式结构一致,另一方面便于模型实现对情感动态衍化过程以及多尺度特征关联关系的建模。具体来说,暂不考虑文本内容之外的影响因素,利用主题模型提取文本的语义空间向量表示,依次预测篇章级、段落级和句子级不同尺度的情感信息。各尺度内部,形成由认知到心理、生理再到发音的衍化关系,发音描述作为最终目标,其他成分作为其子目标,子目标依次作为后续预测目标的部分已知信息参与到后续模块的训练；不同尺度之间,构成由上至下的层级结构,大尺度单元的预测结果作为小尺度单元的部分已知信息参与到小尺度单元的预测,为其提供更为全局的上下文参考。最后通过实验验证了所提方法的有效性,加入情感衍化关系以及多尺度特征间关联关系的影响,使模型最终预测结果的召回率、精准率和F1值分别相对提升了31.8%、10.3%和22.8%。本文工作的主要创新点在于：(1)基于言语情感生成过程的分析归纳,提出多视角情感描述模型：模型细致刻画了言语情感生成过程中各成分的变化及之间的衍化关系,并以发音描述作为连接情感与语音的接口,用于指导后续合成语音时声学参数的调整；(2)基于深度神经网络,构建文本-情感计算模型：模型综合考虑了言语情感生成过程中来自不同尺度特征的影响以及不同情感成分间的衍化关系,支持多尺度特征融合以及动态衍化关系刻画；(3)将先验知识引入深度神经网络,实现网络中间结构的部分可见化：通过网络结构的直接显性设定,有效利用了言语情感生成的先验知识,降低了训练数据与网络规模的开销,预测性能亦有所提升。
[Abstract]:As one of the most important communication tools of human beings, phonetics is not only conveys literal information, but also transfers emotion through the change of mood. The current study of emotional voice mainly focuses on the exploration of the relationship between certain emotional states and speech signals, although some of the directivity clues associated with the changes of the acoustic parameters are observed, but the emotion is due to emotion. The diversity and complexity of performance results in more discrete characteristics of the numerical distribution of emotional acoustic parameters. Emotional information is derived from artificial or specific database analysis, and the research on emotional prediction of text content and scene factors is still in the stage of emotion speech synthesis. The focus of this paper is based on text analysis. The current speech emotion prediction solves the problem of text emotional information analysis in emotional speech synthesis. The research mainly involves the following questions: 1) the related theories need to be sublimated, especially to solve the accurate depiction of emotion and the description of the dynamic evolution process; 2) the modeling technology needs a breakthrough, considering the influence of emotional factors and emotion generation. The complexity of the process, the characteristic parameters needed to be processed may come from a number of layers. The model should be able to support multiscale feature processing and dynamic characterization. In the first question, the thesis adopts psycholinguistics, perceptual phonetics experiments and data under the guidance of related theories and practices, such as psychology, reading, broadcasting and phonetics. The combined method of analyzing the relationship between emotional expression and speech characteristics in the creative sound language activities, such as Chinese reading or broadcasting, and then the induction of speech emotion generation and evolution mechanism. On the basis of this, a multi perspective emotional description scheme is proposed, from cognitive evaluation, psychological feeling, physiological response and pronunciation description, respectively. The four perspectives describe the different aspects of speech emotion. Each angle of view complement each other and forms a distributed expression of speech emotion. Each angle of view forms a direct or indirect derivative relationship based on the process of speech emotion generation. The pronunciation description is the final output result of the process of speech emotion generation, forming a connection of emotional description and acoustic parameters. The interface helps to find a more explicit mapping relationship between the two. Based on this description, a news speech emotion database is constructed. The process of speech emotion generation and the rationality of the description scheme are verified by the implementation of the speech emotion tagging and the establishment of the follow-up prediction model. The deep neural network is adopted for the second problems. On the one hand, the multi layer nonlinear mapping structure of the depth network is consistent with the multi-layer distributed structure of the multi view description model. On the other hand, the model is convenient for modeling the dynamic evolution process and the multi-scale feature correlation. The theme model extracts the semantic space vector representation of text by using the theme model to predict the emotional information of the text level, the paragraph level and the sentence level in turn. Within each scale, the relationship between the cognitive to the psychology, the physiology and the pronunciation is formed. The pronunciation description is the final target, the other components are its sub targets and the sub goals are in turn in turn. The part known information of the subsequent prediction target is involved in the training of the follow-up module; between the different scales, the hierarchical structure is formed from top to bottom. The prediction results of the large scale unit are involved in the prediction of the small scale units as part of the small scale unit, which provides a more global context reference. Finally, the experimental verification is carried out. The effectiveness of the proposed method, the relationship of emotional evolution and the influence of multi-scale features, the recall rate, precision and F1 value of the final prediction results of the model are increased by 31.8%, 10.3% and 22.8%., respectively. The main innovations of this paper are as follows: (1) based on the analysis and induction of the process of speech emotion generation, the multiple perspectives are put forward. The model of emotion description: the model depicts the changes of all components in the process of speech emotion generation and the relationship between them, and uses the pronunciation description as the interface to connect emotion and speech. It is used to guide the adjustment of acoustic parameters in the subsequent synthetic speech. (2) a text emotion calculation model based on the depth neural network is constructed: model synthesis Considering the influence of different scale features in the process of speech emotion generation and the derivative relationships among different emotional components, it supports multi-scale feature fusion and dynamic characterization. (3) introducing prior knowledge into the deep neural network to realize the visible division of the network intermediate structure: through direct explicit setting of network structure It effectively utilizes the priori knowledge generated by speech emotion, reduces the overhead of training data and network scale, and improves the prediction performance.

【学位授予单位】：北京交通大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TN912.33

【相似文献】