基于深层神经网络的声学特征提取及其在LVCSR系统中的应用

发布时间：2018-05-08 14:33

本文选题：深层神经网络 + 大词汇量连续语音识别　；参考：《中国科学技术大学》2014年博士论文

【摘要】：近年来,深层神经网络(Deep Neural Network, DNN)的崛起在诸多研究领域都引起了强烈的反响,并越来越受到关注。在语音识别领域,DNN相关技术能够使声学模型的识别性能得到显著性提升,已然成为新的研究热点。DNN通常以两种方式被有效地应用到声学建模当中：1)与隐马尔科夫模型(Hidden Markov Model, HMM)组成混合架构模型DNN-HMM,代替高斯混合模型(Gaussian Mixture Model, GMM)进行状态输出概率的计算；2)作为前端的声学特征提取器,为传统的GMM-HMM声学建模架构提供更有效的声学特征。本文将主要围绕基于DNN的声学特征提取及其在LVCSR系统中的应用展开研究,包括Tandem特征(或叫作概率特征)提取和瓶颈(bottleneck)特征提取两方面。首先,本文针对中文LVCSR系统提出了一种构建音素建模单元集的方法。这里,音素建模单元集的构建包括将传统声韵母建模单元集中的韵母细化到拼音音素级别、以一些先验知识为指导对这些单元进行调整并构造相对应的词典,以及首次基于扩展元音三角图设计问题集。音素建模单元集的单元数目相对较少,显得更为精简,冗余度和重合性降低,音素单元之间的区分性增强,在提取Tandem特征时可以有效的减少神经网络的输出层节点数目,这样既可以降低神经网络的复杂度,也更有利于Tandem特征的提取。实验表明,相比于声韵母建模单元集,新的音素建模单元集在基线系统和Tandem特征提取两方面都取得了较优的识别性能。其次,本文搭建了基于DNN提取bottleneck特征的基线系统,并结合一些启发式的技术进行了相应的优化。为了提取bottleneck特征,我们通常是将DNN中间隐层的节点数设置成一个较小的数值(往往与基础MFCC或PLP特征维度一致),这个隐层就被形象地称作为bottleneck层,而这种具有特殊结构的DNN被称作为bottleneck DNN, bottleneck层的输出就是所期望得到的基线bottleneck特征。实验表明,通过引入一些启发式的技术,比如利用线性变换技术PCA去相关、一阶差分和二阶差分系数、在解码过程中使用声学规整因子重新调整声学模型得分和语言模型得分之间的相对重要性等,bottleneck特征取得了较为显著的性能提升,甚至能与DNN-HMM混合架构模型的性能相比拟。其中,引入声学规整因子尤为重要。第三,本文提出了两种DNN非相干训练(Incoherent Training)算法,即最小化bottleneck层权重矩阵的相干性和最小化每个1nini-batch中数据的相关系数。算法通过在DNN训练的原始目标函数上增添归整项,使得bottleneck特征各维之间的相关性能够在DNN的训练过程中较为显式地被直接控制并被自动去除,以便于更好地用于后续对角化GMM-HMM的建模。实验结果表明,两种非相干训练算法都取得了一定的效果,使得基于bottleneck特征的GMM-HMM模型在识别性能上超越了DNN-HMM. 最后,本文引入序贯区分性训练(Sequential Discriminative Training,SDT)算法对bottleneck DNN进行优化,从而得到性能更好的bottleneck特征。SDT算法在GMM-HMM建模架构中已经取得了卓越的效果,其目标函数中包含的语音帧之间的序贯信息对于语音识别问题至关重要,而这恰恰是传统的DNN训练算法(主要是基于帧分类的交叉熵算法)所缺少的,因此,我们借助SDT算法来对DNN参数进一步优化,并且采用了两种新的bottleneckDNN结构。两种结构都是将bottleneck层后移至最后一个隐层,但是,第一种结构的其他隐层节点数目相同,另一种则采取“宽”、“窄”隐层交织的结构。从实验结果中可以看到,SDT算法有助于得到更好的bottleneck特征,并且在后移结构上能够取得更优的性能,而使用交织结构能够减少提取bottleneck特征的计算量,同时保证识别性能基本没有损失。
[Abstract]:In recent years , the rise of Deep Neural Network ( DNN ) has caused a strong echo in many fields , and more and more attention has been paid . In the field of speech recognition , DNN technology can improve the recognition performance of the acoustic model , which has become a new research hotspot . The DNN is usually applied to acoustic modeling in two ways : 1 ) and Hidden Markov Model ( HMM ) .
2 ) As the acoustic feature extractor at the front end , it provides more effective acoustic characteristics for the traditional acoustic modeling architecture of the gmm - HMM . This paper mainly focuses on the feature extraction based on DNN and its application in LVCSR system , including the extraction of tandem feature ( or probability feature ) extraction and bottleneck feature extraction .

Firstly , this paper proposes a method of constructing a phoneme modeling unit set for Chinese LVCSR system . The construction of the phoneme modeling unit includes thinning the vowel in the traditional vowel modeling unit to the phoneme level , adjusting and constructing the corresponding dictionary based on some prior knowledge , and reducing the number of output layer nodes of the neural network for the first time .

Secondly , this paper sets up a baseline system based on DNN extraction feature , and optimizes it with some heuristic techniques . In order to extract the characteristic , we usually set the number of nodes of the middle hidden layer of DNN to a smaller value ( which is often consistent with the basic MFCC or PLPs feature dimension ) , and this hidden layer is referred to as the base layer , and the output of this special structure is called as the base line characteristic . The experiment shows that by introducing some heuristic techniques , such as using linear transformation technique PCA to re - adjust the relative importance between the acoustic model score and the language model score , this feature has achieved remarkable performance improvement , and even can be compared with the performance of the DNN - HMM hybrid architecture model .

Thirdly , two kinds of DNN non - coherent training ( Incoherent Training ) algorithms are proposed to minimize the coherence of the weight matrix and minimize the correlation coefficient of data in every 1nini - batch .

In the end , this paper introduces sequential discriminative training ( sdt ) algorithm to optimize the speech recognition problem , so as to get better performance . The algorithm is very important to the speech recognition problem in the model of gmm - HMM , and the two new structures are the structure of the traditional DNN training algorithm ( mainly based on the cross entropy algorithm of frame classification ) .

【学位授予单位】：中国科学技术大学
【学位级别】：博士
【学位授予年份】：2014
【分类号】：TN912.3;TP18

【共引文献】