面向异构人脸识别的跨模态度量学习研究

发布时间：2018-07-27 19:28

【摘要】：异构人脸识别是指待比对识别的人脸图像来自两个不同模态的人脸识别,如近红外图像与可见光图像人脸识别,素描与真人照片的人脸识别,低分辨率与高分辨率图像人脸识别等,本文重点研究了异构人脸识别中的跨模态度量学习问题,针对带有模态干扰的异构人脸特征表示,学习距离度量,消除模态的干扰,使得跨模态人脸的同类与不同类距离可分。具体的,针对异构人脸识别应用中跨模态度量学习的不同问题,本文主要提出了以下的四个创新方法:(1)提出了一种基于间隔的跨模态度量学习方法(Margin Based Cross-Modal Metric Learning,简称为 MCM2L)。针对异构人脸识别中,受模态干扰的影响,跨模态同类距离与跨模态不同类距离不可分的问题,提出了一种最大化跨模态三元组距离约束中同类与不同类距离之间的间隔的方法。具体的,采用的度量函数为基于公共子空间的跨模态度量函数,可以对两个模态下的特征找到一个公共子空间,在公共子空间中对特征进行距离度量,学习该度量函数的目标包括两部分,第一部分是最小化跨模态同类样本对的距离,第二部分是约束跨模态三元组中的同类样本对的距离小于不同类样本对的距离一个间隔,该方法可以更关注于优化那些同类与不同类样本距离不可分的样本。所提的方法还被进一步扩展为基于核的方法(Kernelized Margin Based Cross-Modal Metric Learning,简称为KMCM2L)来处理数据非线性可分的问题。所提出的方法在三个异构人脸数据集上进行了测试,验证了所提算法相对于基准算法能取得更优的识别效果。(2)提出了一种基于AUC优化的跨模态度量学习方法(Cross-Modal Metric Learning for AUC Optimization,简称为CMLAuC)。已有的度量学习方法关注于最小化定义在同类和不同类样本对上的距离损失,而通常在异构人脸数据集上,能构造出的同类与不同类的样本对的数量是严重不均衡的,在数据分布不均衡的情况下,采用AUC(Area Under the ROC Curve)指标更具有实际意义。因此,提出了一种优化定义在跨模态样本对上的AUC的跨模态距离度量方法,该方法被进一步扩展为可以优化部分AUC(partial AUC,简称为pAUC),pAUC是在一个特定的假阳率范围内的AUC,这对于一些要求在特定假阳率范围内有较好性能的应用尤其有用。所提算法被建模为一个基于对数行列式正则化的凸优化问题,为了快速的对所提的算法进行优化,提出了一种小批量邻近点优化算法,每轮随机采样一部分的跨模态同类样本对以及跨模态不同类样本对进行优化。所提算法在三个跨模态数据集以及一个单模态数据集上进行了测试,证明了该算法能有效提升基准算法的性能,此外,基于pAUC优化的度量在一些评价指标,如Rank-1,VR@FPR=0.1%上取得了更好的效果。(3)提出了一种稀疏跨模态度量集成学习方法(Ensemble of Sparse Cross-Modal Metrics,简称为 ESPAC)。异构人脸识别中,除了模态不同带来的干扰,人脸图像上通常还存在着很多其它的干扰因素,包括,遮挡,表情变化,光照变化等,针对该问题提出了一种可进行特征选择的跨模态度量学习方法。具体的,首先给出了一种弱的跨模态距离度量学习方法,可以在两类跨模态三元组上学习秩为一的跨模态距离度量,同时进行基于组的特征选择来消除人脸特征中的噪声特征(对应于遮挡,表情变化,光照变化等);通过集成学习的方法来学习一系列可相互补充的弱距离度量,并将它们集成为一个强距离度量。实验证明所提算法在有强遮挡的情况下,可以有效的通过特征选择来提升性能,此外,在三个异构人脸数据集上,证明了所提算法相较于基准算法能有更好的识别效果。(4)提出了一种干扰鲁棒的跨模态度量学习方法(Variation Robust Cross-Modal Metric Learning,简称为 VR-CM2L)。该方法针对解决了漫画人脸识别中度量漫画与照片距离的问题,漫画人脸识别是一种特殊的异构人脸识别问题,识别过程会受到各种干扰因素的影响,与漫画相关的干扰因素包括面部特征夸张,绘画风格变化等,其它干扰因素包括视角变化,表情变化,光照变化等,这些干扰因素使得漫画特征与照片特征之间存在严重的误配准。针对该问题,提出了一种干扰鲁棒的跨模态度量学习方法。具体的,提出了一种特别设计的基于人脸关键点的异构特征抽取方法,照片人脸特征在固定视角以及尺度的人脸关键点周围抽取,漫画特征在同样的人脸关键点周围,在不同的视角以及不同尺度下抽取。为了度量这样的异构特征表示之间的距离,在每个人脸关键点处学习一个跨模态度量,该跨模态度量中采用了距离池化的方法来对齐每个关键点处漫画的多个特征与照片的单个特征。最终漫画与照片之间的距离是所有基于关键点的距离度量的组合,为了保证学习得到的组合度量的全局最优性,所有的基于人脸关键点的跨模态度量是在一个统一的优化框架下学习的。在两个漫画数据集上验证了所提方法在各种干扰情况下的有效性,同时验证了所提出的异构特征抽取方法与VR-CM2L结合,相较于同构的特征抽取方法取得了更好的效果。
[Abstract]:Heterogeneous face recognition refers to face recognition from two different modes, such as near infrared image and visible image face recognition, face recognition of sketch and reality photos, low resolution and high resolution face recognition. This paper focuses on cross modal measurement learning in heterogeneous face recognition. Aiming at the representation of heterogeneous face features with modal interference, learning distance measurement and eliminating modal interference, the similar and different distance of cross modal faces can be divided. Specifically, in view of the different problems of cross modal measurement learning in the application of heterogeneous face recognition, the following four innovative methods are proposed in this paper: (1) a kind of new method is proposed. Margin Based Cross-Modal Metric Learning (MCM2L) based on interval mode (abbreviated as MCM2L). Aiming at the problem of isomeric face recognition, which is influenced by modal interference and the distance between the same type of cross mode and the different class distance of the cross mode, a kind of maximum cross modal three tuple distance constraint is proposed for the same and different class distance. The measure function is a cross mode metric function based on the common subspace, which can find a common subspace for the characteristics under two modes and measure the distance in the common subspace. The target of learning the metric function includes two parts. The first part is the minimization of the cross mode. The distance between the same sample pairs, the second part is that the distance of the same sample pair in the constrained cross modal three tuples is less than the distance from the different class of sample pairs. The method can be more concerned about optimizing the samples of the same kind and the different samples. The proposed method is further extended to a kernel based method (Kernelized Ma). Rgin Based Cross-Modal Metric Learning, called KMCM2L), is used to deal with the problem of data nonlinear separable. The proposed method is tested on three heterogeneous face data sets. It is proved that the proposed algorithm can achieve better recognition effect compared with the benchmark algorithm. (2) a cross modal metric learning method based on AUC optimization is proposed. Cross-Modal Metric Learning for AUC Optimization, referred to as CMLAuC). The existing metric learning method is concerned with minimizing the distance loss of the definition in the same and different sample pairs, but usually on a heterogeneous face data set, the number of similar and dissimilar sample pairs is seriously unevenly matched in the data distribution. In equilibrium, the use of the AUC (Area Under the ROC Curve) index is more practical. Therefore, a cross modal distance metric method is proposed to optimize the definition of AUC on the cross modal sample pair. This method is further extended to be able to optimize part AUC (partial AUC, for short), and pAUC is within a specific false positive rate range. AUC, which is particularly useful for applications that require better performance in a specific false positive rate range. The proposed algorithm is modeled as a convex optimization problem based on the regularization of logarithmic determinants. In order to optimize the proposed algorithm quickly, a small batch neighborhood point optimization algorithm is proposed. The proposed algorithm has been tested on three cross modal data sets and one single mode data set, which proves that the algorithm can effectively improve the performance of the benchmark algorithm. In addition, the pAUC based optimization measures have been achieved on some evaluation indicators, such as Rank-1 and VR@FPR=0.1%. Good results. (3) a sparse cross modal measurement integrated learning method (Ensemble of Sparse Cross-Modal Metrics, called ESPAC) is proposed. In the heterogeneous face recognition, there are many other interference factors, including, occlusion, expression change and illumination change, in the face image. In this paper, a cross modal metric learning method, which can be selected for feature selection, is presented. Firstly, a weak cross modal distance metric learning method is given, which can learn the cross modal distance measure of the rank one in two types of cross modal three tuples, and perform the feature selection based on the group to eliminate the noise characteristics of the face features. We learn a series of complementary weak distance measures by integrated learning, and integrate them into a strong distance measure. Experiments show that the proposed algorithm can effectively improve performance through feature selection in the case of strong occlusion. In addition, three heterogeneous face data are used. It is proved that the proposed algorithm can have better recognition results than the benchmark. (4) an interference robust cross modal metric learning method (Variation Robust Cross-Modal Metric Learning, called VR-CM2L) is proposed. This method aims at solving the problem of measuring the distance between comics and photographs in comic face recognition, and comic face recognition It is a special heterogeneous face recognition problem. The identification process is affected by various interference factors. The comic related factors include exaggeration of facial features, change of painting style, and other interference factors including visual angle change, expression change, illumination change, etc. these interference factors make comic features and photo characteristics exist between them. In the case of serious misregistration, a robust cross modal metric learning method is proposed. In particular, a specially designed heterogeneous feature extraction method based on the key points of the face is proposed. The feature of the photo face is extracted around the fixed point of view and the key point of the face, and the character of the comic is in the same face key. In order to measure the distance between these heterogeneous features, a cross modal measurement is learned at each key point of each face, and a distance pooling method is used to align the multiple features and the individual features of the pictures at each key point. The distance between the picture and the picture is a combination of all the distance metrics based on the key points. In order to ensure the global optimality of the combined measure obtained by the learning, all the cross modal measurements based on the key points of the face are learned under a unified optimization framework. The two comic data sets verify the interference situation of the proposed method. The validity of the proposed method is verified by the combination of the heterogeneous feature extraction method and VR-CM2L, which achieves better results than the isomorphic feature extraction method.
【学位授予单位】：南京大学
【学位级别】：博士
【学位授予年份】：2017
【分类号】：TP391.41

【相似文献】