基于Web图像的Kinship关系验证研究

发布时间：2018-08-18 08:07

【摘要】：计算机视觉系统的最终目标是要获得自适应能力、自学习能力、在各种解决方案中权衡的能力、对新的上下文情景和应用场合进行泛化的能力,以及和其它系统(包括人)进行沟通的能力。人脸作为计算机视觉领域中的一种重要研究对象,因其在图像获取过程中的便利性和廉价性,受到了模式识别和机器学习等相关领域科研人员的广泛关注,经过近三十年的发展,人脸识别系统已经开始从实验室进入商业领域。然而,在这一从实验室到具体应用场景的转移过程中,存在多种不同性质的人脸识别问题,其中一些还非常困难,例如对从网页中采集的人脸图像进行亲属关系验证的问题。基于人脸图像进行亲属关系验证面临新的问题和挑战,这些问题主要来自于组图像的表示和验证器的设计两个方面。其中组图像表示方面的问题包括由成像环境、表情、遮挡、姿态和遗传特性等造成的人脸外观上的丰富变化。而验证器的设计则面临组图像刻画困难、目标类信息缺失和遗传差异大等因素。正是由于这些挑战存在,使得之前的人脸验证算法难以直接被用于处理亲属关系验证,急需研究新的解决方案来应对这些问题。本文重点研究了基于Web图像的鲁棒的亲属关系验证问题。本文重点讨论亲属关系验证中涉及的三个核心问题,即亲属关系主体对象的表示学习,亲属关系验证器的设计和在实际应用场合中的推广。针对第一个核心问题,提出了一种基于软投票的亲属关系人脸特征块的选择算法;针对第二个问题,探讨了嵌入一定先验信息的组亲属关系验证模型;针对在实际应用场合中的推广,提出了混合亲属关系验证问题及其模型设计方法。具体地,本文的主要贡献和创新点可以总结为如下几点:(1)提出一种考虑组关系的亲属关系验证问题并发布一个包含超过1000组家庭的亲属关系人脸数据集。亲属关系验证学习可以被看作是向刻画多个视觉对象之间互信息的迈进,然而已有的亲属关系验证研究大多考虑的是对关系,即父—子,父—女,母—子和母—女关系,但在实际应用领域,亲属关系包括更加复杂的主体关系,而在所有人类社会关系中的核心基础单元是父母—儿子和父母—女儿家庭关系,理解该亲属关系将促进人工智能对人类社会行为的理解,也是实现计算机视觉系统从对单一对象的刻画到多个主体对象描述的飞跃,另外,相较于更复杂的亲属关系验证,组亲属关系验证更容易实现,因为其涉及的范畴是可控的,且问题本身也更容易定义。(2)提出了一种基于软投票的亲属关系人脸特征块选择方法。探讨了基于有监督方式的亲属关系表示学习,实现亲属关系特征提取的判别性和鲁棒性。主要针对现有亲属关系表示学习仅使用家庭主体中的某个单一对象,而亲属关系主体之间又具有一定空间结构关系问题,考虑挖掘主体对象之间的相关性并利用这些相关性探寻亲属主体间的判别信息。具体地,在给定图像中每个位置上的所有单个特征完成相互之间的竞争后,再选择一些图像组,而这些组所包含的获胜单特征的比例更高。该方法的主要优点是相较于主流的人脸局部特征选择算法更加灵活,因为其是在一种更加精细的级别进行特征选择,因此可以获得更高的性能。(3)提出了一种嵌入人类社会学知识的相对对称的组亲属关系验证模型。考虑到现有亲属关系验证必须要面对的问题,即小样本问题,而借助额外判别信息又是解决小样本问题的一个有力手段,受人类社会学研究成果的启发,将孩子和父母中某一方较为相似的先验信息嵌入模型,提出一种相对对称双线性模型,在TSKinFace和KinFaceW亲属关系人脸数据集上验证了算法的有效性。另外,当父母双方信息均已知时,该方法还可用于解决对亲属关系验证问题,相对于基于父母中一方进行判定的方法具有较好的推广性,一定程度上弥补了待验证人脸的身份信息,在TSKinFace数据集上验证了算法的有效性。最后,所提方法可以被看作为一个框架,在该框架中可以通过有效地嵌入先验信息的方式整合任何一种用于处理对亲属关系验证的方法来应对组亲属关系验证问题。(4)提出了一种混合亲属关系验证问题及其模型设计方法。主要针对现有亲属关系验证都是基于给定主体的性别种类分别进行研究而为实际应用带来额外的性别标注工作量的问题,探讨了亲属关系验证模型在实际应用场景中的推广,提出了混合对亲属关系验证。具体的,受人类社会学研究成果的启发,即一些人脸外观,如眼睛、头发颜色、酒窝、皮肤等表现出极强的遗传性,将不同亲属关系看作为不同但相互之间有相关性的任务,并使用多任务学习框架将每个任务模型分解为两个部分,即一部分在所有任务间共享,另一部分则被每种任务独享。这两部分在一个联合框架下同时学习,使得所提算法能利用到多个任务之间的共有信息。另外,该方法的优点是,当每个任务仅有很少训练样本时,能通过在任务间迁移信息的手段互补判别性信息以达到提高算法泛化性的目的。进一步,为了使算法更加鲁棒,提出了一个多视图多任务的混合对亲属关系验证模型,其中通过为不同的特征学习各自不同的权重融合多种特征以提高混合对亲属关系验证的性能。
[Abstract]:The ultimate goal of computer vision systems is to acquire the ability of self-adaptation, self-learning, the ability to weigh among solutions, the ability to generalize new contexts and applications, and the ability to communicate with other systems (including people). Because of its convenience and low cost in the process of image acquisition, it has attracted extensive attention of researchers in the fields of pattern recognition and machine learning. After nearly 30 years of development, face recognition system has begun to enter the commercial field from the laboratory. However, in the process of the transition from the laboratory to the specific application scenario, there exists a lot of problems. There are many different kinds of face recognition problems, some of which are still very difficult, such as the problem of kinship verification of face images collected from web pages. The problems of group image representation include the rich changes of facial appearance caused by imaging environment, expression, occlusion, posture and genetic characteristics. The design of the validator is faced with the difficulties of group image description, target class information missing and genetic differences. This paper focuses on the robust relational validation problem based on Web images. This paper focuses on three core issues involved in relational validation, namely, representation learning of relational subject objects and relational validator. Aiming at the first core problem, this paper proposes an algorithm for selecting the feature blocks of relatives based on soft voting; for the second problem, a group relatives validation model embedding certain prior information is discussed; for the promotion in practical application, a hybrid relatives algorithm is proposed. Specifically, the main contributions and innovations of this paper can be summarized as follows: (1) A relational validation problem considering group relationships is proposed and a relational face dataset containing more than 1000 families is published. Mutual information advances between visual objects, however, most of the existing kinship validation studies have considered pairwise relationships, i.e. father-son, father-daughter, mother-son and mother-daughter relationships. In practical applications, kinship includes more complex subject relationships, and the core unit of all human social relationships is parent-son. Understanding a parent-daughter family relationship will facilitate AI's understanding of human social behavior, as well as a leap in computer vision systems from depicting a single object to describing multiple subject objects. In addition, group kinship validation is easier to implement than more complex kinship validation because of its involvement. (2) A method of feature block selection based on soft voting is proposed for relational facial feature extraction. The method is based on supervised relational representation learning to realize the discriminability and robustness of relational feature extraction. A single object in a family subject, and the relatives have a certain spatial structure relationship between them. Considering mining the relativity between the subject objects and exploring the discriminant information between the relatives, all the individual features in each position in a given image compete with each other. The main advantage of this method is that it is more flexible than the mainstream face feature selection algorithm, because it is a more fine level of feature selection, so it can obtain higher performance. (3) A new embedded human face feature selection algorithm is proposed. Relatively symmetrical group relational validation model of sociological knowledge. Considering the existing problems in relational validation, that is, the small sample problem, and the use of additional discriminant information is a powerful means to solve the small sample problem, inspired by the results of anthropological sociology, children and one of the parents are more similar. A priori information embedding model is proposed, and a relative symmetric bilinear model is proposed to verify the validity of the proposed algorithm on TSKinFace and KinFaceW relational face datasets. In addition, when both parents'information is known, this method can also be used to solve the problem of relational validation. Finally, the proposed method can be regarded as a framework in which any method used to process relational validation can be integrated by effectively embedding prior information. (4) Propose a hybrid kinship validation problem and its model design method. Mainly aim at the problem that the existing kinship validation is based on the gender type of given subject and brings extra gender labeling workload for practical application, and discuss the kinship validation model. A hybrid approach is proposed to validate kinship in practical scenarios. Specifically, inspired by anthropological research, some facial features, such as eyes, hair color, dimples, and skin, exhibit strong inheritance. Different kinship relationships are viewed as different but related tasks and are used widely. Task learning framework decomposes each task model into two parts, one shared by all tasks and the other shared by each task. The two parts learn simultaneously in a joint framework, enabling the proposed algorithm to take advantage of the common information between multiple tasks. Furthermore, in order to make the algorithm more robust, a multi-view and multi-task hybrid pairwise kinship verification model is proposed, in which different weights are fused by learning for different features. Features to enhance the performance of hybrid validation for kinship.
【学位授予单位】：南京航空航天大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TP391.41

【相似文献】