人脸本征图像分解及其应用

发布时间：2018-06-12 22:48

本文选题：人脸 + 本征图像　；参考：《浙江大学》2017年博士论文

【摘要】：在互联网大数据的时代背景下,人们通过互联网上传、分享的信息已经从原本的文字内容渐渐过渡到了以图片、视频为主的多媒体素材。从早期的博客、Twitter,到现在的微信和Instagram,以及微信朋友圈最新改版中加入的短视频功能,都印证了这一发展趋势。根据国际影像产业协会一项针对手机拍摄内容的研究,以及一项针对图片分享网站Flickr用户上传信息的调查,超过60%的用户分享内容都是以人为主体,并且大多包含了人脸。随着目前刚刚兴起的视频直播等新兴应用的不断发展,这一比例还会进一步提升。在中国,以人脸图片美化为主要功能的美图秀秀已经服务于过十亿用户,并且日均影响2亿张照片。基于上述应用背景,本文针对众多图片、视频中最重要的对象——人脸,深入研究了其本征图像、反射模型、几何结构,以及相关的环境光照模型,通过综合利用人脸特有的几何结构和皮肤的生物学性质,对人脸图片的辐射校准、高光分离、本征图像分解等问题提出了切实有效的解决方案。同时,我们以人脸的本征图像分解为基础,提出了一套成熟的人脸照片处理、增强的流水线,并且实现了虚拟化妆这一具有商业价值和现实意义的应用技术。我们的主要工作包括以下内容:1.为了从相机拍摄的非线性图片中还原原本的场景辐射度信息,我们提出了一种基于人体皮肤反射率梯度空间下低秩属性的相机辐射校准算法。该低秩性质不仅具有色彩不变性,同时相对人脸皮肤存在的微弱高光十分稳定。为了利用该性质并获取到皮肤反射率本征图,我们结合了以几何结构为额外约束的本征图像分解算法,将反射率本征图与其亮度本征图分开。相较之前的单张图像相机辐射校准算法,该算法在大幅降低相机响应函数估算误差的同时并没有带来额外的计算开销。2.我们提出了一种利用人脸皮肤物理光学、人脸几何结构统计学特性来实现的在复杂光照环境下分离人脸图片中镜面高光部分的技术。首先,人脸的皮肤颜色主要取决于皮肤表皮层黑色素和真皮层血红蛋白的含量,这一皮肤的光学性质可以用来约束皮肤的漫反射率,从而可以间接约束因皮肤镜面反射而形成的高光部分的强度。其次,利用人脸几何结构高度相似的统计学特性,并且结合对环境光照效果的近似和建模,则可以直接形成对镜面高光强度的约束。该技术不仅可以对在复杂光照环境下拍摄的照片进行处理,还能够有效处理因过曝而引起的颜色饱和。3.在人脸高光分离技术的基础上,我们进一步利用人脸的反射参数的统计模型作为约束,提出了一种更加准确的人脸本征图像分解算法。除此之外,传统的本征图像分解大多基于理想的漫反射模型——朗伯模型,该模型并不能很好地模拟皮肤等半透明材质的漫反射属性。因此,本文还引入了一种基于物理光学的半透明材质漫反射模型来对皮肤的漫反射亮度性进行了重新建模,从而进一步提高了本征图像分解的准确性。4.针对图片美化中虚拟化妆这一应用,我们以人脸本征图像分解为基础,在不同的本征图层上从物理光学的角度对不同化妆品的化妆效果都进行了建模,从而实现了对各种化妆品效果的真实模拟。这些化妆品包括了但不限于粉底、口红、腮红、眼影等。区别于传统的虚拟化妆技术,本文提出的方法不仅速度快,而且避免了光照不一致、人脸皮肤特征与化妆效果混杂等传统虚拟化妆技术难以避免的问题。更进一步,我们通过与现有的人脸检测、追踪技术相结合,使得该算法不仅可以作用在静态图片上,还可以实现对视频流中姿态、表情同时发生变化的人脸部分进行真实感虚拟化妆。
[Abstract]:In the era of big internet data, people uploaded through the Internet, sharing information from original text content to picture, video based multimedia material. From the early blogs, Twitter, to the current WeChat and Instagram, and the short video features added to the latest version of WeChat's circle of friends, all of them are printed. According to a research on the content of mobile phones by the international image industry association, and a survey of Flickr users uploading information on the picture sharing website, more than 60% of the users share the content of human beings and mostly include human faces. With the emerging applications, such as the emerging video live broadcast, and other emerging applications In China, in China, Meitu Xiu Xiu has served one billion users with face picture beautification as the main function, and affects 200 million photos at the same day. Based on the above application background, this article studies the most important object in the video, the face, and studies its intrinsic image in depth. The ejection model, geometric structure, and the related environmental illumination model, through the comprehensive use of the geometric structure of the face and the biological properties of the skin, put forward a practical and effective solution to the radiation calibration of the face images, the high light separation, the decomposition of the intrinsic image and so on. A set of mature face photo processing, enhanced pipelining, and the implementation of virtual make-up, a commercial and practical application technology. Our main work includes the following: 1. in order to restore the original scene radiance information from the nonlinear picture taken by the camera, we propose a kind of human skin based on the human skin. The low rank attribute of the camera radiation calibration algorithm under the reflectivity gradient space. The low rank property not only has the color invariance, but also the weak high light is very stable relative to the face skin. In order to use this property and obtain the skin reflectivity eigengraph, we combine the eigenimage decomposition algorithm with the geometric structure as the extra constraint. The albedo eigengraph is separated from the luminance eigengraph. Compared with the previous single image camera radiometric calibration algorithm, this algorithm reduces the estimation error of the camera response function and does not bring additional computing overhead.2.. We propose a method of using the physical optics of the face skin and the geometric characteristics of the face. First, the skin color of the face is mainly dependent on the skin epidermal melanin and the content of the dermal hemoglobin. The optical properties of the skin can be used to restrain the diffuse reflectance of the skin, which can be indirectly constrained by the reflection of the skin. The intensity of the high light part. Secondly, by using the statistical characteristics of the highly similar geometric structure of the face, and combining the approximation and modeling of the environmental illumination effect, the constraint on the high light intensity of the mirror can be formed directly. This technique can not only deal with the photos taken in the complex light environment, but also effectively handle the overexposure. On the basis of the face high light separation technology caused by color saturation.3., we further utilize the statistical model of the reflection parameters of the face as constraints, and propose a more accurate decomposition algorithm for face eigenimages. In addition, the traditional eigenimage decomposition is mostly based on the ideal diffuse model Lambert model. It is not good to simulate the diffuse reflectance properties of skin and other semitransparent materials. Therefore, this paper also introduces a translucent material diffuse model based on physical optics to remodel the diffuse reflectance of the skin, thus further improving the accuracy of the intrinsic image decomposition.4. for the virtual make-up of the picture beautification. One application, based on the face eigenimage decomposition, we modeled the cosmetic effects of different cosmetics from the physical optics angle on different eigenlayers, thus realizing the real simulation of the effects of various cosmetics. These cosmetics include but not limited to the foundation, lipstick, blush, eye shadow, etc., which are different from the traditional deficiency. In this paper, the method proposed in this paper is not only fast, but also avoids the problem that the traditional virtual make-up technology can not be avoided, such as the inconsistency of light and the mix of facial skin features and cosmetic effects. Furthermore, by combining with the existing face detection and tracking techniques, the algorithm can not only function on the static picture, but also make the algorithm not only function on the static picture but also the static image. It can realize the virtual virtual make-up of the face part of the video stream in which the gestures and facial expressions change simultaneously.
【学位授予单位】：浙江大学
【学位级别】：博士
【学位授予年份】：2017
【分类号】：TP391.41

【相似文献】