基于单目视频相机的实时人脸跟踪与动画方法研究

发布时间：2018-08-04 20:32

【摘要】：使用计算机生成具有真实感的人脸动画一直是计算机图形学中的重要研究课题。近几十年来,研究者们围绕着这一课题进行了大量的研究工作,取得了丰硕的成果。在工业级应用如影视游戏制作中,人们通常依赖于一些特殊设备,并且需要各种复杂的操作和大量运算,来精确跟踪人脸运动并生成具有高度真实感的人脸动画。但是,这些昂贵的设备和耗时的计算并不适用于面向于普通用户的应用。对于普通用户来说,使用单目视频相机对人脸运动进行实时跟踪并生成人脸动画是最简单、最有效的方法。当前基于单目视频相机的实时人脸跟踪与动画技术在人脸跟踪的准确性、稳定性,以及生成人脸动画的真实感、细节丰富性、表现力等方面,与基于特殊设备的方法之间还存在着很大的差距。本文围绕着基于单目视频相机的实时人脸跟踪与动画技术进行了深入、系统的研究,提出了一系列具有创新性的算法,为普通用户使用单目视频相机进行准确、高效的人脸运动跟踪,以及生成具有真实感的人脸动画提供了一种有效途径。本文研究涵盖了人脸动画技术的三个核心组成部分:人脸模型表示、人脸运动跟踪捕获以及数字化替身的生成。具体工作如下:1.在人脸模型表示方面,我们针对现有人脸数据库中表情表现力不足的问题,研制了FaceWarehouse,一个专用于可视计算应用的三维人脸表情数据库。FaceWarehouse使用一个RGB-D相机扫描采集了 150个用户在20个不同表情下的几何纹理数据。基于这些采集数据,我们为每个用户生成一个特定的表情融合模型,包含了该用户对应于人脸动作编码系统描述的47个基本表情。最后,我们用这150个用户的表情融合模型构建了一个双线性人脸模型。该双线性人脸模型可以用于表示不同用户在不同表情下的人脸形状,因此可以被用于各种可视计算应用中。2.在人脸运动跟踪捕获方面,我们提出了三种基于单目视频相机的实时人脸运动跟踪方法。a)首先,我们提出了一种基于三维形状回归的实时人脸跟踪方法。该方法为每个特定用户训练生成一个特定的三维人脸形状回归器,并利用该回归器在用户的人脸视频中准确跟踪人脸特征点的三维位置。b)针对前述方法中需要为每个特定用户进行预处理的问题,我们提出了偏移动态表情(DDE)模型这一新颖的人脸形状表示方法,并基于DDE模型提出了一个基于单目视频相机的全自动实时人脸运动跟踪方法。该方法可以对任意用户进行准确的人脸运动跟踪,而不需要任何的预处理过程。c)在前述工作基础上,我们提出了一个实时的高精度人脸运动跟踪捕获方法。该方法可以从人脸的局部外观中实时计算得到局部细节形状,进而重建高精度的人脸几何模型,包含了丰富的脸部细节特征如皱纹等。3.在数字化替身生成方面,我们提出了一种基于图像的动态替身表达方法。该方法基于对一个用户采集的几十张图像,为该用户构建人脸的表情融合模型和头发的形变模型。这些采集图像和构建的模型一起构成了该用户的动态替身表达。该动态替身在人脸运动跟踪系统的驱动下,可以生成具有真实感的人脸动画,包含了用户人脸丰富的细节特征,以及具有真实感的头发运动效果。
[Abstract]:The use of computer to generate realistic face animation has always been an important research topic in computer graphics. In recent decades, researchers have carried out a lot of research and fruitful achievements around this topic. In industrial applications such as video game making, people usually rely on some special equipment, and It requires a variety of complex operations and large numbers of operations to accurately track face motion and create a highly realistic face animation. However, these expensive devices and time-consuming calculations do not apply to applications for ordinary users. For ordinary users, a single visual frequency camera is used to track and generate face motion in real time. Face animation is the simplest and most effective method. The current face tracking and animation technology based on monocular video camera has a large gap between the accuracy, stability, the authenticity, the richness and the expressiveness of face animation, and the method based on the special device. The real-time face tracking and animation technology based on monocular video camera has been deeply studied. A series of innovative algorithms are proposed. It provides an effective way for the ordinary users to use monocular video cameras for accurate, efficient face motion tracking and the generation of realistic face animation. It covers three core components of face animation technology: face model representation, face motion tracking capture and digital substitutes generation. The specific work is as follows: 1. in face model representation, we developed a FaceWarehouse for visual computation for the lack of expressive expressiveness in the existing face database. The 3D facial expression database.FaceWarehouse uses a RGB-D camera to scan the geometric texture data of 150 users under 20 different expressions. Based on these data, we generate a specific expression fusion model for each user, including the user's 47 basis for the description of the face action coding system. In the end, we construct a bilinear face model with the 150 users' expression fusion model. The bilinear face model can be used to represent the shape of the face of different users under different expressions. Therefore, we can be used in various visual computing applications for.2. in the face tracking and capture of face movement. We propose three kinds of methods based on this model. A real-time face motion tracking method for monocular video cameras.A) first, we propose a real-time face tracking method based on 3D shape regression. This method generates a specific 3D face shape regression for each specific user, and uses the regression device to track face features accurately in the user's face video. Three dimensional position.B), aiming at the problem of preprocessing for each particular user in the foregoing method, we propose a novel face shape representation method of the offset dynamic expression (DDE) model, and a fully automatic real time face motion tracking method based on the DDE model is proposed. This method can be used arbitrarily. The user carries out accurate face motion tracking without any preprocessing process.C). On the basis of the previous work, we propose a real-time and high precision face motion tracking capture method. This method can calculate the local details from the local appearance of the face, and then reconstruct the high precision face geometric model, including the face geometric model. With the rich facial details, such as wrinkles, such as wrinkles and other.3., we propose an image based dynamic substitute expression based on a user's collection of dozens of images, the facial expression fusion model and the hair deformation model for the user. These images and models are built. The dynamic substitute, driven by the human face motion tracking system, can generate a realistic face animation, including the rich details of the user's face, and the effect of a realistic hair movement.
【学位授予单位】：浙江大学
【学位级别】：博士
【学位授予年份】：2016
【分类号】：TP391.41

【相似文献】