3D Face Recreation from Monocular or Stereo Pictures.

Presentation Transcript

3D Face Reconstruction from Monocular or Stereo Images. Thomas Vetter Universit y of Basel Switzerland http://gravis.cs.uni bas.ch

Change Your Image ...

Analysis by Synthesis show parameter Analysis Image Model Synthesis Image 3D World Image Description

Approach: Example based demonstrating of confronts 2D Image 3D Face Models 2D Image 2 D Face Examples = w 1 * + w 2 * + w 3 * + w 4 * +. . .

Morphing 3D Faces 1 __ 2 3D Blend 3D Morph 1 __ = + 2

Shape and Texture Vectors Reference Head Example i

Surface enlistment: Which representation?

Registration in various representations Curvature Guided Level Set Registration utilizing Adaptive Finite ElementAndreas Dedner, Marcel Lüthi, Thomas Albrecht and Thomas Vetter IN: Proceedings DAGM'07: Heidelberg 2007 Optimal Step Nonrigid ICP Algorithms for Surface Registration Brian Amberg, Sami Romdhani and Thomas Vetter IN: Proceedings, CVPR'07, Minneapolis, USA 2007. A Morphable Model for the Synthesis of 3D Faces. Volker Blanz and Thomas Vetter IN: SIGGRAPH'99 Conference Proceedings, 187-194 Implicit: Triangulated: Parameterized:

Database of 3D Faces

Vector space of 3D countenances. A Morphable Model can produce new faces. a 1 * + a 2 * + a 3 * + a 4 * +. . . = b 1 * + b 2 * + b 3 * + b 4 * +. . .

Manipulation of Faces Modeler

Continuous Modeling in Face Space Caricatur e Original Average Anti Face

Model l ing the Appearance of Faces A face is spoken to as a point in face space. Which bearings code for particular characteristics ?

Learning from Labeled Example Faces Fitting a relapse work

Facial Attributes Weight Subjective Attractiveness Gender Original

3D Shape from Images Face Analyzer Input Image 3D Head

Matching a Morphable 3D-Face-Model R = Rendering Function = Parameters for Pose, Illumination, ... Find ideal a, b, r !

Automated Parameter Estimation Ambient: power, shading Parallel : force, shading, bearing Color: differentiate, picks up, balances Face Parameters 150 shape coefficients an i 150 surface coefficients b i head position head introduction central length 3D Geometry Light and Color

Image Formation: at every Vertex k Rigid Transformation Normals Phong Illumination Perspective Projection Color Transformation b i an i

Error Function Image contrast (pixel power cost work) Plausible parameters Minimize

movement by Volker Blanz.

Using Multiple Features �� 

Which Feature to utilize? some Edge finder

Edge Feature Rigid Transformation Normals Phong Illumination Perspective Projection Color Transformation b i an i

Edge Fitting Results

Multi-Features Fitting Algorithm

Multi-Features Fitting Algorithm 1 2 3 4 5 At stage 4:

Recognition from Images Complex Changes in Appearance Images: CMU-PIE database.

3D Computer Graphics

Correct Identification "1 out of 68" (%) 99.5 83.0 97.8 86.2 79.5 85.7 92.3 95.0 89.0 exhibition front side profile test front 99.8 side 99.9 profile 98.3 aggregate CMU-PIE database: 4488 pictures of 68 people 3 postures x 22 enlightenments = 66 pictures for every person

Reanimation of Images V. Blanz, C. Basso, T. Poggio & T. Vetter Reanimating Faces in pictures and Video Proc. of Eurographics 2003

Expression Transfer Fitting Rendering

Analysis by Synthesis show parameter Image Processing Edges Highlights Segmentation … Image Model some ║ X Analysis Synthesis 3D World Image Description Image

Segmenting hair a general necessity ? No exception identification with anomaly veil

Skin division We have to cover out non-skin areas/anomalies 3DMM is not adequate

Shading Problem Skin locales contain solid power angles that make a division troublesome!

Illumination Compensation

Illumination Compensation Skin Detail Analysis for Face Recognition Jean Sebastian Pierrard , Thomas Vetter CVPR 2007 Local fitting

Segmentation Results GrabCut Skin Detail Analysis for Face Recognition Jean Sebastian Pierrard , Thomas Vetter CVPR 2007 Thresholding

Try New Hairstyles 3D Angle, Position Illumination, Foreground, Background 3D Shape and Texture

More Hairstyles 3D Shape and Texture 3D Angle, Position Illumination, Foreground, Background

Using more than a solitary picture ? Reproducing High Quality Face-Surfaces utilizing Model Based Stereo Brian Amberg, Andrew Blake, Andrew Fitzgibbon, Sami Romdhani and Thomas Vetter   IN: Proceedings ICCV 2007 Rio de Janeiro, Brazil

Model Based Stereo

Model Based Stereo

Silhouette Term

Color Difference Term

Results

Results

Results on Flash Data Ground Truth Monocular Stereo

Acknowledgment Volker Blanz Sami Romdhani Brian Amberg Jaen Sabastian Pierrard