Morphing Faces Experiment

Part 1: Defining Correspondence Points

To morph two images, correspondence points must be defined on each image in the same order. I used the cpselect tool in MATLAB to select points on a portrait of myself and a portrait of the famous Chinese movie star, Daniel Wu. Each image has 48 selected points.

Portrait 1 Portrait 2 Corresponding Points

Using these points, I computed the triangulation for the midway face between the two images.

Triangulation Result

Part 2: Computing the Mid-Way Face

To create a mid-way face between two different faces:

  1. Compute the average shape, which is the mean of each set of corresponding points.
  2. Warp both faces into this average shape.
  3. Average the colors of the warped faces (cross-dissolve).

For the first step, the average shape is just the average of each correspondence—the mean correspondences that we used for a Delaunay triangulation.

For warping, I iterated through all triangles in the original images. For each triangle, I computed its bounding rectangle using cv2.boundingRect, created a mask to isolate the triangle, and warped this to the target location. The triangle's effective part was then extracted using the mask and added to the target image.

After morphing the shapes, the colors of the two images are averaged to create the mid-way image.

Mid-Way Face

Part 3: Creating a Morph Sequence

To create a morph sequence from one face to another, the process from Part 2 was repeated with varying fractions of warping and cross-dissolving. The parameter controlling the amount of warping and cross-dissolving varied from 0 to 1, with a step size of 1/30.

Morph Sequence GIF

Part 4: Utilizing the FEI Face Database

The photographs used are from the FEI face database, which contains images of 200 individuals from the Artificial Intelligence Laboratory of FEI in São Bernardo do Campo, São Paulo, Brazil. The database is divided into neutral faces and expression subsets.These images are annotated with 46 correspondences.

To compute the mean face of a population:

  1. Find the average shape by averaging all correspondences.
  2. Warp each face into this average shape.
  3. Compute the average color from all warped faces to derive the mean face.

Below are examples of faces from the dataset, warped into the average shape for both neutral and expression subsets:

Warped Face 1 Warped Face 2 Warped Face 3 Warped Face 4 Warped Face 5 Warped Face 6

Here are the average neutral and expression faces:

Average Neutral Face Average Expression Face

Here is how my face looks when warped into the average geometry (a little bit ugly).

Part 5: PCA Analysis

Using the dataset and the computed average face, I conducted a PCA analysis to obtain the principal components for neutral and smiling faces, displaying the first eight components:

PCA Component of Neutral Faces PCA Component of Smiling Faces

To reconstruct my average face, I projected it onto the principal component space. As more principal components are used, the reconstructed face increasingly resembles the original image:

Reconstructed Face