Blending 2D and 3D Face Recognition

Over the last decade, performance of face recognition algorithms systematically improved. This is particularly impressive when considering very large or challenging datasets such as the FRGC v2 or Labelled Faces in the Wild. A better analysis of the structure of the facial texture and shape is one of the main reasons of improvement in recognition performance. Hybrid face recognition methods, combining holistic and feature-based approaches, also allowed to increase efficiency and robustness. Both photometric information and shape information allow to extract facial features which can be exploited for recognition. However, both sources, grey levels of image pixels and 3D data, are affected by several noise sources which may impair the recognition performance. One of the main difficulties in matching 3D faces is the detection and localization of distinctive and stable points in 3D scans. Moreover, the large amount of data (tens of thousands of points) to be processed make the direct one-to-one matching a very time-consuming process. On the other hand, matching algorithms based on the analysis of 2D data alone are very sensitive to variations in illumination, expression and pose. Algorithms, based on the face shape information alone, are instead relatively insensitive to these sources of noise. These mutually exclusive features of 2D- and 3D-based face recognition algorithm call for a cooperative scheme which may take advantage of the strengths of both, while coping for their weaknesses. We envisage many real and practical applications where 2D data can be used to improve 3D matching and vice versa. Towards this end, this chapter highlights both the advantages and disadvantages of 2D- and 3D-based face recognition algorithms. It also explores the advantages of blending 2D- and 3D data-based techniques, also proposing a novel approach for a fast and robust matching. Several experimental results, obtained from publicly available datasets, currently at the state of the art, demonstrate the effectiveness of the proposed approach.