EP2497052A1

EP2497052A1 - Method for illumination normalization on a digital image for performing face recognition

Info

Publication number: EP2497052A1
Application number: EP09835896A
Authority: EP
Inventors: Ngoc-Son Vu; Alice Caplier
Original assignee: Institut Polytechnique de Grenoble; VESALIS
Current assignee: Wisimage; Institut Polytechnique de Grenoble
Priority date: 2009-11-06
Filing date: 2009-11-06
Publication date: 2012-09-12
Also published as: WO2011055164A1

Abstract

A method for performing illumination normalization on a digital image, comprising : a) receiving an input image; b) determining, for each pixel, a first adaptation factor F1(p); c) applying to said input image a first adaptive non linear function using said first adaptation factor F1(p) to get a first modified image (Ila1); d) determining, for each pixel, a second adaptation factor F2(p) using the first modified image (Ila1); e) applying to the resulting image (Ila1) of step c a second adaptive non linear function using said second adaptation factor F2(p) to get a second modified image (IIa2); f) applying to the resulting image (Ila2) of step e a Difference of Gaussians (DoG) filter to get a normalized image (In). A face recognition system and a method for performing face recognition are also provided.

Description

METHOD FOR ILLUMINATION NORMALIZATION ON A DIGITAL IMAGE FOR PERFORMING FACE RECOGNITION

FIELD OF THE INVENTION

[0001]The present invention relates to a method for performing illumination normalization on a digital image, a face recognition system and a method for performing face recognition.

BACKGROUND OF THE INVENTION

[0002] Illumination variations that occur on face images degrade dramatically the performance of face recognition systems. Methods developed in order to overcome the illumination problem can be divided into three categories: illumination invariant features extraction, illumination modeling and illumination variation removing.

[0003]The first approach seeks illumination invariant features to represent face images. Examples of such features include edge maps, image intensity derivation and images convolved with 2D Gabor-like filters.

[0004] However, none of these representations is sufficient by itself to overcome illumination variations because of changes in the illumination direction. Methods in the second category use multiple images of each person under various lighting conditions to learn an appropriate model of illumination variations. Generally, methods belonging to this category can achieve good recognition results. However, they require many images captured under different lighting conditions for each subject. Such methods are not adapted to many applications such as video surveillance for example. The third approach transforms image to canonical form in which the illumination variations are erased. Example of such techniques are Histogram Equalization (HE), Gamma Correction (GC) or recent methods based on retinex theory.

[0005] In retinex theory, an image l(x, y) is modeled as the product of the reflectance R and the illumination L. The problem of obtaining R from an input image I can be solved by estimating L. Several methods have been presented to estimate L, such as Single Scale Retinex (SSR), Multi Scale Retinex (MSR) and self-quotient image (SQI). However, these methods still can not exactly estimate L so that large illumination variations are not completely removed.

SUMMARY OF THE INVENTION

[0006] It is therefore an object of the present invention to provide a method and a device adapted to remove or reduce illumination variations in digital images, and more particularly to improve the performances of face recognition systems and methods.

[0007] It is another object of the invention to provide a method that is usable in any type of application or any type of environment.

[0008] It is a further objet of the invention to provide a face recognition system enabling to perform face recognition in video surveillance applications, in various lightning conditions.

[0009]According to the invention, these aims are achieved by means of a method for performing illumination normalization on a digital image, comprising :

a) receiving an input image ;

b) determining, for each pixel , a first adaptation factor F1(p) ;

c) applying to said input image a first adaptive non linear function using said first adaptation factor F1(p) to get a first modified image (Ila1) ;

d) determining, for each pixel, a second adaptation factor F2(p) using the first modified image (Ila1) ; e) applying to the resulting image (Ila1) of step c a second adaptive non linear function using said second adaptation factor F2(p) to get a second modified image (Ha2) ;

f) applying to the resulting image (Ila2) of step e a Difference of Gaussians (DoG) filter to get a normalized image (In).

[0010]The method is an improvement of the retina filter to normalize illumination. The proposed method combines two adaptive nonlinear functions and a Difference of Gaussians filter. This can be related to the performance of two layers of the retina: the photoreceptors and the outer plexiform layer.

[0011]The different steps of the disclosed method enables removing illumination variations, noise but also enhance the image edges.

[0012] When used for face recognition, the modified image is further applied to the task of face detection. The resulting recognition performances are considerably enhanced.

[0013] In a preferred embodiment, the method further comprises a first dynamic range reseating step on the normalized image (In) using a zero-mean normalization to get a reseated normalized image (Rln).

[0014] In a further embodiment, the method further comprises a truncation step on the rescaled normalized image (Rln) in order to enhance the image contrast and get a truncated rescaled normalized image (TRIn). This step is advantageous in order to adjust the image data with a median value corresponding for instance to 0. The truncation threshold level is thus easier to determine.

[0015]The method preferably comprises a second dynamic range rescaling step on the truncated rescaled normalized image (Trln) to get a final normalized image (Ifn). [0016] In a preferred embodiment, at least one adaptive nonlinear function, and preferably both operations, is applied in order to perform a light adaption filter.

[0017]ln an aspect of the invention, the first adaptation factor F1(p) depends on the intensity of the input image and the second adaptation factor F2(p) depends on the intensity of the first modified image (Ila1).

[0018] In another aspect of the invention, the first adaptation factor F1(p) for a given pixel depends on the intensity of said pixel and on the mean value of the input image, the second adaptation factor F2(p) for a given pixel depends on the intensity of said pixel and on the mean value of on the first modified image.

[0019] In a further aspect of the invention, the first adaptation factor F1(p) is based on a two dimensional Gaussian low pass filter with a first standard deviation Si. ,.

[0020] In a still further embodiment, the second adaptation factor F2(p) is based on a two dimensional Gaussian low pass filter with a second standard deviation s₂.

[0021] In a still further embodiment, Si and s₂ values are comprised between 0,3 and 5. In a prefered variant, si value is set to 1 and s₂ value is set to 3.

[0022] In a still further embodiment, the Difference of Gaussians (DoG) filter uses two low pass filters having standard deviations s_Ph and Sh values comprised between 0,1 and 5. In a preferred variant, s_ph value is set to 0,5 and s_h value is set to 4.

[0023] In a further variant, each two dimensional Gaussian low pass filter is preferably replaced by two one dimensional Gaussian low pass filters (the horizontal and vertical ones) with the same standard deviation. This allows reducing the processing time.

[0024] In a preferred embodiment, the input digital image includes a human face representation.

[0025] In another aspect, the invention also provides a face recognition system, comprising:

a) a facial database with data of known individuals;

b) an input for providing a facial image of unknown subject;

c) said data in step a and/or b being pre-processed using the illumination normalization as previously described;

d) means for representing the reference set of images of individuals as a set of reference feature vectors;

e) means for representing said input facial image as a query feature vector;

f) means for calculating, for each vector in the reference set, its similarity score with said query;

g) means for providing a recognition signal based on the set of similarity scores .

[0026] The image variations of the same face due to illumination are often larger than the variations due to the change of face identity. Therefore, illumination normalization steps making the intra-person faces compact by removing all illumination variation of images improves the recognition performances.

[0027]The invention further provides a method for performing face recognition, comprising:

a) providing a facial database with data of known individuals;

b) providing an input facial image of unknown subject;

c) pre-processing said data of step a and/or b using the illumination normalization method previously described;

d) representing the reference set of images of individuals as a set of reference feature vectors;

e) representing said input facial image as a query feature vector;

f) calculating, for each vector in the reference set, its similarity score with said query;

g) providing a recognition signal based on the set of similarity scores. BRIEF DESCRIPTION OF THE DRAWINGS

[0028]The foregoing and other purposes, features, aspects and advantages of the invention will become apparent from the following detailed description of embodiments, given by way of illustration and not limitation with reference to the accompanying drawings, in which:

-Figure 1 presents the main steps of an illumination normalization method according to the invention;

-Figure 2 presents the main steps relating to the light adaptation filter;

-Figure 3 presents the main steps of a known face recognition method;

-Figure 4 presents the main steps of a method for performing face recognition with specific steps involving illumination normalizations.

DETAILED DESCRIPTION OF THE INVENTION

[0029] Basically, the retina is made of three layers: the photo receptors layer with cones and rods; the Outer Plexiform Layer (OPL) with horizontal, bipolar and amacrine cells; the Inner Plexiform Layer (IPL) with ganglion cells. The goal is not to precisely model the dynamics of retinal processing, but rather to identify which processing acts on the retinal signal for illumination normalization.

[0030] Light adaptation filter rods and cones have quite different properties: rods have the ability to see at night, under conditions of very low illumination; cones have the ability to deal with bright signals. But both photoreceptors are sensible to light variations and play a crucial rule as light adaptation filter. To exploit this property, an adaptive nonlinear function could be applied on the input signal.

[0031] Photoreceptors perform not only as a light adaptation filter but also as a low pass filter. Horizontal cells perform the second low pass filter. In OPL, bipolar cells calculate the difference between photoreceptor and horizontal cell responses. Then, bipolar cells act as a band pass filter: they remove high frequency noise and low frequency illumination. To simulate the processes of OPL, two Gaussian low pass filters with different standard deviations corresponding to the effects of photoreceptors and horizontal cells are used.

[0032] Finally, bipolar cells are simulated with a Difference of Gaussians filter (DoG), to enhance the image edges.

[0033]As mentioned above, a model with a nonlinear operation and a DoG filter can be used for variation illumination removing. In this model, two consecutive nonlinear operations are used for a more efficient light adaptation filter and a truncation is used to enhance the global image contrast.

[0034] Duplex nonlinear operations act as an efficient light adaptation filter. Then, two consecutive adaptive nonlinear functions are applied. The adaptation factor of the first non-linear function is preferably computed for each pixel by performing a low pass filter on the input image as follows: where p is the current pixel; F1(p) is the adaptation factor at pixel p; Iin is the intensity of the input image; ^* denotes the convolution operation; Iin is the mean value of the input; and G1 is a 2D Gaussian low pass filter with standard deviation ah

[0035]The input image is then processed according to a Naka-Rushton type equation using the adaptation factor F1: ϊΐαιίρ) = (/iniraa-e) + i¾(p)} [0036] The term lin(max) + F1(p) is a normalization factor where lin(max) is the maximal value of the image intensity. The second nonlinear function works similarly, the light adapted image Ila2 is obtained by: in which 2(p)

[0037]An advantage of this light adaptation filter is that the image Ila2 does not change with different low pass filter sizes. In a preferred embodiment, σΐ and σ2 are set to 1 and 3 respectively.

[0038] The image Ila2 is then transmitted to bipolar cells and processed by using a difference of Gaussians (DoG) filter, as follows: in which DoG is given by: the terms oph and OH correspond to the standard deviations of the low pass filters modeling photoreceptors and horizontal cells. In a preferred embodiment, they are set to 0.5 and 4 respectively. A zero-mean normalization is used in the next step to rescale the dynamic range of the image. The substraction of the mean μ/b/p is not necessary because it is near to 0.

[0039]A drawback of DoG filter is an inherent reduction in overall image contrast. Therefore, a preferred final step is recommended, to enhance the image contrast: the extreme values are removed by a truncation with a threshold Th which is set to 5 in this preferred embodiment.

[0040] In comparison with other methods, the method if the invention is more robust to illumination variations and is of lower complexity. Only one image per subject is required for the training set, the used techniques are simple. The calculation requiring the highest processing resources is the convolution operation with Gaussian kernel. Supposing that the size of a normalized image is m x n. By replacing one 2D Gaussian calculation by two independent 1 D Gaussian ones, the computational complexity of algorithm is O (mnw) where w is the size of 1D Gaussian kernel and w = 6σ in this example.

[0041]As the method is of low complexity, it can be applied as a preprocessing technique to real-time applications such as video surveillance.

[0042] Figure 1 illustrates the main steps of the illumination normalization method. Steps 1 to 9 present an example of a preferred embodiment. Step 1 relates to the acquisition of an image to be processed. Image 10 illustrates an example of such an image. Step 2 relates to a light adaptation filter, detailed in Figure 2. Image 20 illustrates the evolution of the input image due to the light adaptation filter. Steps 3 and 4 correspond to Gaussian low pass filters. Images 30 and 40 illustrate how the processed images may be modified in each corresponding step. [0043] Step 5 and image 50 illustrate the bipolar cell step. Step 6 and image 60 correspond to a dynamic range rescaling. Image 70 is related to steps 7 truncation, 8 rescaling and 9 output. In fact, in most cases, the results of the calculations preformed in these steps are usually not visible, even when considering the data modification from step to step.

[0044] In Figure 2, the light adaptation filter steps are presented with more details. In step 21, the image 10 is received as an input. Step 22 and image 220 relate to a first adaptation factor. Step 23 and image 230 relate to a first light adapted image. Step 24 and image 240 relate to a second adaptation factor. Step 25 and image 250 relate to a second light adapted image. The resulting image 20 is shown once again in step 2 of Figure 1.

Figure 3 illustrates the main steps of a know type face recognition process. Figure 4 presents the main steps of a face recognition process involving illumination normalization. Data relating to an unknown subject and data relating to known individuals are used in input steps 201 and 202. The data related to the facial image to be detected are processed with illumination normalization as previously described. In a preferred embodiment, the data related to the known individuals are also processed with illumination normalization, though these data could be processed under a different approach or not processed according to variants. For instance, the reference database could be build with images acquired under standard optimized light conditions, whereas those of the subject to be detected are obtained under various light conditions, requiring further processing to normalize illumination.

The further steps are typical face recognition steps, such as feature extraction 206, vector transformation of query 207 and reference 208. Similarity scores and then calculated in step 209. Recognition results are obtained in step 210.

[0045] The proposed algorithm has been tested and evaluated on the Yale B database, the Feret illumination database by using two face recognition methods: PCA based and Local Binary Pattern based (LBP). Experimental results show that the proposed method achieves very high recognition rates even for the most challenging illumination conditions. Moreover, processing resources are optimized.

[0046]Those skilled in the art will also understand that the here above described face recognition approach can be modified, considering the fact that many variants of face recognition approaches are well known in the art. Such alterations, modifications and improvements are intended to be within the spirit and scope of the invention.

[0047]Accordingly, the foregoing description is by way of example only and is not intended to be limiting.

Claims

1. A method for performing illumination normalization on a digital image |, comprising :

a) receiving an input image ;

b) determining, for each pixel , a first adaptation factor F1(p) ;

d) determining, for each pixel, a second adaptation factor F2(p) using the first modified image (Ila1)| ;

e) applying to the resulting image (Ila1) of step c a second adaptive non linear function using said second adaptation factor F2(p) to get a second modified image (Ila2) ;

2. A method for performing illumination normalization on a digital image according to claim 1, further comprising a first dynamic range rescaling step on the normalized image (In) using a zero-mean normalization to get a rescaled normalized image (Rln).

3. A method for performing illumination normalization on a digital image according to claim 2, further comprising a truncation step on the rescaled normalized image (Rln) in order to enhance the image contrast and get a truncated rescaled normalized image (TRIn).

4. A method for performing illumination normalization on a digital image according to claim 3, further comprising a second dynamic range rescaling step on the truncated rescaled normalized image (Trln) to get a final normalized image (Ifn).

5. A method for performing illumination normalization on a digital image according to any one of preceding claims, wherein at least one adaptive nonlinear function, and preferably both operations, is applied in order to perform a light adaption filter.

6. A method for performing illumination normalization on a digital image according to any one of preceding claims, wherein the first adaptation factor F1(p) depends on the intensity of the input image and the second adaptation factor F2(p) depends on the intensity of the first modified image (Ila1).

7. A method for performing illumination normalization on a digital image according to any one of preceding claims, wherein the first adaptation factor F1(p) for a given pixel depends on the intensity of said pixel and on the mean value of the input image, the second adaptation factor F2(p) for a given pixel depends on the intensity of said pixel and on the mean value of on the first modified image.

8. A method for performing illumination normalization on a digital image according to any one of preceding claims, wherein the first adaptation factor F1(p) is based on a two dimensional Gaussian low pass filter with a first standard deviation Si.

9. A method for performing illumination normalization on a digital image according to any one of preceding claims, wherein the second adaptation factor F2(p) is based on a two dimensional Gaussian low pass filter with a second standard deviation S2..

10. A method for performing illumination normalization on a digital image according to claim 8 or 9, wherein Si and s₂ values are comprised between 0,3 and 5.

11. A method for performing illumination normalization on a digital image according any one of preceding claims, wherein said Difference of Gaussians (DoG) filter uses two low pass filters having standard deviations s_ph and s_h values comprised between 0,1 and 5.

12. A method for performing illumination normalization on a digital image according to any one of preceding claims, wherein said input digital image includes a human face representation.

13. A face recognition system, comprising:

a) a facial database with data of known individuals;

b) an input for providing a facial image of unknown subject;

c) pre-processing said data of step a and/or b using the illumination normalization method according to any one of claims 1 to 12;

e) means for representing said input facial image as a query feature vector;

g) means for providing a recognition signal based on the set of similarity scores.

14. A method for performing face recognition, comprising:

a) providing a facial database with data of known individuals;

b) providing an input facial image of unknown subject;

e) representing said input facial image as a query feature vector;

g) providing a recognition signal based on the set of similarity scores.