JP2000090191A - Device and method for face recognition - Google Patents

Abstract

PROBLEM TO BE SOLVED: To precisely identify a person from an image obtained by photographing the face. SOLUTION: Deformation is removed from the face image by applying a perturbation method. Namely, combinations of parameters based upon affine transformation of deformation factors of the face images such as the position shift, tilt, up/down direction, and right/left direction of the face are prepared as many as all assumed variations of deformation. For the inputted face image, reverse transformation 31 of deformation using many combinations of parameters is performed at the same time in parallel. Then each discriminator 33 compares a normalized face pattern obtained by the reverse transformation 31 with the various prepared face patterns to calculate similarities. Lastly, the discrimination results of respective discriminators 33 are put together and the person having the largest similarity is selected (37).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a face image recognition apparatus used for personal authentication using a face image.

[0002]

2. Description of the Related Art There is known a face image recognition technique in which image data obtained by photographing a human face part from a camera is input and a personal authentication is performed by a computer. FIG. 1 shows a general flow of this fiber recognition method. That is, (1) input: a face is photographed by a camera to obtain face image data. (2) Preprocessing: Determine the exact position of the face from the input image, extract (cut out) the face pattern, and normalize the size of the face pattern.
(3) Feature extraction: Features are extracted from each face pattern using information based on appearance. (4) Identification: An individual is identified by comparing a face pattern obtained from an input image with a previously prepared face pattern.

In the pre-processing described above, it is not sufficient to simply detect and cut out a face pattern, and it is necessary to perform accurate position and size normalization on the face pattern. This is because, at the time of photographing, a minute change in the direction of the face, a change in the inclination of the face, a change in the expression, and the like occur, and as a result, the face pattern changes. As a conventional face pattern recognition technology, a standard face pattern is prepared, compared with a partial region of an image with some similarity such as a square error, and a region that exceeds a predetermined threshold is determined as a face. And improved techniques thereof have been commonly used. In this specification, such a method is referred to as a “standard matching method”.

[0004]

In the conventional face image clipping method, the position and size of a face in a general image are normalized using an average face. Because there are individual differences, the positions of the eyes and nose are often shifted from the average face. In addition, a human face does not always face a certain direction. For this reason, the conventional standard face matching method has a problem that it is not always possible to accurately detect the position and size of a face.

As a face detection method other than the standard face matching method, there is a technique of detecting the position of a face part using a template such as an average eye and detecting the size and position of the face from the relative positional relationship. However, there is a problem that a template such as an eye or a nose becomes a small image and is easily confused with a variety of background objects.

Accordingly, it is an object of the present invention to enable more accurate face recognition by a computer than in the past.

It is another object of the present invention to remove a deformation from a face image so that a standard face pattern can be accurately extracted.

[0008]

SUMMARY OF THE INVENTION The present invention relates to a perturbation method (Pert method).
The present invention provides a face recognition method that applies a technique called urbation method to a face image. Originally, the perturbation method is a technique used for character recognition, and is a technique for absorbing various deformations by applying an inverse transformation to a previously assumed deformation to an input character (for example, “Thien M. HA and Horst BUNKE, "Handw
ritten NumeralRecognition by Perturbation Method ",
Proc. Of the Fourth Int. Workshopon Frontiers of
Handwriting Recognition, Taipei, Taiwan, Dec. 7-9,
1994, pp. 97-106). This perturbation method is
Conventionally, it is only known as a technique for character recognition.

[0009] Characters and faces have greatly different causes and modes of deformation. In particular, while a character is a two-dimensional object, a face is a three-dimensional object, and its posture varies in a complicated manner in a three-dimensional space. In addition, the deformation of the face includes a troublesome element called expression. Since such a character and a face have greatly different modes of deformation, the idea of applying the perturbation method in the field of character recognition to face recognition is extremely novel. The present invention provides a novel method for applying the perturbation method to a face image under this novel idea.

According to the present invention, in the process of removing the deformation of the face image using the perturbation method, an operation of applying the inverse transformation including the deformation element due to the rotation of the face about an axis parallel to the screen to the face image is performed. As a result, it is possible to remove a deformation element due to a three-dimensional difference in face direction. For example, a deformation element for enlarging or reducing the face image upward or downward (that is, a deformation element due to rotation about a vertical axis parallel to the screen) and a deformation element for enlarging or reducing right or left (i.e., , A transformation element due to the rotation of the face about the horizontal axis parallel to the screen) to remove the transformation element due to the difference in the vertical direction of the face and the transformation element due to the difference in the horizontal direction. Can be.

The deformation of a face image can be classified into factors according to causes: (1) a difference in face position, (2) a difference in size of a face reflected on a camera, and (3) a face orientation (up and down). (4) due to differences in face inclination, and (5) due to differences in facial expressions. Then, assuming that the face is cut out, for example, in a rectangular area, deformation elements of the face image other than (5) can be removed by the following inverse transformation operation. In case (1),
Taking into account the parallel movement of the rectangular area, a rectangular area translated from the standard position is cut out. In the case of (2), taking into account the enlargement / reduction of the rectangular area, a rectangular area enlarged / reduced from the standard size is cut out, and this rectangular area is converted into the standard size. In the case of (3), taking into account the rotation of the rectangular area around an axis parallel to the screen, for example, a trapezoidal area deformed based on perspective is cut out, and this trapezoidal area is converted into a rectangle. In the case of (4), taking into account the rotation of the rectangular area around the vertical axis of the screen, a rectangular area rotated from a standard direction is cut out and converted into a non-rotated rectangular area. The actual deformation of the image can be considered as a combination of the deformation elements (1) to (4). Thus, in a preferred embodiment, a combination of inverse transforms for removing the deformed elements (1) to (4) is applied to the face image.

Further, according to the present invention, a plurality of inverse transformation operations for applying different transformations to a face image are performed independently, identification is performed using a pattern obtained by each inverse transformation, and the plurality of inverse transformation operations are performed. Based on the result of the identification (for example, selecting the individual with the highest similarity)
Determine the final identification result. According to this method, it is possible to reduce an erroneous recognition rate caused by a change in facial expression, particularly, the fact that the eyes are closed at the time of photographing (this is the main cause of erroneous recognition in the related art). In other words, from among a plurality of face patterns obtained by removing different expected deformations, the pattern that best matches the reference pattern and the light and shade pattern of the skin is selected. This is because they can be recognized without being affected.

In a preferred embodiment, a combination of affine transformation parameters that define the deformation elements (1) to (4) is prepared in advance for all possible variations of the deformation, and a large number of parameters are prepared. Are performed simultaneously and in parallel using the combination of.

Although the technique of the present invention can be typically implemented by a computer, a computer program for this can be installed or loaded into the computer through various media such as a disk storage, a semiconductor memory, and a communication network. it can.
Part or all of the recognition method of the present invention can be implemented by dedicated hardware without using a computer.

[0015]

FIG. 2 shows an overall configuration of an embodiment of the present invention.

In FIG. 1, an external device 1 is, for example, a CCD.
It is a camera and inputs face image data to the face image recognition device 2. The face image recognition device 2 cuts out a face pattern from an image input from the external device 1, extracts features, and identifies an individual. The face image used in the face recognition is not a black and white binary image as in the case of character recognition but a gray scale image of, for example, 256 gradations expressing the shade of the skin. The output device 3 is, for example, a CRT or a printer, and displays the identification result.

In the face image recognition device 2, image data from the external device 1 is read by the image reading unit 21 and sent to the preprocessing unit 22. In the preprocessing unit 22, a face pattern is cut out and normalized according to the teaching of the present invention. The normalized face pattern is sent to the feature extraction unit 23. In the feature extraction unit 23, the face pattern is dimensionally compressed into a low-dimensional feature vector that sufficiently expresses the distant face of a different person and absorbs the influence of a change in appearance as much as possible. As a method of this feature extraction, for example, “M.Turk
and A. Pentland, "Eigenfaces for recognition", JC
ognitive Neurosci, vol.3, no.1, pp.71-86, March 1
The technology described in "991" can be used. The feature vector obtained by the feature extraction unit 23 is sent to the identification unit 24. The identification unit 24 compares each of the feature vectors of various persons stored in the feature storage unit 25 with the feature vectors obtained from the input image, and specifies the individual of the input image. As a method of this identification, for example, a method of selecting a feature angle or the smallest one between two feature vectors can be adopted.

Next, the specific processing of the face image recognition device 2 will be described with reference to FIG.

As shown in FIG. 3, first, a grayscale face image of a face is input (input processing 30). Next, a large number of deformation parameters based on the affine transformation in consideration of all expected deformations are prepared in advance, and a face pattern is cut out from the input image using each of the deformation parameters, and the cut out face pattern is inversely transformed. Then, a square normalized pattern is obtained (inverse transformation 31). Next, a feature vector is extracted from the normalized face pattern obtained by each of the inverse transforms 31 and compared with reference feature vectors of various individuals prepared in advance to calculate a similarity. The ID of the individual having the highest degree and the similarity thereof are output (identifier 33). Next, the similarity output from each discriminator 33 is multiplied by a weight coefficient based on the frequency of occurrence of the deformation removed by the corresponding inverse transform 31 (multiplication 35).

The significance of the multiplication 35 is as follows. That is, not all expected deformations actually occur at the same frequency. Generally, a deformation having a small degree of deformation (for example, slightly upward or slightly tilting right) has a high frequency of occurrence, but has a high degree of deformation. (For example, large upwards, large inclination to the right, etc.) are low in occurrence frequency. Therefore,
By applying an inverse transformation with a large degree of deformation to a face image, the possibility that the face is transformed into a face close to another person and erroneously identified as another person is higher than in the case of performing an inverse transformation with a small degree of deformation. Therefore, for all the expected deformations, a weighting coefficient based on each occurrence frequency (a coefficient value is smaller as the occurrence frequency is lower) is prepared in advance together with each deformation parameter, and is output from each discriminator 33. The similarity of the most similar individual is calculated using the inverse transformation 3
By multiplying by the weight coefficient based on the frequency of occurrence of the deformation removed in step 1, the influence of the identification result erroneously identified by another person can be reduced, and the final recognition accuracy can be improved.

The inverse transformation 31, the identification 33, and the multiplication 35 for each of the above-described modifications are performed simultaneously and in parallel as shown in the figure. Note that, as shown in the center of the figure, a routine that does not perform inverse conversion is also performed at the same time assuming that there is no deformation.

Finally, these identification results are collected to determine whether the result is a specific individual or not as an overall result (combination 37). Various variations are conceivable for the determination method here. For example, the plurality of multipliers 35 described above
From the identification results collected from the plurality of multipliers 35, or a method of selecting the individual who appeared in the largest number of identification results by majority decision of the identification results collected from the plurality of multipliers 35, or Although this is also a kind of majority decision, the similarities included in the identification results collected from the plurality of multipliers 35 are totaled for each individual, and the individual with the highest total similarity is selected, while the predetermined value or more is selected. If there is no individual who has obtained the similarity of, there is a method of determining that there is no corresponding person. Note that the multiplication 35 may be omitted, and the weight coefficient used in the multiplication 35 may be taken into account in the combination 37.

When the processing steps of FIG. 3 correspond to the configuration of FIG. 2, the input processing 30 is performed by the image reading unit 21 of FIG.
The inverse transform 31 is provided to the pre-processing unit 22 shown in FIG.
Corresponding to

Next, the inverse transformation 31 performed by the preprocessing unit 22 will be specifically described.

When the deformation of the face image is decomposed into elements according to the causes, (1) the face displacement, (2) the difference in the size of the face reflected on the camera, (3) the face orientation (up, down, left, right) Orientation), (4) differences in facial inclination, and (5) differences in facial expressions. Now, when trying to cut out the face from the input image in a rectangular area, the deformation elements of the face image other than (5) can be removed by the following method. That is, in the case of (1), taking into account the parallel movement of the rectangular area, the rectangular area translated from the standard position is cut out. In the case of (2), taking into account the enlargement / reduction of the rectangular area, a rectangular area enlarged / reduced from the standard size is cut out, and this rectangular area is converted into the standard size. In the case of (3), taking into account the rotation of the rectangular area around the axis parallel to the screen, for example, a trapezoidal area deformed based on perspective is cut out, and this trapezoidal area is converted into a rectangle. In the case of (4), taking into account the rotation of the rectangular area around the axis perpendicular to the screen, a rectangular area rotated from a standard direction is cut out and converted into a non-rotated rectangular area.

FIG. 4 shows this principle more specifically. The pattern 41 shown on the left side of FIG. 4 is a standard face pattern that can be cut out as a square area without deformation. The deformed patterns 51A and 51B shown in (a) on the right side of FIG. 4 take into account the inclination of the face to the left and right, and rotate by a predetermined angle around the axis (z axis) perpendicular to the screen with respect to the standard pattern 41. are doing. The deformation patterns 53A and 53B in (b) take into account the size of the reflection of the face, and are enlarged or reduced in the horizontal or vertical direction with respect to the standard pattern. (c)
The deformation patterns 55A and 55B take into account the rotation of the face to the left and right (to the right and to the left). Deformation, that is, reduction or enlargement biased rightward or leftward. (d) deformation pattern 57
A and 57B consider the vertical rotation (upward and downward) of the face, and the deformation when the standard pattern 41 is rotated around a horizontal axis (x-axis) parallel to the screen, that is, , Or a reduction or enlargement biased upward or downward. Although not shown in FIG. 4, the deformed pattern in consideration of the face misalignment is obtained by translating the standard pattern 41 in the vertical and horizontal directions (y and x directions).

Such a deformation pattern for each deformation element is as follows:
It can be defined using affine transformation parameters. The actual deformation of the image is a combination of the deformation elements (1) to (4) described above. Therefore, a set of affine transformation parameters obtained by synthesizing the patterns of the deformation elements (a) to (d) shown in FIG. For example, x, y,
If the rotation angles around the z-axis are α, β, γ, the amounts of parallel movement in the x, y, and z-axis directions are x, y, z, and the length-to-width ratio is r, α, β, γ, x , Y, z, and r, a variable range is set, and each parameter is gradually changed within the variable range, so that a large number of parameter sets {α, β respectively corresponding to all expected deformation variations. , Γ, x, y, z, r} are prepared. Then, as shown in FIG. 3, by performing a number of inverse transformations 31 using each of the number of parameter sets simultaneously and in parallel, a number of normalizations removing each of the expected deformations are performed. For the completed face pattern. Then, each of the normalized patterns is compared with the pattern of various persons by each discriminator 33 to calculate the similarity. Finally, a large number of the identification results are collected, and a person who obtains, for example, the highest similarity is selected as a final identification result.

According to the above method, a constant face pattern can be cut out without being influenced by subtle changes in the pattern due to the direction and inclination of the face and the expression (particularly the opening and closing of the eyes), so that the recognition accuracy can be improved. .

The embodiment of the present invention has been described above.
This embodiment is merely an example for explaining the present invention, and is not intended to limit the present invention to only these embodiments. Therefore, the present invention can be implemented in various modes other than the above-described embodiment.

[Brief description of the drawings]

FIG. 1 is a flowchart showing a general flow of face recognition.

FIG. 2 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 3 is a flowchart illustrating a processing flow of the face image recognition device.

FIG. 4 is an explanatory diagram for explaining a principle of removing deformation of a face pattern.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 External device 2 Face image recognition device 3 Output device 21 Image reading part 22 Preprocessing part 23 Feature extraction part 24 Identification part 25 Feature storage part 30 Input 31 Inverse transformation 33 Classifier

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Akira Sakano 3-3-3 Toyosu, Koto-ku, Tokyo F-term in NTT Data Corporation (reference) 5B043 AA09 BA04 CA01 EA05 EA12 EA13 EA15 GA02 5B064 AA10 CA05 CA07 CA11 DA10 EA30

Claims (13)

[Claims] A plurality of operable independently of each other, each of which receives the same face image and outputs a normalized face pattern by performing inverse transformation for removing different deformations on the face image. An inverse transform unit, and inputs the normalized face patterns output from each of the plurality of inverse transform units, and calculates the similarity by comparing each face pattern with a reference pattern of a plurality of persons prepared in advance. An identification unit, and a combining unit that collects identification results of the face patterns from the plurality of inverse conversion units by the identification unit, and specifies a person corresponding to the face image based on the identification results, The transformation that each inverse transform unit intends to remove is composed of a combination of a plurality of kinds of transformation elements, and the plurality of kinds of transformation elements include a transformation element due to rotation of the face around an axis parallel to the screen. Recognition equipment Place. 2. The face recognition device according to claim 1, wherein the plurality of types of deformation elements further include a deformation element due to a position shift, a difference in inclination of the face, and a difference in size reflected on a camera. 3. The deformation element due to the rotation of the face about an axis parallel to the screen includes a deformation element about a vertical axis parallel to the screen and a deformation element about a horizontal axis parallel to the screen. Item 3. The face recognition device according to item 1. 4. Each of the inverse transform units has a set of affine transformation parameters corresponding to a combination of the plurality of deformation elements, each having a different parameter set,
2. The face recognition apparatus according to claim 1, wherein a deformed face pattern is cut out from the face image using the parameter set and inverse transformation is performed. 5. The face recognition device according to claim 1, wherein the plurality of inverse transform units operate simultaneously and in parallel. 6. The face recognition apparatus according to claim 5, further comprising a plurality of identification units respectively corresponding to the plurality of inverse conversion units, wherein the identification units operate simultaneously and in parallel. 7. The combination unit according to claim 1, wherein a person having the highest similarity is specified based on the collected identification results.
The face recognition device according to the above. 8. The face recognition apparatus according to claim 1, wherein the combining unit specifies a person by a majority decision method based on the collected identification results. 9. The similarity calculated for the normalized face pattern from each of the inverse transform units by using a weighting factor based on the occurrence frequency prepared in advance for the deformation to be removed by each of the inverse transform units. The face recognition device according to claim 1, wherein the degree is corrected. 10. A plurality of independent face images each of which receives the same face image and outputs a normalized face pattern by subjecting the face image to inverse transformation for removing different deformations. A conversion step, and inputting each of the normalized face patterns output from each of the plurality of inverse conversion steps, and comparing each face pattern with a reference pattern of a plurality of persons prepared in advance to calculate a similarity. An identification step, and a combination step of collecting identification results of the identification step for the face pattern from the plurality of inverse transformation steps and identifying a person corresponding to the face image based on the identification results, The deformation to be removed in each inverse transformation step consists of a combination of a plurality of types of deformation elements, and the plurality of types of deformation elements include a face around an axis parallel to the screen. A face recognition method that includes a deformation element due to the rotation of. 11. The face recognition method according to claim 10, wherein the plurality of types of deformation elements further include a deformation element due to a positional shift of the face, a difference in inclination, and a difference in size reflected on a camera. 12. The face recognition method according to claim 10, wherein the plurality of inverse transformation steps are performed simultaneously and in parallel. 13. A plurality of independent face images each of which receives the same face image and outputs a normalized face pattern by subjecting the face image to inverse transformation for removing different deformations. A conversion step, and inputting each of the normalized face patterns output from each of the plurality of inverse conversion steps, and comparing each face pattern with a reference pattern of a plurality of persons prepared in advance to calculate a similarity. An identification step, and a combination step of collecting identification results of the identification step for the face pattern from the plurality of inverse transformation steps and identifying a person corresponding to the face image based on the identification results, The deformation to be removed in each inverse transformation step consists of a combination of a plurality of types of deformation elements, and the plurality of types of deformation elements include a face around an axis parallel to the screen. A computer-readable recording medium carrying a program for causing a computer to execute a face recognition method including a deformation element due to rotation of a computer.