JP2008304979A - Face authentication device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that a photographic image must be corrected since a monocular camera has no effective method for photographing a collation image facing the same direction as a registered image. <P>SOLUTION: The device comprises a general face detector detecting faces having a difference of pose change larger than the registered image; a registered pose face detector for detecting a face of the same pose as the registered image from the faces detected by the general face detector; and a display device for displaying the detection states of the general face detector and the registered pose face detector to a user. According to this, the photographic image can be used for collation without correction. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to image recognition that is a technique for collating a person in a face image photographed by a camera with a person in a face image in a registered face image database.

  In the collation of a face image taken with a monocular camera and a face image in a pre-registered face image database, the difference in face orientation between the photographed face image and the face image in the registered image is large. This will cause the authentication performance to deteriorate. As a method for solving such a problem, a method of correcting a photographed face image or feature amount and a registered face image or feature amount and treating them as face-to-face matching in the same direction. There is.

  For example, there is a technique of synthesizing a pseudo registered face image when viewed from the shooting direction of a camera using a registered face image and a wire frame reference model (Patent Document 1). Further, there is a technique that corrects a feature amount extracted from a face image and corrects a feature amount obtained from a captured face image to a feature amount when viewed from the same direction as a registered face image (patent) Reference 2).

JP2003-263639 Japanese Patent Application No. 4-320236

  As described above, in the case of a method for correcting a registered face image or feature value to a face image or feature value of a face direction photographed by a camera, an error accompanying the correction occurs, and the original registered image The face image or feature that was not visible in the face image or collation image has a problem that it is difficult to create by correction, and in the case of a monocular camera and a single face model, there are various There is a problem that it is difficult to accurately correct the difference in human face orientation. In other words, the problem is that a front face image that does not require correction could not be properly obtained.

  The face authentication device according to the present invention includes a registered image feature amount extraction unit that extracts a feature amount from a registered image in which a person's face is registered, and a front face that takes the person's face image and has the same face orientation as the registered image. In the case of a front face, a registered pose face detection unit that outputs a face image to the collation image feature amount extraction unit, a feature amount extracted from the collation image feature amount extraction unit, and a registered image feature amount extraction unit The present invention is characterized in that a collation unit for collating the extracted feature amount is provided.

  According to the face authentication device of the present invention, a registered image feature amount extraction unit that extracts a feature amount from a registered image in which a person's face is registered, and a front face that takes the person's face image and has the same face orientation as the registered image. In the case of a front face, a registered pose face detection unit that outputs a face image to the collation image feature amount extraction unit, a feature amount extracted from the collation image feature amount extraction unit, and a registered image feature amount extraction Since the collation part which collates with the feature-value extracted from the part is provided, a front face image can be obtained appropriately.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing an object recognition apparatus according to the first embodiment. FIG. 2 is a diagram for explaining an example of a registered face image according to the first embodiment. Furthermore, FIG. 3 is explanatory drawing regarding the face which the general purpose face detector by Embodiment 1 detects. FIG. 4 is an explanatory diagram relating to the face detected by the registered pose face detector according to the first embodiment.

  The registered image database storage unit 6 stores a normalized face image obtained by cutting a face image (front face) taken from the front direction into a certain size, like the image of the registered image database in FIG. A normalized face image is stored with the size of the image and the positions of feature points such as eyes, nose and mouth fixed.

  The image input unit 1 inputs a face image taken with a monocular camera and sends the face image to the general-purpose face detection unit 2. The general-purpose face detection unit 2 detects a face area having a large face pose change including the front face from the image and sends it to the registered pose face detection unit 3 (see FIG. 3). The registered pose face detection unit 3 detects an image in the same face direction as the registered pose from the face area sent from the general-purpose face detection unit 2, that is, a front face (see FIG. 4). Taking FIG. 3 as an example, the third and third face images from the left are front faces.

  Also, the registered pose face detection unit 3 sends to the display device 4 a result indicating whether the face area sent from the general-purpose face detection unit 2 is the same front face as the registered image database. Further, in the case of the front face, the registered pose face detection unit 3 sends the face area determined to be the front face to the collation image feature amount extraction unit 5. Therefore, the matching image feature amount extraction unit 5 extracts feature points of the front face.

  The display device 4 uses the output results from the general-purpose face detection unit 2 and the registered pose face detection unit 3 to (a) non-detect a face, (b) a face in a direction different from the registered pose (= non-front face) Or (c) which of the three states of detecting the front face in the same direction as the registered pose is presented to the user. In addition to the two states where the front face is detected / non-detected on the display device 4, (a) a face is not detected, (b) a face is detected but a non-front face, and (c) a front face is detected. By presenting the user with three states having a front face, it is possible to detect a front face by interaction with the user.

  Therefore, by detecting the degree of matching of the face orientation with the registered image for the face area in the registered image, a more desirable face image for authentication can be selected for verification. Here, the general-purpose face detection unit 2 and the registered pose face detection unit 3 are distinguished from each other. Further, when the front face of (c) is detected, it is naturally possible to collate without outputting a signal to that effect to the display device 4 that is a display unit to the user.

  When the registered pose face detection unit 3 detects a front face, the collation image feature amount extraction unit 5 extracts a feature amount necessary for collation from the front face region sent from the registration pose face detection unit 3. The registered image input unit 9 extracts a registered image from the registered image database storage unit 6, and the registered image feature amount extraction unit 8 performs feature amount extraction from the extracted registered image. The collation unit 7 performs collation using the feature amount of the collation image and the feature amount of the registered image, and a collation result indicating whether the registered image and the person in the face image input from the image input unit 1 are the same person or different persons. Is output.

  FIG. 5 is a flowchart showing the operation of the face authentication apparatus according to the first embodiment. 6 and 7 show an example of the image scanning method according to the first embodiment. In step ST10, the image input unit 1 inputs a face image, and in step ST11, the basic scanning area 20 is enlarged and moved as shown in FIG. 6 to scan the partial area 22 in the image 21. As a scanning method, as shown in FIG. 7, the size of the image 21 is reduced while the size of the basic scanning region 20 is fixed, and the scanning is performed by moving the basic scanning region 20 having a fixed size. There is also a way to do it.

  In step ST <b> 12, the general-purpose face detection unit 2 determines whether the partial area in the image is a face or a non-face, and if it is non-face, the determination result (non-face) is sent to the display device 4. . Here, as a face detection method, a general method may be used. For example, a method using a neural network, a method using boosting, or the like may be used. In addition, the neural network is described in H. Rowley, S. Baluja, and T. Kanade. Neural network-based face detection. In IEEE Patt. Anal. Mach. Intell., Volume 20, pages 2238, 1998. Boosting is described in Paul Viola and Michael Jones, Robust real-time object detection, International Journal of Computer Vision, 2002.

  In step ST13, the registered pose face detection unit 3 determines whether the partial area (= face area) in the image detected by the general-purpose face detection unit 2 in step ST12 is a front face or a non-front face. The result is sent to the display device 4, and the display device 4 presents the three states of the non-face, the non-front face, and the front face to the user in, for example, red, yellow, and green colors together with the determination result in step ST12. While looking at the color of the display device 4, the user changes the face direction so that green lights up (= front face).

  When green is turned on and a front face is detected, in step ST14, the collation image feature amount extraction unit 5 detects the position of a feature point such as an eye-nose mouth from a partial region in the detected image, and As shown in FIG. 2, the normalized face image is generated by applying affine transformation to the image so that the position is located at a specific position in the image having a certain width and height. Here, as a technique for detecting the position of a feature point such as an eye-nose mouth, the same technique as that used for face detection in the general-purpose face detection unit 2 can be used.

  In step ST15, the collation image feature amount extraction unit 5 extracts a collation feature amount from the normalized face image. Here, as a feature amount extraction method, a general feature amount extraction method may be used. For example, a method using Gabor Wavelet, a method using PCA, or the like may be used. For Gabor Wavelet, WISKOTT, L., FELLOUS, J.-M., AND VON DER MALSBURG, C. 1997. Face recognition by elastic bunch graph matching. IEEE Trans. Patt. Anal. Mach. Intell. 19, PCA is described in TURK, M. AND PENTLAND, A. 1991. Eigenfaces for recognition. J. Cogn. Neurosci. 3, 7286.

  In step ST <b> 17, the registered image input unit 9 extracts a registered image from the registered image database storage unit 6.

  In step ST18, the registered image feature value extraction unit 8 extracts a feature value for verification from the registered image. Here, as a feature amount extraction method, a general feature amount extraction method may be used as in the collation image feature amount extraction unit 5. For this reason, the same method as the method used in the collation image feature amount extraction unit 5 can be used.

  In step ST19, the collation unit 7 determines whether or not the input face image and the registered image are the same person using the feature amount of the collated image and the feature amount of the registered image. Here, a general collation method may be used as the feature amount collation method. For example, in the case of the technique using Gabor Wavelet, it can be determined by the normalized cross-correlation coefficient of the Wavelet coefficient. In the case of a technique using PCA, there is a method of thresholding the Euclidean distance between feature quantity vectors using the coefficient of each base vector obtained by PCA as a feature quantity vector.

  As described above, a registered image feature amount extraction unit that extracts a feature amount from a registered image in which a person's face is registered, and detects whether or not the face is the same face as the registered image by taking a person's face image In the case of a front face, a registered pose face detection unit that outputs a face image to the collation image feature amount extraction unit, a feature amount extracted from the collation image feature amount extraction unit, and a feature amount extracted from the registration image feature amount extraction unit And a collation unit that collates the front face image can be obtained appropriately.

  The registered pose face detection unit outputs a signal to the display unit when the front face cannot be detected or when the face cannot be detected, and the display unit performs display based on the signal. Can be guided to an appropriate front face that does not require correction.

  This makes it possible to easily detect a face image in the same direction as the registered image, and to realize a high-performance face authentication device by preventing a decline in authentication performance due to the difference in face orientation between the registered image and the collation image can do. Further, it is possible to detect a face image with a smaller difference in face direction from the registered image with respect to a change in face direction in the vertical direction, and it is possible to realize a higher-performance face authentication device.

Embodiment 2. FIG.
Embodiment 2 will be described. FIG. 8 is a block diagram showing an object recognition apparatus according to the second embodiment. When compared with the first embodiment, instead of the display device 4, there is a camera-integrated display device 10 integrated with a camera, and a camera-integrated display device 11 and an image input unit corresponding to a plurality of cameras. 12, a general-purpose face detection unit 13, a registered pose face detection unit 14, and a camera selection unit 15 are different. In the drawings, the same reference numerals denote the same or corresponding parts, and this is common throughout the entire specification. Further, the description of the constituent elements appearing in the whole specification is merely an example and is not limited to these descriptions.

  The process from the camera-integrated display device 10 to the image input unit 1, the general-purpose face detection unit 2, and the registered pose face detection unit 3 is the same as in the first embodiment, and the camera-integrated display device 11 also has a camera and a display device 4. Are the same as those of the camera-integrated display device 10 except that are integrated.

  Since there are a plurality of camera-integrated display devices 10 and 11, for example, the camera-integrated display device 10 and the camera-integrated display device 11 are arranged side by side in the height direction. The integrated display device 10 or the camera integrated display device 11 is selected, the face area is displayed only on one side, and the face area to be sent to the collation image feature amount extraction unit 5 is displayed as the camera integrated display device 10 or the camera. Either of the integrated display devices 11 can be selected.

  FIG. 9 is a flowchart showing the object recognition apparatus according to the second embodiment. The operation of the face authentication apparatus in the second embodiment is almost the same as that in the first embodiment. The part for performing face detection from step ST10 to step ST13 is the same as that in the first embodiment, and there are two cameras, and therefore similar steps ST20 to ST23 exist.

  In ST30, the face positions in the height direction in the images obtained from the general-purpose face detection results in steps ST12 and ST22 are compared, and the camera-integrated display device 10 or 11 close to the center in the image is selected. The results of the face, the non-front face, and the front face are sent to the camera-integrated display device 10 (or 11). In the selected camera-integrated display device 10 (or 11), red (= non-face), yellow (= non-front face), and green (= front face) are displayed in accordance with the face detection result. The camera-integrated display device 11 (or 10) that has not been selected can be prevented from being displayed at all. Thereby, the user can take a pose using a camera close to the front, and can perform face detection.

  Here, the case where there are two camera-integrated display devices has been described as an example, but the number is not limited to two, and a camera that captures a human face image and a display unit may be integrated and provided in the vertical direction. Regardless of the height difference, by enabling interaction with the camera closer to the front of the face, it is possible to capture a face with a smaller difference in face orientation from the registered image. At this time, only the camera-integrated display device selected by the camera selection unit 15 does not display the identification state (non-face, non-front face, front face), but other camera-integrated display devices The user can be notified of which camera-integrated display device is trying to identify the front face with an “arrow mark” or “camera number”.

  The front face of the camera-integrated display device 10 (or 11) selected in step ST30 is sent to normalization in step ST14.

  Therefore, by using two camera-integrated display devices arranged in the height direction and selecting a camera-integrated display device that is more directly facing the user, the difference in the vertical direction of the face is also A face in the same direction as the registered image taken from the front direction can be detected.

  Providing multiple front face images by providing multiple cameras in the vertical direction to capture human face images, and by integrating multiple cameras in the vertical direction and cameras that capture human face images. Can get to.

1 is a block diagram illustrating an object recognition device according to Embodiment 1. FIG. 6 is a diagram for describing an example of a registered face image according to Embodiment 1. FIG. It is explanatory drawing regarding the face which the general purpose face detector by Embodiment 1 detects. It is explanatory drawing regarding the face which the registration pose face detector by Embodiment 1 detects. 5 is a flowchart showing an operation of the face authentication apparatus according to the first embodiment. 2 is an example of an image scanning method according to the first embodiment. 2 is an example of an image scanning method according to the first embodiment. 6 is a block diagram illustrating an object recognition apparatus according to Embodiment 2. FIG. 6 is a flowchart illustrating an object recognition apparatus according to a second embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Image input part, 2 General purpose face detection part, 3 Registration pose face detection part, 4 Display apparatus, 5 Matching image feature-value extraction part, 6 Registration image database memory | storage part, 7 Matching part, 8 Registration image feature-value extraction part, 9 Registered image input unit, 10 camera integrated display device, 11 camera integrated display device, 12 image input unit, 13 general-purpose face detection unit, 14 registered pose face detection unit, 15 camera selection unit, 20 basic operation area, 21 image, 22 Partial area.

Claims (4)

A registered image feature amount extraction unit that extracts a feature amount from a registered image in which a person's face is registered;
Registered pose face detection that takes a face image of the person and detects whether or not the face is the same face as the registered image, and outputs the face image to the collation image feature amount extraction unit in the case of the front face And
A face authentication apparatus comprising: a collation unit that collates the feature amount extracted from the collation image feature amount extraction unit and the feature amount extracted from the registered image feature amount extraction unit. The registered pose face detection unit outputs a signal to the display unit when the front face cannot be detected and when the face cannot be detected,
The face authentication apparatus according to claim 1, wherein the display unit performs display based on the signal. The face authentication apparatus according to claim 1, wherein a plurality of cameras that take a face image of a person are provided in a vertical direction. 3. The face authentication apparatus according to claim 2, wherein a camera for taking a human face image and a display unit are integrated and provided in a plurality in the vertical direction.

Images