JP6891633B2 - Face recognition device, face recognition method, and program - Google Patents

Description

The present invention relates to a face recognition device, a face recognition method, and a program, and more particularly to a face recognition device, a face recognition method, and a program using features of facial organs such as eyes, nose, and mouth.

Conventionally, an authentication system that detects facial features from an image and authenticates a person has been proposed. For example, in the method of Non-Patent Document 1, feature points of each facial organ (eyebrows, eyes, nose, mouth, chin, etc.) included in the face region 301 in the input image 300 are detected and defined by a plurality of feature points. Facial organ data 332 can be generated (FIG. 10 (a)). In this way, using the facial organ data 332 representing the facial features of the person, the facial organ data 332 (FIG. 10A) generated from the input image 300 and the facial organ data 382a of a plurality of registered persons are registered. , 382b, 382c, ... (FIG. 10 (b)), and the similarity of faces can be evaluated. If the facial organ data is used for the evaluation of the degree of similarity in this way, the points are compared, so that the authentication process can be performed at a higher speed than when machine learning or the like is used. It also has the advantage that it does not require the collection of learning data or the learning process.

By the way, when evaluating the similarity, it is necessary to align the facial organ data 332 detected from the input image 300 with the registered facial organ data 382a, 382b, 382c, ... In advance. Alignment is generally performed using feature points of the entire face, for example, as disclosed in Non-Patent Document 2. Specifically, as shown in FIG. 11, translation is performed so that the difference between the corresponding feature points (specifically, the sum of squares of the Euclidean distances between the feature points) is the smallest for the entire face. Performs image processing such as rotation and enlargement / reduction.

In addition, the evaluation of the degree of similarity after the alignment is generally performed based on the Euclidean distance between the corresponding feature points. For example, the average value of the Euclidean distance between each feature point is calculated, and the smaller the average value, the more similar faces are judged.

Vahid Kazemi, 1 others, 2014, "OneMillisecond Face Alignment with an Ensemble of Regression Trees", [online], Royal Institute of Technology, Computer Vision and Active Perception Lab (CVPR), [Searched April 25, 2017] , Internet, <URL: http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Kazemi_One_Millisecond_Face_2014_CVPR_paper.pdf> TF Cootes and CJTaylor, 2004, "Statistical Models of Appearance for Computer Vision", p102 &#8211; 104, [online], [Searched April 25, 2017], Internet, <URL: http: // www. face-rec.org/algorithms/AAM/app_models.pdf>

However, as described above, if the alignment is performed so that the difference between the corresponding feature points is the smallest in the entire face, individual differences are less likely to occur and the accuracy of authentication is lowered. Furthermore, when only the Euclidean distance between feature points is used as the evaluation index of similarity, the evaluation result of similarity largely depends on the organ having many feature points, and even if the same person is evaluated by facial expression, the similarity is evaluated. The accuracy of authentication is reduced because the results change significantly.

The present invention has been made in view of such a problem, and an object of the present invention is to provide a face recognition device, a face recognition method, and a program capable of obtaining a face recognition result with high authentication accuracy.

In order to solve the above-mentioned problems, the first invention stores a plurality of face images and organ data in which the positions and shapes of the respective organs of each face image are defined by a plurality of feature points as registered face data. A storage means, an input means for inputting a face image, an organ detection means for detecting each organ from a face region included in the face image input to the input means, and an organ data for generating organ data of each detected organ. The generation means, the input organ data which is the organ data generated by the organ data generation means, and the alignment means for aligning the face image stored in the storage means with reference to a specific organ, and the alignment means. It is characterized by including a similarity calculation means for calculating the similarity between the input face image and the face image stored in the storage means, and an authentication processing means for performing face authentication processing based on the similarity. It is a face recognition device.

According to the first invention, a plurality of face images and organ data in which the positions and shapes of the respective organs of each face image are defined by a plurality of feature points are linked and stored as registered face data in a storage means and input. Each organ is detected from the face area included in the face image, the organ data of each detected organ is generated, and this input organ data and the pre-stored face image are aligned with respect to a specific organ. The degree of similarity between the aligned input face image and the memorized face image is obtained, and face recognition processing is performed based on the degree of similarity. By aligning the face image with reference to a specific organ, it becomes easy to identify individual differences and it becomes possible to obtain an accurate face recognition result.

In the first invention, for example, it is desirable that the similarity calculation means calculates the similarity using feature points for each organ. Since the degree of similarity for each organ is calculated, it becomes easy to identify an individual. Further, the alignment means aligns the input organ data in the specific organ and the organ data of the registered face data so that the difference between the corresponding feature points is minimized. Thereby, the alignment of the feature points of the entire face is preferably performed using the feature points of a specific organ. Further, for example, the alignment means is positioned by image processing of at least one or more of translation, rotation, enlargement, and reduction for either or both of the input organ data and the organ data of the registered face data. Make a match. As a result, restrictions such as the posture at the time of shooting and the position of the camera can be reduced, and the camera becomes easy to use.

Further, the similarity calculation means calculates the average value and variance of the Euclidean distance between the corresponding feature points in the organ data of the input organ data and the registered face data for each facial organ, and the above-mentioned all organs It is desirable to calculate the similarity of the entire face based on the average value and variance of the Euclidean distance. By evaluating not only the Euclidean distance between feature points but also the variance, it becomes easier to identify the same person regardless of changes in facial expressions, and authentication accuracy is improved.

Further, it is desirable that the number of feature points of the organ data is defined for each organ. As a result, features can be extracted with the optimum number of feature points according to each organ, and facial features can be easily expressed. As a result, it is possible to obtain a robust face recognition result that makes it easy to identify individual differences and is less dependent on facial expressions.

The second invention is a step in which a computer stores a plurality of face images and organ data in which the positions and shapes of the respective organs of each face image are defined by a plurality of feature points in a storage unit as registered face data. A step of inputting a face image, a step of detecting each organ from the face area included in the input face image, a step of generating organ data of each detected organ, and an input organ which is the generated organ data. The step of aligning the data and the face image stored in the storage unit with reference to a specific organ, and the degree of similarity between the aligned input face image and the face image stored in the storage unit. The face recognition method is characterized by including a step of calculating and a step of performing face recognition processing based on the similarity.

According to the second invention, a plurality of face images and organ data in which the positions and shapes of the respective organs of each face image are defined by a plurality of feature points are linked and stored in a storage unit as registered face data and input. Each organ is detected from the face area included in the face image, the organ data of each detected organ is generated, and this input organ data and the pre-stored face image are aligned with respect to a specific organ. The degree of similarity between the aligned input face image and the memorized face image is obtained, and face recognition processing is performed based on the degree of similarity. By aligning the face image with reference to a specific organ, it becomes easy to identify individual differences and it becomes possible to obtain an accurate face recognition result.

A third invention is a storage means, a face, which stores a computer as registered face data by associating a plurality of face images with organ data in which the positions and shapes of the respective organs of each face image are defined by a plurality of feature points. Input means for inputting an image, organ detection means for detecting each organ from the face area included in the face image input to the input means, organ data generation means for generating organ data of each detected organ, and the organ data generation. Alignment means for aligning the input organ data, which is the organ data generated by the means, and the face image stored in the storage means with reference to a specific organ, the aligned input face image and the storage means. It is a program for functioning as a similarity calculation means for calculating the similarity with the face image stored in the above, and an authentication processing means for performing face authentication processing based on the similarity.

The third invention makes it possible to make the computer function as the face recognition device of the first invention.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide a face recognition device, a face recognition method, and a program capable of obtaining a face recognition result with high authentication accuracy.

A block diagram showing a functional configuration of the face recognition device 1 according to the present invention. Internal configuration diagram of a computer that functions as a face recognition device 1 The figure which shows the data composition example of the registered face organ data 8. A flowchart showing the flow of the authentication process executed by the face recognition device 1. The figure explaining the process of detecting the facial organ data 32 from the image 30. The figure explaining the alignment of the input facial organ data 32 and the registered facial organ data 82. Flowchart showing the flow of similarity evaluation processing The figure explaining the similarity calculation for each facial organ The figure explaining the comparison between the corresponding feature points of the input facial organ data 32d and the registered facial organ data 82d. The figure explaining the process which detects the face organ data 332 from the input image 300 and compares it with the registered face organ data 382a, 382b, ... The figure explaining the alignment of the conventional facial organ data

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing a functional configuration of the face recognition device 1 according to the present invention, and FIG. 2 is a diagram showing an example of an internal configuration of a computer functioning as the face recognition device 1.

As shown in FIG. 1, the face recognition device 1 includes an input unit 2, a face area detection unit 3, a face organ detection unit 4, an alignment unit 5, a similarity evaluation unit 6, an output unit 7, a registration processing unit 9, and a registration processing unit 9. A storage unit 12 is provided. The registered face data 8 is stored in the storage unit 12.

The input unit 2 inputs the image 30 to the face recognition device 1. The input image 30 (hereinafter referred to as an input image) is an image (face image) including a face region, and may be a still image or a moving image. The input unit 2 may acquire the image 30 taken by the camera 20 directly from the camera 20, or may acquire the face image stored in the storage unit 12, the recording medium, or the like.

The face area detection unit 3 determines whether or not the input image 30 input from the input unit 2 includes a person's face, and if the person's face is detected, detects the face area 31 from the input image 30 ( (See FIG. 5).

The face organ detection unit 4 detects each organ from the face region 31 detected by the face region detection unit 3. The facial organ detection unit 4 detects a predetermined feature point from the detected image information of the face region 31 and estimates the position and shape of each facial organ. The facial organ is an organ included in the face, and is, for example, a right eyebrow 32a, a left eyebrow 32b, a right eye 32c, a left eye 32d, a nose 32e, a mouth 32f, a chin 32g, and the like. A known method may be used for detecting the facial organs. For example, the method of Non-Patent Document 1 is known. (Non-Patent Document 1: Vahid Kazemi, 1 other person, "One Millisecond Face Alignment with an Ensemble of Regression"
Trees ", [online], Royal Institute of Technology, Computer Vision and Active Perception Lab (CVPR), <URL: http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Kazemi_One_Millisecond_Face_2014_CVPR_paper.pdf>). According to the method of Non-Patent Document 1, organ data 32 (hereinafter, referred to as facial organ data) in which the positions and shapes of the facial organs 32a to 32g are defined by a plurality of feature points can be generated from the face region 31. In the present invention, the facial organ detection unit 4 may detect each facial organ 32a to 32g from the image 30 and generate facial organ data 32 in which the position and shape of each facial organ 32a to 32g are defined by a plurality of feature points. The method for detecting the facial organs 32a to 32g is not limited to the method of Non-Patent Document 1. The facial organ data 32 will be described later.

The face organ detection unit 4 outputs the face organ data generated from the input image 30 to the registration processing unit 9 in the registration mode, and outputs the face organ data to the alignment unit 5 in the face recognition mode. The registration mode is a processing mode in which the face image of the user who uses the face authentication device 1 and the face organ data are linked and registered. The face authentication mode is a processing mode for executing the face authentication process of the user of the face authentication device 1. Whether the face recognition device 1 executes the registration mode or the face recognition mode shall be selected in advance by the operator.

The registration processing unit 9 stores the facial organ data generated by the facial organ detection unit 4 in the registration mode as the facial organ data 82 in the storage unit 12 in association with the original face image 81 (see FIG. 3). Further, the user information (hereinafter referred to as user information 83) may be stored in the storage unit 12 in association with the face image 81 and the face organ data 82. The user information 83 is information input by the user to the face recognition device 1 in the registration process, and includes, for example, attribute information such as the user's name, age, and gender.

FIG. 3 is a diagram illustrating an example of registered face data 8 stored in the storage unit 12. As shown in FIG. 3, the registered face data 8 includes a user ID 80, which is user identification information, a user's face image 81, facial organ data 82 generated from the face image 81, and user attributes. Information 83 is associated and stored. The user's attribute information is name, age, gender, etc. In the storage unit 12, the face image 81 of a plurality of users, the face organ data 82, and the user information 83 are associated with each other and stored as the registered face data 8.

On the other hand, in the face recognition mode, the face organ data detected by the face organ detection unit 4 is input to the alignment unit 5. The facial organ data detected by the facial organ detection unit 4 is referred to as input facial organ data 32.

The alignment unit 5 identifies the input image 30 (referred to as the input face image 30) input by the input unit 2 in the face authentication mode and the face image (registered face image 81) stored in advance in the storage unit 12. Align with reference to the facial organs. The alignment is performed by (1) aligning the face organ data (input face organ data 32) detected and generated from the input face image and the face image (registered face image 81) stored in advance in the storage unit 12. You may (2) align the face images (input face image 30 and registered face image 81) with each other, or (3) face organs stored in advance in the input face image 30 and the storage unit 12. The data (registered facial organ data 82) may be aligned. In any case, in the present invention, the alignment unit 5 uses the input face image 30 (or input face organ data 32) and the registered face image 81 (or registered face organ data 82) as a specific face organ (for example, chin). ) Is used as a reference. As a specific example, the alignment unit 5 has a difference between the corresponding feature points of the input facial organ data 32 and the registered facial organ data 82 in a specific facial organ (specifically, the Euclidean distance between the feature points). Alignment is performed so that the sum of squares of) is minimized. The alignment of the facial organ data 32 and 82 will be described later.

The similarity evaluation unit 6 calculates the degree of similarity between the input face image 30 and the registered face image 81 aligned by the alignment unit 5 in the face recognition mode. The similarity evaluation unit 6 calculates the similarity between the input face image 30 and the registered face image 81 using the feature points for each facial organ. Specifically, the average value and the variance of the Euclidean distance between the corresponding feature points in the input facial organ data 32 and the registered facial organ data 82 are calculated for each facial organ. In addition, the score of the similarity of the entire face is calculated based on the average value and the variance of the calculated Euclidean distances of all the facial organs. The specific calculation method will be described later.

The similarity evaluation unit 6 compares the calculated similarity score with a predetermined threshold value, and when the similarity score is equal to or less than the predetermined threshold value, the input facial organ data 32 and the registered facial organ data 82 are similar. Judge that it is. In the face recognition mode, the alignment and similarity evaluation of all the registered face organ data 82 registered in the storage unit 12 and the input face organ data 32 are performed. In the similarity evaluation, a person whose similarity score is equal to or less than a predetermined threshold value and which is associated with the registered facial organ data having the smallest similarity score is authenticated as the person. If the similarity score is greater than the predetermined threshold, authentication fails. A specific example of the calculation of the similarity and the authentication process in the similarity evaluation unit 6 will be described later.

The output unit 7 outputs the evaluation result (result of the authentication process) in the similarity evaluation unit 6. The output is, for example, a notification of the authentication result to the login processing unit that executes the login process in the case of the authentication process at the time of login of the PC, and unlocking the gate in the case of the face authentication process in the gate unlocking or the like. This is a notification of the authentication result to the unlocking processing unit to be executed. The output method is not limited to these, and the evaluation result of the similarity evaluation unit 6 may be notified by any method.

FIG. 2 is a diagram showing a configuration example of a computer that functions as the face recognition device 1. As shown in FIG. 2, the face recognition device 1 includes a control unit 11, a storage unit 12, a media input / output unit 13, a peripheral device I / F unit 14, an input unit 15, a display unit 16, a communication control unit 17, and the like. It is configured by being connected via 18. A camera 20 is connected to the peripheral device I / F unit 14. When the computer functions as the face recognition device 1, the control unit 11 of the computer is realized by executing the program describing the process shown in FIG.

The control unit 11 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU calls and executes a program stored in the storage unit 12, ROM, recording medium, etc. in the work memory area on the RAM, and drives and controls each unit connected via the bus 18. The ROM permanently holds a computer boot program, a program such as a BIOS, data, and the like. The RAM temporarily holds the loaded program and data, and also includes a work area used by the control unit 11 to perform various processes. The control unit 11 functions as each means of the face recognition device 1 by reading and executing the above program.

The storage unit 12 is, for example, a storage device such as a hard disk drive. The storage unit 12 stores a program executed by the control unit 11, data necessary for executing the program, an operating system, and the like. These program codes are read by the control unit 11 as necessary, transferred to the RAM, read by the CPU, and executed. The storage unit 12 stores the registered face data 8 in which the registered face image 81, which is a face image of each user, and the registered face organ data 82, which is the face organ data generated from those face images, are linked. (See FIG. 3).

The media input / output unit 13 is, for example, a drive device for various recording media (media) such as a CD, DVD, and MO, and inputs / outputs (writes / reads) data to and from the media.

The peripheral device I / F (interface) unit 14 is a port for connecting peripheral devices, and transmits / receives data to / from the peripheral device via the peripheral device I / F unit 14. The peripheral device I / F unit 14 is composed of USB or the like, and usually has a plurality of peripheral device I / Fs. The connection form with peripheral devices may be wired or wireless.

The input unit 15 is, for example, an input device such as a touch panel, a keyboard, a pointing device such as a mouse, or a numeric keypad, and outputs the input data to the control unit 11.
The display unit 16 is composed of, for example, a display device such as a liquid crystal panel or a CRT monitor, and a logic circuit (video adapter or the like) for executing display processing in cooperation with the display device, and is input under the control of the control unit 11. Display the display information on the display device. The input unit 15 and the display unit 16 may be a touch panel display (operation unit 25) in which an input device such as a touch panel is integrally provided on the display screen.

The communication control unit 17 has a communication control device, a communication port, and the like, and controls communication with the network 10 and the like. The network 10 includes a LAN (Local Area Network), a WAN (Wide Area Network) connected to a wider area, a public communication line such as the Internet, a base station, and the like. The network 10 may be wired or wireless. The face recognition device 1 can access a server (not shown) via the network 10 and send and receive various programs and data.
The bus 18 is a route that mediates the transfer of control signals, data signals, and the like between the devices.

Next, with reference to FIG. 4, the flow of the face authentication process executed by the face authentication device 1 will be described. FIG. 4 is a flowchart illustrating a processing flow in the face recognition mode. The control unit 11 of the face recognition device 1 reads out the program and data related to the processing shown in FIG. 4 from the storage unit 12, and executes the processing based on the program and data. Before executing this face recognition process, it is assumed that the registered face data 8 of a plurality of users is registered in the storage unit 12. The face organ data 82 registered in the registered face data 8 may be created by the same device as the face recognition device 1 that executes the following face recognition process, or may be created by another device and stored in the storage unit 12. It may be the one memorized in.

The control unit 11 (input unit 2) of the face recognition device 1 acquires an image taken by the camera 20 or an image stored in the storage unit 12 as an input image 30 (step S101). The input image 30 acquired in step S101 may be a still image or a moving image. In the case of a moving image, the image of each frame included in the moving image is used as the input image 30 and the following processing is performed.

The control unit 11 (face area detection unit 3) detects the face area 31 of the person from the input image 30 (step S102). The control unit 11 (face organ detection unit 4) detects facial organs 32a to 32g from the detected face region 31 and generates facial organ data 32 (step S103). The facial organ data 32 generated in step S103 is referred to as input facial organ data 32.

FIG. 5 is a diagram illustrating a process of generating facial organ data 32 from the input image 30. As described above, the control unit 11 recognizes the face region 31 from the image 30 (the image of each frame when acquiring the image) taken by the camera 20, detects the facial organs 32a to 32g from the face region 31, and detects the facial organs 32a to 32g. Facial organ data 32 is generated. As a method for generating facial organ data 32 from the image 30, for example, a known algorithm shown in Non-Patent Document 1 described above may be used. The method for generating facial organ data 32 is not limited to the method described in Non-Patent Document 1. The present invention may generate facial organ data 32 using other techniques.

FIG. 5A is a diagram showing the face region 31 and the facial organ data 32 detected from the input image 30. As shown in FIG. 5B, the facial organ data 32 has a plurality of positions and shapes of each facial organ (right eyebrow 32a, left eyebrow 32b, right eye 32c, left eye 32d, nose 32e, mouth 32f, chin 32g). It is the data defined by the feature points. The number of feature points of each facial organ 32a to 32g is defined for each facial organ. Specifically, for example, the right eyebrow 32a, the left eyebrow 32b, the right eye 32c, and the left eye 32d each have 20 feature points. The nose 32e is defined by 17 feature points. The mouth 32f is defined by 54 feature points. The jaw is defined by 41 feature points. The number of these feature points is an example, and may be another number.

In step S103, if the face cannot be captured in front due to the installation position of the camera 20 or the movement of the face of a person, the image (frame) may be excluded from the target of facial organ detection. ..

Next, the control unit 11 acquires the registered facial organ data 82, which is the facial organ data of the registered face image 81 to be compared with the input facial organ data 32 generated from the input image 30, from the storage unit 12 (step S104). It is assumed that the registered facial organ data 82 of the user is stored in the storage unit 12 as shown in FIG.

The control unit 11 (alignment unit 5) aligns the facial organ data 32 (input facial organ data 32) generated in step S103 with the registered facial organ data 82 called from the storage unit 12 in step S104 (step). S105).

In the alignment process in step S105, the control unit 11 aligns the input facial organ data 32 and the registered facial organ data 82 with reference to a specific facial organ. That is, the alignment is performed so that the difference between the corresponding feature points (the sum of squares of the Euclidean distances between the feature points) in the specific facial organs of the input facial organ data 32 and the registered facial organ data 82 is minimized. ..

It is preferable that the specific facial organ used as the reference for alignment is an organ that does not easily change depending on the facial expression, such as the chin shown in FIG. Further, which face organ is used as a specific face organ may be changed by a preset operation by the operator of the face recognition device 1. In this way, by aligning the input facial organ data 32 and the registered facial image 81 (registered facial organ data 82) with reference to one specific organ (for example, the chin), individual differences in the face can be further compared. It will be easier to do.

In the alignment process of step S105, the control unit 11 has at least one or more of translation, rotation, enlargement, and reduction for either or both of the input facial organ data 32 and the registered facial organ data 82. Image processing is performed, and alignment is performed so that the difference between the corresponding feature points of a specific facial organ (the sum of squares of the Euclidean distances between the feature points) is minimized. In this way, the alignment of the feature points of the entire face is preferably performed using the feature points of a specific facial organ. In addition, since at least one of parallel movement, rotation, enlargement, and reduction is performed for alignment, restrictions such as posture and camera position at the time of shooting can be reduced, and it is easy to use. It becomes a thing.

FIG. 6 shows a state in which the input facial organ data 32 and the registered facial organ data 82 are aligned with respect to the jaw 32 g.

In the alignment process of step S105, an example of aligning the input face organ data 32 and the registered face organ data 82 is shown, but the present invention is not limited to this example, and the input face organ data 32 and the registered face image 81 May be aligned, face images (input face image 30 and registered face image 81) may be aligned, or input face image 30 and registered face organ data 82 may be aligned. .. In either case, the input face image 30 (or input face organ data 32) and the registered face image 81 (or registered face organ data 82) are aligned with respect to a specific face organ (for example, the chin).

Next, the control unit 11 (similarity evaluation unit 6) evaluates the similarity between the aligned input face image 30 and the registered face image 81 (step S106). In the similarity evaluation process, first, the similarity between the aligned input facial organ data 32 and the registered facial organ data 82 is calculated. When calculating the similarity, the control unit 11 calculates the average value and the variance of the Euclidean distance between the corresponding feature points in the input facial organ data 32 and the registered facial organ data 82 for each facial organ. Next, the overall facial similarity is comprehensively evaluated using the weighted sum of the similarity scores for each facial organ.

Since the shape of each facial organ is almost constant for the same person, the value of the distance between the corresponding feature points does not vary, and it is predicted that the variance will be small. In this way, if the average value and variance of the distance between the feature points are calculated for each facial organ and the overall similarity is evaluated comprehensively, the similarity of each facial organ alone is evaluated and the entire face is evaluated. The degree of similarity can also be evaluated. As a result, the same person can be identified without depending on the change in facial expression. In addition, the degree of similarity can be evaluated for the entire face without being biased to a specific facial organ.

The similarity evaluation process in step S106 will be specifically described with reference to FIG. 7.
As shown in FIG. 7, first, K, which is a value indicating the facial organ to be calculated, is initialized (K = 1; step S201), and for the facial organ K, the input facial organ data 32 and the registered facial organ data 82 are combined. calculating a Euclidean distance _{d n} between the corresponding feature points n (step S202).

The position coordinates of the feature point n in the input facial organ data 32 and the registered facial organ data 82 are respectively.

_{Then, the Euclidean distance d n} between the images for the feature point n (between the input facial organ data 32 and the registered facial organ data 82) can be expressed by the following equation (1).

Further, if the set K is placed on the facial organ K as K = {k | k is a feature point included in the facial organ K}, the Euclidean distance between the images at the feature point k included in the facial organ K The average μ _K of the distance is as shown in the following equation (2).

Here, the facial organ K is the number of the facial organ to be calculated, as shown in FIG. 8, for example, the right eyebrow is K = 1, the left eyebrow is K = 2, the right eye is K = 3, and the left eye. Is K = 4, the nose is K = 5, the mouth is K = 6, the jaw is K = 7, and so on. num _K is the number of feature points contained in the facial organ K.

_{Next, the control unit 11 obtains the variance σ K} ² of the Euclidean distance between the corresponding feature points of the facial organ K in the input facial organ data 32 and the registered facial organ data 82 (step S203). _{The variance σ K} ² of the Euclidean distance between the corresponding feature points of the facial organ K in the input facial organ data 32 and the registered facial organ data 82 is as shown in the following equation (3). The variance σ _K ² is a value that becomes smaller as the shape of the facial organ is closer.

FIG. 9 is a diagram showing the feature points of the right eyes 32d and 82d in the input facial organ data 32 and the registered facial organ data 82. When the input facial organ data 32 and the registered facial organ data 82 are of the same person, as shown in FIG. 9, the shapes of the facial organs are almost the same, but the positions may shift due to changes in facial expressions or the like. is there. In such a case, it can be assumed that the value of the distance between the corresponding feature points of the input facial organ data 32 and the registered facial organ data 82 has a small variation (variance). By using variance to evaluate the degree of similarity, individual differences in faces can be evaluated more accurately.

The control unit 11 repeats the calculation of the Euclidean distance and the variance between the corresponding feature points for all the facial organs K (step S204; No → step S205 → step S202 to step S203).

When the Euclidean distance and variance between the corresponding feature points are calculated for all facial organs K (step S204; Yes), the control unit 11 calculates the overall similarity score of the entire face (step S206). The similarity score is, for example, the following equation (4).

In equation (4), μ _K and σ _K ² are the average value and variance of the Euclidean distance between images at the feature points included in the facial organ K. w _K1 and w _K2 are weight parameters. The value of the weight parameter shall be an appropriate value determined experimentally. Face is a set of detected facial organs. For example, in the example shown in FIG. 7, Face = {right eyebrow, left eyebrow, right eye, left eye, nose, mouth, chin}.

The similarity score calculated by the above formula (4) becomes smaller as the input facial organ data 32 and the registered facial organ data 82 are similar.

The control unit 11 outputs the value of the similarity score calculated in step S206 as the similarity evaluation result between the input face image 30 and the registered face image 81 acquired in step S104. The control unit 11 repeatedly executes the processes of steps S104 to S106 (acquisition of registered face image 81 (registered face organ data 82), alignment process, similarity evaluation process) for all registered face images 81. To do. As a result, the control unit 11 obtains a similarity score between the input face image 30 and the registered face image 81.

In the authentication process of step S107 of FIG. 4, the control unit 11 authenticates the person based on the similarity evaluation result (similarity score) between the input face image 30 and the registered face image 81 (step S107). In the personal authentication, the control unit 11 determines which registered face image 81 the person in the input image 30 is associated with. The control unit 11 authenticates the person associated with the registered face image 81 whose similarity score is equal to or less than a predetermined threshold value and whose similarity score is the smallest as the person. If the similarity score is larger than a predetermined threshold value, it is determined that the face recognition has failed.

The control unit 11 (output unit 7) outputs the authentication result of step S107 (step S108 in FIG. 4). The output is, for example, a notification of the authentication result to the login processing unit in the case of authentication processing at the time of PC login, and a notification of the authentication result to the unlocking processing unit in the case of face recognition processing such as gate unlocking. Is. The result output method is not limited to these, and any output method may be used.

As described above, the face authentication device 1 according to the present embodiment has each face organ (right eyebrow 32a, left eyebrow 32b, right eye 32c, left eye 32d, nose 32e) from the face area 31 included in the input image 30. (Mouth 32f, jaw 32g, etc.) is detected, and facial organ data 32 in which the positions and shapes of the detected facial organs 32a to 32g are defined by a plurality of feature points is generated, and the input facial organ data 32 and the user in advance are generated. The registered face organ data 82 of the registered face image 81 registered in association with the information 83 is aligned with reference to a specific face organ (for example, jaw 32 g), a similarity score is obtained, and the similarity score is obtained. Perform face recognition processing. By aligning with a specific facial organ as a reference, it becomes easier to identify individual differences and it becomes possible to obtain an accurate face recognition result. Further, the similarity score is a value obtained by integrating the average value and the variance of the Euclidean distance between images at the feature points included in each facial organ K in all facial organs. By evaluating not only the Euclidean distance between the corresponding feature points but also the variance, it becomes easier to identify the same person regardless of the change in facial expression, and the authentication accuracy is improved.

Although preferred embodiments of the face recognition device and the like according to the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above examples. For example, in the above-described embodiment, an example in which a computer executes a face recognition process has been described, but the present invention is not limited to this, and can be applied to, for example, a process in which a robot identifies a person from a face. Is. Further, it is clear that a person skilled in the art can come up with various modified examples or modified examples within the scope of the technical idea disclosed in the present application, and these also naturally belong to the technical scope of the present invention. It is understood that it is.

1 ……………… Face recognition device (computer)
2 ……………… Input unit 3 ……………… Face area detection unit 4 ……………… Face organ detection unit 5 ……………… Alignment unit 6 ……………… Similarity Evaluation unit 7 ……………… Output unit 8 ……………… Registered face data 9 ……………… Registration processing unit 10 ……………… Network 11 ……………… Control unit 12 ………… …… Storage unit 13 ……………… Media input / output unit 14 ……………… Peripheral device I / F unit 15 ……………… Input unit 16 ……………… Display unit 17 ……………… Communication control Part 18 ……………… Bus 30 ……………… Image 31 ……………… Face area 32 ……………… Facial organ data (input facial organ data)
32a-32g ……………… Facial organ 80 ……………… User ID
81 …………… Face image (registered face image)
82 ……………… Facial organ data (registered facial organ data)
83 …………… User attribute information

Claims (8)

A storage means for storing a plurality of face images and organ data in which the positions and shapes of the respective organs of each face image are defined by a plurality of feature points as registered face data.
Input means for inputting face images and
An organ detecting means for detecting each organ from a face region included in a face image input to the input means, and an organ detecting means.
Organ data generation means that generates organ data of each detected organ,
An alignment means for aligning the input organ data, which is the organ data generated by the organ data generation means, and the face image stored in the storage means with reference to a specific organ.
A similarity calculation means for calculating the similarity between the aligned face image and the face image stored in the storage means, and a similarity calculation means.
An authentication processing means that performs face recognition processing based on the similarity, and
A face recognition device characterized by being equipped with. The face recognition device according to claim 1, wherein the similarity calculation means calculates the similarity using feature points for each organ. The alignment means aligns the input organ data in the specific organ and the organ data of the registered face data stored in the storage means so that the difference between the corresponding feature points is minimized. The face recognition device according to claim 1 or 2, wherein the face recognition device is characterized. The alignment means performs image processing of at least one or more of translation, rotation, enlargement, and reduction with respect to either or both of the input organ data and the organ data of the registered face data. The face recognition device according to any one of claims 1 to 3, wherein alignment is performed. The similarity calculation means calculates the average value and variance of the Euclidean distance between the corresponding feature points in the organ data of the input organ data and the registered face data for each organ, and determines the Euclidean distance of all the organs. The face recognition device according to any one of claims 1 to 4, wherein the similarity of the entire face is calculated based on the average value and the variance. The face recognition device according to any one of claims 1 to 5, wherein the number of feature points of the organ data is defined for each organ. The computer
A step of associating a plurality of face images with organ data in which the positions and shapes of each organ of each face image are defined by a plurality of feature points and storing them as registered face data in a storage unit.
Steps to enter the face image and
Steps to detect each organ from the face area included in the input face image,
Steps to generate organ data for each detected organ,
A step of aligning the input organ data, which is the generated organ data, and the face image stored in the storage unit with reference to a specific organ.
A step of calculating the degree of similarity between the aligned face image and the face image stored in the storage unit, and
The step of performing face recognition processing based on the similarity and
A face recognition method characterized by including. Computer,
A storage means for storing a plurality of face images and organ data in which the positions and shapes of the respective organs of each face image are defined by a plurality of feature points as registered face data.
Input means for inputting face images,
Organ detecting means for detecting each organ from a face region included in a face image input to the input means,
Organ data generation means, which generates organ data of each detected organ,
An alignment means for aligning input organ data, which is organ data generated by the organ data generation means, and a face image stored in the storage means with reference to a specific organ.
A similarity calculation means for calculating the similarity between the aligned face image and the face image stored in the storage means.
An authentication processing means that performs face recognition processing based on the similarity,
A program to function as.

Images