WO2019216091A1

WO2019216091A1 - Face authentication device, face authentication method, and face authentication system

Info

Publication number: WO2019216091A1
Application number: PCT/JP2019/015539
Authority: WO
Inventors: 郁雄渕上
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2018-05-10
Filing date: 2019-04-10
Publication date: 2019-11-14
Also published as: JP2019197426A

Abstract

In this face authentication system, a flat panel detection unit allows inputs of a first captured image and a second captured image that are respectively obtained by two imaging devices capturing the same face image. Two face detection units each detect a face image in the first captured image and/or the second captured image. A determination unit determines whether or not two face images are in a plane, on the basis of the two face images and feature points corresponding to the two face images. A validity determination unit determines the validity of a face authentication result, on the basis of a determination result as to whether or not the two face images are in a plane.

Description

Face authentication device, face authentication method, and face authentication system

The present disclosure relates to a face authentication device, a face authentication method, and a face authentication system that authenticate a target person using an image of a face portion in a captured image captured by a camera.

In Patent Literature 1, a face image sequence for a predetermined time is input, and whether or not a face included in the face image sequence is impersonated based on temporal changes in predetermined color information extracted from the face image sequence. An information processing apparatus for determining whether or not is disclosed.

International Publication No. 2015/122183

However, in the configuration of Patent Document 1, when an improper person uses a portable display terminal such as a tablet terminal to display an image showing the face of an authorized registrant and impersonates the image, Even if the used face authentication is performed, fraud cannot be detected, and there is a possibility that the authentication result is approved.

The present disclosure has been devised in view of the above-described conventional situation, detects impersonation using an image that can be performed at the time of authentication of a target person with high accuracy, and effectively prevents malicious third party fraud by impersonation. An object of the present invention is to provide a face authentication device, a face authentication method, and a face authentication system to be suppressed.

In the present disclosure, an image input unit that inputs a first captured image and a second captured image in which the same person is captured by each of a plurality of cameras, and a face area of at least one of the first captured image and the second captured image A plane detection unit that detects whether or not the face image of the face area is a plane based on face information corresponding to the face area and the first and second captured images And a validity determination unit that determines the validity of the authentication result of the person's face information based on a determination result of whether or not the face image of the face area is a plane. provide.

Further, the present disclosure is a face authentication method in a face authentication apparatus, the step of inputting a first captured image and a second captured image in which the same person is captured by each of a plurality of cameras, and the first captured image And detecting a face area of at least one of the second captured images, and based on the face information corresponding to the face area and the first captured image and the second captured image, the face image of the face area is a plane. Determining whether or not there is a step of determining the validity of the authentication result of the face information of the person based on the determination result of whether or not the face image of the face region is a plane, Provide a face recognition method.

In addition, the present disclosure is a face authentication system in which a plurality of cameras and a face authentication device are connected to be able to communicate with each other, and the plurality of cameras each include a first captured image and a second captured image capturing the same person. An image is output to the face authentication device, the face authentication device receives the first captured image and the second captured image, detects at least one face region of the first captured image and the second captured image, Based on the face information corresponding to the face area and the first captured image and the second captured image, it is determined whether the face image of the face area is a plane, and the face image of the face area is a plane. Provided is a face authentication system that determines the validity of an authentication result of the person's face information based on a determination result of whether or not there is.

According to the present disclosure, it is possible to detect impersonation using an image that can be performed at the time of authentication of the target person with high accuracy, and to suppress malicious third party fraud by impersonation.

FIG. 1 is a diagram illustrating a system configuration example of the face authentication system according to the first embodiment. FIG. 2 is a diagram illustrating an arrangement example of four imaging devices. FIG. 3 is a block diagram illustrating a functional configuration example realized by the cooperation of the processor and the memory built in the face authentication apparatus according to the first embodiment. FIG. 4 is a block diagram illustrating an example of a functional configuration and an outline of operation of the flat panel detection unit. FIG. 5 is an explanatory diagram showing an example of an outline of operation for performing plane determination by capturing a face image projected on a flat panel. FIG. 6 is an explanatory diagram showing an example of an outline of operation for performing plane determination by imaging an actual human face. FIG. 7 is a flowchart illustrating an example of an operation procedure of face authentication according to the first embodiment in time series. FIG. 8 is a flowchart illustrating an example of the operation procedure of the face image plane determination according to step S6 in time series. FIG. 9 is a flowchart illustrating an example of another face authentication operation procedure according to the first embodiment in time series. FIG. 10 is a diagram illustrating an arrangement example of the other three imaging devices according to the first embodiment. FIG. 11 is a block diagram illustrating a functional configuration example realized by cooperation of a processor and a memory built in the face authentication apparatus according to the second embodiment. FIG. 12 is an explanatory diagram illustrating an operation outline example of the frame detection processing by the frame detection unit. FIG. 13 is an explanatory diagram illustrating an operation outline example of the arrangement determination processing by the arrangement determination unit. FIG. 14 is a flowchart illustrating an example of a face authentication operation procedure according to the second embodiment in time series.

Hereinafter, embodiments that specifically disclose the face authentication device, the face authentication method, and the face authentication system according to the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the claimed subject matter.

(Embodiment 1)
FIG. 1 is a diagram illustrating a system configuration example of a face authentication system 100 according to the first embodiment. The face authentication system 100 performs face authentication of a person to be authenticated by collating face image data obtained by capturing an authentication target person with an imaging device and face image data of a regular person registered in advance. The face authentication system 100 outputs the face authentication result of the person to be authenticated to the control target device 40.

The control target device 40 is, for example, a usage management device used by a service provider such as car sharing or a security area entrance / exit management device, but is not limited thereto.

When the control target device 40 is a car sharing use management device, the use management device permits the use of a car share target vehicle when the user is a registered regular user as a face authentication result. On the other hand, when the user is not a registered regular user, the usage management device refuses to use the vehicle to be shared. When permitting or denying the use of a vehicle, the usage management device sends an authentication result indicating whether or not the vehicle can be used to an electronic control unit (ECU: Electronic Control Unit) mounted on the vehicle, for example. The electronic control unit sets the start (for example, ignition on) of the own vehicle to be valid or invalid based on the authentication result sent from the usage management device.

When the control target device 40 is a security area entrance / exit management device, the entrance / exit management device permits a person who is permitted to enter / exit as a face authentication result to pass the person to the security area. On the other hand, if the entrance / exit management device is not a permitted person as a face authentication result, the entrance / exit management device blocks the person from entering the security area or intrudes the administrator or operator of the face authentication system 100. Notification that the person has been detected.

The face authentication system 100 includes a face authentication device 20, a plurality of (for example, two)

imaging devices

11A and 11B, a display device 12, and an ID reading device 13. When there is no need to distinguish between the two

imaging devices

11A and 11B, they are simply referred to as the imaging device 11. The two

imaging devices

11A and 11B are arranged at positions where, for example, the face of a person walking in a passage can be simultaneously imaged from different directions. The imaging device 11 includes an imaging lens, an imaging device, an image signal processing circuit, a communication interface, and the like. The imaging device 11 captures an imaging region including the authentication target person 30 as a subject, and outputs the captured image to the face authentication device 20. Note that the number of imaging devices is not limited to two, and may be any number. The imaging device 11 may be a monitoring camera for monitoring a predetermined area, for example.

FIG. 2 is a diagram illustrating an arrangement example of the four

imaging devices

11e, 11f, 11g, and 11h. The four

imaging devices

11e, 11f, 11g, and 11h are arranged in a direction perpendicular to the surface of the passage through which the person 30 to be authenticated passes (that is, a direction perpendicular to the horizontal plane). The four

imaging devices

11e, 11f, 11g, and 11h image the presence range SA of the face of the person 30 to be authenticated so that the angles of view overlap in the passage width wd. The optical axes of the four

imaging devices

11e, 11f, 11g, and 11h are all parallel, for example.

For example, the face of a person such as a short child is imaged by two

imaging devices

11e and 11f installed at a low position. In this case, the two

imaging devices

11e and 11f are used as the

imaging devices

11A and 11B in FIG. Further, for example, the face of a person such as a tall adult is imaged by two

imaging devices

11g and 11h installed at a high position. In this case, the two

imaging devices

11g and 11h are used as the

imaging devices

11A and 11B in FIG. When the

imaging devices

11e, 11f, 11g, and 11h are not particularly distinguished, they are referred to as the imaging device 11. Further, when three or more face images of the target person are picked up by three or more image pickup devices, two face images of these face images are used for face authentication. Note that three or more face images may be temporally continuous three frames or temporally discontinuous three frames.

The ID reader 13 is composed of an information reader such as a card reader. The ID reading device 13 reads the identification information (for example, ID information) of the ID card 35 for personal identification possessed by the person 30 to be authenticated, and outputs the read identification information to the face authentication device 20. Examples of the ID card 35 include a membership card, an identification card, a driver's license, and a my number card.

The face authentication device 20 is configured by a PC (personal computer) or the like, and includes a processor, a memory, a communication interface, and the like. The face authentication device 20 performs face authentication using the image information of the captured image acquired from the imaging device 11 and the ID information acquired from the ID reading device 13, and outputs the face authentication result to the control target device 40. Details of the face authentication process will be described later.

Display device 12 includes a display device such as a liquid crystal display. The display device 12 displays various data or information when the face authentication device 20 executes face authentication. Various data or information includes an operation guidance screen, a captured image, a face detection result, a frame detection result, a face matching result, an authentication result including face authentication, and the like. The display device 12 is disposed in a monitoring room or the like that is visible to the administrator or operator of the face authentication system 100. The display device 12 may be disposed in the vicinity of the imaging device 11 so that guidance display can be performed for the person 30 to be authenticated.

FIG. 3 is a block diagram showing an example of a functional configuration realized by the cooperation of the processor and the memory built in the face authentication apparatus 20 according to the first embodiment. The face authentication device 20 includes a face recognition unit 21, a face collation unit 24, an validity determination unit 25, and a flat panel detection unit 70. The face recognizing unit 21, the face collating unit 24, the validity determining unit 25, and the flat panel detecting unit 70 are, for example, predetermined processors stored in the memory by the cooperation of the processor and the memory built in the face authentication device 20 which is a PC. This is realized by executing the program.

For example, the face recognition unit 21 inputs data of captured images GZ1 and GZ2 (that is, captured images GZ1 and GZ2 including face images) as shown in FIG. Then, the face of the person 30 to be authenticated included in the captured image is detected by face recognition processing using a known face recognition technique. The face recognition unit 21 acquires face feature information and position information in the captured image (in other words, face arrangement information) by face recognition processing. The face feature information may include vital information such as skin color, blood flow, and hair color in addition to the feature points of the face.

The face collating unit 24 uses the face feature information detected by the face recognizing unit 21 and the collation data of the face information of the regular registrant, and the face in the captured image detected by the face recognizing unit 21 and the regular registrant. It is determined whether or not the face matches. The collation data is recorded on, for example, a recording medium built in the face authentication device 20, and includes image data of a regular registrant's face image and feature data of face feature information. For example, the administrator or operator of the face authentication system 100 sends the verification data included in the ID information recorded on the ID card 35 possessed by the person 30 to be authenticated to the face verification unit 24 of the face authentication device 20 as the verification data. input. The input verification data is stored in a recording medium that can be read by the face verification unit 24.

The validity determination unit 25 determines whether or not the face of the person to be authenticated 30 included in the captured image is valid (that is, whether or not the face authentication is approved) as a result of the face matching by the face matching unit 24, Outputs the face authentication result.

The flat panel detection unit 70 determines whether or not the face image detected in the captured image of the person 30 to be authenticated is a plane (see FIG. 4).

FIG. 4 is a block diagram illustrating an example of a functional configuration and an outline of operation of the flat panel detection unit 70. The flat panel detection unit 70 includes a

face detection unit

71, 72, a face correspondence determination unit 73, a feature

point detection unit

74, 75, a feature point matching unit 76, a projective transformation estimation unit 77, a projective transformation unit 78, a correlation calculation unit 79, A determination unit 80 is included.

The

face detection units

71 and 72 detect the face of the target person included in the corresponding captured images GZ1 and GZ2 in which the authentication target person 30 is captured, and acquire face arrangement information. The face arrangement information is feature point detection ranges rc1 and rc2 (see FIG. 5), and is information representing a rectangular range including, for example, eyes, nose and mouth. The face detection unit 71 performs face detection on the captured image GZ1 captured by the imaging device 11A. The face detection unit 72 performs face detection on the captured image GZ2 captured by the imaging device 11B.

Feature point detectors

74 and 75 detect facial feature points included in the facial image specified by the face arrangement information, for example, by a feature point detection process using a known facial feature point detection technique. As facial feature points, for example, eyes, nose, mouth position, mouth position, deep nostril position, eye corner position, bangs curve on the forehead, distance between the eye corners, facial contour, protruding A part that characterizes a facial part such as the position of the tip of the nose is included. The feature point detection unit 74 detects a feature point of the face included in the face image fg1 (see FIG. 5) obtained by the face detection unit 71. The feature point detection unit 75 detects a feature point of the face included in the face image fg2 (see FIG. 5) obtained by the face detection unit 72. Here, as an example, the case where the feature point is the position of the mouth on both sides, the position of the nostril, and the bangs curve on the forehead is shown. Further, the feature

point detection units

74 and 75 detect, for example, a face corner portion (in other words, a contour) to detect a face feature point, or perform SIFT (Scale-Invariant Feature Transform) or SURF (Speeded-Up Robust). The feature points of the face may be detected using a known feature point extraction algorithm such as Features.

The feature point matching unit 76 performs a feature point matching process using the outputs from the feature

point detection units

74 and 75. In the feature point matching process, the same feature points correspond to the plurality of feature points p1a, p2a, p3a detected in the face image fg1 and the plurality of feature points p1b, p2b, p3b detected in the face image fg2. Attached. In the face images fg1 and fg2, the feature points are associated with the line segments c1 and p3b between the feature points p1a and p1b, between the feature points p2a and p2b, and between the feature points p3a and p3b, respectively. It is expressed by connecting with c2 and c3 (see FIG. 5).

The projective transformation estimation unit 77 calculates a projective transformation matrix based on the coordinates of two corresponding feature points included in the face images fg1 and fg2. The projective transformation matrix is a matrix for transforming coordinates in order to project a certain plane onto another plane. Here, the projective transformation matrix performs coordinate transformation for projecting the face of the face image fg2 onto the face of the face image fg1. The projective transformation estimation unit 77 may calculate a projective transformation matrix that projects the face of the face image fg1 onto the face of the face image fg2. The projective transformation matrix is represented by a 3 × 3 matrix, for example. In this projective transformation matrix, each element of a matrix represented by a 3 × 3 matrix is calculated by associating the coordinates of at least four matching feature points. Further, the projective transformation estimation unit 77 may calculate a projective transformation matrix using a least square method or the like using four or more corresponding points (that is, coordinates indicating the positions of feature points that match each other). . Thereby, the projective transformation estimation unit 77 can calculate a more accurate projective transformation matrix.

The projective transformation unit 78 performs projective transformation on the face image fg2 using the projective transformation matrix calculated by the projective transformation estimation unit 77, and obtains a transformed image fg2a (see FIG. 5).

The correlation calculation unit 79 calculates a correlation value indicating the similarity between the two images based on the face image fg1 and the converted image fg2a. In the calculation of the correlation value, template matching is performed in which a position similar to a designated image (for example, a template) is searched from among the images. Here, normalized cross-correlation (NNC) is used for the calculation of the correlation value. For the calculation of the correlation value, the sum of squares of differences in luminance values (SSD: SumSofSSquared Difference), the sum of absolute values of differences in luminance values (SAD: Sum of Absolute Difference), and the like are used. Also good.

Based on the correlation value calculated by the correlation calculation unit 79, the determination unit 80 determines whether or not the face image fg1 is flat (that is, a two-dimensional plane) as projected on the flat panel. The determination result is output. When the correlation value is high, for example, when the correlation value is higher than a preset threshold value, neither the face image fg1 nor the face image fg2 has a three-dimensional structure (for example, an actual human face). A two-dimensional plane). On the other hand, when the correlation value is low, for example, when the correlation value is equal to or less than a preset threshold value, at least one of the face image fg1 and the face image fg2 is not a plane but a three-dimensional object (for example, the face of an actual person) 3D solid object having a sense of depth.

FIG. 5 is an explanatory diagram showing an example of an outline of operation for capturing a face image projected on the flat panel 150 and performing plane determination. The flat panel 150 is, for example, a display unit (so-called display device such as a display) such as a tablet terminal, a digital photo frame, a smartphone, or a notebook PC (Personal Computer). The

imaging devices

11A and 11B respectively capture the face images displayed on the flat panel 150, and obtain captured images GZ1 and GZ2. At this time, the imaging device 11 </ b> A captures images from the front direction with respect to the flat panel 150. The imaging device 11B captures an image by slightly tilting the flat panel 150 from the front direction.

The

face detection units

71 and 72 perform face detection on the captured images GZ1 and GZ2, respectively, to obtain feature point detection ranges rc1 and rc2. The feature

point detection units

74 and 75 detect face feature points p1a, p2a, and p3a and p1b, p2b, and p3b included in the face images fg1 and fg2 that are surrounded by the feature point detection ranges rc1 and rc2, respectively.

The feature point matching unit 76 associates a plurality of feature points p1a, p2a, and p3a detected in the face image fg1 with a plurality of feature points p1b, p2b, and p3b detected in the face image fg2, and each of them is a line segment. Connect with c1, c2, and c3.

The projective conversion unit 78 performs projective conversion on the face image fg2 to obtain a converted image fg2a. The correlation calculation unit 79 calculates a correlation value between the face image fg1 and the converted image fg2a. Here, the correlation value calculated by the correlation calculation unit 79 is 0.999, which is very close to the value 1 and is a high value. The determination unit 80 determines that the face image fg <b> 1 and the face image fg <b> 2 are two-dimensional images, and are flat (that is, flat) face images displayed on the flat panel 150. In addition to the determination result, the determination unit 80 outputs a superimposed image mg1 in which the face image fg1 and the converted image fg2a are superimposed, and displays the superimposed image mg1 on the display device 12. The superimposed image mg1 is displayed like a single face image in which the face image fg1 and the converted image fg2a substantially coincide with each other and there is almost no double.

Thus, the face image projected on the flat panel shows a high correlation value. Accordingly, face authentication of a person who impersonates a regular registrant by holding a flat panel on which a regular registrant's face image is projected is denied.

FIG. 6 is an explanatory diagram showing an example of an outline of operation for performing plane determination by imaging an actual human face. The

imaging devices

11A and 11B respectively capture the face of the actual person hm and obtain captured images GZ3 and GZ4. At this time, the imaging device 11A captures an image of the face of the actual person hm from the front direction. The imaging device 11B captures an image by slightly tilting the face of the actual person hm from the front direction.

The

face detection units

71 and 72 perform face detection on the captured images GZ3 and GZ4, respectively, and obtain feature point detection ranges rc3 and rc4. The feature

point detection units

74 and 75 detect face feature points p4a, p5a, p6a, and p7a and p4b, p5b, p6b, and p7b included in the face images fg3 and fg4, which are surrounded by the feature point detection ranges rc3 and rc4, respectively. To do.

The feature point matching unit 76 associates a plurality of feature points p4a, p5a, p6a, and p7a detected in the face image fg3 with a plurality of feature points p4b, p5b, p6b, and p7b detected in the face image fg4, These are connected by line segments c4, c5, c6, and c7.

The projective conversion unit 78 performs projective conversion on the face image fg4 to obtain a converted image fg4a.

The correlation calculation unit 79 calculates a correlation value between the face image fg3 and the converted image fg4a. Here, the correlation value calculated by the correlation calculation unit 79 is 0.8832, which is lower than the case of FIG. The determination unit 80 determines that the face image fg3 and the face image fg4 are three-dimensional objects with a sense of depth, and are actual face images of the person hm. In addition to the determination result, the determination unit 80 outputs a superimposed image mg2 in which the face image fg3 and the converted image fg4a are superimposed, and displays the superimposed image mg2 on the display device 12. In the superimposed image mg2, the face image fg3 and the converted image fg4a are slightly shifted from each other, and there are not a few doubles. Here, the converted image fg4a is displayed in magenta and the face image fg3 is displayed in green. Since red purple and green have a complementary color relationship, in the superimposed image mg2, the image portion where they match is displayed in monotone, and the mismatched image portion is displayed in red purple or green. Therefore, the superimposed image mg2 is displayed like two doubled face images in which the green image portion gn and the reddish purple image portion rd are mixed.

Thus, the face image of an actual person shows a low correlation value because the face surface has irregularities. Therefore, face authentication is approved when an actual person's face is imaged by two imaging devices.

Next, the operation of the face authentication system 100 according to Embodiment 1 will be described with reference to FIGS.

FIG. 7 is a flowchart for explaining an example of a face authentication operation procedure according to the first embodiment in time series.

In FIG. 7, the face recognition unit 21 of the face authentication device 20 takes captured images GZ1 and GZ2 captured by the

imaging devices

11A and 11B (S1). The face recognition unit 21 detects the face of the target person in the captured images GZ1 and GZ2 (S2). The face recognition unit 21 determines whether or not the face of the target person has been detected (S3). When the face is not detected, the process of the face authentication device 20 returns to step S1.

On the other hand, when a face is detected, the face collation unit 24 collates the detected face and authenticates the user (S4). The face matching unit 24 determines whether the facial feature information of the target person matches the matching data of the registrant's face (S5).

When the facial feature information of the target person does not match the collation data of the registrant's face, the validity determination unit 25 determines invalidity, and a message indicating rejection (ie, message indicating NG) as the authentication result of face authentication. ) Is output (S9). Thereafter, the face authentication device 20 ends the process shown in FIG.

On the other hand, when the facial feature information of the target person matches the registrant's face collation data, the flat panel detection unit 70 determines the plane of the face image (S6). The determination unit 80 determines whether or not the face image is a plane (S7). In the case of a flat surface, the validity determination unit 25 determines invalidity in step S9 and outputs denial as the authentication result of face authentication. If the plane is not a plane in step S7, the validity determination unit 25 determines the validity and outputs an approval message (that is, an OK message) as the authentication result of face authentication (S8). Thereafter, the face authentication apparatus 20 ends this process.

FIG. 8 is a flowchart for explaining an example of the operation procedure of the face image plane determination according to step S6 in time series.

In FIG. 8, the

face detection units

71 and 72 of the flat panel detection unit 70 detect the face images in the captured images, respectively. The

feature point detectors

74 and 75 each detect a feature point in the face image (S11).

The feature point matching unit 76 performs feature point matching between two face images (S12). The projective transformation estimation unit 77 calculates a projective transformation matrix based on the result of feature point matching (S13). The projective transformation unit 78 performs projective transformation on one of the two face images, and acquires a transformed image. (S14).

The correlation calculation unit 79 calculates a correlation value between the face image not subjected to projective conversion and the converted image (S15). The determination unit 80 determines whether the face image is a plane based on the correlation value (S16). Thereafter, the face authentication device 20 ends the process shown in FIG. Thereby, the process of the face authentication apparatus 20 proceeds to step S7 in FIG.

As described above, in the face authentication system 100 according to the first embodiment, the flat panel detection unit 70 (an example of the image input unit) of the face authentication device 20 includes the two

imaging devices

11A and 11B (an example of a plurality of cameras). A captured image GZ1 (an example of a first captured image) and a captured image GZ2 (an example of a second captured image) in which the same face image (person) is captured are input. The

face detection units

71 and 72 detect at least one face image fg1 or fg2 (face region) of the captured image GZ1 and the captured image GZ2, respectively. The determination unit 80 (an example of a plane determination unit) determines whether the face images fg1 and fg2 are planes based on feature points (face information) corresponding to the two face images fg1 and fg2 and the face images fg1 and fg2. Determine whether or not. The validity determination unit 25 (an example of the validity determination unit) determines the validity of the face authentication result based on the determination result of whether or not the face images fg1 and fg2 are planes.

As a result, the face authentication device 20 or the face authentication system 100 detects impersonation using an image that can be performed at the time of authentication of the target person with high accuracy, and effectively suppresses malicious acts of a malicious third party due to impersonation. be able to. For example, even if a regular registrant's face image (still image, moving image) is displayed on a display unit such as a tablet terminal and impersonation is attempted, the face image is determined to be a plane. That is, this face image is determined to be different from the face image of a human face that is a three-dimensional object. Therefore, face authentication is denied.

In addition, the face matching unit 24 of the face authentication device 20 uses the face feature information (an example of face information corresponding to the face area) of the detected face image and the registered matching data (an example of face matching data). Match. The validity determination unit 25 determines the validity of the face authentication result of the target person based on the face comparison result and the determination result of whether or not the face image is a plane. In this way, the face authentication device 20 or the face authentication system 100 can reduce the number of times of plane determination by performing plane determination on the face images with matching face matching.

Further, the face authentication device 20 includes an ID reading device 13 (an example of an ID acquisition unit) that acquires person identification information (an example of ID information). The validity determination unit 25 determines the validity of the authentication result based on the determination result whether the face image is a plane and the ID information. As described above, the face authentication device 20 or the face authentication system 100 can improve the accuracy of face authentication by using the ID information.

In addition, the flat panel detection unit 70 (an example of a plane determination unit) captures one of the face images fg1 and fg2 in the direction of capturing the one face image fg1 and the other face image fg2. Whether or not the face image is a plane is determined according to the correlation value with the converted image fg2a (an example of the projective conversion image) obtained by projective conversion in the imaging direction at that time. As described above, the face authentication device 20 or the face authentication system 100 can objectively determine whether or not it is a plane by using the correlation value, so that the result of the plane determination does not cause doubt, and the determination accuracy increases.

Further, the plurality of imaging devices 11 (an example of a camera) are arranged at different heights in the vertical direction. The optical axes of the plurality of imaging devices 11 are parallel to each other. Thereby, each imaging device 11 can reliably capture the face image of the authentication target person 30 for two frames regardless of the height of the authentication target person 30. In addition, by making the optical axes of the plurality of imaging devices 11 parallel, it is possible to prevent the face image from being greatly displaced in the vertical direction. Further, since the captured images are from the same direction, the processing for calculating these correlation values can be reduced, and the calculation accuracy of the correlation values is increased.

(Another example of face authentication processing in Embodiment 1)
FIG. 9 is a flowchart illustrating an example of another face authentication operation procedure according to the first embodiment in time series. In the description of FIG. 9, the same step process as the face authentication process shown in FIG. 7 is denoted by the same step number or the corresponding step number, the description is simplified or omitted, and different contents will be described.

In FIG. 9, in another face authentication process, the face authentication device 20 performs plane determination first after a face image is detected. Thereafter, face comparison of the target person is performed. When the face recognition unit 21 detects a face image in step S3, the flat panel detection unit 70 performs plane determination of the face image (S6). The determination unit 80 determines whether or not the face image is a plane (S7). In the case of a plane, the determination unit 80 determines whether or not the number of times determined to be a plane is a predetermined number or more (S7A). When the number is less than the predetermined number, the face authentication device 20 returns to the process of step S1. On the other hand, when the number of times determined to be a plane is equal to or greater than the predetermined number, the validity determination unit 25 determines invalidity and outputs a message indicating rejection (that is, a message indicating NG) as the authentication result of face authentication. (S9). Thereafter, the face authentication apparatus 20 ends the process shown in FIG.

On the other hand, if the face image is not a plane in step S7, the face collation unit 24 collates the detected face and authenticates the person (S4). The face matching unit 24 determines whether or not the feature information of the target person's face matches the matching data of the registrant's face (S5). If it does not match the collation data, the validity determination unit 25 determines invalidity in step S9, and outputs denial as the authentication result of face authentication. If it matches with the collation data in step S5, the validity determination unit 25 determines the validity and outputs a message indicating approval (ie, a message indicating OK) as the authentication result of face authentication (S8). . Thereafter, the face authentication apparatus 20 ends the process shown in FIG.

Thus, face matching can be performed only when it is determined that the face image is not flat. Therefore, it is possible to reduce the number of face matching operations that require a large processing load.

Further, when it is determined that the face image is not a plane, the face matching unit 24 checks whether or not the face feature information of the target person matches the registrant's face matching data. The validity determination unit 25 determines the validity of the authentication result of the person's face information based on the collation result. Thereby, the face authentication apparatus 20 or the face authentication system 100 can omit the face authentication process when the face image is determined to be a plane, and can reduce the processing load.

Also, the validity determination unit 25 denies the validity of the verification result as to whether or not the person is the person when it is determined a predetermined number of times that the face image of the face area is a plane. Thereby, for example, when the face authentication device 20 or the face authentication system 100 determines that the face of the person is a flat surface not only once but multiple times, there is a possibility of erroneous determination that the face is a single flat surface. It is possible to omit the execution of face collation processing with a large processing load while rejecting, and output denial as the authentication result of the person's face authentication. Therefore, in the face authentication device 20 or the face authentication system 100, the processing load can be reduced.

(Example of Arrangement of Other Imaging Devices in Embodiment 1)
FIG. 10 is a diagram illustrating an arrangement example of the other three

imaging devices

11i, 11j, and 11k according to the first embodiment. The three

imaging devices

11i, 11j, and 11k are arranged in a direction perpendicular to the plane of the passage through which the person 30 to be authenticated passes (that is, a direction perpendicular to the horizontal plane). The three

imaging devices

11i, 11j, and 11k image the face existence range SA of the authentication target person 30 so that the angles of view overlap in the passage width wd. In this case, the face of a person such as a short child is imaged by the imaging device 11k installed at a low position and the imaging device 11i installed at a high position. In addition, the face of a person having a general height is imaged by the three

imaging devices

11i, 11j, and 11k. Further, the face of a tall person such as an adult is imaged by the imaging device 11i installed at a high position and the imaging device 11j installed at an intermediate position. The optical axes of the three

imaging devices

11i, 11j, and 11k are arbitrarily set according to the face existence range SA and are not parallel. Therefore, when a person with a different height tries to pass through the passage, a face image is captured by at least two imaging devices.

Thus, the plurality of imaging devices are arranged at different heights in the vertical direction, and the optical axes of the plurality of imaging devices are not parallel to each other. Accordingly, the face image of the target person can be reliably captured by two or more imaging devices regardless of the height of the subject. In addition, by making the optical axes of a plurality of cameras non-parallel, it is possible to easily capture face images of target persons having different heights. Therefore, in order to capture two face images, the number of installed image pickup devices can be reduced.

(Embodiment 2)
In the first embodiment, when the face image is a plane, it is determined that the result of the face authentication is denial by assuming that the display unit is impersonated. In the second embodiment, in addition to the configuration of the first embodiment, a case where it is determined that the display unit is impersonated even when a frame image is detected in the captured image is shown.

In the face authentication system according to the second embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof is simplified or omitted.

FIG. 11 is a diagram illustrating a functional configuration example realized by cooperation of a processor and a memory built in the face authentication apparatus 20A according to the second embodiment. 20 A of face authentication apparatuses have the frame detection part 22 and the arrangement | positioning determination part 23 other than the face recognition part 21, the face collation part 24, the effectiveness determination part 25, and the flat panel detection part 70 which were shown in Embodiment 1. FIG. The face recognition unit 21, the frame detection unit 22, the arrangement determination unit 23, the face collation unit 24, the validity determination unit 25, and the flat panel detection unit 70 are, for example, a processor built in the face authentication device 20 </ b> A that is a PC (Personal Computer) and This is realized by the processor executing a predetermined program stored in the memory in cooperation with the memory.

The frame detection unit 22 uses the captured image and the background image to detect a linear frame in the input image obtained by removing the background image from the captured image. The frame detection unit 22 acquires frame position information (in other words, frame arrangement information) when there is a frame of the display unit in the captured image.

The arrangement determination unit 23 determines the arrangement of whether or not the face is located within the frame of the display unit based on the face arrangement information and the frame arrangement information acquired by the face recognition unit 21. As a result of the placement determination, the placement determination unit 23 detects that a display unit (see above) is present in the captured image and a face image is displayed within the frame of the display unit.

Based on the results of face matching by the face matching unit 24, plane determination by the flat panel detecting unit 70, and frame detection by the placement determining unit 23, the validity determining unit 25 determines whether the face of the target person shown in the captured image is valid. It is determined whether it is approval or disapproval as the validity of the result of face authentication. The validity determination unit 25 outputs the authentication result to the control target device 40.

FIG. 12 is an explanatory diagram showing an example of an outline of the operation of the frame detection process performed by the frame detection unit 22. Here, a situation is assumed in which spoofing is performed by reproducing an image (still image increase, moving image) in which a face of a regular registrant is reflected on a portable display unit. The

imaging devices

11A and 11B each image the display unit.

The captured image captured by one of the

imaging devices

11A and 11B is set as an input image 51 input to the frame detection unit 22. The input image 51 includes a housing 53 of the display unit and a person 54 displayed on the screen of the display unit. The frame detection unit 22 performs a known contour extraction (for example, edge detection) process on the input image 51 to generate an edge image 52. The edge image 52 includes a contour 55 of the housing of the display unit and a contour 56 of a person displayed on the screen of the display unit. The frame detection unit 22 performs straight line detection processing on the edge image 52 and extracts a straight line portion (line segment) 57 in the image. For the straight line detection processing, for example, straight line detection using Hough transform or the like is used. The frame detection unit 22 determines whether there are two parallel straight lines and a straight line perpendicular thereto in the straight line portion 57 in the extracted image, extracts a frame 58 surrounded by four sides, and extracts frame coordinates. To get. Note that the frame detection unit 22 is surrounded by a line segment based on a predetermined condition, even if it is a set of square line segments whose four sides are not closed, or a group of line segments orthogonal to three sides or two sides. It may be determined as a frame and the frame 58 may be extracted.

Further, the frame detection unit 22 may perform straight line detection and frame extraction after removing the background from the captured image in the frame detection process. The accuracy of frame detection can be improved by removing the background portion from the captured image.

FIG. 13 is an explanatory diagram showing an operation outline example of the arrangement determination processing by the arrangement determination unit 23. The arrangement determination unit 23 acquires face arrangement information of the target person obtained as one of the face detection results by the face recognition unit 21. The face arrangement information includes a face frame 68 indicating the contour position of the face in the captured image. Further, the arrangement determination unit 23 acquires frame arrangement information of the display unit obtained as one of the frame detection results by the frame detection unit 22. The frame arrangement information includes a frame 58 indicating the contour position of the display unit detected in the captured image.

As the placement determination process, the placement determination unit 23 compares the positions of both the face frame 68 and the frame 58, and determines whether or not the face frame 68 exists in the area inside the frame 58, that is, the target in the display unit frame. It is determined whether or not a person's face is located. As shown in FIG. 13, when there is a face frame 68 inside the frame 58, the arrangement determination unit 23 outputs an arrangement determination result that there is a face image in the frame.

The validity determination unit 25 regards the face image detected in the captured image as the face image displayed on the display unit when the face image is present in the frame based on the result of the placement determination by the placement determination unit 23, It is determined that the verification result is invalid. The validity determination unit 25 outputs a denial to the control target device 40 as a result of the face authentication.

Next, the operation of the face authentication system 100 according to Embodiment 2 will be described with reference to FIG.

FIG. 14 is a flowchart illustrating an example of a face authentication operation procedure according to the second embodiment in time series. About the same step process as the face authentication process shown in FIG. 7, the description is simplified or abbreviate | omitted by attaching | subjecting the same step number.

If it is determined in step S7 that the plane is not a plane, the frame detection unit 22 detects a frame in the captured image (S7B). The arrangement determination unit 23 determines the arrangement of the face position by face detection and the frame position by frame detection (S7C). The arrangement determining unit 23 detects a frame in the captured image and determines whether or not a face exists inside the frame, that is, whether or not a frame surrounding the outside of the target person's face exists (S7D). The arrangement determination unit 23 passes the determination result of the frame existence to the validity determination unit 25. If there is a frame surrounding the outside of the target person's face as a result of the frame presence determination, the validity determination unit 25 determines that the face authentication is invalid and outputs a rejection to the control target device 40 as the face authentication result. If there is no frame surrounding the outside of the target person's face as a result of the frame presence determination, the validity determination unit 25 determines that the face authentication is valid, and outputs approval to the control target device 40 as the face authentication result.

Here, if it is determined in step S7 that the plane is not a plane, the frame detection processing has been performed in steps S7B to S7D. If they match, frame detection processing similar to steps S7B to S7D may be performed.

In the face authentication process, the face detection process and the frame detection process may be executed in parallel. Moreover, after performing one process in series, you may perform the other process, and the implementation timing and order of a process are not limited.

As described above, in the face authentication system 100 according to the second embodiment, the face detection of the target person is performed by using the result of the frame detection by the arrangement determination unit 23 in the face authentication determination in addition to the face matching and plane determination processing. Can be performed with high accuracy.

Further, even when it is determined that the face image of the target person is not a plane, face authentication can be denied if the captured image including the face image includes a frame image. For example, when the target person is captured by a plurality of imaging devices while inverting the orientation of the tablet terminal in small increments or continuously switching the face image displayed on the tablet terminal to 3D display or 2D display, May be determined to be non-planar. Even in such a case, when there is a frame image in the captured image, it is determined that the face image is reflected on the tablet terminal, and the face image of the target person can be denied.

On the other hand, even if there is an action such as hiding the frame of the display unit on which the face image is displayed, such an action can be dealt with by plane determination. Further, by combining the plane determination, it is possible to avoid a false detection that a face authentication result is NG only by detecting a frame even for a person who is not a face authentication target.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It will be apparent to those skilled in the art that various changes, modifications, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims. It is understood that they belong to the technical scope of the present disclosure. In addition, the constituent elements in the various embodiments described above may be arbitrarily combined without departing from the spirit of the invention.

The present disclosure accurately detects impersonation using an image that can be performed at the time of authentication of a target person, and effectively suppresses malicious acts of a malicious third party due to impersonation, a face authentication method, and face authentication Useful as a system.

11A, 11B Imaging device 12 Display device 13

ID reader

20, 20A Face authentication device 21 Face recognition unit 22 Frame detection unit 23 Arrangement determination unit 24 Face collation unit 25 Effective determination unit 40 Control target device 70 Flat

panel detection unit

71, 72 Face detection unit 73 Face

correspondence determination unit

74, 75 Feature point detection unit 76 Feature point matching unit 77 Projection conversion estimation unit 78 Projection conversion unit 79 Correlation calculation unit 80 Determination unit 100 Face authentication system

Claims

An image input unit that inputs a first captured image and a second captured image in which the same person is captured by each of a plurality of cameras;
A face detector for detecting at least one face area of the first captured image and the second captured image;
A plane determination unit that determines whether the face image of the face area is a plane based on the face information corresponding to the face area and the first captured image and the second captured image;
A validity determination unit that determines the validity of an authentication result of the person's face information based on a determination result of whether or not the face image of the face region is a plane;
Face recognition device.
A collation unit that collates face information corresponding to the detected face area and registered face collation data;
The effectiveness determination unit includes:
Based on the collation result of the face information and the determination result of whether or not the face image of the face area is a plane, the validity of the authentication result of the person's face information is determined.
The face authentication apparatus according to claim 1.
A collation unit that collates face information corresponding to the detected face area and registered face collation data;
The collation unit
When it is determined that the face image of the face area is not a plane, the face information is collated,
The effectiveness determination unit further includes:
Determining the validity of the authentication result of the person's face information based on the face information verification result;
The face authentication apparatus according to claim 1.
An ID acquisition unit for acquiring ID information of the person,
The effectiveness determination unit includes:
Determining the validity of the authentication result based on the determination result whether the face image of the face area is a plane and the ID information;
The face authentication apparatus according to claim 1.
The plane determination unit
Depending on the correlation between one of the first captured image and the second captured image and a projective transformation image obtained by projective transformation of the imaging direction of the other captured image into the imaging direction of the one captured image. Determining whether the face image of the face area in the one captured image is a plane,
The face authentication apparatus according to claim 1 or 2.
The effectiveness determination unit includes:
When it is determined a predetermined number of times that the face image of the face area is a plane, the validity of the collation result as to whether the person is the person is denied,
The face authentication apparatus according to claim 1.
A face authentication method in a face authentication device,
Inputting a first captured image and a second captured image in which the same person is captured by each of a plurality of cameras;
Detecting at least one face area of the first captured image and the second captured image;
Determining whether the face image of the face area is a plane based on the face information corresponding to the face area and the first captured image and the second captured image;
Determining the validity of the authentication result of the person's face information based on the determination result of whether or not the face image of the face region is a plane,
Face authentication method.
A face authentication system in which a plurality of cameras and a face authentication device are communicably connected to each other,
The plurality of cameras are:
Outputting a first captured image and a second captured image each capturing the same person to the face authentication device;
The face authentication device
Inputting the first captured image and the second captured image;
Detecting at least one face area of the first captured image and the second captured image;
Based on the face information corresponding to the face area and the first captured image and the second captured image, it is determined whether or not the face image of the face area is a plane,
Determining the validity of the authentication result of the person's face information based on the determination result of whether or not the face image of the face region is a plane;
Face recognition system.
The plurality of cameras are:
The optical axes of the plurality of cameras are parallel to each other and are arranged at different heights in the vertical direction.
The face authentication system according to claim 8.
The plurality of cameras are:
The optical axes of the plurality of cameras are non-parallel to each other while being arranged at different heights in the vertical direction.
The face authentication system according to claim 8.