JP2019197426A - Face authentication device, face authentication method, and face authentication system - Google Patents

Abstract

To accurately detect impersonation by use of an image obtained upon authentication of a target person and effectively suppress malicious acts of malicious third parties through the impersonation.SOLUTION: In a face authentication system, a flat panel detection part 70 receives input of a captured image GZ1 and a captured image GZ2 which have been respectively obtained by capturing the same face image by two capturing devices. Face detection parts 71 and 72 respectively detect face images fg1 and fg2 of at least one of the captured images GZ1 and GZ2. A determination part 80 determines based on feature points corresponding to the two face images fg1 ang fg2 and the face images fg1 and fg2 whether or not the face images fg1 and fg2 are planes. Based on results of the determination whether or not the face images fg1 and fg2 are planes, a validity determination part 25 determines the validity of the face authentication results.SELECTED DRAWING: Figure 4

Description

  The present disclosure relates to a face authentication device, a face authentication method, and a face authentication system that authenticate a target person using an image of a face portion in a captured image captured by a camera.

  In Patent Literature 1, a face image sequence for a predetermined time is input, and whether or not a face included in the face image sequence is impersonated based on temporal changes in predetermined color information extracted from the face image sequence. An information processing apparatus for determining whether or not is disclosed.

International Publication No. 2015/122183

  However, in the configuration of Patent Document 1, when an improper person uses a portable display terminal such as a tablet terminal to display an image showing the face of an authorized registrant and impersonates the image, Even if the used face authentication is performed, fraud cannot be detected, and there is a possibility that the authentication result is approved.

  The present disclosure has been devised in view of the above-described conventional situation, detects impersonation using an image that can be performed at the time of authentication of a target person with high accuracy, and effectively prevents malicious third party fraud by impersonation. An object of the present invention is to provide a face authentication device, a face authentication method, and a face authentication system to be suppressed.

  In the present disclosure, an image input unit that inputs a first captured image and a second captured image in which the same person is captured by each of a plurality of cameras, and a face area of at least one of the first captured image and the second captured image A plane detection unit that detects whether or not the face image of the face area is a plane based on face information corresponding to the face area and the first and second captured images And a validity determination unit that determines the validity of the authentication result of the person's face information based on a determination result of whether or not the face image of the face area is a plane. provide.

  Further, the present disclosure is a face authentication method in a face authentication apparatus, the step of inputting a first captured image and a second captured image in which the same person is captured by each of a plurality of cameras, and the first captured image And detecting a face area of at least one of the second captured images, and based on the face information corresponding to the face area and the first captured image and the second captured image, the face image of the face area is a plane. Determining whether or not there is a step of determining the validity of the authentication result of the face information of the person based on the determination result of whether or not the face image of the face region is a plane, Provide a face recognition method.

  In addition, the present disclosure is a face authentication system in which a plurality of cameras and a face authentication device are connected to be able to communicate with each other, and the plurality of cameras each include a first captured image and a second captured image capturing the same person. An image is output to the face authentication device, the face authentication device receives the first captured image and the second captured image, detects at least one face region of the first captured image and the second captured image, Based on the face information corresponding to the face area and the first captured image and the second captured image, it is determined whether the face image of the face area is a plane, and the face image of the face area is a plane. Provided is a face authentication system that determines the validity of an authentication result of the person's face information based on a determination result of whether or not there is.

  According to the present disclosure, it is possible to detect impersonation using an image that can be performed at the time of authentication of a target person with high accuracy, and to suppress malicious third party fraud by impersonation.

The figure which shows the system configuration example of the face authentication system which concerns on Embodiment 1. FIG. The figure which shows the example of arrangement | positioning of four imaging devices FIG. 3 is a block diagram showing a functional configuration example realized by cooperation of a processor and a memory built in the face authentication apparatus according to the first embodiment. The block diagram which shows an example of a functional structure and an operation | movement outline | summary of a flat panel detection part. Explanatory drawing which shows the operation | movement outline example which image | photographs the face image projected on the flat panel, and performs plane determination Explanatory drawing which shows the example of an operation outline which picturizes an actual person's face, and performs plane judgment A flowchart for explaining an example of an operation procedure of face authentication according to the first embodiment in time series Flowchart for explaining an example of an operation procedure of face image plane determination according to step S6 in time series A flowchart illustrating an example of another face authentication operation procedure according to the first embodiment in time series. The figure which shows the example of arrangement | positioning of the other three imaging devices which concern on Embodiment 1. FIG. FIG. 9 is a block diagram illustrating a functional configuration example realized by cooperation of a processor and a memory built in the face authentication apparatus according to the second embodiment. Explanatory drawing which shows the operation | movement outline example of the frame detection process by a frame detection part. Explanatory drawing which shows the operation | movement outline example of the arrangement | positioning determination process by an arrangement | positioning determination part. Flowchart for explaining an example of an operation procedure of face authentication according to the second embodiment in time series

  Hereinafter, an embodiment that specifically discloses a face authentication device, a face authentication method, and a face authentication system according to the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

(Embodiment 1)
FIG. 1 is a diagram illustrating a system configuration example of a face authentication system 100 according to the first embodiment. The face authentication system 100 performs face authentication of a person to be authenticated by collating face image data obtained by capturing an authentication target person with an imaging device and face image data of a regular person registered in advance. The face authentication system 100 outputs the face authentication result of the person to be authenticated to the control target device 40.

  The control target device 40 is, for example, a usage management device used by a service provider such as car sharing or a security area entrance / exit management device, but is not limited thereto.

  When the control target device 40 is a car sharing usage management device, the usage management device permits the use of the car sharing target vehicle when the user is a registered regular user as a face authentication result. On the other hand, when the user is not a registered regular user, the usage management device refuses to use the vehicle to be shared. When permitting or denying the use of a vehicle, the usage management device sends an authentication result indicating whether or not the vehicle can be used, for example, to an electronic control unit (ECU) mounted on the vehicle. The electronic control unit sets the start (for example, ignition on) of the own vehicle to be valid or invalid based on the authentication result sent from the usage management device.

  When the control target device 40 is a security area entrance / exit management device, the entrance / exit management device permits a person who is permitted to enter / exit as a face authentication result to pass the person to the security area. On the other hand, if the entrance / exit management device is not a permitted person as a face authentication result, the entrance / exit management device blocks the person from entering the security area or intrudes the administrator or operator of the face authentication system 100. Notification that the person has been detected.

  The face authentication system 100 includes a face authentication device 20, a plurality of (for example, two) imaging devices 11A and 11B, a display device 12, and an ID reading device 13. When there is no need to distinguish between the two imaging devices 11A and 11B, they are simply referred to as the imaging device 11. The two imaging devices 11A and 11B are arranged at positions where, for example, the face of a person walking in a passage can be simultaneously imaged from different directions. The imaging device 11 includes an imaging lens, an imaging device, an image signal processing circuit, a communication interface, and the like. The imaging device 11 captures an imaging region including the authentication target person 30 as a subject, and outputs the captured image to the face authentication device 20. Note that the number of imaging devices is not limited to two, and may be any number. The imaging device 11 may be a monitoring camera for monitoring a predetermined area, for example.

  FIG. 2 is a diagram illustrating an arrangement example of the four imaging devices 11e, 11f, 11g, and 11h. The four imaging devices 11e, 11f, 11g, and 11h are arranged in a direction perpendicular to the surface of the passage through which the person 30 to be authenticated passes (that is, a direction perpendicular to the horizontal plane). The four imaging devices 11e, 11f, 11g, and 11h image the presence range SA of the face of the person 30 to be authenticated so that the angles of view overlap in the passage width wd. The optical axes of the four imaging devices 11e, 11f, 11g, and 11h are all parallel, for example.

  For example, the face of a person such as a short child is imaged by two imaging devices 11e and 11f installed at a low position. In this case, the two imaging devices 11e and 11f are used as the imaging devices 11A and 11B in FIG. Further, for example, the face of a person such as a tall adult is imaged by two imaging devices 11g and 11h installed at a high position. In this case, the two imaging devices 11g and 11h are used as the imaging devices 11A and 11B in FIG. When the imaging devices 11e, 11f, 11g, and 11h are not particularly distinguished, they are referred to as the imaging device 11. Further, when three or more face images of the target person are picked up by three or more image pickup devices, two face images of these face images are used for face authentication. Note that three or more face images may be temporally continuous three frames or temporally discontinuous three frames.

  The ID reader 13 is composed of an information reader such as a card reader. The ID reading device 13 reads the identification information (for example, ID information) of the ID card 35 for personal identification possessed by the person 30 to be authenticated, and outputs the read identification information to the face authentication device 20. Examples of the ID card 35 include a membership card, an identification card, a driver's license, and a my number card.

  The face authentication device 20 is configured by a PC (personal computer) or the like, and includes a processor, a memory, a communication interface, and the like. The face authentication device 20 performs face authentication using the image information of the captured image acquired from the imaging device 11 and the ID information acquired from the ID reading device 13, and outputs the face authentication result to the control target device 40. Details of the face authentication process will be described later.

  The display device 12 includes a display device such as a liquid crystal display. The display device 12 displays various data or information when the face authentication device 20 executes face authentication. Various data or information includes an operation guidance screen, a captured image, a face detection result, a frame detection result, a face matching result, an authentication result including face authentication, and the like. The display device 12 is disposed in a monitoring room or the like that is visible to the administrator or operator of the face authentication system 100. The display device 12 may be disposed in the vicinity of the imaging device 11 so that guidance display can be performed for the person 30 to be authenticated.

  FIG. 3 is a block diagram illustrating a functional configuration example realized by cooperation of a processor and a memory built in the face authentication apparatus 20 according to the first embodiment. The face authentication device 20 includes a face recognition unit 21, a face collation unit 24, an validity determination unit 25, and a flat panel detection unit 70. The face recognizing unit 21, the face collating unit 24, the validity determining unit 25, and the flat panel detecting unit 70 are, for example, predetermined processors stored in the memory by the cooperation of the processor and the memory built in the face authentication device 20 which is a PC. This is realized by executing the program.

  The face recognition unit 21 inputs the data of the captured images GZ1 and GZ2 (that is, the captured images GZ1 and GZ2 including the image of the face portion) sent from the two imaging devices, and uses a known face recognition technique. The face of the person 30 to be authenticated included in the captured image is detected by the face recognition process. The face recognition unit 21 acquires face feature information and position information in the captured image (in other words, face arrangement information) by face recognition processing. The face feature information may include vital information such as skin color, blood flow, and hair color in addition to the feature points of the face.

  The face collating unit 24 uses the face feature information detected by the face recognizing unit 21 and the collation data of the face information of the regular registrant, and the face in the captured image detected by the face recognizing unit 21 and the regular registrant. It is determined whether or not the face matches. The collation data is recorded on, for example, a recording medium built in the face authentication device 20, and includes image data of a regular registrant's face image and feature data of face feature information. For example, the administrator or operator of the face authentication system 100 sends the verification data included in the ID information recorded on the ID card 35 possessed by the person 30 to be authenticated to the face verification unit 24 of the face authentication device 20 as the verification data. input. The input verification data is stored in a recording medium that can be read by the face verification unit 24.

  The validity determination unit 25 determines whether or not the face of the person to be authenticated 30 included in the captured image is valid (that is, whether or not the face authentication is approved) as a result of the face matching by the face matching unit 24, Outputs the face authentication result.

  The flat panel detection unit 70 determines whether the face image detected in the captured image of the person 30 to be authenticated is a plane (see FIG. 4).

  FIG. 4 is a block diagram illustrating an example of a functional configuration and an outline of operation of the flat panel detection unit 70. The flat panel detection unit 70 includes a face detection unit 71, 72, a face correspondence determination unit 73, a feature point detection unit 74, 75, a feature point matching unit 76, a projective transformation estimation unit 77, a projective transformation unit 78, a correlation calculation unit 79, A determination unit 80 is included.

  The face detection units 71 and 72 respectively detect the face of the target person included in the corresponding captured images GZ1 and GZ2 in which the authentication target person 30 is captured, and acquire face arrangement information. The face arrangement information is feature point detection ranges rc1 and rc2 (see FIG. 5), and is information representing a rectangular range including, for example, eyes, nose and mouth. The face detection unit 71 performs face detection on the captured image GZ1 captured by the imaging device 11A. The face detection unit 72 performs face detection on the captured image GZ2 captured by the imaging device 11B.

  The feature point detection units 74 and 75 detect face feature points included in the face image specified by the face arrangement information by, for example, feature point detection processing using a known face feature point detection technique. As facial feature points, for example, eyes, nose, mouth position, mouth position, deep nostril position, eye corner position, bangs curve on the forehead, distance between the eye corners, facial contour, protruding A part that characterizes a facial part such as the position of the tip of the nose is included. The feature point detection unit 74 detects a feature point of the face included in the face image fg1 obtained by the face detection unit 71. The feature point detection unit 75 detects a feature point of the face included in the face image fg2 obtained by the face detection unit 72. Here, as an example, the case where the feature point is the position of the mouth on both sides, the position of the nostril, and the bangs curve on the forehead is shown. The feature point detectors 74 and 75 detect, for example, a corner portion (in other words, a contour) of a face to detect a feature point of the face, or a SIFT (Scale-Invariant Feature Transform) or SURF (Speeded-Up Robust). The feature points of the face may be detected using a known feature point extraction algorithm such as Features.

  The feature point matching unit 76 uses the outputs from the feature point detection units 74 and 75 to perform feature point matching processing. In the feature point matching process, the same feature points correspond to the plurality of feature points p1a, p2a, p3a detected in the face image fg1 and the plurality of feature points p1b, p2b, p3b detected in the face image fg2. Attached. In the face images fg1 and fg2, the feature points are associated with the line segments c1 and p3b between the feature points p1a and p1b, between the feature points p2a and p2b, and between the feature points p3a and p3b, respectively. It is expressed by connecting with c2 and c3 (see FIG. 5).

  The projective transformation estimation unit 77 calculates a projective transformation matrix based on the coordinates of two corresponding feature points included in the face images fg1 and fg2. The projective transformation matrix is a matrix for transforming coordinates in order to project a certain plane onto another plane. Here, the projective transformation matrix performs coordinate transformation for projecting the face of the face image fg2 onto the face of the face image fg1. The projective transformation estimation unit 77 may calculate a projective transformation matrix that projects the face of the face image fg1 onto the face of the face image fg2. The projective transformation matrix is represented by a 3 × 3 matrix, for example. In this projective transformation matrix, each element of a matrix represented by a 3 × 3 matrix is calculated by associating the coordinates of at least four matching feature points. Further, the projective transformation estimation unit 77 may calculate a projective transformation matrix using a least square method or the like using four or more corresponding points (that is, coordinates indicating the positions of feature points that match each other). . Thereby, the projective transformation estimation unit 77 can calculate a more accurate projective transformation matrix.

  The projective transformation unit 78 performs projective transformation on the face image fg2 using the projective transformation matrix calculated by the projective transformation estimation unit 77, and obtains a transformed image fg2a.

  The correlation calculation unit 79 calculates a correlation value indicating the similarity between the two images based on the face image fg1 and the converted image fg2a. In the calculation of the correlation value, template matching is performed in which a position similar to a designated image (for example, a template) is searched from among the images. Here, normalized cross-correlation (NNC) is used for the calculation of the correlation value. The calculation of the correlation value uses the sum of squares of differences in luminance values (SSD: Sum of Squared Difference) or the sum of absolute values of differences in luminance values (SAD: Sum of Absolute Difference). Also good.

  Based on the correlation value calculated by the correlation calculation unit 79, the determination unit 80 determines whether or not the face image fg1 is flat (that is, a two-dimensional plane) as projected on the flat panel. The determination result is output. When the correlation value is high, for example, when the correlation value is higher than a preset threshold value, neither the face image fg1 nor the face image fg2 has a three-dimensional structure (for example, an actual human face). A two-dimensional plane). On the other hand, when the correlation value is low, for example, when the correlation value is equal to or less than a preset threshold value, at least one of the face image fg1 and the face image fg2 is not a plane but a three-dimensional object (for example, the face of an actual person) 3D solid object having a sense of depth.

  FIG. 5 is an explanatory diagram showing an example of an outline of operation for performing plane determination by capturing a face image displayed on the flat panel 150. The flat panel 150 is a display unit (so-called display device such as a display) such as a tablet terminal, a digital photo frame, a smartphone, or a notebook PC. The imaging devices 11A and 11B respectively capture the face images displayed on the flat panel 150, and obtain captured images GZ1 and GZ2. At this time, the imaging device 11 </ b> B images the flat panel 150 from the front direction. The imaging device 11A captures an image by slightly tilting the flat panel 150 from the front direction.

  The face detection units 71 and 72 perform face detection on the captured images GZ1 and GZ2, respectively, and obtain feature point detection ranges rc1 and rc2. The feature point detection units 74 and 75 detect face feature points p1a, p2a, and p3a and p1b, p2b, and p3b included in the face images fg1 and fg2 that are surrounded by the feature point detection ranges rc1 and rc2, respectively.

  The feature point matching unit 76 associates a plurality of feature points p1a, p2a, and p3a detected in the face image fg1 with a plurality of feature points p1b, p2b, and p3b detected in the face image fg2, and assigns them to line segments. Connect with c1, c2, and c3.

  The projective conversion unit 78 performs projective conversion on the face image fg2 to obtain a converted image fg2a. The correlation calculation unit 79 calculates a correlation value between the face image fg1 and the converted image fg2a. Here, the correlation value calculated by the correlation calculation unit 79 is 0.999, which is very close to the value 1 and is a high value. The determination unit 80 determines that the face image fg <b> 1 and the face image fg <b> 2 are two-dimensional images, and are flat (that is, flat) face images displayed on the flat panel 150. In addition to the determination result, the determination unit 80 outputs a superimposed image mg1 in which the face image fg1 and the converted image fg2a are superimposed, and displays the superimposed image mg1 on the display device 12. The superimposed image mg1 is displayed like a single face image in which the face image fg1 and the converted image fg2a substantially coincide with each other and there is almost no double.

  Thus, the face image displayed on the flat panel shows a high correlation value. Accordingly, face authentication of a person who impersonates a regular registrant by holding a flat panel on which a regular registrant's face image is projected is denied.

  FIG. 6 is an explanatory diagram showing an example of an outline of operation for performing plane determination by imaging an actual human face. The imaging devices 11A and 11B respectively capture the face of the actual person hm and obtain captured images GZ3 and GZ4. At this time, the imaging device 11A captures an image of the face of the actual person hm from the front direction. The imaging device 11B captures an image by slightly tilting the face of the actual person hm from the front direction.

  The face detection units 71 and 72 perform face detection on the captured images GZ3 and GZ4, respectively, and obtain feature point detection ranges rc3 and rc4. The feature point detection units 74 and 75 detect face feature points p4a, p5a, p6a, and p7a and p4b, p5b, p6b, and p7b included in the face images fg3 and fg4, which are surrounded by the feature point detection ranges rc3 and rc4, respectively. To do.

  The feature point matching unit 76 associates a plurality of feature points p4a, p5a, p6a, and p7a detected in the face image fg3 with a plurality of feature points p4b, p5b, p6b, and p7b detected in the face image fg4, These are connected by line segments c4, c5, c6, and c7.

  The projective conversion unit 78 performs projective conversion on the face image fg4 to obtain a converted image fg4a. The correlation calculation unit 79 calculates a correlation value between the face image fg3 and the converted image fg4a. Here, the correlation value calculated by the correlation calculation unit 79 is 0.8832, which is lower than the case of FIG. The determination unit 80 determines that the face image fg3 and the face image fg4 are three-dimensional objects with a sense of depth, and are actual face images of the person hm. In addition to the determination result, the determination unit 80 outputs a superimposed image mg2 in which the face image fg3 and the converted image fg4a are superimposed, and displays the superimposed image mg2 on the display device 12. In the superimposed image mg2, the face image fg3 and the converted image fg4a are slightly shifted from each other, and there are not a few doubles. Here, the converted image fg4a is displayed in magenta and the face image fg3 is displayed in green. Since red purple and green have a complementary color relationship, in the superimposed image mg2, the image portion where they match is displayed in monotone, and the mismatched image portion is displayed in red purple or green. Therefore, the superimposed image mg2 is displayed like two doubled face images in which the green image portion gn and the reddish purple image portion rd are mixed.

  Thus, an actual person's face image shows a low correlation value because the face surface has irregularities. Therefore, face authentication is approved when an actual person's face is imaged by two imaging devices.

  Next, the operation of the face authentication system 100 according to Embodiment 1 will be described with reference to FIG. 7 and FIG.

  FIG. 7 is a flowchart illustrating an example of an operation procedure of face authentication according to the first embodiment in time series.

  In FIG. 7, the face recognition unit 21 of the face authentication device 20 takes captured images GZ1 and GZ2 captured by the imaging devices 11A and 11B (S1). The face recognition unit 21 detects the face of the target person in the captured images GZ1 and GZ2 (S2). The face recognition unit 21 determines whether or not the face of the target person has been detected (S3). When the face is not detected, the process of the face authentication device 20 returns to step S1.

  On the other hand, when a face is detected, the face collation unit 24 collates the detected face and performs personal authentication (S4). The face matching unit 24 determines whether the facial feature information of the target person matches the matching data of the registrant's face (S5).

  When the facial feature information of the target person does not match the collation data of the registrant's face, the validity determination unit 25 determines invalidity, and a message indicating rejection (ie, message indicating NG) as the authentication result of face authentication. ) Is output (S9). Thereafter, the face authentication device 20 ends the process shown in FIG.

  On the other hand, when the facial feature information of the target person matches the collation data of the registrant's face, the flat panel detection unit 70 performs a plane determination of the face image (S6). The determination unit 80 determines whether or not the face image is a plane (S7). In the case of a flat surface, the validity determination unit 25 determines invalidity in step S9 and outputs denial as the authentication result of face authentication. If the plane is not a plane in step S7, the validity determination unit 25 determines the validity and outputs an approval message (that is, an OK message) as the authentication result of face authentication (S8). Thereafter, the face authentication apparatus 20 ends this process.

  FIG. 8 is a flowchart illustrating an example of the operation procedure of the face image plane determination according to step S6 in time series.

  In FIG. 8, face detection units 71 and 72 of the flat panel detection unit 70 detect face images in the captured images, respectively. The feature point detectors 74 and 75 each detect a feature point in the face image (S11).

  The feature point matching unit 76 performs feature point matching between the two face images (S12). The projective transformation estimation unit 77 calculates a projective transformation matrix based on the result of feature point matching (S13). The projective transformation unit 78 performs projective transformation on one of the two face images, and acquires a transformed image. (S14).

  The correlation calculation unit 79 calculates a correlation value between the face image not subjected to projective transformation and the converted image (S15). The determination unit 80 determines whether the face image is a plane based on the correlation value (S16). Thereafter, the face authentication device 20 ends the process shown in FIG. Thereby, the process of the face authentication apparatus 20 proceeds to step S7 in FIG.

  As described above, in the face authentication system 100 according to the first embodiment, the flat panel detection unit 70 (an example of the image input unit) of the face authentication device 20 includes the two imaging devices 11A and 11B (an example of a plurality of cameras). A captured image GZ1 (an example of a first captured image) and a captured image GZ2 (an example of a second captured image) in which the same face image (person) is captured are input. The face detection units 71 and 72 detect at least one face image fg1 or fg2 (face region) of the captured image GZ1 and the captured image GZ2, respectively. The determination unit 80 (an example of a plane determination unit) determines whether the face images fg1 and fg2 are planes based on feature points (face information) corresponding to the two face images fg1 and fg2 and the face images fg1 and fg2. Determine whether or not. The validity determination unit 25 (an example of the validity determination unit) determines the validity of the face authentication result based on the determination result of whether or not the face images fg1 and fg2 are planes.

  As a result, the face authentication device 20 or the face authentication system 100 detects impersonation using an image that can be performed at the time of authentication of the target person with high accuracy, and effectively suppresses malicious acts of a malicious third party due to impersonation. be able to. For example, even if a regular registrant's face image (still image, moving image) is displayed on a display unit such as a tablet terminal and impersonation is attempted, the face image is determined to be a plane. That is, this face image is determined to be different from the face image of a human face that is a standing object. Therefore, face authentication is denied.

  In addition, the face matching unit 24 of the face authentication device 20 uses the face feature information (an example of face information corresponding to the face area) of the detected face image and the registered matching data (an example of face matching data). Match. The validity determination unit 25 determines the validity of the face authentication result of the target person based on the face comparison result and the determination result of whether or not the face image is a plane. In this way, the face authentication device 20 or the face authentication system 100 can reduce the number of times of plane determination by performing plane determination on the face images with matching face matching.

  Further, the face authentication device 20 includes an ID reading device 13 (an example of an ID acquisition unit) that acquires person identification information (an example of ID information). The validity determination unit 25 determines the validity of the authentication result based on the determination result whether the face image is a plane and the ID information. As described above, the face authentication device 20 or the face authentication system 100 can improve the accuracy of face authentication by using the ID information.

  In addition, the flat panel detection unit 70 (an example of a plane determination unit) captures one of the face images fg1 and fg2 in the direction of capturing the one face image fg1 and the other face image fg2. Whether or not the face image is a plane is determined according to the correlation value with the converted image fg2a (an example of the projective conversion image) obtained by projective conversion in the imaging direction at that time. As described above, the face authentication device 20 or the face authentication system 100 can objectively determine whether or not it is a plane by using the correlation value, so that the result of the plane determination does not cause doubt, and the determination accuracy increases.

  The plurality of imaging devices 11 (an example of a camera) are arranged at different heights in the vertical direction. The optical axes of the plurality of imaging devices 11 are parallel to each other. Thereby, each imaging device 11 can reliably capture the face image of the authentication target person 30 for two frames regardless of the height of the authentication target person 30. In addition, by making the optical axes of the plurality of imaging devices 11 parallel, it is possible to prevent the face image from being greatly displaced in the vertical direction. Further, since the captured images are from the same direction, the processing for calculating these correlation values can be reduced, and the calculation accuracy of the correlation values is increased.

(Another example of face authentication processing in Embodiment 1)
FIG. 9 is a flowchart illustrating an example of another face authentication operation procedure according to the first embodiment in time series. In the description of FIG. 9, the same step process as the face authentication process shown in FIG. 7 is denoted by the same step number or the corresponding step number, the description is simplified or omitted, and different contents will be described.

  In FIG. 9, in another face authentication process, the face authentication device 20 performs plane determination first after a face image is detected. Thereafter, face comparison of the target person is performed. When the face recognition unit 21 detects a face image in step S3, the flat panel detection unit 70 performs plane determination of the face image (S6). The determination unit 80 determines whether or not the face image is a plane (S7). If it is a plane, the determination unit 80 determines whether or not the number of times determined to be a plane is a predetermined number or more (S7A). If the number is less than the predetermined number, the face authentication device 20 returns to the process of step S1. . On the other hand, when the number of times determined to be a plane is equal to or greater than the predetermined number, the validity determination unit 25 determines invalidity and outputs a message indicating rejection (that is, a message indicating NG) as the authentication result of face authentication. (S9). Thereafter, the face authentication apparatus 20 ends the process shown in FIG.

  On the other hand, if the face image is not a flat surface in step S7, the face collation unit 24 collates the detected face and authenticates the person (S4). The face matching unit 24 determines whether or not the feature information of the target person's face matches the matching data of the registrant's face (S5). If it does not match the collation data, the validity determination unit 25 determines invalidity in step S9, and outputs denial as the authentication result of face authentication. If it matches with the collation data in step S5, the validity determination unit 25 determines the validity and outputs a message indicating approval (ie, a message indicating OK) as the authentication result of face authentication (S8). . Thereafter, the face authentication apparatus 20 ends the process shown in FIG.

  Thereby, face collation can be performed only when it is determined that the face image is not flat. Therefore, it is possible to reduce the number of face matching operations that require a large processing load.

  Further, when it is determined that the face image is not a plane, the face matching unit 24 checks whether or not the face feature information of the target person matches the registrant's face matching data. The validity determination unit 25 determines the validity of the authentication result of the person's face information based on the collation result. Thereby, the face authentication apparatus 20 or the face authentication system 100 can omit the face authentication process when the face image is determined to be a plane, and can reduce the processing load.

  In addition, the validity determination unit 25 denies the validity of the collation result as to whether or not the person is the person when it is determined a predetermined number of times that the face image of the face area is a plane. Thereby, for example, when the face authentication device 20 or the face authentication system 100 determines that the face of the person is a flat surface not only once but multiple times, there is a possibility of erroneous determination that the face is a single flat surface. It is possible to omit the execution of face collation processing with a large processing load while rejecting, and output denial as the authentication result of the person's face authentication. Therefore, in the face authentication device 20 or the face authentication system 100, the processing load can be reduced.

(Example of Arrangement of Other Imaging Devices in Embodiment 1)
FIG. 10 is a diagram illustrating an arrangement example of the other three imaging devices 11i, 11j, and 11k according to the first embodiment. The three imaging devices 11i, 11j, and 11k are arranged in a direction perpendicular to the plane of the passage through which the person 30 to be authenticated passes (that is, a direction perpendicular to the horizontal plane). The three imaging devices 11i, 11j, and 11k image the face existence range SA of the authentication target person 30 so that the angles of view overlap in the passage width wd. In this case, the face of a person such as a short child is imaged by the imaging device 11k installed at a low position and the imaging device 11i installed at a high position. In addition, the face of a person having a general height is imaged by the three imaging devices 11i, 11j, and 11k. Further, the face of a tall person such as an adult is imaged by the imaging device 11i installed at a high position and the imaging device 11j installed at an intermediate position. The optical axes of the three imaging devices 11i, 11j, and 11k are arbitrarily set according to the face existence range SA and are not parallel. Therefore, when a person with a different height tries to pass through the passage, a face image is captured by at least two imaging devices.

  As described above, the plurality of imaging devices are arranged at different heights in the vertical direction, and the optical axes of the plurality of imaging devices are not parallel to each other. Accordingly, the face image of the target person can be reliably captured by two or more imaging devices regardless of the height of the subject. In addition, by making the optical axes of a plurality of cameras non-parallel, it is possible to easily capture face images of target persons having different heights. Therefore, in order to capture two face images, the number of installed image pickup devices can be reduced.

(Embodiment 2)
In the first embodiment, when the face image is a plane, it is determined that the result of the face authentication is denial by assuming that the display unit is impersonated. In the second embodiment, in addition to the configuration of the first embodiment, a case where it is determined that the display unit is impersonated even when a frame image is detected in the captured image is shown.

  In the face authentication system according to the second embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof is simplified or omitted.

  FIG. 11 is a diagram illustrating a functional configuration example realized by cooperation of a processor and a memory built in the face authentication apparatus 20A according to the second embodiment. 20 A of face authentication apparatuses have the frame detection part 22 and the arrangement | positioning determination part 23 other than the face recognition part 21, the face collation part 24, the effectiveness determination part 25, and the flat panel detection part 70 which were shown in Embodiment 1. FIG. The face recognition unit 21, the frame detection unit 22, the arrangement determination unit 23, the face collation unit 24, the validity determination unit 25, and the flat panel detection unit 70 are, for example, a processor and a memory incorporated in the face authentication device 20A that is a PC. Thus, the processor executes the predetermined program stored in the memory.

  The frame detection unit 22 uses the captured image and the background image to detect a linear frame in the input image obtained by removing the background image from the captured image. The frame detection unit 22 acquires frame position information (in other words, frame arrangement information) when there is a frame of the display unit in the captured image.

  The arrangement determination unit 23 determines the arrangement of whether or not the face is located within the frame of the display unit based on the face arrangement information and the frame arrangement information acquired by the face recognition unit 21. As a result of the placement determination, the placement determination unit 23 detects that a display unit (see above) is present in the captured image and a face image is displayed within the frame of the display unit.

  Based on the results of face matching by the face matching unit 24, plane determination by the flat panel detecting unit 70, and frame detection by the placement determining unit 23, the validity determining unit 25 determines whether the face of the target person shown in the captured image is valid. It is determined whether it is approval or disapproval as the validity of the result of face authentication. The validity determination unit 25 outputs the authentication result to the control target device 40.

  FIG. 12 is an explanatory diagram illustrating an example of an outline of the operation of the frame detection process performed by the frame detection unit 22. Here, a situation is assumed in which spoofing is performed by reproducing an image (still image increase, moving image) in which a face of a regular registrant is reflected on a portable display unit. The imaging devices 11A and 11B each image the display unit.

  A captured image captured by one of the imaging devices 11 </ b> A and 11 </ b> B is an input image 51 input to the frame detection unit 22. The input image 51 includes a housing 53 of the display unit and a person 54 displayed on the screen of the display unit. The frame detection unit 22 performs a known contour extraction (for example, edge detection) process on the input image 51 to generate an edge image 52. The edge image 52 includes a contour 55 of the housing of the display unit and a contour 56 of a person displayed on the screen of the display unit. The frame detection unit 22 performs straight line detection processing on the edge image 52 and extracts a straight line portion (line segment) 57 in the image. For the straight line detection processing, for example, straight line detection using Hough transform or the like is used. The frame detection unit 22 determines whether there are two parallel straight lines and a straight line perpendicular thereto in the straight line portion 57 in the extracted image, extracts a frame 58 surrounded by four sides, and extracts frame coordinates. To get. Note that the frame detection unit 22 is surrounded by a line segment based on a predetermined condition, even if it is a set of square line segments whose four sides are not closed, or a group of line segments orthogonal to three sides or two sides. It may be determined as a frame and the frame 58 may be extracted.

  Further, the frame detection unit 22 may perform straight line detection and frame extraction after removing the background from the captured image in the frame detection process. The accuracy of frame detection can be improved by removing the background portion from the captured image.

  FIG. 13 is an explanatory diagram illustrating an operation outline example of the arrangement determination processing by the arrangement determination unit 23. The arrangement determination unit 23 acquires face arrangement information of the target person obtained as one of the face detection results by the face recognition unit 21. The face arrangement information includes a face frame 68 indicating the contour position of the face in the captured image. Further, the arrangement determination unit 23 acquires frame arrangement information of the display unit obtained as one of the frame detection results by the frame detection unit 22. The frame arrangement information includes a frame 58 indicating the contour position of the display unit detected in the captured image.

  As the placement determination process, the placement determination unit 23 compares the positions of both the face frame 68 and the frame 58, and determines whether or not the face frame 68 exists in the area inside the frame 58, that is, the target in the display unit frame. It is determined whether or not a person's face is located. As shown in FIG. 13, when there is a face frame 68 inside the frame 58, the arrangement determination unit 23 outputs an arrangement determination result that there is a face image in the frame.

  The validity determination unit 25 considers that the face image detected in the captured image is the face image displayed on the display unit when the face image exists in the frame based on the result of the placement determination by the placement determination unit 23, It is determined that the verification result is invalid. The validity determination unit 25 outputs a denial to the control target device 40 as a result of the face authentication.

  Next, the operation of the face authentication system 100 according to Embodiment 2 will be described with reference to FIG.

  FIG. 14 is a flowchart illustrating an example of a face authentication operation procedure according to the second embodiment in time series. About the same step process as the face authentication process shown in FIG. 7, the description is simplified or abbreviate | omitted by attaching | subjecting the same step number.

  If it is determined in step S7 that the plane is not a plane, the frame detection unit 22 performs frame detection in the captured image (S7B). The arrangement determination unit 23 determines the arrangement of the face position by face detection and the frame position by frame detection (S7C). The arrangement determining unit 23 detects a frame in the captured image and determines whether or not a face exists inside the frame, that is, whether or not a frame surrounding the outside of the target person's face exists (S7D). The arrangement determination unit 23 passes the determination result of the frame existence to the validity determination unit 25. If there is a frame surrounding the outside of the target person's face as a result of the frame presence determination, the validity determination unit 25 determines that the face authentication is invalid and outputs a rejection to the control target device 40 as the face authentication result. If there is no frame surrounding the outside of the target person's face as a result of the frame presence determination, the validity determination unit 25 determines that the face authentication is valid, and outputs approval to the control target device 40 as the face authentication result.

  Here, if it is determined in step S7 that the plane is not a plane, the frame detection processing is performed in steps S7B to S7D. If they match, frame detection processing similar to steps S7B to S7D may be performed.

  In the face authentication process, the face detection process and the frame detection process may be executed in parallel. Moreover, after performing one process in series, you may perform the other process, and the implementation timing and order of a process are not limited.

  As described above, in the face authentication system 100 according to the second embodiment, the face detection of the target person is performed by using the result of the frame detection by the arrangement determination unit 23 in the face authentication determination in addition to the face matching and plane determination processing. Can be performed with high accuracy.

  Even when it is determined that the face image of the target person is not a plane, face authentication can be denied if the captured image including the face image includes a frame image. For example, when the target person is captured by a plurality of imaging devices while inverting the orientation of the tablet terminal in small increments or while continuously switching the face image displayed on the tablet terminal to 3D display or 2D display, May be determined to be non-planar. Even in such a case, when there is a frame image in the captured image, it is determined that the face image is reflected on the tablet terminal, and the face image of the target person can be denied.

  On the other hand, even if there is an action such as hiding the frame of the display unit on which the face image is displayed, such an action can be dealt with by plane determination. Further, by combining the plane determination, it is possible to avoid a false detection that a face authentication result is NG only by detecting a frame even for a person who is not a face authentication target.

  While various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is obvious for those skilled in the art that various changes, modifications, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure. In addition, the constituent elements in the various embodiments described above may be arbitrarily combined without departing from the spirit of the invention.

  The present disclosure accurately detects impersonation using an image that can be performed at the time of authentication of a target person, and effectively prevents fraudulent acts of a malicious third party due to impersonation, a face authentication method, and face authentication Useful as a system.

11A, 11B Imaging device 12 Display device 13 ID reader 20, 20A Face authentication device 21 Face recognition unit 22 Frame detection unit 23 Arrangement determination unit 24 Face collation unit 25 Effective determination unit 40 Control target device 70 Flat panel detection unit 71, 72 Face detection unit 73 Face correspondence determination unit 74, 75 Feature point detection unit 76 Feature point matching unit 77 Projection conversion estimation unit 78 Projection conversion unit 79 Correlation calculation unit 80 Determination unit 100 Face authentication system

Claims (10)

An image input unit that inputs a first captured image and a second captured image in which the same person is captured by each of a plurality of cameras;
A face detector for detecting at least one face area of the first captured image and the second captured image;
A plane determination unit that determines whether the face image of the face area is a plane based on the face information corresponding to the face area and the first captured image and the second captured image;
A validity determination unit that determines the validity of an authentication result of the person's face information based on a determination result of whether or not the face image of the face region is a plane;
Face recognition device. A collation unit that collates face information corresponding to the detected face area and registered face collation data;
The effectiveness determination unit includes:
Based on the collation result of the face information and the determination result of whether or not the face image of the face area is a plane, the validity of the authentication result of the person's face information is determined.
The face authentication apparatus according to claim 1. A collation unit that collates face information corresponding to the detected face area and registered face collation data;
The collation unit
When it is determined that the face image of the face area is not a plane, the face information is collated,
The effectiveness determination unit further includes:
Determining the validity of the authentication result of the person's face information based on the face information verification result;
The face authentication apparatus according to claim 1. An ID acquisition unit for acquiring ID information of the person,
The effectiveness determination unit includes:
Determining the validity of the authentication result based on the determination result whether the face image of the face area is a plane and the ID information;
The face authentication apparatus according to claim 1. The plane determination unit
Depending on the correlation between one of the first captured image and the second captured image and a projective transformation image obtained by projective transformation of the imaging direction of the other captured image into the imaging direction of the one captured image. Determining whether the face image of the face area in the one captured image is a plane,
The face authentication apparatus according to claim 1 or 2. The effectiveness determination unit includes:
When it is determined a predetermined number of times that the face image of the face area is a plane, the validity of the collation result as to whether the person is the person is denied,
The face authentication apparatus according to claim 1. A face authentication method in a face authentication device,
Inputting a first captured image and a second captured image in which the same person is captured by each of a plurality of cameras;
Detecting at least one face area of the first captured image and the second captured image;
Determining whether the face image of the face area is a plane based on the face information corresponding to the face area and the first captured image and the second captured image;
Determining the validity of the authentication result of the person's face information based on the determination result of whether or not the face image of the face region is a plane,
Face authentication method. A face authentication system in which a plurality of cameras and a face authentication device are communicably connected to each other,
The plurality of cameras are:
Outputting a first captured image and a second captured image each capturing the same person to the face authentication device;
The face authentication device
Inputting the first captured image and the second captured image;
Detecting at least one face area of the first captured image and the second captured image;
Based on the face information corresponding to the face area and the first captured image and the second captured image, it is determined whether or not the face image of the face area is a plane,
Determining the validity of the authentication result of the person's face information based on the determination result of whether or not the face image of the face region is a plane;
Face recognition system. The plurality of cameras are:
The optical axes of the plurality of cameras are parallel to each other and are arranged at different heights in the vertical direction.
The face authentication system according to claim 8. The plurality of cameras are:
The optical axes of the plurality of cameras are non-parallel to each other while being arranged at different heights in the vertical direction.
The face authentication system according to claim 8.

Images