JP2017027492A - Face collation apparatus, face collation system with same, and face collation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to pick up a collation image in an illumination condition substantially identical to an illumination condition at a time of picking up a registration image without registering the illumination condition at the time of picking up the registration image.SOLUTION: A face collation apparatus 3 comprises: a registration image acquisition unit 11 acquiring a registration image; an illumination condition estimation unit 13 estimating an illumination condition at a time of picking up the registration image on the basis of a pixel value in a face region in the registration image; an illumination apparatus control unit 14 setting an illumination condition for an illumination apparatus illuminating a collation target person on the basis of the illumination condition at the time of picking up the registration image estimated by the illumination condition estimation unit 13; a collation image acquisition unit 15 acquiring a collation image picked up in the illumination condition set by the illumination apparatus control unit 14; and an image collation unit 16 collating the collation image with the registration image.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to a face matching device that matches a matching image obtained by capturing a face of a person to be verified with a registered image captured in advance, a face matching system including the face matching device, and a face matching method.

  2. Description of the Related Art Conventionally, for the purpose of face authentication, an apparatus that collates an image for collation (collation image) obtained by capturing a face of a person to be collated on the spot and a registered image captured in advance is known. Such an apparatus can be used, for example, for identity verification at a security gate or the like. However, if the illumination state at the time of capturing the registered image is different from the illumination state at the time of capturing the verification image, the appearance of the face in the image changes greatly, and there is a possibility that the verification may fail.

  Therefore, the illumination parameter value (the ambient illuminance, the number of light sources, the position, the direction of the light source, the type of the light source, etc.) indicating the illumination state at the time of capturing the registered image is registered in advance, and the registered illumination parameter value is the target. As a technique, a technique for generating a collation image from an input image by adjusting a parameter value of an input image (an image obtained by imaging a person to be collated) is disclosed (for example, Patent Document 1). The parameter value of the input image is acquired using a sensor such as an illuminance sensor when the input image is captured. In addition, the lighting conditions (strong / weak, ON / OFF, direction, position, etc.) of the lighting device at the time of capturing the registered image are registered in advance, and the lighting device is configured to reproduce the registered lighting conditions at the time of capturing the verification image. A technique for controlling illumination conditions is disclosed (for example, Patent Document 2).

JP 2000-306095 A JP 2003-308524 A

  However, since the prior art as described above needs to register in advance the lighting state and lighting conditions at the time of taking a registered image, for example, passport verification (identity confirmation) in immigration at an airport, etc. In addition, it is not suitable when an unknown registered image in which the illumination state and the illumination condition at the time of imaging are not registered in advance is presented on the spot.

  Moreover, in the prior art of patent document 1, in order to detect the illumination state at the time of imaging of an input image, since sensors, such as an illuminance sensor, are needed, there exists a problem that the structure of an apparatus becomes complicated. Further, there is a case where the collation image is generated by adjusting the illumination parameter value of the input image with the registered illumination parameter value as a target, but a specific method for that is not disclosed.

  The present disclosure has been devised in view of the above-described problems of the prior art, and the collation image is abbreviated as the illumination condition at the time of capturing the registered image without previously registering the illumination condition at the time of capturing the registered image. It is a main object of the present invention to provide a face matching device, a face matching system, and a face matching method that can capture images under the same illumination conditions.

  The face matching device according to the present disclosure is a face matching device that matches a matching image obtained by capturing a face of a person to be verified with a registered image captured in advance, a registered image acquiring unit that acquires the registered image, and the registration An illumination condition estimation unit that estimates an illumination condition at the time of capturing the registered image based on a pixel value of a face area in the image, and an illumination device that illuminates the person to be verified based on the estimated illumination condition A lighting device controller that controls conditions; a matching image acquisition unit that acquires the matching image captured under the controlled lighting condition; and an image matching unit that matches the matching image with the registered image. It is characterized by that.

  According to the present disclosure, it is possible to capture the collation image under substantially the same illumination condition as the illumination condition at the time of capturing the registered image without previously registering the illumination condition at the time of capturing the registered image.

Overall configuration diagram of a face matching system according to the first embodiment of the present disclosure It is a schematic diagram of the collation image imaging device in FIG. 1, (A) is a side view, (B) is a front view. 1 is a block diagram showing a hardware configuration for realizing the face matching system of FIG. FIG. 7 is a flowchart showing a flow of face matching processing by the face matching device according to the first embodiment of the present disclosure; Whole block diagram of face collation system concerning a 2nd embodiment of this indication The block diagram which shows schematic structure of the feedback part of the face collation apparatus in FIG. FIG. 9 is a flowchart showing the flow of face matching processing by the face matching device according to the second embodiment of the present disclosure;

  A first invention made to solve the above-described problem is a face collation apparatus that collates a collation image obtained by capturing a face of a person to be collated with a registered image captured in advance, and acquires the registration image. An image acquisition unit, an illumination condition estimation unit that estimates an illumination condition at the time of capturing the registered image based on a pixel value of a face area in the registered image, and an illumination condition at the time of capturing the estimated registered image Based on an illumination device control unit that sets an illumination condition of an illumination device that illuminates the person to be collated, a collation image acquisition unit that obtains the collation image captured under the set illumination condition, and the collation image And an image collating unit that collates the image with the registered image.

  According to the face collation device according to the first aspect of the present invention, the illumination condition at the time of capturing the registered image is estimated based on the pixel value of the face area in the registered image, and the estimated illumination condition at the time of capturing the registered image is determined. In order to reproduce, the illumination condition of the illumination device that illuminates the person to be collated is set, so that the illumination condition at the time of capturing the registered image is not registered in advance. The lighting conditions can be reproduced. As a result, the collation image can be imaged under substantially the same illumination conditions as when the registered image was captured, so that the accuracy of collation between the collation image and the registered image can be increased.

  According to a second aspect, in the first aspect, the illumination condition includes an illumination direction and light intensity.

  According to the face collation device according to the second aspect of the present invention, by setting the illumination direction and light intensity of the illumination device in order to reproduce the illumination direction and light intensity at the time of capturing the estimated registered image, It is possible to capture a collation image in the same illumination state at the time of imaging.

  Further, in a third invention, in the first invention or the second invention, further comprising a three-dimensional coordinate setting unit for setting a three-dimensional coordinate of a face area in the registered image, wherein the illumination condition estimating unit includes: An illumination condition at the time of capturing the registered image is estimated based on a three-dimensional coordinate and a pixel value of a face area in the registered image.

  According to the face collation device according to the third aspect of the present invention, the illumination condition at the time of capturing the registered image is estimated based on the three-dimensional coordinates of the face area in the registered image and the pixel value. It is possible to more accurately estimate the illumination conditions at the time of imaging. Thereby, it is possible to further improve the accuracy of matching between the matching image and the registered image.

  According to a fourth invention, in any one of the first to third inventions, the illumination device is installed at a position where the person to be collated can be illuminated from the front and above, and is turned on and off. A plurality of light sources capable of individually controlling at least one of the light intensities, and the lighting device control unit individually controls at least one of on / off of each of the light sources of the lighting devices and the light intensity. The illumination condition of the illumination device is adjusted by the above.

  According to the face collation device according to the fourth aspect of the present invention, the illumination device controller is configured to individually control at least one of on / off and light intensity of each light source of the illumination device. Can be adjusted easily and precisely. Thereby, it is possible to better reproduce the illumination conditions at the time of capturing the registered image.

  According to a fifth aspect of the invention, in the third or fourth aspect of the invention, the illumination conditions at the time of capturing the collation image are determined based on the three-dimensional coordinates of the face area in the registered image and the face area in the collation image. A collation image illumination condition estimation unit that estimates based on a pixel value; and an illumination condition determination unit that determines whether or not the illumination condition at the time of capturing the collation image matches the illumination condition at the time of capturing the registered image. It is further provided with a feature.

  According to the face collation device according to the fifth aspect of the present invention, when the collation between the collation image and the registered image in the image collation unit does not match, the illumination condition at the time of capturing the collation image is the same as that at the time of capturing the registration image. It is possible to determine whether or not the illumination condition is met. Thereby, it is possible to confirm whether or not the illumination condition at the time of capturing the collation image is appropriate, so that it is possible to further improve the collation accuracy between the collation image and the registered image.

  According to a sixth aspect, in any one of the third to fifth aspects, a collation image three-dimensional coordinate estimation unit that estimates a three-dimensional coordinate of a face area in the collation image; A collation image face direction detection unit that detects a face direction in the collation image based on the three-dimensional coordinates of the face area, and a face in the registration image based on the three-dimensional coordinates of the face area in the registration image. A registered image face direction detecting unit that detects a direction, and a face direction determining unit that determines whether or not a face direction in the verification image matches a face direction in the registered image. And

  According to the face collation device according to the sixth aspect of the present invention, when the illumination condition determination unit determines that the illumination condition at the time of capturing the verification image does not match the illumination condition at the time of capturing the registered image, Since it is configured to determine whether or not the face orientation matches the face orientation in the registered image, it is possible to confirm whether or not the face orientation in the verification image is appropriate. This makes it possible to confirm whether or not the direction of the face of the person to be collated at the time of capturing the collation image is appropriate, so that the collation accuracy between the collation image and the registered image can be further increased.

  In the seventh invention, in the sixth invention, when the face orientation determination unit determines that the face orientation in the collation image matches the face orientation in the registered image, the registered image A registered image illumination condition re-estimation unit that re-estimates the illumination condition at the time of imaging based on the three-dimensional coordinates of the face region in the collation image and the pixel value of the face region in the registered image. Features.

  According to the face collation device of the seventh aspect of the present invention, when the face orientation determination unit determines that the face orientation in the collation image matches the face orientation in the registration image, the illumination at the time of capturing the registration image Since the condition is configured to be re-estimated based on the three-dimensional coordinates of the face area in the collation image and the pixel value of the face area in the registered image, the illumination condition at the time of capturing the registered image is determined by the illumination condition estimation unit. When the three-dimensional coordinates of the face area in the registered image used for estimation are not appropriate, the illumination conditions at the time of capturing the registered image are estimated again using the three-dimensional coordinates of the face area in the matching image. be able to. Therefore, it is possible to further improve the accuracy of matching between the matching image and the registered image.

  In the eighth invention, when the face orientation determination unit determines in the sixth invention that the face orientation in the collation image does not match the face orientation in the registered image, In contrast, the information processing apparatus further includes a notification unit that notifies the user to change the face direction of the person to be collated to the same face direction as the face in the registered image and to retake the collation image. .

  According to the face collation device according to the eighth aspect of the present invention, when the face orientation determination unit determines that the face orientation in the collation image does not match the face orientation in the registration image, Since it is configured to notify the user to change the orientation of the subject's face to the same orientation as the face in the registered image and re-take the collation image, When the orientation is different from the orientation of the face in the registered image, it is possible to retake the collation image by changing the orientation of the face of the person to be collated to the same face orientation as the registered image. Therefore, it is possible to further improve the accuracy of matching between the matching image and the registered image.

  According to a ninth aspect, the face matching device according to any one of the first to eighth aspects, a registered image input device that inputs a registered image to the face matching device, and an illumination condition can be adjusted. A face collation system including an illumination device and an imaging device that images a person to be collated illuminated by the illumination device.

  According to the face matching system according to the ninth aspect of the present invention, the illumination condition at the time of capturing the registered image is estimated based on the pixel value of the face area in the registered image, and the estimated illumination condition at the time of capturing the registered image is determined. In order to reproduce, the illumination condition of the illumination device that illuminates the person to be collated is set, so that the illumination condition at the time of capturing the registered image is not registered in advance. The lighting conditions can be reproduced. As a result, the collation image can be imaged under substantially the same illumination conditions as when the registered image was captured, so that the accuracy of collation between the collation image and the registered image can be increased.

  According to a tenth aspect, in the ninth aspect, the illumination condition includes an illumination direction and a light intensity.

  According to the face collation system according to the tenth aspect of the present invention, by setting the illumination direction and light intensity of the illumination device to reproduce the estimated illumination direction and light intensity at the time of capturing the registered image, It is possible to capture a collation image in the same illumination state at the time of imaging.

  In the eleventh aspect of the invention, in the ninth aspect or the tenth aspect of the invention, the face matching device further includes a three-dimensional coordinate setting unit that sets a three-dimensional coordinate of a face area in the registered image, The illumination condition estimating unit estimates an illumination condition at the time of capturing the registered image based on a three-dimensional coordinate and a pixel value of a face area in the registered image.

  According to the face collation system of the eleventh aspect of the present invention, the illumination condition at the time of capturing the registered image is estimated based on the three-dimensional coordinates and pixel values of the face area in the registered image. It is possible to more accurately estimate the illumination conditions at the time of imaging. Thereby, it is possible to further improve the accuracy of matching between the matching image and the registered image.

  According to a twelfth aspect, in any one of the ninth to eleventh aspects, the lighting device is installed at a position where the person to be collated can be illuminated from the front and above, and is turned on and off. It has a plurality of light sources that individually adjust and control at least one of the light intensities, and the lighting device control unit individually controls at least one of on / off of each of the light sources of the lighting devices and the light intensity. The illumination condition of the illumination device is adjusted by the above.

  According to the face collation system according to the twelfth aspect of the present invention, the illumination device controller is configured to individually control at least one of on / off and light intensity of each light source of the illumination device. Can be adjusted easily and precisely. Thereby, it is possible to better reproduce the illumination conditions at the time of capturing the registered image.

  According to a thirteenth aspect, in the eleventh aspect or the twelfth aspect, the face collation device determines the illumination condition at the time of capturing the collation image as a three-dimensional coordinate of the face area in the registered image and the collation. A collation image illumination condition estimation unit that estimates based on the pixel value of the face area in the image, and determines whether the illumination condition at the time of capturing the collation image matches the illumination condition at the time of capturing the registered image An illumination condition determining unit is further provided.

  According to the face collation system of the thirteenth aspect of the present invention, if the collation between the collation image and the registered image in the image collation unit does not match, the illumination condition at the time of capturing the collation image is the same as that at the time of capturing the registration image. It is possible to determine whether or not the illumination condition is met. Thereby, it is possible to confirm whether or not the illumination condition at the time of capturing the collation image is appropriate, so that it is possible to further improve the collation accuracy between the collation image and the registered image.

  In the fourteenth invention, in any one of the eleventh to thirteenth inventions, the face matching device estimates a three-dimensional coordinate of a face area in the matching image. A collation image face direction detecting unit for detecting a face direction in the collation image based on the three-dimensional coordinates of the face region in the collation image, and the three-dimensional coordinates of the face region in the registration image. A registered image face direction detecting unit that detects a face direction in the registered image, and a face direction determining unit that determines whether or not the face direction in the verification image matches the face direction in the registered image. It is further provided with a feature.

  According to the face collation system according to the fourteenth aspect of the present invention, when the illumination condition determination unit determines that the illumination condition at the time of capturing the verification image does not match the illumination condition at the time of capturing the registered image, Since it is configured to determine whether or not the face orientation matches the face orientation in the registered image, it is possible to confirm whether or not the face orientation in the verification image is appropriate. This makes it possible to confirm whether or not the direction of the face of the person to be collated at the time of capturing the collation image is appropriate, so that the collation accuracy between the collation image and the registered image can be further increased.

  According to a fifteenth aspect, in the fourteenth aspect, the face matching device determines that the face orientation determination unit determines that the face orientation in the match image matches the face orientation in the registered image. A registration image illumination condition re-estimation unit that re-estimates the illumination condition at the time of capturing the registered image based on the three-dimensional coordinates of the face area in the collation image and the pixel value of the face area in the registration image Is further provided.

  According to the face collation system according to the fifteenth aspect of the present invention, when the face orientation determination unit determines that the face orientation in the collation image matches the face orientation in the registration image, illumination at the time of capturing the registration image Since the condition is configured to be re-estimated based on the three-dimensional coordinates of the face area in the collation image and the pixel value of the face area in the registered image, the illumination condition at the time of capturing the registered image is determined by the illumination condition estimation unit. When the three-dimensional coordinates of the face area in the registered image used for estimation are not appropriate, the illumination conditions at the time of capturing the registered image are estimated again using the three-dimensional coordinates of the face area in the matching image. be able to. Therefore, it is possible to further improve the accuracy of matching between the matching image and the registered image.

  In a sixteenth aspect based on the fourteenth aspect, the face collating apparatus determines that the face orientation in the collated image does not match the face orientation in the registered image in the face orientation determining unit. A notification unit that notifies the user to change the face direction of the person to be collated to the same face direction as the face in the registered image and to take the collation image again. It is characterized by that.

  According to the face collation system of the sixteenth aspect of the present invention, when the face orientation determination unit determines that the face orientation in the collation image does not match the face orientation in the registered image, the face collation system Since it is configured to notify the user to change the orientation of the subject's face to the same orientation as the face in the registered image and re-take the collation image, When the orientation is different from the orientation of the face in the registered image, it is possible to retake the collation image by changing the orientation of the face of the person to be collated to the same face orientation as the registered image. Therefore, it is possible to further improve the accuracy of matching between the matching image and the registered image.

  A seventeenth aspect of the present invention is a face collation method for collating a collation image obtained by capturing a face of a person to be collated with a registered image captured in advance, the step of acquiring the registration image, A step of estimating an illumination condition at the time of capturing the registered image based on a pixel value of a face area; and an illumination device for illuminating the person to be verified based on the estimated illumination condition at the time of capturing the registered image. The method includes a step of setting an illumination condition, a step of obtaining the collation image captured under the set illumination condition, and a step of collating the collation image with the registered image.

  According to the face collation method according to the seventeenth aspect of the present invention, the illumination condition at the time of capturing the registered image is estimated based on the pixel value of the face area in the registered image, and the estimated illumination condition at the time of capturing the registered image is determined. In order to reproduce, the illumination condition of the illumination device that illuminates the person to be collated is set, so that the illumination condition at the time of capturing the registered image is not registered in advance. The lighting conditions can be reproduced. As a result, the collation image can be imaged under substantially the same illumination conditions as when the registered image was captured, so that the accuracy of collation between the collation image and the registered image can be increased.

  According to an eighteenth aspect, in the seventeenth aspect, the illumination condition includes an illumination direction and light intensity.

  According to the face collation method according to the eighteenth aspect of the present invention, by setting the illumination direction and light intensity of the illumination device in order to reproduce the illumination direction and light intensity at the time of capturing the estimated registered image, It is possible to capture a collation image in the same illumination state at the time of imaging.

  According to a nineteenth aspect, in the seventeenth aspect or the eighteenth aspect, the method further includes a three-dimensional coordinate setting step of setting a three-dimensional coordinate of a face area in the registered image, wherein the illumination condition estimating step includes: The illumination condition at the time of capturing the registered image is estimated based on the three-dimensional coordinates and pixel values of the face region in the registered image.

  According to the face collation method of the nineteenth aspect of the present invention, the illumination condition at the time of capturing the registered image is estimated based on the three-dimensional coordinates and pixel values of the face area in the registered image. It is possible to more accurately estimate the illumination conditions at the time of imaging. Thereby, it is possible to further improve the accuracy of matching between the matching image and the registered image.

  Further, in a twentieth aspect according to any one of the seventeenth to nineteenth aspects, the lighting device is installed at a position where the person to be collated can be illuminated from the front and above, and is turned on and off. A plurality of light sources that individually adjust and control at least one of the light intensities, and the lighting device control step individually controls at least one of on / off of each of the light sources of the lighting device and the light intensity; The illumination condition of the illumination device is adjusted by the above.

  According to the face collation method of the twentieth aspect of the invention, since at least one of on / off and light intensity of each light source of the illumination device is individually controlled in the illumination device control step, the illumination condition of the illumination device Can be adjusted easily and precisely. Thereby, it is possible to better reproduce the illumination conditions at the time of capturing the registered image.

  In the twenty-first aspect, in the nineteenth aspect or the twentieth aspect, the illumination condition at the time of capturing the collation image is determined based on the three-dimensional coordinates of the face area in the registered image and the face area in the collation image. A collation image illumination condition estimation step that is estimated based on a pixel value; and an illumination condition determination step that determines whether or not the illumination condition at the time of capturing the collation image matches the illumination condition at the time of capturing the registered image. Furthermore, it is characterized by having.

  According to the face collation method according to the twenty-first aspect of the present invention, if the collation between the collation image and the registered image does not match in the image collation step, the illumination condition at the time of capturing the collation image is the illumination at the time of capturing the registration image. It is possible to determine whether or not the condition is met. Thereby, it is possible to confirm whether or not the illumination condition at the time of capturing the collation image is appropriate, so that it is possible to further improve the collation accuracy between the collation image and the registered image.

  According to a twenty-second aspect, in any one of the nineteenth to twenty-first aspects, a verification image three-dimensional coordinate estimation step for estimating a three-dimensional coordinate of a face area in the verification image; A collation image face orientation detection step for detecting a face orientation in the collation image based on the three-dimensional coordinates of the face area; and a face in the registration image based on the three-dimensional coordinates of the face area in the registration image. A registered image face orientation detecting step for detecting an orientation; and a face orientation determining step for determining whether or not a face orientation in the collation image matches a face orientation in the registered image. To do.

  According to the face collation method of the twenty-second aspect of the present invention, when it is determined in the illumination condition determination step that the illumination condition at the time of capturing the collation image does not match the illumination condition at the time of capturing the registered image, Since it is configured to determine whether or not the face orientation matches the face orientation in the registered image, it is possible to confirm whether or not the face orientation in the verification image is appropriate. This makes it possible to confirm whether or not the direction of the face of the person to be collated at the time of capturing the collation image is appropriate, so that the collation accuracy between the collation image and the registered image can be further increased.

  In the twenty-third invention, in the twenty-second invention, when it is determined in the face orientation determination step that the face orientation in the collation image matches the face orientation in the registered image, the registered image A registered image illumination condition re-estimation step of re-estimating the illumination condition at the time of imaging based on the three-dimensional coordinates of the face region in the collation image and the pixel value of the face region in the registered image. And

  According to the face collation method of the twenty-third aspect of the present invention, when it is determined in the face orientation determination step that the face orientation in the collation image matches the face orientation in the registration image, the illumination at the time of capturing the registration image Since the condition is re-estimated based on the three-dimensional coordinates of the face area in the collation image and the pixel value of the face area in the registered image, the illumination condition at the time of capturing the registered image in the illumination condition estimation step is set. When the three-dimensional coordinates of the face area in the registered image used for estimation are not appropriate, the illumination conditions at the time of capturing the registered image are estimated again using the three-dimensional coordinates of the face area in the matching image. be able to. Therefore, it is possible to further improve the accuracy of matching between the matching image and the registered image.

  In the twenty-fourth aspect, in the twenty-second aspect, when it is determined in the face direction determination step that the face direction in the collation image does not match the face direction in the registered image, On the other hand, the method further includes a notification step of notifying the collation image again by changing the face direction of the person to be collated to the same face direction as the face direction in the registered image.

  According to the face collation method of the twenty-fourth aspect of the present invention, when it is determined in the face orientation determination step that the face orientation in the collation image does not match the face orientation in the registered image, Since it is configured to notify the user to change the orientation of the subject's face to the same orientation as the face in the registered image and re-take the collation image, When the orientation is different from the orientation of the face in the registered image, it is possible to retake the collation image by changing the orientation of the face of the person to be collated to the same face orientation as the registered image. Therefore, it is possible to further improve the accuracy of matching between the matching image and the registered image.

  Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

(First embodiment)
1 is an overall configuration diagram of a face matching system according to a first embodiment of the present disclosure, FIG. 2 is a schematic diagram of a matching image imaging device in FIG. 1, (A) is a side view, and (B) ) Is a front view.

  The face collation system according to the present disclosure uses a face image (hereinafter referred to as “collation image”) of a person to be collated captured by an imaging device as a face image of a person to be collated (hereinafter referred to as “registered image”). For example, it is applicable to passport verification (identity verification) in immigration at an airport or the like. In recent years, an IC passport equipped with an IC (integrated circuit) is used as a passport, and a face image of the holder of the IC passport is recorded on an IC chip. According to the face collation system according to the present disclosure, it is possible to perform identity verification by collating the face image read from the IC passport (passport) with the face image captured at the examination place in the airport. Hereinafter, a case will be described in which the face matching system according to the present disclosure is applied to passport matching (identity verification) in immigration at an airport or the like.

  As shown in FIG. 1, the face matching system 1 according to the present disclosure includes a registered image input device 2, a face matching device 3, and a matching image imaging device 6. The collation image imaging device 6 includes an illumination device 4 and an imaging device 5. The face collation device 3 is connected to the registered image input device 2, the illumination device 4, and the imaging device 5 via wired or wireless communication such as a LAN (Local Area Network). In addition, the connection form of the face collation apparatus 3 and the registered image input apparatus 2, the illuminating device 4, and the imaging device 5 is not specifically limited, For example, you may be the connection via networks, such as the internet. Alternatively, a general-purpose interface (for example, a USB (Universal Serial Bus) interface) may be used to connect to each other with a communication cable.

  The registered image input device 2 is a device for inputting the face image (registered image) of the person to be collated captured in advance to the face collating device 3. As the registered image input device 2, for example, a passport reader (passport reader) for reading registered information such as a registered image from an IC chip of an IC passport can be used. The registered image read by the passport reading device (registered image input device 2) is input to the face matching device 3. The registered image input device 2 is not particularly limited in its form, function, arrangement, quantity, etc. as long as at least the registered image can be input to the face matching device 3, and various changes can be made. Is possible.

  As shown in FIG. 2, the collation image capturing device 6 includes a vertical member 7 extending in the vertical direction, and a horizontal member 8 extending horizontally forward from the front side surface 7 b of the upper end portion 7 a of the vertical member 7. have. The vertical member 7 and the horizontal member 8 have such a height and width that the collation subject 9 can enter the space defined by both members.

  The illuminating device 4 includes a plurality of light sources 4a that are LED light sources, for example. The light source 4a is disposed on the front side surface 7b of the vertical member 7 and the lower side surface 8a of the horizontal member 8 so as to illuminate the person 9 to be collated from the front and above. The light sources 4a are arranged in five rows in the left-right direction and seven rows in the front-rear direction on the lower surface 8a of the horizontal member 8. Further, the light source 4a is arranged on the front side surface 7b of the vertical member 7 so as to surround the imaging device 5 in one vertical row on the left side and on the right side and one horizontal row on the lower side. In addition, the arrangement | positioning form of the light source 4a is not limited to this.

  Each light source 4a is configured to be able to be turned on / off individually, and can be individually turned on / off under the control of the illumination device control unit 14 of the face matching device 3 described later. Thereby, it becomes possible to set the direction or light quantity (namely, light intensity) which illuminates the collation subject 9 to various directions or light intensity. Moreover, you may comprise each light source 4a so that light intensity can be adjusted separately. If it does in this way, it will become possible to adjust the light intensity which illuminates collation subject person 9 more precisely. As long as the illumination device 4 can adjust the illumination conditions (illumination direction and light intensity) to various conditions, the form, function, arrangement, quantity, and the like are not particularly limited, and various changes can be made. Is possible.

  The imaging device 5 is a CCD camera, for example, and is disposed on the front side surface 7b of the vertical member 7 so as to capture the face of the person 9 to be verified from the front. The imaging device 5 captures the person to be collated illuminated by the illumination device 4, and the face image (collation image) of the person to be collated captured by the imaging device 5 is input to the face collating device 3. In addition, as long as the imaging device 5 can capture the face image of the person to be collated, the form, function, arrangement, quantity, and the like are not particularly limited, and various changes can be made.

  The face matching device 3 is a general computer device, and although not shown in detail, a CPU (Central Processing Unit) that centrally executes various types of information processing and control of peripheral devices based on a predetermined control program. It has a hardware configuration including a RAM (Random Access Memory) that functions as a work area, a ROM (Read Only Memory) that stores control programs and data executed by the CPU, and a network interface that executes communication processing via the network. doing. Various functions (for example, image matching processing and the like) of the face matching device 3 described in detail later can be realized by the CPU executing a predetermined control program (for example, an image matching program). Note that the face matching device 3 is not limited to a computer device, and other information processing devices (such as a server) that can perform the same function can also be used. Further, at least a part of the functions of the face matching device 3 may be replaced by processing using other conventionally known hardware.

  As shown in FIG. 1, the face collation device 3 includes a registered image acquisition unit 11, a three-dimensional (3D) coordinate setting unit 12, an illumination condition estimation unit 13, an illumination device control unit 14, and a collation image acquisition unit 15. And an image matching unit 16. Each unit is controlled by a control unit (not shown).

  The registered image acquisition unit 11 is connected to the registered image input device 2, and acquires from the registered image input device 2 a face image (registered image) of the person to be collated captured in advance. The registered image acquired by the registered image acquisition unit 11 from the registered image input device 2 is input to the 3D coordinate setting unit 12, the illumination condition estimation unit 13, and the image verification unit 16.

  The 3D coordinate setting unit 12 sets the three-dimensional coordinates of the face area in the registered image input from the registered image acquisition unit 11. Specifically, the three-dimensional coordinates of the face feature points in the registered image are set. The three-dimensional coordinates of the face feature points can be estimated from the pixel values of the face area in the registered image input from the registered image acquisition unit 11 using a conventionally known technique. For example, the correlation between the pixel value shading of the face area in the registered image and the three-dimensional coordinates of the face feature point is learned in advance using a large number of face images and three-dimensional coordinate data, and based on the learned correlation. Thus, it is possible to estimate the three-dimensional coordinates of the face feature points from the shades of pixel values of the face area in the registered image (for example, “Real-Time Combined 2D + 3D Active Appearance Models, CVPR '04, pp. 535-542 ").

  As described above, when the illumination condition at the time of capturing the registered image is estimated based on the three-dimensional coordinates and the pixel value of the face area in the registered image, the illumination condition at the time of capturing the registered image can be estimated more accurately. It becomes. Note that the method of setting the three-dimensional coordinates of the face region (or face feature point) in the registered image is not particularly limited, and various other known techniques can be used. For example, standard three-dimensional coordinate data prepared in advance may be used, or the user may input using input means (not shown). The three-dimensional coordinates of the face area in the registered image set by the 3D coordinate setting unit 12 are input to the illumination condition estimation unit 13.

  The illumination condition estimation unit 13 is based on the three-dimensional coordinates of the face region in the registered image input from the 3D coordinate setting unit 12 and the pixel value of the face region in the registered image input from the registered image acquisition unit 11. The illumination condition at the time of capturing the registered image is estimated. The estimation of illumination conditions at the time of capturing a registered image can be performed using a conventionally known technique. For example, n light sources are discretely set on a virtual celestial sphere, and the presence or absence or intensity of illumination from each light source at each face feature point in the registered image is determined according to each face feature point in the registered image. Is calculated based on the three-dimensional coordinates. Then, by deriving a linear equation representing the pixel value at each facial feature point in the registered image and obtaining a solution thereof, the illumination direction and light intensity of each light source can be obtained (for example, “based on the shadow of the object” Estimating the light source environment, Vol. 41, No. SIG 10, CVIM 1, Dec. 2000 ”). Note that the method for estimating the illumination condition at the time of capturing a registered image is not particularly limited, and various other known techniques can be used. The illumination condition at the time of capturing the registered image estimated by the illumination condition estimation unit 13 is input to the illumination device control unit 14.

  The illuminating device control unit 14 is connected to the illuminating device 4, and individually controls each light source 4 a of the illuminating device 4 to set the illumination conditions (illumination direction and light intensity) of the illuminating device 4. The illumination condition of the illumination device 4 is set so as to reproduce the illumination condition at the time of capturing the registered image estimated by the illumination condition estimation unit 13. As described above, by turning on / off each light source 4a of the illumination device 4 individually or for each illumination position, or by controlling the light intensity of each light source 4a individually or for each illumination position, the illumination direction and The light intensity can be changed, whereby the illumination condition of the illumination device 4 can be set. For example, when using a model that realizes illumination conditions using n light sources on the celestial sphere as in the present embodiment, the illumination device 4 is also provided with a spherical surface corresponding to the celestial sphere, and n light sources are provided on the spherical surface. And the parameters of the n light sources estimated by the illumination condition estimation unit 13 may be reproduced. Thereby, the illumination condition of the illuminating device 4 can be adjusted easily and precisely, and it becomes possible to reproduce the illumination condition at the time of imaging a registered image better. The lighting device 4 illuminates the person to be verified under the lighting conditions set by the lighting device control unit 14, that is, the lighting conditions that reproduce the lighting conditions at the time of capturing the registered image.

  The collation image acquisition unit 15 is connected to the imaging device 5 and acquires the face image (collation image) of the person to be collated captured by the imaging device 5. Since the imaging device 5 captures the person to be collated illuminated under the illumination conditions set by the illumination device control unit 14, the collation image was captured under substantially the same illumination conditions as the illumination conditions at the time of capturing the registered image. It becomes an image. The verification image acquired by the verification image acquisition unit 15 from the imaging device 5 is input to the image verification unit 16.

  The image collation unit 16 collates the collation image input from the collation image acquisition unit 15 with the registration image input from the registration image acquisition unit 11. Specifically, the similarity between the collation image and the registered image is calculated. When the similarity is high, it is determined that the collation between the collation image and the registration image matches, and when the similarity is low, the collation image and the registration image Is determined not to match. As an example of the similarity calculation method, there is a method of calculating the difference between the facial feature amount of the collation image and the facial feature amount of the registered image. When the difference is small, it is determined that the similarity is high, and when the difference is large, it is determined that the similarity is low. In addition, the method of collating a collation image and a registered image is not specifically limited, It is possible to use various other well-known techniques. The collation result in the image collation unit 16 is output to the outside of the face collation device 3 such as a security system or a surveillance camera system via an output unit (not shown).

  FIG. 2 is a block diagram showing a hardware configuration for realizing the face matching system 1 of FIG. As shown in FIG. 2, the face matching system 1 in FIG. 1 includes an image input unit 21, an illumination unit 22, a photographing unit 23, a communication unit 24, a storage unit 25, a processor 26, a display unit 27, an input unit 28, and the like. This is realized by a hardware configuration including a system bus 29 for connecting the two.

  The image input unit 21 is, for example, a passport reading device for reading registration information such as a registered image from an IC chip of an IC passport, and the face verification device 3 uses a face image (registered image) of a verification target person captured in advance. Used to input to. The image input unit 21 corresponds to the registered image input device 2 in FIG.

  The illuminating unit 22 is an illuminating device having a plurality of light sources (for example, LED light sources), and is used to illuminate the person to be collated from the front and from above. Each light source is configured to be able to individually control at least one of on / off and light intensity in order to adjust illumination conditions (illumination direction and light intensity). The illumination unit 22 corresponds to the illumination device 4 in FIG.

  The imaging unit 23 is, for example, a CCD camera, and is used to capture the face image of the person to be collated illuminated by the illumination unit 22 from the front. The imaging unit 23 corresponds to the imaging device 5 in FIG.

  The communication unit 24 is a general communication module, and is used for the face matching device 3 to communicate with other external devices or systems. The communication unit 24 is used, for example, to output the collation result in the image collation unit 16 of the face collation device 3 to the display unit 27, an external security system, a surveillance camera system, or the like.

  The storage unit 25 is, for example, a ROM or a hard disk, and stores various programs and various data for realizing various functions of the face matching device 3. Various programs implement each process in the registered image acquisition unit 11, the 3D coordinate setting unit 12, the illumination condition estimation unit 13, the illumination device control unit 14, the verification image acquisition unit 15, and the image verification unit 16 of the face verification device 3. For example, a program for controlling the entire face matching device 3, a program for controlling the registered image input device 2, the illumination device 4, and the imaging device 5 connected to the face matching device 3. The various types of data include threshold values used in each process in the above-described units of the face matching device 3 and matching data used for matching the face image in the image matching unit 16.

  The processor 26 is, for example, a CPU, reads the above-described various programs and various data from the storage unit 25 onto a RAM (not shown), the registered image acquisition unit 11, the 3D coordinate setting unit 12, and the illumination condition estimation unit of the face matching device 3. 13, each process in the illuminating device control part 14, the collation image acquisition part 15, and the image collation part 16 is performed. In addition, the processor 26 performs overall control of the face matching device 3 and the peripheral devices (the registered image input device 2, the lighting device 4, and the imaging device 5) of the face matching device 3.

  The display unit 27 is a general display device, and is used for displaying a processing result in the face matching device 3.

  The input unit 28 is a general input device such as a keyboard and a mouse, and is used by the user to input a command to the face collation apparatus 3.

  Next, the flow of face image matching processing by the face matching device 3 according to the first embodiment of the present disclosure shown in FIG. 1 will be described with reference to FIG. FIG. 4 is a flowchart showing the flow of face matching processing by the face matching device 3 according to the first embodiment of the present disclosure.

  First, the registered image acquisition unit 11 acquires a face image (registered image) of the person to be collated captured in advance from the registered image input device 2 (step ST101). Subsequently, the 3D coordinate setting unit 12 sets the three-dimensional coordinates of the face area in the registered image (step ST102). Next, the illumination condition estimation unit 13 estimates the illumination condition at the time of capturing the registered image based on the three-dimensional coordinates of the face region in the registered image and the pixel value of the face region in the registered image (step ST103). ).

  In subsequent step ST104, the illuminating device control unit 14 individually controls each light source 4a of the illuminating device 4 to reproduce the illumination conditions at the time of capturing the registered image estimated by the illumination condition estimating unit 13. Set the lighting conditions. The illuminating device 4 illuminates the person to be verified under the illumination conditions set by the illuminating device control unit 14, and the imaging device 5 images the person to be verified illuminated under the illumination conditions.

  In the next step ST105, the collation image acquisition unit 15 obtains a face image (collation image) of the person to be collated captured from the imaging device 5 under the illumination conditions. In step ST106, the image collation unit 16 collates the collation image with the registered image. The matching result is output to the outside of the face matching device 3 via an output unit (not shown).

  As described above, according to the face matching device 3 according to the first embodiment of the present disclosure, the illumination condition at the time of capturing the registered image at the time of capturing the registered image without previously registering the illumination condition at the time of capturing the registered image. Can be reproduced. As a result, the collation image can be imaged under substantially the same illumination conditions as when the registered image was captured, so that the accuracy of collation between the collation image and the registered image can be increased.

(Second Embodiment)
FIG. 5 is an overall configuration diagram of the face matching system according to the second embodiment of the present disclosure, and FIG. 2 is a block diagram illustrating a schematic configuration of the feedback unit 17 of the face matching device 3 in FIG. The face matching system 1 according to the second embodiment is different from the face matching system 1 according to the first embodiment shown in FIG. 1 in that the face matching device 3 includes a feedback unit 17. Since other configurations are the same as those of the first embodiment, the same components are denoted by the same reference numerals, and description thereof is omitted.

  Although not shown, the feedback unit 17 is connected to the registered image acquisition unit 11, the 3D coordinate setting unit 12, the illumination condition estimation unit 13, the verification image acquisition unit 15, and the image verification unit 16. The feedback unit 17 is controlled by a control unit (not shown). When the collation between the collation image and the registered image is not matched by the image collation unit 16, the illumination condition at the time of capturing the collation image is confirmed, and the face in the collation image is displayed. Confirmation of the lighting direction and re-estimation of the illumination condition at the time of capturing the registered image.

  The feedback unit 17 includes a collation image illumination condition estimation unit 31, an illumination condition determination unit 32, a collation image three-dimensional (3D) coordinate estimation unit 33, a collation image face direction detection unit 34, and a registered image face direction detection unit 35. A face orientation determination unit 36, a notification unit 37, and a registered image illumination condition re-estimation unit 38.

  The collation image illumination condition estimation unit 31 is connected to the 3D coordinate setting unit 12 and can acquire the three-dimensional coordinates of the face area in the registered image from the 3D coordinate setting unit 12. The collation image illumination condition estimation unit 31 is connected to the collation image acquisition unit 15 and can acquire a collation image from the collation image acquisition unit 15. When the collation between the collation image and the registered image does not match in the image collation unit 16, the collation image illumination condition estimation unit 31 and the pixel value of the face region in the collation image Based on the above, the illumination condition at the time of capturing the collation image is estimated. The illumination condition at the time of capturing the collation image estimated by the collation image illumination condition estimation unit 31 is input to the illumination condition determination unit 32.

  The illumination condition determination unit 32 is connected to the illumination condition estimation unit 13 and can acquire the illumination condition at the time of capturing the registered image estimated by the illumination condition estimation unit 13. The illumination condition determination unit 32 determines whether or not the illumination condition at the time of capturing the verification image estimated by the verification image illumination condition estimation unit 31 matches the illumination condition at the time of capturing the registered image estimated by the illumination condition estimation unit 13. Determine whether. Specifically, the difference between the illumination condition at the time of capturing the registered image and the illumination condition at the time of capturing the collation image is calculated, and if the difference falls below a predetermined threshold, the illumination at the time of capturing the registered image When it is determined that the conditions and the illumination conditions at the time of capturing the verification image match, and the difference is equal to or greater than the threshold value, the illumination conditions at the time of capturing the registered image and the illumination conditions at the time of capturing the verification image do not match judge. An example of the difference is the sum of absolute values of differences in light intensity at each illumination position (each light source). The determination result of the illumination condition determination unit 32 is input to the collation image 3D coordinate estimation unit 33.

  When the collation between the collation image and the registered image does not match in the image collation unit 16 and the illumination condition determination unit 32 determines that the illumination condition when the registered image is captured and the illumination condition when the collation image is captured match Even though the illumination conditions are the same, the collation between the collation image and the registered image did not match, so that the collation target person can be determined to be different from the person in the registration image.

  If it is determined that the illumination condition at the time of capturing the registered image and the illumination condition at the time of capturing the verification image do not match, the orientation of the face of the person being verified at the time of capturing the verification image is different from the orientation of the face in the registered image It may be the case. Therefore, it is confirmed whether or not the orientation of the face in the collation image is appropriate. If not, the collation image is taken again with the same face orientation as that of the registered image. It is also conceivable that the three-dimensional coordinates of the face area in the registered image used when the illumination condition estimating unit 13 estimates the illumination condition at the time of capturing the registered image are not appropriate. Therefore, when it is determined that the illumination condition at the time of capturing the registered image and the illumination condition at the time of capturing the matching image do not match, the three-dimensional coordinates of the face area in the matching image are used to capture the registered image. Re-estimate the lighting conditions.

  The verification image 3D coordinate estimation unit 33 is connected to the verification image acquisition unit 15 and can acquire the verification image from the verification image acquisition unit 15. When the illumination condition determination unit 32 determines that the illumination condition at the time of capturing the registered image and the illumination condition at the time of capturing the verification image do not match, the verification image 3D coordinate estimation unit 33 3 of the face area in the verification image Estimate dimensional coordinates. The three-dimensional coordinates of the face area in the collation image can be estimated from the pixel values of the face area in the collation image using a conventionally known technique. For example, a method similar to the method used in the 3D coordinate setting unit 12 can be used. The three-dimensional coordinates of the face area in the collation image estimated by the collation image 3D coordinate estimation unit 33 are input to the collation image face direction detection unit 34.

  The collation image face direction detection unit 34 detects the face direction in the collation image based on the three-dimensional coordinates of the face area in the collation image input from the collation image 3D coordinate estimation unit 33. Detection of the orientation of the face in the image can be performed using a conventionally known face recognition technique. For example, detection is possible based on the position of the face feature point in the face area. The face orientation in the collation image detected by the collation image face direction detection unit 34 is input to the face direction determination unit 36.

  The registered image face orientation detection unit 35 is connected to the 3D coordinate setting unit 12 and can acquire the three-dimensional coordinates of the face area in the registered image from the 3D coordinate setting unit 12. The registered image face orientation detection unit 35 detects the orientation of the face in the registered image based on the three-dimensional coordinates of the face area in the registered image acquired from the 3D coordinate setting unit 12. The direction of the face in the image can be detected using a conventionally known face recognition technique, as in the collation image face direction detection unit 34. The face orientation in the registered image detected by the registered image face orientation detection unit 35 is input to the face orientation determination unit 36.

  The face orientation determination unit 36 determines whether the face orientation in the collation image input from the collation image face orientation detection unit 34 matches the face orientation in the registration image input from the registration image face orientation detection unit 35. Determine whether. This determination can also be performed using a conventionally known face recognition technique, similarly to the collation image face direction detection unit 34 and the registered image face direction detection unit 35. The determination result of the face orientation determination unit 36 is input to the notification unit 37 and the registered image illumination condition re-estimation unit 38.

  The notification unit 37 registers the orientation of the face of the person to be collated with respect to the user when the face orientation determination unit 36 determines that the face orientation in the collation image does not match the face orientation in the registration image. A notification is made to change the face orientation to the same face orientation in the image and retake the collation image. The notification may be performed by the notification unit 37 itself, or may be performed via a notification unit (not shown) such as a display device. This makes it possible to re-take a collation image by changing the face direction of the person to be collated to the same face direction as the registered image. Therefore, it is possible to further improve the accuracy of matching the matching image with the registered image.

  The registered image illumination condition re-estimation unit 38 is connected to the registered image acquisition unit 11 and can acquire a registered image from the registered image acquisition unit 11. The registered image illumination condition re-estimating unit 38 determines the illumination condition at the time of capturing the registered image when the face direction determining unit 36 determines that the face direction in the collated image matches the face direction in the registered image. Re-estimation is performed based on the three-dimensional coordinates of the face area in the collation image and the pixel values of the face area in the registered image. The illumination condition at the time of capturing the registered image re-estimated by the registered image illumination condition re-estimator 38 is input to the illumination device controller 14. As a result, when the three-dimensional coordinates of the face area in the registered image used when the illumination condition estimation unit 13 estimates the lighting condition at the time of capturing the registered image are not appropriate, The illumination conditions at the time of capturing the registered image can be estimated again using the three-dimensional coordinates, and this makes it possible to further improve the accuracy of matching the matching image with the registered image.

  Next, the flow of face image matching processing by the face matching device 3 according to the second embodiment of the present disclosure shown in FIGS. 5 and 6 will be described with reference to FIG. FIG. 7 is a flowchart illustrating a flow of face matching processing by the face matching device 3 according to the second embodiment of the present disclosure.

  Steps 201 to 205 are the same as steps ST101 to ST105 in the flowchart of FIG.

  In step 206, the image collating unit 16 determines whether or not the collation between the collation image and the registered image matches. If it is determined that the collation matches (ST206: Yes), the process ends. If it is determined that the collation does not match (ST206: No), the process proceeds to step ST207.

  In step ST207, the collation image illumination condition estimation unit 31 estimates the illumination condition at the time of capturing the collation image based on the three-dimensional coordinates of the face area in the registered image and the pixel value of the face area in the collation image. In subsequent step ST208, the illumination condition determination unit 32 determines whether or not the illumination condition at the time of capturing the collation image matches the illumination condition at the time of capturing the registered image. When it is determined that they match (ST208: Yes), the process proceeds to step ST209, and when it is determined that they do not match (ST208: No), the process proceeds to step ST210. In step ST209, since the collation between the collation image and the registered image does not match even though the illumination conditions are the same, it is determined that the collation target person is different from the person in the registration image, and then the process ends. .

  In step ST210, the collation image 3D coordinate estimation unit 33 estimates the three-dimensional coordinates of the face area in the collation image. Subsequently, in step ST211, the collation image face direction detection unit 34 detects the face direction in the collation image based on the three-dimensional coordinates of the face area in the collation image estimated by the collation image 3D coordinate estimation unit 33. To do. In subsequent step ST212, the registered image face orientation detection unit 35 detects the orientation of the face in the registered image based on the three-dimensional coordinates of the face region in the registered image acquired from the 3D coordinate setting unit 12.

  In next step ST213, the face orientation determination unit 36 determines whether or not the face orientation in the verification image matches the face orientation in the registered image. When it is determined that they match (ST213: Yes), the process proceeds to step ST214, and when it is determined that they do not match (ST213: No), the process proceeds to step ST215.

  In step ST214, the registered image illumination condition re-estimation unit 38 re-estimates the illumination condition at the time of capturing the registered image based on the three-dimensional coordinates of the face region in the collation image and the pixel value of the face region in the registered image. To do. Then, it progresses to step ST204 and repeats the process after ST204.

  In step ST215, the notification unit 37 notifies the collation target person to change the face direction to the same face direction as the face direction in the registered image and to retake the collation image. Then, it progresses to step ST205 and repeats the process after ST205.

  As described above, according to the face collation device 3 according to the second embodiment of the present disclosure, when the collation between the collation image and the registered image does not match in the image collation unit 16, illumination at the time of capturing the collation image Confirmation of conditions, confirmation of the orientation of the face in the collation image, and re-estimation of the illumination conditions at the time of capturing the registration image, thereby further improving the collation accuracy between the collation image and the registration image It becomes possible.

  As mentioned above, although this indication was explained based on specific embodiments, these embodiments are illustrations to the last, and this indication is not limited by these embodiments. Note that the constituent elements of the face matching device according to the present disclosure, the face matching system including the face matching system, and the face matching method according to the present disclosure shown in the above embodiments are not necessarily all required, and at least as long as they do not depart from the scope of the present disclosure. It is possible to select appropriately.

  The face matching device, the face matching system and the face matching method including the face matching device according to the present disclosure are substantially the same as the lighting conditions at the time of capturing the registered image without previously registering the lighting conditions at the time of capturing the recorded image. It is useful as a face collation apparatus that can capture images under the illumination conditions, a face collation system including the face collation apparatus, a face collation method, and the like.

DESCRIPTION OF SYMBOLS 1 Face collation system 2 Registration image input device 3 Face collation device 4 Illumination device 5 Imaging device 6 Collation image imaging device 11 Registration image acquisition part 12 Three-dimensional (3D) coordinate setting part 13 Illumination condition estimation part 14 Illumination apparatus control part 15 Collation Image acquisition unit 16 Image collation unit 17 Feedback unit 31 Collation image illumination condition estimation unit 32 Illumination condition determination unit 33 Collation image three-dimensional (3D) coordinate estimation unit 34 Collation image face direction detection unit 35 Registered image face direction detection unit 36 Face direction Determination unit 37 Notification unit 38 Registered image illumination condition re-estimation unit

Claims (24)

A face matching device for matching a matching image obtained by capturing a face of a person to be verified with a registered image captured in advance,
A registered image acquisition unit for acquiring the registered image;
An illumination condition estimating unit that estimates an illumination condition at the time of capturing the registered image based on a pixel value of a face area in the registered image;
Based on the estimated illumination conditions at the time of capturing the registered image, an illumination device control unit that sets illumination conditions of an illumination device that illuminates the person to be verified, and
A collation image acquisition unit that acquires the collation image captured under the set illumination conditions;
A face collation apparatus comprising: an image collation unit that collates the collation image with the registered image.   The face collation apparatus according to claim 1, wherein the illumination condition includes an illumination direction and a light intensity. A three-dimensional coordinate setting unit for setting the three-dimensional coordinates of the face area in the registered image;
The said illumination condition estimation part estimates the illumination conditions at the time of imaging of the said registration image based on the three-dimensional coordinate and pixel value of the face area | region in the said registration image, The Claim 1 or Claim 2 characterized by the above-mentioned. The face matching device described in 1. The illumination device has a plurality of light sources that are installed at positions where the person to be collated can be illuminated from the front and above, and at least one of on / off and light intensity is individually adjusted and controlled,
The lighting device control unit adjusts lighting conditions of the lighting device by individually controlling at least one of on / off and light intensity of each light source of the lighting device. Item 4. The face matching device according to any one of Items 3 to 4. A collation image illumination condition estimation unit that estimates illumination conditions at the time of capturing the collation image based on the three-dimensional coordinates of the face area in the registered image and the pixel value of the face area in the collation image;
The illumination condition determination unit for determining whether or not an illumination condition at the time of capturing the reference image matches an illumination condition at the time of capturing the registered image is further provided. The face matching device described. A collation image three-dimensional coordinate estimation unit for estimating a three-dimensional coordinate of a face area in the collation image;
A collation image face direction detection unit for detecting the orientation of the face in the collation image based on the three-dimensional coordinates of the face region in the collation image;
A registered image face orientation detection unit that detects the orientation of the face in the registered image based on the three-dimensional coordinates of the face region in the registered image;
6. The face orientation determination unit for determining whether or not the orientation of the face in the collation image matches the orientation of the face in the registered image. The face matching device described in 1.   When the face orientation determination unit determines that the face orientation in the verification image matches the orientation of the face in the registered image, the illumination condition at the time of capturing the registered image is set as the face area in the verification image. The face collation apparatus according to claim 6, further comprising a registered image illumination condition re-estimation unit that re-estimates based on a three-dimensional coordinate of the image and a pixel value of a face area in the registered image.   When the face orientation determination unit determines that the face orientation in the collation image does not match the face orientation in the registered image, the face orientation of the person to be collated is indicated to the registered image to the user. The face collation apparatus according to claim 6, further comprising a notification unit that notifies the user to change the face direction to the same face direction and retake the collation image. A face matching device according to any one of claims 1 to 8,
A registered image input device for inputting a registered image to the face matching device;
A lighting device capable of adjusting lighting conditions;
A face matching system comprising: an imaging device that images a person to be matched illuminated by the lighting device.   The face matching system according to claim 9, wherein the illumination condition includes an illumination direction and light intensity. The face matching device further includes a three-dimensional coordinate setting unit that sets three-dimensional coordinates of a face region in the registered image,
The illumination condition estimation unit estimates an illumination condition at the time of capturing the registered image based on a three-dimensional coordinate and a pixel value of a face area in the registered image. The face matching system described in. The illumination device has a plurality of light sources that are installed at positions where the person to be collated can be illuminated from the front and above, and at least one of on / off and light intensity is individually adjusted and controlled,
The lighting device control unit adjusts lighting conditions of the lighting device by individually controlling at least one of on / off and light intensity of each light source of the lighting device. Item 12. The face matching system according to any one of Items 11. The collation image illumination condition estimation unit that estimates the illumination condition at the time of capturing the collation image based on the three-dimensional coordinates of the face area in the registered image and the pixel value of the face area in the collation image. When,
The illumination condition determination unit that determines whether or not an illumination condition at the time of capturing the reference image matches an illumination condition at the time of capturing the registered image is further provided. The face matching system described. A collation image three-dimensional coordinate estimation unit for estimating a three-dimensional coordinate of a face region in the collation image;
A collation image face direction detection unit for detecting the orientation of the face in the collation image based on the three-dimensional coordinates of the face region in the collation image;
A registered image face orientation detection unit that detects the orientation of the face in the registered image based on the three-dimensional coordinates of the face region in the registered image;
The face orientation determination unit for determining whether or not the orientation of the face in the collation image matches the orientation of the face in the registered image. Face matching system.   When the face collation apparatus determines that the face orientation in the collation image matches the face orientation in the registered image by the face orientation determination unit, the illumination condition at the time of capturing the registered image is The registered image illumination condition re-estimation unit that re-estimates based on the three-dimensional coordinates of the face region in the collation image and the pixel value of the face region in the registered image. Face matching system.   When the face collation apparatus determines that the face orientation in the collation image does not match the face orientation in the registered image by the face orientation determination unit, the face of the person to be collated is displayed to the user. The face verification system according to claim 14, further comprising a notification unit that notifies the user to change the direction of the face to the same face direction as the face in the registered image and retake the verification image. . A face matching method for matching a matching image obtained by capturing a face of a person to be verified with a registered image captured in advance,
A registered image obtaining step for obtaining the registered image;
An illumination condition estimating step for estimating an illumination condition at the time of capturing the registered image based on a pixel value of a face area in the registered image;
A lighting device control step of setting a lighting condition of a lighting device that illuminates the person to be collated based on a lighting condition at the time of capturing the estimated registered image,
A collation image capturing step for obtaining the collation image captured under the set illumination conditions;
A face matching method comprising: an image matching step for matching the matching image with the registered image.   The face matching method according to claim 17, wherein the illumination condition includes an illumination direction and light intensity. A three-dimensional coordinate setting step of setting a three-dimensional coordinate of the face area in the registered image;
19. The illumination condition estimating step estimates an illumination condition at the time of capturing the registered image based on a three-dimensional coordinate and a pixel value of a face area in the registered image. The face matching method described in 1. The illumination device has a plurality of light sources that are installed at positions where the person to be collated can be illuminated from the front and above, and at least one of on / off and light intensity is individually adjusted and controlled,
The lighting device control step adjusts the lighting condition of the lighting device by individually controlling at least one of on / off and light intensity of each light source of the lighting device. Item 20. The face matching method according to any one of Items 19. A collation image illumination condition estimation step for estimating an illumination condition at the time of capturing the collation image based on a three-dimensional coordinate of the face area in the registered image and a pixel value of the face area in the collation image;
21. The illumination condition determination step of determining whether or not an illumination condition at the time of capturing the collation image matches an illumination condition at the time of capturing the registered image. Face matching method. A collation image three-dimensional coordinate estimation step for estimating a three-dimensional coordinate of a face region in the collation image;
A collation image face direction detection step of detecting a face direction in the collation image based on a three-dimensional coordinate of a face region in the collation image;
A registered image face direction detecting step for detecting a face direction in the registered image based on a three-dimensional coordinate of a face region in the registered image;
The face orientation determination step of determining whether or not the orientation of the face in the collation image matches the orientation of the face in the registered image, according to any one of claims 19 to 21. The face matching method described.   When it is determined in the face orientation determination step that the face orientation in the collation image matches the face orientation in the registration image, the illumination condition at the time of capturing the registration image is set as the face region in the collation image. 23. The face collation method according to claim 22, further comprising a registered image illumination condition re-estimation step for re-estimation based on the three-dimensional coordinates of the image and a pixel value of the face area in the registered image.   When it is determined in the face orientation determination step that the face orientation in the collation image does not match the face orientation in the registered image, the face orientation of the person to be collated is indicated to the registered image to the user. 23. The face matching method according to claim 22, further comprising a notification step of notifying the user to change the orientation of the face to the same face orientation and retake the matching image.

Images