JP4141090B2 - Image recognition apparatus, shadow removal apparatus, shadow removal method, and recording medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image recognition apparatus, a shadow removal apparatus, a shadow removal method, and the like used for personal authentication.
[0002]
[Prior art]
For example, as a technique for personal authentication, a biometric personal authentication technique for identifying an individual using a human biometric feature is known. As one of the personal authentication techniques, there is a face image recognition technique for recognizing an individual using a face image.
In the authentication process using this face image recognition technology, a human face is captured by a camera to acquire a face image, and after performing predetermined preprocessing such as position and size normalization on the face image, feature extraction is performed. And matching with a face image pattern registered in advance to identify a person.
[0003]
[Problems to be solved by the invention]
However, in the authentication apparatus using the face image recognition technique as described above, one of the major problems is a deterioration in recognition performance due to a change in illumination during photographing.
In order to solve this problem, stable lighting can be achieved by applying strong illumination light from the front or covering the camera and person with a box such as an ID photo camera so that shadows do not appear on the face during shooting. There is a technique for taking a picture under illumination. However, in this case, since a powerful illumination device, a box, or the like is used, the system configuration becomes large and the installation location is limited.
[0004]
In this case, the same problem may occur in recognition processing of not only a photographed image of a person's face but also a photographed image of a general object.
[0005]
The present invention has been made in view of the above-described circumstances, and an object of the present invention is to provide an image recognition device and the like that can cope with a change in illumination at the time of photographing in recognition of an object by a photographed image.
Another object of the present invention is to provide a shadow removal apparatus and the like that can remove a shadow in a captured image, which is suitable for an object recognition apparatus that recognizes an object from a captured image.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, an image recognition apparatus according to the first aspect of the present invention provides:
In an image recognition device that recognizes an image,
Means for storing a plurality of distance images generated in advance based on measurement results of the three-dimensional shapes of a plurality of objects and having a distance value between a predetermined position and the surface of the object as a pixel value;
An input means for inputting a captured image of the object;
A composite image is generated by changing the three-dimensional shape parameter of the object surface shape model representing the shape of the surface of the object acquired from the distance image and the illumination direction parameter, and has the highest similarity with the input image A selection means for selecting a three-dimensional shape parameter and an illumination direction parameter of the composite image;
Brightness correction means for performing brightness correction by dividing the brightness value of each corresponding pixel of the input image by the brightness value of each pixel of the composite image selected by the selection means;
Means for performing a predetermined recognition process on the input image whose luminance has been corrected;
It is characterized by providing.
[0007]
According to such a configuration, an input using a model representing the shape of the surface of the object, which is generated from a plurality of distance images having a pixel value as a distance value between the predetermined position and the surface of the object such as a human face. The three-dimensional shape of the surface of the object in the image and the direction of illumination can be estimated, and the luminance of the input image can be corrected based on the estimated shape to remove the shadow. As a result, it is possible to recognize a face image that is shaded by a change in illumination. In addition, each face image can be recognized with a simple system configuration without using a special device so as not to generate a shadow.
[0008]
Moreover, you may memorize | store further the average image of several images without a shadow,
The similarity between the input image input by the input unit and the average image may be calculated, and each process by the selection unit and the luminance correction unit may be performed when the similarity is lower than a predetermined value.
[0009]
A shadow removing apparatus according to the second aspect of the present invention provides:
A shadow removal apparatus that removes shadows from a captured image of an object,
Means for storing a plurality of distance images generated in advance based on measurement results of the three-dimensional shapes of a plurality of objects and having a distance value between a predetermined position and the surface of the object as a pixel value;
An input means for inputting a captured image of the object;
A composite image is generated by changing the three-dimensional shape parameter of the object surface shape model representing the shape of the surface of the object acquired from the distance image and the illumination direction parameter, and has the highest similarity with the input image A selection means for selecting a three-dimensional shape parameter and an illumination direction parameter of the composite image;
Means for correcting the luminance by dividing the luminance value of each corresponding pixel of the input image by the luminance value of each pixel of the composite image selected by the selecting means, and removing the shadow of the input image;
It is characterized by providing.
[0010]
According to such a configuration, an input using a model representing the shape of the surface of the object, which is generated from a plurality of distance images having a pixel value as a distance value between the predetermined position and the surface of the object such as a human face. The three-dimensional shape of the surface of the object in the image and the direction of illumination can be estimated, and the luminance of the input image can be corrected based on the estimated shape to remove the shadow.
[0011]
The shadow removal method according to the third aspect of the present invention is as follows.
A shadow removal method for removing shadows from a captured image of an object,
Based on the measurement results of the three-dimensional shape of a plurality of objects, generating a plurality of distance images having a distance value between a predetermined position and the surface of the object as pixel values;
Inputting a captured image of the object;
A composite image is generated by changing the three-dimensional shape parameter of the object surface shape model representing the shape of the surface of the object acquired from the distance image and the illumination direction parameter, and has the highest similarity with the input image A selection step of selecting a three-dimensional shape parameter and an illumination direction parameter of the composite image;
Performing luminance correction by dividing the luminance value of each corresponding pixel of the input image by the luminance value of each pixel of the composite image selected in the selection step, and removing the shadow of the input image;
It is characterized by providing.
[0012]
According to such a configuration, an input using a model representing the shape of the surface of the object, which is generated from a plurality of distance images having a pixel value as a distance value between the predetermined position and the surface of the object such as a human face. The three-dimensional shape of the surface of the object in the image and the direction of illumination can be estimated, and the luminance of the input image can be corrected based on the estimated shape to remove the shadow.
[0013]
A recording medium according to the fourth aspect of the present invention is
A computer-readable recording medium recording a program for causing a computer to function as an image recognition device for recognizing an image,
The computer
Means for storing a plurality of distance images generated in advance based on measurement results of the three-dimensional shapes of a plurality of objects and having a distance value between a predetermined position and the surface of the object as a pixel value;
An input means for inputting a photographed image of an object;
A composite image is generated by changing the three-dimensional shape parameter of the object surface shape model representing the shape of the surface of the object acquired from the distance image and the illumination direction parameter, and has the highest similarity with the input image Selection means for selecting a three-dimensional shape parameter and an illumination direction parameter of the composite image;
A luminance correction unit that performs luminance correction by dividing the luminance value of each corresponding pixel of the input image by the luminance value of each pixel of the composite image selected by the selection unit;
Means for performing a predetermined recognition process on the luminance-corrected input image;
Record the program to make it function as
[0014]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, a face image recognition apparatus according to an embodiment of the present invention will be described with reference to the drawings.
As shown in FIG. 1, the face image recognition apparatus 1 according to the embodiment of the present invention includes a storage unit 11, an image capture unit 13, a shadow removal unit 15, and a recognition unit 17. A camera 2 for photographing the face of the person is connected.
[0015]
For example, the storage unit 11 averages the brightness of each pixel of the face image obtained by photographing the face image of the face image obtained in front of, for example, about hundreds of people with no shadow on the face. The obtained average image (average face image) is stored.
Further, the storage unit 11 stores information such as a face distance image and an average face distance image. These information is obtained by measuring the three-dimensional shape of the face in advance using the non-contact three-dimensional shape input device for the above-mentioned hundreds of persons, inputting the measurement result to a predetermined computer, and performing a predetermined calculation process. It is the information obtained by doing.
[0016]
The face distance image is an image generated by a conversion process into a two-dimensional image using a distance value between the measuring device (three-dimensional shape input device) and the face surface as a pixel value. The face distance image represents the unevenness of the face, that is, the face shape, by each pixel value. By applying a parameter related to the illumination direction (illumination direction parameter) to the face distance image and synthesizing the image, it is possible to synthesize a face image that is shaded by illumination from an arbitrary direction.
[0017]
The average face distance image is, for example, an average image of face distance images obtained by calculating an average value of pixel values of each face distance image. This average face distance image represents an average face shape of a human.
[0018]
Further, raster scanning is performed on each of the face distance images for several hundred people, and an image vector having pixel values in the image as a one-dimensional array is obtained. Then, the principal component analysis is performed on the acquired image vectors for several hundred people, and as a result of the principal component analysis, several eigenvectors having a high contribution ratio are selected and stored in the storage unit 11. A face distance image of an arbitrary person can be approximated by a linear combination of the selected several eigenvectors. Here, if the number of selected eigenvectors is n, the face shape of the person can be substantially reproduced by the eigenvector coupling coefficients (A1, A2,..., An). In this system, this eigenvector coupling coefficient (A1, A2,..., An) is used as a parameter (hereinafter referred to as a three-dimensional parameter) and used to construct a human face shape.
Note that the facial shape model expressed by the linear combination of the eigenvectors and the three-dimensional parameters described above can be statistically applied to all other persons by sufficiently increasing the number of facial distance images to be subjected to principal component analysis. Can also be applied, and a more general human face shape model can be obtained.
[0019]
The face distance image and the face image are unified with respect to the face direction, size, etc., for example, by taking a picture based on the center point of the eyeballs of both eyes of each person's face.
[0020]
The image capturing unit 13 acquires an input image (face image) from the camera 2, measures the similarity with the average face image stored in the storage unit 11, and calculates the shadow of the face image that is the input image. Determine presence or absence. Specifically, if the similarity is lower than a predetermined value, it is determined that there is a shadow, and if the similarity is equal to or higher than a predetermined value, it is determined that there is no shadow. If it is determined that there is a shadow, the shadow removal unit 15 is requested to remove the shadow. If it is determined that there is no shadow, the recognition unit 17 is requested to recognize the input image.
[0021]
The shadow removing unit 15 roughly determines the illumination direction parameter in the input image in response to a request from the image capturing unit 13. Here, the illumination parameter indicates from which direction the illumination is applied to the face. The origin is set at a predetermined position (for example, the center of gravity of the face), and the X axis, Y axis, and Z axis are set. The coordinate values (α, β, γ) of the position of the light source when each coordinate axis is set. The shadow removal unit 15 changes the illumination direction parameters (α, β, γ) of the average face distance image stored in the storage unit 11 and synthesizes the face image in which the shadow is generated. Then, the similarity between each composite image and the input image is measured, and the illumination direction parameters (α0, β0, γ0) when the similarity is the highest are obtained.
[0022]
Next, the shadow removing unit 15 uses the illumination direction parameters (α0, β0, γ0) obtained previously as a reference, and the illumination direction parameters (α, β, γ) and the three-dimensional shape parameters (A1, A2,... ., An)) are changed little by little to generate respective composite images. Then, the similarity between the composite image and the input image is measured, and the optimum illumination direction parameters (α ′, β ′, γ ′) when the similarity is the highest, and the optimum three-dimensional shape parameters (A1 ′, A2 ′,. .., An ′) is obtained.
[0023]
Next, the shadow removing unit 15 uses the optimum illumination direction parameters (α ′, β ′, γ ′) and the optimum three-dimensional shape parameters (A1 ′, A2 ′,..., An ′) obtained previously. Then, the brightness of the input image is corrected to remove the shadow. Specifically, by dividing the luminance value of the corresponding pixel of the input image by the luminance value of each pixel of the face image synthesized with the optimal illumination direction parameter and the optimal three-dimensional shape parameter, the input image is displayed as a shadowless image. To correct. The shadow removal unit 15 requests the recognition unit 17 to recognize the input image from which the shadow is removed.
[0024]
The recognition unit 17 uses, for example, a known face recognition method (for example, eigenface method) for the input image from the image capture unit 13 or the input image from which the shadow removal unit 15 has removed the shadow. Predetermined recognition processing such as feature extraction and matching is performed, and a recognition result is output. In addition, since the shadow is removed from the input image to the recognition unit 17, a recognition method that does not correspond to the influence of illumination can be used as the recognition method here.
[0025]
Next, the operation of the face image recognition apparatus 1 according to the present embodiment will be described with reference to the flowchart of FIG.
First, a face image of a person to be authenticated is photographed by the camera 2 connected to the face image recognition device 1, and the photographed image is input to the image capturing unit 13 (step S1).
The image capturing unit 13 measures the similarity between the input image and the average face image stored in the storage unit 11 (step S2), and whether or not shade is generated in the input image based on the measurement result. Is discriminated (step S3). For example, if the measured similarity is lower than a predetermined value, it is determined that the input image is shaded, and the shadow removal unit 15 is requested to remove the shadow.
[0026]
In response to the shadow removal request, the shadow removal unit 15 performs a shadow removal process for removing the shadow of the input image (step S4). In this shadow removal processing, for example, as shown in FIG. 3, first, with respect to the average face distance image, the illumination direction parameters (α, β, γ) are changed to create a composite image with each parameter value (step S11). ), The degree of similarity between each composite image and the input image is measured, and the illumination direction parameters (α0, β0, γ0) when the degree of similarity is the highest are calculated (step S12).
[0027]
Next, based on the illumination direction parameters (α0, β0, γ0) obtained in step S12, the shadow removal unit 15 and the illumination direction parameters (α, β, γ) and the three-dimensional shape parameters (A1, A2,. .., An) are changed little by little to generate a composite image at each parameter value (step S13), the similarity between the composite image and the input image is measured, and the optimum illumination direction parameter that provides the highest similarity (Α ′, β ′, γ ′) and optimum three-dimensional shape parameters (A1 ′, A2 ′,..., An ′) are obtained (step S14).
[0028]
Next, the shadow removal unit 15 performs facial image synthesis using the optimal illumination direction parameters (α ′, β ′, γ ′) and the optimal three-dimensional shape parameters (A1 ′, A2 ′,..., An ′). By dividing the luminance value of the corresponding pixel in the input image by the luminance value of each pixel, the shadow of the input image is removed (step S15), and the recognition unit 17 is requested to recognize the input image, and the process returns to the main flow.
[0029]
In response to a recognition request from the shadow removal unit 15, the recognition unit 17 recognizes a face image by using a general recognition method such as the eigenface method for the input image from which the shadow has been removed by the shadow removal processing. For example, a person is identified from the face image registered in the storage unit 11 (step S5), and the recognition result is output (step S6).
[0030]
In step S3, if the measured value of the similarity between the input image and the average face image is equal to or greater than a predetermined value, the image capturing unit 13 determines that no shadow has occurred in the input image, and directly recognizes the recognition unit. 17 requests the recognition of the input image.
[0031]
In this way, when a human statistical three-dimensional model is constructed and a shadow is generated in the face image to be recognized, the three-dimensional shape of the face and the direction of illumination are estimated from the three-dimensional model. Based on the above, it is possible to correct the brightness of the face image and remove the shadow of the image. As a result, it is possible to recognize a face image that is shaded by a change in illumination. In addition, each face image can be recognized with a simple system configuration without using a special device so as not to generate a shadow.
[0032]
In addition, by using a three-dimensional model acquired from a face distance image generated based on a measurement result of a three-dimensional shape of a face, for example, a three-dimensional model is estimated from a plurality of two-dimensional images, and the estimation result is used. Therefore, the amount of calculation is small compared to the processing to be performed, so that it is not necessary to use a high-performance computer and it can be realized by a standard computer.
[0033]
Further, the optimum three-dimensional shape parameter and the optimum illumination direction parameter may be obtained using direct or other indirect means without obtaining the rough illumination direction parameter.
Further, all input images may be subjected to the shadow removal process without determining the presence or absence of the shadow.
[0034]
The face image recognition device 1 may generate data such as each face distance image, average face distance image, and average face image stored in the storage unit 11 and store the data in the storage unit 11.
Note that the image used in the present apparatus may be a gray scale image representing gradation in gradation, or a color image or the like.
[0035]
Further, the illumination parameter is not limited to the above coordinate values (α, β, γ), and may be any as long as it indicates the direction from which the face is illuminated, and the content and display format are arbitrary. is there.
[0036]
The three-dimensional shape model is not limited to the linear combination of eigenvectors selected by principal component analysis of each pixel value of the facial distance image, and other models that can represent the facial shape may be used.
[0037]
Further, the average face image is not limited to the average image of the face image, and a standard face image obtained by another method may be used. For example, an average face image may be selected from a plurality of photographed images and used as the average face image.
[0038]
Further, a shadow removing device that includes the storage unit 11, the image capturing unit 13, and the shadow removing unit 15 and supplies an image from which the shadow has been removed to the recognition device may be realized.
[0039]
The system of the present invention can be realized using a normal computer system, not a dedicated system. For example, by installing the program from a medium (floppy disk, CD-ROM, etc.) that stores the program for executing the above-described operation in a computer, a face image recognition device that executes the above-described processing is configured. Can do. Note that when the OS realizes the above functions by sharing the OS or jointly of the OS and the application, only the part other than the OS may be stored in the medium.
[0040]
It is also possible to superimpose a program on a carrier wave and distribute it via a communication network. For example, the program may be posted on a bulletin board (BBS) of a communication network and distributed via the network.
The above-described processing can be executed by starting this program and executing it in the same manner as other application programs under the control of the OS.
[0041]
【The invention's effect】
As described above, according to the present invention, a model representing the shape of the surface of an object, which is generated from a plurality of distance images having pixel values as distance values from a predetermined position to the surface of the object such as a human face. It is possible to estimate the three-dimensional shape of the surface of the object and the direction of illumination in the input image, and to correct the luminance of the input image based on that to remove the shadow. As a result, it is possible to recognize a face image that is shaded by a change in illumination.
[Brief description of the drawings]
FIG. 1 is a system configuration diagram of a face image recognition apparatus according to an embodiment of the present invention.
FIG. 2 is a flowchart for explaining the operation of the face image recognition apparatus of FIG. 1;
FIG. 3 is a flowchart for explaining shadow removal processing;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Face image recognition apparatus 11 Memory | storage part 13 Image capture part 15 Shadow removal part 17 Recognition part 2 Camera

Claims (5)

In an image recognition device that recognizes an image,
Means for storing a plurality of distance images generated in advance based on measurement results of the three-dimensional shapes of a plurality of objects and having a distance value between a predetermined position and the surface of the object as a pixel value;
An input means for inputting a captured image of the object;
A composite image is generated by changing the three-dimensional shape parameter of the object surface shape model representing the shape of the surface of the object acquired from the distance image and the illumination direction parameter, and has the highest similarity with the input image A selection means for selecting a three-dimensional shape parameter and an illumination direction parameter of the composite image;
Brightness correction means for performing brightness correction by dividing the brightness value of each corresponding pixel of the input image by the brightness value of each pixel of the composite image selected by the selection means;
Means for performing a predetermined recognition process on the input image whose luminance has been corrected;
An image recognition apparatus comprising: Further memorize the average image of multiple images without shadows,
Calculating the degree of similarity between the input image input by the input means and the average image, and performing processing by the selection means and the luminance correction means when lower than a predetermined value;
The image recognition apparatus according to claim 1. A shadow removal apparatus that removes shadows from a captured image of an object,
Means for storing a plurality of distance images generated in advance based on measurement results of the three-dimensional shapes of a plurality of objects and having a distance value between a predetermined position and the surface of the object as a pixel value;
An input means for inputting a captured image of the object;
A composite image is generated by changing the three-dimensional shape parameter of the object surface shape model representing the shape of the surface of the object acquired from the distance image and the illumination direction parameter, and has the highest similarity with the input image A selection means for selecting a three-dimensional shape parameter and an illumination direction parameter of the composite image;
Means for correcting the luminance by dividing the luminance value of each corresponding pixel of the input image by the luminance value of each pixel of the composite image selected by the selecting means, and removing the shadow of the input image;
A shadow removing device comprising: A shadow removal method for removing shadows from a captured image of an object,
Based on the measurement results of the three-dimensional shape of a plurality of objects, generating a plurality of distance images having a distance value between a predetermined position and the surface of the object as pixel values;
Inputting a captured image of the object;
A composite image is generated by changing the three-dimensional shape parameter of the object surface shape model representing the shape of the surface of the object acquired from the distance image and the illumination direction parameter, and has the highest similarity with the input image A selection step of selecting a three-dimensional shape parameter and an illumination direction parameter of the composite image;
Performing luminance correction by dividing the luminance value of each corresponding pixel of the input image by the luminance value of each pixel of the composite image selected in the selection step, and removing the shadow of the input image;
A shadow removal method comprising: A computer-readable recording medium recording a program for causing a computer to function as an image recognition device for recognizing an image,
The computer
Means for storing a plurality of distance images generated in advance based on measurement results of the three-dimensional shapes of a plurality of objects and having a distance value between a predetermined position and the surface of the object as a pixel value;
An input means for inputting a photographed image of an object;
A composite image is generated by changing the three-dimensional shape parameter of the object surface shape model representing the shape of the surface of the object acquired from the distance image and the illumination direction parameter, and has the highest similarity with the input image Selection means for selecting a three-dimensional shape parameter and an illumination direction parameter of the composite image;
A luminance correction unit that performs luminance correction by dividing the luminance value of each corresponding pixel of the input image by the luminance value of each pixel of the composite image selected by the selection unit;
Means for performing a predetermined recognition process on the luminance-corrected input image;
A computer-readable recording medium in which a program for functioning as a computer is recorded.

Images