JP2009086926A - Image recognition method and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image recognition method and device, having resistance to fluctuation of photographic environment or photographic equipment. <P>SOLUTION: An LBP (Local Binary Pattern) of a notice pixel Pij of a reference face image D and an LBP of a correspondence pixel P'ij of a position corresponding to the notice pixel Pij in a referred face image Dk are compared in each corresponding bit, "1" is allocated to the bit having an according value, "0" is allocated to the bit having a discording value, and a bit sum corresponding to one pixel becomes a similar score of the notice pixel Pij. That is, when the LBP of the notice pixel Pij of the reference face image D is (10001011) and when the LBP of the correspondence pixel P'ij of the referred face image Dk is (00011010), the bits second, third, fifth, sixth, and seventh from the MSB accord, so that the similar score of the notice pixel Pij becomes "5". It is repeated with respect to all the pixels of the reference face image D, and a total ΣCij of the bit sums of all the pixels becomes a similar score between the reference face image D and the referred face image Dk. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image recognition method and apparatus, and more particularly, to an image recognition method and apparatus suitable for facial image recognition.

  Research and development of face recognition that automatically detects a face area from a face photograph and identifies who the face area is active. In face authentication, the face, which is originally a three-dimensional structure, is recognized as a two-dimensional image, so that the appearance of the face and the identification result differ greatly depending on the influence of various variable elements added to the face. There is. Typical variation factors include face orientation, lighting, camera characteristics, facial expressions, aging, and the like. Since it is difficult to handle and solve all the variable elements at once, it is common to handle the variable elements separately.

  Here, focusing on fluctuations due to differences in lighting and camera characteristics, there are a wide variety of shooting environments such as indoor, outdoor, sunny, cloudy, under fluorescent lighting, and under incandescent lighting. In response, the shadow of the face image changes. These are called “lighting fluctuations”, and the appearance of the face changes variously due to the lighting fluctuations.

  Also, in equipment such as a camera that shoots a face and a scanner that captures images, it is generally more natural to adjust the relative relationship between color data such as images and signals when they are actually output. A correction operation is performed to obtain a display. A typical correction operation is gamma correction. The γ (gamma) value is the ratio of the change in the voltage conversion value to the change in the brightness of the image, and it is ideal that this value approaches 1. However, the value varies depending on the device depending on the characteristics of the element. For this reason, gamma correction is performed to reproduce a display faithful to the original data.

Patent Document 1 discloses a technique for suppressing such “illumination fluctuation” and “characteristic fluctuation” of photographing equipment, which are problems of face authentication. In this prior art, a plurality of faces are photographed in advance under various environments and cameras, an eigenvector is created by KL expansion, and this is registered. As a result, the registration data includes a variable element, so that it is possible to correctly authenticate a face photograph to which the change is added.
JP-A-11-175718

  However, in the above-described prior art, in order to include a variable element in the registration data, it is necessary to take several face photographs while changing illumination and photographing equipment. Therefore, when there is only one camera or when it is difficult to change the illumination, registration data including a variable element cannot be created. In addition, it is difficult to reflect all the variation factors relating to the characteristics of lighting and photographing equipment in the registered data. If an unexpected variation factor occurs, the recognition accuracy is lowered.

  In addition, even when the variation factors at the time of shooting are biased, there is a risk that the similarity score will increase due to a stronger response to the variation factors than the characteristics of the face itself. For example, an illumination variation with light from the upper right is added to the face photograph taken at the time of authentication, and the element of the illumination variation is not included in the registered data of the person, but the variation element is included in the registration data of the other person. If it is included, there is a possibility that the similarity score with other person's registered data may increase.

  An object of the present invention is to solve the above-described problems of the prior art and provide an image authentication method and apparatus that is resistant to variations in illumination and characteristics of photographing equipment.

  In order to achieve the above-described object, the present invention provides an image recognition method for calculating a similarity score between a reference image and a referenced image, and for each pixel of each image, the pixel and a plurality of peripheral pixels. And a procedure for obtaining a relative value sequence in which relative values of feature amounts are arranged according to the positions of the surrounding pixels, and a relative value sequence of corresponding pixels of the reference image and the referenced image for each corresponding relative value. And a procedure for calculating a similarity score between relative value sequences and a procedure for calculating a similarity score between the reference image and the referenced image based on the similarity between the relative value sequences of each pixel. It is characterized by.

  According to the present invention, images are compared based on the adjacent relative value sequence (LBP) that is resistant to variations in illumination and characteristics of the photographic equipment, and image recognition is performed based on the similarity score. It is possible to perform image authentication that is resistant to variations in lighting and characteristics of photographing equipment.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the best embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the main part of a face image recognition system to which the present invention is applied. The photographed face image is compared with a large number of face images already registered in the database, and the degree of similarity is high. The registered face image is output as a recognition result.

  The captured face image is captured by the face image detection unit 10. The face detection unit 10 detects a face area from the input face image. In the present embodiment, coordinate information of a face area is output based on a known face image detection method described in Document A below. When the face area is defined by a rectangle, the coordinate information is the upper left coordinates, width, and height of the face area. If a plurality of face areas are detected, coordinate information of each face area is output.

  Reference A: P. Viola, M.J. Jones, "Robust real-time object detection," in: Second International Workshop on Theories of Visual Modeling, Learning, Computing, and Sampling, 2001.

  The reference point detection unit 11 detects the positions of the eyes and nose as organs that are the reference points of the face from the detected face area. In the present embodiment, the reference point is detected using a circular ring filter, which is a typical technique for detecting organs such as the eyes and nose from the face region.

  FIG. 2 is a diagram conceptually showing an organ detection method using a circular ring filter. First, a face region R is specified [FIG. 2 (a)] based on coordinate information. Next, conceptually, a white ring filter whose outer periphery is radius r2 and a black circular ring filter whose inner periphery is radius r1 [FIG. In the circular ring filter, the output becomes large at the point where the difference between the black eye and the white eye is large, so that the filter output (isolated point) can be obtained at the corresponding position of the eye and the nose [FIG. In the present embodiment, two isolated points P1 and P2 having a large output value of the circular ring filter above the face region are recognized as “eyes”, and two isolated points P3 and P4 having a large output value of the circular ring filter are detected below. Is recognized as “nose” [(c)].

  Returning to FIG. 1, the normalization unit 12 determines the face cutout position based on the reference point, and normalizes the face image cut out from the cutout position so that the size (number of pixels) is constant. Turn into.

  In the present embodiment, as shown in FIG. 3, the distance between the detected coordinates P1, P2 of both eyes is set to “10”, and each vertical line of the distance “3” from the eyes to the left and right directions based on this distance. Is the left and right contours L1 and L2 of the face area, the horizontal line with a distance “6” upward from the eyes is the upper contour L3 of the face area, and the horizontal line with a distance “14” downwards from the eyes is the lower contour L4 of the face area. 20:16] is determined as a face clipping region. Next, the face image is normalized by enlarging / reducing so that the number of pixels in the cut-out area becomes [40 × 50].

  Returning to FIG. 1, the adjacent relative value sequence conversion unit 13 sets each pixel of interest in the normalized face image as a relative feature amount between the pixel of interest and a plurality of surrounding pixels adjacent to the pixel of interest. Is converted into a numerical string (represented as an adjacent relative value string) arranged in accordance with the position of each peripheral pixel. In the present embodiment, LBP (Local Binary Pattern) is adopted as the adjacent relative value sequence, and the LBP of each pixel of interest is obtained.

  LBP is a texture analysis technique, and is mainly used for region segmentation. This LBP is similar to edge extraction filter (Laplacian filter, Canny filter, etc.) processing in the field of image processing, and the output value is determined based on the relative value of the feature quantity between the target pixel and surrounding pixels.

  FIG. 4 is a diagram conceptually illustrating LBP. LBP is information indicating a relative value (in this embodiment, a luminance difference) between feature amounts of one pixel of interest Pij and its surrounding pixels, as shown in FIG. LBP indicates the relative size of each peripheral pixel with respect to the pixel of interest, and LBP is considered to change little under global illumination in nature.

  In the present embodiment, for a 3 × 3 pixel block including one target pixel extracted from a face image and eight peripheral pixels adjacent to the target pixel, “1” and “0” are assigned to small peripheral pixels, respectively, and a numerical string composed of 8 bits clockwise with the upper left peripheral pixel being the MSB is output as an LBP. In the example of FIG. 4, the LBP value is “131 (decimal number)”.

  5 (a) and 5 (b) are LBP images obtained by visualizing LBP obtained from two face images obtained by photographing the same person's face under different illuminations, and each pixel corresponds to LBP. Expressed in brightness. Comparing (a) and (b) in the figure, it can be seen that although the face image has a luminance difference and a contrast difference corresponding to the illumination fluctuation, the influence of the illumination fluctuation is suppressed in the LBP image.

  Returning to FIG. 1, the switching unit 14 registers the LBP of each face image obtained as described above as a referenced face image or compares it with a reference face image already registered as a reference face image. Switch whether to calculate the similarity score. The LBP of the referenced face image is registered in advance in the database (DB) by the registration unit 15 together with a name and identification information that can identify the person. The similarity calculation unit 17 compares the LBP of the reference face image with the LBP of each referenced face image registered in the DB 16, and calculates a similarity score based on the similarity between the two.

  FIG. 6 is a diagram schematically showing a similarity calculation method by the similarity calculation unit 17. As a general method for determining the similarity between images, a method is known in which the distance between pixels at the same position in an image is measured and a square error is used as a similarity score. However, when the feature amount of each pixel is represented by LBP, since LBP is a value obtained from the relationship with adjacent pixels, it is meaningless to see the similarity of LBP in pixel units.

  For example, if only the magnitude relationship between the pixel of interest and the surrounding pixel on the left side is different, the LBP is (0000001) = 1, and the distance from LBP (00000000) = 0 when all the magnitude relationships match is small. . However, even if only one neighboring pixel has a different magnitude relationship, if only the magnitude relationship between the pixel of interest and the upper left neighboring pixel is different, the LBP is (10000000) = 128, and all magnitude relationships match. The distance from LBP (00000000) = 0 becomes enormous.

  As described above, when the feature amount of each pixel is represented by LBP, an LBP image in which the influence of illumination variation is suppressed can be obtained, but it is difficult to apply it to face image recognition using an image similarity score as a parameter. Therefore, in the present invention, the LBP of the target pixel Pij of the reference face image D and the LBP of the pixel (corresponding pixel) P′ij at the position corresponding to the target pixel Pij in the reference face image Dk are each corresponding bit. To be compared. Then, “1” is assigned to the bit having the same value and “0” is assigned to the non-matching bit, and the bit sum for one pixel becomes the similarity score of the target pixel Pij.

  In the example shown in FIG. 6, the LBP of the target pixel Pij of the reference face image D is (10001011), the LBP of the corresponding pixel P′ij of the reference face image Dk is (00011010), Since the third, fifth, sixth and seventh bits coincide with each other, the similarity score of the target pixel Pij is “5”. This is repeated for all the pixels of the reference face image D, and the sum ΣCij of the similarity scores of all the pixels becomes the similarity score between the reference face image D and the referenced face image Dk.

  Returning to FIG. 1, the result output unit 18 sorts all the referenced face images Dk based on the similarity score, and outputs the highest-numbered referenced face images as recognition results, or the only one having the highest similarity. Only the referred face image is output as a recognition result.

  FIG. 7 is a flowchart showing a face image recognition procedure using LBP. Here, a case where a face image similar to the reference face image D is detected from a number of referenced face images Dk will be described as an example. To do.

  In step S1, a referenced face image Dk as a comparison target is selected. In step S2, one pixel of the reference face image D is selected as the current pixel of interest Pij. In step S3, the LBP of the pixel of interest Pij is calculated. In step S4, the LBP of the pixel (corresponding pixel) P′ij at the position corresponding to the current pixel of interest Pij is fetched from the DB 16 for the current referenced face image Dk. In step S5, the LBP of the target pixel Pij and the LBP of the corresponding pixel are compared for each corresponding bit.

  If it is determined in step S6 that the values of the respective bits match, the score Cij is incremented in step S7. In step S8, it is determined whether or not the above comparison has been completed for all the bits of the LBP (8 bits in this embodiment). If not completed, the process returns to step S5, and the above-described processes are performed while switching the target bit. Is repeated.

  Thereafter, regarding the current pixel of interest Pij, when the comparison of all the bits of the LBP is completed and the bit sum is obtained, the process proceeds to step S9. In step S9, it is determined whether or not the above-described processing has been completed for all the pixels of the reference face image D. If it has not been completed, the processing returns to step S2 and the above-described processing is repeated while switching the target pixel Pij.

  Thereafter, when the above-described processing is completed for all the pixels of the reference face image D, the process proceeds to step S10, and the sum ΣCij of the scores Cij of all the pixels indicates the similarity between the current face image Dk and the reference face image D. It is registered as a representative similarity score Ck. In step S11, it is determined whether or not the above comparison has been completed for all the referenced face images Dk. If not completed, the process returns to step S1, and the above-described processes are repeated while switching the referenced face image Dk.

  Thereafter, when the above-described processing is completed for all the referenced face images Dk and the similarity Ck with the reference face image D is obtained, the process proceeds to step S12. In step S12, the referenced face image Dk is sorted based on the similarity Ck, and only the higher-order referenced face image is output as a recognition result.

  Next, modifications of the present invention will be described. In the above-described embodiment, for a 3 × 3 pixel block, an 8-bit value based on the magnitude relationship between the target pixel Pij located in the center and the eight neighboring pixels adjacent to the target pixel Pij. As described with reference to FIG. 8, for example, a target pixel Pij located at the center of a 5 × 5 pixel block, and a circle having a radius of 2 pixels from the target pixel Pij, as illustrated in FIG. An 8-bit LBP may be obtained based on the magnitude relationship of the feature amounts with the eight peripheral pixels P1 to P8 arranged at equal intervals in the circumferential direction. At this time, for the virtual pixels straddling the four pixels, such as the peripheral pixel P1 (and P3, P5, P7), the average value of the feature values of the four pixels P11, P12, P13, and P14 is set as the peripheral pixel. It can be adopted as a feature quantity of P1.

  Then, a similarity score is calculated using each LBP obtained for each pixel block having a different size, and only a referenced face image having a higher similarity score is output as a recognition result regardless of the size of the pixel block. You may do it.

  In the above-described embodiment, the similarity calculation unit 17 compares the LBP of the target pixel Pij of the reference face image D and the LBP of the corresponding pixel P′ij of the referenced face image Dk for each corresponding bit. The number of matching bits Cij is calculated for each pixel, and the sum ΣCij for all the pixels is described as the similarity score Ck between the reference face image D and the referenced image Dk, but the present invention is not limited to this. Instead, for each target pixel Pij of the reference face image D, it is determined whether or not the number of matching bits Cij is equal to or greater than a predetermined threshold Cref. The total number may be the similarity score Ck.

  Further, in the above-described embodiment, it has been described that the similarity is calculated based on the LBP of the face image normalized to the size of 40 × 50 pixels by the normalization unit 12, but the present invention is only this As shown schematically in FIG. 9, a plurality of normalized images having different numbers of pixels are obtained for both the reference image D and the referenced image Dk, and similar for each normalized image. A score may be calculated, and only a referenced face image with a higher similarity score may be output as a recognition result regardless of the size of the normalized image.

  However, since the similarity score Ck depends on the area of the normalized image, the similarity score Ck1 normalized by multiplying by a coefficient corresponding to the area ratio other than the similarity score obtained from the normalized image of 40 × 50 pixels. , Ck2, Ck3, and Ck4, and the normalized similarity scores Ck1, Ck2, Ck3, and Ck4 are preferably compared with each other.

  Furthermore, in the above-described embodiment, the present invention is applied to face image recognition. However, the present invention can be similarly applied to, for example, object recognition, texture recognition, or image recognition.

It is a block diagram of a face image recognition system to which the present invention is applied. It is the figure which showed notionally the organ detection method using a circular ring filter. It is a figure explaining the normalization method of a face image. It is the figure which demonstrated LBP notionally. It is the figure which showed an example of the LBP image. It is the figure which expressed the similarity calculation method typically. It is the flowchart which showed the face image recognition procedure using LBP. It is FIG. (1) explaining the modification of this invention. It is FIG. (2) explaining the modification of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Face image detection part, 11 ... Reference point detection part, 12 ... Normalization part, 13 ... Adjacent relative value sequence conversion part, 14 ... Switching part, 15 ... Registration part, 16 ... Database (DB), 17 ... Similarity degree Calculation unit, 18 ... result output unit

Claims (8)

In an image recognition method for comparing a reference image with a referenced image and calculating a similarity score between the two,
For each pixel of each image, a procedure for obtaining a relative value sequence in which relative values of feature amounts of the pixel and a plurality of peripheral pixels are arranged according to the positions of the peripheral pixels;
A procedure for comparing relative value sequences of corresponding pixels of the reference image and the referenced image for each corresponding relative value, and calculating a similarity between the relative value sequences;
An image recognition method, comprising: calculating a similarity score between a reference image and a referenced image based on a similarity between relative value sequences of each pixel. In an image recognition device that compares a reference image with a referenced image and calculates a similarity score between the two,
Means for obtaining a relative value sequence in which relative values of feature amounts of the pixel and a plurality of peripheral pixels are arranged in accordance with positions of the peripheral pixels for each pixel of each image;
Means for comparing the relative value sequences of corresponding pixels of the reference image and the referenced image for each corresponding relative value, and calculating a similarity between the relative value sequences;
An image recognition apparatus comprising: means for calculating a similarity score between a reference image and a referenced image based on a similarity between relative value sequences of each pixel. The means for calculating the similarity of the relative value sequences compares the relative value sequences for each corresponding relative value, and obtains the number of matching relative values for each pixel,
3. The image recognition apparatus according to claim 2, wherein the means for calculating the similarity score uses the sum of the number of matches of the relative values as a similarity score. The means for calculating the similarity of the relative value sequences compares the relative value sequences for each corresponding relative value, and obtains the number of matching relative values for each pixel,
The means for calculating the similarity score determines whether or not the number of matches of the relative values exceeds a predetermined reference value, and sets the number of pixels for which the number of matches of the relative values exceeds a predetermined reference value as a similarity score The image recognition apparatus according to claim 2. The means for obtaining the relative value sequence obtains a plurality of relative value sequences having different peripheral pixel ranges for each pixel of the reference image and the referenced image,
The means for calculating the degree of similarity of the relative value sequence includes, for each relative value sequence having a different range of the peripheral pixels, for each corresponding relative value, a relative value sequence of corresponding pixels of the reference image and the referenced image. Compare and calculate the similarity of each relative value column,
The means for calculating the similarity score calculates a similarity score between the reference image and the referenced image based on the similarity between the relative value sequences of each pixel. The image recognition apparatus described. A plurality of normalization means for normalizing the reference image and the referenced image to the same number of pixels;
Each normalizing means normalizes the reference image and the referenced image to different numbers of pixels,
The means for obtaining the relative value sequence obtains, for each pixel of each normalized image, a relative value sequence in which the relative values of the feature amounts of the pixel and the plurality of peripheral pixels are arranged according to the positions of the peripheral pixels,
The means for calculating the similarity of the relative value sequence compares the relative value sequences of the corresponding pixels of the reference image and the referenced image for each corresponding relative value for each normalized image, Calculate the similarity between each other,
The means for calculating the similarity score includes means for calculating a similarity score between the reference image and the referenced image based on the degree of similarity of the relative value sequence of each pixel for each normalized image. The image recognition apparatus according to claim 2.   7. The image recognition apparatus according to claim 2, wherein the relative value sequence is an LBP (Local Binary Pattern). The reference image and the referenced image are face images;
Face detection means for detecting a face region from the input face image;
Reference point detection means for detecting a plurality of organs as reference points from the face region;
Normalizing means for normalizing the size of the face image based on the distance between the reference points,
The image recognition apparatus according to claim 2, wherein the means for obtaining the relative value sequence obtains a relative value sequence for each pixel of each normalized face image.