JP2006031388A - Image similarity computation device, image recognition device using the computation device, image siimilarity computation method, image recognition method using the computation method, image-collating computer program and recording medium to which the program is recorded - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image similarity computation device and an image similarity computation method which can precisely compute the similarity of a pair of images, and also provide an image recognition device and an image recognition method which use the computation device and the computation method. <P>SOLUTION: A characteristic area setting part 45 sets T pieces of characteristic area A<SB>j</SB>(1≤j≤T) in common in a first image I<SP>1</SP>and a second image I<SP>2</SP>. For each of T pieces of the characteristic area A<SB>j</SB>, a first and second characteristic quantity computation parts 46 and 47 compute an area value x<SP>1</SP><SB>k</SB>, and x<SP>2</SP><SB>k</SB>of p<SB>j</SB>pieces of partial area a<SB>k</SB>(1≤k≤p<SB>j</SB>) in the characteristic area A<SB>j</SB>of the first and second images I<SP>1</SP>and I<SP>2</SP>. Based on the area values, a correlation value computation part 48 computes a correlation value (1) in the characteristic area A<SB>j</SB>of the first and second images I<SP>1</SP>and I<SP>2</SP>. The correlation value C<SB>j</SB>is compared with a threshold value θ<SB>j</SB>in a comparison part 49. The result of the comparison is added by multiplying a weighting coefficient w<SB>j</SB>by means of a weighting addition part 50 to find the similarity. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to, for example, an image similarity calculation device applicable to face image and other image recognition, an image recognition device using the same, an image similarity calculation method, an image recognition method using the image recognition method, an image verification computer program, and The present invention relates to a recording medium on which it is recorded.

  When restricting access to specific areas, personal authentication is performed at the entrance. One example of an apparatus conventionally used for personal authentication is a card reader. In this case, an identification card such as a magnetic card is distributed in advance to those who are permitted to enter. A person who intends to enter the restricted access area causes the card reader arranged at the entrance to read the identification information recorded on the identification card. If the read identification information matches the identification information registered in advance, entry is permitted and the gate installed at the entrance is opened or the door lock is unlocked.

However, in such a system, if a person forgets to carry a card or loses a card, even a person who is permitted to enter cannot enter. Another problem is that security is easily broken due to theft of the card.
Another example of a conventional device for personal authentication is a password input device. In this case, a password is taught in advance to those who are permitted to enter. A person who wants to enter the restricted access area inputs a password from a password input device arranged at the entrance. If the input result matches a pre-registered password, entry is permitted and the gate installed at the entrance is opened or the door lock is unlocked.

With this system, there is no risk of forgetting or losing the card. However, if a person who is allowed access has forgotten his password, that person will not be able to enter. Also, if a password is leaked, security is broken.
Recently, personal authentication technology using human biometric information (biometric data) has been actively developed. If personal authentication using biometric information is used, all of the above-mentioned problems are solved.

  Personal authentication techniques using biometric information include those using fingerprints, those using irises, and those using facial images. However, when using a fingerprint, it is necessary to register the fingerprint in advance. Therefore, there are many users who have a sense of resistance. In addition, for personal authentication using an iris, the user must position his eyes at a position in close proximity to the camera at the time of authentication. Therefore, the user is strongly aware of the personal authentication procedure and feels annoyed. Personal authentication using face images does not have such disadvantages, and there is a possibility that a personal authentication system with less resistance can be constructed.

  For personal authentication using a face image, for example, an image recognition device configured by a computer system is used to recognize the face image (Patent Document 1 and Patent Document 2). This image recognition apparatus includes collation data storage means in which image data of a face image of a person permitted to enter a restricted access area or feature data of the image data is registered in advance as collation data. Further, the image recognition device includes an imaging device that captures a face image of a person to be recognized and converts the image into image data, a collation unit that collates image data output by the imaging device and registration data in the collation data storage unit, And determining means for outputting a recognition result based on the collation result by the collating means.

The collation means calculates the similarity between the face image captured by the imaging device and the face image corresponding to the collation data registered in the collation data storage means. The determination means compares the calculated similarity with a predetermined recognition threshold value, and if the similarity exceeds the recognition threshold value, the person to be recognized is one of the persons registered in the collation data storage means. Outputs the result of determination to match.
The prior art for collating face images is shown, for example, in Non-Patent Document 1 below. In the prior art disclosed in Non-Patent Document 1, a plurality of filters previously selected by learning using a boosting algorithm are used. The plurality of filters set filter regions in the face image that are different in shape, size, or position. The filter area is divided into two partial areas, and the sum of the pixel values in each partial area is obtained. Then, the difference between the pixel value sums of the two partial areas is obtained as a filter response value. If this filter response value is approximated between the face image to be matched and the registered face image, it can be determined that they are likely to be image pairs of the same person, and if the filter response value is not approximated It can be determined that there is a high possibility that it is an image pair of another person.

A plurality of such filters are obtained by boosting, and the similarity between the image pairs is obtained by weighting and adding the identification results (weak hypotheses) using the individual filters. If the similarity is higher than a predetermined threshold value, it is determined that the image pair belongs to the same person. If the similarity is equal to or less than the threshold value, the image pair belongs to another person. To be judged.
JP 2003-346149 A JP 2001-266151 A Michael J. Jones et al., “Face Recognition Using Boosted Local Features”, 2003, MITSUBISHI ELECTRIC RESEARCH LABORATORIES, [Search June 28, 2004], Internet <URL: http://www.merl.com/reports/ docs / TR2003-25.pdf> Yoav Freund et al., “A decision-theoretic generalization of on-line learning and an application to boosting”, Journal of Computer and System Sciences, 1997, p.119-139.

However, in the above-described prior art for recognizing an image using a difference in filter response values as a feature amount, there are cases where the similarity (likeness of the same person) cannot always be detected accurately. Moreover, the robustness with respect to fluctuation factors such as the illumination conditions at the time of imaging of the recognition target person, the posture of the recognition target person, and their facial expressions is not necessarily sufficient.
For example, regarding the pair of images I ¹ and I ² of the same person, the pixel values at a plurality of locations in the first and second regions P ₁ and P ₂ set in common in these images are as shown in FIG. Suppose that it takes a large value. In this case, the image in the first region P ₁ includes a large change, whereas the image in the second region P ₂ includes only a minute change such as noise. That is, the first region P ₁ clearly contains more important information.

However, the difference between the filter response values in the first region P ₁ between the images I ¹ and I ² is | (6-1) − (7-0) | = 2, and the filter response value in the second region P ₂ . The difference between the images I ¹ and I ² is | (1-0) − (1-1) | = 1. The filter response value has the meaning that the smaller the absolute value, the more likely it is to be the same person. Therefore, in the learning process, the second region P ₂ is selected as the filter region without selecting the first region P _1. .

As a result, it is not always possible to accurately calculate the similarity, and face image recognition processing that is sensitive to fluctuations in noise or the like may be performed, and robustness against fluctuations may be insufficient.
Accordingly, an object of the present invention is to provide an image similarity calculation device and an image similarity calculation method capable of accurately calculating the similarity between a pair of images, and an image recognition device and an image recognition method using them. It is.

  Another object of the present invention is to provide a computer program capable of operating a computer as a device capable of accurately performing a pair of image collation and a recording medium on which the computer program is recorded.

In order to achieve the above object, the invention according to claim 1 is an apparatus for obtaining the similarity between the first image I ¹ and the second image I ² , wherein the first image I ¹ and the second image I 2 are obtained. For each of T (T is a natural number greater than or equal to 2) feature regions A _j (j is a natural number of 1 ≦ j ≦ T) that can be set in common in I ² , the feature region A in the first image I ¹ p _j number (p _j is a natural number of 2 or more) partial area a _k (k is 1 ≦ k ≦ p _j is a natural number) of domain values x ¹ _k and the feature region of the second in the image I ² in the in the _j based on the area value x ² _k of p _j-number of partial areas a _k corresponding in the a _j, the correlation value of the first image I ¹ and the second image I ² of the feature region a _j
Based on the correlation value calculation means for calculating the correlation value C _j for the T feature regions A _j obtained by the correlation value calculation means, the similarity between the first image I ¹ and the second image I ² An image similarity calculation device including correlation-based similarity calculation means for calculating.

According to this configuration, the correlation value between the first and second image I ^1, the first and second images I relating to the T of feature areas A ₁ to A _T set in common in I ^{2 1,} I ² C ₁ -C first and second image I on the basis of _T ^1, the similarity of the I ² is calculated.
For example, consider a case where the correlation values of the first and second regions P ₁ and P ₂ are obtained for the image pair as shown in FIG. The correlation value for the first region P ₁ is 1 × 0 + 6 × 7 = 43, and the correlation value for the second region P ₂ is 0 × 1 + 1 × 1 = 1. The larger the correlation value, the more likely it is that the person is the same person. Therefore, for example, if the feature region A _j is selected by learning, the first region P ₁ is preferentially selected.

In this way, by using a correlation value, when a feature region is determined by learning, a region containing more important information can be selected as a feature region, and the priority of a region containing only noise-level information is reduced. Can do. Therefore, it is possible to obtain an image similarity calculation device that can accurately determine the similarity and is robust against fluctuations (robust).
“Similarity” is a probability that the first and second images I ¹ and I 2 are images of the same object (likeness of the same object), or a probability of being an image of the same kind of object (likeness of the same kind of object). Say. Further, the “region value” may be an average value of pixel values in the region, or may be a total sum of pixel values in the region. In general, this region value corresponds to the luminance of the partial region.

The T feature regions are preferably different in at least one of size, shape, position, and partial region. The partial region a _k is preferably a region obtained by dividing the feature region A _j into p _j (for example, equally divided into p _j ). In this case, if the feature area A _j is divided differently, it is preferably handled as a different feature area even if the size, shape, and position are the same.

The coefficient α _j may be defined as α _j = β / p _j (β is a constant. For example, β = 1). However, when p _j takes the same value for the _T feature regions A _{1 to} A _T (that is, p ₁ = p ₂ =... = P _T ), α _j = γ (γ is a constant) Can be determined.
The invention according to claim 2 is an apparatus for determining the similarity between the first image I ¹ and the second image I ^2, and is T (T is a natural number of 2 or more) feature regions A _j (j is A natural number of 1 ≦ j ≦ T) in common in the first image I ¹ and the second image I ² , and T feature regions A _j set by the feature region setting unit , P _j (p _j is a natural number of 2 or more) partial areas a _k (k is a natural number of 1 ≦ k ≦ p _j ) in the feature area A _j in the first image I ^1. Based on x ¹ _k and the region value x ² _k of the corresponding p _j partial regions a _k in the feature region A _j in the second image I ² , the first image I ¹ and the second image I ² correlation values in the feature area A _j
Based on the correlation value calculation means for calculating the correlation value C _j for the T feature regions A _j obtained by the correlation value calculation means, the similarity between the first image I ¹ and the second image I ² An image similarity calculation device including correlation-based similarity calculation means for calculating.

With this configuration, the same effect as that of the first aspect of the invention can be realized.
In the present invention, the feature areas A _{1 to} A _T are set in common to the first and second images I ¹ and I ² by the feature area setting means, and the first and second images I for each feature area A _j are set. A correlation value C _j between ¹ and I ² is obtained. Therefore, for example, the first image I ¹ may be registered in advance, and the image data itself of the first image I ¹ may be registered when collating with the second image I ² . That is, it is possible to reduce the prior processing.

According to a third aspect of the invention, a device for obtaining the first image I ¹ and the similarity between the second image I ^2, can be set in common to the first image I ¹ and the second in the image I ² p _j number (p in such respect to T (T is a natural number of 2 or more) each feature region a _j of the (j is a natural number of 1 ≦ j ≦ T), characterized in the area a _j of the first in the image I ¹ Collation data in which a region value x ¹ _k of a partial region a _k (k is a natural number of 1 ≦ k ≦ p _j ) of _j is a natural number of 2 or more is stored as collation data representing the feature amount of the first image I ¹ storage means, the T number of feature areas a _j, and the feature region setting means for setting the second in the image I ^2, for each of the set T number of feature areas a _j this feature region setting means, the second image p in I ² in the feature region a _j of the _j partial area a _k region value x wherein the ² _k second image I A feature value calculating means for calculating the feature value of ² and the region value x ¹ _k of the first image I ¹ stored in the matching data storage means for each of the T feature areas A _j , and the feature quantity Based on the region value x ² _k of the second image I ² calculated by the calculating means, the correlation value in the feature region A _{j of} the first image I ¹ and the second image I ²
Based on the correlation value calculation means for calculating the correlation value C _j for the T feature regions A _j obtained by the correlation value calculation means, the similarity between the first image I ¹ and the second image I ² An image similarity calculation device including correlation-based similarity calculation means for calculating.

With this configuration, the same effect as that of the first aspect of the invention can be realized.
In the present invention, for the first image I ¹ , the region value x ¹ _k of each partial region a _k of the feature regions A _{1 to} A _T is calculated in advance as collation data and stored in the collation data storage means. . Therefore, it is possible to reduce the amount of calculation when calculating the degree of similarity between the first and second images I ¹ and I ² , and the degree of similarity can be calculated in a short time. Accordingly, it is possible to shorten the processing time when the first image I ¹ is registered in advance and the second image I ² is input to the apparatus to obtain the similarity.

According to a fourth aspect of the present invention, the correlation-based similarity calculating means determines the correlation value C _j obtained by the correlation value calculating means for each predetermined threshold value θ _j (preferably for each feature region A _j. If the correlation value C _j exceeds the threshold θ _j , the first image I ¹ and the second image I ² are similar (more specifically, A first weak discrimination value h _j (I ¹ , I ² ) = H ₁ representing the same target image or the same target image), and the correlation value C _j is less than or equal to the threshold θ _j If there is, the second weak identification value h _j indicating that the first and second images I ¹ and I ² are dissimilar (more specifically, they are not the same target image or the same target image). ^{(I 1, I 2) =} H 2 ( however, H _1> H ₂₎ and comparing means for outputting the output of said comparison means for said T number of feature areas a _j h _j The (I ^1, I ^2), the weighting coefficient w _j (preferably coefficients determined for each individual feature areas A _j) by a linear combination by multiplying the first image I ¹ and the second image I Similarity of ²
4. The image similarity calculation device according to claim 1, further comprising weighted addition means for calculating

According to this configuration, a plurality of characteristic regions A ₁ to A _T plurality of correlation values C ₁ -C _T determined respectively for a predetermined threshold value theta ₁ through? _T and comparing means for comparing each provided Yes. Then, the weak identification values h ₁ (I ¹ , I ² ) to h _T (I ¹ , I ² ) representing the comparison results of the plurality of correlation values C _{1 to} C _T are weighted and added to obtain the first and The similarity between the second images I ¹ and I ² is obtained.
The weighting factors w _{1 to} w _T respectively multiplied by the weak identification values h ₁ (I ¹ , I ² ) to h _T (I ¹ , I ² ) corresponding to the feature regions A _{1 to} A _T are obtained by learning, for example. be able to. As a result, the weaker identification value of the feature region that is effective for the similarity / dissimilarity determination of the first and second images I ¹ and I ² can be weighted more heavily, and therefore the similarity can be accurately obtained.

The invention according to claim 5 is characterized in that the partial area a _k is an area obtained by dividing the first image I ¹ and the second image I ² into a characteristic area A _j only in a predetermined direction. An image similarity calculation device according to any one of claims 1 to 4.
According to this configuration, the partial area a _k is determined by dividing the characteristic area A _j with respect to a predetermined direction, and a correlation value (unidirectional correlation value) regarding such a partial area a _k is obtained. Can be made more robust. This is because image features can be easily extracted by dividing the image one-dimensionally. More specifically, the image is compared with the case where the region of the image is divided into two orthogonal directions (x direction and y direction) (that is, divided into a lattice shape) and the features of the image are extracted. Extract features by dividing only in the first direction (x direction) and then extract features by dividing the image only in the second direction (y direction). can do. That is, the feature of the image can be extracted by decomposing it into an x-direction component and a y-direction component.

The partial area a _k may be an area obtained by dividing the feature area A _j by a dividing line (straight line) orthogonal to the predetermined one direction.
According to a sixth aspect of the present invention, if the image similarity calculation device according to any one of the first to fifth aspects and the similarity calculated by the image similarity calculation device exceed a predetermined recognition threshold value. The first image I ¹ and the second image I ² are determined to be similar (more specifically, they are images of the same target or the same type of target) and calculated by the image similarity calculation device. If the similarity is equal to or less than the recognition threshold value, it is determined that the first image I ¹ and the second image I ² are dissimilar (more specifically, they are not images of the same object or similar objects). And an image recognition device characterized by including a determination means.

According to this configuration, the similarity between the first and second images I ¹ and I ² (specifically, whether the images are the same object or the same object) is determined based on the accurately calculated similarity. Can be determined. Moreover, the configuration for calculating the similarity is robust against fluctuation factors. Therefore, it is possible to realize an image recognition apparatus that can accurately recognize an image and is robust against a variation factor.
The recognition object for image recognition may be an object whose appearance changes itself. Examples of changes in appearance include changes in the shape of the object itself (dynamic changes and changes over time), apparent changes depending on illuminance, and changes due to the presence or absence of an accessory (such as glasses or makeup). Examples of such recognition objects include animal (especially human) faces and animal physiques. “Form” means a concept including shape, pattern or color, or any combination thereof.

In addition, the recognition target object may be the same kind of object having various appearances (an object having a different form or the like within a certain range). Examples of such recognition objects include fruits, empty bottles, automobiles, machine parts, and ground objects (such as buildings) viewed from an aircraft.
The image data of the processing target image may be input from an imaging device (camera) that captures the recognition target and outputs the image data, or reproduces the image data recorded on a recording medium such as a memory, an optical disk, or the like. It may be input from the image reproduction device or acquired by communication.

The invention according to claim 7 is a method for determining the similarity between the first image I ¹ and the second image I ² , wherein T (T is a natural number of 2 or more) feature regions A _j (j is A first feature region setting step of setting 1 ≦ j ≦ T) in the first image I ¹ and a second feature of setting the T feature regions A _j in the second image I ^2. a region setting step, for each of the first T-number set in the image I ^1, wherein the area a _j, p _j number in the feature region a _j of the first in the image I ¹ (p _j is 2 or more A first region value calculating step for calculating a region value x ¹ _k of a partial region a _k (k is a natural number of 1 ≦ k ≦ p _j ) and T features set in the second image I ² for each region a _j, second territory for computing the area value x ² _k of p _j-number of partial areas a _k in the characteristic region a _j of the second in the image I ² Value calculating step, wherein the T for each characteristic region A _j, p _j-number of partial areas a _k region values x ¹ _k and the second image in the feature area A _j of the first in the image I ¹ based on the area value x ² _k corresponding p _j number of partial areas a _k within the feature region a _j in I ^2, the first image I ¹ and the second image I ² the feature region a _j Correlation value
A correlation value calculating step for calculating the similarity between the first image I ¹ and the second image I ² based on the correlation values C _j for the T feature regions A _j obtained An image similarity calculation method comprising: a degree calculation step.

With this method, the similarity between the first and second images I ¹ and I ² can be accurately obtained by using the correlation between the images.
Invention of claim 8, compared the correlation-based similarity computing step, the correlation value C _j obtained by said correlation value calculation step with respect to the feature region A _j with a predetermined threshold theta _j, the correlation value If C _j exceeds the threshold value θ _j , the first weak identification value h _j (I ¹ , I ² ) = representing that the first image I ¹ and the second image I ² are similar. If H ₁ is an identification value related to the feature region A _j and the correlation value C _j is equal to or less than the threshold value θ _j, it indicates that the first and second images I ¹ and I ² are dissimilar. the second weak identification value _{^{h j (I 1, I 2}} ) = a comparison step of H ₂ and the identification value relating to the feature region a _j, the T number of feature areas a _j the identification value h _j for (I ¹ , I ² ) are multiplied by a weighting coefficient w _j and linearly combined to obtain a similarity between the first image I ¹ and the second image I ² .
The image similarity calculation method according to claim 7, further comprising a weighted addition step of calculating.

According to this configuration, the accurate similarity is determined by appropriately determining the coefficient w _j so that the feature region effective for the similarity / dissimilarity determination of the first and second images I ¹ and I ² is weighted more appropriately. Can be requested.
The invention according to claim 9 is characterized in that the partial area a _k is an area obtained by dividing the first image I ¹ and the second image I ² by dividing the feature area A _j only in a predetermined direction. The image similarity calculation method according to claim 7 or 8.

According to this method, since the similarity is calculated based on the unidirectional correlation value, it is possible to calculate the similarity that is robust against the variation factor.
According to a tenth aspect of the present invention, the similarity calculation step by the image similarity calculation method according to any one of the seventh to ninth aspects, and the similarity calculated by the similarity calculation step has a predetermined recognition threshold value. If it exceeds, it is determined that the first image I ¹ and the second image I ² are similar, and if the similarity calculated by the image similarity calculation device is equal to or less than the recognition threshold value, The image recognition method includes a determination step of determining that the first image I ¹ and the second image I ² are dissimilar.

By this method, since the similarity obtained by the similarity calculation process that is accurate and robust against the variation factor is used, the similarity / dissimilarity between the first and second images I ¹ and I ² is accurately determined. It is possible to reduce the influence of fluctuation factors.
The invention according to claim 11 is a computer program for operating a computer as a device for collating the first image I ¹ and the second image I ² , wherein the computer program With respect to each of T (T is a natural number of 2 or more) feature regions A _j (j is a natural number of 1 ≦ j ≦ T) that can be set in common in one image I ¹ and second image I ² , the first The region values x ¹ _k of the p _j (p _j is a natural number of 2 or more) partial regions a _k (k is a natural number of 1 ≦ k ≦ p _j ) in the feature region A _j in the image I ¹ and the second based on the area value x ² _k of p _j-number of partial areas a _k corresponding within the feature region a _j in the image I ^2, the first image I ¹ and a second image I ² of the feature region a _j Correlation value in
And a similarity degree between the first image I ¹ and the second image I ² based on the correlation value C _j regarding the T feature regions A _j obtained by the correlation value calculation means. It is a computer program characterized by functioning as a correlation base similarity calculation means for calculating.

By executing the computer program on a computer, the image similarity calculation device according to claim 1 can be configured.
The invention described in claim 12 is a computer program for operating a computer as a device for collating the first image I ¹ and the second image I ² , wherein the computer program includes T computers. Feature region setting means for commonly setting (T is a natural number of 2 or more) feature regions A _j (j is a natural number of 1 ≦ j ≦ T) in the first image I ¹ and the second image I ² , For each of the T feature regions A _j set by the feature region setting means, p _j (p _j is a natural number of 2 or more) partial regions a _k in the feature region A _j in the first image I ^1. (k is 1 ≦ k ≦ p natural number _j) domain values x ² _k corresponding p _j number of partial areas a _k in the region value x ¹ _k and the second the features in the image I ² in the area a _j of Based on the first image I ¹ and the second image I ² Correlation value in feature area A _j
And a similarity degree between the first image I ¹ and the second image I ² based on the correlation value C _j regarding the T feature regions A _j obtained by the correlation value calculation means. It is a computer program characterized by functioning as a correlation base similarity calculation means for calculating.

By executing the computer program on a computer, the image similarity calculation device according to claim 2 can be configured.
The invention according to claim 13 is a computer program for operating a computer as a device for collating the first image I ¹ and the second image I ² , the computer comprising the first image I ¹ and for each feature region a _j of the common settable to T (T is a natural number of 2 or more) (j is a natural number of 1 ≦ j ≦ T) during the second image I ^2, of the first in the image I ¹ The region value x ¹ _k of p _j (p _j is a natural number of 2 or more) partial regions a _k (k is a natural number of 1 ≦ k ≦ p _j ) in the feature region A _j is _obtained from the first image I ¹ . Collation data storage means stored as collation data representing feature quantities, wherein the computer program sets the T feature areas A _j in the second image I ² ; T pieces set by the feature region setting means For each feature region A _j of calculating an area value x ² _k of p _j-number of partial areas a _k in the second the features in the image I ² in the area A _j as the feature quantity of the second image I ² For each of the T feature areas A _j , the area value x ¹ _k of the first image I ¹ stored in the collation data storage means and the feature value calculation means calculated by the feature quantity calculation means 2 on the basis of the area value x ² _k of the image I ^2, the correlation value of the first image I ¹ and the second image I ² of the feature region a _j
And a similarity degree between the first image I ¹ and the second image I ² based on the correlation value C _j regarding the T feature regions A _j obtained by the correlation value calculation means. It is a computer program characterized by functioning as a correlation base similarity calculation means for calculating.

By executing this computer program on a computer, the image similarity calculation device of claim 3 can be configured.
In the fourteenth aspect of the invention, the correlation base similarity calculation means compares the correlation value C _j obtained by the correlation value calculation means with a predetermined threshold value θ _j, and the correlation value C _j If the threshold value θ _j is exceeded, a first weak discrimination value h _j (I ¹ , I ² ) = H ₁ indicating that the first image I ¹ and the second image I ² are similar is output. If the correlation value C _j is less than or equal to the threshold value θ _j , a second weak identification value h _j (I ¹ , I 2) indicating that the first and second images I ¹ and I ² are dissimilar. ² ) = H ₂ (where H ₁ > H ₂ ) and the output value h _j (I ¹ , I ² ) of the comparison means for the T feature regions A _j are weighted coefficients The similarity between the first image I ¹ and the second image I ² by multiplying w _j and linearly combining them
The computer program according to claim 11, further comprising weighted addition means for calculating.

By executing this computer program on a computer, the image similarity calculation device of claim 4 can be configured.
The invention according to claim 15 is characterized in that the partial area a _k is an area obtained by dividing the first image I ¹ and the second image I ² by dividing the feature area A _j only in a predetermined direction. A computer program according to any one of claims 11 to 14.

By executing this computer program on a computer, the image similarity calculation device of claim 5 can be configured.
According to the sixteenth aspect of the present invention, when the similarity calculated by the weighted addition means exceeds a predetermined recognition threshold, the first image I ¹ and the second image I ² If the similarity calculated by the image similarity calculation device is equal to or less than the recognition threshold value, the first image I ¹ and the second image I ² are dissimilar. The computer program according to claim 11, wherein the computer program is made to function as a determination unit that determines that

By executing this computer program on a computer, the image recognition apparatus of claim 6 can be configured.
The invention described in claim 17 is a computer-readable recording medium in which the computer program according to any one of claims 11 to 16 is recorded.
By causing a computer to execute the computer program recorded on the recording medium, the computer can function as the image recognition apparatus as described above.

  The computer program may be recorded on the recording medium in any format, for example, may be recorded in a compressed format. As the recording medium, any recording format including a magnetic recording medium, an optical recording medium, and a magneto-optical recording medium can be applied.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a schematic block diagram showing a configuration of a personal authentication system according to an embodiment of the present invention. This personal authentication system is used, for example, for entrance management to a specific area where entry is restricted. This personal authentication system is arranged at the entrance to the restricted access area, and has a camera 1 as an imaging device that captures a face image of a person appearing at the entrance, and a computer connected to the camera 1 via a cable 2. 3, a display 4 disposed at the entrance, and a gate switch 7 that opens and closes a gate installed at the entrance (or unlocks / locks the door lock).

  The computer 3 is connected to a display 5 as a man-machine interface and an input operation unit 6 (for example, a keyboard and a mouse). The display device 4 may be a two-dimensional display device that can display a message in response to a command signal from the computer 3, for example, and includes a plurality of color LED lamps controlled by the computer 3. It may be a lamp display device. The gate switch 7 opens and closes the gate (or unlocks / locks the door lock) in accordance with a control signal from the computer 3 to permit or prohibit entry of a person who appears at the entrance.

  The computer 3 includes a CPU 10, a ROM 11 and a RAM 12, and has a general configuration in which these are connected to a bus 13. The bus 13 further includes a hard disk drive (HDD) 14 as an external storage device, a CD-ROM drive 15 as a recording medium reader capable of reading data recorded on a CD-ROM 8 which is an example of a recording medium, A display controller 16 for controlling the display 5 and an input unit controller 17 (for example, a keyboard controller and a mouse controller) for controlling the input operation unit 6 are connected. The image input interface unit 18 for receiving image data corresponding to the captured image generated from the camera 1, the display control interface unit 19 for outputting a control signal to the display 4, and the control signal to the gate switch 7 Is connected to the bus 13.

  The face image of the person permitted to enter the restricted access area is captured in advance by the camera 1, for example, and the individual verification data corresponding to the face image is a predetermined storage area in the hard disk drive 14 in advance. Registered in the storage unit 25. As will be described later, representative verification data representing individual verification data of each person is also registered in the verification data storage unit 25.

  On the other hand, an image of a person who appears at the entrance is captured by the camera 1. Image data representing the captured image is transmitted to the computer 3, and the person's face image data is extracted from the image data. The extracted face image data is collated with the individual collation data registered in advance in the collation data storage unit 25 of the hard disk drive 14. As a result of this collation, if any face image registered in the form of the individual collation data matches the face image of the person appearing at the entrance, that is, the recognition threshold value determined by the similarity between both face images is set in advance. If it exceeds, the computer 3 opens the gate (or unlocks the door lock) by the gate switch 7 and permits the person to enter.

  A computer program for realizing such an operation by the computer 3 may be provided by being recorded on a CD-ROM 8 which is an example of a recording medium. In this case, by loading the CD-ROM 8 into the CD-ROM drive 15, performing a predetermined installation operation, and installing the computer program in the computer 3, the computer 3 can realize the functions as described above. Become.

  When the installed computer program is executed on the computer 3, for example, a screen as shown in FIG. The computer 3 executing the computer program operates as an image recognition device, extracts a face image from a registration mode for registering a face image of a person permitted to enter, and an image captured by the camera 1. It operates in one of the recognition modes for performing recognition processing.

  When registering individual verification data of a face image of a person permitted to enter the restricted access area in the verification data storage unit 25, the operator operates the input operation unit 6 to operate the computer 3 in the registration mode. Let In this state, a person who is permitted to enter is imaged by the camera 1. At this time, the image data generated from the camera 1 is transmitted to the computer 3 and stored in the image memory 12a constituted by a storage area in the RAM 12 (and a virtual memory area in the hard disk drive 14 as necessary). . Further, face image data is extracted from the image data. Individual collation data corresponding to the extracted face image data is registered in the collation data storage unit 25. The individual verification data may be face image data itself or feature amount data representing a predetermined feature amount extracted from the face image data.

  Human face images change in various ways depending on facial expressions, change greatly depending on the presence or absence of accessories such as glasses, and are also affected by imaging conditions (such as illuminance). Therefore, when registering the individual verification data of a person who is allowed to enter, various facial expressions can be taken for the person, and a face image can be taken, and images can be taken with and without accessories. Imaging is performed under imaging conditions. In this way, a plurality of (for example, 50 to 100) face images are captured for each person, and individual matching data corresponding to each face image is registered in the matching data storage unit 25.

Hereinafter, a person who has completed the registration of the individual verification data as described above is referred to as a “registrant”, and a person who has not registered the individual verification data is referred to as an “unregistered person”.
FIG. 3 is a block diagram for explaining a functional configuration of the computer 3 in the execution state of the computer program. The computer 3 substantially includes a plurality of function processing units realized by the CPU 10 cooperating with the ROM 11 and the RAM 12 by causing the CPU 10 to execute the computer program. The plurality of function processing units extract a face image area from the image data generated by the camera 1 to generate face image data, and the face image data extracted by the face image detection unit 30. A face recognition processing unit 40 for collating with individual collation data registered in advance is provided.

  The face image detection unit 30 cuts out a face image area from the input image data, and further outputs face image data normalized in size and inclination (left and right inclination) as face image data to be verified. The face image area to be cut out is an area that is not easily affected by the hairstyle and is not easily affected by the vertical movement of the jaw during speech. More specifically, for example, a rectangular region including the range from the eyebrows to the mouth is included as a face image region including the space between both eyes in the left-right direction and the range from the eyebrows to the mouth in the up-down direction. The normalized face image data corresponds to, for example, a square image of 32 pixels × 32 pixels.

  The face recognition processing unit 40, if a certain degree of agreement is found between the face image data to be collated and any individual collation data, the face of the registrant corresponding to the individual collation data corresponds to the face image data to be collated. Judged to be an image. In this case, it is determined that the person imaged by the camera 1 is a registered person, and the computer 3 opens the gate (or unlocks the door lock) by the gate switch 7 and permits the person to enter. On the other hand, the face recognition processing unit 40 determines that the face image to be checked is the face image of the registrant unless individual matching data that matches a certain degree of matching with the face image data to be checked is registered in the matching data storage unit 25. Judge that there is no. In this case, it is determined that the person imaged by the camera 1 is an unregistered person, and the computer 3 holds the gate in the closed state (or holds the door lock in the locked state) by the gate switch 7. Is prohibited from entering.

  The face recognition processing unit 40 calculates a similarity (similarity of person) between a face image to be verified and a face image corresponding to the verification data (individual verification data or representative verification data), and a verification data storage A collation data selection unit 42 for selecting which collation data to be collated among the collation data registered in the unit 25, and the similarity calculated by the similarity calculation unit 41 as a predetermined recognition threshold value. It has a determination unit 43 (determination means, determination step) for determining whether or not the face image to be checked is a registrant's face image by comparison.

  If the highest value (highest similarity) among the similarities obtained for the plurality of individual verification data exceeds the recognition threshold, the determination unit 43 determines that the person of the face image to be verified is a registrant. It is judged that there is, and recognition result data indicating that is output. The recognition result data may include information (for example, name information and registration number information of the registrant) for specifying the registrant corresponding to the individual matching data with the highest similarity. Receiving the recognition result data, the display unit 4 displays the recognition result. This display may simply be a display indicating that the user is a registrant, or may be a display of specific information (for example, name) of the registrant.

On the other hand, if there is no individual verification data whose similarity exceeds the recognition threshold value, the determination unit 43 determines that the person of the face image to be verified is not a registrant, and outputs identification result data indicating that fact. The In this case, the display 4 displays that the person's registration cannot be confirmed.
The similarity calculating unit 41 is based on the collation data stored in the collation data storage unit 25 and the collation feature dictionary stored in the collation dictionary storage unit 26 configured by the storage area in the hard disk drive 14. The similarity between the face image to be collated and the registered face image (stored in the form of collation data) is calculated. The matching feature dictionary holds data (matching feature data) serving as an index when calculating the similarity between two images in a fixed data structure. The similarity calculation unit 41 extracts a feature amount from the face image data to be collated based on the collation feature data held in the collation feature dictionary, and collates it with the collation data.

  FIG. 4 is an illustrative view for explaining a registration state of collation data in the collation data storage unit 25. FIG. 4 shows an example in which there are m registrants and n individual verification data corresponding to n face images are registered for each registrant. More specifically, registration numbers 1, 2,..., M are assigned to individual registrants, and registrant names (name 1, name 2,. Name m), each representative collation data (representative collation data 1, representative collation data 2,..., Representative collation data m), and each n individual collation data (individual collation data 1-1, 1- 2, ..., 1-n; 2-1, 2-2, ..., 2-n; ...; m-1, m-2, ..., mn) are registered. Of course, the number of individual verification data may differ for each registrant.

The representative collation data of each registrant is collation data representing n pieces of individual collation data corresponding to the registrant. More specifically, the representative collation data of each registrant may be average value data of n pieces of individual collation data registered corresponding to the registrant.
Such representative collation data may be automatically generated by the operation of the CPU 10 or the like and written to the collation data storage unit 25 when the individual collation data is registered in the collation data storage unit 25. The representative collation data is created by an operator's manual operation using registered face image data or individual collation data, and is also written into the collation data storage unit 25 by the operator's manual operation. Also good.

  FIG. 5 is a diagram more intuitively showing the registration state of the collation data in the collation data storage unit 25. In this example, for each of nine registrants A to I, one representative face image data is registered as representative collation data, and each three face image data is registered as individual collation data. ing. As in this example, a plurality of face image data is registered as individual verification data for each registrant, and representative face image data obtained by calculating an average value of the plurality of face image data is registered as representative verification data. Also good. Also, instead of the image data itself, feature amounts used for similarity calculation by the similarity calculation unit 41 are extracted in advance from individual face images, and the feature amounts are registered in the verification data storage unit 25 as individual verification data. Alternatively, the average value of the individual verification data (feature value) for each registrant may be registered in the verification data storage unit 25 as representative verification data.

Thus, the collation data storage unit 25 functions as a collation data storage unit that stores collation data to be collated with the face image data to be collated.
FIG. 6 is a flowchart for explaining an example of processing by the face recognition processing unit 40. When the face image data to be collated is input to the face recognition processing unit 40, the collation data selection unit 42 refers to the collation data storage unit 25 and sequentially selects the representative collation data of the registrant (step S1). The similarity calculation unit 41 calculates the similarity between the representative face image corresponding to the representative collation data selected by the collation data selection unit 42 and the face image to be collated, and the representative similarity storage area 12b (see FIG. 1 and FIG. 3) (step S2). Thus, when the similarity is obtained for the representative collation data of all the registrants registered in the collation data storage unit 25, the collation data selection unit 42 extracts individual collation data to be collated with the face image data to be collated. Processing is executed (step S3). This process is simply a process for excluding the individual collation data of the registrant who is considered not to agree with the face image data to be collated, from the collation target.

  One specific example of the processing content for extracting the individual collation data to be collated is shown in FIG. In this example, the collation data selection unit 42 compares the similarity obtained for the representative collation data with a predetermined rejection threshold (step S11). The matching data selection unit 42 excludes (rejects) the registrant corresponding to the representative matching data whose similarity does not exceed the rejection threshold (rejected) (step S12). That is, a plurality of individual verification data registered in the verification data storage unit 25 for the registrant are collectively rejected. As a result, only the registrant corresponding to the representative collation data whose similarity exceeds the rejection threshold value is subjected to collation (step S13). The collation data selection unit 42 performs the same process for all the representative collation data (step S14), and ends the extraction of the individual collation data to be collated.

  Another example of the process for extracting the individual collation data to be collated is shown in FIG. In this example, the collation data selection unit 42 ranks (sorts) the representative collation data of all registrants stored in the representative similarity storage area 12b (step S15), and registers a predetermined number or a predetermined ratio of the higher rank. For the person, the individual collation data is extracted as a collation target (step S16). The collation data selection unit 42 collectively excludes the plurality of individual collation data of the remaining registrants from the collation target (step S17). In this case as well, if there is a similarity within the upper predetermined number or within the upper predetermined ratio that does not exceed the predetermined rejection threshold, the individual verification data of the registrant corresponding to such similarity May be excluded from verification targets.

  The collation data selection unit 42 sequentially selects the individual collation data of a part of the registrants extracted in this way one by one as the target of the collation calculation by the similarity calculation unit 41 (step S4 in FIG. 6). . The similarity calculation unit 41 calculates the similarity between the face image corresponding to the selected individual verification data and the face image to be verified. Thus, the similarity is calculated for all the individual verification data extracted by the verification data selection unit 42 (step S5). The calculated similarity is written in the individual similarity storage area 12 c in the RAM 12.

  Next, the determination unit 43 refers to the individual similarity storage area 12c and searches for the highest value (highest similarity) of the calculated similarity (step S6). Then, the determination unit 43 compares the retrieved maximum similarity with a predetermined recognition threshold value (step S7). As a result, if the highest similarity exceeds the recognition threshold value (YES in step S6), the determination unit 43 determines that the collated face image is a registrant's face image, and the highest similarity is determined. The registrant corresponding to the individual collation data is identified as the person of the face image to be collated (step S8). On the other hand, when the highest similarity is not more than the recognition threshold value, it is determined that the person of the face image to be verified is not registered (step S9).

  In this way, the individual collation data to be collated can be narrowed down by collating the face image data to be collated with the representative collation data. Thereby, the collation calculation of the face image data to be collated and the individual collation data can be greatly reduced, and the time required for the recognition process can be greatly shortened. Also, when the number of individual verification data per registrant is increased to improve recognition accuracy, the time required for recognition does not increase significantly. Therefore, the face image can be accurately recognized by a short time process.

  FIG. 9 is a flowchart for explaining another example of processing by the face recognition processing unit 40. When the face image data to be collated is input to the face recognition processing unit 40, the collation data selection unit 42 refers to the collation data storage unit 25 and sequentially selects the representative collation data of the registrant (step S21). The similarity calculation unit 41 calculates the similarity between the representative image corresponding to the representative collation data selected by the collation data selection unit 42 and the face image to be collated, and writes it in the representative similarity storage area 12b (step S22).

Thus, when the similarity is obtained for the representative collation data of all registrants registered in the collation data storage unit 25, the collation data selection unit 42 executes a process for extracting individual collation data to be collated. This process is simply a process for excluding the individual collation data of the registrant who is considered not to agree with the face image data to be collated, from the collation target.
More specifically, the collation data selection unit 42 refers to the representative similarity storage area 12b and sorts the representative collation data of all registrants in descending order according to the similarity (step S23). Furthermore, the collation data selection part 42 divides a registrant into a some group according to the order of similarity (step S24). For example, if collation data as in the example of FIG. 5 is registered in the collation data storage unit 25, the three people with the highest similarity are classified into the first group, and the three people with the middle similarity are the first. You may classify | categorize into 2 groups and classify 3 people with low similarity into a 3rd group. Of course, the grouping of registrants based on similarity does not have to be equal among a plurality of groups.

The collation data selection unit 42 further extracts the individual collation data of the registrant of the highest group (for example, the first group described above) as a collation target (step S25), and the other groups (second and second in the above example). Individual collation data of registrants of the third group is provisionally excluded from collation targets (step S26).
The collation data selection unit 42 sequentially selects the individual collation data of the registrants of the highest group extracted in this manner one by one as a collation target (step S27). The similarity calculation unit 41 calculates the similarity between the face image corresponding to the selected individual verification data and the face image to be verified. Thus, the similarity is calculated for all the individual collation data of the registrants in the highest group (step S28). The calculated similarity is written in the individual similarity storage area 12 c in the RAM 12.

  Next, the determination unit 43 refers to the individual similarity storage area 12c and searches for the maximum value (highest similarity) of the calculated similarity (step S29). Then, the retrieved maximum similarity is compared with a predetermined recognition threshold (step S30). As a result, if the maximum similarity exceeds the recognition threshold value (YES in step S30), the determination unit 43 determines that the face image to be verified is a registrant's face image, and determines the highest similarity. The registrant corresponding to the individual verification data is specified as the person of the face image to be verified (step S31).

  On the other hand, when the highest similarity is equal to or less than the recognition threshold value (NO in step S30), the collation data selection unit 42 collates the individual collation data for all registrants registered in the collation data storage unit 25. It is determined whether or not it has been completed (step S32). This determination is equivalent to the determination of whether or not the collation processing has been completed for all the groups in which the registrant is classified according to the similarity of the representative collation data. When the collation process for any registrant is not completed (NO in step S32), the collation data selection unit 42 is the highest group among the groups that have been temporarily excluded from the candidates (that is, The registrant belonging to the next group (for example, the second group in the above example) is set as a candidate for collation, and the processing from step S27 is repeated. That is, the individual verification data of the registrants of the next group are sequentially selected.

Similar to the case described above, the similarity calculation unit 41 calculates the similarity between the face image corresponding to the selected individual verification data and the face image to be verified. Thus, the similarity is calculated for all the collation data of the registrants of the next group (step S28).
Next, the determination unit 43 searches for the highest calculated similarity (the highest similarity) (step S29). Then, the retrieved maximum similarity is compared with the recognition threshold (step S30). As a result, if the maximum similarity exceeds the recognition threshold value, the registrant corresponding to the individual matching data of the similarity is specified as the person of the face image to be compared (step S31).

  In this way, the same processing is performed until individual collation data having a similarity exceeding the recognition threshold is found. If the individual collation data whose similarity is higher than the recognition threshold value is not found even though the collation processing is performed on all the individual collation data stored in the collation data storage unit 25 ( The determination unit 43 determines that the person of the face image to be verified has not been registered (step S34).

In the process shown in FIG. 9, when an appropriate recognition result cannot be obtained, the individual verification data once rejected is restored as a verification target. As a result, it is possible to reliably avoid a situation in which the registrant is not recognized.
In the example of FIG. 9, when it is determined that the person of the face image to be verified is unregistered, the verification process for the individual verification data is performed for all of the registrants. However, when the similarity obtained as a result of matching between the face image data to be matched and the representative matching data is extremely low, the probability that a similarity higher than the recognition threshold value obtained by matching with the individual matching data is low. Therefore, for the group whose similarity calculated for the representative face image data does not reach the predetermined rejection threshold value, the individual collation data of the registrant may not be revived as a collation target.

  In this embodiment, an example is shown in which one representative collation data is registered for one registrant, but a plurality (two or more) of representative collation data are collated for one registrant. It may be registered in the data storage unit 25. For example, in the case of a registrant who uses spectacles, first representative collation data representing individual collation data in the state of wearing spectacles and second representative collation data representing individual collation data in a state of wearing no spectacles. And may be registered. Also in this case, it is only necessary to determine whether or not a corresponding group of individual collation data is to be collated for each representative collation data based on the similarity obtained for the representative collation data.

Next, calculation of the similarity of face images will be described.
As shown in FIG. 10, a case where the similarity between the first face image I ¹ and the second face image I ² whose size and inclination are normalized will be described. The first face image I ¹ is, for example, a registered face image corresponding to the individual collation data stored in the collation data storage unit 25, and the second face image I ² is a face from an image captured by the camera 1, for example. This corresponds to the face image data to be verified extracted by the image detection unit 30.

First, T feature regions A _j (j is a natural number of 1 ≦ j ≦ T) having different sizes, shapes, positions, and at least one of the types described later (T is a natural number of 2 or more), sequentially, It is set in common in the first face image I ¹ and the second face image I ² . The feature area A _j is an area that includes at least two pixels and has the same size as or smaller than the first and second face images I ¹ and I ² . As the shape of the feature region A _j , an arbitrary shape such as a rectangle (including a square), a circle, or a bowl shape can be selected, and even if the shape is different among the plurality of feature regions A _{1 to AT.} Good.

For example, assume that the shape of the feature region is a square, the first and second face images I ¹ and I ² are squares of 32 pixels × 32 pixels, and the feature region is divided into two parts for analysis (ie, Assuming a case where the number of divisions p _j described later is fixed to “2”), the minimum feature area is a square having a size of 2 pixels × 2 pixels, and the maximum feature area is a size of 32 pixels × 32 pixels. In addition, it is allowed only when the size is the even number of pixels (represented by the number of pixels along one side of the square area). However, since it has been empirically known that the feature area of 2 pixels × 2 pixels is too small to be meaningful, the size of the minimum feature area is actually 4 pixels × 4 pixels. The feature region of 4 pixels × 4 pixels can take 29 × 29 different positions within the square region of 32 pixels × 32 pixels, and the feature region of 6 pixels × 6 pixels of the next size is 32 pixels × 32 27 × 27 different positions can be taken within the square area of the pixel. Based on the same consideration, a total of 4495 feature areas can be set in the first and second face images I ¹ and I ^{2 of} 32 pixels × 32 pixels. Further, a separate feature region is distinguished between the case where the region having the same position and size is divided into two in the horizontal direction (horizontal direction) x and the case where the analysis is divided into two in the vertical direction (vertical direction) y. In this case, a total of 8990 feature areas can be set. Some or all of these feature regions are used as feature regions A _j (j = 1, 2,..., T) for calculating similarity.

When the feature region A _j is set, next, p _j (p _j is a natural number of 2 or more) partial regions a _k (k is 1 ≦ k ≦) in the feature region A _j in the first face image I ¹ . The region value x ¹ _k in the natural number of p _j and the region value x ² _k in the corresponding p _j partial regions a _k in the feature region A _j in the second image I ² are obtained. The number of divisions p _j of the feature area A _j may be different for each feature area, and the value p (= p ₁ = p ₂ = …… = p _T ) common to the feature areas A _{1 to} A _T. Also good.

The partial area a _k, for example, FIG. 11 (a), the region (Fig obtained by equally dividing the inside characteristic region A _j to p _j pieces vertically horizontal dividing line 55 along the horizontal direction x 11 (a) may be a rectangular area long in the x direction). Further, as shown in FIG. 11B, the partial area a _k is an area obtained by equally dividing p _j in the horizontal direction by a vertical dividing line 56 along the vertical direction y in the characteristic area A _j (FIG. 11). (b) may be a rectangular region that is long in the y direction. Further, the division direction of the regions may be different among the plurality of feature regions A ₁ to _AT . In the following, feature regions with different region division directions are treated as different feature regions even if they have the same shape, size, and position.

Domain values x ¹ _k in the partial area a _k, x ² _k, for example, an average value of data of a plurality of pixels included in the partial area a _k (average pixel value) of the partial region a _k It corresponds to the luminance value. However, when the partial area _ak includes only one pixel (for example, in the case of a feature area of 2 pixels × 1 pixel), the data of the pixel itself becomes the area value of the partial area _ak . Domain values x ¹ _k, x ² _k of the partial regions a _k of feature areas A _j is the feature amount of the feature region A _j first and second face images related to I ^1, I ^2.

When the above-described region values x ¹ _k and x ² _k are obtained for the p _j partial regions a _k , the correlation in the feature region A _{j of} the first and second face images I ¹ and I ² is then obtained. It is done. More specifically, the correlation value C _j is calculated by the following equation.

However, when the division numbers p _j in all the feature regions A _{1 to} A _T are all equal, the coefficient 1 / p _{j on the} right side may be omitted.
The operation for obtaining the correlation between the images may be performed using a partial region obtained by dividing the feature region in two directions of the x direction and the y direction (that is, dividing in a lattice shape). However, in this embodiment, an operation for obtaining the correlation is performed using the partial region a _k obtained by dividing the feature region A _j in either the x direction or the y direction. Hereinafter, the correlation obtained in this way is referred to as “unidirectional correlation”.

Such unidirectional correlate seek operation, the are performed on a plurality of characteristic regions A ₁ to A _T, thereby, the correlation values _{C 1, C 2, ......,} C T is determined.
The correlation value C _j of each feature area A _j thus obtained is applied to a weak classifier (Weak Classifier) h _j (I ¹ , I ² ) defined for each feature area A _j .

That is, the correlation value C _j for each feature area A _j is compared to a predetermined threshold value theta _j determined in advance for each feature region A _j. As a result, if the correlation value C _j exceeds the threshold value θ _j , the first identification value “+1” (1) indicating that the first and second face images I ¹ and I ² are face images of the same person. A weak hypothesis (Weak Hypothesis) is generated. On the other hand, if the correlation value C _j is equal to or _smaller than the threshold value θ _j , the second identification value “−1” (1) indicating that the first and second face images I ¹ and I ² are not the same person's face image. A weak hypothesis representing a negative conclusion) is generated.

When the output values h ₁ (I ¹ , I ² ) to h _T (I ¹ , I ² ) (weak hypothesis) of the weak classifiers for each of the _T feature regions A _{1 to} A _T are obtained, Next, these are multiplied by a weighting coefficient w _j determined for each feature region A _j and added (linear combination). Thereby, the similarity (likeness of the same person) between the first and second face images I ¹ and I ² is obtained. That is, the similarity is given by the following equation.

The selection of the feature regions A _{1 to} A _T applied to the calculation of the unidirectional correlation value C _j , the threshold value θ _j in the weak classifier, and the weighting coefficient w _j are determined by prior learning. For this learning, for example, a boosting learning algorithm (AdaBoost) described in Non-Patent Document 2 is applied.

  Specifically, as shown in the flowchart of FIG. 12, a plurality of pairs of different face image data of the same person are prepared in advance as positive samples (samples in which the conclusion that they are the same person is correct), Prepare multiple pairs of face image data of other people as negative samples (samples where the conclusion that they are the same person is incorrect), and as an initialization process, weight these samples uniformly (equally). (Weight) is distributed (step S50).

Next, a feature region A (for example, a square feature region) is set in the normalized face image data, and the unidirectional correlation value C expressed by the above equation (1) is set to all samples (positive samples). And negative samples). This calculation is executed with respect to all feature areas A (for example, square areas in which at least one of position, size, and division direction of the areas is different) that can be set in the normalized face image (step). S51). Then, T feature regions A _{1 to} A _T are selected, and T times of learning are performed to determine _T weak classifiers h ₁ (I ¹ , I ² ) to h _T (I ¹ , I ² ). (For t = 1 to T loop) is started (step S52).

  Specifically, first, all positive samples in which weights (uniform weights when t = 1) are given to each unselected individual feature area A (all feature areas A when t = 1). With respect to all negative samples, a histogram for each unidirectional correlation value C (weighted histogram corresponding to a sample given a weight) is created (step S53). An example of this histogram is shown in FIG.

In general, the distribution of negative samples appears in a relatively small range of correlation values C, and the distribution of positive samples appears in a relatively large range of correlation values C. The threshold value θ is determined so that the misidentification rate (error rate) when the threshold value and the correlation value C are compared is minimized (step S54).
For example, when the threshold θ is set to a large value θ _L , misidentification (False Accept error, false positive error) that an image pair (negative sample) of different persons is an image pair of the same person is Although the number is reduced, misidentification (False Reject error, false negative error) that the image pair (positive sample) of the same person is an image pair of different persons increases. Conversely, if the threshold value θ is set to a small value θ _S , misidentification (an error for accepting another person, a false positive error) that images pairs of different persons (negative samples) are image pairs of the same person increases. In addition, misidentification (person rejection error, false negative error) that an image pair (positive sample) of the same person is an image pair of different persons is reduced.

The threshold value θ is set to a value that minimizes the erroneous identification as a whole. In general, the threshold θ is a value between the distribution range of the positive sample and the distribution range of the negative sample, or an overlap interval thereof. Along with this threshold value θ, an erroneous identification rate corresponding to the threshold value θ is obtained.
In this way, by determining the threshold value θ of all unselected feature regions A, the weak classifier h corresponding to all the feature regions A is created.

Among these weak classifiers h, the weak classifier having the lowest misclassification rate (error rate) is the t-th classifier h _t (when t = 1, the first classifier h ₁ (feature area A _1). )) (Step S55), and a weighting coefficient w _t (weighting coefficient w ₁ when t = 1) corresponding to the misidentification rate is determined (step S56).
Then, the samples correct by the t weak classifier h _t is obtained given a small weight, redistribution of the weights so as to provide a large portion in the sample to be incorrect by the t weak classifier h _t (re- weighting), and learning from step S52 is performed on this (step S57). That is, the entire feature area unselected (feature region A the non-characteristic regions A ₁ to A _t), weighted histogram is generated for the sample in which the weight is given (step S53). Of the weak classifiers corresponding to the feature region A, the weak classifier having the smallest misclassification rate is set as the (t + 1) th classifier h _{t + 1} (step S55). That is, for example, a weak classifier that gives a correct answer with high probability to a sample that is easily misidentified by the first classifier h ₁ is selected as the second classifier h ₂ . Then, the first (t + 1) weighting factor w _{t + 1} according to the misidentification rate of weak classifier h _{t + 1} of is determined (step S56).

The same operation is repeated until _T weak classifiers h _{1 to} h _T are obtained, and T feature regions A _{1 to} A _T suitable for calculating similarity are selected by performing T learning. Will be. Then, by multiplying the _T weak discriminators h _{1 to} h _T by the respective weighting coefficients w _{1 to} w _T and linearly combining them, the similarity calculation formula according to the above-mentioned formula (3) is obtained.
Then, the similarity obtained by the equation (3) is compared with a predetermined recognition threshold value _Rth . When the obtained similarity exceeds the recognition threshold value R _th , it is determined that the first and second face images I ¹ and I ² are face images of the same person, and the recognition threshold value R _th or less. The first and second face images I ¹ and I ² are determined to be face images of different persons.

  That is, the determination result is given by the following equation.

FIG. 14 is a diagram for explaining the contents held in the collation dictionary storage unit 26 (see FIGS. 1 and 3). The collation dictionary storage unit 26 stores information on the feature areas A _{1 to} A _T corresponding to the similarity calculation expression of the expression (3). That is, corresponding to the feature numbers _{1 to T} , it is applied to the feature region designation information for designating the feature regions A _{1 to AT and} the correlation values C ₁ to CT of the feature regions A ₁ to _AT . and threshold theta ₁ through? _T is, the weighting factor is given and w ₁ to w _T is stored for weak classifiers h ₁ to h _T of the feature regions a ₁ to a _T.

The feature area designation information is information for designating the position, shape, size, and type of the feature area. FIG. 14 shows an example in which the shape of the feature region is fixed to a fixed shape (for example, a square) and the number of divisions p _j of the feature region is fixed to a fixed value p. Therefore, the position, size, and type are registered as the feature area designation information. The position of the feature region represents the position in the two face images I ¹ and I ² to be collated. Specifically, for example, as shown in FIG. 11, when the feature area is rectangular, the lower left corner is regarded as the origin, and the position of the feature area is represented by the x and y coordinates of the origin. The size is represented by, for example, the length of one side (number of pixels) in the case of a square feature region. The type is information indicating whether the feature region is divided in the x direction (horizontal direction) (see FIG. 11B) or divided in the y direction (vertical direction) (see FIG. 11A).

FIG. 15 is a block diagram for explaining a configuration example of the similarity calculation unit 41 (see FIG. 3). The similarity calculation unit 41 of this configuration example includes a first face image I ^{1 to be} collated (for example, a representative face image corresponding to representative collation data or a registered face image corresponding to individual collation data) and a second face image I ^2. A feature region setting unit 45 (feature region setting means) that sets a feature region A _{j in} common (for example, a face image to be checked captured by the camera 1) and a feature amount of the first face image I ¹ are calculated. A first feature quantity computing section 46 (feature quantity computing means) and a second feature quantity computing section 47 (feature quantity computing means) for computing the feature quantity of the second face image I ² .

The feature region setting unit 45 reads the feature regions A _{1 to AT} from the collation dictionary storage unit 26 and sequentially sets them in the first and second face images I ¹ and I ² (first and second feature region setting). Step).
The first feature quantity calculation unit 46 refers to the collation dictionary storage unit 26, and sets the region according to the “type” information for the feature region A _j in the first face image I ¹ set by the feature region setting unit 45. dividing the obtained area value x ¹ _k of that broken portion each area a _k as the characteristic amount (first region value calculation step). Similarly, the second feature amount calculation unit 47 refers to the collation dictionary storage unit 26 and sets the “type” of the feature region A _j in the second face image I ² set by the feature region setting unit 45. dividing the area according to the information, it obtains the domain values x ² _k of that broken portion each area a _k as the characteristic amount (second region value calculation step).

The similarity calculation unit 41 further includes a correlation value calculation unit 48 that calculates a correlation value C _j based on the region values x ¹ _k and x ² _k obtained by the first and second feature amount calculation units 46 and 47. A correlation value calculating means, a correlation value calculating step), and a comparing section 49 (comparing means, comparing means) for comparing the correlation value C _j obtained by the correlation value calculating section 48 with a threshold value θ _j corresponding to the feature area A _j . A comparison step) and a weighting addition unit 50 (weighting addition means, weighting addition step) for multiplying the comparison result h _j (I ¹ , I ² ) by the comparison unit 49 by a weighting coefficient w _j and adding the result. The comparison unit 49 and the weighted addition unit 50 constitute a correlation base similarity calculation means in the present invention, and a correlation base similarity calculation step is executed by these functions.

The comparison unit 49 reads the threshold value θ _j corresponding to the feature region A _j with reference to the matching dictionary storage unit 26. If the correlation value C _j is larger than the threshold value θ _j , the comparison unit 49 has the first identification value “+1” (a weak hypothesis indicating that the first and second face images are face images of the same person). Is output. On the other hand, if the correlation value C _j is less than or equal to the threshold value θ _j , the comparison unit 49 determines that the second identification value “−1” (a weak hypothesis indicating that the first and second face images are the face images of others). ) Is output.

The weighting addition unit 50 multiplies the output value of the comparison unit 49 by a weighting coefficient corresponding to the feature region A _j , and linearly combines the output values of the comparison unit 49 for the feature regions A _{1 to} A _T to obtain the (3 The similarity represented by the formula is obtained and output. That is, the weighting addition unit 50 multiplies the comparison result h _j (I ¹ , I ² ) for the feature regions A _{1 to AT} given from the comparison unit 49 by the weighting coefficient w _j and adds them.

FIG. 16 is a block diagram for explaining another configuration example of the similarity calculation unit 41. In FIG. 16, the same functional parts as those shown in FIG. 15 are given the same reference numerals as those in FIG. The configuration of FIG. 16 is suitable when face image collation data (representative collation data and individual collation data) are registered in the collation data storage unit 25 in the form of feature amounts. In this case, the collation data storage unit 25 stores the feature amounts related to the feature areas A _{1 to AT} for each registered face image as individual collation data. Similarly, as representative collation data, feature quantities related to the feature areas A _{1 to} A _T for the representative face image are stored. These feature values are stored in the matching data storage unit 25 for each of the feature areas A _{1 to AT} for each matching data. The stored feature amount is a region value x ¹ _k of the partial region a _k obtained by dividing each feature region A _j .

The feature region setting unit 45A (feature region setting means) refers to the collation dictionary storage unit 26 and sequentially sets the feature regions A _{1 to AT} in the second face image I ² . The feature quantity calculation unit 47 refers to the collation dictionary storage unit 26 and calculates the feature value x ² _k for each partial area a _k for the feature area A _j set in the second face image I ² . Calculate as
The calculated region value x ² _k is given to the correlation value calculation unit 48. The correlation value calculation unit 48 reads the region value x ¹ _k corresponding to the feature region A _j of the first face image I ¹ from the collation data storage unit 25. The correlation value calculation unit 48 calculates the correlation value C _j based on the region value x ¹ _k read from the collation data storage unit 25 and the region value x ² _k calculated by the feature amount calculation unit 47. To do.

The functions of the comparison unit 49 and the weighting addition unit 50 are the same as those in the configuration of FIG.
In the case of this configuration example, for the first face image I ¹ (registered face image or representative face image), the region value x ¹ _k is obtained in advance and registered as collation data in the collation data storage unit 25. . Therefore, the amount of calculation for calculating the similarity can be reduced. Accordingly, the calculation time can be shortened, so that the time required for recognition of the face image can be shortened.

Next, the process by the face image detection unit 30 (see FIG. 3) will be outlined.
FIG. 17 is a functional block diagram for explaining a functional configuration of the face image detection unit 30. The face image detection unit 30 extracts a face image region from an input image captured by the camera 1 and generates face image data (matched face image data) whose size and inclination are normalized. The face image detection unit 30 executes a rough detection process as a first stage process and a fine detection process as a second stage process. In the rough detection process, the entire area of the image captured by the camera 1 is roughly scanned, and a part of the area that is likely to contain a face image is extracted as a peripheral area of the face image, and the remaining area is rejected. This is a process of excluding from the precise detection process target. In other words, the precision detection process is a process of extracting and cutting out a face image area within the face image peripheral area extracted by the rough detection process.

  In order to execute the rough detection process, the face image detection unit 30 includes a rough detection window setting unit 31 that sets a partial region in the captured image as a processing target region (rough detection window), and the rough detection window setting unit 31. A peripheral area identification unit 32 for identifying whether the coarse detection window set by the step is a face image peripheral area including a face image, and a peripheral area feature dictionary storing information for determination by the peripheral area identification unit 32 33. The peripheral area feature dictionary 33 is stored in a storage area in the hard disk drive 14 as shown in FIG.

  In addition, in order to execute the precision detection process, the face image detection unit 30 further sets a partial region of the face image peripheral region extracted by the peripheral region identification unit 32 as a processing target region (precision detection window). The precise detection window setting unit 34 for performing the detection, the face image identifying unit 35 for identifying whether the precise detection window set by the precise detection window setting unit 34 corresponds to the face image region, and the determination by the face image identifying unit 35 And a face image feature dictionary 36 storing information for. The face image feature dictionary 36 is stored in a storage area in the hard disk drive 14 as shown in FIG.

  Further, the face image detection unit 30 includes a face image output processing unit 37 that normalizes and outputs an image in the precision detection window determined to be a face image region by the face image identification unit 35. The face image output processing unit 37 normalizes the image in the precise detection window to a certain size (for example, 32 pixels × 32 pixels), corrects the inclination of the face image in the precise window, and Collation face image data is generated.

  The face image extracted from the captured image is a rectangular area having a predetermined shape inside the outer corners of the human face, such as three images in the center row of FIG. 18 (three images arranged in the vertical direction). (A region having the same shape as the image to be collated by the face recognition processing unit 40. For example, a square region). That is, face images included in various sizes in the captured image are cut out by a rectangular region having a constant aspect ratio. The face image in the middle row in FIG. 18 shows a case where the line segment connecting both eye corners is substantially parallel to the horizontal side of the rectangular area (0 degree), and the face image in the upper row in the middle row in FIG. In contrast, the face image in the lower row of the center row in FIG. 18 is rotated right (+5 degrees) by 5 degrees with respect to the image in the middle row of the same row.

  In this case, the face image feature dictionary 36 holds information for identifying a rectangular image as shown in the center column of FIG. 18 as a face image. More specifically, information regarding the face image detection filter corresponding to the feature pattern of the face image is held in the face image feature dictionary 36. For example, as shown in FIG. 19, the face image detection filter has, for each of a pair of rectangular areas 71 and 72 set in the processing window 70, an average luminance value of each of the plurality of pixels included in the areas 71 and 72. And the difference between the average luminance values is compared with a predetermined threshold value.

  The positions of the rectangular regions 71 and 72 in the processing window 70, the shape (vertically and horizontally long), and the threshold value for the average luminance value are predetermined by learning. This learning is performed using a large number of image samples. More specifically, a large number of image samples having the same shape and size as the processing window 70 are collected and stored in the face image sample storage unit 61 (see FIG. 17) in the storage area of the hard disk drive 14 (see FIG. 1). Stored. The image samples in this case include a face image sample including a face image (true sample) and a non-face image sample not including a face image (fake sample).

  The face image feature calculation unit 63, which is a function processing unit in the computer 3, performs learning based on the image samples stored in the face image sample storage unit 61 and can best separate the true / false samples. Find the detection filter. That is, various combinations of the positions of the rectangular areas 71 and 72 in the processing window 70 and the vertical and horizontal directions are set, and the combination that can best separate the true / false samples is selected. In this case, “being able to best isolate a true / false sample” means that misidentification with a true sample as a false sample and misidentification with a false sample as a true sample are minimized. For the combination of the rectangular regions 71 and 72 selected in this way, a face image detection filter is obtained by setting a threshold value that minimizes the misidentification rate. Information regarding the face image detection filter is stored in the face image feature dictionary 36 in advance.

  On the other hand, the peripheral area feature dictionary 33 holds information for identifying each of the three face images in the left and right columns in FIG. 18 as the face image peripheral area in addition to the face image in the central line in FIG. Will be. In the three images in the left column of FIG. 18, the face image extraction frame is shifted by a predetermined number of pixels (one pixel in the example of FIG. 18; -1 pixel) to the left from the positions of the three images in the central column of FIG. It is an image obtained by making it. In the three images in the right column of FIG. 18, the face image extraction frame is set to a predetermined number of pixels in the right direction from the position of the three images in the center column of FIG. It is an image obtained by shifting.

  The peripheral area feature dictionary 33 holds information related to the peripheral image detection filter similar to the face image detection filter. This peripheral image detection filter is determined by learning similar to the case of the face image detection filter. However, in this learning, in addition to the face image as shown in the center row of FIG. 18, face images as shown in the left and right rows are used as the face image peripheral region sample (true sample). A large number of such true samples and false samples that do not include a face image are stored in advance in the peripheral area image sample storage unit 62 that is a storage area in the hard disk drive 14.

A peripheral region detection filter capable of best separating the genuine samples stored in the peripheral region image sample storage unit 62 is detected by the peripheral region feature calculation unit 64 which is a function processing unit in the computer 3. It is obtained by learning similar to the case of the filter. Information about the surrounding area detection filter is stored in advance in the surrounding area feature dictionary 33.
The face image sample storage unit 61, the peripheral region image sample storage unit 62, the face image feature calculation unit 63, and the peripheral region feature calculation unit 64 are configured for prior learning. That is, these function processing units together with the peripheral area feature dictionary 33 and the face image feature dictionary 36 form an offline execution system 51.

On the other hand, the process of extracting the face image from the captured image is executed by the rough detection window setting unit 31, the peripheral region identification unit 32, the fine detection window setting unit 34, the face image identification unit 35, and the face image output processing unit 37. That is, these function processing units together with the peripheral area feature dictionary 33 and the face image feature dictionary 36 form an online execution system 52.
The coarse detection window setting unit 31 sequentially sets the coarse detection windows in the captured image so as to scan the entire area of the captured image with various sizes of coarse detection windows similar in shape to the processing window 70. More specifically, as shown in FIG. 20, the coarse detection window setting unit 31 starts from a start position 81 at the lower left corner in the captured image 80 and reaches a stop position 82 at the upper right corner. The coarse detection window 77 is set so as to scan the entire range. That is, the rough detection window setting unit 31 sets the first rough detection window 77 at the start position 81, and thereafter sets the rough detection window 77 in order while moving the predetermined plurality of pixels (for example, three pixels) to the right. I will do it. When the coarse detection window 77 reaches the right end of the captured image 80, the coarse detection window setting unit 31 returns the set position of the coarse detection window to the left end, moves upward by one pixel, and opens a new coarse detection window 77. Set. Thereafter, the same operation is repeated until the rough detection window 77 reaches the stop position 82.

  The coarse detection window setting unit 31 performs the above-described operation using the window having the minimum size that can be processed by the peripheral region detection filter as the initial coarse detection window 77, and thereafter sequentially sets the size of the coarse detection window 77 to a predetermined size. The same operation is repeated while increasing the size step by step (for example, by one pixel). Finally, a coarse detection window 77 having the same size as the captured image 80 is set.

  With respect to the partial images in the coarse detection window 77 sequentially set in this way, the peripheral area identifying unit 32 performs processing by the above-described peripheral area detection filter, and whether each coarse detection window 77 corresponds to the peripheral area of the face image. Determine whether or not. When the surrounding area detection filter is applied to the image in the rough detection window 77, the image is enlarged or reduced so as to match the window size of the surrounding area detection filter. With respect to the determined coarse detection window 77 that is the peripheral area of the face image, position information and size information are generated and stored in the RAM 12 (see FIG. 1).

  As described above, the peripheral region detection filter is obtained by learning using not only the sample in which the face image is in the center of the window but also the face image sample shifted by one pixel to the left and right. For this reason, if the rough detection window 77 is shifted and sequentially set in the captured image 80 with a shift width of 3 pixels or less, the region including the face image can be accurately extracted. Of course, if the coarse detection window 77 is sequentially set by shifting by 3 pixels, which is the maximum shift width, the determination of the peripheral area of the face image can be completed in the shortest time.

In general, in the case where the peripheral area detection filter is obtained by learning using the face image in the center row of FIG. 18 and the face image shifted by u pixels (where u is a natural number) to the left and right, 2u + 1 The coarse detection window 77 can be shifted by a shift width equal to or smaller than the pixels.
The precision detection window setting unit 34 sets a precision detection window in the vicinity of the area determined as the face image peripheral area by the peripheral area identification unit 32. For example, the fine detection window setting unit 34 sets a fine detection window having the same size as the rough detection window 77 within a face image peripheral region and a range shifted by one pixel to the left and right. A process by the face image identification unit 35, that is, a process by a face detection filter is executed on the set precision detection window. When a face detection filter is applied to an image in the precision detection window, the image is enlarged or reduced to match the window size of the face image detection filter.

  For the precision detection window determined to be a face image area by the face identification processing unit 35, information representing the position and size is output by the face image identification unit 35. In response to this, the face image output processing unit normalizes the size and inclination of the image in the precision detection window, and generates the face image data to be verified. In this case, “inclination normalization” refers to, for example, rotating an image having an inclination as shown in the upper or lower part of FIG. 18 to an image having substantially no inclination as shown in the middle part of FIG.

  As mentioned above, although one Embodiment of this invention was described, this invention can also be implemented with another form. For example, in the above-described embodiment, the example in which the collation data of a plurality of face images is registered in the collation data storage unit 25 has been described. However, the collation data of only one face image is registered in the collation data storage unit 25. Also good. In this case, for example, the similarity between the image data to be processed and the representative collation data is compared with a predetermined threshold value, and if the similarity does not exceed the threshold value, collation with the individual collation data is performed. What is necessary is just to complete | finish recognition processing, without performing. As a result, when the face image to be collated is clearly different from the face image of the registered person, it is possible to omit the collation processing for a plurality of individual collation data, so that the person of the face image to be collated is an unregistered person. You can make a quick decision that there is.

  In the above-described embodiment, the example in which the present invention for restricting entrance to the restricted access area is applied has been described. For example, the present invention is applied to a robot capable of dealing with people such as a home robot and a customer service robot. It is also possible to incorporate and use the image recognition apparatus. As a result, the robot can accurately recognize the face of the registered person in a short time, so that it is possible to deal with the person quickly and accurately. For example, such a robot can execute an appropriate response according to the recognized person with excellent responsiveness.

The recognition of a face image can also be used for a process of extracting a face image of a specific person from a huge amount of image data (still image or moving image). Specifically, face image recognition processing can be applied to processing of extracting a suspect's face image from video captured by a video camera in a criminal investigation.
Furthermore, although the case where the recognition target object is a face image has been described in the above embodiment, as described above, the present invention can also be applied to recognition of a recognition target object other than a face image.

  In addition, various design changes can be made within the scope of the matters described in the claims.

1 is an illustrative block diagram showing a configuration of a personal authentication system according to an embodiment of the present invention. It is a figure which shows the example of a display of the display connected to the computer which comprises an image recognition apparatus. It is a block diagram for demonstrating the functional structure of the computer in the execution state of the said computer program. It is an illustration figure for demonstrating the registration state of the collation data in a collation data storage part. It is the figure which expressed the registration state of the collation data in a collation data storage part more intuitively. It is a flowchart for demonstrating an example of the process by a face recognition process part. It is a flowchart which shows a specific example of the processing content for extracting the individual collation data of collation object. It is a flowchart which shows the other specific example of the process for extracting the individual collation data of collation object. It is a flowchart for demonstrating the other example of the process by a face recognition process part. It is a figure for demonstrating the calculation which calculates | requires the similarity of a pair of face image. It is a figure for demonstrating the characteristic area used in order to obtain | require the similarity of a face image. It is a flowchart for demonstrating the process for defining a similarity calculating formula by learning. It is a figure which shows an example of the weighting histogram when a common feature area is set to a pair of face images, and the correlation value of a pair of face images in the feature area is obtained. It is a figure for demonstrating the content held by the dictionary storage part for collation. It is a block diagram for demonstrating the structural example of a similarity calculating part. It is a block diagram for demonstrating the other structural example of a similarity calculating part. It is a functional block diagram for demonstrating the functional structure of a face image detection part. It is a figure which shows the example of the face image extracted from a captured image, and the example of the image detected as a peripheral region of a face image. It is a figure which shows the example of the filter for a face image detection process. It is a figure for demonstrating the scanning of the captured image by a rough detection window. It is a figure for the comparison description of the similarity calculation process using a filter process, and the similarity calculation process using a correlation value.

Explanation of symbols

1 Camera 2 Cable 3 Computer 4 Display 5 Display 6 Input Operation Unit 7 Gate Switch 8 CD-ROM
10 CPU
11 ROM
12 RAM
12a Image memory 12b Representative similarity storage area 12c Individual similarity storage area 13 Bus 14 Hard disk drive 15 CD-ROM drive 16 Display controller 17 Input controller 18 Image interface unit 19 Display control interface unit 20 Gate open / close control interface unit 25 Reference data Storage unit 26 Collation dictionary storage unit 30 Face image detection unit 31 Coarse detection window setting unit 32 Peripheral region identification unit 33 Peripheral region feature dictionary 34 Precision detection window setting unit 35 Face image identification unit 36 Face image feature dictionary 37 Face image output processing Unit 40 face recognition processing unit 41 similarity calculation unit 42 collation data selection unit 43 determination unit 45 feature region setting unit 45A feature region setting unit 46 first feature amount calculation unit 47 second feature amount calculation unit 48 correlation value calculation unit 49 comparison Part DESCRIPTION OF SYMBOLS 50 Weighting addition part 51 Offline execution system 52 Online execution system 55 Horizontal parting line 56 Vertical parting line 61 Face image sample storage part 62 Peripheral area image sample storage part 63 Face image feature calculation part 64 Peripheral area feature calculation part 70 Processing window 71 Rectangular Area 72 rectangular area 77 coarse detection window 80 captured image A _j feature area a _k partial area x ¹ _k area value x ² _k area value C _j correlation value I ¹ face image I ² face image

Claims (17)

An apparatus for determining the similarity between the first image I ¹ and the second image I ² ,
For each of T (T is a natural number greater than or equal to 2) feature regions A _j (j is a natural number of 1 ≦ j ≦ T) that can be commonly set in the first image I ¹ and the second image I ² , p _j number in the first image I wherein areas a _j in ¹ (p _j is a natural number of 2 or more) partial area a _k of (k is 1 ≦ k ≦ p natural number _j) domain values x ¹ _k and said Based on the region values x ² _k of the corresponding p _j partial regions a _k in the feature region A _j in the second image I ² , the feature regions of the first image I ¹ and the second image I ² Correlation value in A _j
Correlation value calculation means for calculating
Based on the correlation value C _j relating to the T characteristic region A _j obtained by the correlation value calculation means, correlation-based similarity calculating means for calculating a similarity of the first image I ¹ and the second image I ² An image similarity calculation device comprising: An apparatus for determining the similarity between the first image I ¹ and the second image I ² ,
Feature region setting means for commonly setting T (T is a natural number of 2 or more) feature regions A _j (j is a natural number of 1 ≦ j ≦ T) in the first image I ¹ and the second image I ² When,
For each of the T feature regions A _j set by the feature region setting means, p _j (p _j is a natural number of 2 or more) partial regions a in the feature region A _j in the first image I ^1. _k (k is 1 ≦ k ≦ p natural number _j) area value of the corresponding p _j number of partial areas a _k in the region value x ¹ _k and the second the features in the image I ² in the area a _j of x ² Based on _k , the correlation value in the feature area A _{j of} the first image I ¹ and the second image I ²
Correlation value calculation means for calculating
Based on the correlation value C _j relating to the T characteristic region A _j obtained by the correlation value calculation means, correlation-based similarity calculating means for calculating a similarity of the first image I ¹ and the second image I ² An image similarity calculation device comprising: An apparatus for determining the similarity between the first image I ¹ and the second image I ² ,
For each of T (T is a natural number of 2 or more) feature regions A _j (j is a natural number of 1 ≦ j ≦ T) that can be commonly set in the first image I ¹ and the second image I ² , the p _j number (p _j is a natural number of 2 or more) domain values x ¹ _k partial region a _k of (k is a natural number of 1 ≦ k ≦ p _j) in the first image I wherein the area a _j of ^1, Collation data storage means stored as collation data representing the feature quantity of the first image I ¹ ;
Feature region setting means for setting the T feature regions A _j in the second image I ² ;
For each set the T characteristic region A _j This feature region setting means, the area value x ² _k of p _j-number of partial areas a _k within the feature region A _j of the second in the image I ² A feature amount calculating means for calculating the feature amount of the second image I ² ;
For each of the T feature regions A _j , the region value x ¹ _k of the first image I ¹ stored in the collation data storage unit and the region of the second image I ² calculated by the feature amount calculation unit Based on the value x ² _k , the correlation value in the feature area A _{j of} the first image I ¹ and the second image I ²
Correlation value calculation means for calculating
Based on the correlation value C _j relating to the T characteristic region A _j obtained by the correlation value calculation means, correlation-based similarity calculating means for calculating a similarity of the first image I ¹ and the second image I ² An image similarity calculation device comprising: The correlation-based similarity calculation means includes
The correlation value C _j obtained by the correlation value calculating means is compared with a predetermined threshold value θ _j, and if the correlation value C _j exceeds the threshold value θ _j , the first image I ¹ and A first weak discrimination value h _j (I ¹ , I ² ) = H ₁ representing that the second image I ² is similar is output, and if the correlation value C _j is less than or equal to the threshold θ _j A second weak discrimination value h _j (I ¹ , I ² ) = H ₂ (where H ₁ > H ₂ ) indicating that the first and second images I ¹ and I ² are dissimilar is output. The first image I ^{1 is} obtained by linearly combining the comparison means and the output value h _j (I ¹ , I ² ) of the comparison means for the T feature areas A _j by a weighting coefficient w _j. And the similarity of the second image I ²
The image similarity calculation apparatus according to claim 1, further comprising weighted addition means for calculating 5. The partial area a _k is an area obtained by dividing the first image I ¹ and the second image I ² by dividing the feature area A _j only in a predetermined direction. The image similarity calculation apparatus according to any one of the above. An image similarity calculation device according to any one of claims 1 to 5,
If the similarity calculated by the image similarity calculation device exceeds a predetermined recognition threshold, it is determined that the first image I ¹ and the second image I ² are similar, and the image similarity And determining means for determining that the first image I ¹ and the second image I ² are dissimilar if the degree of similarity calculated by the degree calculating device is equal to or less than the recognition threshold value. An image recognition device. A method for determining the similarity between a first image I ¹ and a second image I ² , comprising:
A first feature region setting step of setting T (T is a natural number of 2 or more) feature regions A _j (j is a natural number of 1 ≦ j ≦ T) in the first image I ¹ ;
A second feature region setting step of setting the T feature regions A _j in the second image I ² ;
Wherein for each of the first image I ¹ on the set T number of feature areas A _j, partial regions of the p _j number in the first image I wherein areas A _j in ¹ (p _j is a natural number of 2 or more) a first region value calculating step of calculating a region value x ¹ _k of a _k (k is a natural number of 1 ≦ k ≦ p _j );
For each of the second image I ² on the set T number of feature areas A _j, the region value x ² _k of p _j-number of partial areas a _k in the characteristic region A _j of the second in the image I ² A second region value calculation step for calculating;
For each of the T feature regions A _j , the region value x ¹ _k of p _j partial regions a _k in the feature region A _j in the first image I ¹ and the corresponding value in the second image I ^2. based on the area value x ² _k of p _j-number of partial areas a _k corresponding in the characteristic region a _j, the correlation value of the first image I ¹ and the second image I ² of the feature region a _j
A correlation value calculating step for calculating
Characterized in that it comprises, based on the obtained T pieces of feature areas A _j related to the correlation value C _j, the correlation-based similarity computing step of computing a similarity of the first image I ¹ and the second image I ² An image similarity calculation method. The correlation-based similarity calculation step includes:
The correlation value C _j obtained by said correlation value calculation step with respect to the feature region A _j is compared with a predetermined threshold value theta _j, if the correlation value C _j exceeds the said threshold theta _j, the first The first weak discrimination value h _j (I ¹ , I ² ) = H ₁ representing that the first image I ¹ and the second image I ² are similar is used as the discrimination value for the feature region A _j , and the correlation value C _{If j} is equal to or less than the threshold value θ _j , a second weak discrimination value h _j (I ¹ , I ² ) = H ₂ indicating that the first and second images I ¹ and I ² are dissimilar. A comparison step in which D is an identification value for the feature region A _j ,
By multiplying the identification values h _j (I ¹ , I ² ) for the T feature regions A _j by a weighting coefficient w _j , the first image I ¹ and the second image I ² Degree of similarity
The image similarity calculation method according to claim 7, further comprising a weighted addition step for calculating. The partial region a _k is according to claim 7, characterized in that the first image I ¹ and the second image I ² is an area obtained by dividing the characteristic region A _j with respect to only one predetermined direction Image similarity calculation method. A similarity calculation step by the image similarity calculation method according to claim 7,
If the similarity calculated in the similarity calculation step exceeds a predetermined recognition threshold, it is determined that the first image I ¹ and the second image I ² are similar, and the image similarity is A determination step of determining that the first image I ¹ and the second image I ² are dissimilar if the degree of similarity calculated by the calculation device is equal to or less than the recognition threshold value. Image recognition method. A computer program for operating a computer as a device for comparing a first image I ¹ and a second image I ² ,
The computer program stores the computer,
For each of T (T is a natural number greater than or equal to 2) feature regions A _j (j is a natural number of 1 ≦ j ≦ T) that can be commonly set in the first image I ¹ and the second image I ² , p _j number in the first image I wherein areas a _j in ¹ (p _j is a natural number of 2 or more) partial area a _k of (k is 1 ≦ k ≦ p natural number _j) domain values x ¹ _k and said Based on the region values x ² _k of the corresponding p _j partial regions a _k in the feature region A _j in the second image I ² , the feature regions of the first image I ¹ and the second image I ² Correlation value in A _j
And a similarity value between the first image I ¹ and the second image I ² based on the correlation value C _j regarding the T feature regions A _j obtained by the correlation value calculation means. A computer program that functions as a correlation-based similarity calculation means for calculating. A computer program for operating a computer as a device for comparing a first image I ¹ and a second image I ² ,
The computer program stores the computer,
Feature region setting means for commonly setting T (T is a natural number of 2 or more) feature regions A _j (j is a natural number of 1 ≦ j ≦ T) in the first image I ¹ and the second image I ² ,
For each of the T feature regions A _j set by the feature region setting means, p _j (p _j is a natural number of 2 or more) partial regions a in the feature region A _j in the first image I ^1. _k (k is 1 ≦ k ≦ p natural number _j) area value of the corresponding p _j number of partial areas a _k in the region value x ¹ _k and the second the features in the image I ² in the area a _j of x ² Based on _k , the correlation value in the feature area A _{j of} the first image I ¹ and the second image I ²
And a similarity value between the first image I ¹ and the second image I ² based on the correlation value C _j regarding the T feature regions A _j obtained by the correlation value calculation means. A computer program that functions as a correlation-based similarity calculation means for calculating. A computer program for operating a computer as a device for comparing a first image I ¹ and a second image I ² ,
The computer has T characteristic regions A _j (j is a natural number of 1 ≦ j ≦ T) that can be set in common in the first image I ¹ and the second image I ² (T is a natural number of 2 or more). for each, the region value x of the partial area a _k of p _j number (p _j is a natural number of 2 or more) in the first image I wherein areas a _j in ¹ (k is a natural number of 1 ≦ k ≦ p _j) Including collation data storage means for storing ¹ _k as collation data representing the feature amount of the first image I ¹ ,
The computer program stores the computer,
Feature region setting means for setting the T feature regions A _j in the second image I ² ;
For each set the T characteristic region A _j This feature region setting means, the area value x ² _k of p _j-number of partial areas a _k within the feature region A _j of the second in the image I ² Feature amount calculating means for calculating the feature amount of the second image I ² ;
For each of the T feature regions A _j , the region value x ¹ _k of the first image I ¹ stored in the collation data storage unit and the region of the second image I ² calculated by the feature amount calculation unit Based on the value x ² _k , the correlation value in the feature area A _{j of} the first image I ¹ and the second image I ²
And a similarity value between the first image I ¹ and the second image I ² based on the correlation value C _j regarding the T feature regions A _j obtained by the correlation value calculation means. A computer program that functions as a correlation-based similarity calculation means for calculating. The correlation-based similarity calculation means includes
The correlation value C _j obtained by the correlation value calculating means is compared with a predetermined threshold value θ _j, and if the correlation value C _j exceeds the threshold value θ _j , the first image I ¹ and A first weak discrimination value h _j (I ¹ , I ² ) = H ₁ representing that the second image I ² is similar is output, and if the correlation value C _j is less than or equal to the threshold θ _j A second weak discrimination value h _j (I ¹ , I ² ) = H ₂ (where H ₁ > H ₂ ) indicating that the first and second images I ¹ and I ² are dissimilar is output. A comparison means;
The first image I ¹ and the second image are linearly combined by multiplying the output value h _j (I ¹ , I ² ) of the comparison means for the T feature regions A _j by a weighting coefficient w _j. Similarity of I ²
The computer program according to claim 11, further comprising weighted addition means for calculating 15. The partial area a _k is an area obtained by dividing the first image I ¹ and the second image I ² by dividing the feature area A _j only in a predetermined direction. A computer program according to any one of the above. Said computer further
If the similarity calculated by the weighted addition means exceeds a predetermined recognition threshold, it is determined that the first image I ¹ and the second image I ² are similar, and the image similarity calculation When the similarity calculated by the apparatus is equal to or less than the recognition threshold value, the first image I ¹ and the second image I ² are made to function as a determination unit that determines that they are dissimilar. Item 16. The computer program according to any one of Items 11 to 15. A computer-readable recording medium on which the computer program according to claim 11 is recorded.