JP2012221061A - Image recognition apparatus, image recognition method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recognize an object with higher accuracy.SOLUTION: An image recognition apparatus comprises: extraction means for cutting out a plurality of local areas from an input image and extracting a feature amount from each of the plurality of local areas; collation means for calculating the degree of similarity between the input image and a registered image for each of the local areas cut out by the extraction means; selection means for selecting a local area based on the state of the local areas cut out by the extraction means; and identification means for selecting a metric matrix to calculate the distance to the registered image based on the local area selected by the selection means, and identifying whether the input image is an image in the same category with the registered image by using the selected metric matrix and the degree of similarity, which is calculated by the collation means, of the local area selected by the selection means.

Description

  The present invention relates to an image recognition apparatus, an image recognition method, and a program.

2. Description of the Related Art Conventionally, there has been known a face recognition technique for performing personal identification by extracting a face region (face image) from an image including a human face and comparing the extracted face image with a face image of a specific person registered in advance. It has been.
This technique is used for security purposes, for example, permitting a person to enter the office when the person in the camera is authenticated as a registrant. On the other hand, there is a demand for using this technique for searching for a photograph showing the same person.
In the former application, high-accuracy recognition is possible by limiting the conditions for photographing a person. However, in the latter case, there are problems that the photographing conditions of the person are diverse and the recognition accuracy is lowered. For example, even if the same person is photographed between photographs with different face orientations, facial expressions, and illumination at the time of shooting, they may be erroneously determined to be different persons.

In order to solve such a problem, a method of extracting a plurality of local regions from a face image and performing recognition based on the similarity of the local regions has been proposed. For example, a method is disclosed in which collation based on principal component analysis is performed for each local region of a face image to enhance resistance to face orientation and hiding (see Non-Patent Document 1). Here, the local area is a part representing a characteristic area of the face such as eyes, nose and mouth.
On the other hand, a metric function that evaluates the similarity between the input face image and the registered face image is learned using a set of face images of the same person and non-identical person that have been collected in advance, and collation is performed using the metric function. A method for improving the identification performance by performing is disclosed (Non-Patent Document 2). That is, the determination of the similarity of face images is not a simple principal component analysis, but a method of extracting components suitable for discrimination between the same person and non-identical persons in the feature amount of the face image from the learning sample. It has been.

Pentland, Moghaddam and Starner. View-based and modular eigenspaces for face recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'94) Guillaumin, Verbeek and Schmid.Is that you? Metric learning approaches for face identification.IEEE International Conference on Computer Vision (ICCV), Oct. 2009 Viola and Jones.Rapid Object Detection using Boosted Cascade of Simple Features.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR2001) Cootes, Edwards and Taylor. Active Appearance Models. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 6, JUNE 2001 Murase, Kaneko, Igarashi. Robust Image Matching by Incremental Sign Correlation. IEICE Transactions (D-II), vol.J83-D-II, no.5, pp.1323-1331, May 2000.

  However, in the method of Non-Patent Document 2, the similarity between feature vectors obtained by simply combining the feature quantities of local regions is evaluated when determining the similarity. The accuracy of recognition decreases when there is a significant change in.

  The present invention has been made in view of such problems, and an object thereof is to recognize an object with higher accuracy.

  Therefore, the image recognition apparatus according to the present invention calculates the similarity between an extraction unit that extracts a plurality of local regions from an input image and extracts a feature amount, and a registered image for each local region extracted by the extraction unit. The distance between the collating unit, the selecting unit that selects the local region based on the state of the local region cut out by the extracting unit, and the registered image based on the local region selected by the selecting unit is calculated. A metric matrix is selected, and whether the input image is an image in the same category as the registered image using the selected metric matrix and the similarity calculated by the matching unit of the local region selected by the selection unit. And identifying means for identifying.

  According to the present invention, an object can be recognized with higher accuracy.

It is a figure which shows an example of a structure of an image recognition apparatus. It is a figure which shows an example of the flowchart which concerns on a recognition process. It is a figure which shows an example of an end point. It is a figure which shows an example of a rectangular area. It is a figure which shows an example of an input image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiments do not limit the present invention, and all the configurations described in the embodiments are not necessarily essential to the means for solving the problems of the present invention.

FIG. 1 is a diagram illustrating an example of a configuration of an image recognition apparatus according to the present embodiment. The image acquisition unit 110 acquires image data captured by an imaging unit such as a camera. The face detection unit 120 cuts out a face area (face image) from the image data acquired by the image acquisition unit 110.
The end point detection unit 130 detects end points of facial parts such as eyes, nose, and mouth that represent facial features from the face image cut out by the face detection unit 120. The local feature extraction unit 140 cuts out a local region from the face image cut out by the face detection unit 120 based on the end point position detected by the end point detection unit 130, and calculates a feature amount for recognizing the face from the cut out local region. Ask.

The dimension compressing unit 150 compresses the feature amount (local feature amount) of the local region obtained by the local feature extracting unit 140 into a low-dimensional feature space. At this time, the dimension compression unit 150 uses a projection matrix stored in the recognition parameter storage unit 210. The feature amount collation unit 160 collates the feature amount compressed by the dimension compression unit 150 with the feature amount registered in the registered face data storage unit 220 in advance, and obtains a similarity between them. The local region selection unit 170 selects a local region used for face identification from the similarity obtained by the feature amount matching unit 160.
The integrated identification unit 180 integrates the similarity of the local regions selected by the local region selection unit 170, and determines whether the person of the input face image is a registered image (registered image). At this time, the integrated identification unit 180 selectively applies the metric matrix stored in the recognition parameter storage unit 210 according to the local region selected by the local region selection unit 170.

The recognition parameter storage unit 210 is a memory that stores the projection matrix obtained by pre-learning used by the dimension compression unit 150 and the metric matrix obtained by pre-learning used by the integrated identification unit 180. Each pre-learning may be performed by an image recognition device, or may be performed by another device different from the image recognition device.
The registered face data storage unit 220 is a memory that stores a feature amount obtained from a face image (an example of a registered image) of a specific person registered in advance used by the feature amount matching unit 160.

Hereinafter, the operation of the image recognition apparatus will be described with reference to the flowchart of FIG. FIG. 2 is a diagram illustrating an example of a flowchart relating to recognition processing.
First, the image acquisition unit 110 acquires image data (input image) captured by an imaging unit such as a camera (S101). The acquired image data is stored in a memory inside the image acquisition unit 110. At this time, it is assumed that the acquired image data is a luminance image. Note that when acquiring a color image such as RGB, the image acquisition unit 110 converts it into a luminance image and stores it.

The face detection unit 120 cuts out a face region from the image data acquired by the image acquisition unit 110 (S102). For example, the face detection unit 120 obtains the position of the face area in the input image stored in the memory of the image acquisition unit 110 by the method disclosed in Non-Patent Document 3. Then, the face detection unit 120 scales the detected face area to a predetermined size (for example, 100 × 100 pixels), and stores the face image in the memory inside the face detection unit 120.
The end point detection unit 130 detects end points of facial parts such as eyes, nose, and mouth representing facial features from the face image cut out by the face detection unit 120 (S103). For example, the end point detection unit 130 detects the positions of the left and right end points of the eyes, the corners of the eyes, and the mouth as shown in FIG. As a technique for detecting these feature points, for example, a method disclosed in Non-Patent Document 4 is used. The coordinates of the detected end point position are stored in a memory inside the end point detection unit 130.

The local feature extraction unit 140 cuts out a local region from the face image cut out by the face detection unit 120 based on the end point position detected by the end point detection unit 130, and calculates a feature amount for recognizing the face from the cut out local region. Obtained (S104). For example, the local feature extraction unit 140 cuts out six regions including the left and right eyes, the left and right eyebrows, the nose, and the mouth region as local regions as indicated by broken lines in FIG.
For example, when extracting the region of the left eye, the local feature extraction unit 140 refers to the memory of the face detection unit 120 with reference to the coordinates of the position of the left eye corner and the eye eye stored in the memory inside the end point detection unit 130. Cut out from the face image stored in. That is, the local feature extraction unit 140 obtains the coordinates of the positions of the four corners of the left eye region in the face image from the coordinates of the positions of the left eye corner and the eye head based on a predetermined geometric relationship, and the left eye region is a predetermined rectangular shape. A local region image is obtained by performing geometric transformation so that The local area is converted so as to be a rectangular area of 30 × 31 pixels, for example.

And the local feature extraction part 140 calculates | requires a local feature-value from the calculated | required local region image. In the present embodiment, the local feature extraction unit 140 obtains an incremental code feature quantity as the local feature quantity. As described in Non-Patent Document 5, the incremental sign feature amount represents a tendency of increase or decrease in luminance of adjacent pixels. Here, the feature amount is represented by 1 bit based on the magnitude relationship between the upper and lower pixels. It is expressed by Note that the incremental code feature amount has a feature that is robust to illumination variations.
The obtained feature amount is stored in the memory inside the local feature extraction unit 140 in 30 × 30 pixels, 1 bit, that is, 900 bits. Similarly, the local feature extraction unit 140 cuts out image data from the face image stored in the memory inside the face detection unit 120 based on the coordinates of the end point positions to be referred to for other local regions. Find the feature quantity.
In addition, the size of the local region may be obtained with an appropriate size for each local region, or may be the same size for all local regions. Further, the feature amount may be obtained from more local regions. In addition, although the case where the incremental code is used as the feature amount has been described here, other feature amounts such as a luminance gradient histogram and a Gabor wavelet may be used, or a feature amount obtained by combining them may be used. Also good.

The dimension compressing unit 150 compresses the local feature amount obtained by the local feature extracting unit 140 into a low-dimensional feature space (S105). Here, the dimension compression unit 150 uses a projection matrix stored in the recognition parameter storage unit 210.
The projection matrix is obtained in advance by performing principal component analysis on feature quantities obtained from learning samples for each local region. That is, many face images are collected in the order of several thousand to several tens of thousands of samples in advance, and the above-described processing of S101 to S104 is performed to obtain a 900-dimensional feature vector expressed in 900 bits for each local region. deep. Then, a principal component analysis is performed on the obtained set of feature vectors, and a projection vector projected onto a higher-order principal component (a predetermined number, for example, 50) is obtained for each local region, and a recognition parameter storage unit 210 is obtained. Remember it.

The dimension compression process is simple and only performs projective transformation of the input local feature quantity using the projection matrix obtained by the above-described method. The converted feature quantity is a low-dimensional feature vector, and only main components constituting the feature quantity of the local region from which the noise component is removed are extracted. The converted feature value is stored in a memory inside the dimension compression unit 150. This dimension compression process is repeated for each local region.
The feature amount collation unit 160 collates the feature amount compressed by the dimension compression unit 150 with the feature amount registered in the registered face data storage unit 220 in advance, and obtains a similarity between them (S106). Here, the feature amount of the registered face (registered face image) is the feature amount obtained by the above-described processing of S101 to S105, similarly to the input face image. The matching between feature amounts is performed by calculating the Manhattan distance for each local region, and the reciprocal thereof represents the similarity.
In addition to the Manhattan distance, the distance calculation method may use a Euclidean distance, a Mahalanobis distance considering data dispersion, or the like. Further, a method for obtaining the similarity by the cosine similarity may be used.

The local region selection unit 170 selects a local region used for face identification from the similarity obtained by the feature amount matching unit 160 (S107). That is, the local region selection unit 170 performs subsequent processing only on a local region having a distance between feature amounts smaller than a predetermined value or a local region having a similarity higher than a predetermined value among the similarities obtained by the feature amount matching unit 160. To use selectively.
FIG. 5 is a diagram showing an input face image when the left eye and the left eyebrows of the face are covered with hair and are hidden. In this case, even if the person of the input face image and the person of the registered face image are the same person, the similarity is extremely low in the local area where the hiding has occurred. As described above, when a part of the input face image is hidden, if the feature amount of the local region is used, the recognition accuracy is lowered. I try to avoid the effects. The same can be said for a case where the face is not frontal but oblique, or when a strong shadow is generated on a part of the face due to the external environment.

The similarity between the input face and the registered face in the selected local area is stored in the memory inside the local area selection unit 170. The processing of S106 and S107 is repeated for each local region.
In the present embodiment, it is determined whether to select a local region based on the similarity of the local feature amount with the registered face image. However, the local region may be selected by another method. For example, the local region selection unit 170 determines whether or not the state of the local region of the input face image for each local region causes a decrease in accuracy in the local region selection unit 170 based on the local feature amount of the input face image. Judgment. In other words, the local region selection unit 170 selects a local region used for face identification based on the state of the local region.
For example, when the hiding as shown in FIG. 5 occurs or when the face is inclined, the local region selection unit 170 determines whether or not the feature amount is obtained when an intense shadow occurs. In this case, the local region is not selected.

The integrated identification unit 180 selects a metric matrix stored in the recognition parameter storage unit 210 according to the local region selected by the local region selection unit 170 (S108).
First, the metric matrix will be described. The metric matrix is a matrix for measuring the distance between two feature vectors as described in Non-Patent Document 2.
Normally, the Mahalanobis distance is measured using the data variance-covariance matrix, but in recent years, an objective function is set based on supervised data, and a matrix that measures the distance to minimize the error of this objective function is used. A method for obtaining by learning has been proposed.
For example, in Non-Patent Document 2, a metric matrix is obtained by applying LDML (Logistic Discriminant based Metric Learning) to a feature amount of a face image. Further, for example, in Non-Patent Document 2, a metric matrix is obtained by applying Information Theoretic Metric Learning (ITML) to a feature amount of a face image. Also in the present embodiment, a metric matrix is learned by LDML or ITML, and a test image having a good result is adopted.

In the present embodiment, the similarity is obtained for each local region in the processing of S106, and these are combined to form the feature vector x. That is, the distance d _M is defined as in (Equation 1) below.
d _M = x ^T Mx (Formula 1)
Here, M is a symmetric positive definite matrix of D rows and D columns, D is the dimension of the feature vector x, and T is transposition. Further, x = (x ₁ , x ₂ , x ₃ ,..., X _N ) ^T , and x _i is the similarity between the input image of the i-th local region and the registered image.

At the time of learning, the processing of S101 to S106 described above is performed for each local region on a face image sample (image sample) collected in advance, and the obtained similarity is classified by category (category) Another). Then, the metric matrix is obtained so that the distance between the feature quantities in the local area of the face image of the same person (distance between the images) is small and that of the non-identical person is large.
In the calculation of the similarity using the metric matrix, the distance d _M is obtained from the feature vector configured by combining the similarities obtained for each local region according to (Equation 1). However, in the present embodiment, only the effective local region is selected in S107, and the similarity of the entire face is calculated for it. Therefore, learning of the metric matrix is performed for all cases of selection of the local region, and the obtained metric matrix is stored in the recognition parameter storage unit 210.

These are, for example, a metric matrix that uses all regions as local regions, and a metric matrix that uses only five regions out of six regions (when only the left eyebrow is not used or when only the right eyebrow is not used).
Then, the metric matrix stored in the recognition parameter storage unit 210 is selected according to the local region selected by the local region selection unit 170. For example, in the case of the face image shown in FIG. 5, a metric matrix for calculating the similarity of the entire face is selected using the similarity of four regions excluding the left eye and the left eyebrow.

Note that if the metric matrix is to be held for all cases of selection of local regions, the memory capacity in the recognition parameter storage unit 210 becomes enormous if the number of local regions is large.
In such a case, only the metric matrix having a large number of elements corresponding to more local regions is stored in the recognition parameter storage unit 210 so that the metric matrix including the local region selected in S107 is selected. To do.
Then, the similarity of the entire face is calculated using only elements corresponding to the local region selected from the selected metric matrix.

Next, the integrated identification unit 180 integrates the similarity of the local regions selected by the local region selection unit 170 and registers the person of the input face image using the metric matrix selected by the local region selection unit 170. It is determined whether or not the person is a face image (S109). The distance d _M ′ is calculated as in (Equation 2) below, and the reciprocal thereof represents the similarity of the entire face.
d _M '= x' ^T M'x '(Formula 2)
Here, M ′ is a metric matrix selected in S108, and x ′ is a feature vector formed by combining the similarity of feature amounts between local regions selected in S107.

Then, the integrated identification unit 180 determines that the person of the input face image is the same person as the person of the registered face image when the similarity of the entire face is greater than or equal to a predetermined value.
The case where there is one registered person has been described above, but the present embodiment can also be applied when there are a plurality of registered persons. When the processes of S106 to S109 described above are repeated for each registered person's face image and there are a plurality of registered face images whose similarity is equal to or greater than a predetermined value, the person of the registered face image having the maximum similarity is determined as the person Judge that.
Further, the integrated identification unit 180 outputs a recognition result (S110).

The example in which this embodiment is applied to personal identification of a face has been described above. As described above, in the present embodiment, the similarity between the input image and the registered image is calculated for each local region, a local region having a high similarity is selected, and a face is identified using a metric matrix corresponding to the local region. I made it.
Therefore, high-accuracy face recognition that takes advantage of both the recognition method that is robust to the influence of hiding, face orientation, and lighting based on local region similarity and the high-precision similarity determination using a metric matrix be able to.
In this embodiment, face recognition has been described as an example, but this embodiment can be widely applied to applications that identify whether an input image is an object of a predetermined category. For example, the present embodiment can be applied to the classification of animal faces, such as whether the face image is a dog face, a cat face, or a dog breed.

<Other embodiments>
The image recognition device may be an information processing device (computer) having a CPU, ROM, RAM, hard disk, or the like, a camera having an information processing device, or the like. In this case, basically, the CPU stores the program stored in the ROM, hard disk or the like in the RAM and executes the program, thereby realizing the functions of the image recognition apparatus and the processing related to the flowchart.
However, you may comprise the function of an image recognition apparatus, a part of process based on a flowchart, or all using dedicated hardware.
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

  According to the configuration of the above-described embodiment, an object can be recognized with higher accuracy.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

Claims (6)

Extraction means for extracting a feature amount by cutting out a plurality of local regions from the input image;
Collating means for calculating a similarity with a registered image for each local region cut out by the extracting means;
Selection means for selecting a local region based on the state of the local region cut out by the extraction unit;
A metric matrix for calculating the distance from the registered image is selected based on the local area selected by the selection means, and the metric matrix calculated by the matching means for the selected metric matrix and the local area selected by the selection means An image recognition apparatus comprising: an identification unit that identifies whether the input image is an image in the same category as the registered image using similarity.   The image recognition apparatus according to claim 1, wherein the selection unit selects a prescribed number of local regions from those having a high degree of similarity calculated by the matching unit.   The metric matrix is obtained by classifying the local regions into categories for image samples collected in advance so that the distance between images in the same category is small and the distance between images in different categories is large. The image recognition apparatus according to claim 1.   The image recognition apparatus according to claim 1, wherein the registered image is a human face image. An extraction step of extracting a feature amount by cutting out a plurality of local regions from the input image;
A matching step for calculating a similarity with a registered image for each local region cut out in the extraction step;
A selection step of selecting a local region based on the state of the local region cut out in the extraction step;
Select a metric matrix that calculates the distance from the registered image based on the local region selected in the selection step, and calculated in the matching step of the selected metric matrix and the local region selected in the selection step And an identification step of identifying whether the input image is an image in the same category as the registered image using similarity.   The program which makes a computer perform each process of the image recognition method of Claim 5.

Images