JP2013218605A - Image recognition device, image recognition method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to obtain an accurate recognition result while suppressing an increase in the number of collation times.SOLUTION: A local area is cut out from a face image, a local feature amount for recognizing a face is obtained from the cutout area, similarity of local feature amounts between adjacent viewpoints is calculated, and the viewpoints of a multi-viewpoint image are labeled based on the calculated similarity. Then, the local feature amounts are integrated based on the labeling result. In comparison with a registered image, similarity is calculated based on the integrated local feature amount, thereby obtaining a recognition result.

Description

  The present invention relates to an image recognition apparatus, an image recognition method, and a program suitable for use in, for example, face recognition.

  2. Description of the Related Art Conventionally, there has been known a face recognition technique for performing individual identification by extracting a face area as a face image from an image including a human face and comparing the extracted face image with a face image of a specific person registered in advance. This face recognition technology is used for security purposes, for example, permitting entry into an office or the like when a person in a camera is authenticated as a registered person.

  On the other hand, there is also a demand for using this face recognition technology for searching for a photograph showing the same person. In security applications such as those described above, high-precision recognition is possible by limiting the shooting conditions when shooting a person. However, when used for search, the recognition accuracy is high because there are various shooting conditions for people. There is a problem that it falls. For example, between photographs with different face orientations, facial expressions, and illumination at the time of shooting, the same person may be mistakenly identified as a different person.

  Therefore, in recent years, various methods using face images in a plurality of directions have been proposed as methods for performing face recognition with high accuracy even when the face orientations are different. For example, Non-Patent Document 1 discloses a method in which a partial space is created from face images in a plurality of directions, and recognition is performed based on the similarity between the partial spaces of the registered face and the input face. Further, Patent Document 1 discloses a method for integrating individual time series face images of the same person acquired from a plurality of cameras as a single image sequence, and performing individual identification after acquiring a change in the appearance of the face. Here, subspace recognition is also used. On the other hand, Patent Document 2 discloses a method of selecting two face images having different directions from face images in a plurality of directions, generating a front face image by field morphing, and collating the front faces. As a third method, face images in a plurality of directions are prepared as registered faces and input faces, and one face image is selected from each of the registered face image group and the input face image group, and comprehensive verification is repeated. Has also been proposed.

JP 2007-317062 A JP 2009-25874 A

Kazuhiro Fukui: "Face image recognition using multi-viewpoint images", Reports of the Japan Society for Precision Engineering Image Applied Technology Technical Committee, Vol.19, No.3, pp.1-10, (2004) Viola and Jones.Rapid Object Detection using Boosted Cascade of Simple Features.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR2001) Cootes, Edwards and Taylor. Active Appearance Models. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 6, JUNE 2001 C. Shan and T. Gritti, Learning Discriminative LBP-Histogram Bins For Facial Expression Recognition. In Proc. British Machine Vision Conference, 2008

  However, in the method using subspaces, if the range of face orientation changes is too wide, the appearance of the face changes greatly, making it difficult to create a subspace that can accurately represent individual features. Become. In addition, the method using image morphing has a problem that the generated front face breaks down if the association between images fails. Therefore, in order to perform face authentication with the highest accuracy, it is desirable to use a method in which collation is comprehensively repeated between the registered face image group and the input face image group. However, this method has a problem that the number of collations increases as the number of face images to be acquired increases, and the amount of calculation becomes enormous.

  In view of the above-described problems, an object of the present invention is to obtain a highly accurate recognition result while suppressing an increase in the number of collations.

  The image recognition apparatus according to the present invention includes an acquisition unit that acquires a multi-viewpoint image including a plurality of single-viewpoint images with different imaging directions of the subject, an extraction unit that extracts a feature amount from each of the plurality of single-viewpoint images, A first similarity calculating unit that calculates a similarity between each feature amount extracted by the extracting unit; a similarity calculated by the first similarity calculating unit; and a viewpoint position of the multi-viewpoint image An integration unit that integrates the feature amounts based on the second feature, a second similarity calculation unit that calculates a similarity between the feature amount integrated by the integration unit and a feature amount of a registered image, and the second Recognizing means for recognizing the subject based on the similarity calculated by the similarity calculating means.

  According to the present invention, it is possible to suppress an increase in the number of verifications even if exhaustive verification between multi-viewpoint images is performed. In addition, recognition accuracy can be maintained in collation between a plurality of image groups.

It is a block diagram which shows the function structural example of the image recognition apparatus which concerns on embodiment. It is a figure which shows an example of the multiview imaging device which comprises the multiview image acquisition part in embodiment. 6 is a flowchart illustrating an example of a processing procedure for registering a multi-viewpoint image in the embodiment. It is a figure which shows an example of a multiview image. It is a figure explaining the position of a feature point. It is a figure explaining the local area | region in a person's face. It is a figure which shows an example of the result of labeling in embodiment. In an embodiment, it is a flow chart which shows an example of a processing procedure which inputs a multi-viewpoint image and collates with a registered multi-viewpoint image.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, an example of performing individual identification of a face from a multi-viewpoint face image will be described.
FIG. 1 is a block diagram illustrating a functional configuration example of an image recognition apparatus 100 according to the present embodiment.
In FIG. 1, a multi-viewpoint image acquisition unit 101 acquires a multi-view imaging device including a plurality of imaging units, an imaging control unit that controls an imaging unit included in the multi-view imaging device, and a multi-view imaging device. And a recording device that records a multi-viewpoint image for each viewpoint position. FIG. 2 shows an example of the multi-eye imaging apparatus 200. As illustrated in FIG. 2, in the multi-lens imaging device 200, 25 imaging units 201 to 225 that acquire image data are evenly arranged on a square lattice. With such a multi-lens imaging device, it is possible to obtain a group of images with different imaging directions in which the same subject is imaged from a plurality of viewpoint positions.

  The face detection unit 102 detects a face region from each single-viewpoint image constituting the multi-viewpoint image acquired by the multi-viewpoint image acquisition unit 101, and outputs information on the position coordinates. The face image normalization unit 103 performs image geometric conversion on the original single-viewpoint image so that the face region has a predetermined arrangement and size from the position coordinates of the face region detected by the face detection unit 102. , Cut out the normalized face image.

  The face feature point detection unit 104 detects end points of facial parts such as eyes, nose, and mouth that represent facial features from the face image normalized by the face image normalization unit 103, and outputs information on the position coordinates. The local feature extraction unit 105 extracts a local region from the face image normalized by the face image normalization unit 103 based on the face feature end point position detected by the face feature point detection unit 104, and recognizes the face from the extracted region. A local feature amount for obtaining the value is obtained. The local feature storage unit 106 is a memory that stores the local feature amount of the face obtained by the local feature extraction unit 105.

  The first similarity calculation unit 107 calculates the similarity of local feature amounts of different single-viewpoint images in the multi-viewpoint image stored in the local feature storage unit 106. The labeling unit 108 labels the viewpoints of the multi-viewpoint image based on the similarity calculated by the first similarity calculation unit 107. The local feature integration unit 109 integrates the local feature amounts stored in the local feature storage unit 106 based on the result of the labeling unit 108. The multi-view face feature registration unit 110 registers the local feature amount integrated by the local feature integration unit 109 in association with the viewpoint position of the multi-view image.

  The second similarity calculation unit 111 calculates the similarity of local feature amounts between multi-viewpoint images. That is, the similarity between the local feature amount integrated by the local feature integration unit 109 and the local feature amount already registered in the multi-viewpoint face feature registration unit 110 is calculated. The similarity integration unit 112 integrates the similarities calculated by the second similarity calculation unit 111 and outputs the similarities between the multi-viewpoint images. The recognizing unit 113 determines whether the person of the multi-view image input based on the similarity of the similarity integrating unit 112 is the same person as the person of the registered multi-view image.

Next, the operation of the image recognition apparatus 100 according to this embodiment will be described.
FIG. 3 is a flowchart illustrating an example of a processing procedure for registering a multi-viewpoint image according to the present embodiment.
First, the multi-view image acquisition unit 101 acquires a multi-view image and stores the acquired multi-view image in the internal memory of the multi-view image acquisition unit 101 (S301). At this time, the acquired multi-viewpoint image is a luminance image, and when a color image such as RGB is acquired, it is converted into a luminance image and stored in the internal memory. FIG. 4 shows an example of a multi-viewpoint image. For example, the face image 401 at the upper left end in FIG. 4 is acquired by the imaging unit 201 in FIG.

  Next, the face detection unit 102 obtains an accurate position and size of the face area from the multi-viewpoint image acquired by the multi-viewpoint image acquisition unit 101 (S302). For example, the position of the face area in the luminance image stored in the internal memory of the multi-viewpoint image acquisition unit 101 is obtained by the method disclosed in Non-Patent Document 3. In this processing, for example, the positions of 25 faces (corresponding to the center position of both eyes) and the size (corresponding to the distance between both eyes) shown in FIG. 4 are obtained and stored in a memory (not shown).

  Next, the face image normalization unit 103 performs geometric conversion so that the detected face area has a predetermined arrangement and size based on the position and size of the face detected by the face detection unit 102, and The data is stored in the illustrated memory (S303). For example, conversion is performed so that the positions of the eyes are 100 × 100 pixel face images arranged at predetermined positions in the image.

  Next, the face feature point detection unit 104 detects end points of facial parts such as eyes, nose, and mouth that represent facial features from the face image normalized by the face image normalization unit 103 (S304). In the present embodiment, as shown in FIG. 5, the positions of the left and right end points of the eyes, eyes, and mouth of both eyes are detected as feature points. As a technique for detecting these feature points, for example, a method disclosed in Non-Patent Document 3 is used. Then, information on the position coordinates of the detected end point is stored in a memory (not shown).

  Next, the local feature extraction unit 105 cuts out a local region from the face image normalized by the face image normalization unit 103 based on the position of the end point detected by the face feature point detection unit 104, and extracts a face from the cut out region. A local feature amount for recognition is obtained (S305). In the present embodiment, as shown by broken lines in FIG. 6, the left and right eyes, the eyebrows, the nose, and the mouth are cut out as local areas. For example, when the left eye region is cut out, it is cut out from the face image with reference to the position coordinates of the left eye corner and eye corner. In other words, the position coordinates of the four corners of the left eye region in the face image are obtained from the position coordinates of the left eye corner and the eye corner based on a predetermined geometric relationship, and subjected to geometric transformation so that the left eye region becomes a predetermined rectangle. Cut out the area. The local area is converted so as to be a rectangular area of 30 × 30 pixels, for example.

  Then, a local feature amount is obtained from the obtained image of the local area. In this embodiment, an LBP histogram (histogram feature) is obtained as the local feature amount. As described in Non-Patent Document 4, the LBP histogram is obtained by obtaining an LBP feature in which an increase or decrease in luminance of neighboring pixels is encoded, and generating a histogram for each value encoded in a local region. . The LBP feature has characteristics that are robust to illumination variations.

  The information of the obtained local feature amount is local together with a viewpoint index (a pointer indicating any one of the imaging units in FIG. 2) and a local area index (a pointer indicating any of the local areas) in the multi-view image of the face image. It is stored in the feature storage unit 106. Similarly, local feature amounts are obtained from the face image based on the position coordinates of the end points to be referred to for other local regions. In the present embodiment, the case where the LBP histogram is used as the local feature amount has been described, but other feature amounts may be used. For example, a luminance gradient histogram, a Gabor wavelet feature amount, or the like may be used, or a feature amount combining them may be used.

  Next, it is determined whether or not the processing of S303 to S305 has been performed for all the faces in the multi-viewpoint image (S306). As a result of the determination, if all the faces have not been processed, the process returns to S303, and if all the faces have been processed, the process proceeds to the next S307.

  Next, the first similarity calculation unit 107 calculates the similarity between local features between adjacent viewpoints for one local region among the local feature amounts stored in the local feature storage unit 106. (S307). For example, when calculating the similarity for the left eye region, first, the viewpoint feature at the upper left corner of FIG. 4 and the local feature amount having the local region index of the left eye are read from the local feature storage unit 106. Then, the viewpoint index adjacent to the right of the read local feature quantity and the local feature quantity having the local region index of the left eye are read, and the similarity is calculated. In the similarity calculation, a histogram intersection of the LBP histogram is calculated. Similarly, the lower side view index and the local feature quantity having the left eye local area index are read, and the similarity is calculated.

  In this embodiment, since the LBP histogram is used as the local feature quantity, the histogram intersection is set as the similarity, but the similarity may be obtained by other methods such as normalized correlation. These processes are repeated for local features between all adjacent viewpoints, and the obtained similarities are sequentially stored in a memory (not shown).

  Next, the labeling unit 108 labels the viewpoints of the multi-viewpoint image based on the similarity calculated by the first similarity calculation unit 107 (S308). For example, assuming that the label at the upper left viewpoint in FIG. 4 is L1, the same label is attached when the similarity calculated in S307 is larger than a predetermined value, and a new label (for example, L2) is attached when different. These processes are repeated for local features between all adjacent viewpoints. FIG. 7 shows an example of the labeling result. Then, the obtained labeling result is stored in a memory (not shown).

  Next, the local feature integration unit 109 integrates the local feature amounts stored in the local feature storage unit 106 based on the result of the labeling unit 108 (S309). In the present embodiment, the local feature quantity labeled with the same label by the labeling unit 108 is read from the local feature storage unit 106 and integrated. In the integration process, a plurality of LBP histograms are added for each histogram bin, and normalized by the number of local feature amounts to be integrated. This integration process is repeated for all the labels attached in S308. As described above, in S307 to S309, the similarity of the local feature amount is calculated between the adjacent viewpoints using the viewpoint index, so that it is possible to accurately label similar local features and integrate the local feature amounts. Good and efficient.

  Next, it is determined whether or not the above-described processing of S307 to S309 has been performed on all local regions (S310). If the result of this determination is that processing has not been performed for all local regions, the process returns to S307. On the other hand, if processing has been performed for all local regions, the process proceeds to the next S311.

  Thus, by performing the processing of S307 to S309 for each local area, it is possible to label the viewpoints of the multi-viewpoint images corresponding to each area, and it is possible to integrate the local feature amounts with high accuracy. For example, since the eye region is substantially flat with respect to the entire face, the appearance of the image in the direction close to the front face is similar. Therefore, when the center image of the multi-viewpoint image is a front face, it is expected that a label as shown in FIG. 7 is attached. On the other hand, since the nose region has a three-dimensional structure with respect to the entire face, the appearance of the image differs even in the direction close to the front face. Therefore, it is expected that different labels are attached even to the center image of the multi-viewpoint image. However, in the direction close to the side, the appearance is similar, so the labels are the same, and it can be expected that the features can be integrated. That is, the labeling result depends on the shape of the target object or object region.

  Next, the multi-view face feature registration unit 110 registers the local feature amount integrated by the local feature integration unit 109 for each local region (S311). In this embodiment, the local feature amount integrated for each label and each label is registered. Further, the labeling result obtained by the labeling unit 108 as shown in FIG. 7 is also registered. That is, the integrated local feature amount is associated with the viewpoint position of the multi-viewpoint image.

FIG. 8 is a flowchart illustrating an example of a processing procedure for collating with a multi-viewpoint image (registered image) registered by inputting a multi-viewpoint image in the present embodiment.
First, the multi-viewpoint image acquisition unit 101 acquires a multi-viewpoint image (S801). The processing here is the same as S301 in FIG. Next, a local feature amount of the face is acquired from the multi-viewpoint image (S802). The processing here is the same as S302 to S309 in FIG.

  Next, the second similarity calculation unit 111 calculates the similarity between the local feature amount of the face acquired in S802 and the local feature amount registered in the multi-view face feature registration unit 110 for each local region ( S803). For example, when the similarity is calculated for the left eye region, the local feature amount having the local region index of the left eye is acquired from the processing result of S802 and acquired from the multi-view face feature registration unit 110.

  For example, first, the local feature amount with the label L1 and the local feature amount with the label L1 registered in the multi-viewpoint face feature registration unit 110 are acquired in the labeling step of S802, and the similarity is calculated. In the calculation of the similarity, the histogram intersection of the LBP histogram is calculated in the same manner as the method performed by the first similarity calculation unit 107. Similarly, similarity is calculated for all label combinations. At this time, since the local feature amounts are integrated by the local feature integration unit 109, the number of combinations is reduced to each stage as compared with all combinations of the number of viewpoints that are not integrated. Moreover, since the local feature-value is integrated based on the similarity calculated by the 1st similarity calculation part 107, the precision of the similarity calculated here is hold | maintained.

  Furthermore, when the viewpoint position of the input multi-view image and the viewpoint position of the registered multi-view image are separated by a predetermined value or more, the calculation of the similarity can be omitted. For example, when the viewpoint position of the input multi-viewpoint image is the upper left corner (corresponding to the imaging unit 201 in FIG. 2) and the viewpoint position of the registered multi-viewpoint image is the lower right corner (corresponding to the imaging unit 225 in FIG. 2). is there. By associating the integrated local feature quantity with the position of the multi-viewpoint image, it is possible to determine a situation with a low possibility of matching such appearances. The obtained similarities are sequentially stored in a memory (not shown). The above processing is repeated for each local region.

  Next, the similarity integration unit 112 integrates the similarities calculated by the second similarity calculation unit 111 and outputs the similarities between the multi-viewpoint images (S804). First, the similarity is integrated for each local region. For example, among the similarities calculated in step S803 for the left eye region, the highest similarity is set as the similarity of the left eye region. Even if there is a difference between the orientation of the face at the time of input and the orientation of the face at the time of registration, it can be expected that the orientations coincide in any of the imaging units of the multi-viewpoint image acquisition unit 101. Extracting the similarity between images with matching face orientations by finding the maximum similarity from the combination of multiple viewpoints between the input multi-view image and the registered multi-view image Can improve accuracy. Similarly, similarities in other local regions are integrated. Then, by summing up the obtained similarities for each local region, the similarities between the multi-viewpoint images are integrated.

  Next, the recognition unit 113 determines whether the person of the input multi-view face image is the same person as the person of the registered multi-view face image based on the similarity integrated by the similarity integration unit 112. (S805). In this process, when the degree of similarity between multi-viewpoint images is a predetermined value or more, it is determined that they are the same person.

  In the above description, the case where there is one registered person has been described. However, the present invention can also be applied to a case where there are a plurality of registered persons. In this case, the above-described steps S803 to S805 are repeated for each registered person's face image, and when there are a plurality of registered faces whose similarity is equal to or greater than a predetermined value, the registered person having the maximum similarity is determined as the corresponding person. .

  The embodiment applied to personal identification of the face has been described above. As described above, in the present embodiment, local feature amounts are integrated from similarities in multi-view images of adjacent viewpoints for each local region, and the similarity between multi-view images is obtained using the integrated local feature amounts. I made it. Therefore, in the collation between multi-viewpoint images, it is possible to suppress an increase in the number of similarity calculations while maintaining recognition accuracy. In the present embodiment, local feature amounts are integrated in the multi-view image to be registered and the input (determined) multi-view image, respectively, and collation between the multi-view images is performed. Local feature amounts may be integrated using only multi-viewpoint images.

  Further, in the present embodiment, the similarity between local feature amounts extracted from images of adjacent viewpoints among multi-viewpoint images is calculated, and the local feature amounts are integrated based on the similarity. It is not limited to the viewpoint. For example, when a multi-viewpoint image is acquired by a multi-view imaging apparatus having 100 × 100 imaging units, it is not always necessary to perform feature extraction or similarity calculation using all viewpoint images. In that case, integration of local feature amounts is performed from an image of a near viewpoint as much as possible.

  In the present embodiment, face recognition has been described as an example, but the present invention can also be applied to identification of a specific object other than a face. In particular, it functions effectively even for an object such as a face, whose appearance changes with respect to an angle change in the shooting direction. In particular, in the present embodiment, since local feature amounts are integrated based on the similarity in the multi-viewpoint image, it is possible to appropriately integrate the local feature amounts even when the shape or orientation of the target is not known in advance. it can.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

101 Multi-viewpoint image acquisition unit 105 Local feature extraction unit 107 First similarity calculation unit 109 Local feature integration unit 111 Second similarity calculation unit 113 recognition unit

Claims (8)

Obtaining means for obtaining a multi-viewpoint image including a plurality of single-viewpoint images having different imaging directions of the subject;
Extraction means for extracting feature amounts from the plurality of single viewpoint images,
First similarity calculation means for calculating a similarity between the respective feature amounts extracted by the extraction means;
Integration means for integrating the feature quantities based on the similarity calculated by the first similarity calculation means and the viewpoint position of the multi-viewpoint image;
Second similarity calculation means for calculating the similarity between the feature quantity integrated by the integration means and the feature quantity of the registered image;
An image recognition apparatus comprising: recognition means for recognizing the subject based on the similarity calculated by the second similarity calculation means. The first similarity calculation means calculates a similarity between feature quantities extracted from a single viewpoint image of a nearby viewpoint among the multi-viewpoint images,
The image recognition apparatus according to claim 1, wherein the integration unit integrates feature quantities extracted from a single viewpoint image of a nearby viewpoint among the multi-viewpoint images. The extraction means extracts a plurality of local regions from the single viewpoint image and extracts feature amounts from the local regions,
The first similarity calculation means calculates a similarity for each local region,
The integration unit integrates feature quantities for each local region,
The image recognition apparatus according to claim 1, wherein the second similarity calculation unit calculates a similarity based on the integrated feature amount for each local region.   The image recognition apparatus according to claim 1, further comprising a registration unit configured to register the integrated feature amount in association with a viewpoint position of the multi-viewpoint image.   The image recognition apparatus according to claim 1, wherein the feature amount is a histogram feature.   The image recognition apparatus according to claim 1, wherein the single viewpoint image is a face image. An acquisition step of acquiring a multi-viewpoint image including a plurality of single-viewpoint images having different imaging directions of the subject;
An extraction step of extracting feature amounts from the plurality of single viewpoint images,
A first similarity calculation step of calculating a similarity between the respective feature amounts extracted in the extraction step;
An integration step of integrating the feature quantities based on the similarity calculated in the first similarity calculation step and the viewpoint position of the multi-viewpoint image;
A second similarity calculation step of calculating the similarity between the feature amount integrated in the integration step and the feature amount of the registered image;
An image recognition method comprising: a recognition step of recognizing the subject based on the similarity calculated in the second similarity calculation step. An acquisition step of acquiring a multi-viewpoint image including a plurality of single-viewpoint images having different imaging directions of the subject;
An extraction step of extracting feature amounts from the plurality of single viewpoint images,
A first similarity calculation step of calculating a similarity between the respective feature amounts extracted in the extraction step;
An integration step of integrating the feature quantities based on the similarity calculated in the first similarity calculation step and the viewpoint position of the multi-viewpoint image;
A second similarity calculation step of calculating a similarity with a registered image based on the feature amount integrated in the integration step;
A program causing a computer to execute a recognition step of recognizing the subject based on the similarity calculated in the second similarity calculation step.

Images