JP4259365B2 - Image recognition apparatus and image recognition method - Google Patents

Description

  The present invention relates to an image recognition apparatus and an image recognition method for recognizing a recognition object from a captured image.

In order to use for personal authentication and the like, an apparatus for capturing a human face and recognizing the human face from the captured image has been developed. As a face recognition device, for example, a reference image of a plurality of facial features (eyes, nose, mouth, etc.) is prepared, and a portion similar to the characteristic portion is searched from the captured image using the reference image. There is a technique for recognizing a face by detecting a position with a high degree as a candidate position of each feature portion and pattern matching the relative positional relationship (see Patent Document 1).
JP 2000-99722 A

  However, in the face recognition device, if a position with a high degree of similarity is shifted even a little, the combination of patterns at relative positions increases. In particular, if the positions of a plurality of feature portions are shifted, the combination of patterns becomes enormous. . As a result, the processing load increases and the processing time becomes longer.

  Therefore, an object of the present invention is to provide an image recognition apparatus and an image recognition method that reduce the processing load.

An image recognition apparatus according to the present invention includes an imaging unit, a reference image holding unit that holds a reference image of a plurality of characteristic parts of a recognition target, a position of the reference image in the entire image corresponding to the reference image, A positional relationship holding unit that holds a positional relationship with an arbitrary point, a comparison unit that compares a captured image captured by the imaging unit with a plurality of reference images held by the reference image holding unit, and a positional relationship holding Based on the positional relationship held by the means, the plurality of comparison results by the comparison means are respectively moved with reference to any point in the image, the plurality of moved comparison results are integrated, and based on the integration result And determining means for determining whether the object is a recognition object.

  In this image recognition apparatus, the reference image holding means holds a plurality of reference images of parts that are characteristics of the recognition target (for example, eyes, nose, mouth, etc. when the recognition target is a human face) For each reference image held in the relationship holding means, the reference image holds the positional relationship between the position in the whole image and any point in the image (for example, the center point, the upper left point, the lower right point). In the image recognition apparatus, a certain object is imaged by an imaging unit, and a captured image is acquired. In the image recognition apparatus, the comparison unit compares the captured image with a plurality of reference images, and acquires a comparison result for each reference image. In this comparison, a region of the size of each reference image is sequentially cut out from the captured image over the entire image while shifting in the vertical direction and the horizontal direction, and the cut out many regions are compared with the reference image. Therefore, as a comparison result, a result is obtained for each reference image with respect to the position of each region corresponding to the number of regions cut out from the captured image. When there is an area similar to the reference image (feature part) in the captured image, the similarity to the position of the area is high, and in the area not similar to the reference image in the captured image, the similarity to the position of the area Becomes lower. Further, in the image recognition apparatus, the determination unit moves the comparison result for each reference image based on the positional relationship corresponding to the reference image, and determines whether the object is a recognition object based on the plurality of moved comparison results. To do. When a certain captured object has a feature portion similar to each reference image, the comparison result of each reference image indicates that the reference image is located in the entire image (the feature portion that is the reference image is The degree of similarity around the existing position) is high. In this case, when each comparison result is moved with reference to an arbitrary point of the image, a position having a high similarity in each comparison result gathers at the arbitrary point. Therefore, when a result indicating that the degree of similarity is high at an arbitrary point is obtained in the plurality of moved comparison results, since there are a plurality of reference image feature parts, the captured object is It can be determined that the object is a recognition object. In this way, in this image recognition apparatus, the reference images and captured images of a plurality of characteristic portions are respectively compared, and the comparison results are simply moved and integrated, so that the processing load (processing time) for each captured image is increased. It becomes constant. As a result, the processing load is reduced and the processing time is shortened. By the way, since a plurality of feature portions are searched as in the past and pattern matching is not performed on the relative positional relationship of the searched feature portions, the combination of patterns does not increase and the processing load does not increase. . Further, in this image recognition device, it is possible to detect whether or not there is a portion similar to the feature portion in the captured image by performing comparison with each reference image, and the obtained comparison result is obtained from the geometric position. By integrating based on the relationship, it is possible to detect whether or not the position of the portion similar to the characteristic portion in the captured image is matched, and thus the recognition accuracy is high. In addition, in this image recognition apparatus, even when there is noise in each comparison result (for example, when there are other regions in the captured image that have high similarity to the reference image), the plurality of comparison results are moved. The noise is suppressed. In this image recognition apparatus, the recognition accuracy increases as the number of reference images (that is, the number of feature portions to be compared) is increased.

  The image recognition apparatus of the present invention includes an image size conversion unit that converts the size of a captured image captured by the imaging unit, and the comparison unit holds the captured image converted by the image size conversion unit and the reference image storage unit. A plurality of reference images may be compared with each other.

  In this image recognition apparatus, the size of the captured image is converted by the image size conversion unit, and the comparison unit performs comparison using the captured image of the converted size. Therefore, it is possible to cope with a change in the size of the object on the captured image and to compare the captured image with the reference image. In addition, when the size of the captured image is reduced, the processing load is further reduced and noise can be reduced.

An image recognition method according to the present invention is an image recognition method for recognizing a recognition object from a captured image, and the reference image of the plurality of characteristic parts of the recognition object and the reference image in the entire image of the reference image A comparison step in which a positional relationship between a position and an arbitrary point in the image is held in advance and the captured image is compared with a plurality of held reference images, respectively, and a plurality of comparison steps based on the held positional relationship. comparison result of moving each relative to the arbitrary point in the image, by integrating the plurality of comparison results obtained by the mobile, to include a determination step of determining whether or not the recognition target object based on the integration result It is characterized by.

  The image recognition method of the present invention includes an image size conversion step for converting the size of the captured image. In the comparison step, the captured image that has been subjected to the size conversion in the image size conversion step is compared with a plurality of held reference images. It is good also as composition to do.

  Each of the above-described image recognition methods has the same effects as the above-described image recognition device.

  According to the present invention, the processing load for recognizing a recognition target object can be reduced.

  Embodiments of an image recognition apparatus and an image recognition method according to the present invention will be described below with reference to the drawings.

  In the present embodiment, the present invention is applied to an image recognition apparatus targeting a person or a car. In the image recognition apparatus according to the present embodiment, when a person is targeted, it is determined whether the recognition target is a face and whether it is a person's face. It is a style and it is discriminated whether it is a certain vehicle type. The image recognition apparatus according to the present embodiment has the same configuration and operation for any one of a person and a car, but the distance (offset amount) between the reference image of the characteristic part, the reference image, and the center of gravity of the image. Are held according to the target. In the present embodiment, first, the configuration of the image recognition device when applied to human face recognition will be described, then the configuration when applied to vehicle type recognition of the automobile will be described, and finally, the image recognition device The operation of No. 1 will be described.

  With reference to FIGS. 1-7, the structure of the image recognition apparatus 1 at the time of applying to a person's face recognition is demonstrated. FIG. 1 is a configuration diagram of an image recognition apparatus according to the present embodiment. FIG. 2 is an explanatory diagram of the inter-image similarity evaluation unit of FIG. FIG. 3 is an example of a reference image when the recognition target is a person's face, (a) is a person's face image for creating a reference image, and (b) is the face image of FIG. Is a reference image having the eyes, nose, and mouth as features. FIG. 4 is an example of a captured image obtained by capturing a person's face from the front. (A) is a captured image, and (b) is a captured image showing a result of recognizing the face. FIG. 5 is a similarity evaluation map for recognizing a face from the captured image of FIG. 4 based on the reference image of FIG. 3, (a) is a similarity evaluation map for the reference images of both eyes, and (b) Is the similarity evaluation map for the nose reference image, and (c) is the similarity evaluation map for the mouth reference image. FIG. 6 is an offset similarity evaluation map obtained by offsetting the similarity evaluation map of FIG. 5 to the center of gravity, (a) is an offset similarity evaluation map for the reference images of both eyes, and (b) is for the reference image of the nose. It is an offset similarity evaluation map, and (c) is an offset similarity evaluation map for a mouth reference image. FIG. 7 is an integrated map in which the three offset similarity evaluation maps of FIG. 6 are integrated.

  The image recognition apparatus 1 is a face recognition apparatus that determines whether or not a face of a person is a captured image obtained by capturing an object. The image recognition apparatus 1 holds offset amounts from the center of gravity of the reference images of both eyes, nose, and mouth of a person's face and the image of the entire face to the center of each reference image. Then, the image recognition device 1 determines whether or not both eyes, nose, and mouth that are similar to the three reference images in a captured image obtained by capturing an object are present at the correct positions. For this purpose, the image recognition apparatus 1 includes a camera 2 and an image ECU [Electronic Control Unit] 3. The image ECU 3 includes a reference image database 10, an offset database 11, an image resolution conversion unit 12, an inter-image similarity evaluation unit 13, and an integration. A processing unit 14 is configured. In this embodiment, the data of each image is handled in pixel units, and the coordinate system is (x, y) in pixel units.

  In the present embodiment, the camera 2 corresponds to the imaging means described in the claims, the reference image database 10 corresponds to the reference image holding means described in the claims, and the offset database 11 claims. The image resolution converting unit 12 corresponds to the image size converting unit described in the claims, and the inter-image similarity evaluation unit 13 is compared in the claims. The integrated processing unit 14 corresponds to a determination unit described in the claims.

  The camera 2 is, for example, a CCD [Charge Coupled Device] camera. The camera 2 captures an object and acquires a captured color image (for example, an image of RGB [Red Green Blue]). When recognizing a person's face, the camera 2 images the person's face from the front. For example, a captured image as shown in FIG. The camera 2 transmits the captured image data to the image ECU 3. Although the camera 2 is color, it is sufficient that at least luminance information can be obtained, so a monochrome camera may be used.

  The image ECU 3 includes a CPU [Central Processing Unit], a ROM [Read Only Memory], a RAM [Random Access Memory], and the like, and each processing unit and each database of the image recognition apparatus 1 are configured. The image ECU 3 holds reference images of a plurality of characteristic portions of the recognition target object and offset amounts of the respective reference images. In the image ECU 3, captured image data is taken from the camera 2, the same area as each reference image is cut out over the entire captured image for each reference image, and the similarity indicating the similarity between the number of cut out areas and the reference image Generate an evaluation map. Further, the image ECU 3 generates an offset similarity evaluation map obtained by offsetting the similarity evaluation map for each reference image by an offset amount, and whether or not the object is a recognition target from an integrated map obtained by integrating a plurality of offset similarity evaluation maps. Determine whether.

  The reference image database 10 is constructed in the ROM and stores a plurality of reference image data. The reference image is an image of a part that is a feature of the recognition target object, and is an image of both eyes, nose, and mouth because the recognition target object is a face. Other than this, reference images of other characteristic portions such as ears and eyebrows may be used. In this embodiment, instead of recognizing the face of a specific person, it is determined whether or not it is a person's face. Therefore, as a face image for creating a reference image, a face of a general person is captured. A captured image may be used, or face images of a large number of persons may be collected and a face image of a person composed of both eyes, nose, and mouth having an average shape and size may be created. If the camera 2 cannot always capture a face that faces the front, an average face image of faces in various directions may be created, and a reference image may be created from the average face image. In addition, since the face has various expressions, an average face image of faces having various expressions may be created, and a reference image may be created from the average face image.

  In order to create a reference image, for example, a face image as shown in FIG. In this face image, the face is the smallest rectangle in contact with the face including both eyes, nose and mouth, and a face area in which the four sides of the rectangle are parallel to the vertical or horizontal axis of the image is set (broken line in FIG. 3A). Rectangle area). Further, the center of the face area is set as the center of gravity Cen_B (= (0, 0)) (circle mark in FIG. 3A). Further, from this face image, a rectangle including both eyes, a rectangle including the nose, and a rectangle including the mouth, and a region in which the four sides of each rectangle are parallel to the vertical axis or the horizontal axis of the image is cut out (FIG. 3B). 3 rectangular regions indicated by broken lines). The center of each region is defined as a feature point P_B_i (i = 0, 1, 2) (three crosses in FIG. 3B). Each image partitioned by the cut-out rectangular area is a reference image, and the data of the reference image is stored in the reference image database 10. The reference image data is a luminance value for each pixel. Note that it is desirable that the size of the reference image has a certain size since noise is increased in the similarity evaluation map if this size is small.

The offset database 11 is constructed in the ROM and stores an offset amount for each reference image. The offset amount is a deviation between the center of gravity Cen_B of the face area and each feature point P_B_i, and is represented by ΔP_B_i (x, y). As the offset amount, an average value of the distance between the center of gravity Cen_B of the face region and each feature point P_B_i is set. That is, positions where feature points of many human faces exist are collected, and each distance is obtained using an average value of the positions. The offset database 11 stores an offset amount ΔP_B_i (i = 0, 1, 2) in association with each reference image.

  As shown in Expression (1), if the offset amount ΔP_B_i is known, the center of gravity Cen_B can be obtained if each feature point P_B_i is determined.

  The image resolution conversion unit 12 converts the size (resolution) of the captured image into an arbitrary size in order to match the resolution of the object on the captured image with the resolution of the reference image. In this embodiment, since the similarity is calculated using the luminance value, the image resolution conversion unit 12 extracts the luminance value of each pixel from the captured image data, and converts the size of the luminance value for each pixel. A captured image is generated. As a size conversion method, for example, the luminance value of each pixel on the converted image is determined by linear interpolation, or pixels are thinned out when the image size is reduced.

The inter-image similarity evaluation unit 13 sequentially extracts reference images from the reference image database 10. Then, in the image similarity evaluation unit 13, for each reference image, the center point (xc, yc) of the clipped region is shifted by one pixel in the x direction or the y direction, and is the same as the reference image from the size-converted captured image. A region of the size of is cut out (see FIG. 2). Further, the inter-image similarity evaluation unit 13 calculates the absolute value of the difference in luminance value for each pixel at the corresponding position by using the expression (2) between the image of each cut-out region and the reference image. The sum of absolute value differences for pixels is calculated.

  In Expression (2), T (x, y) is the luminance value of the pixel at the position (x, y) of the captured image, and S (x, y) is the pixel at the position (x, y) of the reference image. Luminance value. Assuming that the size of the reference image is m pixels in the x direction and n pixels in the y direction, the absolute values of m × n differences are calculated, and m × n absolute value differences are integrated. Therefore, the higher the similarity between the image of the cutout area and the reference image (the more correlation there is), the smaller the absolute value difference sum value, and when the image of the cutout area matches the reference image, the absolute value difference summation The value is 0. In addition, although the luminance value was used, you may calculate using the other value of an image. For example, saturation values and hues may be used, or R values, G values, and B values of RGB images may be used. When a value other than the luminance value is used, a reference image is created according to the value to be used, and the image resolution conversion unit 12 generates a captured image whose size has been converted.

  Further, the inter-image similarity evaluation unit 13 subtracts the absolute value difference sum value from the maximum luminance value to obtain a luminance value indicating the similarity between the image of each cut-out region and the reference image. Therefore, the higher the degree of similarity between the image in the cutout region and the reference image, the larger the luminance value. When the image in the cutout region matches the reference image, the luminance value becomes the maximum value. Then, the inter-image similarity evaluation unit 13 sets the luminance value to the coordinates (xc, yc) that are the center of the cut out region, and generates the similarity evaluation map Map_B_i (see FIG. 2). The similarity evaluation map Map_B_i is generated for each reference image, and is a map in which luminance values that are similarities to the reference image are arranged in each pixel at the center point (xc, yc) of all cutout regions. When the similarity evaluation map Map_B_i is displayed on the screen, the higher the similarity is, the larger the luminance value becomes, and the portion becomes white. In addition, in the similarity evaluation map Map_B_i, when there is a feature portion similar to the reference image in the captured image, the similarity is a peak in the vicinity where the feature portion is located, and thus the luminance value around the feature point P_B_i increases. .

  The inter-image similarity evaluation unit 13 generates, for example, a similarity evaluation map Map_B_i (i = 0, 1, 2) as shown in FIG. FIG. 5A shows the similarity evaluation map Map_B_0 of the reference images of both eyes, and the luminance value around the feature point P_B_0 is large. FIG. 5B shows the similarity evaluation map Map_B_1 of the nose reference image, and the luminance value around the feature point P_B_1 is large. FIG. 5C displays a similarity evaluation map Map_B_2 of the mouth reference image, and the luminance value around the feature point P_B_2 is large. In FIGS. 5A to 5C, there are places where the luminance value becomes large (white) other than the feature points, but these places are noise. By the way, in the case of a face, the background and skin images have the same brightness value, but the brightness value changes at the facial features and the border between the background and the face. When calculating, similarity may appear, and this becomes noise.

  The integration processing unit 14 extracts the offset amount ΔP_B_i corresponding to the reference image from the offset database 11. Then, the integration processing unit 14 offsets each pixel of the similarity evaluation map Map_B_i by the offset amount ΔP_B_i to generate an offset similarity evaluation map ΔMap_B_i. The offset similarity evaluation map ΔMap_B_i is generated for each reference image, and is a map obtained by moving the similarity evaluation map ΔMap_B_i so that the feature point P_B_i of each reference image is located at the center of gravity Cen_B. In the offset similarity evaluation map ΔMap_B_i, when there is a feature portion similar to the reference image in the captured image, the similarity peak moves from each feature point P_B_i by the offset amount ΔP_B_i, so the luminance value around the center of gravity Cen_B increases. . Note that the offset similarity evaluation map ΔMap_B_i has the same size as the similarity evaluation map Map_B_i, and 0 is set as the luminance value for pixels for which no luminance value is set from the similarity evaluation map Map_B_i.

For example, the integration processing unit 14 generates an offset similarity evaluation map ΔMap_B_i (i = 0, 1, 2) as shown in FIG. 6A to 6C show the respective offset similarity evaluation maps ΔMap_B_i obtained by offsetting the similarity evaluation map Map_B_i for each reference image shown in FIG. 5 by the offset amount ΔP_B_i indicated by the arrows. The luminance value around the center of gravity Cen_B is large. In the offset similarity evaluation map ΔMap_B_i, noise is left because the similarity evaluation map Map_B_0 is merely offset.

  Further, the integration processing unit 14 adds the luminance values of all the offset similarity evaluation maps ΔMap_B_i for each corresponding pixel according to the expression (3) to generate an integrated map T_Map. The integrated map T_Map is a map in which all the offset similarity evaluation maps ΔMap_B_i are integrated. In the integrated map T_Map, when a portion having a large luminance value matches in all the offset similarity evaluation maps ΔMap_B_i, the luminance value at that portion increases. Therefore, in the integrated map T_Map, when there are all feature portions similar to the plurality of reference images in the captured image, the similarity peaks around the centroid Cen_B are integrated, so the luminance value around the centroid Cen_B becomes the largest. . Further, in the integrated map T_Map, even if noises (locations with large luminance values) exist in different locations in each offset similarity evaluation map ΔMap_B_i, they are canceled out by integrating them, and noise is reduced. For example, the integration processing unit 14 generates an integration map T_Map as shown in FIG. FIG. 7 shows an integrated map T_Map in which the three offset similarity evaluation maps ΔMap_B_i shown in FIG. 6 are integrated. Only the luminance value around the center of gravity Cen_B is increased, and noise is eliminated.

  Then, the integration processing unit 14 compares the luminance value of each pixel around the center of gravity Cen_B of the integrated map T_Map with a threshold value. If the luminance value is larger than the threshold value, it is determined that a face exists in the captured image. In the following cases, it is determined that no face exists in the captured image. FIG. 4B shows a determination result that a face is present in the captured image of FIG. 4A, and the face area including the feature portion is drawn with a solid line rectangle.

  Next, with reference to FIG. 1 and FIGS. 8 to 12, the configuration of the image recognition device 1 when applied to vehicle type recognition of an automobile will be described. FIG. 8 is an example of a reference image when the recognition target is the back style of an automobile. FIG. 9 is an example of a captured image obtained by capturing an automobile from the rear. FIG. 10 is a similarity evaluation map for recognizing a car from the captured image of FIG. 9 based on the reference image of FIG. 8, and (a) is a similarity evaluation map for the reference image of the right tail lamp. ) Is a similarity evaluation map for the reference image of the right tire, (c) is a similarity evaluation map for the reference image of the number plate, (d) is a similarity evaluation map for the reference image of the left tail lamp, e) is a similarity evaluation map for the reference image of the left tire. 11 is an offset similarity evaluation map obtained by offsetting the similarity evaluation map of FIG. 10 to the center of gravity, (a) is an offset similarity evaluation map for a reference image of the right tail lamp, and (b) is a reference of the right tire. An offset similarity evaluation map for an image, (c) is an offset similarity evaluation map for a reference image of a number plate, (d) is an offset similarity evaluation map for a reference image of a left tail lamp, and (e) is It is an offset similarity evaluation map with respect to the reference image of the left tire. FIG. 12 is an integrated map in which the five offset similarity evaluation maps of FIG. 11 are integrated.

  The image recognition device 1 is a vehicle type recognition device that determines whether a vehicle is a recognition target vehicle type from a captured image obtained by imaging an object. The image recognition device 1 holds the left and right tail lamp parts of the back style of the automobile, the left and right tire parts, the reference image of the number plate part, and the offset amount from the center of gravity of the back style image of the automobile to the center of each reference image. . Then, the image recognition apparatus 1 determines whether left and right tail lamps, left and right tires, and number plates similar to the five reference images in a captured image obtained by imaging a certain automobile from the rear are present at the correct positions. Here, only differences from the case of recognizing a person's face in the configuration of the image recognition apparatus 1 will be described below.

  The camera 2 captures an image of the automobile from behind and transmits data of the captured image to the image ECU 3. For example, it is a captured image as shown in FIG.

  The reference image stored in the reference image database 10 is an image of the left and right tail lamps, the left and right tires, and the number plate because the vehicle type is discriminated. In the case of an automobile, the position of the tail lamp, tire, and number plate differs depending on the vehicle type, and the size and shape of the tail lamp and tire are also different. Other than this, reference images of other characteristic portions such as a bumper and a rear wing may be used. In this embodiment, since a specific vehicle type is discriminated, the back style image of the vehicle for creating the reference image is an image obtained by capturing the vehicle of that vehicle type from behind. If the camera 2 cannot always capture a car from behind, an average back style image of various back styles from various directions may be created, and a reference image may be created from the average back style image.

  In order to create the reference image, for example, a back style image of a car as shown in FIG. 8 is prepared. In this image, it is the smallest rectangle including the left and right tail lamps, the left and right tires, and the number plate, and a backstyle area in which the four sides of this rectangle are parallel to the vertical axis or the horizontal axis of the image is set (a dashed-dotted line in FIG. 8). Rectangle area). Further, the center of the backstyle area is set as the center of gravity Cen_B (= (0, 0)) (circle mark in FIG. 8). Further, from this image, there are a rectangle including a right tail lamp, a rectangle including a right tire, a rectangle including a number plate, a rectangle including a left tail lamp, and a rectangle including a left tail lamp. A region parallel to the horizontal axis is cut out (five rectangular regions indicated by broken lines in FIG. 8). The center of each region is defined as a feature point P_B_i (i = 0, 1, 2, 3, 4) (five crosses in FIG. 8). Each image partitioned by the cut-out rectangular area is a reference image, and the data of the reference image is stored in the reference image database 10.

  The offset amount ΔP_B_i (i = 0, 1, 2, 3, 4) stored in the offset database 11 is the difference between the center Cen_B of the backstyle area and each feature point P_B_i (i = 0, 1, 2, 3, 4). Deviation.

  The inter-image similarity evaluation unit 13 generates, for example, a similarity evaluation map Map_B_i (i = 0, 1, 2, 3, 4) as shown in FIG. FIG. 8A displays a similarity evaluation map Map_B_0 of the reference image of the right tail lamp, and the luminance values around the feature point P_B_0 and around the feature point P_B_3 of the left tail lamp are large. FIG. 8B shows a similarity evaluation map Map_B_1 of the reference image of the right tire, and the vicinity of the brightness value of the feature point P_B_1 and the feature point P_B_4 of the left tire is large. FIG. 8C shows the similarity evaluation map Map_B_2 of the reference image of the number plate, and the luminance value around the feature point P_B_2 is large. FIG. 8D shows a similarity evaluation map Map_B_3 of the reference image of the left tail lamp, and the luminance values around the feature point P_B_3 and around the feature point P_B_0 of the right tail lamp are large. FIG. 8E displays a similarity evaluation map Map_B_4 of the reference image of the left tire, and the luminance values around the feature point P_B_4 and around the feature point P_B_1 of the right tire are large. In FIGS. 8A to 8E, there are places where the luminance value becomes large (white) other than the feature points, but these places are noise.

  For example, the integration processing unit 14 generates an offset similarity evaluation map ΔMap_B_i (i = 0, 1, 2, 3, 4) as shown in FIG. FIGS. 11A to 11E show the respective offset similarity evaluation maps ΔMap_B_i obtained by offsetting the similarity evaluation map for each reference image shown in FIG. 10 by the offset amount ΔP_B_i indicated by the arrows. All of the luminance values around the center of gravity Cen_B are large. In the offset similarity evaluation map ΔMap_B_i, noise remains. Further, the integration processing unit 14 generates, for example, an integration map T_Map as shown in FIG. FIG. 12 shows an integrated map T_Map in which the five offset similarity evaluation maps ΔMap_B_i shown in FIG. 11 are integrated, and only the luminance value around the center of gravity Cen_B is increased, and noise is eliminated.

  The integration processing unit 14 compares the luminance value of each pixel around the center of gravity Cen_B of the integrated map T_Map with a threshold value. If the luminance value is larger than the threshold value, the vehicle of the captured image is determined as the recognition target vehicle type, and the luminance value is If it is less than or equal to the threshold value, it is determined that the vehicle in the captured image is not the vehicle model to be recognized.

  The operation in the image recognition apparatus 1 will be described with reference to FIG. Before performing recognition, depending on the object to be recognized (the face of any person or a specific vehicle model), the characteristic parts (both eyes, nose, mouth or left and right tail lamps, left and right tires, number plate) Extract multiple. Then, a reference image of the plurality of characteristic portions is created, and data of the reference image (luminance value for each pixel) is stored in the reference image database 10. Further, an offset amount ΔP_B_i between the feature point P_B_i that is the center of each feature portion and the center of gravity Cen_B that is the center of the image is obtained, and the offset amount ΔP_B_i is stored in the offset database 11 in association with the reference image.

  The camera 2 captures an image of an object (a person's face or the back style of a car), and transmits the captured image data to the image ECU 3. The image ECU 3 converts the size of the captured image in order to match the size (resolution) with the reference image. The size-converted captured image data is a luminance value. Next, the image ECU 3 extracts a reference image from the reference image database 10. Then, the image ECU 3 sequentially cuts out a region having the same size as the extracted reference image from the size-converted captured image, and calculates a luminance value indicating the similarity between the extracted region and the reference image. Further, the image ECU 3 sets the calculated luminance value to the center coordinates (xc, yc) of the extracted area, and generates a similarity evaluation map Map_B_i. The image ECU 3 generates a similarity evaluation map Map_B_i for each of a plurality of reference images. When there is a feature portion similar to the reference image in the captured image, the brightness value around the feature point P_B_i is large in the similarity evaluation map Map_B_i.

  Subsequently, the image ECU 3 extracts the offset amount ΔP_B_i corresponding to the reference image from the offset database 11. Then, in the image ECU 3, in order to place the feature point P_B_i at the center of gravity Cen_B, the similarity evaluation map Map_B_i is offset by the offset amount ΔP_B_i. The image ECU 3 generates an offset similarity evaluation map ΔMap_B_i for each of the plurality of reference images. When there is a feature portion similar to the reference image in the captured image, the luminance value around the centroid Cen_B is large in the offset similarity evaluation map ΔMap_B_i.

  Further, the image ECU 3 integrates the luminance values of all the offset similarity evaluation maps ΔMap_B_i for each pixel to generate an integrated map T_Map. When there are all feature parts similar to a plurality of reference images in the captured image, the luminance value around the center of gravity Cen_B is the largest in the integrated map T_Map. In view of this, the image ECU 3 compares the luminance value of each pixel around the center of gravity Cen_B of the integrated map T_Map with a threshold value, and if it is greater than the threshold value, determines that the object is a recognition target (an arbitrary person's face or a specific vehicle type). If it is equal to or less than the threshold, it is determined that the object is not a recognition object.

  According to the image recognition apparatus 1, the processing load (processing speed) is constant regardless of the position, shape, and size of the characteristic parts (both eyes, nose, mouth or tail lamp, tire, number plate), so the processing load is Lighter and faster processing speed. Incidentally, since the image recognition apparatus 1 does not perform pattern matching, the amount of processing does not change even if the position, shape, and size of the characteristic part of the object in the captured image are slightly different from the reference image.

  Further, the image recognition apparatus 1 can detect whether or not there is a portion similar to the feature portion in the captured image by obtaining the similarity to the reference image, and the obtained plurality of similarity evaluation maps Map_B_i By integrating with Cen_B as a reference, it is possible to detect whether or not the position of the portion similar to the characteristic portion in the captured image is correct, so that the recognition accuracy is high. Furthermore, in the image recognition apparatus 1, the recognition accuracy is improved by increasing the number of feature portions used as reference images. Further, in the image recognition device 1, even if noise appears in each similarity evaluation map Map_B_i, the noise can be suppressed by integrating them.

  Further, in the image recognition device 1, when the size of the captured image is reduced by conversion, the processing load can be further reduced, and noise in the captured image can be suppressed. Further, in the image recognition apparatus 1, since the similarity is indicated by the luminance value, the similarity can be visually determined.

  As mentioned above, although embodiment which concerns on this invention was described, this invention is implemented in various forms, without being limited to the said embodiment.

  For example, in the present embodiment, the present invention is applied to the recognition of a person or a car, but it can be applied to the recognition of various other things. In the case of recognizing a person, the present invention can also be applied to personal authentication for authenticating a specific individual. In this case, a reference image of a feature portion is created from a face image obtained by capturing the face of the recognized individual. Further, the present invention can be applied to various recognitions such as a specific race, a specific age group, men and women, and a reference image is prepared for each recognition. For example, when a certain race is recognized, a reference image is created from an average face image of that race. When recognizing an automobile, the present invention can be applied to recognition in a large framework such as a small car, a medium car, and a large car. In this case, average values of automobiles of various sizes are used for the reference image and the offset amount.

  Further, in the present embodiment, the point serving as the reference for the offset amount is set at the center of gravity (center) of the image, but may be set at any position in the image such as the upper left corner or the lower right corner of the image. In this case, the offset amount is obtained for the set position. Incidentally, since the similarity in the offset similarity evaluation map and the integrated map is high around the set position, the pixel to be compared with the threshold needs to be a pixel around the set position.

  In the present embodiment, the resolution (size) of the captured image is converted. However, when the resolution of the target object and the reference image in the captured image can be matched, the resolution of the captured image may not be converted. .

  In this embodiment, the similarity to the reference image is set as the luminance value, but the absolute value difference sum value itself may be used as the similarity instead of the luminance value, or the similarity to the reference image may be determined by another method. You may ask for it.

It is a block diagram of the image recognition apparatus which concerns on this Embodiment. It is explanatory drawing of the similarity evaluation part between images of FIG. It is an example of a reference image when the recognition target is a person's face, (a) is a person's face image for creating a reference image, and (b) is both eyes and nose in the face image of FIG. The reference image has a mouth as a characteristic part. It is an example of the captured image which imaged the face of a certain person from the front, (a) is a captured image, (b) is a captured image which shows the result of having recognized the face. 4 is a similarity evaluation map for recognizing a face from the captured image of FIG. 4 based on the reference image of FIG. 3, (a) is a similarity evaluation map for the reference images of both eyes, and (b) is a nose reference. It is a similarity evaluation map for images, and (c) is a similarity evaluation map for mouth reference images. 5 is an offset similarity evaluation map obtained by offsetting the similarity evaluation map of FIG. 5 to the center of gravity, (a) is an offset similarity evaluation map for the reference images of both eyes, and (b) is an offset similarity evaluation for the reference images of the nose. (C) is an offset similarity evaluation map for the mouth reference image. 7 is an integrated map obtained by integrating the three offset similarity evaluation maps of FIG. 6. It is an example of the reference image at the time of making recognition object the back style of a car. It is an example of the captured image which imaged a certain automobile from back. 9 is a similarity evaluation map for recognizing a car from the captured image of FIG. 9 based on the reference image of FIG. 8, (a) is a similarity evaluation map for the reference image of the right tail lamp, and (b) is a right tire. (C) is a similarity evaluation map for the reference image of the number plate, (d) is a similarity evaluation map for the reference image of the left tail lamp, and (e) is the left side. It is a similarity evaluation map with respect to the reference image of a tire. 10 is an offset similarity evaluation map obtained by offsetting the similarity evaluation map of FIG. 10 to the center of gravity, (a) is an offset similarity evaluation map for a reference image of the right tail lamp, and (b) is an offset similarity for a reference image of the right tire. (C) is an offset similarity evaluation map for the reference image of the number plate, (d) is an offset similarity evaluation map for the reference image of the left tail lamp, and (e) is a reference of the left tire. It is an offset similarity evaluation map with respect to an image. 12 is an integrated map obtained by integrating the five offset similarity evaluation maps in FIG. 11.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image recognition apparatus, 2 ... Camera, 3 ... Image ECU, 10 ... Reference image database, 11 ... Offset database, 12 ... Image resolution conversion part, 13 ... Inter-image similarity evaluation part, 14 ... Integration processing part

Claims (4)

Imaging means;
Reference image holding means for holding reference images of a plurality of characteristic parts of the recognition object;
A positional relationship holding means for holding a positional relationship between the position of the reference image in the entire image and an arbitrary point in the image corresponding to the reference image;
A comparison unit that compares the captured image captured by the imaging unit with a plurality of reference images held by the reference image holding unit;
Based on the positional relationship held by the positional relationship holding unit, each of the plurality of comparison results by the comparison unit is moved with reference to an arbitrary point in the image, and the plurality of comparison results moved are integrated, An image recognition apparatus comprising: a determination unit that determines whether the object is a recognition object based on the integration result . Image size conversion means for converting the size of the captured image captured by the imaging means;
The image recognition apparatus according to claim 1, wherein the comparison unit compares the captured image whose size has been converted by the image size conversion unit with a plurality of reference images held by the reference image holding unit. . An image recognition method for recognizing a recognition object from a captured image,
Corresponding to the reference image and the reference image of the plurality of characteristic parts of the recognition target object, the positional relationship between the position of the reference image in the entire image and an arbitrary point in the image is held in advance.
A comparison step of comparing the captured image and the plurality of held reference images respectively;
Based on the held positional relationship, the plurality of comparison results in the comparison step are moved with reference to any point in the image, the plurality of moved comparison results are integrated, and based on the integration result And a determination step of determining whether the object is a recognition object. Including an image size conversion step of converting the size of the captured image;
4. The image recognition method according to claim 3, wherein in the comparison step, the captured image whose size has been converted in the image size conversion step is compared with the plurality of held reference images.

Images