JPH09128542A - Image recognision and retrieval device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To stably extract an object even when the object is somewhat hidden or the direction of the object is fluctuated without utilizing any shape information by collating the object while utilizing the local color histogram of attentional area, and repeating this processing while changing its position or the like. SOLUTION: In an object learning device 1, an object area extracting device 4 extracts the area of the object from the image of a registered object while using a window function, etc., and a color histogram preparing device 5 calculates the color histogram to be used as a dictionary on the stage of recognition while using the picture elements in the extracted partial area. At an object retrieving device 2, the partial image in the specified area (concerned area) is extracted while using the same window function as the time of learning and the color histogram is calculated while using the color values of these picture elements. The color histograms are successively calculated while scanning the concerned areas and further, this processing is repeated while changing the size of the input image. Then, a distance measuring instrument 10 calculates the value of distance with the histogram at the time of learning, and when that value is less than a certain threshold value, the area is defined as the learnt object.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image retrieval system for searching an image including an object registered in advance from an image and detecting where the object is located in the image, and in particular, it is specified by an automatic monitoring device. To detect the location of
The present invention relates to an image recognition search device that searches an image database for an image containing a specific object.

[0002]

2. Description of the Related Art Conventionally, two types of methods have been proposed for a device for extracting a region of a target object from an image. The first method is a method of using the shape information of the object. As one example, there is a method of extracting geometrical partial features such as corners and ridgelines and extracting the position of the object from the positional relationship. Another method using shape information is a method in which a two-dimensional template is prepared and the partial image of the input image is correlated to detect a region having a large correlation value as the region of the object.

On the other hand, the second method is a method of utilizing the attribute of an object. For example, the red color is searched for. In addition, a method of checking whether or not an object is included in an image based on the similarity using a color histogram of the entire image has been proposed.

[0004]

However, the first method is still in the research stage because it is difficult to accurately extract geometric features such as corners and ridges from a general image with a lot of noise.

Further, in the method utilizing the correlation, the apparent image of a three-dimensional object largely changes depending on the viewing direction and the like, so that the accuracy cannot be high. Although a method of storing all cases as templates for changes in the orientation and size of an object has been proposed, it is difficult to handle all cases in which a part of an object is hidden by another object.

[0006] In the second method, it is difficult for the method using a single attribute of an object such as a red object to stably detect an object composed of a plurality of colors. The method using the color histogram of the entire image cannot specify where the object is in the image. As a method improved on this method, for example, a method of dividing an image into 8 × 8 and calculating a color histogram in each area can be considered, but with this method, an object is stably extracted against changes in the size and position of the object. Is difficult to do.

The present invention has been made in order to solve the above problems, and its purpose is to stably extract an object against some hiding or a change in the direction of the object without using shape information. An object is to provide an image recognition search device.

[0008]

In order to achieve the above-mentioned object, in an image retrieval apparatus for finding an area of an object registered in advance from an image including a plurality of objects, a multi-band space (for example, RGB of registered objects) Means for pre-calculating a histogram in the color space), means for setting a region of interest such as a window function in the input image, and calculating a histogram in the multi-band space in the region of interest, and registering the histogram in advance. Means to calculate the distance or similarity to the histogram of the object that was created, to repeat these processes while changing the position of the region of interest, to change the size of the input image or to change the size of the region of interest However, it has means for repeating this embedding and means for judging whether or not the object exists there depending on the value of the distance or the similarity.

[0009]

BEST MODE FOR CARRYING OUT THE INVENTION When the size of an object changes, the direction of the object changes a little, or a part of the object is hidden by another object by the above-mentioned means. On the other hand, it is possible to search for an object in a complicated image.

[0010]

EXAMPLES The details of the present invention will be described below with reference to one example.

FIG. 1 is a block diagram of an image recognition / retrieval apparatus according to an embodiment of the present invention. The device is roughly divided into an object learning device 1 and an object search device 2.

The object learning device 1 is a device for creating dictionary data required for recognition from a two-dimensional image of a registered object to be searched. Hereinafter, each unit of the object learning device 1 will be described.

The image input device 3 includes a color TV camera and an A
It is composed of a / D converter and takes in a two-dimensional image of an object as learning data and sends it to the object region extraction device 4.

The object area extracting device 4 is composed of means for extracting an object area from an image using a window function or the like.
As the window function of an object, there are window functions such as a rectangular shape for extracting the inside of the object, a circular shape for extracting only the central portion of the circle, and a function for extracting only the inside of the object according to the shape of the object. Here, for simplicity, a case of a rectangular shape will be described. FIG. 2 shows an example of an object to be searched extracted in a rectangular shape.

The color histogram creation device 5 calculates a color histogram using the color values of the pixels of the partial area extracted by the window function from the input image. For example, in the case of three colors of R (red), G (green), and B (blue), h
A three-dimensional histogram of _{r, g, and b} is constructed. A histogram is an amount obtained by dividing a three-dimensional color space of R, G, and B into fine cubes, projecting each pixel of the target area into the color space, and adding the number of pixels in each cube. is there. Learning is completed by calculating color histograms for all registered objects. At the recognition stage, this color histogram is used as a dictionary. FIG. 3 is a three-dimensional graph showing an example of the histogram of the object shown in FIG.

The object search device 2 is a device for detecting the position and size of an object included in an input image based on dictionary data. The object search device 2 will be specifically described below.

The image input device 6 is a device for inputting a search image including a plurality of objects. This is the same as the image input device 3 used in the learning means. The input image is sent to the input image size conversion device 7. An example of the input image is shown in FIG.

The input image size conversion device 7 converts the input image into various magnifications. For example α
^-1x , α ^- 2x, α ^- 3x, etc. Here, α is a numerical value of about 1 to 2, for example. The meaning of creating images with various magnifications is to cope with a large change in the size of an object by enlarging or reducing the input image. Further, a small change in the size of the object does not have much influence because the color histogram has a characteristic of gently changing. The image whose size has been variously converted is sent to the attention area scanning extraction device 8.

The attention area scanning extraction device 8 extracts a partial image of a specific area and sends it to the distance measuring device 10. The shape of this particular area uses the same window function as that used in the object area extraction device 4. The position of this attention area is sequentially scanned in the input image in a raster scan manner, for example, and all of it is sent to the input color histogram calculation device 9.

The input color histogram creation device 9 calculates a color histogram using the color values of the pixels in the partial area extracted from the input image by the window function. While scanning the area of interest, a color histogram is sequentially calculated for the transmitted data, and the result is transferred to the distance calculation device. Further, this process is repeated by changing the size of the input image. A color histogram in which the size of the input image is s at the positions x and y obtained as a result is represented by g (x, y, s) _{r, g, b} . FIG. 5 shows an example of the attention area, and FIG. 6 shows a color histogram of the area.

The distance measuring device 10 receives the color histogram g sent from the input color histogram creating device 9.
The distance value between (x, y, s) _{r, g, b} and the color histogram h _{r, b, g} created by the image learning device 1 is calculated. Although various definitions are possible for the distance value, the Euclidean distance

[0022]

(Equation 1) City block distance

[0023]

(Equation 2) Overlap similarity

[0024]

(Equation 3) Are used.

Pay attention when the distance value (for example, the above equations (1) and (2)) is less than a certain threshold value or the similarity (for example, the above equation (3)) is more than a certain threshold value. The region in which the object is a learned object is its position (x,
y) is output. Further, since the size of the object can be determined from the value of the magnification of the input image output from the input image size conversion device at that time, the detection result s of the size of the object is also output. That is, the position and size of the object used for learning can be detected from the input image. The graph shows the distance value at each point (x, y) with respect to the input figure of the magnification shown in FIG. 7.

[0026]

As described above, according to the present invention,
By combining the collation that uses the local color histogram of the input image with the region of interest set in it and the process of repeating this process while changing the position of the region of interest, etc. It has become possible to detect the position of a specific object. If it is used as an image database or the like, it is possible to search for an image containing a specific object.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of one embodiment of the present invention.

FIG. 2 is an example of an object to be searched extracted in a rectangular shape, which is a printing result of image output by a computer.

FIG. 3 is an example of a three-dimensional histogram of the object shown in FIG.

FIG. 4 is an example of an input image, which is a print result of image output by a computer.

FIG. 5 is an example of a region of interest.

6 is a three-dimensional color histogram of the area shown in FIG.

FIG. 7 is a graph showing distance values at respective points (x, y) with respect to an input figure having a certain magnification.

FIG. 8 is a detection result of an object in an image, which is a printing result of image output by a computer.

[Explanation of symbols]

 1 Object Learning Device 2 Object Search Device 3 Image Input Device 4 Object Area Extraction Device 5 Color Histogram Creation Device 6 Image Input Device 7 Input Image Size Conversion Device 8 Region of Interest Scan Extraction Device 9 Input Color Histogram Creation Device 10 Distance Measuring Device

Claims (3)

[Claims] 1. An image recognition search device for searching a region of a registered object from an image including a plurality of objects, comprising means for calculating a histogram of a registered object in a multiband space in advance, and an input image. A target area such as a window function is set, means for calculating a histogram in the multiband space in the target area, and means for calculating the distance or the similarity between the histogram and the histogram of an object registered in advance, An image recognition / retrieval device having means for repeating these processes while scanning the area of interest, and means for determining whether or not an object exists in the area of interest based on the value of the distance or the similarity obtained by the repetition. 2. The image recognition / retrieval device according to claim 1, wherein the means for repeating these processes while scanning the area of interest has means for varying the size of the area of interest and repeating the processing. 3. The image recognition and retrieval apparatus according to claim 1, wherein the means for repeating these processes while scanning the region of interest has means for changing the size of the input image and repeating the processes.