JPH11161788A - Method for generating recognition model and method for recognizing picture - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce workload related with the generation of a recognition model without designating the area of an object by a human-being. SOLUTION: Plural pictures in which the objects of the generation of a recognition model are commonly projected are inputted 101, each inputted picture is decomposed into partial pictures 111, and a feature vector is extracted from each decomposed partial picture 112. The group of the feature vectors is classified into a cluster group having a hierarchical structure 113, and a cluster constituted of the feature vector of the partial picture included in the object area is obtained from the cluster group 114. The hyperspheroid of the feature vector with the center of gravity of the obtained object cluster as a center, and also the hyperspheroid including all the feature vectors of the object cluster is obtained as an area for featuring the object 116, and the recognition model of the object is generated by making language expression correspond to the hyperspheroid of the feature vector space 117.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a method for generating a recognition model for recognizing an object in an image and an image recognition method based on the generated recognition model.

[0002]

2. Description of the Related Art In recent years, many methods have been proposed for generating a recognition model for recognizing an object in an image. One of them is a method of directly generating a recognition model of a target object from an image on which the target object to be generated is generated. This conventional method learns image features related to the color or texture of an object by manually designating where in the image the image of the object is located. The object image recognition model is generated by associating the obtained image features with the linguistic expressions characterizing the object (Ref. 1: RW Picard and TP Minka,
"Vision texture for annotation", J. Amer. Multimedi
a Systems 3, pp. 3-14 (1995)). For example, when generating a recognition model of the object “sky” from an image in which the object “sky” is captured, the region of the object “sky” in the image is manually spatially designated, and the object “sky” is designated. The recognition model is generated by learning the image feature of “sky” and associating the learned image feature with the linguistic expression “sky”.

In Reference 1, a landscape image is decomposed into partial mesh images, and then a plurality of partial mesh images included in an area of a target for which a recognition model is to be generated in the image are manually designated. Learn the image features of objects,
We propose a method for generating a recognition model of an object by associating the learned image features with the linguistic expressions that characterize the object. For example, when generating a recognition model of the object “sky” from a landscape image in which the object “sky” is captured, first, the landscape image is decomposed into a partial mesh image, and the area of the object “sky” in the landscape image is The image feature of the object "sky" is learned by manually specifying a plurality of partial mesh images included in the image, and the recognition model of the object "sky" is associated by associating the learned image feature with the linguistic expression "sky". Has been generated.

[0004]

SUMMARY OF THE INVENTION A recognition model of an object is generated by manually designating where in the image the object to be recognized is to be captured. However, the conventional method has a problem that when the number of types of the target for which the recognition model is to be generated increases, the amount of work for specifying the region of the target in the image increases.

[0005] The problem to be solved by the present invention is that a plurality of images on which an object to be a recognition model is to be generated can be obtained without manually specifying where in the image the object appears. It is an object of the present invention to propose a method of automatically finding an area and learning an image feature of an object and generating a recognition model of the object.

Another object of the present invention is to propose a method for recognizing whether or not a target object appears in an unknown image by using a generated recognition model.

[0007]

According to the present invention, there is provided a recognition model generating method comprising the steps of: inputting a plurality of images in which an object to be recognized is generated in common and a language expression characterizing the object; Dividing each of the divided images into, for example, equally-spaced meshes and decomposing them into partial mesh images, extracting feature vectors related to colors or textures from the respective partial mesh images, and setting a set of the extracted feature vectors. Classifying into hierarchical clusters,
Obtaining a cluster composed of the feature vectors of the partial mesh images included in the target object region from the obtained cluster group; and determining a superposition of a feature vector space centered on the center of gravity of the obtained target cluster. Generating a hypersphere that is a sphere and includes all feature vectors of the object cluster as a region that characterizes the image feature of the object, and associating the input linguistic expression with the hypersphere in the feature vector space And generating a recognition model of the object.

In the image recognition method according to the present invention, the input unknown image is divided into, for example, equally spaced meshes and decomposed into partial mesh images, and a feature vector is extracted from each partial mesh image of the unknown image. And a step of determining whether or not the target object is shown in the unknown image by a collation calculation between the extracted feature vectors and the recognition model of the target object generated by the recognition model generation method. Features.

Here, a plurality of images in which the object to be recognized is generated in common are set to satisfy the following two conditions (condition 1) and (condition 2). (Condition 1) The region of the target object for which the recognition model is to be generated in the image for generating the recognition model has image characteristics similar in color or texture among all the images for generating the recognition model. (Condition 2) The background region other than the region of the object for which the recognition model is to be generated in the recognition model generation image or a part of the background region is similar in color or texture between all the images for recognition model generation. No image features.

A feature vector group extracted from the partial mesh image included in the area of the object for which the recognition model is to be generated is represented by:
Because of (Condition 1), one cluster is formed to be close in the feature vector space. Therefore, if this cluster can be obtained from the group of clusters obtained in the hierarchical clustering step, it is possible to learn the image characteristics of the object without manually specifying the region of the object in the image. This makes it possible to generate a recognition model of the object.

By the way, since this desired cluster is (condition 1) and (condition 2), the following (feature 1) and (feature 2)
May be characterized as a cluster that satisfies (Feature 1) The cluster includes a feature vector extracted from at least one partial mesh image of all images of the recognition model generation image. (Feature 2) The cluster has the lowest hierarchy among the clusters that satisfy Feature 1. That is, when the hierarchical structure is represented by a dendrogram, this cluster is located at the end of the dendrogram among the clusters satisfying the feature 1.

Therefore, a desired cluster can be obtained by searching for a cluster satisfying these two features at the same time from the cluster group obtained in the hierarchical clustering stage. In the characterization region generation stage, the cluster obtained in such a manner is generalized to a larger hypersphere including the cluster, and then the input linguistic expression is associated with the hypersphere to obtain an object. Generate a recognition model for. The generated recognition model is stored as a recognition dictionary and used for recognition of an object in an unknown image.

[0013]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing the flow of processing for implementing the recognition model generation method of the present invention. This processing is realized by using a so-called computer system including a keyboard, an image scanner, a display, a CPU, a memory device, and the like.

First, using an image scanner or the like,
A plurality of images in which objects to be recognized model generation targets are commonly captured are input in advance and stored in a memory or the like (step 101). In addition, using a keyboard or the like, a linguistic expression for attaching the recognition model generation target is input and set in a register or the like (step 102). Here, the above two conditions (Condition 1) and (Condition 2) are included in each of a plurality of images in which an object to be recognized is generated.
Is satisfied.

Next, the following processing is executed on the CPU to automatically generate a recognition model of the object. First, each input image is divided into meshes at equal intervals and decomposed into partial mesh images (step 111). Next, for each image, a feature vector relating to color or texture is extracted from each partial mesh image (step 112), and a set of the extracted feature vectors is classified into a cluster group having a hierarchical structure (step 113). . For each cluster of the obtained cluster group, whether the set of partial mesh images from which each feature vector constituting the cluster is extracted includes at least one partial mesh image of all images of the recognition model generation image Is checked, and if it is included, this cluster is registered as a cluster candidate that characterizes the image feature of the object (step 114). next,
Among the registered candidate clusters, a cluster having the smallest number of feature vectors constituting the cluster is selected, and for each feature vector constituting the minimum number of clusters, an extraction source partial mesh image is used as a recognition model generation target. It is checked whether it is included in the area of a certain object, and if all of the extracted partial mesh images are included in the object area, the cluster is determined as a cluster characterizing the image feature of the object. Acquisition (step 115).
Next, a hypersphere in the feature vector space centered on the center of gravity of the obtained object cluster and including all the feature vectors of the object cluster is generated as a region characterizing the image feature of the object ( Step 116). Finally, for the generated hypersphere in the feature vector space,
A recognition model is generated by associating the input linguistic expressions (step 117). The generated recognition model is stored in a dictionary memory as a recognition dictionary.

Next, a specific example of processing relating to generation of a recognition model according to the present invention will be described with reference to FIGS.

First, an image 1,..., And an image N in which an object, which is a recognition model generation target, is commonly captured is input. However, the following two conditions are satisfied for each image. (Condition 1) The region of the target object for which the recognition model is to be generated in the image for generating the recognition model has image characteristics similar in color or texture among all the images for generating the recognition model. (Condition 2) The background region other than the region of the object for which the recognition model is to be generated in the recognition model generation image or a part of the background region is similar in color or texture between all the images for recognition model generation. No image features.

Here, for example, it is assumed that a recognition model of an object "blue clear sky" is generated, and images 1,... Is selected and input so that (condition 1) and (condition 2) are satisfied. According to (condition 1), image 1
..., Each image N has a target object “blue clear sky”
And a rectangular region having image characteristics similar to each other. In FIG. 2, the area of the target object for which the recognition model is to be generated is indicated by hatching. In this example, a (1, 1), a (2, 1), a (N, K) and the like are rectangular regions having similar image features. Here, the size of the rectangular area can be made the same by making the rectangular size the smallest one. On the other hand, according to (Condition 2), the rectangular regions having such properties are limited to those included in the target object “blue clear sky” region of the original image. Therefore, if it is possible to automatically find a rectangular region of the same size having similar image characteristics from each image of the recognition model generation image, the image characteristics of the rectangular region are learned, and It is possible to learn the image feature of “blue sunny sky” without manually specifying the area. However, at this point, since the processing system does not know at which position of the input image the rectangular area exists, the input image is divided into equally-spaced meshes of an appropriate size for the time being. . That is, the input image 1,..., Image N is divided into images (step 11).
In 1), as shown in FIG. 2, the image is decomposed into partial mesh images a (1, 1),..., A (N, K) of an appropriate size. FIG. 2 shows an example of division into 4 × 4 meshes. If the learning of the image features of the object "blue clear sky" cannot be achieved even after performing the following series of processing,
It is assumed that the division size of the equally-spaced mesh is automatically changed to a smaller one, and the processing is restarted from the beginning.

Next, feature vector extraction processing (step 1)
In 12), a feature vector v (1, 1),.
(N, K) is extracted. This is also shown in FIG. In FIG. 2, the dimension of the feature vector space is two-dimensional. Next, in the hierarchical clustering process (step 113), a set of feature vectors {v (1, 1),.
(N, K)} are classified into cluster groups having a hierarchical structure as shown in FIG. FIG. 3 shows a state of cluster formation after repeating the integration processing three times, and the number given to the cluster indicates the number of times the cluster was formed in the integration processing.

As for the feature vector extraction, for example, “YI Ohta T. Kanadeand T. Sakai,“ Co
lor information for region segmentation ", Comp.
and Img. Proc., 13: 222-241 (198).
0) ”(Ref. 2) and“ J. Mao and AK Jai
n, “Texture classification and segmentation using
multiresolution simultaneous autoregressive model
s ", Patt. Rec., 25 (2): 173-188 (1
992) ”(Ref. 3). Regarding hierarchical clustering, for example,“ Mikio Takagi, Hirohisa Shimoda, “Image Analysis Handbook”, University of Tokyo Press (19)
91) "(reference document 4).

Next, candidate cluster registration processing (step 1)
In 14), for each of the obtained classes, as shown in FIG. 4, a set of partial mesh images from which the feature vectors constituting the cluster are extracted is converted into at least one partial mesh of all the input images. It is checked whether or not an image is included. If the image is included, this cluster is registered as a cluster candidate that characterizes the image feature of the target object. That is, the cluster C of interest is C = ｛v (i1, j1), v (i2, j2),.
v (ip, jp)}, a set C composed of partial mesh images from which the p feature vectors are extracted
＾ is C ＾ = {a (i1, j1), a (i2, j2),.
, A (ip, jp)} where the set C ＾ includes at least one partial mesh image of all the input N images, the cluster C is defined as an object Is registered as a cluster candidate that characterizes the image feature. FIG. 4 shows that cluster C includes feature vectors extracted from at least one partial mesh image of all N images.

Next, in the object cluster acquisition process (step 115), a cluster C _min having the smallest number of feature vectors constituting the cluster is selected from the registered candidate clusters, and then a cluster C _min is formed. Check whether the partial mesh image from which each feature vector is extracted is included in the region of the target object for which the recognition model is to be generated, and if all the partial mesh images from which the extraction source is included are included in the target object region , The cluster C _min is determined and acquired as a cluster characterizing the image feature of the object. That is, if the cluster C _{min is, C min = {v (i1} min, j1 min), v (i2 min, j2 min), ···, v (iq min, jq min)} represented as a cluster C _min C _min ^ is set in the extract the partial mesh image _{composed, C min ^ = {a (} i1 min, j1 min), a (i2 min, j2 min), ···, a (iq min, jq _min )}, where each partial mesh image a (i
k _min, jk _min) is, whether included in the area of the object "blue clear sky", as shown in FIG. 4, each partial mesh image a (ik _min, jk _min) to put an outline Each partial mesh image a is highlighted and examined in the original image.
(Ik _min , jk _min ) are all objects “blue clear sky”
Is included in the region, the cluster C _min is determined and acquired as a cluster characterizing the image feature of the target object.

Hierarchical clustering processing (step 11)
3) is a process of sequentially integrating similar feature vectors to grow a cluster. Therefore, all feature vectors extracted from each partial mesh image are finally integrated into one cluster _Cmax . Therefore, the cluster C _max is also included in the candidate clusters registered in the candidate cluster registration process (step 114). However, in the input recognition model generation image, there is generally another region other than the region of the target object “blue clear sky”. Therefore, the target object “blue clear sky” is included in the cluster C _max. The feature vector extracted from the partial mesh image included in another area other than the area of "" is also included.
For example, in the input image 1, the object "blue clear sky"
In addition, if the object "asphalt road" is imaged, for example, the feature vector extracted from the partial mesh image included in the area of the object "asphalt road" is also included in _Cmax . The reason for selecting a cluster having the smallest number of feature vectors constituting the cluster from the candidate clusters registered in the candidate cluster registration process is that a feature vector that is overgrown like a cluster C _max and is irrelevant to the object is selected. This is to exclude clusters that may be included.

If at least one of the feature vectors constituting the cluster C _min does not include the partial mesh image from which the feature vector is extracted, in the region of the object for which the recognition model is to be generated. Then, a series of processes from the feature vector extraction process (step 112) is repeated again using a feature amount different from the used feature amount.

Next, a characterization area generating process (step 1)
In 16), a hypersphere in a feature vector space whose center is the center of gravity of the object cluster and whose radius is the maximum value of the distance between the center of gravity and each feature vector of the object cluster is converted into an image of the object "blue clear sky". It is generated as a region characterizing the feature. Then, a linguistic expression "blue sunny sky" is manually input (step 102). Finally, in the recognition model generation process (step 117), the input linguistic expression “blue clear sky” is associated with the hypersphere in the feature vector space characterizing the target object “blue clear sky”. Generate a recognition model for the object "blue sunny sky".

FIG. 5 is a diagram showing a flow of processing for implementing the image recognition method of the present invention based on the recognition model generated as described above. This processing is also actually realized on the computer system.

First, an unknown image such as a camera or an image recording medium is input (step 201). The input unknown image is divided into equally spaced meshes and decomposed into a plurality of partial mesh images (step 211), and a feature vector is extracted from each partial mesh image of the unknown image (step 21).
2). These are basically steps 111 and 11 in FIG.
Same as 2. It is determined whether or not the target object is included in the unknown image by performing a collation calculation with each of the extracted feature vectors and the recognition model generated and stored using the above-described recognition model generation method (step 213).

Next, a specific processing example of the image recognition method based on the recognition model of the present invention will be described with reference to FIG.
Here, it is assumed that it is determined whether or not the target object “blue clear sky” is captured in the unknown image X.

An unknown image X is input. First, in an image dividing process (step 211), the unknown image X is decomposed into partial mesh images b (1),..., B (K). next,
In the feature vector extraction process (step 212), each partial mesh image b (j) of the unknown image X (1 ≦ j ≦ K)
By using the same feature amount as the feature amount that generated the recognition model of the object “blue sunny sky”, the feature vector w (1),
.., W (K) are extracted. As described above, the feature amount extracted from each partial mesh image b (j) needs to be the same as the feature amount that generated the recognition model of the object to be determined. Finally, in the recognition model matching process (step 213), it is determined whether or not the target object "blue clear sky" is shown in the unknown image X by a calculation of matching with the recognition model of the target object "blue clear sky". I do. That is,
As shown in FIG. 6, the extracted feature vectors w (1),.
.., W (K) is checked to see if at least one feature vector is included in the hypersphere of the feature vector space characterizing the image feature of the object, and if at least one feature vector is included, It is determined that the target object “blue clear sky” is shown in the unknown image X, and the linguistic expression “blue clear sky” is given to the unknown image X.

[0030]

As described above, according to the present invention,
By providing a plurality of images in which the target object for which the recognition model is to be generated is shared, the processing system can specify the target object region without manually specifying where in the image the target object region is located. It is possible to automatically find out and learn the image features of the object, and generate a recognition model of the object. As a result, the amount of work involved in generating a recognition model can be reduced as compared with a conventional method in which a human specifies a region of a target object. Also, based on the recognition model generated in this way, the target object can be easily recognized from the unknown image.

[Brief description of the drawings]

FIG. 1 is a diagram showing a flow of processing according to a recognition model generation method of the present invention.

FIG. 2 is a diagram illustrating an example in which an image for generating a recognition model is divided into equally-spaced meshes and an example in which a feature vector is extracted from each partial mesh image.

FIG. 3 is a diagram showing a process of classifying a set of extracted feature vectors into clusters having a hierarchical structure.

FIG. 4 is a diagram showing each feature vector constituting one cluster;
It is a figure which shows the process of checking from which partial mesh image it was extracted.

FIG. 5 is a diagram showing a flow of processing according to the image recognition method of the present invention.

FIG. 6 is a diagram specifically illustrating a flow of a process of determining a target in an unknown image X;

[Explanation of symbols]

101 Image Input for Recognition Model Generation 102 Linguistic Expression Input 111 Image Segmentation Processing for Image for Recognition Model Generation 112 Feature Vector Extraction Processing for Image for Recognition Model Generation 113 Hierarchical Clustering Processing 114 Candidate Cluster Registration Processing 115 Object Cluster Acquisition Processing 116 Features Attached area generation processing 117 Recognition model generation processing 201 Unknown image input 211 Image segmentation processing for unknown images 212 Feature vector extraction processing for unknown images 213 Recognition model matching processing

Claims (2)

[Claims] 1. A method for generating a recognition model by inputting a plurality of images in which an object of recognition model generation is commonly taken and a linguistic expression characterizing the object, wherein each of the input images is Decomposing into partial images; extracting feature vectors from each of the decomposed partial images; classifying a set of the extracted feature vectors into clusters having a hierarchical structure; Acquiring a cluster composed of feature vectors of partial images included in the object region from the cluster group; and a hypersphere in a feature vector space centered on the center of gravity of the acquired object cluster, and Generating a hypersphere including all the feature vectors of the object cluster as a region characterizing the object; and inputting the hypersphere in the generated feature vector space to the hypersphere. Generating a recognition model of an object by associating a language expression with the object. 2. A method for recognizing an object of an input unknown image based on a recognition model generated by the recognition model generation method according to claim 1, wherein the input unknown image is decomposed into partial images. Performing a step of extracting a feature vector from each partial image of the unknown image; and performing a matching calculation between each of the extracted feature vectors and the generated recognition model of the object to determine whether the target object is included in the unknown image. Determining whether or not the image is recognized.