JP2003271957A - Device and method for calculating similarity between images, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately express similarity between images. <P>SOLUTION: The first representative vector group a3 and the second representative vector group b3 for representing respective feature quantity vector groups are calculated by a representative vector group calculating part 22, based on the first feature vector quantity group b1 and the second feature vector quantity group b2 showing features of the respective images of the first comparison-objective image a1 and the second comparison-objective image b1 calculated by a feature quantity vector group calculating part 21, and a representative intervector-group distance ab from the first representative vector group a3 to the second representative vector group b3 for indicating a relative distance to the second representative vector group b3 viewed from the first representative vector group a3, and a representative intervector-group distance ba from the second representative vector group b3 to the first representative vector group a3 for indicating a relative distance to the first representative vector group a3 viewed from the second representative vector group b3 are calculated by a representative intervector-group distance calculating part 23. The larger one thereof is selected as an inter-image distance c by a distance selecting part 24, and the inter-image similarity is defined using the inter-image distance c to be output, in a similarity output part 25. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention compares the characteristics of two images, such as color information, grayscale information, background, main subject, etc., and calculates the similarity between the two images. The present invention relates to an apparatus and method, and a program for causing a computer to realize such a function of calculating the similarity between images.

[0002]

2. Description of the Related Art An inter-image similarity calculation device that compares features such as color information, grayscale information, background, main subject, and the like between two images and calculates the similarity between these two images It is often used in a system for calculating and retrieving similar images, and is disclosed in JP-A-11-096368 and 2000.
-029885, "Proc. Of Int. Symposium on
Multimedia Information Proccssing, Dec 1997. "and so on.

In the inter-image similarity calculation method disclosed in these documents, generally, two comparison target images 101 and 102 are input as shown in FIGS. 12A and 12B (step S101). ), Each comparison target image 101, 102
After performing the feature analysis (step S102), the color histogram, the texture feature amount, the shape feature amount, and the like are expressed as one feature amount vector for one image (step S103). Then, when calculating the similarity between images, distances such as Euclidean distance and Manhattan distance between these feature amount vectors are calculated (step S10).
4) The smaller the distance (Distance), the greater the degree of similarity (relatively similar images), and the greater the distance, the smaller the degree of similarity (relatively similar images).
The degree of similarity is calculated so that (step S105).

The use of the similarity thus calculated is to calculate the similarity between one image and a plurality of images and output the images in the order of similarity, or the similarity is above a certain threshold (distance is below the threshold). It is used for the application that outputs things as a certain class.

[0005]

However, in the above-mentioned conventional technique, since many features originally consisting of many pixel data groups are made into one vector for each image, the feature amount extraction method is quite appropriate. If this is not the case, the loss of information will increase, and the features of the image cannot be represented faithfully.

This will be described by taking the color histogram analysis of an image as an example. Pixel data distributed in the RGB space as shown in FIG. 13 is histogrammed for each of the RGB axes by histogram analysis. At this time, the histogram data of each axis are arranged in order to form one feature amount vector.
However, in this case, the three-dimensional position information of each data in the RBG space is lost. Therefore, it becomes impossible to faithfully represent the characteristics of the image.

Further, the characteristics of the image (for example, the distribution of pixel data) greatly differ depending on the image, and it is necessary to faithfully express the characteristics in any distribution shape. The fidelity of will be greatly different.

If the features of the images cannot be represented faithfully as described above, the degree of similarity between the images cannot be represented accurately.

For example, as shown in FIG. 14A, the distribution of the first comparison object image 101 (area A) and the distribution of the second comparison object image 102 (area B) are similar in shape and close in position. In this case, the distance between the feature amount vector 103 of the area A and the feature amount vector 104 of the area B is small and the distance between the two images is small even if the feature amount extraction method is not so appropriate. That is, an output that the desired result is high in similarity is obtained. Further, as shown in FIG. 14B, when the position is distant regardless of the distribution shape, even if the feature quantity extraction method is not so appropriate, the feature quantity vector 103 of the area A and the feature quantity of the area B The distance to the quantity vector 104 is large, and the distance between the two images is large, that is, the output that the desired result is low similarity is obtained.

On the other hand, if the shapes of the distributions are different even if the positions are close, the feature quantity extraction method is not appropriate, that is, the feature quantity of the region A as shown in FIG. If the vector 103 and the feature quantity vector 104 of the region B are extracted, the distance between the feature vectors becomes small, and the distance between the two images is small, that is, the similarity that is an erroneous result is high. The output will be obtained. Similarly, as shown in (D) of FIG. 14, even when the positions are close to each other, the distribution of the feature values is significantly different when one (region A) greatly includes the other (region B). If the extraction method is not appropriate, the distance between the feature amount vector 103 of the area A and the feature amount vector 104 of the area B will be small, and the distance between the two images will be small, that is, an erroneous result. The output will be high.

The present invention has been made in view of the above points, and compares features such as color information, grayscale information, background, main subject, and the like between two images to determine the similarity between the two images. When calculating, feature amounts that faithfully reproduce the image features of various images can be extracted without devising the feature amount extraction method so that there is no loss of information, and the similarity between images is accurately expressed. An object is to provide an inter-image similarity calculation device, an inter-image similarity calculation method, and a program that can be performed.

[0012]

In order to achieve the above object, the inter-image similarity calculation device according to the present invention is configured to compare the characteristics of images with respect to a first comparison target image and a second comparison target image. An inter-image similarity calculation device for calculating inter-image similarity, comprising an image input unit for inputting the first comparison target image and the second comparison target image, the first comparison target image and the second comparison target image. From the target image, showing the characteristics of each image,
A feature quantity vector group calculating unit that calculates the first feature quantity vector group and the second feature quantity vector group, and first to second indicating the relative distance between the second feature quantity vector group and the first feature quantity vector group. An inter-vector-group distance for calculating the inter-vector-group distance to 2 and a second-to-first inter-vector-group distance indicating the relative distance of the first feature-quantity vector group viewed from the second feature-quantity vector group A calculation unit, a distance selection unit that selects a larger one of the first to second vector group distances and the second to first vector group distances as the image distance, and the image distance And a similarity output unit that defines and outputs the similarity between images using the distance.

That is, according to the inter-image similarity calculation apparatus of the present invention, instead of extracting a single vector as a feature amount for each image, a large number of feature amount vector groups are extracted, and then the vector groups are extracted. Since the similarity between groups is calculated in units of, the similar vector groups can be output as having a high similarity, and the dissimilar vector groups can be output as having a low similarity. Therefore, the feature amount may remain as a vector group, and it is not necessary to use a single vector, so it is not necessary to devise a feature amount extraction method so as to prevent information loss. That is, it is possible to accurately extract the feature amount that faithfully reproduces the image feature for various images without devising the feature amount extraction method so that there is no loss of information, and to accurately express the similarity between images. An inter-similarity calculation device can be provided.

Further, the inter-image similarity calculation apparatus according to the present invention is an inter-image similarity calculation apparatus for calculating inter-image similarity between a first comparison target image and a second comparison target image whose image features are to be compared. And the first comparison target image and the second comparison target image
An image input unit for inputting a comparison target image, a first feature amount vector group and a second feature amount vector group showing the features of the respective images from the first comparison target image and the second comparison target image. A feature quantity vector group calculation unit for calculating
A representative vector group calculation unit that calculates a first representative vector group and a second representative vector group that represent each of the feature amount vector groups from the feature amount vector group and the second feature amount vector group, and the first representative The distance from the first representative vector group to the second representative vector group indicating the relative distance of the second representative vector group viewed from the vector group and the first representative vector group viewed from the second representative vector group.
A representative vector group distance calculating unit that calculates a second to first representative vector group distance indicating a relative distance of the representative vector group, and the first to second representative vector group distances and the first to second representative vector group distances. A distance selecting unit that selects a larger one of the representative vector group distances from 2 to 1 as the image distance; and a similarity output unit that defines and outputs the image similarity using the image distance. It is characterized by including.

That is, according to the inter-image similarity calculation apparatus of the present invention, instead of extracting a single vector as a feature amount for each image, a large number of feature amount vector groups are extracted, and then a vector group is extracted. Since the similarity between groups is calculated in units of, the similar vector groups can be output as having a high similarity, and the dissimilar vector groups can be output as having a low similarity. Therefore, the feature amount may remain as a vector group, and it is not necessary to use a single vector, so it is not necessary to devise a feature amount extraction method so as to prevent information loss. That is, it is possible to accurately extract the feature amount that faithfully reproduces the image feature for various images without devising the feature amount extraction method so that there is no loss of information, and to accurately express the similarity between images. An inter-similarity calculation device can be provided. Moreover, instead of using all of the extracted feature vector groups, the similarity between groups is calculated by using a representative vector group representing each feature vector group as a unit. The amount is small and the speed can be increased.

The number of vectors of the first representative vector group and the number of vectors of the second representative vector group calculated by the representative vector group calculation unit are the same, and the number of vectors between the representative vector group calculation units is calculated. The distance between the first representative vector group and the second representative vector group is each vector of the first representative vector group, which is the distance between each vector of the first representative vector group and the nearest neighbor vector in the second representative vector group. And the distances between the representative vector groups from the second representative vector group calculated by the representative vector group distance calculating unit are the vectors of the second representative vector group and the nearest neighbors of the first representative vector group. The distance to the vector may be the sum of the respective vectors of the second representative vector group.

Further, at this time, when the distance between the first representative vector group and the second representative vector group is calculated by the representative vector group calculating unit, the number of the first representative vector group with respect to the total number of the second representative vector groups is calculated. The ratio of the number of second representative vectors that are closest to each vector is calculated, and the larger the ratio is, the smaller the distance between the representative vector groups from the first to the second is corrected, and the representative vector group calculating unit calculates. When calculating the distance between the second representative vector group and the first representative vector group, the ratio of the number of first representative vectors most adjacent to each vector of the second representative vector group to the total number of the first representative vector group is calculated. It is preferable to calculate and correct the distance between the representative vector groups from the second to the first as the ratio increases.

Further, when the distance between the representative vector groups from the first to the second is calculated by the distance between representative vector groups, the distribution of the first feature quantity vector group or the first representative vector group. According to the magnitude of the distribution, the larger the distribution is, the smaller the distance between the representative vector groups from the first to the second is corrected, and the distance between the representative vector groups is calculated from the second by the representative vector group distance calculating unit. When the distance between the representative vector groups to the first is calculated, the larger the distribution size is, the larger the second characteristic vector group or the second representative vector group is. The distance from the representative vector group to the first representative vector may be corrected to be smaller.

The inter-image similarity calculation method according to the present invention is an inter-image similarity calculation method for calculating the inter-image similarity between the first comparison target image and the second comparison target image whose image features are to be compared. And the first comparison target image and the second comparison target image
A comparison target image is input, and a first feature amount vector group and a second feature amount vector group indicating the features of each image are calculated from the first comparison target image and the second comparison target image, From the first feature quantity vector group and the second feature quantity vector group, a first representative vector group and a second representative vector group representing each feature quantity vector group are calculated, and the first representative vector group is calculated.
A distance from the first representative vector group to a second representative vector group indicating the relative distance of the second representative vector group viewed from the representative vector group;
A distance from the second representative vector group to the first representative vector group, which indicates the relative distance of the first representative vector group from the second representative vector group, is calculated, and the distance from the first representative vector group to the second representative vector group is calculated. Of the second to first representative vector group distances, the larger one is selected as the image distance, and the image distance is defined and output using the image distance.

That is, according to the inter-image similarity calculation method of the present invention, instead of extracting a single vector as a feature amount for each image, a large number of feature amount vector groups are extracted, and then a vector group is extracted. Since the similarity between groups is calculated in units of, the similar vector groups can be output as having a high similarity, and the dissimilar vector groups can be output as having a low similarity. Therefore, the feature amount may remain as a vector group, and it is not necessary to use a single vector, so it is not necessary to devise a feature amount extraction method so as to prevent information loss. That is, feature amounts that faithfully reproduce the image features of various images can be extracted without devising the feature amount extraction method so that there is no loss of information, and the similarity between images can be accurately expressed. become. Moreover, instead of using all of the extracted feature vector groups, the similarity between groups is calculated by using a representative vector group representing each feature vector group as a unit. The amount is small and the speed can be increased.

Also, the program according to the present invention causes the computer to calculate the first image comparison degree when calculating the inter-image similarity between the first image comparison object and the second image comparison image to be compared in image characteristics. From the function of inputting an image and the second comparison target image, and from the first comparison target image and the second comparison target image, a first feature quantity vector group and a second feature quantity indicating a feature of each image. A first representative vector group and a second representative vector group representing each feature amount vector group are calculated from the function of calculating the vector group and the first feature amount vector group and the second feature amount vector group. Function to
The distance from the first representative vector group to the second representative vector group indicating the relative distance of the second representative vector group viewed from the first representative vector group, and the relative distance of the first representative vector group viewed from the second representative vector group. Of the second to the first representative vector group distance, and the first to the second representative vector group distance and the second to the first representative vector group distance. Of these, the function of selecting the larger one as the inter-image distance and the function of defining and outputting the inter-image similarity using the inter-image distance are realized.

That is, according to the program of the present invention, when the computer calculates the inter-image similarity between the first comparison image and the second comparison image for comparing the features of the images, Instead of extracting a single vector as a feature amount, a large number of feature amount vector groups are extracted and then the similarity between groups is calculated with the vector group as a unit. The vector groups are output as having high similarity, and the vector groups that are not similar are output as having low similarity. Therefore, the feature amount may remain as a vector group, and it is not necessary to use a single vector, so it is not necessary to devise a feature amount extraction method so as to prevent information loss. That is, feature amounts that faithfully reproduce the image features of various images can be extracted without devising the feature amount extraction method so that there is no loss of information, and the similarity between images can be accurately expressed. become. Moreover, instead of using all the extracted feature vector groups,
Since the degree of similarity between groups is calculated with a representative vector group representing each feature amount vector group as a unit, the amount of calculation as a whole can be small and the speed can be increased.

[0023]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

[First Embodiment] FIG. 2A is a diagram showing the configuration of an inter-image similarity calculation apparatus according to the first embodiment of the present invention. Is composed of an input device 10, an arithmetic device 20, and an output device 30.

Here, the input device 10 is for inputting an image to be compared with the characteristics of the image. For example, the input device 10 reads an image from a recording medium such as an FD or a CD-ROM, or reads an image. It may be a scanner to be performed or an input via a network such as the Internet or a LAN. Further, it may be possible to directly input from a digital camera or the like.

The arithmetic unit 20 is for performing various arithmetic operations for calculating the similarity between images, and may be a dedicated device or a personal computer or the like. The arithmetic unit 20 includes a CPU 20A and a C
The program memory 20B stores a program executed by the PU 20A. When the device is dedicated, a dedicated program is fixedly stored in the program memory 20B. Further, when the apparatus is composed of a personal computer, the program loaded in the program memory 20B is FD or CD-RO.
It may be obtained by reading a preinstalled recording device (not shown) of the arithmetic unit 20 from a recording medium such as M or from a network such as the Internet or a LAN, or if necessary from the recording medium or the network. It may be obtained by reading.

Further, in this program memory 20B,
An application program that uses the calculated image similarity can also be stored. Examples of the application include, for example, searching for similar images from an image database, calculating the similarity between one image and a plurality of images and outputting the images in the order of similarity (categorization), and the similarity is above a certain threshold. Outputting a class (distance less than threshold) as a class (grouping).

The output device 30 is a display or printer for outputting the result of the application. Alternatively, the result may be output to the outside via a network. Of course, not only the application result but also the image similarity calculation result may be output.

FIGS. 1 and 2B are a functional block diagram and an operational flowchart of such an inter-image similarity calculation device.

That is, first, the image input unit 11 as one functional unit of the input device 10 inputs the first comparison target image a1 and the second comparison target image b1 whose features are to be compared (step S1).

Subsequently, the feature amount vector group calculation unit 21 as one functional unit of the arithmetic unit 20 shows the features of each image from the first comparison target image a1 and the second comparison target image b1. , A first feature amount vector group a2 (first comparison target image a1) obtained by vectorizing RGB values of pixels, for example.
And a second feature amount vector group b2 (extracted from the second comparison target image b1) are calculated (step S2). Here, since the feature quantity vector group generally has a very large number of vectors, in the representative vector group calculation unit 22, subsequently, from the first feature quantity vector group a2 and the second feature quantity vector group b2, First representing each feature vector group
The representative vector group a3 and the second representative vector group b3 are calculated (step S3). At this time, the number of vectors in the representative vector group may be smaller than the number of vectors in the feature amount vector group. When the number of feature amount vector groups is small, or when the feature amount vector group is realized by using a computer having high calculation capability, the feature vector group itself may be used as the representative vector group.

Subsequently, in the representative vector group distance calculating section 23 as one functional section of the arithmetic unit 20, the second representative vector group b3 viewed from the first representative vector group a3.
From the first representative vector group distance ab from the first representative vector group b3 indicating the relative distance of the first representative vector group a3 to the second representative vector group a3 relative to the second representative vector group b3. And the inter-vector group distance ba is calculated (step S4).

The relative distance between the vector groups can be set as follows, if generalized and described. As shown in the conceptual diagram of FIG. 3, if the region formed by the first representative vector group a3 is set as the region A and the region formed by the second representative vector group b3 is set as the region B, the above-mentioned first to second The vector group distance ab is considered to be the sum or average of the distances from each point in the area A to reach the area B in the shortest time, and the vector group distance ba from the second to the first vector group is the area B. It is considered as the sum or average of the distances from each point in the area to reach the area A in the shortest time. At this time, if the number of vectors is the same, the sum of the distances may be the distance between the vector groups, and if the number of vectors is different, the average of the distances may be the distance between the vector groups. The concept of summation and average is as follows:
It may be replaced with the area integration in and the normalization in the area integration. Vector group distance ab and vector group distance ba
Is generally referred to as a relative distance because it differs depending on the shape of the area and the relative positional relationship. This distance is, for example, from the town A to the town B how much the people in town A must walk to go to town B, assuming that the area is the town and the representative vector group is the person who lives in that town. Relative distance from (choose how close people in town A feel to town B), how far people in town B must go to town A anyway from town B to town A ( B
How close people feel to town A is) (however, it is possible that towns may overlap).

This relative distance is calculated from (A) to (D) of FIG.
As shown in, it has the following properties.

Case 1. If the two vector groups have similar distribution shapes and the positions are close, both (A → B, B
→ The relative distance of A) becomes very small ((A) of FIG. 4)
reference).

Case 2. Regardless of the distribution shape of the two vector groups, if the positions are far, both (A → B, B → A)
Also becomes larger (see FIG. 4B).

Case 3. If the shapes of the distributions are different even if the positions of the two vector groups are close, the following case 4. Except in the above case, the relative distance between the two (A → B, B → A) also becomes large (see FIG. 4C).

Case 4. Even if the positions of two vector groups are close to each other, one (for example, A: large distribution) and the other (for example, B:
If the size of the distribution is greatly different to include the smaller distribution, one of the two (A → B, B → A) (B → B)
A) is small and one (A → B) is large (see (D) of FIG. 4).

After the relative distance between the representative vector groups is calculated by the distance calculating unit 23, the distance selecting unit 24 as one functional unit of the arithmetic unit 20 calculates the distance between the representative vector groups from the first to the second. Of the distance ab and the representative vector group distance ba from the second to the first, the larger one is the inter-image distance c.
(Step S5). That is, an important point in outputting the similarity between images is to output similar images so that the similarity is high, and to output dissimilar images so that the similarity is low. The two relative distances have the properties of Cases 1 to 4 above, but by selecting the larger one of the two relative distances as the inter-image distance, the similarity becomes low except for Case 1 (that is,
The distance can be increased) and the above points can be realized.

Therefore, the feature quantity vector of the area A and the feature quantity vector of the area B are inappropriately extracted as in the prior art described with reference to FIGS. 14C and 14D. Sometimes, the inter-image distance is not determined to be small, and in the present embodiment, it can be correctly determined that the inter-image distance is large.

Then, in the similarity output section 25 as one functional section of the arithmetic unit 20, the selected inter-image distance c
Is used to define the similarity between images and output (step S
6). Generally, the larger the distance between images, the lower the similarity,
The smaller the inter-image distance, the higher the similarity. The output definition of the inter-image similarity may be set according to how the main inter-image similarity calculation device is used. If you want to define the similarity output to be larger as the similarity between two images is higher, you can set the reciprocal of the distance between the images as the similarity so that the calculated similarity is higher as the distance is smaller. When it is desired to define the similarity output to be smaller as the similarity between the images is higher, the inter-image distance may be set as the similarity as it is.

When the inter-image similarity is defined in this manner, a desired application is executed by using the inter-image similarity, and the result is output by the output device 30 (step S7). . For example, a similar image search application in which the similarity between images output as a numerical value by the similarity output unit 25 is subjected to threshold processing and an image having an similarity between images that is equal to or larger than the threshold is displayed and output on a display can be considered.

Hereinafter, the similarity calculation method will be described more specifically with reference to FIGS. 5A to 5D. Note that calculation of the inter-image similarity of image pairs as shown in FIG. 5A will be considered here.

First, the image input section 11 inputs the first comparison target image a1 and the second comparison target image b1 (see FIG. 5A) for which the features should be compared and the similarity should be calculated. These images are commercially available electronic image collections (CD-ROM), for example.
It is possible to input from.

Subsequently, the feature quantity vector group calculating section 21
Then, from the first comparison target image a1 and the second comparison target image b1, as a feature amount indicating the feature of each image, a first feature amount vector group a obtained by vectorizing RGB values of pixels.
2 (extracted from the first comparison target image a1) and the second feature amount vector group b2 (extracted from the second comparison target image b1) are calculated (see (B) of FIG. 5).

Subsequently, in the representative vector group calculating section 22, from the first feature amount vector group a2 and the second feature amount vector group b2, a first representative vector group a3 representing each feature amount vector group is obtained. The second representative vector group b3 is calculated (see FIG. 5C). The calculation of the representative vector group is performed by, for example, self-organizing feature mapping or K-me.
A clustering method such as the ans method or a vector quantization method may be used. Although the number of vectors in the representative vector group is set to be smaller than the number of vectors in the feature amount vector group, it is not necessarily the same for each image (of course, it may be the same). In addition, when the number of feature quantity vector groups is small, or when it is realized by using a computer with high calculation ability, the feature quantity vector group may be used as it is as the representative vector group.

Subsequently, in the representative vector group distance calculating unit 23, the first to second representative vector group distances indicating the relative distances of the second representative vector group b3 from the first representative vector group a3. Ab and the inter-vector-group distance ba from the second representative vector group a3 to the first representative vector group a3 viewed from the second representative vector group b3 are calculated (see (D) of FIG. 5). As the distance between vector groups, various mathematical expressions can be defined as described above, but here, the distance ab between vector groups from the first to the second is the second distance from each point of the first representative vector group a3. Representative vector group b
3 is the average of the distances to the nearest representative vectors in the third representative vector, and similarly the distance ba between the second and first vector groups is the distance from each point in the second representative vector group b3 to the nearest representative vector in the first representative vector group a3. Is the average of the distance from the representative vector of. Note that, as described above, the concept of summation is that the area A occupied by the first representative vector group a3 or the area B occupied by the second representative vector group b3.
In the area integration, and the concept of the average may be mathematically replaced with normalization by the area integration value.

After the above-mentioned relative distance is calculated by the distance between representative vector groups 23, the distance between the representative vector groups ab from the first to the second is calculated by the distance selecting unit 24 (in FIG. Of “5.0”) and the distance ba between representative vector groups from the second vector to the first vector (“5.5” in FIG. 5D), the larger one (“5. 5 ") as the inter-image distance c. The distance value in the figure is a value that is temporarily set for the sake of explanation.

Finally, the similarity output section 25 defines and outputs the inter-image similarity using the inter-image distance c. As the definition setting of the inter-image similarity, the inter-image distance c is directly used as the similarity. This corresponds to defining the similarity output to be smaller as the similarity between the two images is higher. In the example of FIG. 5A, the similarity output between the two images is small (the distance is small: “5.5”), and the similarity between the images is high.

Similarly, FIGS. 6A to 6D.
Shows an example of a dissimilar image pair. In the case of this example, the distance a → b (“50.0”) and the distance b → a
As a result of selecting the larger one in (“7.0”), the similarity output becomes large (the distance is large: “50.0”) and the similarity between images is low. At this time, distance b →
In a, the similarity looks high with a small value of “7.0”, but by selecting the larger one, the resulting distance becomes large, and it is noted that there is an effect that the similarity can be output as low. I want to.

Although the first embodiment has been described above, according to this embodiment, instead of extracting a single vector as a feature amount for each image, a large number of feature amount vector groups are extracted. Then, the degree of similarity between groups is calculated in units of vector groups. As described above, the similarity between the vector groups has a property that it is possible to output the similar vector groups as having a high similarity and output the dissimilar vector groups as having a low similarity. is there. Therefore, the feature amount may remain as a vector group, and it is not necessary to use a single vector, so it is not necessary to devise a feature amount extraction method so as to prevent information loss. That is, it is possible to accurately extract the feature amount that faithfully reproduces the image feature for various images without devising the feature amount extraction method so that there is no loss of information, and to accurately express the similarity between images. An inter-similarity calculation device can be provided.

[Second Embodiment] Next, a second embodiment of the present invention will be described with reference to FIG.

The configuration of this embodiment is the same as that of the first embodiment, and the operation is also the same except for the function of the representative vector group distance calculation unit 23. Therefore, only the representative vector group distance calculation unit 23 will be described, and the other description will be omitted.

That is, in the present embodiment, the first representative vector group a calculated by the representative vector group calculating section 22.
The number of vectors of 3 and the number of vectors of the second representative vector group b3 are the same. Then, the first to second representative vector group distance ab calculated by the representative vector group distance calculating unit 23 is the same as that of each vector of the first representative vector group a3 and the second representative vector group b3. The distance to the most adjacent vector is a value obtained by summing up the respective vectors of the first representative vector group a3.

Similarly, the distance ba from the second representative vector group to the first representative vector group is the distance between each vector of the second representative vector group b3 and the most adjacent vector in the first representative vector group a3. The sum of the values of the respective vectors of the second representative vector group b3 is set.

That is, as described above, various calculation methods are conceivable for obtaining the distance between vector groups, but if the numbers of vectors of the representative vector group for each vector group are arranged in this way, There is an advantage that the calculation amount of distance calculation can be kept constant regardless of the number of vectors of the feature amount vector group. The number of vectors of the feature amount vector group often changes for each image depending on the size of the image, etc. However, with this embodiment, the calculation amount of distance calculation is constant regardless of the image size. If the number of representative vectors is set to a certain number or less, it is possible to reduce the calculation amount.

[Third Embodiment] Next, a third embodiment of the present invention will be described with reference to FIGS. 8 and 9A and 9B.

The configuration of this embodiment is the same as that of the first embodiment, and the operation is also the same except for the function of the representative vector group distance calculating unit 23. Therefore, only the representative vector group distance calculation unit 23 will be described, and the other description will be omitted.

That is, in the present embodiment, when the representative vector group distance calculating unit 23 calculates the first to second representative vector group distance ab, first, the second representative vector In the group b3, the number of vectors most adjacent to each vector of the first representative vector group a3 is obtained, and the ratio to the total number of the second representative vector group b3 is calculated. Subsequently, the larger (smaller) the ratio, the smaller (larger) the distance ab between the representative vector groups from the first to the second.

Similarly, in the representative vector group distance calculating unit 23, the second to first representative vector group distance b is obtained.
When calculating a, first, the first representative vector group a3
Among these, the number of vectors most adjacent to each vector of the second representative vector group b3 is obtained, and the ratio to the total number of the first representative vector group a3 is calculated. Subsequently, the larger (smaller) the ratio is, the smaller (larger) the distance ba between the representative vector groups from the second to the first is corrected.

Here, as a method of correction, for example, the following equation is used.

Correction distance ab '(first → second) = distance ab between representative vector groups from the first to second / proportion (in the second) correction distance ba ′ (second → first) = from the second Distance ba between representative vector groups to the first ratio / ratio (in the first) Distance ab ′ between representative vector groups from the first to second and distance b between representative vector groups from the second corrected to the above
With respect to a ′, the distance selecting unit 24 selects the larger one as the inter-image distance.

The effects of the present embodiment will be described with reference to FIGS. 9 (A) and 9 (B).

Although the distance between the vector groups has been described in the first embodiment, in general, between similar areas, as in Example 1 shown in FIG. The ratio (7/7 = 100%) of the number (7 in this example) of the representative vector of the corresponding region (A in this example) to the corresponding vector (7 in B in this example) tends to be large. There is. Therefore, at this time, if the distance is corrected to be smaller according to this ratio, it can be determined that the area A and the area B are more similar.

Similarly, between the dissimilar areas, as shown in FIG.
As shown in Example 2 in (B) of the above, for the total number of representative vectors (7 in B area in this example), the corresponding area (A in this example)
The number of adjacent representative vectors of a region (2 in this example)
The ratio (2/7 = 29%) tends to be small. Therefore, at this time, if the distance is corrected to increase according to this ratio, it can be determined that the areas A and B are not more similar.

Therefore, according to the present embodiment, the images that are similar to each other can have a higher degree of similarity and the images that are not similar to each other can have a lower degree of similarity. An inter-image similarity calculation device that can be expressed can be provided.

[Fourth Embodiment] Next, FIG. 10 and FIG.
A fourth embodiment of the present invention will be described with reference to 1 (A) and (B).

The configuration of this embodiment is the same as that of the first embodiment, and the operation is also the same except for the function of the representative vector group distance calculating unit 23. Therefore, only the representative vector group distance calculation unit 23 will be described, and the other description will be omitted.

That is, in the present embodiment, when the representative vector group distance calculating section 23 calculates the first to second representative vector group distances ab, first, the first feature amount is calculated. The size of the distribution of the vector group a2 or the first representative vector group a3 is obtained, and according to the size,
The larger (smaller) the distribution is, the smaller (larger) the distance ab between the representative vector groups from the first to the second is corrected.

Similarly, the size of the distribution of the second feature amount vector group b2 or the second representative vector group b3 is obtained, and the larger (smaller) the distribution size is, the larger the distribution size becomes. The distance ba between the representative vector groups from the second to the first is corrected to be small (large).

Here, as a method of correction, for example, the following equation is used.

Correction distance ab ″ (first → second) = distance ab between representative vector groups from first to second / size of distribution (first area) correction distance ba ″ (second → first) = Distance ba between representative vector groups from second to first / size of distribution (second region) Further, in combination with the third embodiment, the correction distance ab ″ (first → second) = Distance between representative vector groups from first to second ab / {distribution size (first region)
× Proportion (in the second)} Corrected distance ba ″ (second → first) = distance ba between representative vector groups from the second to the first / {size of distribution (second area)
Of course, it does not matter if x ratio (1st middle)}.

With respect to the first-to-second representative vector group distance ab ″ and the second-to-first representative vector group distance ba ″ corrected as described above, the distance selecting unit 24 determines the larger one. Select as the distance between images.

Although FIG. 10 shows a case where the correction method of the third embodiment is combined with the correction method of the third embodiment, it goes without saying that the correction method of the present embodiment may be performed independently.

The effect of the present embodiment will be described with reference to FIGS. 11 (A) and 11 (B).

Generally, an area having a small distribution tends to have a small distance from an area corresponding to another image, and an area having a large distribution tends to have a large distance. Therefore, at this time, depending on the size of this distribution (eg, the area A when the distance AB is obtained), the larger (smaller) the size of the distribution is,
By correcting the distance ab between the representative vector groups so as to be small (large), both in the case of Example 1 as shown in FIG. 11A and in the case of Example 2 as shown in FIG. 11B. , Can be corrected to almost the same degree of similarity.

The size of the distribution may be obtained from the feature vector group or the representative vector group. You can directly calculate the standard deviation etc. of the distribution, but for a certain area for which you want to find the size of the distribution, calculate the distances to various areas and calculate the maximum value (the distribution is The larger / smaller area also tends to have a larger / smaller maximum distance from an arbitrary area). That is, the following equation may be used.

Correction distance ab ″ (first → second) = distance ab between representative vector groups from the first to the second / the maximum correction distance of the distances between the first area and the constituent areas of other images ba ″ (second → first) = distance ba between representative vector groups from the second to the first / the maximum value of the distance between the second area and the constituent area of another image, according to the present embodiment, By correcting the similarity based on the size of the distribution, there is no system instability such that the similarity between images is judged differently depending on the distribution of the feature amount, and the similarity between images can be determined. It is possible to provide an inter-image similarity calculation device that can be expressed more accurately.

Although the present invention has been described based on the above embodiments, the present invention is not limited to the above embodiments, and various modifications and applications are possible within the scope of the gist of the present invention. Of course.

That is, the feature amount in the above embodiments is exemplified by vectorizing the RGB values of pixels, but of course,
Anything that can characterize the image, such as another color, shape, or texture, can be used. For example, an image may be divided into small blocks and pixels may be arranged as a vector for each block, or a co-occurrence matrix component, a GABOR filter output component, or the like may be used as a feature amount describing a texture.

Further, although the representative vector group calculation unit 22 uses a clustering method such as the self-organizing feature mapping or the K-means method or a vector quantization method to set the number smaller than the number of vectors of the feature amount vector group. When the number of feature amount vector groups is small or when the feature amount vector group is realized by using a computer having high calculation ability, the feature amount vector group may be used as it is as the representative vector group. In that case, the representative vector group calculation unit 22 may directly set all the feature amount vectors as the representative vector without performing a special setting process.

Further, the similarity after the similarity between images is calculated may be used for similar image retrieval, or in order to collect a group of images into several similar scenes, any similar scenes may be used. It may be used as a target of the threshold processing of whether or not it belongs to.

[0083]

As described in detail above, according to the present invention,
When comparing features such as color information, grayscale information, background, main subject, etc. between two images and calculating the similarity between images, there is no need to devise a feature extraction method so that there is no loss of information. To provide an inter-image similarity calculation device, an inter-image similarity calculation method, and a program capable of extracting a feature amount that faithfully reproduces image features of various images and accurately expressing the similarity between images. You can

[Brief description of drawings]

FIG. 1 is a functional block diagram of an inter-image similarity calculation device according to a first embodiment of the present invention.

FIG. 2A is a block configuration diagram of an inter-image similarity calculation device according to the first embodiment, and FIG. 2B is a diagram showing an operation flowchart thereof.

FIG. 3 is a conceptual diagram for explaining a relative distance between vector groups.

FIG. 4 is a diagram for explaining a property of a relative distance.

FIG. 5 is a diagram for explaining a similarity calculation method according to the first embodiment.

FIG. 6 is a diagram for explaining a similarity calculation method according to the first embodiment.

FIG. 7 is a functional block diagram of an inter-image similarity calculation device according to a second embodiment of the present invention.

FIG. 8 is a functional block diagram of an inter-image similarity calculation device according to a third embodiment of the present invention.

FIG. 9 is a diagram for explaining the effect of the third embodiment.

FIG. 10 is a functional block diagram of an inter-image similarity calculation device according to a fourth embodiment of the present invention.

FIG. 11 is a diagram for explaining the effect of the fourth embodiment.

FIG. 12 is a diagram for explaining a conventional inter-image similarity calculation method.

FIG. 13 is a diagram illustrating a method of calculating a feature vector in a conventional inter-image similarity calculation method.

[Fig. 14] Fig. 14 is a diagram for explaining a problem of a conventional inter-image similarity calculation method.

[Explanation of symbols]

10 Input device 11 Image input section 20 arithmetic unit 20A CPU 20B program memory 21 Feature Vector Group Calculation Unit 22 Representative vector group calculator 23 Representative Vector Group Distance Calculation Unit 24 Distance Selector 25 Similarity output section 30 Output device

Claims (7)

[Claims] 1. An inter-image similarity calculation apparatus for calculating an inter-image similarity between a first comparison target image and a second comparison target image whose image features are to be compared, wherein the first comparison target image and the first comparison target image are compared with each other. 2 From an image input unit for inputting a comparison target image, the first comparison target image and the second comparison target image,
A first feature amount vector group and a second feature amount vector group showing the features of each image.
A feature quantity vector group calculation unit that calculates a feature quantity vector group, and a distance between the first and second vector groups that indicates a relative distance of the second feature quantity vector group from the first feature quantity vector group, An inter-vector-group distance calculator that calculates a second to first inter-vector-group distance that indicates the relative distance of the first feature-quantity vector group from the viewpoint of the second feature-quantity vector group; A distance selecting unit that selects a larger one of the inter-vector group distance to 2 and the second to first vector group distance as the inter-image distance, and the inter-image similarity using the inter-image distance. An inter-image similarity calculation device comprising: a similarity output unit that defines and outputs. 2. An inter-image similarity calculation apparatus for calculating an inter-image similarity between a first comparison target image and a second comparison target image whose image features are to be compared, wherein the first comparison target image and the first comparison target image are compared with each other. 2 From an image input unit for inputting a comparison target image, the first comparison target image and the second comparison target image,
A first feature amount vector group and a second feature amount vector group showing the features of each image.
A feature quantity vector group calculating unit that calculates a feature quantity vector group, and a first representative vector group and a first representative vector group that represent each feature quantity vector group from the first feature quantity vector group and the second feature quantity vector group. A representative vector group calculation unit that calculates two representative vector groups; a first to second representative vector group distance indicating a relative distance of the second representative vector group viewed from the first representative vector group; 2 A representative vector group distance calculating unit that calculates a second to first representative vector group distance indicating the relative distance of the first representative vector group viewed from the representative vector group, and the first to second Between the representative vector groups of the
From the first to the first representative vector group distance, a distance selecting unit that selects a larger one as the image distance, a similarity output unit that defines and outputs the image similarity using the image distance, An inter-image similarity calculation device comprising: 3. The number of vectors of the first representative vector group and the number of vectors of the second representative vector group calculated by the representative vector group calculation unit are the same, and calculated by the distance between representative vector group calculation unit. The distance between the first representative vector group and the second representative vector group is each vector of the first representative vector group, which is the distance between each vector of the first representative vector group and the nearest neighbor vector in the second representative vector group. And the distances from the second representative vector group to the first representative vector group calculated by the representative vector group distance calculating unit are the respective vectors of the second representative vector group and the nearest neighbors of the first representative vector group. The inter-image similarity calculation device according to claim 2, wherein a distance to a vector is a total sum for each vector of the second representative vector group. 4. When calculating the distance between the first representative vector group and the second representative vector group in the representative vector group calculating unit, each vector of the first representative vector group with respect to the total number of the second representative vector group And the ratio of the number of second representative vectors that are closest to each other is calculated, and the larger the ratio, the first to second
To the total number of the first representative vector groups when the representative vector group calculating unit calculates the distance from the second representative vector group to the first representative vector group. , Calculating the ratio of the number of first representative vectors closest to each vector of the second representative vector group,
The inter-image similarity calculation device according to claim 3, wherein the larger the ratio is, the smaller the distance between the representative vector groups from the second to the first is. 5. The distribution of the first feature vector group or the first representative vector group when calculating the distance between the first representative vector group and the second representative vector group in the representative vector group distance calculating unit. According to the magnitude, the larger the distribution is, the smaller the distance between the representative vector groups from the first to the second is corrected, and the distance between the representative vector groups is calculated from the second to the second. In calculating the distance between the representative vector groups to 1, the larger the distribution size of the second feature vector group or the second representative vector group is, the larger the distribution size of the second feature vector group or the second representative vector group becomes. The inter-image similarity calculation device according to claim 3 or 4, wherein the distance to the first representative vector group is corrected to be small. 6. An inter-image similarity calculation method for calculating inter-image similarity between a first comparison target image and a second comparison target image for which image features are to be compared, the first comparison target image and the first comparison target image 2 comparison target image is input, and from the first comparison target image and the second comparison target image,
A first feature amount vector group and a second feature amount vector group showing the features of each image.
Calculating a characteristic amount vector group, and calculating a first representative vector group and a second representative vector group representing each characteristic amount vector group from the first characteristic amount vector group and the second characteristic amount vector group. Of the first representative vector group from the first representative vector group and the second representative vector group distance from the first representative vector group indicating the relative distance of the second representative vector group from the first representative vector group. A second to first representative vector group distance indicating a relative distance is calculated, and the first to second representative vector group distance and the second representative vector group distance are calculated.
To the first representative vector group distance, the larger one is selected as the image distance, and the image similarity is defined and output using the image distance, and the image similarity is characterized by Calculation method. 7. The first comparison target image and the second comparison target image are calculated by a computer when calculating the inter-image similarity between the first comparison target image and the second comparison target image whose image features are to be compared. From the function of inputting the target image and the first comparison target image and the second comparison target image,
A first feature amount vector group and a second feature amount vector group showing the features of each image.
A first representative vector group and a second representative vector group which represent each of the feature amount vector groups from the first feature amount vector group and the second feature amount vector group. And a distance between first and second representative vector groups indicating the relative distance of the second representative vector group viewed from the first representative vector group, and the first representative vector group viewed from the second representative vector group. A function of calculating a second to first distance between representative vector groups indicating a relative distance of the representative vector group; and a distance from the first to second representative vector group and the second
From the first to the first representative vector group distance, a function of selecting the larger one as the image distance, and a function of defining and outputting the image similarity using the image distance, program.