JP2007257470A - Similarity discrimination device, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively execute processing of similarity discrimination whether two images are similar or not. <P>SOLUTION: A histogram calculation part 32 calculates histograms of the two images of which similarity is determined. A feature amount calculation part 34 divides the two histograms to a prescribed number of regions, and calculates a representative value of the respective regions, and calculates a feature amount showing correlation of the region for each of the regions to which the two histograms corresponds. A discrimination part 36 executes similarity discrimination of the two images based on the feature amount. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a similarity determination apparatus and method for determining whether two images are similar, and a program for causing a computer to execute the similarity determination method.

Conventionally, a plurality of images have been classified into a plurality of groups based on information on shooting time and shooting location given to each image (see Patent Document 1). Also, for each of a plurality of images, a feature amount such as color, brightness, frequency information, and edge information is calculated, and this feature amount is weighted to calculate a comprehensive feature amount, which is compared between images. Thus, a method for classifying a plurality of images into a group composed of a plurality of similar images has been proposed (see Patent Documents 1 and 2). Further, a method has been proposed in which learning is performed using a similar image as a learning pattern in advance, and whether or not two given images are similar using the learning result is determined (see Patent Document 3). Also proposed is a method of determining the orthogonal transformation coefficient of two images that are the target of similarity determination, obtaining a color histogram representing the appearance frequency of each color included in the orthogonal transformation coefficient, and comparing the color histograms to perform similarity determination. (See Patent Document 4). Furthermore, a technique has also been proposed in which a histogram of partial areas of two images to be subjected to similarity determination is obtained, and the similarity between the two images is obtained by determining the similarity between the histograms (see Patent Document 5). By using the histogram in this way, it is possible to easily determine the similarity between two images with a small amount of calculation.
JP 2000-112997 A JP 2000-48181 A JP 2000-90113 A JP 2002-82985 A JP 2002-163653 A

  However, in the technique of performing similarity determination using the histograms described in Patent Documents 4 and 5, similarity determination is performed by obtaining a histogram correlation between corresponding values in the histogram (that is, values on the horizontal axis of the histogram). As a result, the correlation between the corresponding regions in the histogram is lowered only by slightly different colors and brightness of the two images. For this reason, depending on the method of determining similarity using histograms as described in Patent Documents 4 and 5, it is determined that two images that are determined to be similar if viewed by humans are not similar. There is a risk that.

  The present invention has been made in view of the above circumstances, and an object thereof is to effectively perform similarity determination processing.

The similarity determination apparatus according to the present invention is a similarity determination apparatus that determines whether two images are similar,
Histogram calculating means for calculating a histogram of hues of the two images;
A feature value for dividing the two histograms into a predetermined number of regions, calculating a representative value of the frequency of each region, and calculating a feature value representing a correlation between the representative values for each corresponding region of the two histograms A calculation means;
And a discriminating means for discriminating the similarity between the two images based on the feature amount of each region.

  In the similarity determination apparatus according to the present invention, when calculating the feature value, the feature value calculation unit calculates the feature value by referring to the frequency of the histogram in the vicinity of the corresponding region. It may be a means.

  In the similarity determination apparatus according to the present invention, the determination means calculates a total feature amount by weighting and adding the feature amount of each region with a predetermined weighting factor, and the similarity based on the total feature amount It is good also as a means to perform discrimination.

  In this case, the predetermined weighting factor is a plurality of feature values obtained from a plurality of correct answer data composed of two sample images known to be similar to a plurality of feature images obtained from two sample images known to be dissimilar. It is good also as a weighting coefficient which weights the said feature-value obtained by learning beforehand the said feature-value obtained from incorrect answer data.

  As the learning, for example, a machine learning technique can be used. As a machine learning method, for example, a known method such as a neural network or boosting can be used.

The similarity determination method according to the present invention is a similarity determination method for determining whether or not two images are similar.
Calculating a hue histogram of the two images;
Dividing the two histograms into a predetermined number of regions, calculating a representative value of the frequency of each region,
For each corresponding region of the two histograms, a feature amount representing the correlation between the representative values is calculated,
The similarity determination of the two images is performed based on the feature amount of each region.

  In addition, you may provide as a program for making a computer perform the similarity determination method by this invention.

  According to the present invention, the hue histograms of two images are calculated, the two histograms are divided into a predetermined number of regions, and the representative value of each region is calculated. Then, a feature amount representing a correlation between representative values is calculated for each corresponding region of the two histograms, and similarity determination between the two images is performed based on the calculated correlation feature amount of each region. As described above, in the present invention, when calculating the feature value representing the correlation between the two histograms, the representative value of each region is calculated. It will be possible to absorb such changes. Therefore, it can be determined that two images having slightly different colors, which are determined not to be similar if the correlation between the histograms is seen, can be determined to be similar, and thus a similarity determination that is strong against changes in the image can be performed.

  Further, when calculating the feature amount, by calculating the feature amount with reference to the frequency of the histogram of not only the region in which the feature amount is calculated but also the neighborhood of the region, when calculating the feature amount, The change in color between the two images can be further absorbed. Therefore, it is possible to perform a strong similarity determination based on a change in the image.

  Also, by calculating the total feature amount by weighting and adding the feature amount with a predetermined weighting factor, and using the weighting factor obtained by learning in advance when performing similarity determination based on the total feature amount Similarity determination can be performed using a weighting factor suitable for an existing image. Therefore, it is possible to more effectively determine the similarity between images.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing a configuration of a similarity determination apparatus according to an embodiment of the present invention. As shown in FIG. 1, the similarity determination apparatus 1 according to the present embodiment performs various controls such as recording control and display control of image data, and also controls each part of the apparatus 1 and a program for operating the CPU 12. A system memory 14 including a viewer software for browsing images, a ROM in which various constants are recorded, and a RAM serving as a work area when the CPU 12 executes processing, and various instructions to the apparatus 1 An input unit 16 including a keyboard and a mouse for performing the display and a display unit 18 including a liquid crystal monitor for performing various displays are provided.

  In addition, the similarity determination apparatus 1 includes a card slot 20 for reading image data from the memory card 2 on which image data is recorded, and recording image data on the memory card 2, and a technique represented by JPEG for the image data. A compression / decompression unit 22 that compresses or decompresses compressed image data; a hard disk 24 that records image data and various programs executed by the CPU 12; and a memory control that controls the system memory 14, the card slot 20, and the hard disk 24 Unit 26 and a display control unit 28 for controlling the display of the display unit 18.

  The similarity determination apparatus 1 includes a low-frequency image creation unit 30, a histogram calculation unit 32, a feature amount calculation unit 34, and a determination unit 36.

  Next, processing performed by the low-frequency image creation unit 30, the histogram calculation unit 32, the feature amount calculation unit 34, and the determination unit 36 will be described. FIG. 2 is a flowchart showing processing performed in the present embodiment. It is assumed that the two images that are subjected to similarity determination are read from the memory card 2 and stored in the hard disk 24. When the user issues an instruction for similarity determination from the input unit 16, the CPU 12 starts processing, and the low-frequency image creation unit 30 creates a low-frequency image representing a low-frequency component for each of the two images for which similarity determination is performed. (Step ST1).

  Specifically, the low-frequency image creation unit 30 has a predetermined resolution lower than that of the image before processing by performing wavelet transform on the image or by reducing the size of the pixels constituting the image. Create a low-frequency image. Here, when the wavelet transform is used, the LL component obtained by the wavelet transform is used as the low frequency image. When reducing the image, it is possible to effectively remove the high-frequency component of the image by reducing it using a higher-order interpolation equation such as a Cubic spline rather than simply reducing it by linear transformation. The influence of noise in similarity determination can be removed. In addition, when the resolutions of the two images are different, the low-frequency image creation unit 30 creates a low-frequency image so that the resolutions of the two images match. Further, in order to easily perform subsequent processing, the image may be deformed so as to be a square.

  Next, the histogram calculation unit 32 calculates the hue at each pixel constituting the image (step ST2). Further, the histogram calculation unit 32 calculates a hue histogram for each of the two images by plotting the hue value on the horizontal axis and the number of appearances of the hue on the vertical axis (step ST3). Specifically, since the hue takes a value of 0 to 359 degrees, the scale of the hue is calculated by setting the scale of the horizontal axis to 0 to 359 degrees and plotting the number of appearances in increments of 1 degree.

  Next, the feature amount calculation unit 34 divides the hue histograms of the two images calculated by the histogram calculation unit 32 into a predetermined number of regions, respectively (step ST4).

  FIG. 3 is a diagram for explaining division of regions in the hue histogram. As illustrated in FIG. 3, the feature amount calculation unit 34 divides the hue histogram H <b> 0 into a predetermined number of regions (eight in this case). The number of divisions is not limited to 8 divisions, and may be set as appropriate, such as 4 divisions, 16 divisions, 32 divisions, or the like.

  Further, the feature amount calculation unit 34 calculates an average value of appearance frequencies for each divided region (step ST5). Specifically, for each of the two histograms, the sum of frequencies is obtained for each of a plurality of regions Ck (k = 1 to n, where n is the number of divisions) obtained by the division, and this is expressed by the number of steps in the region. By dividing, an average value Mk of frequencies is calculated for each divided region Ck. FIG. 4 is a diagram showing a histogram of average values for each divided area. Since the histogram is created in increments of 1 to 0 to 359 degrees, when the division number is 8, the number of increments in each region Ck is 45. Instead of the average value Mk, the total frequency, median value, maximum value, minimum value, or the like in each region Ck may be used.

  Further, the feature amount calculation unit 34 flattens the average value Mk for each region Ck (step ST6). Specifically, according to the following equation (1), the average value Mk of the region Ck is weighted and added to the average values Mk−1 and Mk + 1 of two adjacent regions Ck−1 and Ck + 1. A flattened average value Mk ′ is calculated by flattening the average value Mk for each region Ck.

Mk '= a.Mk-1 + b.Mk + c.Mk + 1 (1)
However, a, b, and c are weighting factors, and for example, values of a = c = 0.25 and b = 0.5 are used so that a + b + c = 1.

  Here, since the hue takes a value of 0 to 359 degrees and the hue value is looped, the flattening average value M1 ′ is calculated using the average value Mn as shown in the following formula (2). can do. Further, the flattening average value Mn ′ can be calculated using the average value M1 as shown in the following formula (3).

M1 '= a.Mn + b.M1 + c.M2 (2)
Mn '= a.Mn-1 + b.Mn + c.M1 (3)
Here, when flattening the average value Mi, the average value of two areas adjacent to the area where the flattened average value is calculated is used, but flattening is performed using the average value of more adjacent areas. May be performed. For example, as shown in the following formula (4), an average value of four adjacent regions may be used.

Mk '= a.Mk-2 + b.Mk-1 + c.Mk + d.Mk + 1 + e.Mk + 2 (4)
However, a, b, c, d, e are weighting factors, and a + b + c + d + e = 1.

  Further, the feature amount calculation unit 34 calculates a feature amount Tk representing the correlation between the two histograms by calculating the reciprocal of the absolute value of the difference between the flattened average values Mk ′ between the corresponding regions in the two histograms. (Step ST7). Here, the feature amount Tk represents the correlation between the regions Ck of the two histograms, and the larger the value, the higher the degree of similarity between the regions Ck.

  Subsequently, the determination unit 36 calculates the total similarity by weighting and adding the plurality of feature amounts Tk calculated for the two images according to the following equation (4) (step ST8).

Total similarity Sm1 = ΣαkTk (4)
Here, αk is a weighting coefficient for weighting the feature amount Tk. The weighting coefficient αk was obtained from feature values obtained from a large number of correct answer data consisting of two sample images known to be similar to each other and from a lot of incorrect answer data consisting of two sample images known to be not similar to each other. The feature amount is obtained by learning in advance using, for example, a machine learning method.

  During learning, the number of histogram divisions is set to various division numbers such as 8 divisions, 16 divisions, and 32 divisions, so that the weighting coefficient αk corresponding to the division number of the color region is obtained. Can do. Further, by performing learning using various types of images such as landscape images, person images, and template images as sample images, the weighting coefficient αk corresponding to the type of image can be obtained. As a result, if the type of image to be subjected to similarity determination is set in advance, the total similarity Sm1 can be calculated using the optimum weighting factor αk corresponding to the type of image.

  In addition, by performing learning for each user, it is possible to obtain the optimum weighting factor αk for the user who has acquired the image for which similarity determination is performed. Therefore, the overall similarity is determined using the weighting factor αk corresponding to the user's shooting tendency. The degree Sm1 can be calculated.

  Then, the determination unit 36 determines whether or not the total similarity Sm1 is equal to or greater than a predetermined threshold value Th1 (step ST9). If step ST9 is affirmed, it is determined that the two images are similar (step ST10). The process is terminated. If step ST9 is negative, it is determined that the two images are not similar (step ST11), and the process ends.

  As described above, in the present embodiment, when calculating the feature value Tk representing the correlation between two histograms, the average value Mk of each region Ck of the histogram is calculated. Therefore, when calculating the feature value Tk, A slight change in the colors of the two images can be absorbed. Therefore, it is possible to determine that two images having slightly different feature amounts are similar, which are determined not to be similar if the correlation of the histogram is seen, thereby making it possible to perform similarity determination that is strong against changes in the image. .

  Further, by calculating the feature quantity Tk with reference to the frequency of the histogram in the vicinity of each divided area, it is possible to further absorb slight changes in the colors of the two images when calculating the feature quantity Tk. As a result, it is possible to perform a similarity determination stronger with changes in the image.

  In addition, by learning the weighting coefficient αk for weighting the feature amount in advance, it is possible to perform similarity determination using a weighting coefficient suitable for an existing image, so that image similarity determination is performed more effectively. Can do.

  In the above embodiment, the weighting factor αk is obtained by learning. However, a weighting factor αk that is artificially set in advance may be used.

  Although the apparatus according to the embodiment of the present invention has been described above, the computer functions as a unit corresponding to the low-frequency image creation unit 30, the histogram calculation unit 32, the feature amount calculation unit 34, and the determination unit 36, and is illustrated in FIG. A program that performs the processing as shown is also one embodiment of the present invention. A computer-readable recording medium that records such a program is also one embodiment of the present invention. Further, such a program may be incorporated in viewer software for browsing images.

1 is a schematic block diagram showing the configuration of a similarity determination device according to an embodiment of the present invention. A flowchart showing processing performed in the present embodiment Diagram showing segmentation in hue histogram The figure which shows the histogram of the average value for every divided area

Explanation of symbols

1 Similarity Discriminator 2 Memory Card 12 CPU
DESCRIPTION OF SYMBOLS 14 System memory 16 Input part 18 Display part 20 Card slot 22 Compression / decompression part 24 Hard disk 26 Memory control part 28 Display control part 30 Low frequency image creation part 32 Color histogram calculation part 34 Feature-value calculation part 36 Discrimination part

Claims (6)

In a similarity determination device that determines whether two images are similar,
Histogram calculating means for calculating a histogram of hues of the two images;
A feature value for dividing the two histograms into a predetermined number of regions, calculating a representative value of the frequency of each region, and calculating a feature value representing a correlation between the representative values for each corresponding region of the two histograms A calculation means;
A similarity determination apparatus comprising: a determination unit configured to determine similarity between the two images based on a feature amount of each region.   2. The feature amount calculating unit is a unit that calculates the feature amount by referring to a frequency of the histogram in a region near the corresponding region when calculating the feature amount. Similarity discrimination device of description.   The discrimination means is means for calculating a total feature quantity by weighting and adding the feature quantity of each region with a predetermined weighting factor, and performing the similarity discrimination based on the total feature quantity. The similarity determination apparatus according to claim 1 or 2.   The predetermined weighting factor is a plurality of incorrect answer data consisting of two sample images known to be similar to the feature quantity obtained from a large number of correct answer data consisting of two sample images known to be similar. The similarity determination apparatus according to claim 3, wherein the similarity determination device is a weighting coefficient for weighting the feature amount, which is obtained by learning the feature amount obtained from the above in advance. In a similarity determination method for determining whether or not two images are similar,
Calculating a hue histogram of the two images;
Dividing the two histograms into a predetermined number of regions, calculating a representative value of the frequency of each region,
For each corresponding region of the two histograms, a feature amount representing the correlation between the representative values is calculated,
A similarity determination method, wherein similarity determination of the two images is performed based on a feature amount of each region. In a program for causing a computer to execute a similarity determination method for determining whether two images are similar to each other,
A procedure for calculating a histogram of hues of the two images;
A procedure for dividing the two histograms into a predetermined number of regions and calculating a representative value of the frequency of each region;
A procedure for calculating a feature amount representing a correlation between the representative values for each of the corresponding regions of the two histograms;
A program for determining similarity between the two images based on a feature value of each region.

Images