JP2007257466A - Similarity discrimination device, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively execute similarity discrimination whether two images are similar or not. <P>SOLUTION: A low frequency image generating part 30 generates a low frequency image showing a low frequency component of two images of which similarity is discriminated. A color histogram calculation part 32 calculates a color histogram of the two low frequency images. A feature amount calculation part 34 divides the two color histograms to a prescribed number of color regions, and a feature amount showing correlation of the two color histograms is calculated for each of the divided color regions. A discrimination part 36 executes similarity discrimination of the two images based on the feature amount. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a similarity determination apparatus and method for determining whether two images are similar, and a program for causing a computer to execute the similarity determination method.

Conventionally, a plurality of images have been classified into a plurality of groups based on information on shooting time and shooting location given to each image (see Patent Document 1). Also, for each of a plurality of images, a feature amount such as color, brightness, frequency information, and edge information is calculated, and this feature amount is weighted to calculate a comprehensive feature amount, which is compared between images. Thus, a method for classifying a plurality of images into a group composed of a plurality of similar images has been proposed (see Patent Documents 1 and 2). Further, a method has been proposed in which learning is performed using a similar image as a learning pattern in advance, and whether or not two given images are similar using the learning result is determined (see Patent Document 3). Also proposed is a method of determining the orthogonal transformation coefficient of two images that are the target of similarity determination, obtaining a color histogram representing the appearance frequency for each color included in the orthogonal transformation coefficient, and comparing the color histograms to perform similarity determination. (See Patent Document 4). Here, since the color histogram is hardly affected by the position and deformation of the subject included in the image, similarity determination that is strong against movement and deformation of the subject can be performed by performing similarity determination using the color histogram.
JP 2000-112997 A JP 2000-48181 A JP 2000-90113 A JP 2002-82985 A

  However, in the method of performing similarity determination using the color histogram described in Patent Document 4, the comparison of the color histograms of two images is performed for all the colors included in each image. The process has a very long time.

  The present invention has been made in view of the above circumstances, and an object thereof is to efficiently perform a similarity determination process.

The similarity determination apparatus according to the present invention is a similarity determination apparatus that determines whether two images are similar,
Low-frequency image creation means for creating a low-frequency image representing each low-frequency component of the two images;
Color histogram calculating means for calculating a color histogram of the two low-frequency images;
A feature amount calculating means for dividing the two color histograms into a predetermined number of color regions and calculating a feature amount representing a correlation between the two color histograms for each of the divided color regions;
And a discriminating means for discriminating the similarity between the two images based on the feature amount of each color region.

  In the similarity determination apparatus according to the present invention, the feature amount calculation means may be means for dividing the two color histograms according to human visual characteristics.

  In the similarity determination apparatus according to the present invention, the determination unit calculates two total feature amounts by weighting and adding the feature amounts of the respective color regions with a predetermined weighting coefficient, and the two total feature amounts are calculated. Means for performing the similarity determination based on the difference may be used.

  Further, in the similarity determination apparatus according to the present invention, it is known that the predetermined weight coefficient is not similar to the feature amount obtained from a plurality of correct answer data composed of two sample images that are known to be similar. A weighting factor for weighting the feature amount obtained by learning in advance the feature amount obtained from a large number of incorrect answer data composed of two sample images may be used.

  As the learning, for example, a machine learning technique can be used. As a machine learning method, for example, a known method such as a neural network or boosting can be used.

The similarity determination method according to the present invention is a similarity determination method for determining whether or not two images are similar.
Creating a low frequency image representing the low frequency component of each of the two images;
Calculating a color histogram of the two low frequency images;
Dividing the two color histograms into a predetermined number of color regions;
Calculating a feature amount representing a correlation between the two color histograms for each of the divided color regions;
The similarity determination of the two images is performed based on the feature amount of each color region.

  In addition, you may provide as a program for making a computer perform the similarity determination method by this invention.

  According to the present invention, a low frequency image representing each low frequency component of two images is created, and a color histogram of the two created low frequency images is calculated. Next, the two color histograms are divided into a predetermined number of color regions, and a feature amount representing the correlation between the two color histograms is calculated for each divided color region. Then, similarity determination between the two images is performed based on the calculated feature amount of each color region. As described above, according to the present invention, the color histogram is divided into a predetermined number of color regions, and the feature amount representing the correlation is calculated for each of the divided color regions. Therefore, the color histograms are compared for all the colors included in the image. Compared to the case, the calculation time for similarity determination can be shortened. In addition, since similarity determination is performed using a low-frequency image representing a low-frequency component of the image, the calculation time for similarity determination can be shortened compared to the case where the image itself is used. In addition, since the correlation is obtained for each color area, even if the image contains noise, the noise can be absorbed in the color area, and thus the accuracy is not affected by the noise. A determination can be made.

  Further, by dividing the two color histograms according to the human visual characteristics, it is possible to perform similarity determination reflecting the human visual characteristics.

  Also, learning is performed in advance when calculating the two total feature amounts by weighting and adding the feature amounts for each color region with a predetermined weighting coefficient, and performing similarity determination based on the difference between the two total feature amounts. By using the weighting coefficient obtained by the above, similarity determination can be performed using a weighting coefficient suitable for an existing image. Therefore, it is possible to more effectively determine the similarity between images.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing a configuration of a similarity determination apparatus according to an embodiment of the present invention. As shown in FIG. 1, the similarity determination apparatus 1 according to the present embodiment performs various controls such as recording control and display control of image data, and also controls each part of the apparatus 1 and a program for operating the CPU 12. A system memory 14 including a viewer software for browsing images, a ROM in which various constants are recorded, and a RAM serving as a work area when the CPU 12 executes processing, and various instructions to the apparatus 1 An input unit 16 including a keyboard and a mouse for performing the display and a display unit 18 including a liquid crystal monitor for performing various displays are provided.

  In addition, the similarity determination apparatus 1 includes a card slot 20 for reading image data from the memory card 2 on which image data is recorded, and recording image data on the memory card 2, and a technique represented by JPEG for the image data. A compression / decompression unit 22 that compresses or decompresses compressed image data; a hard disk 24 that records image data and various programs executed by the CPU 12; and a memory control that controls the system memory 14, the card slot 20, and the hard disk 24 Unit 26 and a display control unit 28 for controlling the display of the display unit 18.

  The similarity determination device 1 includes a low-frequency image creation unit 30, a color histogram calculation unit 32, a feature amount calculation unit 34, and a determination unit 36.

  Next, processing performed by the low-frequency image creation unit 30, the color histogram calculation unit 32, the feature amount calculation unit 34, and the determination unit 36 will be described. FIG. 2 is a flowchart showing processing performed in the present embodiment. It is assumed that the two images that are subjected to similarity determination are read from the memory card 2 and stored in the hard disk 24. When the user issues an instruction for similarity determination from the input unit 16, the CPU 12 starts processing, and the low-frequency image creation unit 30 creates a low-frequency image representing a low-frequency component for each of the two images for which similarity determination is performed. (Step ST1).

  Specifically, the low-frequency image creation unit 30 has a predetermined resolution lower than that of the image before processing by performing wavelet transformation on the image or by reducing the size of pixels constituting the image. Create a low-frequency image. Here, when the wavelet transform is used, the LL component obtained by the wavelet transform is used as the low frequency image. When reducing the image, it is possible to effectively remove high-frequency components of the image by reducing it using a higher-order interpolation equation such as Cubic spline rather than simply reducing it by linear transformation. The influence of noise in similarity determination can be removed. Further, when the resolutions of the two images are different, the low-frequency image creation unit 30 creates a low-frequency image so that the resolutions of the two images match. Further, in order to easily perform subsequent processing, the image may be deformed so as to be a square.

  Next, the color histogram calculation unit 32 calculates a color vector in each pixel constituting the image (step ST2). Here, when the image is composed of RGB colors, pixel values of R, G, and B colors are calculated as color vectors. If the image is composed of YCC components, the values of Y, Cb, and Cr components are calculated as color vectors. Further, when the image is composed of HSV components, the values of the H, S, and V components are calculated as color vectors.

In the present embodiment, it is assumed that an image is composed of RGB colors, and pixel values of R, G, and B colors are used as color vectors. Here, when two images have 8-bit color numbers, values of 0 to 255 are obtained for each of the R, G, and B colors, so that 256 ³ color vectors are calculated.

  Further, the color histogram calculation unit 32 calculates a color histogram by plotting on the color space the number of pixels that appear for all the calculated color vectors for each of the two images (step ST3). FIG. 3 is a diagram showing a color histogram. Here, since the color histogram plots the number of appearances for a three-dimensional color vector composed of three components R, G, and B, it is obtained by plotting the number of appearances in the RGB three-dimensional space. However, FIG. 3 shows a state in which the color histogram is represented two-dimensionally for convenience.

The color histogram shown in FIG. 3 is obtained by plotting the value of the color vector on the horizontal axis and the number of appearances of each color vector on the vertical axis. Since the horizontal axis takes 256 ³ values from (R, G, B) = (0, 0, 0) to (255, 255, 255), in order to simplify the explanation, FIG. Only a part of the color histogram is shown.

  Next, the feature amount calculation unit 34 divides the color histograms for the two images calculated by the color histogram calculation unit 32 into a predetermined number of color regions, respectively (step ST4).

FIG. 4 is a diagram for explaining division of color regions in the color histogram. As shown in FIG. 4, the feature amount calculation unit 34 divides the color histogram into a predetermined number of color regions (eight in this case). The number of divisions is not limited to 8 divisions, and may be set as appropriate, such as 4 divisions, 16 divisions, 32 divisions, or the like. In practice, since the color histogram exists in the RGB three-dimensional space, each color component is divided into a predetermined number of color regions in the three-dimensional space. For example, when the division number of each color component is 4, divided into 4 ^three color regions as shown in FIG.

  Further, the feature amount calculation unit 34 calculates a feature amount representing the correlation between the two color histograms for each divided color region (step ST5). Specifically, for two color histograms, the sum of frequencies is obtained for each of a plurality of color regions Ci (i = 1 to n, n is the number of divisions) obtained by division, and this is used as the number of color vectors in the region. The frequency average value is calculated as a statistical value for each divided color area Ci. Instead of the average value, the total frequency, median value, maximum value, minimum value, or the like may be used as the statistical value. Then, with respect to the two color histograms, the reciprocal of the absolute value of the difference between the statistical values between the corresponding color regions Ci is calculated as the feature amount Ti. Here, the feature amount Ti represents the correlation between the color regions Ci of the two color histograms, and the greater the value, the higher the degree of similarity between the color regions Ci.

  Subsequently, the determination unit 36 calculates the total similarity by weighting and adding the plurality of feature amounts Ti calculated for the two images according to the following equation (1) (step ST6).

Total similarity Sm = ΣαiTi (1)
Here, αi is a weighting coefficient for weighting the feature amount Ti. The weighting factor is a feature amount obtained from a large number of correct answer data composed of two sample images known to be similar to a feature amount obtained from a large number of incorrect answer data composed of two sample images known to be dissimilar. The quantity is obtained by learning in advance using, for example, a machine learning method.

  At the time of learning, the weight coefficient αi corresponding to the number of divisions of the color area is obtained by setting the number of divisions of the color histogram to various division numbers such as 8, 16 and 32 divisions. be able to. Further, by performing learning using various types of images such as landscape images, person images, and template images as sample images, the weighting coefficient αi corresponding to the image type can be obtained. As a result, if the type of an image to be subjected to similarity determination is set in advance, the total similarity Sm can be calculated using the optimum weighting coefficient αi corresponding to the type of image.

  In addition, by performing learning for each user, it is possible to obtain the optimum weighting factor αi for the user who has acquired the image for which similarity determination is performed. The degree Sm can be calculated.

  Then, the determination unit 36 determines whether or not the total similarity Sm is equal to or greater than a predetermined threshold value Th1 (step ST7). If step ST7 is affirmed, it is determined that the two images are similar (step ST8). The process is terminated. If step ST7 is negative, it is determined that the two images are not similar (step ST9), and the process ends.

  As described above, in the present embodiment, the color histogram is divided into a predetermined number of color areas, and the feature amount representing the correlation is calculated for each divided color area. Therefore, the color histograms are calculated for all the colors included in the image. Compared with the case of comparison, the calculation time for similarity determination can be shortened. In addition, since the correlation is obtained for each color area, even if noise is included, the noise can be absorbed in the color area, and thereby, it is possible to perform similarity determination with high accuracy without being affected by noise. It can be carried out.

  In addition, since similarity determination is performed using a low-frequency image representing a low-frequency component of the image, the calculation time for similarity determination can be shortened compared to the case where the image itself is used.

  Moreover, a low frequency image can be easily obtained by creating a low frequency image by performing wavelet transform on two images. In addition, since high-frequency components can be removed by wavelet transform, similarity determination that is strong against noise can be performed.

  Also, by learning the weighting coefficient αi for weighting the feature amount in advance, the similarity determination can be performed using the weighting coefficient suitable for the existing image, so that the image similarity determination is performed more effectively. Can do.

  In the above-described embodiment, the color histogram is divided into a predetermined number of color regions Ci. However, the color histogram may be divided for each RGB color component according to the number of divisions considering human visual characteristics. Here, since human visual characteristics have higher sensitivity in the order of G, R, and B, for example, as shown in FIG. 6, the G component is divided into 8 parts, the R component is divided into 4 parts, and the B component is divided into 2 parts. Thus, the color region may be divided so that the number of divisions satisfies G> R> B.

  In the above embodiment, the weighting factor αi is obtained by learning. However, a weighting factor αi that is artificially set in advance may be used.

  Further, in the above embodiment, when the feature amount Ti is calculated from the absolute value of the difference between the average values of the color regions of the color histogram, the average value (Mi) of the color region Ci is set to two adjacent regions Ci−. The average value Mi for each region Ci is flattened by weighting and adding the average value of 1, Ci + 1 (referred to as Mi-1, Mi + 1), and the flattened average value is calculated. The feature amount Ti may be calculated from the absolute value of the difference. Here, the flattened average value (Mi ′) can be calculated by the following equation (2).

Mi '= a.Mi-1 + b.Mi + c.Mi + 1 (2)
However, a, b, and c are weighting factors, and for example, values of a = c = 0.25 and b = 0.5 are used so that a + b + c = 1.

  As a result, it is possible to determine that two images having slightly different colors that are similar are determined to be similar if the correlation between the color histograms is seen, and thus it is possible to perform similarity determination that is strong against changes in the image. it can.

  Although the apparatus according to the embodiment of the present invention has been described above, the computer is caused to function as means corresponding to the low-frequency image creation unit 30, the color histogram calculation unit 32, the feature amount calculation unit 34, and the determination unit 36, and FIG. A program that performs the processing shown in FIG. 6 is also one embodiment of the present invention. A computer-readable recording medium that records such a program is also one embodiment of the present invention. Further, such a program may be incorporated in viewer software for browsing images.

1 is a schematic block diagram showing the configuration of a similarity determination device according to an embodiment of the present invention. A flowchart showing processing performed in the present embodiment Figure showing the calculated color histogram in two dimensions Diagram for explaining division of color histogram Diagram for explaining division of color histogram Illustration for explaining color histogram division considering human visual characteristics

Explanation of symbols

1 Similarity Discriminator 2 Memory Card 12 CPU
DESCRIPTION OF SYMBOLS 14 System memory 16 Input part 18 Display part 20 Card slot 22 Compression / decompression part 24 Hard disk 26 Memory control part 28 Display control part 30 Low frequency image creation part 32 Color histogram calculation part 34 Feature-value calculation part 36 Discrimination part

Claims (6)

In a similarity determination device that determines whether two images are similar,
Low-frequency image creation means for creating a low-frequency image representing each low-frequency component of the two images;
Color histogram calculating means for calculating a color histogram of the two low-frequency images;
A feature amount calculating means for dividing the two color histograms into a predetermined number of color regions and calculating a feature amount representing a correlation between the two color histograms for each of the divided color regions;
A similarity determination apparatus comprising: a determination unit configured to determine similarity between the two images based on a feature amount of each color region.   The similarity determination apparatus according to claim 1, wherein the feature amount calculation unit is a unit that divides the two color histograms according to human visual characteristics.   The discrimination means is a means for calculating two total feature quantities by weighting and adding the feature quantities of the respective color regions with a predetermined weighting factor, and performing the similarity discrimination based on a difference between the two total feature quantities. The similarity determination apparatus according to claim 1, wherein the similarity determination apparatus is provided.   The predetermined weighting factor is a plurality of incorrect answer data consisting of two sample images known to be similar to the feature quantity obtained from a large number of correct answer data consisting of two sample images known to be similar. 4. The similarity determination apparatus according to claim 1, wherein the similarity determination device is a weighting coefficient for weighting the feature amount, which is obtained by learning in advance the feature amount obtained from. In a similarity determination method for determining whether or not two images are similar,
Creating a low frequency image representing the low frequency component of each of the two images;
Calculating a color histogram of the two low frequency images;
Dividing the two color histograms into a predetermined number of color regions;
Calculating a feature amount representing a correlation between the two color histograms for each of the divided color regions;
A similarity determination method, wherein similarity determination of the two images is performed based on a feature amount of each color region. In a program for causing a computer to execute a similarity determination method for determining whether two images are similar to each other,
Creating a low frequency image representing the low frequency component of each of the two images;
Calculating a color histogram of the two low frequency images;
Dividing the two color histograms into a predetermined number of color regions;
A procedure for calculating a feature value representing a correlation between the two color histograms for each of the divided color regions;
And a procedure for determining similarity between the two images based on the feature amount of each color region.

Images