JP2007316950A - Method, apparatus and program for processing image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately find out similarity between images on the basis of the contents of the images. <P>SOLUTION: An image is divided into M partial areas on the basis of a previously set rule, and then inter-partial areas similarity Sab(i, j) which is the similarity of partial area images of all combinations of M partial area images Pa(i)i=1, 2, ..., M constituting an image Fa and M partial area images Pb(j)i=1, 2, ..., M constituting an image Fb is calculated. Out of M(M-1)/2 combinations of the partial area images Pa(i) and Pb(j), effective partial areas are detected by referring to a parameter to be used for effective partial area decision in a storage means and image similarity between two images Fa and Fb is calculated from the inter-partial areas similarity Sab(i, j) of N partial area pairs decided to be the effective partial areas. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing method, apparatus, and program, and in particular, calculates the similarity between two images required for image classification and image search for efficient image management and browsing. The present invention relates to an image processing method, apparatus, and program.

  The following is a conventional technique for calculating the similarity between two images.

(1) Method for matching the entire image:
This is a method for obtaining the similarity based on the feature amount extracted from the entire image. As a feature quantity, a color histogram representing the frequent distribution of colors in the entire image is often used. In this case, the distance between the color histograms extracted from two images is used as the similarity between the images. This method is characterized by being realized by simple processing.

(2) Method by collation between regularly divided areas:
Instead of collating the entire image, the image is regularly divided into rectangular blocks, and the blocks are collated with the feature values extracted from each block area. As a method of collating between blocks, there is a method of obtaining a similarity between images by a sum of similarities between blocks having the same position in the image. For all combinations of each block of two images, the similarity is obtained by multiplying the similarity between the colors between the blocks by the color histogram and the similarity regarding the position of the block. In this method, it is possible to obtain a similarity between images such that the image includes a larger number of blocks whose colors are similar to the positional relationship in the image (see, for example, Non-Patent Document 1).

(3) Method by collation between object areas:
A method for obtaining similarity between images from the similarity between the associated object regions after performing preprocessing for dividing the image into regions corresponding to the objects, associating the object regions extracted from each image It is.
Seiji Hotta, Kohei Inoue, Kiichi Urahama "Similar search of video based on distance between image sets" VOL.54, NO.11, pp.1653-1656, 2000

  However, in the above-described method (1) for collating the entire image, the positional configuration in the image is not sufficiently reflected in the similarity calculation, and thus the characteristics of the entire image are similar even if the image content is not similar. In some cases, there is a problem that the degree of similarity becomes high.

  The above-described method (2) based on collation between the regularly divided regions can perform collation reflecting the positional configuration in the image, so that the problem of the collation method for the entire image can be improved. However, this method has the same image component such as different subject positions even if the image contents are the same, but there is a problem that the degree of similarity is low when the positional relationship is different.

  The method (3) based on collation between object areas can solve this problem, but it is assumed that the object area can be detected correctly. In limited situations, such as when the background is uniform or the object appearing in the image can be identified, it is possible to extract the object region, but it is difficult to extract the object region from a general image with high accuracy. It is.

  As described above, it is difficult to accurately obtain the similarity between images reflecting whether the image contents are similar with respect to a general image.

  The present invention has been made in view of the above points, and an object of the present invention is to provide a method for accurately obtaining similarity between images based on image contents.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention (Claim 1) is an image processing method for calculating the similarity between two images Fa and Fb when the images are input.
An area dividing step (step 1) in which the area dividing means divides the image into M partial areas based on a preset rule;
The inter-region similarity calculation means constructs an image Fb with M partial region images Pa (i) i = 1, 2,... M constituting the image Fa obtained in the region dividing step (step 1). Inter-region similarity for calculating the partial region similarity Sab (i, j), which is the similarity of the partial region images for all combinations of M partial region images Pb (j) j = 1, 2,. A calculation step (step 2);
The effective partial area detection means includes a storage means for storing parameters used for determining the effective partial area among M (M−1) / 2 combinations of the partial area images Pa (i) and Pb (j). Referring to an effective partial area detecting step (step 3) for detecting an effective partial area with reference to a storage means storing parameters used for determining an effective partial area;
The image similarity calculation means calculates two images Fa from the inter-partial region similarity Sab (i, j) of the N partial region pairs determined to be effective partial regions in the effective partial region detection step (step 3). And an image similarity calculation step (step 4) for calculating the image similarity between Fb and Fb.

Further, the present invention (Claim 2), in the effective partial region detection step (Step 3),
Associating the partial areas of the image Fa with the partial areas of the image Fb in the descending order of the similarity Sab (i, j) between the partial areas;
A step of obtaining a threshold value of similarity between regions determined to be an effective partial region from the distribution of similarity between regions for K pairs having high similarity of the M partial region pairs associated with each other.

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 3) is an image processing apparatus that calculates the similarity between two images Fa and Fb when they are input.
Parameter storage means 5 for storing parameters used for effective partial area determination;
Area dividing means 2 for dividing an image into M partial areas based on a preset rule;
M partial area images Pa (i) i = 1, 2,... M constituting the image Fa and M partial area images Pb (j) j constituting the image Fb obtained by the area dividing means 2. = 1, 2,..., M between the partial area similarity calculating means 3 for calculating the partial area similarity Sab (i, j) which is the similarity of the partial area image;
An effective partial region detecting unit 4 for detecting an effective partial region with reference to the parameter storage unit among M (M−1) / 2 combinations of the partial region images Pa (i) and Pb (j); ,
The effective partial region detecting means 4 calculates the image similarity between the two images Fa and Fb from the inter-partial region similarity Sab (i, j) of the N partial region pairs determined to be effective partial regions. Image similarity calculation means 5.

Further, the present invention (Claim 4) is provided in the effective partial area detecting means 4,
Means for associating the partial area of the image Fa and the partial area of the image Fb in the descending order of the similarity Sab (i, j) between the partial areas;
Means for determining a threshold value of similarity between regions determined to be an effective partial region from a distribution of similarity between regions for K pairs having high similarity of the M partial region pairs associated with each other.

  The present invention (Claim 5) is an image processing program for causing a computer to execute each means of the image processing apparatus according to Claim 3 or 4.

  As described above, according to the present invention, the components of the two images, such as the background and the subject, are similar even if the positional relationship of the components in the image fluctuates. It is possible to calculate the inter-image similarity that reflects the similarity of the image contents so that the similarity is high.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, the outline of the present invention will be described.

  In the present invention, in the method based on the collation between object areas, the object area detection problem is solved by regarding the area divided regularly like a rectangular block without using the preprocessing for detecting the object area as the object area. Resolve. By using regularly divided partial areas, the similarity in the partial area near the boundary of the subject or background is essentially similar, but by dividing into small areas, The problem of high similarity even though they are not similar is solved by the following policy.

  The similarity between the images containing similar components may be low in the partial area near the boundary of the vehicle body and background, although the similarity may be low, but the similarity of the partial area that hardly includes the vicinity of the boundary. It is considered that there are many such partial areas. In addition, images that do not include similar components may be similar in small partial areas, but such partial areas are considered to be small.

  Therefore, the partial area whose similarity is reliable without being influenced by the vicinity of the vehicle body or the background boundary is p% of the whole, and the partial area whose similarity is high in a small partial area even though it is not originally similar. Is considered to be q << p, as shown in FIG. In this case, after associating the most similar partial areas between images, if only the similarity of the upper p% partial areas having a high similarity is adopted, the original image as shown in FIG. For images with the same configuration, only a partial region whose degree of similarity is reliable is adopted, and as shown in FIG. 3B, similar images are erroneously similar for images that are not originally similar in image configuration. Since the ratio of the partial region with a high degree is as small as q / p, the influence is small.

However, since the ratio p of the partial area whose similarity can be trusted without being affected by the vicinity of the boundary between the subject and the background varies depending on the image, it is necessary to measure from the image to be processed, but it is difficult to perform automatic measurement. . Therefore, an average value of actually measured values of p for many images is preset as an initial value p ₀ , and K pairs having a high similarity are selected from the partial region pairs associated between the images (p ₀ = K / M), the similarity threshold T1 shown in FIG. 3 is estimated based on the similarity distribution of K partial region pairs.

  In the present invention, the following apparatus configuration and operation are performed based on the above concept.

  FIG. 4 shows the configuration of the image processing apparatus according to the embodiment of the present invention.

  The image processing apparatus shown in FIG. 1 includes an image acquisition unit 1, an area division processing unit 2, an inter-region similarity calculation unit 3, an effective partial region detection unit 4, a parameter storage unit 5, and an image similarity calculation unit 6. .

  The image acquisition unit 1 reads two images Fa and Fb to be processed and outputs these images to the region division processing unit 2.

  The region division processing unit 2 divides each image received from the image acquisition unit 1 into M partial regions based on a preset rule. For example, a method of dividing the block into a total of M = m1 × m2 blocks by dividing m1 equally in the horizontal direction and m2 equally in the vertical direction may be used. The partial area image Pa (i) = i = 1, 2,..., M of the image Fa and the partial area image Pb (j) j = 1, 2,. Output.

  The inter-region similarity calculation unit 3 is a portion that is the similarity of the partial region image with respect to M (M−1) / 2 combinations of all combinations of the received partial region images Pa (i) and Pb (j). The similarity between regions Sab (i, j) is calculated. The calculated similarity Sab (i, j) is output to the effective partial region determination unit 4 and the image similarity calculation unit 6. As a similarity of the partial area image, a color histogram is obtained from the partial area image, a method of calculating based on the difference of the color histogram, a feature amount such as an average, variance, skewness, kurtosis, etc. representing the distribution shape of the color histogram is obtained. A method of calculating from the distance in the feature space, a method of calculating the color signature of the partial area image as a feature amount, and calculating based on the Earth Mover's Distance are effective.

  Upon receiving the inter-partial similarity, Sab (i, j) from the inter-region similarity calculation unit 3, the effective partial region determination unit 4 uses the effective partial region determination from the parameter storage unit 5 used for the effective partial region determination. A parameter is read, and a valid pair is detected from among the combinations of M (M-1) / 2 partial areas using the parameter. The detected N effective partial region pairs are output to the image similarity calculation unit 6. A specific method for detecting the effective partial area will be described later.

  The parameter storage unit 5 used for the effective partial region determination stores parameters used for the effective partial region determination. Upon receiving a read request from the effective partial region determination unit 4, the stored parameter is determined as the effective partial region determination. Output to part 4.

  When the image similarity calculation unit 6 receives N effective partial region pairs from the effective partial region detection unit 4, the image similarity calculation unit 6 calculates the average value of the partial region similarity Sab (i, j) for the N effective partial region pairs as follows: The similarity between the two images Fa and Fb is calculated and output.

  Next, a processing procedure in the above configuration will be described.

  FIG. 5 is a flowchart of the operation of the image processing apparatus according to the embodiment of the present invention.

  Step 201) The image acquisition unit 1 reads two images Fa and Fb.

  Step 202) The area division processing unit 2 performs a process of dividing each input image into M areas. FIG. 6B shows an example of division. The figure shows an example in which the area is divided into 4 × 4 = 16 rectangular areas. The divided areas are temporarily stored in a memory (not shown).

  Step 203) In the inter-region similarity calculation unit 3, the partial region image Pa (i) i = 1, 2,... M of the image Fa and the partial region image Pb (the image Fb) read from the memory (not shown). j) Interregion similarity Sab (i, j) is calculated for all combinations of j = 1, 2,. As shown in FIG. 4C, M (M-1) / 2 types of similarity between regions are obtained.

  Step 204) The partial areas of the image Fa and the partial areas of the image Fb are associated with each other in the descending order of the similarity between partial areas Sab (i, j). The association process is performed by the following method.

  1) First, rearrange M (M-1) / 2 types of similarity Sab (i, j) between partial areas in descending order.

  2) Next, an area pair having the maximum similarity between partial areas Sab (i, j) is detected, and this pair is assumed to be associated. Let this region pair be Pa (ii1) and Pb (jj1).

  3) Then, in a partial area excluding Pa (ii1) and Pb (jj1), a pair of areas having the highest similarity between partial areas Sab (i, j) is detected, and this pair is associated. Let this pair be Pa (ii2) and Pb (jj2).

  Thereafter, the same processing is repeated to associate all partial areas.

  Step 205) The effective partial region detection unit 4 calculates the average u of the similarity between regions and the standard deviation σ for K pairs having a large similarity between the partial regions associated in Step 204. Note that the value of K is read from the parameter storage unit 5 used for effective partial region determination.

  Step 206) The effective partial region detection unit 4 obtains a threshold value of the similarity between regions for determining the effective partial region pair. For K pairs having high similarity between the partial areas associated in step 204, u−λσ is set as a threshold value using the average u of the similarity between areas and the standard deviation σ. In addition, about the value of (lambda), it reads from the parameter memory | storage part used for similar preparation machine determination.

  Step 207) A pair between partial areas whose similarity between partial areas is u−λσ or more is detected as an effective partial area pair.

  Step 208) The image similarity calculation unit 6 calculates and outputs the average value of the similarity between the partial regions of the effective partial region pair detected by the effective partial region detection unit 4 as the image similarity.

  Note that the output destination of the image similarity is a display device or a storage device.

  In addition, according to the present invention, the functions of the components shown in FIG. 4 can be constructed as a program, installed in a computer used as an image processing apparatus, executed, or distributed via a network.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention is applicable to a technique for detecting the similarity between two images, for example, a technique such as video search.

It is a figure for demonstrating the principle of this invention. It is a figure for demonstrating the principle of this invention. It is a figure for demonstrating the outline | summary of this invention. 1 is a configuration diagram of an image processing apparatus according to an embodiment of the present invention. It is a flowchart of the whole process in one embodiment of this invention. It is a figure explaining the example of the partial area | region in one embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image acquisition part 2 Area division means, Area division process part 3 Inter-region similarity calculation means, Inter-region similarity calculation part 4 Effective partial area detection means, Effective partial area detection part 5 Parameter storage means, Parameter storage part 6 Image similarity Degree calculation means, image similarity calculation unit

Claims (5)

An image processing method for calculating the similarity between two images Fa and Fb,
An area dividing step in which the area dividing means divides the image into M partial areas based on a preset rule;
The inter-region similarity calculating means obtains M partial region images Pa (i) i = 1, 2,... M constituting the image Fa and M constituting the image Fb obtained in the region dividing step. Inter-region similarity calculation for calculating the similarity between partial regions Sab (i, j), which is the similarity of partial region images for all combinations of the partial region images Pb (j) j = 1, 2,. Steps,
Storage means in which the effective partial area detecting means stores parameters used for determining the effective partial area among the M (M-1) / 2 combinations of the partial area images Pa (i) and Pb (j). And an effective partial region detection step for detecting an effective partial region,
The image similarity calculation means calculates two images Fa and Fb from the inter-partial region similarity Sab (i, j) of the N partial region pairs determined to be effective partial regions in the effective partial region detection step. An image similarity calculating step for calculating the image similarity of
And an image processing method. In the effective partial region detection step,
Associating the partial areas of the image Fa with the partial areas of the image Fb in the descending order of the similarity between the partial areas Sab (i, j);
Obtaining a threshold value of the similarity between regions determined to be an effective partial region from the distribution of similarity between regions for K pairs having high similarity of the M partial region pairs associated with each other;
The image processing method according to claim 1, wherein: An image processing apparatus that calculates the similarity between two images Fa and Fb,
Parameter storage means for storing parameters used for determining the effective partial area;
Area dividing means for dividing the image into M partial areas based on a preset rule;
M partial area images Pa (i) i = 1, 2,... M constituting the image Fa and M partial area images Pb (j) constituting the image Fb obtained by the area dividing means. ) An inter-region similarity calculating means for calculating an inter-region similarity Sab (i, j), which is the similarity of the partial region images for all combinations of j = 1, 2,.
Effective partial area detection means for detecting an effective partial area with reference to the parameter storage means in M (M-1) / 2 combinations of the partial area images Pa (i) and Pb (j) When,
The image similarity between the two images Fa and Fb is calculated from the similarity between partial regions Sab (i, j) of the N partial region pairs determined as effective partial regions by the effective partial region detection means. Image similarity calculation means;
An image processing apparatus comprising: The effective partial area detecting means includes
Means for associating the partial areas of the image Fa with the partial areas of the image Fb in the descending order of the similarity between the partial areas Sab (i, j);
Means for obtaining a threshold value of similarity between regions determined to be an effective partial region from the distribution of similarity between regions for K pairs having high similarity of the M partial region pairs associated with each other;
The image processing apparatus according to claim 3, further comprising: On the computer,
5. An image processing program for causing each means of the image processing apparatus according to claim 3 or 4 to be executed.

Images