JP2010072682A - Color histogram generation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and method for generating a color histogram for precisely displaying the features of colors. <P>SOLUTION: This color histogram generation device includes a color space database; and a histogram generation part 2. The color space database includes the information of a plurality of subspaces generated by clustering a data group belonging to a color space. A histogram generation part 2 decides which subspace pixels in an input image belong to, and generates a histogram based on this. The color space database is, for example, a lookup table 4, wherein the data belonging to a color space used in the input image and a subspace to which the data should belong are associated with each other. In the case of using the lookup table 4, the subspace to which the pixels belong is determined by referring to the lookup table 4 by using the data of the pixels in the input image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for generating a color histogram. More specifically, the present invention relates to a technique for generating a color histogram for representing image characteristics.

  In order to determine the similarity between images, a method of extracting feature amounts from images and comparing the feature amounts is known. Since the feature amount can generally be acquired as multidimensional vector data, the similarity can be determined using the distance between the vector data.

A color histogram is known as a kind of image feature amount (see Patent Document 1 below). A typical method for generating a color histogram is as follows.
Divide the color space to obtain a partial space (so-called bin).
Determine which bin each pixel constituting the input image belongs to.
Normalize the number of pixels belonging to each bin by the total number of pixels.

The number of pixels for each bin thus obtained becomes a color histogram. Since this color histogram is considered to represent the characteristics of the colors in the image, the similarity between the images can be determined using the color histogram as a reference.
JP 2006-155619 A

  By the way, conventionally, as a method of dividing the color space, a method of dividing the space by a separation line or a separation curve is used. For example, in the technique of Patent Document 1, a bihexagonal pyramid model in the HSV space is used as the color space. A subspace (that is, a bin) is generated by basically dividing this space on the basis of lightness.

  However, when the space is divided in this way, the color difference between the partial spaces varies. That is, there are cases where the colors are relatively similar or not similar between adjacent partial spaces. For example, a certain pixel and a certain pixel may belong to different bins even though they are similar in color. On the other hand, even if the pixels are different from each other in terms of feeling, they may belong to the same bin.

  That is, in the prior art, there is room for improvement in accurately expressing the color features by the generated color histogram.

  The present invention has been made in view of the above situation. One of the objects of the present invention is to provide an apparatus or method for generating a color histogram capable of accurately expressing color characteristics.

  The present invention is configured as described in any of the following items.

(Item 1)
A color space database and a histogram generator,
The color space database includes information on a plurality of partial spaces generated by clustering data groups belonging to the color space,
The histogram generation unit is configured to determine which partial space a pixel in an input image belongs to, and generate a histogram based on the determination.

  In the invention of this item, a plurality of subspaces are generated using clustering. Here, clustering is performed on a data group constituting a color space. Therefore, in this apparatus, a partial space can be formed by clustering instead of drawing a dividing line with respect to the color space.

  Furthermore, the histogram generation unit can determine which partial space each pixel in the input image belongs to generate a histogram.

  In this apparatus, a partial space is formed by clustering data groups in a color space. For this reason, the generated histogram can accurately represent color features.

(Item 2)
The color space database further includes central data in the generated partial spaces,
The histogram generation unit
A process of calculating a distance between a pixel in the input image and the central data in each of the partial spaces;
Item 2. The color histogram generation device according to Item 1, wherein the color histogram generation device is configured to perform a process of causing the pixel to belong to a partial space to which the center data closest to the pixel belongs.

  Here, “belonging a pixel to a subspace” means counting as the number of pixels to be included in the subspace (so-called bin).

(Item 3)
The color space database is a lookup table in which data belonging to the color space used in the input image is associated with the partial space to which the data should belong,
The color according to item 1 or 2, wherein the histogram generation unit is configured to determine the partial space to which the pixel belongs by referring to the lookup table using pixel data in the input image. Histogram generator.

  By using the lookup table, it is possible to generate a color histogram at high speed. When the pixel data and the lookup table data are different, the subspace can be determined based on the lookup table data closest to the pixel data.

(Item 4)
The partial space in the color space database is configured such that the minimum value of the distance between the central point of each generated partial space and the central point of another partial space is constant. The color histogram production | generation apparatus of any one of these.

  By performing clustering in this way, it is possible to further improve the accuracy of the color histogram (the sensory features received from the image match the color histogram).

(Item 5)
Each of the partial spaces in the color space database is generated such that the maximum distance between the data belonging to each partial space and the center point of each partial space is constant. The color histogram generation device described.

  By performing clustering in this way, the accuracy of the color histogram can be further improved.

(Item 6)
The plurality of partial spaces in the color space database further includes a partial space for achromatic colors,
6. The item according to claim 1, wherein the achromatic color partial space is generated by clustering an achromatic color data group among data groups belonging to the color space. Color histogram generator.

  In the invention of this item, some partial spaces are for achromatic colors.

(Item 7)
Furthermore, it has a subspace generator,
The color histogram generation apparatus according to any one of Items 1 to 6, wherein the partial space in the color space database is generated by the partial space generation unit.

(Item 8)
The color histogram generation apparatus according to any one of Items 1 to 7, wherein the data group clustered by the partial space generation unit is a grid point in the color space.

  The lattice point means data composed of integer values. By making the lattice points clustering targets, the calculation cost for clustering can be kept low.

(Item 9)
The color histogram generation apparatus according to any one of Items 1 to 7, wherein the data group clustered by the partial space generation unit is a representative point extracted from the color space.

  It is also possible to perform clustering by appropriately thinning out data in the color space. The degree of thinning can be determined in view of the required clustering accuracy and calculation cost. By performing clustering using representative points extracted from the color space, the calculation cost for clustering can be further reduced.

(Item 10)
Providing a color space database having information on a plurality of partial spaces generated by clustering data groups belonging to the color space;
And determining which partial space the pixel in the input image belongs to, and generating a histogram based on the partial space.

(Item 11)
A computer program for causing a computer to execute the steps according to item 10.

  This computer program is stored in an appropriate recording medium (for example, an optical recording medium such as a CD-ROM or a DVD disk, a magnetic recording medium such as a hard disk or a flexible disk, or a magneto-optical recording medium such as an MO disk). Can be stored. This computer program can be transmitted via a communication line such as the Internet.

  According to the present invention, it is possible to provide an apparatus or method for generating a color histogram capable of accurately expressing color characteristics.

(Apparatus configuration of the first embodiment)
A color histogram generation device according to a first embodiment of the present invention will be described with reference to the accompanying drawings.

  The color histogram generation device of this embodiment includes a subspace generation unit 1, a histogram generation unit 2, and a lookup table generation unit 3 as main components (see FIG. 1).

  The partial space generation unit 1 is configured to generate a plurality of partial spaces by clustering data groups belonging to the color space.

  The histogram generation unit 2 is configured to determine which partial space the pixel in the input image belongs to and generate a histogram based on this.

  The lookup table generation unit 3 is configured to generate a lookup table 4 for associating data belonging to the color space of the input image with the partial space to which the data belongs. The lookup table 4 corresponds to an example of a color space database in the present invention.

  The histogram generation unit 2 in the present embodiment is configured to determine the partial space to which the pixel belongs by referring to the lookup table 4 using the pixel data in the input image.

  Specific operations in the apparatus of this embodiment will be described in detail below.

(Operation of the first embodiment)
A method of generating a color histogram in the present embodiment will be described with further reference to FIGS.

(Overall procedure)
First, the overall procedure will be described with reference to FIG. Then, a more detailed procedure for each process will be described.

(Step SA-1 in FIG. 2)
First, the data group belonging to the color space is clustered by the partial space generation unit 1. Thereby, a plurality of partial spaces can be generated.

  In this embodiment, clustering is performed on a data group belonging to a color space before generating a histogram. A plurality of subspaces can be generated by this clustering. In this embodiment, clustering is performed on a data group belonging to the color space regardless of the pixels of the input image. In the present embodiment, a partial space can be formed by clustering instead of drawing a dividing line with respect to the color space.

(Steps SA-2 and SA-3 in FIG. 2)
Subsequently, the histogram generation unit 2 determines which partial space the pixel in the input image belongs to. Based on this, a histogram can be generated and output. In this apparatus, a partial space is formed by clustering data groups in a color space. For this reason, the generated histogram can accurately represent color features.

(Subspace generation procedure)
Next, the procedure for generating the partial space will be described in more detail with reference to FIG.

(Step SB-1 in FIG. 3)
This embodiment will be described on the assumption that an image is acquired in RGB. Of course, it is also possible to handle data using a color space other than RGB.

First, all RGB combination data is generated in a pseudo manner as follows. This data is only prepared for clustering, not actual input image data.
(R, G, B) = (0, 0, 0) to (255, 255, 255)

  As a result, 256 × 256 × 256 numbers of data are generated. Thus, in this embodiment, grid point data is selected. A lattice point is data represented by an integer value. By using the data of the lattice points, it becomes possible to perform the calculation described later at high speed. However, it is not essential to use grid point data, and appropriate data can be used.

  In the present embodiment, it is not necessary to use all the lattice points, and thinning can be performed as appropriate. The amount of calculation can be further reduced by thinning out (that is, using representative points).

Next, the RGB color space is converted to La ^* b ^* . This is because La ^* b ^* is said to express human color sense more appropriately than RGB. However, the color space used in clustering is not limited to La ^* b ^* , and other color spaces (for example, HSV, Lu ^* v ^*, etc.) can be used. In this embodiment, in order to generate a lookup table to be described later, association is performed between the converted La ^* b ^* data and the original RGB data. That is, the other data can be referred to from one data.

(Step SB-2 in FIG. 3)
Next, in the La ^* b ^* space, all the data of the lattice points described above are clustered to obtain a cluster. As a clustering method, general clustering methods such as Nearest Neighbor method and k-mean method can be used. A detailed implementation example of clustering will be described later. The calculation of the distance for clustering may use the Euclidean distance in the La ^* b ^* space, or may use a color difference formula defined in each color space. Both cases are expressed as distance in this specification.

  Clustering divides the space evenly. A partial space (so-called bin) is formed by each generated cluster.

(Step SB-3 in FIG. 3)
Next, the subspace generation unit 1 calculates center data of each cluster. The center data of each cluster is a color representing each bin (representative color). As a method for calculating the center of each cluster, a conventionally used method can be used.

A situation where the color space 6 of La ^* b ^* is divided into a plurality of clusters 7 is schematically shown in FIG. The color space 6 of La ^* b ^* has an irregular shape as shown in the figure. Each cluster 7 is represented by a circle. The center point (center data) 8 of each cluster 7 is a representative color.

  In this embodiment, each bin is a partial space in which the center points 8 of the cluster 7 are separated by lines. However, in this embodiment, the partial space is not formed by actually defining the line. By making a pixel belong to a cluster having a center point closest to it, a bin having such a pseudo boundary can be grasped.

(Step SB-4 in FIG. 3)
On the other hand, the lookup table generation unit 3 generates a lookup table that associates data belonging to the color space with the partial space to which the data belongs. Specifically, first, it is determined to which cluster each data in the La ^* b ^* space belongs. This determination can be performed by causing each data in the La ^* b ^* space to belong to the cluster having the closest distance from the cluster center. Then, the lookup table 4 is created for all combinations of the bin number of each cluster and the RGB values associated with the color of the La ^* b ^* space belonging to it.

  An example of the lookup table 4 is shown in FIG. Here, a bin number is assigned to each value of RGB.

(Clustering implementation example)
Next, an implementation example of the clustering described above will be described with further reference to FIG.

(Step SC-1 in FIG. 6)
First, when starting clustering, an initial cluster is generated. That is, the center point of each initial cluster is determined first. This center point may be selected randomly from the color space, but by making the point on the lattice the center of the initial cluster in the three-dimensional space of the color space, the subspace can be generated efficiently. When generating an achromatic partial space, a plurality of initial achromatic clusters (clusters having a and b of 0 and L having a value other than 0) are generated.

(Step SC-2 in FIG. 6)
Next, one point in the La ^* b ^* color space is selected as the target point. However, the target point may be a representative point extracted from the color space. By using the representative points, the calculation cost for clustering can be kept low.

(Step SC-3 in FIG. 6)
Next, the evaluation index when the target point is added to each cluster is calculated. The calculation of the evaluation index can be performed as follows, for example.

(1A) Assuming that the target point is added to any one cluster (initial cluster), the center of the cluster (center data) is obtained. As the center, for example, the average of all points constituting the cluster can be used. Alternatively, the point where the sum of the distances to other points in the cluster is the smallest may be the center. The value of the achromatic subspace (described later) L when calculating the center point in the case of (La ^* b ^* luminance) only with the calculated a, b (La ^* b ^* of a, b) is 0 And

  (2A) Next, the radius is obtained for all the clusters. In order to obtain the radius, for example, the distance between the center point of the cluster and all the points belonging to the cluster is obtained. The maximum value of the obtained distance can be used as the radius.

  (3A) Next, an average radius μa and variance σa are calculated for all clusters.

  (4A) For each cluster center, determine the distance to the center of another cluster. And let the minimum value of distance be the distance between adjacent clusters. Further, the average μb and variance σb of this distance are calculated.

(5A) Furthermore, the evaluation index P is calculated by the following formula, for example.
P = k1 ・ μa + k2 ・ σa + k3 ・ μb + k4 ・ σb
Here, k1 to 4 are weighting coefficients.

  Assuming that target points are added to each cluster, the evaluation index P is calculated for each case.

(Step SC-4 in FIG. 6)
A target point is added to the cluster having the smallest evaluation index P.

(Step SC-5 in FIG. 6)
If there are remaining points in the color space, the process returns to step SC-2 and the above-described processing is repeated. When the process is completed for all points, the process proceeds to the next step SC-6.

(Step SC-6 in FIG. 6)
Since the result of the clustering depends on the initial cluster, the optimal subset is not always generated even if the above procedure is performed. Therefore, the following repetitive operations are performed.

  (1B) All points (sample sets) for generating a subspace are determined. This point may be any integer value in the color space. However, a sample set can be extracted and used by sampling at an appropriate interval.

  (2B) An initial cluster (center point) is determined. It may be determined randomly or in a lattice shape. When generating an achromatic subspace, a plurality of achromatic initial clusters (clusters in which a and b are always 0 and L has an arbitrary value) are determined.

  (3B) Steps SC-2 to SC-5 described above are performed on the sample set.

  (4B) When this step (3B) ends, the points included in the cluster before the end and the points included in the cluster after the end are compared. If the points included in both clusters are the same, the process ends. If the points included in the cluster are different, the processing is repeated by returning to step (3B) for the sample set again with the center of the cluster as the start point of the cluster.

  According to the clustering implementation example described above, the partial space is generated so that the minimum value of the distance between the central point of each generated partial space and the central point of the other partial space from the central point is constant. It will be.

  By performing clustering in this way, it is possible to further improve the accuracy of the color histogram (the sensory features received from the image match the color histogram).

  Furthermore, according to the clustering implementation example described above, in each generated partial space, the partial space is generated such that the maximum value of the distance between the data belonging to the partial space and the central point is constant. Become.

  In this respect as well, in this implementation example, the accuracy of the color histogram can be further improved.

(Histogram generation procedure)
Next, a procedure for generating a histogram using the histogram generator 2 will be described with further reference to FIG.

(Step SD-1 in FIG. 7)
First, an input image is acquired. The input image is, for example, a query image input for image search. However, the image is not limited to the query image, and may be used for any application as long as it is necessary to generate a color histogram. It is assumed that the input image of this embodiment is expressed in the RGB color space. This is because there are generally many images expressed in RGB. However, in principle, there are no particular restrictions on the color space that represents the input image.

(Step SD-2 in FIG. 7)
In this embodiment, pixel data in the input image is expressed in RGB as described above. In the present embodiment, the RGB value closest to the input data is selected from the lookup table 4. The distance between the input data and the data in the lookup table can be calculated using, for example, the Euclidean distance.

  Next, when the corresponding data is found, the pixel is made to belong to the bin number shown in the lookup table 4. That is, the number of data of the bin number is increased.

  This process is performed for each pixel in the input image.

(Step SD-3 in FIG. 7)
After determining which bin belongs to all the pixels in the input image, the number of pixels in each bin is normalized by the total number of pixels. Thereafter, the obtained color histogram can be output. This color histogram can be used as data indicating the feature amount of the image.

  In the present embodiment, as described above, clustering is performed on data belonging to the color space. A color histogram is generated by allocating pixels of the input image to the generated cluster. For this reason, in this embodiment, it is possible to accurately generate a color histogram according to a human sense.

  In this embodiment, since the lookup table 4 is used as the color space database, a color histogram can be generated at high speed.

  In the embodiment, the lookup table 4 is generated and a histogram is generated using the lookup table 4. However, the generation and use of a lookup table can be omitted. That is, by comparing each pixel in the input image with the center data of each cluster, the bin to which the pixel should belong can be determined. In general, the pixel belongs to the bin (cluster) having the nearest center data. That is, the color space database only needs to have information on a plurality of partial spaces generated by clustering into data groups belonging to the color space. Using this information, a histogram can be generated by associating the pixel of the input image with a partial space.

  In the above-described embodiment, the above-described evaluation formula is used in clustering. However, the following method is also possible when performing simply.

  That is, a distance from each target point in the color space to the center point of each cluster (as a distance, a color difference formula defined in the color space may be used). Then, the target point can be added to the nearest cluster.

  In the above-described embodiment, the description starts from the stage of generating the color space database. However, it is possible to reuse the generated color space database. Therefore, a color space database that has already been generated can be prepared and a histogram can be generated using the database.

(Second Embodiment)
Next, a method for generating a color histogram in the second embodiment will be described with reference to FIG. In the description of the second embodiment, the same reference numerals are used for the elements common to the first embodiment described above, thereby avoiding complicated description.

  In the first embodiment, when generating a histogram, the pixel belongs to a cluster having a center closest to the pixel of the input image. On the other hand, in the second embodiment, instead of assigning pixels to one cluster, weights are assigned and weights are assigned to a plurality of pixels.

(Step SE-1 in FIG. 9)
The input image is scanned pixel by pixel from the upper left to the lower right, and the target pixel is selected.

(Step SE-2 in FIG. 9)
Determine the bin containing the pixel of interest. This process can be performed using the method of the first embodiment.

(Step SE-3 in FIG. 9)
Add similarity to the target pixel to all bins. The similarity S can be calculated by the following equation, for example.
S = 1-d (Ci, S) / M
here,
Ci: central data of i-th cluster,
S: Target pixel data
d (): Color difference formula (distance)
M: Maximum color difference. By performing this calculation for all clusters, the similarity between each pixel and each bin can be added to all bins.

(Step SE-4 in FIG. 9)
When all pixels have been processed, the process ends. Otherwise, the process returns to step SE-1.

(Step SE-5 in FIG. 9)
All are normalized by dividing the count in the bin by the number of pixels.

  In the second embodiment, similarity weights can be counted for all bins similar to each pixel, and the accuracy of the histogram can be further improved.

  Note that a look-up table can be used in the processing described above in the second embodiment. That is, the data constituting the color space and the similarity between each bin can be calculated in advance to generate a lookup table (not shown). Then, the similarity can be obtained by referring to the look-up table on the basis of data close to the input image. Thereby, the generation of the color histogram in the input image can be performed at high speed.

  The operations of the above-described embodiments can be implemented by incorporating appropriate computer software into the computer.

  The contents of the present invention are not limited to the above embodiment. In the present invention, various modifications can be made to the specific configuration within the scope of the claims.

  For example, each component described above may exist as a functional block, and may not exist as independent hardware. As a mounting method, hardware or computer software may be used. Furthermore, one functional element in the present invention may be realized by a set of a plurality of functional elements, and a plurality of functional elements in the present invention may be realized by one functional element.

  Moreover, the functional element may be arrange | positioned in the position physically separated. In this case, the functional elements may be connected by a network. It is also possible to realize functions or configure functional elements by grid computing.

1 is a block diagram illustrating a basic configuration of a color histogram generation device according to a first embodiment of the present invention. It is a flowchart for demonstrating schematically the whole flow of the color histogram generation method which concerns on 1st Embodiment of this invention. It is a flowchart for demonstrating the generation method of the partial space in 1st Embodiment. It is explanatory drawing which shows an example of the partial space produced | generated in 1st Embodiment. It is explanatory drawing which shows an example of the look-up table (part) in 1st Embodiment. It is a flowchart for demonstrating the implementation example of the clustering in 1st Embodiment. It is a flowchart for demonstrating the procedure which produces | generates the color histogram about an input image. It is explanatory drawing which shows an example of the produced | generated color histogram. It is a flowchart for demonstrating the generation method of the color histogram in 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Subspace generation part 2 Histogram generation part 3 Lookup table generation part 4 Lookup table (color space database)
6 color space 7 cluster 8 cluster center (center data of subspace)

Claims (11)

A color space database and a histogram generator,
The color space database includes information on a plurality of partial spaces generated by clustering data groups belonging to the color space,
The histogram generation unit is configured to determine which partial space a pixel in an input image belongs to, and generate a histogram based on the determination. The color space database further includes central data in the generated partial spaces,
The histogram generation unit
A process of calculating a distance between a pixel in the input image and the central data in each of the partial spaces;
The color histogram generation device according to claim 1, wherein the color histogram generation device is configured to perform a process of causing the pixel to belong to a partial space to which central data closest to the pixel belongs. The color space database is a lookup table in which data belonging to the color space used in the input image is associated with the partial space to which the data should belong,
3. The configuration according to claim 1, wherein the histogram generation unit is configured to determine the partial space to which the pixel belongs by referring to the lookup table using pixel data in the input image. Color histogram generator. The partial space in the color space database is configured such that a minimum value of a distance between a central point of each generated partial space and a central point of another partial space is constant. 4. The color histogram generation device according to any one of 3 above. Each said partial space in the said color space database is produced | generated so that the maximum value of the distance of the data which belong to each partial space, and the center point of each partial space may become fixed. The color histogram generation device described in 1. The plurality of partial spaces in the color space database further includes a partial space for achromatic colors,
6. The achromatic color partial space is generated by clustering an achromatic color data group among data groups belonging to the color space. Color histogram generator. Furthermore, it has a subspace generator,
The color histogram generation apparatus according to claim 1, wherein the partial space in the color space database is generated by the partial space generation unit. The color histogram generation apparatus according to any one of claims 1 to 7, wherein the data group clustered by the partial space generation unit is a grid point in the color space. The color histogram generation apparatus according to claim 7, wherein the data group clustered by the partial space generation unit is a representative point extracted from the color space. Providing a color space database having information on a plurality of partial spaces generated by clustering data groups belonging to the color space;
And determining which partial space the pixel in the input image belongs to, and generating a histogram based on the partial space.   A computer program for causing a computer to execute the steps according to claim 10.