KR20010048785A - Multimedia Feature And Method Of Multimedia Retrieval System Using Binary Color Histogram - Google Patents

Abstract

PURPOSE: A method for extracting feature of a multi media object, a feature structure and a method for searching a multi media using a binary color histogram are provided to maintain a superior performance although a rapid searching time and a small feature storing space are used by using a color histogram, thereby satisfying requesting matters generated in a multi media searching. CONSTITUTION: A color histogram is created, and each bin of the created color histogram is grouped as an unmixed color system, a middle color system and a gray color system. Reference points of the unmixed color system and the gray color system are decided as lower than that of the middle color system by searching a binary-reference point for binarizing bins of the middle color system from a sample group. A multi media feature is extracted by creating a binary color histogram by binarizing each histogram bin value. When a multi media is searched, the color histogram is used by properly combining the binary color histogram according to a group by including binary color histogram information being applied different reference points according to a group into a multi media feature information structure with the existing color histogram information.

Description

Feature Extraction, Feature Structure, and Multimedia Retrieval of Multimedia Objects Using Binary Color Histograms {Multimedia Feature And Method Of Multimedia Retrieval System Using Binary Color Histogram}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia system that uses a color histogram to search for multimedia objects, and in particular, to binarize color information by group, and by using a binary color histogram for each group. By searching for an object, the present invention relates to a multimedia feature structure having a high search performance while maintaining a small space and a fast search speed, a method of extracting the feature, and a multimedia search method using the feature.

More specifically, the present invention categorizes each bin of the color histogram into N groups as a feature of multimedia, and sets a reference value for binarizing the color histogram for each group according to the color series for each group. The present invention relates to a multimedia feature structure and a retrieval method using the structure that enable a multimedia search using a binary color histogram.

More specifically, the present invention classifies each bin of the color histogram into a pure color series, a gray series, and a middle color series as a feature of multimedia, and sets the binarization reference value differently for each of the classified pure colors, gray, and middle color series. The present invention relates to a multimedia feature structure and a retrieval method using the structure that enable a multimedia search using a binary color histogram.

Recently, as the technology of searching for multimedia based on the contents has emerged, researches on multimedia features that influence search performance have been actively conducted.

Currently, the most popular search engines use global and local color information and texture information for image retrieval. Among them, color information is known as the most important factor in image retrieval.

Therefore, more effective color features have been developed, and attempts have been made to develop more efficient color spaces.

However, despite these numerous studies, the most commonly used feature is the color histogram.

The color histogram represents the distribution of colors appearing in the whole or region of the image, and the number of bins constituting the histogram depends on the quantization of the colors.

In general, the numerical value of each bin of the color histogram is expressed as a float, and the similarity measure in the search is calculated as the distance between these color histograms.

Therefore, a small number of calculations are required to calculate the distance between two images during retrieval, and a space several times as much as the empty storage space for expressing the prime number is required to express the characteristics of each image.

In general, distance calculation using a color histogram is not included in mathematically complicated calculations. However, since a faster search time is required when searching a large number of images, many efforts are being made to reduce the time required to calculate the distance of a histogram. .

The present invention satisfies the requirements of multimedia search using a color histogram, thereby enabling an effective method to maintain performance that does not drop significantly compared to a search performance using a conventional color histogram while using a fast search time and a small feature storage space. Provides a multimedia search method.

In particular, in the present invention, by dividing a color group into three series of pure color series, gray series, and middle color series, and selecting a different binarization reference point for each group, the color distribution can be efficiently represented by a binary histogram, and the existing binary The present invention provides a multimedia retrieval method with high efficiency, which is an advantage of binary color histograms while improving very low retrieval performance which is a problem of color histograms.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram showing an example of probability cumulative distribution of bin values at 256 levels for explaining the present invention.

FIG. 2 is a diagram showing an example of probability cumulative distribution of bin values at level 128 for explaining the present invention. FIG.

3 is a diagram showing an example of a color histogram and a binary color histogram for explaining the present invention.

4 is a cross-sectional view of the MMD in the HMMD color space to explain the present invention.

5 is a view showing a color sequence assigned to each bin when the MMD cross section is quantized to 18 levels for explaining the present invention.

FIG. 6 is a view showing variance of bin values of color groups of pure, neutral and gray in the present invention. FIG.

7 is a view showing the average of the bin values of the color group of the pure, middle, and gray series in the present invention

8 is a flowchart of a feature extraction method for explaining the present invention;

The present invention relates to a multimedia search using a binary color histogram, (a). As a feature of the multimedia object, each bin of the color histogram is classified into N groups, and (b). The reference value for binarization of the classified N groups is adaptively set according to the characteristics of each group to obtain a binary color histogram and perform multimedia search.

In addition, the N color bin group in the present invention;

(a). A pure color series representing pure colors, and (b). A gray series representing a gray line, and (c). A middle series representing the rest of the middle color system; Characterized in that consisting of.

In the present invention, the pure color series is; (a). In a typical color space, it represents areas of low coverage of white and black, or (b). S and V represent relatively large areas of the HSV color space, or (c). The difference between MAX (R, G, B) and MIN (R, G, B) represents a relatively large area.

In the present invention, the gray series is; (a). In the general color space, represent areas with high coverage of white and black, or (b). S and V represent relatively small regions in the HSV color space, or (c). The difference between MAX (R, G, B) and MIN (R, G, B) represents a relatively small area.

In the present invention, the pure color series and the gray series are characterized by being binarized by a lower binarization reference point than the middle series.

Further, in the present invention, the binarization reference value is lowered as the number of color quantizations, that is, the number of color histogram bins.

In addition, the multimedia search method of the present invention; To perform a multimedia search using a binary color histogram, (a). Generating a color histogram of the multimedia, and (b). Grouping each bin of the color histogram into a color sequence, a middle sequence, and a gray sequence; and (c). Finding a binarization reference point for binarizing the bins of the middle series from the sample population, and (d). Determining a reference point of a pure color series and a gray series lower than the determined binarization reference point of the intermediate series, and (e). And a binarization color histogram by binarizing each histogram bin value based on the determined binarization reference value.

In addition, the present invention is characterized in that the multimedia feature structure further comprises a binarized color histogram information for each group extracted by the feature extraction method.

Hereinafter, the multimedia searching method, the feature extraction method, and the multimedia feature structure of the present invention made as described above will be described in more detail.

In general, when a color feature of an image is represented by a color histogram representing a color probability distribution between 0 and 1, the color probability value of each bin is represented by a very small value.

Therefore, when each bin is represented by one prime number, the unexpressed regions of the prime representable regions are very large. Therefore, only the remaining regions except for these regions are subjected to proper quantization, and each bin has a smaller space than the few representation spaces. Can be expressed.

However, this method saves storage space for the feature, but does not significantly improve the time required for the histogram comparison in retrieval.

If the fraction of two decimals is expressed as a binary histogram, the distance calculation of the two histograms is an exclusive OR of bits, and the calculation is very fast, and the storage space is as much as the number of color bins. Only by the number of bits is possible.

Thus, if the color histogram can be represented as a binary histogram, there is a big advantage in space and time.

However, binary histograms are not commonly used because binary data histograms are lost because information loss is so large that searching performance is poor.

The present invention provides a binary color histogram that expresses the maximum color characteristics by assigning a binarization reference value differently for each color group when constructing a binary histogram, so that the performance does not drop significantly compared to when using a conventional color histogram. It retains the advantages of a binary color histogram while maintaining it.

By analyzing the image color distribution from the sample population, we can see that the color distribution is divided into three color groups: pure color, neutral color, and gray color.

Among them, the middle color series is usually the color series occupied by the largest area of the image, and is an important part in determining the search performance in general image searching.

Therefore, when binarizing bin values, it is efficient to find the binarization reference point based on the intermediate color series.

The characteristics of the pure color series tend to be biased toward a specific image rather than appearing uniformly in each image, so select a reference point lower than the binary reference point of the intermediate color series.

Gray series, on the other hand, has a property of appearing evenly in all images, and since the difference of the value does not affect image retrieval, it is also used to classify small but exceptional images by selecting a low reference point.

The intermediate binarization threshold value, which is the center of the binarization threshold, generally decreases as the number of color quantizations, i.e., the number of bins in the color histogram, increases.

This is because the larger the number of color quantizations, the lower each bin value as a whole.

As such, the present invention divides a color group into three series of pure color, gray color, and intermediate color, and selects a different binarization reference point for each group to efficiently express a color distribution in a binary histogram, which is a problem of the conventional binary color histogram. It provides a highly efficient search method that improves very low search performance while at the same time an advantage of binary color histograms.

As described above, the color histogram is a representation of the color characteristics of an image as a color probability distribution between 0 and 1.

Therefore, the color probability value of each bin is represented by a very small value. For example, FIGS. 1 and 2 show probability cumulative distributions of bin values for each bin when color is quantized at 256 and 128 levels, respectively.

As can be seen from the figure, most of the bins show a small probability value, and it can be seen that more than 95% of the bins have a value within 0.1.

Normally, each of these bins is represented as a decimal number between 0 and 1, and because of this characteristic, when a bin is represented as a prime number, an area of 0.1 or more does not affect image search very much.

Therefore, using this distribution of bin values, we can determine the appropriate reference value for binarizing the histogram.

The binarization of the histogram by the reference value Th is illustrated in FIG. 3, where each histogram bin value is based on the threshold value Th and the value above the threshold value is '1', and the threshold value Th is represented. The nonzero value is binarized by mapping the color histogram to zero.

The converted binary histogram has higher efficiency than the conventional color histogram in terms of storage space and search time.

First, in terms of storage space, if each bin of the existing color histogram is represented by a decimal number of 4 bytes and the number of bins is N, the total storage space of the histogram for representing an image becomes 4N bytes.

In contrast, the storage of a binary histogram of the same number of bins requires only one bit per bin, resulting in N bits, or N / 8 bytes. This results in 32 times more storage efficiency than conventional histograms.

In terms of the computation time required for the search, the most commonly used Euclidean distance measurement (sum of squares of the difference between the reference and the histogram of the target image) is used. 'Decimal operation is required.

On the other hand, binary histograms are fast enough that they do not need to be specified numerically since N XOR bit operations are required.

4 is a cross-sectional view of Min_Max_Diff in the color space (HMMD) expressed as Hue, Min (R, G, B), Max (R, G, B), and Diff (Max-Min). Is showing through.

In the drawing, region 401 is a region where the diff is above a certain threshold, and shows a pure color sequence, region 402 is a middle region of the color region, and region 403 represents a region where the diff is below a certain threshold.

Based on these criteria, Figure 5 shows the color sequence assigned to each bin when the Min_Max_Diff cross section is quantized to 18 levels.

Based on the color series criteria defined in FIG. 4, in FIG. 5, bin 0 corresponds to a pure color series, bin 1 to bin 9 correspond to a middle series, and the rest corresponds to a gray series.

FIG. 6 shows the variance of the bin values of the three series of color groups and FIG. 7 shows the mean.

As can be seen from the figure, both the variance and the average value of the pure color series are low, which means that the pure color series hardly appears depending on the image or appears in a specific pure color only in a specific image.

Therefore, the pure color series can reflect the color characteristics of the image well when binarized with a lower reference value than other series.

Gray series have both large variances and average values.

This means that the difference between the histogram bin values for each image is only the difference in the value itself, so that the gray line does not affect most image searches.

However, since there are groups of images that can be distinguished by some sparse grays, it may be effective to binarize the histogram with a low reference value to distinguish these groups of images.

8 illustrates a multimedia feature extraction process for implementing the present invention.

First, a color histogram is generated as in the general method, and each bin of the generated color histogram is grouped into a color sequence, a middle sequence, and a gray sequence.

Then, the binarization reference point for binarizing the bins of the middle series is found from the sample population, and the reference point of the pure color series and the gray series is determined to be lower than the determined binarization reference point of the intermediate series.

The multimedia feature is extracted by generating a binary color histogram by binarizing each histogram bin value according to the binarization reference value determined as described above.

The multimedia feature information extracted in this way may be included as the corresponding multimedia feature.

That is, by including binary color histogram information with different reference points for each group together with the existing color histogram information in the multimedia feature information structure, the color histogram and the group binary color histogram are appropriately combined during the multimedia search. It will be available.

As described above, the present invention uses a binary color histogram in a multimedia search using a color histogram, thereby having an effect of having a very fast searching time and a very small feature space, and at the same time using a conventional binary color histogram. Compared to the original color histogram, the search performance is higher than that of the original color histogram.

In particular, the present invention uses a binarized color histogram applying different binarization reference points for each color group in a multimedia search using a color histogram, thereby providing a multimedia search engine with sufficient performance while having faster search time and less feature storage space. It can be effective.

Claims (10)

In a multimedia retrieval method using a binary color histogram as a feature of a multimedia object, (a). Each bin of the color histogram is classified into N groups, and (b). A reference point for binarizing the color histogram for each of the classified N groups is set differently from each other (c). And performing a multimedia search by binarizing the color histogram on the basis of the binarization reference point set differently for each group. 2. The apparatus of claim 1, wherein the N color bin groups are in color space; (a). A pure color series representing pure colors, (b). A gray series representing a gray series, and (c). Multimedia search method using a binary color histogram, characterized in that the classification of the remaining middle color except the pure color and gray series. The method of claim 2, wherein the pure color series represents a region having a low content ratio of white and black, a region where S and V are relatively large in the HSV color space, or Max (R, G, B in the HMMD color space. ) And a multimedia search method using a binary color histogram, characterized in that the difference between Min (R, G, B) represents a relatively large area. The method of claim 2, wherein the gray series represents a region having a high content ratio of white and black, a region where S and V are relatively small in the HSV color space, or Max (RGB) and Min in the HMMD color space. A multimedia retrieval method using a binary color histogram, characterized by a relatively small area difference between (RG and B). 3. The multimedia retrieval method according to claim 2, wherein the binarization reference point is determined based on an intermediate color series to determine a binarization reference point of a pure color series and a gray series. 6. The multimedia retrieval method using a binary color histogram according to claim 2 or 5, wherein the binarization reference point of the pure color series and the gray series is binarized by a binarization reference point lower than that of the intermediate color series. . 3. The multimedia retrieval method according to claim 1 or 2, wherein the binarization reference value is determined in inverse proportion to the color quantization number (bin number of the color histogram). In a multimedia retrieval method using a binary color histogram as a feature of a multimedia object, (a). Creating a color histogram, (b). Classifying each bin of the color histogram into a pure color series, a gray series, and a middle color series; (c). Finding a binarization reference point of the intermediate series; (d). Determining a reference point of a pure color series and a gray series lower than the binarization reference point of the intermediate series; (e). Generating a binary color histogram by binarizing the bins of each histogram by the binarization reference value determined for each series; Multimedia feature extraction method characterized in that consisting of. 9. The method of claim 8, wherein the binarization reference point of the intermediate series determines a reference point for binarizing the bins of the intermediate series from a sample population. A multimedia feature structure comprising the feature feature extracted by the method according to claim 8 in a multimedia feature structure.