KR20100104844A - Method for searching images - Google Patents

Abstract

PURPOSE: A method of searching images is provided to efficiently search desirable UCC(User Created Contents) among UCC images on the Internet at a high speed. CONSTITUTION: An MGST(Modified Generalized Symmetry Transform) feature is extracted from a resized input query image(S204), and an angle division characteristic is extracted from an input query image(S206). A color characteristic is extracted from the input query image(S208). If a three-dimensional image identifier for the input query image, the similarity between the input query image and a reference image is compared through a three dimensional image identifier(S210). Through the matching between three-dimensional image identifiers, image searching is performed(S212).

Description

How to Search for Images {METHOD FOR SEARCHING IMAGES}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to image retrieval, and in particular, a modified generalized symmetry transform (MGST) feature, an angular partition feature, and a color for retrieving an image from digital content such as user created contents (UCC) including various geometric deformations. The present invention relates to an image retrieval method for generating a three-dimensional image identifier having a three-dimensional histogram structure, which is a combination of color features, and searching for a desired image by matching the three-dimensional image identifiers.

The present invention is derived from the research conducted as part of the IT new growth engine technology development project of the Ministry of Knowledge Economy and the Ministry of Information and Telecommunications Research and Development (Task Management No .: 2008-S-024-01, Title: Rich UCC Technology Development).

In recent years, as the demand for digital contents increases, a vast amount of multimedia contents such as video, music, and images are continuously generated, produced, distributed, and serviced. Among them, UCC video is the content that various users create, process, or edit independently. Such UCC video is explosive due to the spread of high performance, portable digital camera, mass storage device and portable storage media, and price drop. Usage is increasing.

Meanwhile, a technique for searching for such UCC images is called a UCC image search technique. Until now, UCC image search techniques have been searched for using a desktop PC or a mobile terminal with a built-in browser. It is common to input metadata of text as text and search.

However, as shown in FIG. 1, the original characteristics of the UCC image may change in size, contents, shape, or quality due to changes in the Internet or playback apparatus, processing, re-editing, or compositing of the user's contents. In this case, even if the user inputs previously input information such as metadata or an identifier, there is a problem that it is impossible to search because there is no separate metadata or identification information on the newly modified content.

Accordingly, the present invention is devised to solve the case that it is impossible to search for a necessary image in the case of no metadata or geometrically deformed digital content such as UCC, and in the absence of metadata or image of digital content such as UCC In order to search the original image when geometric deformation such as cropping, shifting, size conversion, rotation, etc., a 3D image identifier that is unique feature information of the image is generated, and an image retrieval method for matching between 3D image identifiers is provided. .

The above-described present invention is an image retrieval method comprising the steps of resizing an input query image, generating a three-dimensional image identifier for the resized input query image, and using the three-dimensional image identifier on the input query image. And performing an image search for the image.

The generating of the 3D image identifier of the present invention may include extracting an MGST feature of the input query image, extracting an angular segmentation feature of the input query image, and extracting a color feature of the input query image. It includes a step.

In the extracting of the MGST feature of the present invention, the method may include: dividing a pair of symmetrical pixel pairs based on a center pixel in a predetermined region of the input query image, and accumulating the symmetry of each pixel pair. Comprising the step of calculating.

The extracting of the color features of the present invention may include obtaining an average RGB value of a 3 × 3 area around each pixel of the input query image, and subdividing the average RGB value into HUE values of an HSI color space. It comprises the step of.

According to the present invention, there is an advantage that a high-speed search of various and vast still images including geometric deformations and a huge amount of image databases, or UCC images distributed on the Internet, can search for a desired UCC at high speed and efficiency.

In addition, it is possible to manage various versions of digital photos, so that even if the original image characteristics such as the size, shape, and quality of the image change according to the type of Internet or playback device, the identifier, which is a common characteristic of the modified image, is used. There is an advantage that can be managed by the same group of pictures.

In addition, it is possible to search for illegal video contents, and when there is illegal distribution of a video produced by copying, processing, editing and synthesizing the original video, there is an advantage that it can be determined and searched as an illegal copy of the original.

Hereinafter, with reference to the accompanying drawings will be described in detail the operating principle of the present invention. In the following description of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. The following terms are defined in consideration of the functions of the present invention, and may be changed according to the intentions or customs of the user, the operator, and the like. Therefore, the definition should be made based on the contents throughout the specification.

2 illustrates a UCC image search operation flow using a 3D image identifier according to an embodiment of the present invention.

First, when a query image to be searched for an image is input (S200), a resize is performed as a first step for generating a 3D image identifier for the input query image (S202). In this case, the query image may be a large number of images stored in a storage device such as a hard disk in a computer.

That is, in order to generate a 3D image identifier from an input query image, first, a resizing is performed on the input query image, and the image size after the resizing is 256 × N. In this case, the length of the shorter side of the input query image is 256 and the rest is enlarged or reduced to fit the ratio of the input query image.

On the other hand, since the feature used in the 3D histogram is a value calculated for each color and the black and white image, the resizing of the image must be performed separately in the color and the black and white. First, a black and white image of an input query image is constructed to produce a resized black and white image. For this purpose, the brightness component is extracted using the RGB to YUV conversion. Equation 1 below is a conversion equation for extracting the brightness component (luminance).

Subsequently, the composed black and white image is resized to a predetermined size by bi-cubic interpolation. Next, to make a resized color image, the input image is divided into three channels (Red, Green, Blue), and each channel is resized to a predetermined size by applying a third-order line interpolation method. Then, each resized channel image is matched again to produce a resized color image.

As described above, after resizing the input query image, the MGST (Modified Generalized Symmetry Transform) feature, which is one of three-dimensional image identifiers used for image retrieval, is extracted from the resized input query image (S204).

Hereinafter, the operation of extracting the MGST feature will be described. MGST is a transformation for calculating the degree of symmetry of the region. In order to apply a symmetry transform, an area is set based on one pixel as shown in FIG. 3.

Subsequently, a total of 68 pixel pairs which are symmetric with respect to the center pixel in the set region are distinguished, and a symmetric transformation is applied to each pixel pair as shown in Equation 2 below.

는 각각 화소쌍의 두 화소를 나타내고, 화소쌍의 대칭 정도(

)는 거리 가중 함수(

)와 위상 가중 함수(

), 그리고 두 화소의 체적(

)을 곱해서 계산된다. 거리 가중 함수(

)는 아래의 [수학식 3]에 의해 정의된다.

Represents two pixels of each pixel pair, and the degree of symmetry of the pixel pair (

) Is the distance weighting function (

) And the phase weighting function (

), And the volume of the two pixels (

Calculated by multiplying Distance weighting function (

) Is defined by Equation 3 below.

는 대칭 변환이 일어나는 영역의 크기이다. 또한 위상 가중 함수(

)는 아래의 [수학식 4]에 의해 계산된다.

Is the size of the region where symmetrical transformation takes place. Also, the phase weighting function (

) Is calculated by Equation 4 below.

는 각각

에서의 윤곽선의 방향을 나타내고,

는 두 화소를 연결한 직선과 수평선이 이루는 각도를 나타낸다. 마지막으로 각 화소의 체적(

)은 아래의 [수학식 5]에 의해서 계산된다.

Respectively

Indicates the direction of the contour at

Denotes the angle formed by a straight line connecting two pixels and a horizontal line. Finally, the volume of each pixel (

) Is calculated by Equation 5 below.

은 체적을 계산하기 위한 영역의 크기이고,

는 중심 화소의 화소값이고,

은 중심으로부터

만큼 떨어진 위치의 화소들의 평균 화소값이다. 68개의 화소쌍에 대해서 각각 대칭도를 계산하고, 영역의 최종 대칭도는 68개의 화소쌍의 대칭도를 전부 누적한 값이 된다. 이렇게 계산된 대칭도 값은 도 4에서 도시된 바와 같이 양자화 함수에 의해서 7단계로 구분된다.

Is the size of the area for calculating the volume,

Is the pixel value of the center pixel,

From the center

It is the average pixel value of the pixels at positions apart. Symmetric degrees are calculated for the 68 pixel pairs, respectively, and the final symmetry of the region is a value obtained by accumulating all the symmetry degrees of the 68 pixel pairs. The calculated symmetry values are divided into seven levels by the quantization function as shown in FIG. 4.

Subsequently, after extracting the MGST feature from the resized input query image, the angular segmentation feature, which is one of three-dimensional image identifiers used for image search, is extracted from the resized input query image (S206).

Hereinafter, an operation of extracting an angle division feature will be performed to calculate a feature within a local area of radius 12 for each pixel of the input query image. That is, as shown in FIG. 5, the average pixel value given the weight of the semicircle is obtained by moving the reference line separating the circle in half by 18 degrees. In this case, the weight is applied according to the distance from the center, and the absolute difference (HCD (j)) with respect to the average value of the two semicircles is calculated as shown in [Equation 6] while moving by 18 degrees. Find the acute angles of the baseline from which the maximum and minimum values are calculated from the 10 calculated absolute differences.

Here, if the difference between the maximum absolute difference and the minimum absolute difference is less than 10, this local area is considered as a flat area having no angular characteristic, and has the characteristic of the average pixel value instead of the angular characteristic. In this case, the average pixel value is quantized in eight steps. In the case of the angular characteristic AP (x, y), the average pixel value is calculated in an angle-divided area by 18 degrees. have. Accordingly, the angular splitting characteristic consists of 13 steps in total.

Subsequently, after extracting the angular split feature from the resized input query image, the color feature, which is one of three-dimensional image identifiers used for image retrieval, is extracted from the resized input query image (S208).

Hereinafter, an operation of extracting an angle division feature will be performed. The average RGB of the 3x3 area is obtained and segmented around each pixel of the input query image and mapped into Hue of a hue saturation intensity (HSI) color space. A lookup table for converting RGB to Hue is created through the same process as in FIG. 6.

That is, dividing each RGB channel in half in an RGB cube is divided into eight cubes, as shown in FIG. Next, as shown in (b) of FIG. 6, the RGB of the divided cube is doubled, and the value is converted into Hue information as shown in (c) of FIG. This process is performed for eight partitioned cubes to complete Hue's lookup table for RGB.

Accordingly, the value of 360 Hz is quantized in nine steps, and the color characteristic is composed of ten steps in total without the case where there is no Hue information.

As described above, by generating a three-dimensional image identifier consisting of the MGST feature, the angle division feature, the color feature used in the image search from the input query image, three feature values are extracted for each pixel, the three-dimensional as shown in FIG. The 3D histogram may be configured using the image identifier, and the search may be performed between the input query image and the reference image set as a comparison target.

That is, when the 3D image identifier for the input query image is generated, the similarity is compared using the 3D image identifier between the input query image and the reference image (S210), and image searching is performed by matching between the 3D image identifiers. It is performed (S212).

First, in the similarity calculation, if the difference in the similarity is greater than a predetermined standard by calculating the similarity based on the representative color between the input query image and the reference image, the two images are determined to be different and then the 3D image identifier matching is performed. You will not.

In this case, the similarity of the representative colors is calculated by the order of five representative color values among the color features extracted from the color feature extraction, and the similarity (similarity) of the representative colors is calculated by Equation 8 below.

은 두 영상의 대표색상의 개수 중에서 작은 값이고,

는 각각 동일한 색상 값이 두 영상에서 몇 번째 대표 색상인지를 나타내는 순위 값이다. 이들 대표 색상의 유사도는 다음의 3차원 히스토그램를 이용하는 3차원 영상 식별자 정합 이전에 차이가 큰 값을 일차 제거하는 용도로 활용된다.

Is the smaller of the representative colors of the two images,

Is a rank value indicating the number of representative colors in each of the same color values. The similarity of these representative colors is used to first remove a large difference value before matching the 3D image identifier using the following 3D histogram.

Next, in the matching of the three-dimensional image identifier, the difference in the similarity is less than a predetermined criterion is performed on the two images determined to have high similarity.

That is, the difference value is calculated by matching each pixel between the 3D histogram as shown in FIG. 7 using the 3D image identifier for the input query image and the 3D histogram using the 3D image identifier for the reference image, and calculating the difference. It is determined whether the input query image is a search target image similar to the reference image based on the value, and if it is a similar image, it is output as a search result.

At this time, the matching of the three-dimensional histogram is calculated by the difference value Dist (Q, R) by Equation 9 below.

은 각각 두 영상(입력 질의영상, 기준영상)의 3차원 히스토그램을 1차원 으로 나열한 것이다.

Are three-dimensional histograms of two images (input query image and reference image) in one dimension.

1, m, and n are indexes of a symmetry feature, an AP feature, and a color feature, respectively. Therefore, the maximum value of l is 7, the maximum value of m is 13, and the maximum value of n is 10.

In the three-dimensional histogram matching, the distance calculation is performed separately for each symmetry level, so d and s are calculated for each symmetry level, and d1, s1, d2, s2,... Find d7 and s7.

nd is the difference for each level of symmetry. nd1 = d1 / s1, which is the sum of difference values for each level divided by the sum of samples. When nd values are obtained for each level, the difference between the two is finally the average of 7 difference values nd1 + nd2 + nd3 +... + nd7 / 7

As described above, the present invention generates a three-dimensional image identifier of the three-dimensional histogram structure that is a combination of the MGST feature, angle segmentation feature, color feature feature to retrieve the image from digital content such as UCC including a variety of geometric deformation , By searching images through matching 3D image identifiers, it is possible to quickly and efficiently search for a wide range of still images including geometric deformations and a large amount of image databases or UCC images distributed on the Internet. You can search by.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention. Therefore, the scope of the invention should be defined by the claims rather than by the described embodiments.

1 illustrates various geometric or complex variations of a conventional image;

2 is a flowchart of an image search operation using a 3D image identifier according to an embodiment of the present invention;

3 is a structural diagram of an MGST feature of an image according to an embodiment of the present invention;

4 illustrates an example of quantization of an MGST feature according to an embodiment of the present invention;

5 is a structural diagram of an angle division feature of an image according to an embodiment of the present invention;

6 is a color feature structure diagram of an image according to an embodiment of the present invention;

7 is a three-dimensional histogram structure diagram of a three-dimensional image identifier according to an embodiment of the present invention.

Claims (15)

As an image search method, Resizing the input query image; Generating a 3D image identifier for the resized input query image; Performing an image search for the input query image using the 3D image identifier Image search method comprising a. The method of claim 1, The resize step, Extracting a black and white image and a color image from the input query image; Resizing the black and white image; Resizing the color image Image search method comprising a. The method of claim 2, The black and white image, An image retrieval method resized by three-line interpolation. The method of claim 2, Resizing the color image, Dividing the input query image into three channels; Resizing the image of each channel by third-line interpolation; Resizing the color image by re-matching the images of the resized channels; Image search method comprising a. The method of claim 1, The generating of the three-dimensional image identifier, Extracting an MGST feature of the input query image; Extracting an angle division feature of the input query image; Extracting color features of the input query image; Image search method comprising a. The method of claim 5, Extracting the MGST feature, Dividing a pair of symmetrical pixel pairs based on a center pixel in a predetermined region of the input query image; Calculating the MGST feature by accumulating respective symmetry for each pixel pair Image search method comprising a. The method of claim 6, The MGST features, Image retrieval method that is divided into 7 levels through quantization. The method of claim 5, The angle division feature, Image retrieval method divided into 13 levels through quantization. The method of claim 5, Extracting the color feature, Obtaining an average RGB value of a 3 × 3 area around each pixel of the input query image; Subdividing the average RGB value into a HUE value in an HSI color space Image search method comprising a. The method of claim 9, The HUE value is, Image retrieval method that is divided into values of 10 steps through quantization. The method of claim 1, Performing the image search, Calculating a similarity between the representative color of the input query image and the representative color of the reference image; Matching the three-dimensional image identifier with respect to the two images when the similarity is higher than a predetermined criterion; Image search method comprising a. The method of claim 11, The representative color of the input query image is, The image retrieval method comprises five representative colors selected from among colors extracted as color features from the input query image. The method of claim 11, The matching step of the 3D image identifier, Reading a three-dimensional image identifier of the input query image and a three-dimensional image identifier of the reference image; Performing matching by calculating a difference between feature values of each pixel among three-dimensional image identifiers of the images; Image search method comprising a. The method of claim 13, The three-dimensional image identifier, An image retrieval method formed of a three-dimensional histogram structure. The method of claim 1, The input query image is Image retrieval method that is UCC image.

Images