KR100302366B1 - Apparatus and method for searching layout base image - Google Patents

Abstract

PURPOSE: An apparatus for searching a layout base image and a method thereof are provided to automatically index an image when inputting an image in an image database and use an image itself as a searching word when searching the image. CONSTITUTION: When a user requests a database input, a database input section(10) analyzes an image in an image directory and makes it in a database form. A small-sized image generator(60) divides a current input image into a small image and stores it in a small-sized image directory. A feature sampling section(20) samples a feature vector with respect to a current input image using a wavelet transform. A similarity calculator(30) calculates a mutual similarity with respect to all images when the feature vector is sampled and generates an index thereof. A question section(80) analyzes a question word suited to the user's searching request and outputs an image searched form the index by the similarity calculator(30). A result viewer(90) reads a small image corresponding to the searched image outputted by the question section(80) from the small-sized image directory and displays it to the user.

Description

Layout based image retrieval device and method

The present invention relates to a layout-based image retrieval apparatus and method, and more particularly, to layout-based image retrieval, which automatically indexes an image when the image is input to the image database and searches using the contents of the image itself. An apparatus and method are provided.

In the conventional database, all data can be searched only by input keywords or input values that can be calculated. Therefore, in order to store an image using such a database, there is an inconvenience of having to input a comment (comment) separately from the stored image.

In order to make up for this drawback, the conventional method of comparing the histogram of color is used. This method displays the image as a color distribution graph and compares the graphs obtained from each image. How to judge.

As another method, there is a method of comparing only the average color of an image, and this method is a method of representing the image by arithmetic mean value of colors distributed throughout the image.

In the case of layout, after dividing an image into several blocks (for example, 5x5), the layout similarity between the two images was determined based on the average color of each block.

However, the conventional method described above has a disadvantage in that the color distribution is slightly similar to the whole when it is slightly bright or dark, and there is a leap of logic that the color of the entire image is represented by one color. In the case of the conventional layout similarity determination method, it is difficult to represent the entire block by the average color of each block.

Therefore, in view of the above-described conventional circumstances, the present invention converts an image into a small image having a predetermined size through wavelet transform and extracts a predetermined number of layout-texture feature vectors using the difference image information generated in this process. It is an object of the present invention to provide a layout-based image retrieval device that automatically indexes an image by using a layout similarity measurement method that determines a similar degree in comparison, and uses the image itself as a search word when searching.

In addition, the present invention, in view of the above-mentioned conventional matters, converts an image into a small image having a predetermined size through wavelet transform, and extracts a predetermined number of layout-textured feature vectors using difference image information generated in this process. It is an object of the present invention to provide a layout-based image retrieval method in which images are automatically indexed by using a layout similarity measuring method that compares them to determine similarity, and uses the image itself as a search term when searching.

1 is a block diagram of a layout-based image retrieval apparatus according to an embodiment of the present invention;

2 is a flowchart illustrating an operation of the database input unit shown in FIG. 1;

3 is a flowchart showing a process of extracting a layout feature vector shown in FIG. 2;

FIG. 4 is a diagram illustrating a result of the wavelet transform operation illustrated in FIG. 3.

<Description of the reference numerals for the main parts of the drawings>

10: database inputter 20: feature extractor

25: Feature Vector Directory 30: Similarity Calculator

40: Image Directory 50: Small Image Generator

60: small image directory 70: index

80: Query Tool 90: Result Viewer

In order to achieve the above object, a layout-based image retrieval apparatus according to an exemplary embodiment of the present invention includes: database input means for analyzing and databaseting an image in an image directory as a database input request from a user is input;

A small image generating means for dividing a currently input image into small images under a control of the database input means and storing the image into a small image directory;

Feature extraction means for extracting a feature vector for a currently input image using wavelet transform and storing the feature vector in a feature vector directory under control of the database input means;

Similarity calculation means for generating an index by calculating mutual similarity for all images as feature extraction for the image in the image directory is completed;

Query means for analyzing a query word according to a user's search request and outputting an image searched from an index by the similarity calculation means;

And a result viewer that reads the small image corresponding to the searched image output by the querying means from the small image directory and shows the searched image to the search user.

In addition, the layout-based image retrieval method according to an embodiment of the present invention to achieve the above object, the image input process for analyzing and database the image as a database input request from the user is input;

A query process of receiving a user's query and extracting a similar image from a database;

The image input process may include: extracting layout features of an image stored in an image directory to create a feature vector; calculating similarity with feature vectors of other images in the image directory using the extracted feature vectors; Calculating similarity for all images in the image directory based on the calculated similarity;

The query process may include extracting a feature vector by receiving an image to be searched by a user's search request, and using layout feature vectors of other images previously calculated and input into the image directory using the extracted feature vector. Extracting a similarity of the; and exporting to the user in order based on the extracted similarity.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram of a layout-based image retrieval apparatus according to an embodiment of the present invention. Reference numeral 10 denotes a database input unit (DB input unit) that automates all processes for databaseting an image. 10) automatically determines whether an image to be processed is already input to the image directory 40, and if it is not input, drives the small image generator 50 to facilitate display of a new thumbnail. The image is made into a small image of about 100 × 100) and stored in the small image directory 60. In addition, since there is no feature vector for the new image, the feature extractor 20 is driven to perform a feature vector extraction operation for the new image, and the extracted feature vector is stored in the feature vector directory 25.

Here, the feature extractor 20 extracts a feature vector for an image using a wavelet transform. In an embodiment of the present invention, the wavelet transform includes an L color component, a color component, and b color component. Each is performed for and is repeatedly converted until the size of the low frequency image becomes a constant size (ie, 8 × 8 size).

Meanwhile, the feature extractor 20 extracts the feature vector after extracting the original image to be processed from the image directory 40, decompressing the image, and normalizing its size. Since the RGB color coordinate system is far from the human cognitive system, the wavelet transformation is performed after conversion into the CIE-Lab color coordinate system.

Accordingly, the feature vector extracted by the feature extractor 20 is a combination of the color layout feature vector extracted from the low frequency band as a result of the wavelet transform and the texture layout feature vector extracted from the high frequency band, and the color layout feature vector. Is a value of the low frequency band image according to the wavelet transform result, and the texture layout feature vector is a value obtained by projecting the values of the points of the low frequency band image for each direction component obtained through the wavelet transform with respect to the L color coordinates irrespective of scale. to be.

In an embodiment of the invention, the texture layout feature vector consists of 192 values.

If the same image exists in the image directory 40, the database input unit 10 selects a subsequent image without further processing the image and repeats the above operation.

Reference numeral 30 denotes a similarity calculator that calculates a mutual similarity for all images into a table and generates an index 70. The similarity calculator 30 displays the feature extractor 20 for all new images input by the user. When the feature extraction operation is completed by the control by the database input unit 10.

In this way, the similarity calculator 30 calculates the mutual similarity of all the images, creates a table, and then generates the index 70.

Reference numeral 80 denotes a querying tool used by a general user to search for an image in a pre-entered image directory 40. The querying tool 80 analyzes a query according to a user's search request and then extracts the feature extractor. The image retrieved from the index 70 is sent to the result viewer 90 by using the similarity calculator 30 and the similarity calculator 30.

In this case, the query tool 80 may be a sample image arbitrarily selected by the result viewer 90, or may be a picture drawn by a user or another image data input from an image input apparatus such as a scanner.

The result viewer 90 receives the image searched from the index 70 and inputs the received image and displays the small image from the small image directory 60 to the search user.

The operation performed by the database input unit 10 described above will be described in more detail with reference to the flowchart of FIG. 2.

The database inputter 10 extracts an image input by the user from the image directory 40 to determine whether a small image exists first (step 101). As a result of the determination, when the small image does not exist (NO in step 101), the small image generator 50 is driven to create a small image and store it in the small image directory 60 (step 102). If there is a small image ("YES" in step 101), it is not necessary to make the small image again, and it is determined whether the feature vector for the original image exists in the feature vector directory 25 (step 103).

Here, the small image is intended to make the original image larger in size and reproduce it to the user by making it smaller in size of 100 × 100 size, and reduce the size according to the aspect ratio of the original image and reduce the size to 100 × 100. Fill in the remaining areas of the square with white.

In addition, for all processed images, an arbitrary space is allocated to the feature vector directory 25 to store the feature vector, so that the same process is not repeated when the database input process is performed again.

If it is determined in step 103 that the feature vector exists, the process proceeds directly to step 105 in which it is determined whether the processing is completed for all the images in the image directory 25, whereas if the feature vector does not exist, the process proceeds to step 104. The feature vector is extracted, stored in the feature vector directory 25, and the process proceeds to step 105 above.

In this way, when the feature vector is present or the extraction of the feature vector is completed, it is determined in step 105 whether all the images in the image directory 40 have been processed. While returning and continuing processing of the step and subsequent operations, when the processing is completed for all the images in step 105, the similarity calculator 30 is driven to create a similarity table (step 106).

The similarity table is a table in which mutual similarity is calculated in advance for all images, and thus the retrieval time can be shortened by allowing similarity for all images to be obtained without calculating similarity at the time of retrieval.

The similarity table created in this way is used as an index 70 used in the result viewer 90, that is, an index 70 for finding a small image meeting the user's search request.

Meanwhile, in the embodiment of the present invention, since the wavelet transform is used when performing the layout feature vector extraction operation in step 104, the wavelet transform will be described.

The wavelet transform is a tool that separates a signal into different frequency components and analyzes each frequency component, and is defined as in Equation 1 below.

Here, ψ: analytical wavelet, a: scale, b: position.

In Equation 1, ψ (t) is a wavelet function, and there are various types such as Haar and Daubechies according to its characteristics. In the present invention, Daubechies wavelet function is used.

When wavelet transform is performed on an image, the image is transformed into a small image of 1/4 size, the other 1/4 is the difference image of the horizontal component, the other 1/4 is the difference image of the vertical component, The other quarter is reconstructed with the difference image of the diagonal component. In general, an image reduced to a quarter size is called a low frequency band, and the remaining difference image part is called a high frequency band.

When the wavelet transform is again performed on the image reduced to the quarter size, the low frequency band image reduced to the quarter size is obtained again, and the high frequency band image is obtained for the remaining region.

As a result, two consecutive wavelet transforms result in one low frequency band image of 1/16 size, three high frequency band images of 1/16 size, and three high frequency band images of 1/4 size. This conversion process may be repeatedly performed if the size of the image of the low frequency band is a convertible size.

The layout feature vector extraction operation (ie, step 104) using the wavelet transform will be described in detail as follows.

Referring to the flowchart of FIG. 3, first, the original image to be processed is taken from the image directory 40 and decompressed into a form convenient for use (step 201). The reason for the decompression is that the video data is generally very large. Therefore, the compressed data is usually stored for storage. Since the compressed video data cannot be used as it is, the compressed information is released and used again.

Next, the size of the decompressed image is normalized (step 202). In order to perform the wavelet transform, 256 × 256 images having the same aspect ratio and the same aspect ratio are easy to process. Convert to an image of size.

The decompressed image is composed of three color components of red, green, and blue. Since the color coordinates (RGB coordinate system) are suitable for physical display methods, they are far from human cognitive systems, so the normalized images Color information of the RGB coordinate system with respect to the CIE-Lab color coordinate system is converted (step 203).

Here, the CIE-Lab color coordinate system is similar to the human cognitive system, and the euclidean distance between the two colors also shows a difference in numerical meaning similar to that of the human cognitive system.

When the conversion to the CIE-Lab color coordinate system is completed, wavelet transform is performed on the previously processed image (step 204). The wavelet transform is performed on the L color component, the a color component, and the b color component, respectively, The conversion operation is repeatedly performed until the size of the band image is 8 × 8.

As a result of the wavelet transformation, 64 feature vectors are constructed by constructing 64 values for each color coordinate obtained from the low frequency band image and a total of 192 values for each color coordinate (step 205).

Thereafter, a texture feature vector is constructed for the L color component representing the intensity of the image for the high frequency band image (step 206).

Referring to FIG. 4, region 1 is a region where a low frequency component image appears, and region 2, region 3, and region 4 both include horizontal image difference image information, and region 5, region 6, and region 7 are diagonal. The region includes difference image information of components, and the region 8, region 9, and region 10 include regions of vertical image difference images.

In an exemplary embodiment of the present invention, the wavelet transform is applied again to the difference image information of the high frequency band and the wavelet transform is performed until the size of the low frequency image of each region becomes 8 × 8.

As a result, 64 values are obtained for each direction component by projecting the values of the low frequency components of the 8 × 8 size high frequency region into each direction component for each region. That is, in FIG. 4, the values of the blackened points of the region 2, the region 3, and the region 4 are all added together to obtain one value and replaced with the region 2. In this way, 64 values are obtained.

Similarly, 64 values are obtained in the same manner in the regions 5, 6 and 7, and 64 values are obtained in the same manner in the regions 8, 9 and 10, respectively.

In the present invention, the 192 values thus obtained are used as texture layout feature vectors, and 64 feature vectors of the low frequency band and 192 feature vectors of the high frequency band obtained as described above are used as the layout feature vectors of the image.

Subsequently, the similarity between the two images is calculated using the layout feature vector of the image obtained by the above operation (step 207). The similarity between the two images is calculated by the following equation (2).

S (A, B) = D (A, B) + αT (A, B)

Here, D (A, B) represents the similarity of the color layouts of the two images, and the similarity of the color layouts of the two images is calculated as the Euclidean distance as shown in Equation 3 below. The smaller the distance, the more similar the image.

In addition, T (A, B) in Equation 2 represents the similarity of the texture layout, and the similarity of the texture layout is calculated as in Equation 4 below. .

As described above, the similarity between the two images in the present invention is applied to the similarity of Equations 3 and 4 with respect to the pair of comparison images.

In addition, α in Equation 2 is an experimentally obtained weight because there is no linear relationship between D (A, B) and T (A, B), and the two similarities are summed and expressed as one similarity. . In addition, this weight α allows the user to adjust arbitrarily in the querying process, thereby resulting in a result closer to the user's request.

According to the present invention as described above, when the image data is input to the database, the feature of the data can be extracted and used as an index in the database, so that the image data itself can be used as a search word. By using to determine the layout similarity, the result is superior to that of the conventional method, that is, the method of dividing an image into blocks and comparing the average color of each block.

In addition, after the wavelet transform, the low frequency band image has a characteristic of expressing the color layout of the original image well, and in order to determine the layout similarity, by extracting the texture characteristic through wavelet transformation from the difference image of the high frequency band, It is much superior to the conventional method of comparing only colors.

In addition, the wavelet transform reduces the amount of data, thereby reducing processing time.

In addition, this invention is not limited only to the above-mentioned embodiment, It can implement by a correction, a deformation | transformation, and addition in the range which does not deviate from the summary of this invention. The technical ideas formed by such modifications, variations and additions should be regarded as technical ideas belonging to the following claims.

Claims (20)

Database input means for analyzing and databaseting an image in an image directory as a database input request from a user is input; A small image generating means for dividing a currently input image into small images under a control of the database input means and storing the image into a small image directory; Feature extraction means for extracting a feature vector for a currently input image using wavelet transform and storing the feature vector in a feature vector directory under control of the database input means; Similarity calculation means for generating an index by calculating mutual similarity for all images as feature extraction for the image in the image directory is completed; Query means for analyzing a query word according to a user's search request and outputting an image searched from an index by the similarity calculation means; And a result viewer which reads a small image corresponding to the searched image output by the querying means from the small image directory and shows the search user to the search user. 2. The layout-based image of claim 1, wherein the wavelet transform is performed on each of the L color component, the a color component, and the b color component, and is repeatedly converted until the size of the low frequency band image is constant. Search device. The apparatus of claim 2, wherein the predetermined size is 8 × 8 size. The apparatus of claim 3, wherein the extracted feature vector comprises a combination of a color layout feature vector extracted from a low frequency band as a result of the wavelet transform and a texture layout feature vector extracted from a high frequency band. 5. The apparatus of claim 4, wherein the color layout feature vector extracted from the low frequency band is a value of a low frequency image according to a wavelet transform result. The texture layout feature vector extracted from the high frequency band is a value obtained by projecting the values of the points of the low frequency image of each direction component obtained through wavelet transformation on L color coordinates irrespective of scale. Layout based image search device. 7. The apparatus of claim 6, wherein the texture layout feature vector is composed of 192 values. The apparatus of claim 1, wherein the query means is a sample image selected and displayed by the result viewer. The apparatus of claim 1, wherein the query means is a picture drawn by a user. The apparatus of claim 1, wherein the query means is image data input from an image input means. The apparatus of claim 1, wherein the similarity between the two images by the similarity calculating means is calculated by the following equation. S (A, B) = D (A, B) + αT (A, B) here, α is the weight. An image input process of analyzing and databasening an image as a database input request from a user is input; A query process of receiving a user's query and extracting a similar image from a database; The image input process may include: extracting layout features of an image stored in an image directory to create a feature vector; calculating similarity with feature vectors of other images in the image directory using the extracted feature vectors; Calculating a similarity degree for all images in the image directory based on the calculated similarity level, The query process may include extracting a feature vector by receiving an image to be searched by a user's search request, and using layout feature vectors of other images previously calculated and input into the image directory using the extracted feature vector. And extracting a similarity between the two and a similarity, and exporting to the user in order based on the extracted similarity. The method of claim 12, wherein the extracting the layout feature to create a feature vector comprises: releasing the compressed image data, normalizing the decompressed image data to a constant size image, and performing the normalized image. Converting the RGB color coordinate system for the data into a predetermined color coordinate system, wavelet transforming the image data of the converted result, and color layout feature vectors and high frequency bands extracted from the low frequency band according to the wavelet transform result. Layout-based image retrieval method comprising the step of constructing a feature vector consisting of a combination of texture layout feature vectors extracted from the. The method of claim 13, wherein the RGB color coordinate system is converted into a CIE-Lab color coordinate system. 15. The layout-based image of claim 14, wherein the wavelet transform is performed on each of the L color component, the a color component, and the b color component, and is repeatedly converted until the size of the low frequency image becomes constant. Search method. The method of claim 15, wherein the predetermined size is 8x8 size. The method of claim 15, wherein the color layout feature vector extracted from the low frequency band is a value of a low frequency image according to a wavelet transform result. 16. The method of claim 15, wherein the texture layout feature vector extracted from the high frequency band is a value obtained by projecting the values of the points of the low frequency band image of each direction component obtained through wavelet transformation on L color coordinates irrespective of scale. Layout based image retrieval method. 19. The method of claim 18, wherein the texture layout feature vector comprises 192 values. The method of claim 12, wherein the similarity of the layout features is calculated by the following equation. S (A, B) = D (A, B) + αT (A, B) here, α is the weight.