KR20130059058A - Preprocessing apparatus for query image and searching image in content based image retrieval using sketch query and methof therefor - Google Patents

Abstract

PURPOSE: A preprocessing apparatus of a query image and a search object image in a content-based image searcher which uses a sketch query and a method thereof are provided to improve image search performance through the sketch query by transforming the sketch query and a database image. CONSTITUTION: A sketch query preprocessing unit(100) extracts an image feature by transforming an inputted sketch query. A database image preprocessing unit(200) extracts an image feature by transforming an image which is stored in a database(210). A similarity measurement unit(300) measures a similarity of two images which are processed through the sketch query preprocessing unit and the database image preprocessing unit. Based on the measured similarity, an image search unit(400) searches an image which has a high similarity with the sketch query from the database image preprocessing unit. [Reference numerals] (100) Sketch query preprocessing unit; (110) User input module; (120) Sketch query preprocessing module; (130) Sketch query feature extracting module; (200) Database image preprocessing unit; (210) Database module; (220) Database image preprocessing module; (230) Database image feature extracting module; (300) Similarity measurement unit; (400) Image search unit; (AA) Sketch query

Description

Preprocessing apparatus and method for query image and search target image in content-based image searcher using sketch query {PREPROCESSING APPARATUS

The present invention relates to a preprocessing device and a method of preprocessing a query image and an image to be searched, and more particularly, to a sketch query for improving the search performance when performing an image search through a sketch query in a content-based image searcher. A method and apparatus for preprocessing data images.

As the demand for multimedia services such as images and videos increases and portable multimedia devices are widely used, there is an increasing need for an efficient multimedia retrieval system that manages a large amount of multimedia data and quickly finds and provides contents desired by consumers.

Recently, a lot of attention has been paid to CBIR (Contents-Based Image Retrieval) technology for searching similar images based on images. Conventional text-based image retrieval is a method of assigning a text-type tag to a searched database image and extracting an image having a tag that matches a search word input by a user. This method is not only required to tag a large amount of images, but also has difficulty in fully satisfying the needs of users because it is difficult to express images containing visual information only by text.

CBIR, on the other hand, uses the image itself rather than text as a query, which eliminates the need for cumbersome image tagging and enables search through visual depictions that are difficult to express with text alone. have. In particular, a technique that uses a sketch drawn by a user as a query image is called Sketch-Based Image Retrieval, which recently used a touch pad that can draw simple sketches such as tablet PCs and smartphones. The popularity of mobile multimedia devices has attracted much attention.

On the other hand, in relation to sketch-based image retrieval technology, the Journal of KIISE: Software and Applications, Vol. 30, No. 8 (2003.8), "Web-Based Image Retrieval and Classification System through Sketch Query" Some are public.

The apparatus according to the prior art classifies images, freezes the semantic information of the high level of the images, reduces the search space when searching, and enables quick searching, and actively displays the images desired by the user through a sketch query.

Also, unlike CBIR, Sketch-Based Image Retrieval (SBIR) technology has the advantage of being more convenient because users can draw on the fly without having to prepare an image for search. SBIR's application fields are diverse, such as art search, image patent protection, and suspect face search using montage.

When using SBIR technology to operate an image search, the SBIR system takes as input a garment sketch image created by the user with a touch pad or tablet, extracts semantic information, and selects and retrieves images with similar semantic information from the image database. You can show it as a result. If an image can be searched through a sketch like this, the user can find the desired image using only the sketch drawn from the image that comes to mind without having to remember the same specific term entered by the user in order to find the image such as the search result. While SBIR is more convenient and useful for applications than CBIR, there are disadvantages as well. Because sketched images contain much less information than regular images, it is more difficult to extract meaning from them and find similar images.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and an object thereof is to improve image retrieval performance through sketch queries by modifying sketch queries and database images.

The present invention for achieving the technical problem relates to a pre-processing device of the query image and the search target image in the content-based image searcher using a sketch query, a sketch query preprocessor for extracting image features by modifying the input sketch query ; A database image preprocessor configured to extract an image feature by modifying an image stored in a database; A similarity measurer for measuring similarity between two images processed by the sketch query preprocessor and a database image preprocessor; And an image retrieval unit for retrieving an image having a high similarity to a sketch query from the database image preprocessor based on the similarity measured by the similarity measurer. .

The present invention relates to a preprocessing method of a query image and a search target image in a content-based image searcher using a sketch query, comprising: (a) extracting an image feature by modifying an input sketch query; (b) extracting image features by modifying an image stored in the database by the database image preprocessor; (c) a similarity measurer measuring the similarity of the two images processed by the sketch query preprocessor and the database image preprocessor; And (d) searching for an image having a high similarity to a sketch query from the database image preprocessor based on the similarity measured by the image searcher through the similarity measurer. .

According to the present invention as described above, in the case of performing the image search through the sketch query in the content-based image searcher, by modifying the sketch query and the database image, there is an effect that can improve the image search performance through the sketch query. .

1 is an overall configuration diagram conceptually showing an apparatus for preprocessing a query image and a search target image in a content-based image searcher using a sketch query according to the present invention;
2 is an exemplary view of a sketch query preprocessing module using morphological thinning according to the present invention.
3 is an exemplary view of a DB image preprocessing module using a Canny edge detector according to the present invention.
4 is a flowchart illustrating a preprocessing method of a query image and a search target image in a content-based image searcher using a sketch query according to the present invention.
5 is a detailed flowchart of a process of extracting image features by modifying an input sketch query according to the present invention;
6 is a detailed flowchart illustrating a process of extracting image features by modifying an image stored in a database according to the present invention.

Specific features and advantages of the present invention will become more apparent from the following detailed description based on the accompanying drawings. It is to be noted that the detailed description of known functions and constructions related to the present invention is omitted when it is determined that the gist of the present invention may be unnecessarily blurred.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in detail with reference to the accompanying drawings.

A preprocessing apparatus for a query image and a search target image in a content-based image searcher using a sketch query according to the present invention will be described with reference to FIGS. 1 to 3.

FIG. 1 is a schematic diagram schematically showing a pre-processing apparatus A of a query image and a search target image in a content-based image searcher using a sketch query according to the present invention. As illustrated, the sketch query preprocessor 100 is illustrated. ), A database image preprocessor 200, a similarity measurer 300, and an image searcher 400.

The sketch query preprocessor 100 performs a function of extracting image features by deforming the input sketch query, as shown in FIG. 1, the user input module 110, the sketch query preprocessing module 120, and Sketch query feature extraction module 130 is included.

The user input module 110 receives a sketch query from the user.

Here, the sketch query is composed of digital images created by various graphic input devices such as a mouse, a tablet, and a touch pad.

The sketch query preprocessing module 120 transforms the sketch query input through the user input module 110.

In detail, the sketch query preprocessing module 120 uses a morphological operation method such as wavelet transformation, morphological thinning, and morphological region filling, from a sketch query. By modifying or improving sketch queries before feature extraction, the performance of the CBIR method can be improved.

2 is an exemplary view of a sketch query preprocessing module 120 using morphological thinning according to the present invention. As shown, an input image is first stored in an image memory, and the stored image is previously defined. The H & M transform is performed using a structured component. The converted image is fed back to the image memory, and the process is repeated until the pixel thickness of the connected component of the image becomes one.

The sketch query feature extraction module 130 extracts a sketch query feature that can be clearly distinguished from other images from the feature extraction target image.

In addition, the database image preprocessor 200 performs a function of extracting image features by transforming the image stored in the database, as shown in FIG. 1, the database module 210 and the DB image preprocessor module 220. And a DB image feature extraction module 230.

The database module 210 stores various images such as natural and synthetic images. At this time, the size of the database module 210 is not limited.

The DB image preprocessing module 220 transforms the image stored in the database module 210.

In detail, the DB image preprocessing module 220 uses a Gaussian filter, a Laplacian filter, a Difference of Gaussian (DoG), and Canny edge detection to search. By modifying or improving the images, it is possible to increase the performance of the CBIR method.

FIG. 3 is an example of a DB image preprocessing module 220 using a Canny edge detector according to the present invention. As shown, noise reduction is performed using a Gaussian filter or the like. A gradient operation for edge component detection (gradient operation), non-maximum suppression for interpolating broken edge lines (non-maximum suppression), and hysteresis thresholding for binarizing edge maps are performed.

The DB image feature extraction module 230 extracts features of the DB image that can be clearly distinguished from other images from the feature extraction target image.

Meanwhile, the sketch query feature extraction module 130 and the DB image feature extraction module 230 may include an edge histogram descriptor (EHD), a tensor field descriptor, and an Angular partitioning of abstract image (APAI). Features can be extracted using the method of.

In addition, the similarity measurer 300 measures the similarity between the two images processed by the sketch query preprocessor 100 and the database image preprocessor 200.

In detail, the similarity measurer 300 measures similarity with respect to a feature extracted from two different images by using a sum of absolute distance (SAD) and an euclidian distance.

The image search unit 400 searches for an image having a high similarity to a sketch query from the database module 210 of the database image preprocessor 200 based on the similarity measured by the similarity measurer 300. .

In this case, the image search unit 400 searches an image by analyzing information of the content itself, not an image tag composed of text, but extracts all data or an edge histogram descriptor (e.g., color, brightness, etc.). The feature may be searched using feature information extracted through an Edge Histogram Descriptor (EHD), a Tensor field descriptor, and an Angular partitioning of Abstract Image (APAI).

Hereinafter, a method of preprocessing a query image and a search target image in a content-based image searcher using a sketch query using the above-described apparatus according to the present invention will be described with reference to FIGS. 4 to 6.

FIG. 4 is a flowchart illustrating a preprocessing method of a query image and a search target image in a content-based image searcher using a sketch query according to the present invention. As shown in the drawing, the sketch query preprocessor 100 receives an input sketch query. A process of extracting image features by deforming (S100), a process of extracting image features by deforming an image stored in a database by the database image preprocessor 200 (S200), and the similarity measurer 300 includes the sketch query preprocessor ( 100) and a process of measuring similarity between the two images processed by the database image preprocessor 200 (S300) and the image retrieval unit 400 based on the similarity measured by the similarity measurer 300. And a step (S400) of retrieving an image having a high similarity to the sketch query from the database image preprocessor 200.

In this case, the third process S300 measures similarity with respect to features extracted from two different images using a sum of absolute distance and a Euclidian distance, and the fourth process S400. ) May search for an image by analyzing information of the content itself, not an image tag composed of text, but may search for all data or feature information that may be extracted from an image including color, brightness, and the like.

5 is a detailed flowchart of a first process (S100) according to the present invention. As shown, the user input module 110 receives a sketch query from a user (S110), and the sketch query preprocessing module 120 is a wavelet. Transform or improve the sketch query before extracting features from the sketch query using morphological operations such as transformation, morphological thinning, and morphological region filling. (S120), the sketch query feature extraction module 130 extracts features using methods such as an edge histogram descriptor (EHD), a tensor field descriptor and an Angular partitioning of abstract image (APAI). A sketch query feature that can be clearly distinguished from other images is extracted from the target image (S130).

6 is a detailed flowchart of a second process (S200) according to the present invention. As shown, the DB image preprocessing module 220 includes a Gaussian filter, a Laplacian filter, and a Gaussian difference. The Gaussian: DoG) and Canny edge detection are used to modify or improve search target images stored in the database module 210 (S210), and the DB image feature extraction module 230 performs an edge histogram descriptor. Features of a DB image that can be clearly distinguished from other images from the feature extraction target image using methods such as Edge Histogram Descriptor (EHD), Tensor field descriptor, and Angular partitioning of Abstract Image (APAI) It is extracted (S220).

While the present invention has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. It will be appreciated by those skilled in the art that numerous changes and modifications may be made without departing from the invention. Accordingly, all such appropriate modifications and changes, and equivalents thereof, should be regarded as within the scope of the present invention.

A: Preprocessing device for query image and searched image in content based image searcher using sketch query
100: sketch query preprocessor 110: user input module
120: sketch query preprocessing module 130: sketch query feature extraction module
200: database image preprocessor 210: database module
220: DB image preprocessing module 230: DB image feature extraction module
300: similarity measurement unit 400: image search unit

Claims (16)

In the pre-processing device of the query image and the search target image in the content-based image searcher using a sketch query,
A sketch query preprocessor 100 extracting an image feature by modifying an input sketch query;
A database image preprocessor 200 extracting an image feature by modifying an image stored in a database;
A similarity measurer 300 measuring the similarity of the two images processed by the sketch query preprocessor 100 and the database image preprocessor 200; And
An image retrieval unit 400 for retrieving an image having a high similarity to a sketch query from the database image preprocessor 200 based on the similarity measured by the similarity measurer 300; Pre-processing device of the query image and the search target image in the content-based image searcher using a sketch query including a.
The method of claim 1,
The sketch query preprocessing unit 100,
A user input module 110 for receiving a sketch query from a user;
A sketch query preprocessing module 120 for modifying a sketch query input through the user input module 110; And
A sketch query feature extraction module 130 for extracting a sketch query feature that can be clearly distinguished from other images from the feature extraction target image; Pre-processing device for the query image and the search target image in the content-based image searcher using a sketch query comprising a.
The method of claim 1,
The sketch query,
A pre-processing device of a query image and a search target image in a content-based image searcher using a sketch query, characterized in that the digital image is created by various graphic input devices including a mouse, a tablet, and a touch pad.
3. The method of claim 2,
The sketch query preprocessing module 120,
An apparatus for preprocessing a query image and a searched image in a content-based image searcher using a sketch query, wherein the sketch query is transformed before extracting features from the sketch query using a morphological operation.
The method of claim 4, wherein
The morphological operation method,
Pre-processing device for query image and searched image in content-based image searcher using sketch query characterized in that any one of wavelet transform, morphological thinning, morphological region filling .
3. The method of claim 2,
The sketch query feature extraction module 130,
In a content-based image retrieval using a sketch query characterized in that it uses any one of an edge histogram descriptor (EHD), a tensor field descriptor, and an Angular partitioning of abstract image (APAI). Preprocessing device for query image and search target image.
The method of claim 1,
The database image preprocessor 200
A database module 210 for storing various images including natural and synthetic images;
DB image preprocessing module 220 for modifying the image stored in the database module 210; And
A DB image feature extraction module 230 for extracting features of the DB image that can be clearly distinguished from other images from the feature extraction target image; Pre-processing device for the query image and the search target image in the content-based image searcher using a sketch query comprising a.
The method of claim 7, wherein
The DB image preprocessing module 220,
The image to be searched is transformed using any one of a Gaussian filter, Laplacian filter, Difference of Gaussian (DoG), and Canny edge detection. A pre-processing device for query images and search target images in a content-based image searcher using a sketch query.
The method of claim 7, wherein
The DB image feature extraction module 230,
In a content-based image retrieval using a sketch query characterized in that it uses any one of an edge histogram descriptor (EHD), a tensor field descriptor, and an Angular partitioning of abstract image (APAI). Preprocessing device for query image and search target image.
The method of claim 1,
The similarity measuring unit 300,
Content-based using sketch query, which measures the similarity of features extracted from two different images, using either method of Sum of Absolute Distance (EAD) and Euclidian distance (Euclidian distance). Pre-processing device for the query image and the search target image in the image searcher.
The method of claim 1,
The image search unit 400,
A pre-processing apparatus of a query image and a search target image in a content-based image searcher using a sketch query, characterized by searching for an image by analyzing information of the content itself in an image including color and brightness configurations.
A method of preprocessing a query image and a search target image in a content-based image searcher using a sketch query,
(a) the sketch query preprocessor 100 extracting the image feature by modifying the input sketch query;
(b) the database image preprocessor 200 extracting an image feature by modifying an image stored in a database;
(c) a similarity measurer 300 measuring the similarity of the two images processed by the sketch query preprocessor 100 and the database image preprocessor 200; And
(d) searching for an image having a high similarity to a sketch query from the database image preprocessor 200 based on the similarity measured by the similarity measurer 300 by the image searcher 400; The pre-processing method of the query image and the search target image in the content-based image searcher using a sketch query including a.
13. The method of claim 12,
In the step (a)
(a-1) the sketch query preprocessor 100 receiving a sketch query from a user;
(a-2) deforming the sketch query before extracting the feature from the sketch query by the sketch query preprocessor 100 using a morphological operation method; And
(a-3) The sketch query preprocessor 100 uses any one of an edge histogram descriptor (EHD), a tensor field descriptor, and an Angular partitioning of abstract image (APAI). Extracting a sketch query feature that can be clearly distinguished from another image from the feature extraction target image; The pre-processing method of the query image and the search target image in the content-based image searcher using a sketch query comprising a.
13. The method of claim 12,
The step (b)
(b-1) The database image preprocessor 200 is any one of a Gaussian filter, a Laplacian filter, a Difference of Gaussian (DoG), and Canny edge detection. Transforming the stored search target images using; And
(b-2) The database image preprocessor 200 uses any one of an edge histogram descriptor (EHD), a tensor field descriptor, and an Angular partitioning of abstract image (APAI). Extracting a feature of the DB image that can be clearly distinguished from another image from the feature extraction target image; The pre-processing method of the query image and the search target image in the content-based image searcher using a sketch query comprising a.
13. The method of claim 12,
In the step (c)
The similarity measurer 300 measures similarity of features extracted from two different images by using any one of a sum of absolute distance (SAD) and an euclidian distance (Euclidian distance). A preprocessing method of a query image and a search target image in a content-based image searcher using a sketch query.
13. The method of claim 12,
In the step (d)
Pre-processing of the query image and the searched image in the content-based image searcher using a sketch query characterized in that the image search unit 400 searches for the image by analyzing the information of the content itself in the image including the color and brightness configuration Way.

Images