KR20030029410A - System for searching image data being based on web and method thereof - Google Patents

Abstract

PURPOSE: A system and a method for searching moving-image data based on a web are provided to perform the content based searching using a query moving-image as well as a query word, to enhance searching performance by integrally using a texture descriptor and a color descriptor, and to realize the efficient content based searching for the vast data on the web by using a meta searching engine. CONSTITUTION: A user interface module(10) inputs the query word or the query moving-image through a user interface on the web and uploads the query moving-image to a local host of a user. The meta searching engine module(20) collects the moving-images scattered on the web based on the query word, and selects and stores various metadata while outputting the collected moving-image. A metadata extracting module(30) extracts the color descriptor and the texture descriptor from the data stored by the meta searching engine module(20) or the user interface module(10). A database(40) stores the extracted descriptors and other metadata. A searching result output module(50) measures a similarity between the data in the database(40) and the query word or the query moving-image, and outputs the searching result data in a constant order according to the measuring result.

Description

System for searching image data being based on web and method

The present invention relates to a web-based image retrieval engine, and more particularly, to a web-based image data retrieval system and method for enabling integrated retrieval based on texture and color descriptors for images in MPEG-7 international standard models. It is about.

Nowadays, with the development of computer and internet technology, multimedia data is increasing exponentially. The multimedia data on the Internet, accessible to anyone around the world, is immense.

In this massive amount of multimedia data retrieval, when a user enters a query to search for a specific image, some commercial search engines perform a text-based search. This is not a search based on the content of the image itself. Often, images are found that have nothing to do with.

Due to the above problems of character-based multimedia data retrieval and the necessity of efficient data management, retrieval and recognition, various retrieval algorithms based on the characteristics of video contents have been developed in the MPEG-7 standardization process.

Here, MPEG-7 is an international standard that has been standardized in ISO / IEC since early 1998, and is officially called a multimedia content description interface.

Unlike MPEG-1 / 2/4, which aims to compress audiovisual data, MPEG-7 is a standard for describing various kinds of multimedia data, and each technician is based on the contents of multimedia data. In addition, it is very suitable for perceptual and cognitive characteristics because it expresses the content characteristics of original multimedia data in small amounts in detail.

In MPEG-7, a description is a tool that associates a feature with a value, description schemes represent relationships between descriptors, and description definition language represents a description structure. .

Recently, in line with MPEG-7 standardization, research on content-based retrieval of images and videos has been actively conducted. However, research on content-based search engines of images using MPEG-7 international standard descriptors is just the beginning.

In addition, the image retrieval system according to the prior art has defined a new image feature called a 'color correlogram' for image indexing and retrieval. It is very effective feature information for searching.

The image retrieval system includes a color feature vector extractor and a similarity measurer to provide a color-based image retrieval function so that a user can search for an image having a color similar to this by using the image as a query and provide the retrieval result.

However, the conventional image retrieval system provides a color-based image retrieval by defining a feature vector expressing color and uses a fixed database. Therefore, the image retrieval system cannot search and collect images for the web, and cannot expand the database. There is a problem in that an image search function by a query cannot be provided.

In addition, the conventional image retrieval system has a method of performing multi-resolution processing in the compressed region and performing content-based image retrieval by measuring correlation in the frequency domain through wavelet transform.

The above method uses a fixed database in the compressed region of the image and measures the correlation in the frequency domain, so it is impossible to process in the uncompressed region, and image search by keyword query is impossible. There is a problem that cannot be extended continuously.

Meanwhile, in the conventional image retrieval system, a method of performing a high-definition search of a desired image using a color by using a color may include a control unit, a sample image input unit, an image color information extraction unit, a contribution factor determination unit, a candidate image determination unit, and the like. It is composed of color image based search function.

Since the color-based image retrieval system searches for images using only color feature values, the accuracy of retrieval is low, and since a fixed database is used, the database cannot be expanded.

The present invention is to solve the above problems, an object of the present invention is to enable a content-based search using a query image as well as a query when the image search, it is possible to improve the search performance by using a texture and color descriptors integrally, The present invention provides a web-based image data retrieval system and method that enables efficient content-based retrieval of vast amounts of data on the web by performing a search on images collected from the web using a meta search engine.

1 is a block diagram showing the configuration of a web-based image data retrieval system according to an embodiment of the present invention.

2 is a flowchart illustrating a web-based image data retrieval method according to an embodiment of the present invention.

FIG. 3 is a diagram illustrating a search result screen of a meta search engine when the administrator inputs a query word 'sunrise'.

4 is a flowchart illustrating a process of measuring similarity between images using texture and color descriptors.

5 is a diagram illustrating screens of a user interface and an image search result.

A feature of the web-based image data retrieval system of the present invention for realizing the above object is a search engine for collecting data existing on the web, a local host of the user who uses the search engine, and an internet network. A web-based image data retrieval system connected through a web, comprising: a user interface module for inputting a query word or a query image through a user interface implemented on a web; A meta search engine module for collecting and outputting images scattered on the web based on a query input by an administrator, and selecting a specific image from the collected images; A metadata extraction module for extracting texture and color descriptors from an image selected through the meta search engine module or a query image input from the user interface module; A database that stores image data selected by the meta search engine module and descriptors extracted by the metadata extraction module; And a search result output module configured to measure the similarity between the image in the database and the query word or query image to be searched and output search result data in a predetermined order according to the measurement result.

The user interface module selects weights to be assigned to texture and color descriptors when a query image is input.

The user interface module may further include an uploading unit uploading a query image in the local host of the user to the system when the query image is input.

The search result output module calculates a geometric distance between the texture and color descriptors of the images in the database and the texture and color descriptors of the query image, and measures similarity by assigning a weight selected by the user to the geometric distance.

In addition, the search result output module of the present invention, the similarity measuring unit for measuring the similarity between the texture and color descriptors of the images stored in the database and the query image by the user input, the similarity measurement And a data output unit for outputting the search results on the web page in order from the image having high similarity according to the negative measurement result.

Meanwhile, a feature of the web-based image data retrieval method of the present invention is a search engine for collecting data existing on the web, and web-based image data connected to a local host of a user who uses the search engine through an internet network. A search method comprising: a first step of collecting images necessary for searching an image on a web, selecting a specific image from the collected images, and storing the image in a database; A second step of extracting texture and color descriptors of the images selected in the first step and storing the descriptors and metadata in a database; A third step of inputting a query word or a query image to be searched through a user interface; A fourth step of extracting texture and color descriptors of the image input in the third step and measuring similarity with descriptors stored in the database; And outputting image data as search results in order according to the similarity measurement result of the fourth step.

In the third step, when the user inputs a query word, the central word of the images stored in the database is compared with the content corresponding to the query word, and the images having similar contents are output.

The third step includes, when the user inputs the query image, the user selects the image on the local host as the query image, uploads it to the system, and specifies a weight to be assigned to the texture and color descriptor for image retrieval. do.

In the second or fourth step, when the texture and color descriptor are extracted, the images collected in the database are converted to a designated color display format and normalized in size, and then the texture and color descriptor are extracted through sampling.

The geometric distance between the texture and color descriptors of the images in the database and the texture and color descriptors of the query image to be searched for by the user is calculated, and the similarity is measured by giving a weight selected by the user to the geometric distance.

In the fifth step, when outputting image data as a search result, a list control is possible so that a user can select at least one of the search result images to use as a query image.

In the fifth step, when the user requests re-search after outputting the image data as a search result, the database is expanded so that the user can directly collect necessary images from the web and search for an image not stored in the database.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a web-based image data retrieval system according to an embodiment of the present invention.

A search engine for gathering data present on the web and a system of an embodiment according to the present invention connected to a local host of a user using such a search engine via an internet network, as shown in FIG. A user interface module 10 for inputting a query word or a query image through a user interface and uploading a query image from a user's local host;

A meta search engine module 20 for collecting images scattered on the web based on a query input by an administrator and selecting and storing various metadata while outputting the collected images;

A metadata extraction module 30 for extracting texture and color descriptors from data stored through the meta search engine module 20 or the user interface module 10;

A database 40 in which descriptors and other metadata extracted from the meta search engine module 20 or the metadata extraction module 30 are stored, and

The search result output module 50 measures the similarity between the data in the database 40 and the query word or query image to be searched and outputs search result data in a predetermined order according to the measurement result.

In particular, metadata represents a category, a core word, a file name, a file path, a URL, and the like.

The user interface module 10 includes a user input unit 11 through which a user inputs a query in a query field or a query image in a query image field through a web-implemented user interface, and is located on the user's local host when the query image is input. It includes an upload unit 13 for uploading an image.

In this case, when the query image is input, the user input unit 11 selects a weight to give more importance to texture or color descriptors.

The meta search engine module 20 collects the images scattered on the web by passing the query word or query image input by the administrator to an existing commercial web search engine and outputs the image to a web page for the administrator to select. 21 and a data storage unit 23 for selecting an image to be stored in the database 40 among the images collected by the data collector 21 and storing the selected image in a specific directory.

The search result output module 50 measures the similarity with the texture and color descriptors of the images stored in the database 40 in the similarity measurer 51, and the order from the images with the high similarity in the data output 53. Print the search results on the web page as it is.

In this case, since the similarity measuring unit 51 uses both texture and color descriptors among the MPEG-7 standard descriptors, the similarity measuring unit 51 weights the L1 difference value, which is a similarity measuring method when the two descriptors are used, respectively, to obtain similar images from the image having the smallest value. Output

The operation of the web-based image data retrieval system of the present invention configured as described above will be described with reference to the flowchart of the attached web-based image data retrieval method.

2 is a flowchart illustrating a web-based image data retrieval method according to an embodiment of the present invention.

As shown in FIG. 2, the data collection unit 21 of the meta search engine module 20 collects images scattered on the web based on a query input by an administrator, and among the collected images, only images necessary for the administrator are needed. The image is output to a web page so that it can be selected (S1). In addition, the data storage unit 23 stores the selected image after viewing the web page in the database 40 (S2).

3 illustrates a search result screen of the meta search engine when the administrator inputs a query term 'sunrise'.

First, when the administrator inputs a query word 'sunrise' into the meta search engine, a result screen such as (a) and (b) shown in FIG. 3 is output. 3A is an upper portion of the result screen for the query word 'sunrise', and (b) is a lower portion of the result screen. It can be seen that images that are not related to the query are output from the upper portion of the result screen to the lower portion of the query word.

Therefore, the administrator can select and store only the image desired by the administrator using the left check button of (a) in the database 40, and input a category at the end of the result screen of (b).

The image stored through the meta search engine module 20 is an image in the form of a JPEG or GIF file and its size varies. Accordingly, the metadata extraction module 30 converts a given image into a format such as red (green, green and blue) or HSV (Hue Saturation Value) through the meta search engine module 20, and normalizes the size of the image. Then, various metadata, including MPEG-7 texture and color descriptor 1, are extracted and stored in the database 40 (S3).

Here, the MPEG-7 texture and color descriptor may be expressed in a vector form having M and N components, such as Equations 1 and 2 below.

In this case, TD is a vector form of a texture descriptor (TD) among standard descriptors of MPEG-7. The texture descriptors consist of the energy averages of the brightness averages and standard deviations of a given image, f _DC , f _SD, and the energy characteristics of all 30 channels created by dividing the frequency domain into five and six radial and angular directions in the radon domain. Energy averages e ₁ , e ₂ , ..., e ₃₀ and standard deviations d ₁ , d ₂ , ..., d ₃₀ .

CD is a vector form of Color Descriptor (CD) among MPEG-7 standard descriptors. The configuration of the color descriptor is f _i , where f _i is the number of bins of the I-th bin of the 64 linear quantized bins.

Next, the user input unit 11 of the user interface module 10 inputs a query word or query image that the user desires to search for a web image using a web implemented user interface (S4).

In the above, when the user inputs a query, the image search method of the system is searched by a simple string comparison with the query field of the database 40.

However, when the user inputs the query image, the user selects the weight w to be assigned to the texture and color descriptor, and then uploads the query image selected by the user from the local host in the uploading unit 13 (S5).

The metadata extraction module 30 extracts the MPEG-7 texture and color descriptor 2 from the uploaded query image. (S6) The MPEG-7 texture and color descriptor 2 is the texture and color descriptor 1. As shown in Equation 1 and Equation 2 in the form of a vector.

After extracting both the MPEG-7 texture and color descriptor 1 and the texture and color descriptor 2 of the query image through the meta search engine module 20, the search result output module 50 may search for the selected image. The similarity between the query image and all images in the database 40 is measured, and the images are sequentially output from the image having the highest similarity (S7, S8).

4 is a flowchart illustrating a process of measuring similarity between images using texture and color descriptors.

Referring to FIG. 4, the metadata extraction module 30 extracts an MPEG-7 texture descriptor and a color descriptor from a query image. (S11) The similarity measurer 51 may be represented by Equations 3 and 4 below. Similarly, the similarity is measured using the geometric distance of the texture and color descriptors of the query image and the images in the database 40.

Here, the geometrical distances of the texture and color descriptors are the texture difference value and the color difference value using the k-th feature value of the database 40. (S12)

In Equation 3, d is the geometric distance between texture descriptors. In addition, alpha (k) is an average of k th feature values for the entire database 40 and is used in the normalization of feature values. Therefore, k has a value from 1 to 62.

In Equation 4, d is the geometric distance between the color descriptors. Here, k represents the k-th feature value and has a value from 1 to 64.

In Equation 5, the merged distance is obtained by multiplying the geometric distance of the color descriptor of Equation 4 by the weight w selected by the user and adding the geometric distance of the texture descriptor of Equation 3. In this way, each geometric distance between the texture descriptor and the color descriptor is weighted (S13), and the geometric distance of the texture and color descriptors obtained by the above Equations 3 and 4 measures the final similarity by the following Equation 5. (S14)

In the above, Equation 3, Equation 4, and Equation 5 are examples of the similarity measurement for explaining the present invention. The present invention is not limited to the geometric distance as described above. Several similarity measurement methods can be used, including the method.

5 is a diagram illustrating screens of a user interface and an image search result.

As shown in FIG. 5, (a) of FIG. 5 is a user interface screen. A user may input and search a query in a keyword field, or input an image of a local host in a query image field. You can also search by selecting weights.

5B is a search result screen based on a query image, and images are output in the order most similar to the query image located on the upper left of the screen. FIG. 5C is a search result screen of a query word, which are images obtained by a simple string comparison between a query word and a query field in a database.

Comparing the screens of (b) and (c) of FIG. 5, the output images of (b) and the central term query of the content based search using the texture and color descriptor which are the MPEG-7 standard descriptor are shown in (c). It can be seen that the output images show high similarity with each other.

Meanwhile, FIG. 5D is an arbitrary search result screen showing 16 images from among images stored in the database. Such an arbitrary search result screen is for displaying images stored in a database to the user in advance so that the user can select an image most similar to the image to be searched as a query image.

When the similarity measurement is completed in the similarity measuring unit 51, the data output unit 53 outputs images in order from the high similarity image, and determines whether the user is satisfied with the search result (S9).

If the user is satisfied with the search result, the video search process is terminated immediately. If the user is not satisfied with the search result, the user can set the list control to perform a more precise search using one of the images output as the image search result. Go back to step

For example, a list control is provided on the right side of the screens (b), (c), and (d) of FIG. 5 so that the user can select one of the search result images to use as a query image.

Alternatively, when the user is not satisfied with the search result, the user may directly search the image that is not stored in the database 40 by using the meta search engine.

The drawings and detailed description of the invention are merely exemplary of the invention, which are used for the purpose of illustrating the invention only and are not intended to limit the scope of the invention as defined in the appended claims or claims. Therefore, those skilled in the art will understand that various modifications and equivalent other embodiments are possible from this. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

The web-based image retrieval system and method according to the present invention utilizes the MPEG-7 descriptor, which is an international standard for content feature descriptors of multimedia data, for image retrieval. There is an effect that can perform an image search by the image.

In addition, the present invention can improve the accuracy of the search by using the texture and color descriptors among the MPEG-7 standard descriptors, enable user-centered search, and collect and store images on the web using a meta search engine. There is an effect that can be used to search for images on the web.

In addition, the present invention may perform a more precise search by using one of the images of the search result already output if the user is not satisfied with the search result for the query word or the query image input by the user, or the user may directly search the meta search engine. You can also randomly search for images that are not stored in the database, so you can continue to expand the database.

Claims (12)

A search engine for collecting data existing on the web, and a web-based image data retrieval system connected to a local host of a user who uses the search engine through an internet network, A user interface module for inputting a query word or a query image through a user interface implemented on a web; A meta search engine module for collecting and outputting images scattered on the web based on a query input by an administrator, and selecting a specific image from the collected images; A metadata extraction module for extracting texture and color descriptors from an image selected through the meta search engine module or a query image input from the user interface module; A database for storing image data selected by the meta search engine module and a texture and color descriptor extracted by the metadata extraction module; And A search result output module which measures the similarity between the image in the database and the query word or query image to be searched and outputs search result data in a predetermined order according to the measurement result Web-based image data retrieval system comprising a. The method of claim 1, The user interface module, Web-based image data retrieval system characterized in that for selecting the weight to be assigned to the texture and color descriptor when the query image input. The method of claim 2, The search result output module, And calculating a geometric distance between the texture and color descriptors of the images in the database and the texture and color descriptors of the query image and measuring the similarity by assigning a weight selected by the user to the geometric distance. The method of claim 1, The user interface module, The web-based image data retrieval system further comprises an upload unit for uploading a query image in the local host of the user when the query image is input. The method of claim 1, The search result output module, A similarity measurer for measuring similarity between texture and color descriptors of images stored in the database and texture and color descriptors of a query image by a user input; Data output unit for outputting the search results in order from the image with a high similarity according to the measurement result of the similarity measuring unit Web-based image data retrieval system comprising a. A search engine for collecting data existing on a web, and a web-based image data retrieval method connected to a local host of a user who uses the search engine through an internet network, A first step of collecting images necessary for retrieving images from the web and selecting a specific image from the collected images and storing the image in a database; A second step of extracting texture and color descriptors of the images selected in the first step and storing the descriptors and metadata in a database; A third step of inputting a query word or a query image to be searched through a user interface; A fourth step of extracting texture and color descriptors of the image input in the third step and measuring similarity with descriptors stored in the database; And A fifth step of outputting image data as search results in order according to the similarity measurement result of the fourth step; Web-based image data retrieval method comprising a. The method of claim 6, In the third step, when the user inputs a query, the web-based image data retrieval method may include comparing the central word of the images stored in the database with contents corresponding to the query and outputting images having similar contents. The method of claim 6, In the third step, when a user inputs a query image, A user selects an image on his local host as a query image, uploads it to the system, and specifies a weight to assign to a texture and color descriptor for image retrieval. The method of claim 6, The second or fourth step is to extract the texture and color descriptors, And converting the images collected in the database into a designated color display format and normalizing the sizes, and extracting textures and color descriptors through sampling. The method of claim 6 or 8, The fourth step is Calculating a geometric distance between a texture and color descriptor of images in the database and a texture and color descriptor of a query image to be searched by a user, and measuring similarity by assigning a weight selected by the user to the geometric distance Based image data retrieval method. The method of claim 6, In the fifth step, when the image data is output as a search result, the list control is performed so that the user can select at least one of the search result images and use the image as a query image. Search method. The method of claim 6, In the fifth step, when the user requests re-search after outputting the image data as a search result, the database is expanded so that the user can directly collect necessary images from the web and search for an image not stored in the database. Web-based image data retrieval method.