KR100955181B1 - Method and System for Searching Image - Google Patents

Abstract

The image retrieval method according to an embodiment of the present invention, which provides search results in consideration of the quality index of each image by calculating the quality index of the images, provides a quality value of each image. Calculating; Calculating a text similarity between a query and each of the images; And calculating a relevance of the image for the query using the quality index of each image and the text similarity of each image.

Image, quality, index, feature, search, rating, freshness, goodness of fit, similarity

Description

Image Search Method and Search System {Method and System for Searching Image}

The present invention relates to a search method, and more particularly, to an image search method.

Due to the development of the Internet and the increase of the Internet, various services are provided using the Internet, and a representative example thereof is a search service. When a search service inputs a word or a combination of words to be searched by a user, the search engine searches a search result document (eg, a web site or an article including a search query entered by the user) corresponding to the search engine. Means services provided to

Recently, in order to meet various search demands of users, search service providers have provided content such as images as search results as well as text search results. In this case, in providing an image as a search result, the existing image search system searches for images including a query input by a user, for example, an image in which the query input by the user is included in a file name or text related to the image. By modeling according to the importance of the image will be provided.

However, when providing an image as a search result, as described above, if the file name of the image includes a query entered by the user or a query entered by the user in a text file related to the image, the image is returned as the search result. Since the images provided in the search results may not be related to the query entered by the user or have poor quality, the reliability of the image search service may be deteriorated when these images are provided as the search results. There is a problem.

In addition, since the existing search system provides an image search result with an image similar to the query input by the user, the image is not selected by the users, that is, the user does not have high preference. May be included in the search results, or images that do not have high user preferences may be disposed at the top of the search results, thereby reducing the accuracy of the image search service.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problems, and provides an image retrieval method and system capable of providing a search result in consideration of the quality index of each image by calculating the quality index of the images. Shall be.

In addition, another object of the present invention is to provide an image retrieval method and system capable of providing a search result in consideration of the preference of each image when providing the image search result by calculating a user preference of each image.

An image retrieval method according to an aspect of the present invention for achieving the above object comprises the steps of: calculating a quality index (Quality Value) of each image; Calculating a text similarity between a query and each of the images; And calculating a relevance of the image for the query using the quality index of each image and the text similarity of each image.

In one embodiment, the image search method comprises the steps of: generating an image search result for the query using the goodness of fit of the image; And providing the image search result to a user.

At this time, in the text similarity calculation step of the image, the text similarity of the image is calculated using the importance of the words included in the query in at least one of the body, the title, and the link address of the image associated with the image. The importance is characterized in that it is calculated using a modified Poisson (2-Poisson) model.

The image quality index may be generated using image feature information including at least one of image basic information, color distribution information, gray distribution information, and texture information extracted from each image. .

The image retrieval method may further include calculating a preference of the respective images, and in the calculating of the fitness, the fitness may be calculated in consideration of the preference of each image.

According to an embodiment, the preference information of each image may include at least one of first preference information indicating a degree of overlap of the image itself and second preference information indicating a degree of overlap of a source of the image, wherein the first preference information includes: The second preference information is calculated using the number of overlaps of the image itself, and the second preference information is calculated using the number of sources including the image.

Meanwhile, the calculating of the goodness of fit may include determining whether at least one of the text similarity, the quality index, the number of overlaps of the image itself, and the number of sources including the image satisfies a predetermined condition. In this case, the fitness is calculated by considering at least one of the first and second preferences together.

The image retrieval method may further include calculating a recency of the image. In the relevance calculation step, the relevance may be calculated in consideration of the recency of the image.

In one embodiment, the latest degree of the image is calculated using a difference value between the creation date and the current date of the image and a threshold date.

The image retrieval method may further include calculating weights for the text similarity of the image and the quality index of the image. In the fitness calculation step, the fitness is the quality index and the weight of the image in which the weight is reflected. Is calculated using the text similarity of the reflected image.

An image retrieval system according to another aspect of the present invention for achieving the above object is a quality index calculation unit for calculating the quality index (Quality Value) of each image; A text similarity calculator configured to calculate a text similarity between the query and the respective images; And a fitness calculation unit for calculating a relevance of the image to the query using the quality index of each image and the text similarity of each image.

As described above, according to the present invention, by generating the image search results in consideration of the quality index of each image quality images are placed on the top of the search results so that users can browse the high-quality images first, thereby searching There is an effect that can improve the satisfaction of the service.

In addition, the present invention is to consider the user preferences of each image when generating the image search results so that the image having a high user preference is placed on the top of the search results, so that many users can view the image first to view the accuracy of the search There is an effect that can be improved.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic block diagram of an image retrieval system according to an embodiment of the present invention. The image retrieval system 100 is connected to the user terminal 104 via the Internet 102 to receive a query input from the user terminal 104, and the image search result corresponding to the received query is received by the user terminal ( 104). Hereinafter, for convenience of description, the term user terminal 104 will be used interchangeably with a user.

As shown, the image search system 100 calculates the image database 110, the quality index calculator 120, the preference calculator 130, the latestness calculator 140, the interface 150, and the text similarity. The unit 160, the fitness calculator 170, and the search result generator 180 may be included.

The image database 110 is a place where various images to be used for a search are stored. In this embodiment, an image refers to a concept including not only a document that is an image but also a document including an image together with text. In the following description, all these documents will be referred to as images for the convenience of description.

Meanwhile, each of the images stored in the image database 110 includes an image quality index calculated by the quality index calculator 120, an image preference calculated by the preference calculator 130, and a latestness calculator. The image latestness calculated by 140 is stored with each image, and in addition, the importance of words included in the image used for calculating text similarity is matched with each image and stored.

The quality index calculator 120 calculates a quality value of each image. Detailed configuration of the quality index calculation unit 120 will be described in more detail with reference to FIG. 2. As illustrated in FIG. 2, the quality index calculator 120 according to an embodiment of the present invention includes a feature information extractor 210, an abnormal image determiner 220, a weight reflector 230, and a quality index. The calculator 240 is included.

The feature information extractor 210 extracts feature information of a corresponding image from each image. The feature information extractor 210 decodes an image compressed in various formats by using an image decoding library. Raw data is generated in the form of (Red, Blue, Green), and feature information is extracted by applying an image processing technique to the generated RGB data.

In one embodiment, the feature information extractor 210 may extract image basic information, color distribution information, gray distribution information, and texture information as image feature information. To this end, the feature information extracting unit 210 is an image basic information extracting unit 212, color distribution information extracting unit 214, gray distribution information extracting unit 216, and texture information extraction, as shown in FIG. Section 218.

The basic image information extracting unit 212 extracts basic image information from each image. In the exemplary embodiment, the basic image information extracting unit 212 may include image size values, image widths, and the like as image basic information. At least one of a ratio of height and compressed information of an image may be extracted.

The image size value is a value representing the size of the image, and the image basic information extractor 212 may determine the size value of the image by comparing a value obtained by multiplying the width and the height of the corresponding image with a threshold value. Specifically, if the product of the width and height of the image is larger than the threshold, the size of the image is determined as 1, and if the product of the width and height of the image is less than or equal to the threshold, the product of the width and height of the image is multiplied. The ratio of the log value of the log and the log value of the threshold may be determined as the size value of the image. In one embodiment, the threshold for determining the size value of the image may be set to a maximum size value of the image, which may be 300 in width and 400 in height. A method of calculating an image size value is expressed by Equation 1 below.

The ratio of the width to the height of the image can be determined within a range of 1 or less, which means that a maximum value of 1 is set for an image of the same height and width.

The image compression information is conceived that the image may have better image quality than that which is compressed much. The image basic information extracting unit 212 calculates bit per pixel constituting the image. Compression information can be calculated. At this time, the number of bits per pixel can be calculated by dividing the byte size constituting the image by the product of the width and height of the image, which is expressed by Equation 2 below.

Next, the color distribution information extractor 214 extracts color distribution information from each image. In one embodiment, the color distribution information extractor 214 uses color histogram information and color as color distribution information. Color channel distribution information and geometric color distribution information may be extracted.

The color histogram means that color bins corresponding to each pixel are determined by using HSV (Hue, Saturation, Value) values of each pixel of the corresponding image, and the probability of occurrence of these color bins is represented graphically. . Where H means the color of the image, S means the brightness of the image, V means the saturation of the image.

The color distribution information extracting unit 214 is color histogram information. The number of color bins in which the probability of occurrence of the color bins included in the image is greater than or equal to the threshold, the maximum value of the probability of occurrence of all the color bins included in the image, Of the color bins of the reference color histogram having a ratio between the maximum value and the second largest value among the total color bins included, the probability of occurrence of each of the color bins constituting the image, and a uniform probability of uniformity. At least one of the difference value of the appearance probability value and the color / monochrome flag is extracted.

For example, as shown in FIG. 3A, when the threshold for the appearance probability value of the color bins is 0.02, among the 512 color bins, there are seven color bins whose probability of occurrence is 0.02 or more, and the maximum probability of the appearance probability is 0.241. It can be seen that the ratio of the maximum value of the probability (0.241) and the second largest value (0.232) is 1.04. Here, the ratio of the maximum value of the probability of occurrence to the second largest value may mean that the image quality is not good.

In addition, the difference between the appearance probability value of each of the color bins constituting the image and the appearance probability value of the color bins of the reference color histogram having a uniform appearance probability value is 0.1 as the probability value of the color bin is 0.1. All can be calculated using the same reference color histogram. At this time, the difference (D) between the appearance probability value of the color bin of the reference color histogram and the color bin of the color histogram of the corresponding image may be calculated using Equation 3 below.

Where I represents an image corresponding to the reference color histogram, I 'represents the image, f _j represents the probability of occurrence of the jth color bin of the reference color histogram, and f _j ' represents _j of the color histogram of the image. The probability of occurrence of the first color bean.

On the other hand, the color / black and white flag indicates whether the image is black and white / color, and whether or not the black and white / color of the image can be determined using the degree that the pixel value of the image is distributed in a specific color. For example, when the pixel values of the image are concentrated only on the black and white axes in the HSV color space, the image is determined as a black and white image.

At least one of an average HSV value of the whole and an average HSV value of the center region of the image. That is, the color distribution information extracting unit 214 constitutes an average of H values, an average of S values, and an average of V values, and a central region of the image, as shown in FIG. 3C. At least one of the average of the H value, the average of the S value, and the average of the V values of all the pixels is calculated as the color channel distribution information of the corresponding image.

On the other hand, the geometric color distribution information means a difference between each average HSV value of the image and the average HSV value of the center region of the image. That is, as shown in FIG. 3C, the color distribution information extracting unit 214 calculates a difference value between each average HSV value of the entire image and the average HSV value of the central region of the image as geometric color distribution information of the corresponding image. It is.

Here, a small difference between the calculated HSV value of the center region and the entire HSV value of the image, which may be characteristic of the image, may mean that the quality of the image is not good. have.

Next, the gray distribution information extractor 216 extracts gray distribution information from the corresponding image. In one embodiment, the gray distribution information extractor 216 extracts gray histogram information as gray distribution information. can do.

The gray histogram is a graph showing probability values of all gray bins constituting the image, and the gray distribution information extractor 216 is included in the image as shown in FIG. 3D as gray histogram information. The number of gray bins whose threshold probability is greater than or equal to the threshold, the maximum value of the probability of occurrence of all gray bins included in the image, and the second maximum value of the probability of occurrence of all the gray bins included in the image. At least one of the ratios of the large values can be extracted.

For example, in FIG. 3D, when the threshold for the appearance probability value of the gray bins is 0.02, among the total 128 gray bins, there are five color bins whose probability of occurrence is 0.02 or more, the maximum value of the probability of occurrence is 0.241, and the maximum value of the probability of appearance. It can be seen that the ratio of (0.241) to the second largest value (0.232) is 1.04.

Like the histogram information, the gray histogram information may mean that the image quality is not good because the ratio of the maximum value of the appearance probability value to the second largest value is large.

Next, the texture information extractor 218 extracts texture information of the image from the image, where texture information is defined as a statistical value for the spatial distribution of gray values constituting the image. In an embodiment, the texture information extractor 218 may calculate texture information of the image using a gray-level co-occurrence matrix (GLCM) technique.

The GLCM technique is a method for expressing a spatial relationship between two pixels. The distance, angle, and correlation between brightness values of neighboring pixels are averaged, contrasted, and correlated using distances and angles between neighboring pixels as shown in FIG. 3E. It is a technique of calculating basic statistics such as relations, and expressing the calculated values by assigning new brightness values to the central pixels in the kernel and expressing them as partial texture features of the image.

In one embodiment, the texture information extractor 218 corresponds to an energy value, an entropy value, a correlation value, an inertia value, and a homogeneity value using GLCM. Can be calculated with the texture information of the image.

Here, the energy value is a value representing the uniformity of the contrast value in the image. If the contrast value appears in a constant or repeated form, the image has a large energy value, otherwise the image has a low energy value. Will be taken.

The entropy value represents the degree of disorder or confusion of the image, that is, the randomness of the intensity distribution. If the difference in contrast between pixels in the image varies, the image has a large entropy value, and the difference in contrast varies. Otherwise, the image will have a small entropy value. In other words, it can be seen that the entropy value tends to be inversely proportional to the energy value.

The correlation value represents a linear dependence of contrast between two neighboring pixels in the corresponding image, and if a specific pattern exists in the distribution of the contrast values, it is determined that there is a correlation. For example, a correlation value of 0 indicates no correlation, and a correlation value of 1 indicates perfect correlation. That is, if a similar contrast pattern appears at a specific distance, the image has a high correlation value.

The inertia value represents the degree of inertia of the contrast within the image. The inertia value can be calculated using the difference between the contrasts of two adjacent pixels. The larger the difference between the two pixels, the greater the inertia of the image. It will have a value.

The homogeneity value indicates the homogeneity of the pixels in the image. The more similar contrast values within a certain distance, the greater the homogeneity of the image. The more contrast values within a certain distance, the smaller the homogeneity of the image. do.

In this case, the texture information extractor 118 may calculate the GLCM used to calculate the energy value, entropy value, correlation value, inertia value, and homogeneity value by using Equation 4 below.

Here, P (i, j; d, Θ) means a joint probability density function, i and j are the first coordinate (k, l) and the second coordinate (m) of the corresponding image, respectively. , n) means a gray value, and d means the distance between the first coordinate and the second coordinate, and is defined as in Equation 5, and Θ means the angle between the first coordinate and the second coordinate. It is defined as Equation 6.

In summary, the feature information extractor 210 uses an image basic information extractor 212, a color distribution information extractor 214, a gray distribution information extractor 216, and a texture information extractor 218. Image basic information, color distribution information, gray distribution information, and texture information are extracted from the corresponding image.

Next, the abnormal image determination unit 220 determines whether the corresponding image is an abnormal image, and the abnormal image is, for example, the image is composed only of a graph or a table, or the image is composed only of black or white. Or the same image as the width-to-height ratio of the image is below the threshold.

According to an embodiment, the abnormal image determiner 220 may determine the corresponding image as an abnormal image when some feature information among the feature information extracted by the feature information extractor 210 is equal to or less than a predetermined value.

For example, among the feature information extracted by the feature information extractor 110, the compressed information of the image, the number of color bins in which the appearance probability value of the color bins included in the image is greater than or equal to a threshold, and the total gray bins included in the image. The maximum value of the probability of appearance, the ratio of the maximum value and the second largest value among the probability values of all the color bins included in the image, the color / monochrome flag, and the gray whose probability of occurrence of the total gray bins in the image is above the threshold. Characteristic information such as the number of bins, the maximum value of the probability of occurrence of the total gray bins included in the image, and the ratio of the maximum value and the second largest value of the probability of occurrence of the total gray bins included in the image may be higher than the threshold. If it is small, the image can be determined as an abnormal image.

If it is determined by the abnormal image determination unit 220 that the image is an abnormal image, the quality index of the image is determined to be zero without undergoing a separate quality index calculation process.

The weight reflector 230 calculates a weight for each feature information extracted by the feature information extractor 210 and reflects the calculated weight for each feature information to each feature information. In one embodiment, the weight reflector 230 configures a training set using predetermined images, and then applies regression analysis techniques such as logistic regression to each training set. Star weights can be calculated.

In the above-described embodiment, the weight reflector 230 is described as an essential component of the quality index calculator 120. However, the weight reflector 230 is optional since the weight reflector is optional. It may be included as. In addition, in the above-described embodiment, the weight reflecting unit 230 calculates a weight for each feature information and reflects it for each feature information. However, in the modified embodiment, one weight is calculated for all feature information. This may be reflected for each feature information.

The quality index calculator 240 calculates the quality index of the image by using the feature information extracted by the feature information extractor 210. Here, the weights calculated for each feature information by the weight reflector 230 may be reflected in the extracted feature information.

In detail, the quality index calculator 240 may calculate the quality index of the corresponding image by summing two or more feature information of the extracted feature information. In this case, the weights calculated for each feature information may be reflected in each feature information.

In one embodiment, when the specific feature information is included within a predetermined range because the quality of the image may be very poor regardless of other feature information, the quality index calculation unit 240 of the extracted feature information If some feature information is included within a predetermined range, a predetermined penalty may be applied to the calculated quality index.

For example, the size value of the image is less than or equal to the threshold, the ratio of the width and height of the image is within the threshold range, the compression information of the image is less than or equal to the threshold, and the probability of occurrence of all color bins constituting the image is predetermined. The number of color bins above the value is less than or equal to the threshold, or the maximum of the probability values of all the gray bins included in the image is greater than or equal to the threshold, or second to the maximum value of the probability of the total gray bins included in the image. If the ratio of the large value is above the threshold or the average S value of the central region of the image is below the threshold, the penalty may be applied to the image.

On the other hand, if the quality index of the image becomes a negative value or the quality index is smaller than the reference value due to the application of the penalty, the quality index calculator 240 calculates the final quality index by compensating for the quality index to which the penalty is applied. Can be.

The quality index calculator 240 may match the calculated quality index with the corresponding image and store it in the image database 110.

Referring back to FIG. 1, the preference calculator 130 calculates a preference of each image. Here, the preference indicates a degree cited by many users, and in one embodiment, the preference indicates a second degree of overlap between first preference information indicating a degree of overlap of the image itself and a source including the image. May include preference information.

The first preference information may be calculated using the number sig_dup_cnt of which the image itself is duplicated. The first preference information may have a higher value as the number of images itself is duplicated.

Meanwhile, the second preference information may be calculated using the number of collections (col_dup_cnt) including an image, where the collection means items such as a blog, a bulletin board, a news, a web, a dictionary, and the like. The second preference information also has a higher value as the number of collections containing images increases.

The preference calculator 130 may match the preference of the image calculated by the above method with each image and store the same in the image database 110.

Next, the recency calculator 140 calculates recency of each image. In one embodiment, the freshness calculator 140 may calculate the freshness of each image by using a monotonic subtractive function having the difference between the date at which each image is generated and the current date and the threshold date as variables. . The latestness calculator 140 may match the calculated latestness with each image and store it in the image database 110.

The interface unit 150 receives a query input from the user terminal 104 and provides a search result generated by the search result generator 180 to be described later to the user through the user terminal 104.

The text similarity calculator 160 calculates a similarity of text between the query received through the interface 150 and the respective images. The text similarity calculator 160 may include a title of the text associated with each image. The text similarity of each image can be calculated by using the weights of the words included in at least one of the body, the text, and the link address for the image (hereinafter referred to as an 'image document'). As described above, the importance may be matched with the corresponding image and stored.

In one embodiment, the importance of the words included in each image document may be calculated using a modified Poisson model. Specifically, the importance of each word is determined by the fact that the word appears in the image document. The number may be calculated using the number of times, the number of image documents in which the corresponding word appears, the number of words included in each image document, and the like.

For example, if a certain image document contains the text, "This is a movie with a beautiful Jun Ji-hyun, it appears that Jun Ji-hyun is acting bizarrely." For each word, the importance of each word is determined using the number of times the word appears in the image document, the number of image documents in which the word appears, and the number of words in each image document. 100, movie: 50, bizarre: 30) and store it with the image.

The text similarity calculator 160 may calculate a text similarity between the query and each image by using the importance of the above-described words. In one embodiment, the text similarity calculator 160 may calculate the text similarity using Equation 7 below.

Equation 7 represents an equation for calculating text similarity between the query q and the image document d, where qtf _i denotes the number of occurrences of the word i in the query q, and weight (term _i , d) Denotes the importance of the word i in the image document d.

For example, if the query entered from the user is "pretty pretty Jun Ji-hyun" and the text "goodness" included in the query yields a text similarity between the image document, the importance of the word "pretty" in the image is 50. , The text similarity for the word " pretty " becomes " 2ⅹ50 = 100 " since the word " pretty " appears twice in the query.

Next, the goodness-of-fit calculation unit 170 is the quality index calculated by the quality index calculation unit 120, the preference calculated by the preference calculation unit 130, the latestness calculated by the latest degree calculation unit 140, And using the text similarity between the query and the image calculated by the text similarity calculating unit 160, the goodness of fit of each image for the input query.

In one embodiment, the goodness-of-fit calculation unit 170 may calculate the goodness of fit of each image by adding up the quality index, the preference, the latestness, and the text similarity of each image. In a modified embodiment, the goodness of fit of each image may be calculated by summing at least one of the quality index, preference, and freshness with the text similarity, rather than adding all of the quality index, preference, and freshness to the text similarity. will be.

Meanwhile, in calculating the fitness, the fitness calculator 170 may calculate the fitness by considering the case only when the predetermined condition is satisfied. In one embodiment, the fitness index is calculated in consideration of the first preference only when the quality index of the image, the text similarity, and the number of overlaps of the image itself are greater than or equal to the threshold, and the image quality index is the second preference. And the fitness may be calculated in consideration of the second preference only when the number of collections including the image is greater than or equal to the threshold.

In the above-described embodiment, the goodness-of-fit calculation unit 170 directly adds the quality index, the preference, the latestness, and the text similarity. However, in the modified embodiment, the weight is reflected after each weight is applied to each of these values. You can also sum the values. To this end, the image retrieval system 100 may further include a weight calculator 175 that calculates a weight for each of the quality index, the preference, the latestness, and the text similarity.

That is, if the weight calculator 175 calculates the weight for each quality index, preference, freshness, and text similarity and provides the result to the fitness calculator 170, the fitness calculator 170 reflects the weight and then the weight is calculated. By summing up all the results, you can calculate the goodness of fit of the image. In this case, the weight calculator 175 may calculate weights for each of the first preference information and the second preference information separately in calculating the weight for the preference.

In the above-described embodiment, the weight calculator 175 calculates the weights for all of the quality index, the preference, and the latestness. However, if neither the quality index, the preference, nor the latestness are used for the calculation of the fitness, The weight calculator 175 may calculate weights only for those used for calculating the goodness of fit.

Next, the search result generator 180 generates an image search result for the query input by the user using the goodness of fit of the image calculated by the fitness calculator 170, and generates the image search result into the interface unit. Provided to user terminal 104 through 150.

In one embodiment, the search result generator 180 may generate an image search result for the input query by sorting the images in the order of fitness calculated by the fitness calculator 170. In this case, in generating the image search results, those having a goodness of fit or less than the reference value may not be included in the search results.

Hereinafter, an image search method according to an embodiment of the present invention will be described in detail with reference to FIG. 4.

First, when a predetermined query is received through the user terminal (S400), a text similarity between the received query and each image is calculated (S410). In one embodiment, the text similarity of each image may be determined by using the importance of words included in at least one of a title, a text, and a link address of the image associated with each image, as described in Equation 7 above. In this case, the importance of the words included in each image may be calculated in advance and matched with the corresponding image and stored.

Here, the importance of the words included in each image document can be calculated using the modified Poisson (2-Poisson) model. Specifically, the importance of each word is the number of times the word appeared in the image document, the word It can be calculated using the number of the image document appeared, the number of words contained in each image document, and the like.

Next, a goodness of fit of each image for the corresponding query is calculated by summing at least one of the quality index, the preference, and the latestness of each image with the text similarity (S420). That is, the fitness of each image for the corresponding query is calculated by summing all of the quality index, the preference, and the latestness with the text similarity, or summing at least one of the quality index, the preference, and the latestness with the text similarity.

In one embodiment, in calculating the goodness of fit, the goodness of fit may be calculated in consideration of this only when the predetermined condition is satisfied. For example, in the case of the first preference, the fitness may be calculated in consideration of the first preference only when the quality index of the image, the text similarity, and the number of overlaps of the image itself are greater than or equal to the threshold, and in the case of the second preference, the image quality index and the image are included. Only when the number of collected collections is greater than or equal to the threshold, the fitness may be calculated in consideration of the second preference.

In the above-described embodiment, the quality index, the preference, the latestness, and the text similarity are described as directly summing. However, in the modified embodiment, weights are calculated for each of the quality index, the preference, the latestness, and the text similarity. Then, the fitness may be calculated by summing at least one of the weighted quality index, the preference, and the latestness and the weighted text similarity.

Meanwhile, the quality index, the preference, and the latestness of each image may be calculated in advance and stored for each image. Hereinafter, a method of calculating the image quality index, the preference, and the latestness will be described in detail.

First, the quality index of an image represents the quality of each image, and may be calculated using feature information extracted from each image. Looking at the image quality index calculation method in detail with reference to Figure 5, first, feature information is extracted from each image (S500). In one embodiment, at least one of basic image information, color distribution information, gray distribution information, and texture information may be extracted as feature information of the corresponding image.

Here, the basic image information includes at least one of a size value of the image, a ratio of the width and height of the image, and compression information of the image, and the color distribution information includes color histogram information, color channel information, and geometric color distribution information. The gray distribution information includes at least one gray histogram of the corresponding image. The color histogram information, the color channel information, the geometric color distribution information, and the gray histogram information may include various pieces of information, which are described in detail in the above description of the image quality index calculation system. Let's do it.

Meanwhile, the texture information refers to a statistical value of a spatial distribution of gray values constituting the image, and may be calculated using GLCM in one embodiment. Specifically, the present invention can calculate the energy value, entropy value, correlation value, inertia value, and homogeneity value from the GLCM value with such texture information.

Next, it is determined whether the corresponding image is an abnormal image using the extracted feature information (S510). Here, the abnormal image refers to an image such as when the image is composed only of a graph or table, the image is composed only of black or white, or the ratio of the width and height of the image is less than a threshold. Types of feature information used to determine whether the corresponding image is an abnormal image have been described in detail in the above description of the abnormal image determination unit, and thus a detailed description thereof will be omitted.

As a result, if it is determined that the image is an abnormal image, the final quality index of the image is determined to be 0 (S580). If it is determined that the image is not an abnormal image, the weight calculated for each feature information is assigned to each feature information. Reflect (S520). In one embodiment, the weight for each feature information may be calculated by constructing a training set using predetermined images and then applying logistic regression to the training set.

In the present embodiment, the weight reflecting step is described as an essential process, but the weight reflecting process may be optionally included.

Next, the quality index of the corresponding image is calculated by summing two or more feature information among the feature information reflecting the weight (S530).

Thereafter, it is determined whether the specific feature information among the feature information extracted in step S500 is included within a predetermined range (S540), and when included within the predetermined range, a predetermined penalty is applied to the quality index calculated in step S530. Here, the reason why the penalty is applied to the quality index is to reflect the quality of the image because the quality of the image may be very poor regardless of other feature information when the specific feature information is included within a predetermined range. Therefore, this penalty application process may be optionally included. Types of feature information used for determining a penalty application are described in detail in the above description of the quality index calculation unit, and thus a detailed description thereof will be omitted.

On the other hand, if the specific feature information is not included within the predetermined range as a result of the determination in step S540 determines the quality index calculated in step S530 as the final quality index of the image (S580).

Subsequently, it is determined whether the quality index to which the penalty is applied is negative or less than the reference value (S560). If it is determined that the quality index is negative or less than the reference value, the quality index to which the penalty is applied is compensated (S570), and the compensated quality index is applied to the corresponding image. Determine the final quality index for (S580). Since the above-described penalty application process is optional, the quality index compensation process may also be optionally included.

On the other hand, if it is determined in step S560 that the quality index to which the penalty is applied is not negative or less than the reference value, the quality index to which the penalty is applied is determined as the final quality index of the image (S580).

Secondly, the preference of an image indicates the degree to which each image is cited by the users, and in one embodiment, the preference is the overlap of the first preference information indicating the degree of overlap of the image itself and the source in which the image is included. It may include second preference information indicating the degree.

Here, the first preference information may be calculated using the number sig_dup_cnt of which the image itself is duplicated. The first preference information has a higher value as the number of images itself is duplicated. In addition, the second preference information may be calculated by using the number of collections (col_dup_cnt) including an image. Here, the collection means items such as a blog, a bulletin board, a news, a web, a dictionary, and the like. The second preference information also has a higher value as the number of collections containing images increases.

Third, the freshness of an image indicates how up-to-date each image is, and the freshness of each image is the monotonic function with the difference between the date when each image was created and the current date and the threshold date. It can be calculated using.

Referring to FIG. 4 again, an image search result for the received query is generated using the goodness of fit calculated in step S420 (S430). In one embodiment, the image search results may be generated by arranging the images in the order of goodness of fit, wherein images having a goodness of fit below the reference value may not be included in the search results.

Finally, the generated image search result is provided to the user through the user terminal (S440).

The above-described image retrieval method may also be implemented in the form of a program that can be executed using various computer means. In this case, the program for performing the image retrieval method may be a hard disk, a CD-ROM, a DVD, a ROM, or a RAM. Or a computer-readable recording medium such as a flash memory.

Those skilled in the art to which the present invention pertains will understand that the present invention can be implemented in other specific forms without changing the technical spirit or essential features.

Therefore, it is to be understood that the embodiments described above are exemplary in all respects and not restrictive. The scope of the present invention is shown by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

1 is a schematic block diagram of an image retrieval system according to an embodiment of the present invention;

FIG. 2 is a block diagram illustrating an example of the quality index calculator shown in FIG. 1.

3A to 3E are views illustrating a method of extracting feature information from an image by a quality index calculator.

4 is a flowchart showing an image retrieval method according to an embodiment of the present invention;

5 is a flowchart showing a method for calculating a quality index according to an embodiment of the present invention.

<Description of the symbols for the main parts of the drawings>

100: image search system 110: image database

120: quality index calculator 130: preference calculator

140: latest calculation unit 150: interface unit

160: text similarity calculator 170: goodness of fit calculator

175: weight calculator 180: search result generator

Claims (24)

Calculating a quality value of each image; Calculating a text similarity between a query and each of the images; And Calculating a relevance of the image for the query using the quality index of each image and the text similarity of each image; Generating an image search result for the query using the goodness of fit of the image; And Providing the image search results to a user Including, Calculating a preference of each image; In the fitness calculation step, the fitness is calculated in consideration of the preference of each image, And the preference information of each image comprises at least one of first preference information indicating a degree of overlap of the image itself and second preference information indicating a degree of overlap of a source of the image. delete The method of claim 1, wherein the calculating of the text similarity of the image comprises: The text similarity of the image is calculated by using the importance of the words included in the query in at least one of the body, the title, and the link address of the image associated with the image. The method of claim 3, The importance is calculated by using a modified Poisson (2-Poisson) model. The method of claim 1, The image quality index is generated using image feature information including at least one of image basic information, color distribution information, gray distribution information, and texture information extracted from each image. How to search for images. delete delete The method of claim 1, And the first preference information is calculated using the number of overlaps of the image itself, and the second preference information is calculated using the number of sources including the image. The method of claim 8, wherein the calculating of the goodness of fit, Determining whether at least one of the text similarity, the quality index, the number of duplicates of the image itself, and the number of sources including the image satisfies a predetermined condition; If it satisfies the image retrieval method characterized in that calculating the goodness of fit in consideration of the first and second preferences. The method of claim 1, Calculating a recency of the image; In the fitness calculation step, the image retrieval method comprising calculating the goodness of fit in consideration of the latest degree of the image. The method of claim 10, The latest degree of the image is calculated using a difference between the date of creation of the image and the current date and the threshold date. The method of claim 1, Calculating weights for the text similarity of the image and the quality index of the image; In the goodness of fit calculation step, the goodness of fit is calculated using the quality index of the image reflecting the weight and the text similarity of the image reflecting the weight. A computer-readable recording medium having recorded thereon a program for performing the method according to any one of claims 1 or 3 to 5 or 8 to 12. A quality index calculator for calculating a quality index of each image; A text similarity calculator for calculating a text similarity between a query and each of the images; A fitness calculation unit for calculating a relevance of the image to the query using the quality index of each image and the text similarity of each image; An interface unit for receiving the query and providing an image search result corresponding to the query; And Search result generation unit for generating an image search results for the query using the goodness of fit of the image Including, Further comprising a preference calculator for calculating the preference (Popularity) of each image, The fitness calculator calculates the fitness by considering the preferences of the images together. And the preference calculator calculates at least one of first preference information indicating a degree of overlap of the image itself and second preference information indicating a degree of source overlap of the image as preference information of the image. delete The method of claim 14, The text similarity calculating unit calculates the text similarity using the importance of a word included in the query in at least one of a body, a title, and a link address of the image associated with the image. . The method of claim 14, The quality index calculator comprises: a feature information extraction unit for extracting image basic information, color distribution information, gray distribution information, and texture information from the respective images as image feature information; And, And a quality index calculator configured to generate the image quality index by using the extracted image feature information. delete delete The method of claim 14, And the preference calculator calculates the first preference information by using the number of overlapping images, and calculates the second preference information by using the number of sources including the image. The method of claim 20, The goodness-of-fit calculation unit determines whether at least one of the text similarity, the quality index, the number of overlapping of the image itself, and the number of sources including the image satisfies a predetermined condition. And calculating the goodness of fit by considering the second preference. The method of claim 14, The apparatus may further include a recency calculator for calculating a recency of each image. The fitness calculation unit calculates the fitness by considering the latest image of each image together. The method of claim 22, And the latestness calculator calculates the latestness of each image by using a difference value between a creation date of the image, a current date, and a threshold date. The method of claim 14, And a weight calculator configured to calculate weights for the text similarity of each image and the quality index of the image. And the fitness calculation unit calculates the fitness using the quality index of the image in which the weight is reflected and the text similarity of the image in which the weight is reflected.

Images