JP2010218479A - Image retrieval device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a retrieval device or a retrieval method for retrieving an image along retrieval intention of a user in image retrieval with the usage of a query image. <P>SOLUTION: A query image is provided for an image search engine executing image retrieval based on a multi-resolution model, and a first retrieval result is obtained. Then, designation of correct and/or incorrect about the image included in the first retrieval result is accepted. Next, weight on a degree of similarity for each of resolutions is calculated in accordance with the designation of correct and/or incorrect. After that, with the usage of the calculated weight, the degree of similarity between the query image and the retrieved image is re-calculated by the image search engine. Thus, a second retrieval result is obtained. Further, a document for output is generated wherein the images by the second retrieval result are arranged in descending order of the re-calculated degree of similarity. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an apparatus and method for searching for an image similar to a query image.

  The image search is roughly classified into a text-based search method and an image-based search method.

  In a text-based image search, the user specifies a text query and the system searches for images that have tags associated with the text query. According to this method, an image that is semantically related to a text query can be searched. However, in order to realize this method, it is necessary to tag the image. This tagging operation usually requires manual intervention. In addition, there is a possibility that the search accuracy is deteriorated due to an error in tag assignment or a mismatch between the input text query and the assigned tag.

  On the other hand, in the image-based search method, the feature amount of the query image specified by the user is usually compared with the feature amount of the image in the image DB, and a similar image is extracted as a similar image. This method has the advantage that image feature amounts can be extracted and feature amounts can be compared automatically by a computer.

  However, in the similar image search using the image feature amount, the calculation itself can be executed with high accuracy, but the searched image and the query image may not be semantically matched. For example, even images that are very similar as feature quantities, such as a yellow box and a yellow car, may be completely different semantically. In addition, since it is generally difficult to read a user's intention from a query image in an image-based search, there is a possibility that an image that does not match the user's intention is searched.

  As a technique for improving the accuracy of image-based search, a search technique using a multi-resolution image model (Multi Resolution Image Similarity Model) has been proposed (see, for example, Patent Document 1 below). The outline of this search method will be described below. In this method, image division is represented by the concept of “resolution”. For example, if the resolution is 2 × 1, it indicates that the image has been divided into upper and lower two sheets. The dividing line in this case extends in the horizontal direction. In addition, if the resolution is 1 × 2, it indicates that the image is divided into two left and right. In this case, the dividing line extends in the vertical direction. How to determine the number of divisions can be determined according to the search purpose and the search accuracy obtained experimentally. As the resolution, for example, 1 × 1, 2 × 1, 1 × 2, 2 × 2, 3 × 1, 1 × 3, 3 × 3, 4 × 4, 5 × 5, 6 × 6, etc. It can be determined as appropriate. Note that the resolution 1 × 1 is an undivided image.

  Then, the similarity between the query image and the target image (image in the DB) is calculated at each resolution. The similarity at a certain resolution can be obtained as the sum of the similarities in each region generated by division. The similarity in each region can be calculated in the same manner as a normal image similarity based on feature quantities such as image color, shape, texture, and the like. Thereafter, by calculating the sum of the similarities at each resolution, the overall similarity between the query image and the target image can be obtained.

  In image retrieval using a multi-resolution model, image retrieval can be performed with emphasis on local similarity in images.

  However, even in an image search using a multi-resolution model, there remains a possibility of searching for an image that is not semantically related to the query image. In addition, there is a possibility that the image desired by the user cannot be searched.

JP 2006-65399 A

  The present invention has been made in view of the above situation. A main object of the present invention is to provide a search device or a search method capable of searching for an image in accordance with a user's search intention in an image search using a query image.

  The present invention is configured as described in any of the following items.

(Item 1)
Obtaining a first search result including a plurality of images similar to the query image by providing a query image to an image search engine that performs an image search based on a multi-resolution model using a similarity for each of a plurality of resolutions In the image search device,
A feedback reception unit that receives designation of correct and / or incorrect answers for the image included in the first search result;
A weight calculation unit that calculates a weight for the similarity for each of the plurality of resolutions in accordance with designation of the correct answer and / or incorrect answer;
A search control unit that causes the image search engine to acquire a second search result by causing the image search engine to calculate a similarity between the query image and the searched image using the calculated weight;
An image search apparatus comprising: an output unit that generates an output document in which images in the second search result are arranged in order of the calculated similarity.

  Here, the process of obtaining the second search result can be performed by calculating the similarity between those images and the query image using the already searched images. Alternatively, this processing can be performed by re-searching the image in the DB and calculating the similarity with the query image using the newly searched image.

(Item 2)
The weight calculation unit, for a resolution having the highest degree of similarity between the query image and the correct image for which the correct answer is designated, has a relatively higher weight for the similarity than a weight for the similarity at other resolutions. The image search device according to item 1, wherein the image search device is configured to perform processing.

(Item 3)
The weight calculation unit is configured such that, for a resolution having the highest similarity between the query image and the incorrect image in which the incorrect solution is designated, the weight for the similarity is relatively higher than the weight for the similarity at another resolution. Item 3. The image search device according to Item 1 or 2, wherein the image search device is configured to perform a process of reducing the size of the image.

(Item 4)
The weight calculation unit is configured to calculate a weight for the similarity at the resolution according to the number of correct images for which the correct answer is specified and / or incorrect images for which the incorrect answer is specified. The image search device according to any one of 1 to 3.

(Item 5)
Further, the weight calculation unit is configured to weight the similarity at the resolution according to the similarity between the query image and the correct image with the correct answer specified and / or the incorrect image with the incorrect answer specified. Item 5. The image search device according to any one of Items 1 to 4, wherein the image search device is configured to calculate.

(Item 6)
Obtaining a first search result including a plurality of images similar to the query image by providing a query image to an image search engine that performs an image search based on a multi-resolution model using a similarity for each of a plurality of resolutions In the image search method,
Receiving a correct and / or incorrect answer designation for the image included in the first search result;
Calculating a weight for the similarity for each of the plurality of resolutions in response to designation of the correct answer and / or incorrect answer;
Causing the image search engine to obtain a second search result by causing the image search engine to calculate a similarity between the query image and the searched image using the calculated weight;
Generating an output document in which the images in the second search result are arranged in order of the calculated similarity.

(Item 7)
A computer program that causes a computer to execute the steps according to item 6.

  This computer program is stored in an appropriate recording medium (for example, an optical recording medium such as a CD-ROM or a DVD disk, a magnetic recording medium such as a hard disk or a flexible disk, or a magneto-optical recording medium such as an MO disk). Can be stored. This computer program can be transmitted via a communication line such as the Internet.

  ADVANTAGE OF THE INVENTION According to this invention, in the image search using a query image, it becomes possible to provide the search apparatus or search method which can search the image according to a user's search intention.

It is a block diagram which shows the outline of the search system using the image search device which concerns on 1st Embodiment of this invention. It is a flowchart which shows the outline of the image search method using the apparatus of 1st Embodiment. It is explanatory drawing for demonstrating the search method using the search device of 1st Embodiment. FIG. 1A shows an example of an input query image. FIG. 5B shows an example of the first search result. FIG. 3C shows an example of feedback about correct / incorrect answers. FIG. 4D shows an example of the second search result considering feedback. It is explanatory drawing which shows an example of the weight attached | subjected to the similarity for every resolution, when acquiring a 1st search result. It is explanatory drawing which shows an example of the weight attached | subjected to the similarity for every resolution, when acquiring a 2nd search result. It is a flowchart which shows the outline of the image search method in 2nd Embodiment of this invention.

(Configuration of the first embodiment)
A search system using the image search apparatus according to the first embodiment of the present invention will be described with reference to FIG. This search system includes a user terminal 1, an image search device 2, a search server 3, and the Internet 4 as main elements.

(User terminal)
The user terminal 1 includes a terminal main body 11, an input device 12, and a display device 13. The terminal body 11 includes a CPU (not shown) and performs calculation processing necessary for the operation of the user terminal 1. The input device 12 is for receiving a command from the user. As the input device 12, for example, a device such as a keyboard, a mouse, or a trackball can be used. The input device 12 according to the present embodiment can specify a query image by the user. The display device 13 presents information to the user. As the display device 13, for example, a display such as an LCD or a CRT can be used. The display device 13 of the present embodiment displays the feedback document generated by the image search device 2.

(Image search device)
The search device 2 includes a query reception unit 21, a feedback reception unit 22, a weight calculation unit 23, a search control unit 24, and an output unit 25.

  The query receiving unit 21 is a part that receives information on a query image input by the user to the user terminal 1. Further, the query receiving unit 21 is configured to acquire a first search result including a plurality of images similar to the query image by providing the search server 3 with the query image. The first search result is presented to the user via the user terminal 1. Thereby, the user can input a correct answer and / or incorrect answer designation for an image included in the first search result (described later).

  The feedback accepting unit 22 is configured to accept designation of correct and / or incorrect answers from the user for images included in the first search result. Specifically, the feedback receiving unit 22 is configured to receive from the user terminal 1 information for specifying correct and / or incorrect answers input using the user terminal 1.

  The weight calculation unit 23 is configured to calculate a weight for the similarity for each of a plurality of resolutions in accordance with designation of a correct answer and / or an incorrect answer from the user. Here, the search engine of the search server 3 calculates the similarity between the query image and the image in the DB using a multi-resolution model (described later). Therefore, the weight calculator 23 calculates a weight for the similarity for each resolution in the multi-resolution model.

  The search control unit 24 causes the image search engine of the search server 3 to calculate the similarity between the query image and the searched image using the weight calculated by the weight calculation unit 23. Thereby, the search control unit 24 causes the image search engine of the search server 3 to acquire the second search result.

  The output unit 25 is configured to generate an output document in which images in the second search result are arranged in descending order of similarity.

(Search server)
The search server 3 includes a search engine 31, an index DB 32, and an image DB 33.

  The search engine 31 performs image search based on the multi-resolution model. Since the multi-resolution model itself is conventionally known, detailed description thereof will be omitted.

  The index DB 32 is a DB that stores an index for speeding up the search for images stored in the image DB 33. The image DB 33 stores a large number of images as search targets. In the image DB 33, an ID is assigned to each image. In addition, information such as image feature amounts is recorded corresponding to each ID. Since the configuration of each DB may be the same as the conventional one, further detailed description is omitted.

(Operation of this embodiment)
Next, a search method using the image search apparatus of the present embodiment will be described with further reference to FIGS.

(Step SA-1 in FIG. 2)
First, the user designates a query image using the input device 12 of the user terminal 1 (see FIG. 3A). Information on the designated query image is sent to the query receiving unit 21 of the image search device 2 via the Internet 4.

  When receiving the query image information, the query receiving unit 21 of the image search device 2 sends the query image to the search engine 31 of the search server 3.

(Step SA-2 in FIG. 2)
Next, the search engine 31 of the search server 3 searches for similar images in the image DB based on the multi-resolution model. In the multi-resolution model, as described above, for example, 1 × 1, 2 × 1, 1 × 2, 2 × 2, 3 × 1, 1 × 3, 3 × 3, 4 × 4, 5 × 5, Similarities between the query image and the target image are obtained for a plurality of resolutions, such as 6 × 6. Then, the search engine 31 sums up the similarities for each resolution to obtain the overall similarity, and identifies a target image having a high similarity overall as a similar image. The fine resolution to be used can be appropriately set according to the search purpose. In the initial search, no weight is given to each resolution. In other words, at the time of the first search, it can be said that the weights at the respective resolutions are equal (see FIG. 4).

  Thereby, the search engine 31 of the search server 3 can acquire the search result of the similar image. The search result obtained in this way is referred to as a first search result in this specification. In the first search result, ranking is performed according to the degree of similarity between the query image and the target image. That is, the first search result specifies an image having a higher similarity ranking. Therefore, the first search result includes a plurality of images similar to the query image (see FIG. 3B).

(Step SA-3 in FIG. 2)
Next, the output unit 25 generates a document for accepting feedback using the first search result and sends it to the user terminal 1. The display device 13 of the user terminal 1 displays a feedback acceptance document. The user can specify a correct answer and / or an incorrect answer for the displayed image using the input device 12 while viewing the display device 13 (see FIG. 3C). In FIG. 3C, the symbol “◯” indicates a correct image, and the symbol “×” indicates an incorrect image. That is, in the figure, the first and fifth from the top are correct images, and the second from the top is an incorrect image. The other images were not designated as correct or incorrect. In this embodiment, both correct and incorrect answers are specified, but only one of them can be specified. Further, the number of images to be specified is not particularly limited.

(Step SA-4 in FIG. 2)
Next, the feedback receiving unit 22 receives information on the correct / incorrect answer specified as described above. Receiving information about the correct image and the incorrect image in this way is referred to as feedback in this specification.

(Step SA-5 in FIG. 2)
Next, the weight calculation unit 23 calculates the weight given to the similarity for each resolution based on the feedback about the correct / incorrect answer. An example of the weight calculation method will be described in detail below.

(Identification of correct best match resolution)
The weight calculation unit 23 identifies “the resolution with the highest similarity between the fed back correct image and the query image”. This resolution is referred to as “correct best match resolution” in this specification. Here, when there are a plurality of correct images, the correct best match resolution is specified for each correct image. A set of correct best match resolutions is referred to as a correct best match resolution set.

(Identification of incorrect answer best match resolution)
In the same manner as described above, the weight calculation unit 23 identifies “the resolution having the highest similarity between the fed back incorrect image and the query image”. This resolution is referred to as “incorrect answer best match resolution” in this specification. Here, when there are a plurality of incorrect answer images, an incorrect answer best match resolution is specified for each incorrect answer image. A set of incorrect answer best match resolutions is referred to as an incorrect answer best match resolution set.

(Weight adjustment)
For the resolutions included in the correct best match resolution set, the weight for the similarity is relatively higher than the similarities at other resolutions.

  For the resolutions included in the incorrect answer best match resolution set, the weight for the similarity is relatively low compared to the similarities at other resolutions.

  Moreover, it is preferable that the weight calculation unit 23 calculates the weight for the similarity for each resolution according to the number of correct images for which correct answers are specified and / or incorrect images for which incorrect answers are specified.

5 shows an example of the adjusted weights, shown by the code w ₁ to w _3. A specific calculation example of the weight adjustment will be described later.

(Step SA-6)
Next, the search control unit 24 causes the image search engine 31 to calculate the similarity between the query image and the searched image using the weight calculated as described above. Thereby, the search control unit 24 causes the image search engine 31 to acquire the second search result. In acquiring this second search result, the image search engine 31 can use the degree of similarity for each resolution determined in step SA-2. That is, the similarity between the query image and the target image can be newly obtained by multiplying the similarity for each resolution already obtained in the above-described step by the weight described above and summing the results. Using this new similarity, the order of items (that is, searched images) in the second search result can be determined. However, it is not necessary to use what has already been calculated as the similarity for each resolution, and it is possible to newly calculate the similarity for each resolution and weight it.

  The similarity between the query image and the target image obtained in this way takes into account feedback about correct / incorrect answers. Therefore, in the second search result, it is expected that the order of correct images and similar images can be increased, and the order of incorrect images and similar images can be decreased.

(Step SA-7 in FIG. 2)
Next, the output unit 25 generates an output document in which images in the second search result are arranged in the order of high similarity calculated using the above-described weighting. Further, the output unit 25 sends an output document to the user terminal 1 via the Internet 4. The user terminal 1 presents an output document to the user using the display device 13. Thereby, the user can see the search result newly ranked based on the correct / incorrect answer information.

  Therefore, according to the present embodiment, there is an advantage that in the image search using the query image, it is possible to search for an image according to the search intention of the user.

  Next, specific calculation examples 1 and 2 for the weight for each resolution will be described below.

(Calculation Example 1)
(Default weight for resolution)
In the following example, four resolutions (resolutions 1 to 4) are used. These resolutions are due to different numbers of divisions. Here, for example, 1 × 2 and 2 × 1 are different numbers of divisions. The number of divisions to be specifically adopted is not important in this example, and thus description thereof is omitted.

  The default weight for each resolution is:

Default resolution and weight = (Resolution 1: W [1] = 1.0, Resolution 2: W [2] = 1.0, Resolution 3: W [3] = 1.0, Resolution 4: W [4] = 1.0)

  That is, the weight is set to 1 for any resolution. W [i] represents a weight for the resolution i.

  Assume that the following ten images are acquired as a result of this search (that is, the first search result).

Search result = (Image 1, Image 2, Image 3, Image 4, Image 5, Image 6, Image 7, Image 8, Image 9, Image 10)

(Feedback for correct / incorrect answers)
Next, feedback about correct / incorrect answers is received from the user. In this example, the feedback result is as follows.

Feedback: correct image = {image 1, image 4, image 5, image 8}, incorrect image = {image 2, image 6}

(Generate best match resolution set)
Next, the correct best match resolution set described above is generated. In this example, it is assumed that:

Correct Best Match Resolution Set = {Resolution 1 (Image 1), Resolution 1 (Image 4), Resolution 4 (Image 5), Resolution 1 (Image 8)}

  That is, in the case of the images 1, 4 and 8 as the correct images, the similarity with the query image was the highest at the resolution 1. In the case of the image 5, the similarity with the query image was the highest at the resolution 4.

  Similarly, an incorrect answer best match resolution set is generated as follows.

  Incorrect answer best match resolution set = {resolution 1 (image 2), resolution 2 (image 6), resolution 2 (image 10)}

  That is, in the case of the image 2 as an incorrect image, the similarity with the query image was the highest at the resolution 1. Further, in the case of the images 6 and 10, at the resolution 2, the similarity with the query image was the highest.

(Adjust weight for resolution)
A method for calculating the weight (W [i]) for the resolution will be described. Prepare the symbols to be used as follows.
PK [i]: frequency of resolution i in correct answer best match resolution set,
NK [i]: frequency of resolution i in incorrect answer best match resolution set,
FW [i]: coefficient for reducing the weight according to the designated number of correct / incorrect answers,
ΔW [i]: difference with respect to initial weight,
W [i]: Calculated weight.

  Then, for resolution 1, the weight can be calculated as follows.

PK [1] (Frequency of resolution 1 in the correct best match resolution set) = 3
NK [1] (frequency of resolution 1 in incorrect answer best match resolution set) = 1
FW [1] = 1 / (PK [1] + NK [1]) = 1 / (3 + 1) = 1/4 = 0.25
ΔW [1] = (PK [1] −NK [1]) × FW [1] = (3-1) × 0.25 = 0.5
W [1] = W [1] + ΔW [1] = 1.0 + 0.5 = 1.5

  That is, the weight for resolution 1 is calculated as 1.5. In this example, it can be said that the weight is calculated according to the number of designated correct / incorrect images. Also, calculating PK [i] -NK [i] is equivalent to canceling the weight specified by the correct answer with the weight specified by the incorrect answer.

As in the case of resolution 1, the weight (W [2]) for resolution 2 can be calculated as follows.
PK [2] (Frequency of resolution 2 in the correct best match resolution set) = 0
NK [2] (frequency of resolution 2 in incorrect answer best match resolution set) = 2
FW [2] = 1 / (PK [2] + NK [2]) = 1 / (0 + 2) = 0.5
ΔW [2] = (PK [2] −NK [2]) × FW [2] = (0−2) × 0.5 = −1.0
・ W [2] = W [2] + ΔW [2] = 1.0 + (− 1.0) = 0.0

The weight (W [3]) for resolution 3 can be calculated as follows.
PK [3] (Frequency of resolution 3 in the correct best match resolution set) = 0
NK [3] (frequency of resolution 3 in incorrect answer best match resolution set) = 0
FW [3] = 1 / (PK [3] + NK [3]) = 1 / (0 + 0) = 0.0 (If the denominator is 0, it is regarded as 0)
ΔW [3] = (PK [3] −NK [3]) × FW [3] = (0−0) × 0.0 = 0.0
W [3] = W [3] + ΔW [3] = 1.0 + 0.0 = 1.0

The weight (W [4]) for the resolution 4 can be calculated as follows.
PK [4] (frequency of resolution 4 in correct best match resolution set) = 1
NK [4] (frequency of resolution 4 in incorrect answer best match resolution set) = 0
FW [4] = 1 / (PK [4] + NK [4]) = 1 / (1 + 0) = 1.0
ΔW [4] = (PK [4] −NK [4]) × FW [4] = (1−0) × 1.0 = 1.0
W [4] = W [4] + ΔW [4] = 1.0 + 1.0 = 2.0

(Weight for each resolution)
As a result of the adjustment, the weight for each resolution is as follows.
Resolution 1: W [1] = 1.5, Resolution 2: W [2] = 0.0, Resolution 3: W [3] = 1.0, Resolution 4: W [4] = 2.0.

  In such a calculation method, when the number of feedbacks is large, the value of FW [i] becomes small, so that the influence of feedback on the weight can be reduced. This is based on the assumption that the greater the number of feedbacks, the lower the certainty of feedback per unit. This method is considered effective when the criteria for correct and incorrect answers are ambiguous.

  In the above example, PK [i] is the frequency of resolution i in the correct best match resolution set, and NK [i] is the frequency of resolution i in the incorrect best match resolution set. However, the similarity between the correct / incorrect answer image and the query image in the correct / incorrect answer best match resolution set can be used instead of the frequency. Specifically, it can be calculated as follows.

As described above, the images corresponding to the resolution 1 of the correct best match resolution set are the image 1, the image 4, and the image 8. In this case, PK [i] at resolution 1 is
PK [1] = (similarity between query image and image 1 at resolution 1 + similarity between query image and image 4 at resolution 1 + similarity between query image and image 8 at resolution 1),
Can be calculated as Similar calculations can be made for other resolutions. This has the advantage that the weight value can be set finely. Also, this calculation is equivalent to performing weighting in consideration of the number of designated correct / incorrect images because the sum of the similarities is used.

(Calculation Example 2)
(Default weight for resolution)
In the calculation example 1 described above, FW [i] (a coefficient for reducing the amount of variation in weight according to the designated number of correct / incorrect answers) is used. On the other hand, in Calculation Example 2, the weight is calculated without using FW [i]. In the description of the calculation example 2, the description common to the calculation example 1 is simplified.

For Resolution 1 in Calculation Example 2, the weight can be calculated as follows.
PK [1] (Frequency of resolution 1 in the correct best match resolution set) = 3
NK [1] (frequency of resolution 1 in incorrect answer best match resolution set) = 1
ΔW [1] = PK [1] −NK [1] = 3−1 = 2.0
W [1] = W [1] + ΔW [1] = 1.0 + 2.0 = 3.0

That is, the weight for resolution 1 is calculated as 3.0. Similarly, the weight (W [2]) for resolution 2 can be calculated as follows.
PK [2] (Frequency of resolution 2 in the correct best match resolution set) = 0
NK [2] (frequency of resolution 2 in incorrect answer best match resolution set) = 2
ΔW [2] = PK [2] −NK [2] = 0−2 = −2.0
・ W [2] = W [2] + ΔW [2] = 1.0 + -2.0 = -1.0

The weight (W [3]) for resolution 3 can be calculated as follows.
PK [3] (Frequency of resolution 3 in the correct best match resolution set) = 0
NK [3] (frequency of resolution 3 in incorrect answer best match resolution set) = 0
ΔW [3] = PK [3] −NK [3] = 0−0 = 0.0
W [3] = W [3] + ΔW [3] = 1.0 + 0.0 = 1.0

The weight (W [4]) for the resolution 4 can be calculated as follows.
PK [4] (frequency of resolution 4 in correct best match resolution set) = 1
NK [4] (frequency of resolution 4 in incorrect answer best match resolution set) = 0
ΔW [4] = PK [4] −NK [4] = 1−0 = 1.0
W [4] = W [4] + ΔW [4] = 1.0 + 1.0 = 2.0

(Weight for each resolution)
As a result of the adjustment, the weight for each resolution is as follows.
Resolution 1: W [1] = 3.0, Resolution 2: W [2] = − 1.0, Resolution 3: W [3] = 1.0, Resolution 4: W [4] = 2.0.

  In such a calculation method, the greater the number of feedbacks, the more the weight tends to fluctuate. As a result, it is possible to obtain a search result that places more importance on the user's intention. This method is considered effective when the criteria for correct and incorrect answers are clear.

  In the calculation example 2, as in the calculation example 1, as the PK [i] and [i], instead of the frequency, the similarity between the correct / incorrect answer image and the query image in the correct / incorrect best match resolution set Can be used.

(Second Embodiment)
Hereinafter, a second embodiment of the present invention will be described with reference mainly to FIG. In the description of the second embodiment, the description of the configuration common to the first embodiment will be simplified.

(Steps SB-1 to SB-6 in FIG. 6)
Since steps SB-1 to SB-6 in the second embodiment may be substantially the same as steps SA-1 to SA-6 in the first embodiment, detailed description thereof is omitted. By these steps, the second search result can be acquired.

(Step SB-7 in FIG. 6)
Next, in the second embodiment, the output unit 25 generates a feedback document (not shown) using the second search result. The appearance of the feedback document may be the same as that generated in step SB-3, but the second search result is used as its contents.

(Step SB-8 in FIG. 6)
Next, the feedback receiving unit 22 waits for feedback from the user to the feedback document using the second search result. If there is feedback, the process returns to step SB-5, and the weight calculation unit 23 recalculates the weight according to the designation of correct / incorrect answer. The process here can be similarly performed by using the second search result (that is, updating the contents of the first search result) as the first search result in the first embodiment. Then, a new second search result can be generated using the updated first search result, and a feedback document can be generated again. As long as there is feedback, this loop can continue. Alternatively, when the number of feedbacks is limited and a predetermined number of loops are continued, it can be set to proceed to the next step SB-9.

(Step SB-9 in FIG. 6)
On the other hand, when there is no feedback in step SB-8, the output unit 25 generates an output document in the same manner as in step SA-7 of the first embodiment. Here, the output document includes the second search result updated by the feedback in step SB-8.

  In the second embodiment, since the feedback to the search result is repeatedly performed, it is possible to perform a search that places more importance on the user's intention.

  The operations of the above-described embodiments can be implemented by incorporating appropriate computer software into the computer.

  The contents of the present invention are not limited to the above embodiment. In the present invention, various modifications can be made to the specific configuration within the scope of the claims.

  For example, each component described above may exist as a functional block, and may not exist as independent hardware. As a mounting method, hardware or computer software may be used. Furthermore, one functional element in the present invention may be realized by a set of a plurality of functional elements, and a plurality of functional elements in the present invention may be realized by one functional element.

  Moreover, the functional element may be arrange | positioned in the position physically separated. In this case, the functional elements may be connected by a network. It is also possible to realize functions or configure functional elements by grid computing.

  In the case where the second search result is obtained in step SA-6 or SB-6 after calculating the weight for each resolution of the multiresolution model, step SA-2 is performed using the multiresolution model in which the weight is changed. The second search result can be obtained as a result of calculating the similarity with respect to the image of the first search result obtained in the above and updating the ranking of the first search result. In this case, for example, the search server is provided with a cache memory, and the first search result is temporarily stored in the cache memory. Recalculation can be performed. Accordingly, in this case, it is possible to obtain an image having a high degree of similarity by reflecting user feedback from images similar to the query image. In addition, since the similarity is recalculated for a predetermined number of images already obtained, the processing can be speeded up.

  In addition, the second search result can be obtained by calculating the similarity with respect to the image stored in the image DB 33 using the multi-resolution model with the weighting changed, and performing the search again. In this case, an image having a high degree of similarity in weighting for each resolution in consideration of user feedback can be obtained from the image DB. For example, there is a possibility that a new search result that is not obtained from the first search result in Step SA-6 or SB-6 is obtained as the second search result.

DESCRIPTION OF SYMBOLS 1 User terminal 11 Terminal main body 12 Input device 13 Display apparatus 2 Image search device 21 Query reception part 22 Feedback reception part 23 Weight calculation part 24 Search control part 25 Output part 3 Search server 31 Image search engine 32 Index DB
33 Image DB
4 Internet

Claims (7)

Obtaining a first search result including a plurality of images similar to the query image by providing a query image to an image search engine that performs an image search based on a multi-resolution model using a similarity for each of a plurality of resolutions In the image search device,
A feedback reception unit that receives designation of correct and / or incorrect answers for the image included in the first search result;
A weight calculation unit that calculates a weight for the similarity for each of the plurality of resolutions in accordance with designation of the correct answer and / or incorrect answer;
A search control unit that causes the image search engine to acquire a second search result by causing the image search engine to calculate a similarity between the query image and the searched image using the calculated weight;
An image search apparatus comprising: an output unit that generates an output document in which images in the second search result are arranged in order of the calculated similarity. The weight calculation unit, for a resolution having the highest degree of similarity between the query image and the correct image for which the correct answer is designated, has a relatively higher weight for the similarity than a weight for the similarity at other resolutions. The image search device according to claim 1, wherein the image search device is configured to perform processing. The weight calculation unit is configured such that, for a resolution having the highest similarity between the query image and the incorrect image in which the incorrect solution is designated, the weight for the similarity is relatively higher than the weight for the similarity at another resolution. The image search device according to claim 1, wherein the image search device is configured to perform a process of reducing the size to a minimum. The weight calculation unit is configured to calculate a weight for the similarity at the resolution according to the number of correct images for which the correct answer is specified and / or incorrect images for which the incorrect answer is specified. Item 4. The image search device according to any one of Items 1 to 3. Further, the weight calculation unit is configured to weight the similarity at the resolution according to the similarity between the query image and the correct image with the correct answer specified and / or the incorrect image with the incorrect answer specified. The image search device according to any one of claims 1 to 4, wherein the image search device is configured to calculate. Obtaining a first search result including a plurality of images similar to the query image by providing a query image to an image search engine that performs an image search based on a multi-resolution model using a similarity for each of a plurality of resolutions In the image search method,
Receiving a correct and / or incorrect answer designation for the image included in the first search result;
Calculating a weight for the similarity for each of the plurality of resolutions in response to designation of the correct answer and / or incorrect answer;
Causing the image search engine to obtain a second search result by causing the image search engine to calculate a similarity between the query image and the searched image using the calculated weight;
Generating an output document in which the images in the second search result are arranged in order of the calculated similarity.   A computer program for causing a computer to execute the steps according to claim 6.

Images