JP2012079186A - Image retrieval device, image retrieval method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize a similar image retrieval with high accuracy, paying attention to characteristic region that a query image has.SOLUTION: A region including partial images in a retrieval target image similar to a partial image in a query image is cut out and a similarity degree between this region and the query image is calculated. Since a rate between the number of partial images included in the region and the number of partial images similar to the partial image extracted from the query image among the number of partial images in the region is weighted, the similarity degree is calculated through paying attention to the partial images similar to the query image.

Description

  The present invention relates to an image search apparatus for searching for an image similar to an image serving as a search key.

  By inputting an image as a search key and comparing image feature amounts (represented by quantifying image features such as color scheme, texture, and shape), an image that is a search key (hereinafter referred to as a “query image”) A technique for searching for an image similar to is known. When the user inputs a query image, the feature amount is extracted from the query image, and the similarity with the feature amount of the search target image is calculated to search for a similar image (for example, Patent Document 1).

  Since the feature amount extracted from one image indicates the feature of the entire image, it is effective when searching for images that are generally similar. On the other hand, when searching for an image having similar images (hereinafter referred to as “partial images”) by partial areas constituting the image, it is assumed that one image is composed of a plurality of areas. Thus, by representing the feature of the image by the feature amount for each partial image, it is possible to perform a similar image search that places importance on the partial image.

  However, if the image is divided into partial images and the feature values calculated from the partial images are simply compared, for example, if two images are divided into 100, the feature values are compared in 100 × 100 patterns. Since the similarity is calculated, the calculation amount is enormous.

  Therefore, a technique called a partial image visual keyword has been devised, which can calculate the degree of similarity while suppressing the amount of calculation using partial images. With visual keywords, a single image is considered to be composed of a plurality of partial images, and partial images are extracted from the images. And a feature vector is generated based on the number of clusters to which each partial image belongs.

  As described above, by using the visual keyword, it is possible to perform an image search with high accuracy not by the feature amount extracted from the entire image but by the feature amount obtained by capturing the image as a fine area.

JP 2001-52175 A

  By the way, the image input by the user as a query includes a characteristic region including the user's search intention, and it is desirable to determine the similarity by paying attention to this “characteristic region”. For example, when a person is included in the query image, it is desirable to determine the similarity by paying attention to the characteristics of the person.

  However, with the conventional visual keyword and the technique of Patent Document 1, since the similarity is calculated by capturing the entire image uniformly, a search specialized for the characteristic region of the image cannot be performed.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to realize a high-accuracy similar image search focused on a characteristic region of a query image.

  To achieve the above object, according to a first aspect of the present invention, there is provided an image search apparatus that calculates a similarity with a query image and searches an image with a high similarity from the search target images. A first extracting means for extracting a partial image; a second extracting means for extracting a plurality of partial images from the search target image; and a partial image in the search target image similar to the partial image extracted from the query image. When selecting the area including the selected partial image from the search target image, and calculating the similarity between the query image and the image in the extracted area, Similarity that gives as a weight the ratio between the number of partial images included in the clipped region and the number of partial images similar to the partial image extracted from the query image among the partial images in the region Calculation It is characterized in that it comprises a stage, a.

  According to the first aspect, a region including a partial image in the search target image that is similar to the partial image in the query image is cut out, and the similarity is calculated between the region and the query image. In addition, the similarity is weighted by a ratio between the number of partial images included in the region and the number of partial images similar to the partial image extracted from the query image among the partial images in the region. Then, the degree of similarity obtained by paying attention to the partial image similar to the query image is calculated. Therefore, it is possible to realize a high-accuracy similar image search that pays attention to a characteristic region of the query image.

  In the second invention, the similarity calculation means includes a number of the partial images included in the query image and a portion similar to the partial image in the search target image among the partial images in the query image. A ratio with the number of images is further given as a weight of the similarity.

  According to the second invention, the ratio between the number of partial images included in the query image and the number of partial images similar to the partial images in the search target image among the partial images in the query image is the degree of similarity Therefore, it is possible to calculate the degree of similarity by paying attention to a region similar to the search target image in the query image. Therefore, it is possible to realize a high-accuracy similar image search that pays attention to a characteristic region of the query image.

  In the third aspect of the present invention, the region cutout unit is located outside the set of partial images when selecting a partial image in the search target image similar to the partial image extracted from the query image. The partial image to be excluded is excluded from selection, and a region including the partial image after the exclusion is cut out.

  According to the third aspect of the present invention, since the partial image in the search target image similar to the partial image extracted from the query image is cut out from the outside by removing the partial image, the region whose similarity is to be calculated Can be narrowed down to a characteristic area. Therefore, it is possible to realize a high-accuracy similar image search that pays attention to a characteristic region of the query image.

  ADVANTAGE OF THE INVENTION According to this invention, the highly accurate similar image search which observed the characteristic area | region which a query image has can be implement | achieved.

The block diagram which shows the function structure of the image search device which concerns on this invention. The flowchart of a feature vector generation process. The figure which shows the mode of the extraction of the area image from image data, and the mapping to a visual keyword. The flowchart of the process of cutting out a comparison area and calculating similarity. The 1st conceptual diagram for demonstrating extraction of a comparison area | region, and calculation of a similarity. The 2nd conceptual diagram for demonstrating extraction of a comparison area | region, and calculation of a similarity. The 3rd conceptual diagram for demonstrating extraction of a comparison area | region, and calculation of a similarity. The conceptual diagram which shows the other example of extraction of a comparison area.

[Configuration of image search device]
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a functional block diagram of an image search apparatus 1 to which the present invention is applied. The image search device 1 is connected to the Internet connected via a communication network, and can collect image data from the web via the Internet. The collected data is accumulated in a database (DB) to create a search target image.

  The image search apparatus 1 receives a query image transmitted from a client terminal such as a personal computer or a mobile terminal connected via a communication network as a search request. Then, a similar image search is performed according to the search request, and search results ranked in the order of similarity are returned to the client terminal.

  The image search apparatus 1 according to the present embodiment indexes images using a visual keyword technique. An image search using visual keywords represents an image as a set of a plurality of image areas, and an image index (feature) based on a feature amount obtained from image areas (hereinafter referred to as “partial images” as appropriate) constituting each image. Vector), which can be said to be an application of text search technology for obtaining feature values of sentences from keywords in text.

  For this reason, in image search using visual keywords, text search technology (transposition index, vector space model, word appearance frequency, etc.) is applied to image region search by treating the image region in the image as a keyword. Scale and high speed can be realized.

As reference technical literature on image search using visual keywords,
Sivic and Zisserman: “Efficient visual search for objects in videos”, Proceedings of the IEEE, Vol.96, No.4, pp.548-566, Apr 2008.
・ Yang and Hauptmann: “A text categorization approach to video scene classification using keypoint features”, Carnegie Mellon University Technical Report, pp. 25, Oct 2006.
・ Jiang and Ngo: “Bag-of-visual-words expansion using visual relatedness for video indexing”, Proc. 31 ^st ACM SIGIR Conf., Pp.769-770, Jul 2008.
・ Jiang, Ngo, and Yang: “Towards optimal bag-of-features for object categorization and semantic video retrieval”, Proc. 6th ACM CIVR Conf., Pp.494-501, Jul. 2007.
・ Yang, Jiang, Hauptmann, and Ngo: “Evaluating bag-of-visual-words representations in scene classification”, Proc. 15 ^th ACM MM Conf., Workshop on MMIR, pp.197-206, Sep. 2007.
Etc.

  In addition, by expressing a single image as a set of a plurality of partial images, unlike general similar image search, search using an image obtained by cutting a part of an image in an arbitrary size or position as a query image is possible. It becomes possible. Therefore, the user can express the query more directly and accurately by an operation such as designating a part as shown by a broken line in the image G1 in FIG. 5 in order to obtain a desired search result.

  As shown in FIG. 1, the image search device 1 includes a query image reception unit 10, a feature vector generation unit 20, a comparison region extraction unit 30, a similarity calculation unit 40, a search result output unit 50, a visual keyword generation unit 60, A visual keyword DB 65, an indexing unit 70, an index DB 75, an area management DB 80, and a search target image DB 90 are provided.

  These functional units are configured by so-called computers, and are associated with a CPU (Central Processing Unit) as an arithmetic / control device, a RAM (Random Access Memory) and a ROM (Read Only Memory) as a storage medium, a communication interface, and the like. It is realized with.

  The query image receiving unit 10 receives and receives a query image that is a search key for similar image search transmitted from a client terminal. This query image includes an image stored in the search target image DB 90, an image cut out by an operation for designating a partial area of the image data, and a newly received image. The query image may be a single image or a combination of a plurality of images.

  The feature vector generation unit 20 extracts a partial image from the query image, performs a feature vector generation process (see FIG. 2) that generates a feature vector based on the feature amount of the partial image, and generates a feature vector from the query image. To do. The feature vector generation process will be described later.

  The comparison area cutout unit 30 performs a process of matching the partial image extracted from the query image with the partial image extracted from the search target image, and cutting out a region including the partial image in the matched search target image as a comparison area. Do. For matching of partial images, that is, selecting similar partial images, feature vectors of the images are used.

  The similarity calculation unit 40 calculates the similarity between the feature vector for each search target image stored in the index DB 75 and the feature vector generated from the query image. For calculating the similarity, a known technique such as a cosine distance or a Bhattacharyya distance is used. The details of extracting the comparison area and calculating the similarity will be described later.

  The search result output unit 50 generates data that ranks the images to be searched based on the similarity calculated by the similarity calculation unit 40. The data output by the search result output unit 50 is, for example, data obtained by sorting the image IDs of search target images based on the similarity. An address (URL) for accessing the search target image DB 90 may be added to the image ID.

  When generating the feature vector of the image data, the visual keyword generation unit 60 generates a classification (cluster) for mapping the partial images in the image. The visual keyword generation unit 60 extracts a plurality of partial images from images used for image search and image data prepared in advance for learning, and clusters them based on the feature amounts of the partial images. As a standard method of clustering, k-means, Hierarchical Agglomerative Clustering (HAC) or the like is used.

  The feature vector generation unit 20 generates a feature vector by mapping (classifying) the partial image detected from the image to a cluster formed by the clustering of the visual keyword generation unit 60. This cluster is referred to as a “visual keyword” as a feature amount space for expressing an image as a collection of visual keywords.

  The visual keyword DB 65 includes a visual keyword ID (VKID) for identifying a cluster formed by clustering by the visual keyword generation unit 60, and a center coordinate that is a coordinate of a center point in the feature amount space (multidimensional space) of the cluster. , A database that stores the radius indicating the cluster range in association with each other.

  The center coordinate is a value indicating an average value of feature amounts of images belonging to each cluster, and is represented by multidimensional coordinates on the feature amount space. The radius is obtained, for example, by the distance from the image farthest from the center coordinate among the images belonging to the cluster.

  The indexing unit 70 generates a feature vector for the image data stored in the search target image DB 90 based on the feature vector generation process of FIG. 2, and associates the generated feature vector with the index DB 75 as an index of the image data. Remember.

  Further, the indexing unit 70 assigns a region ID to the partial image detected from the image data, and stores the VKID of the visual keyword mapping the partial image in the region management DB 80 in association with the image ID and the region ID. The area ID may be an XY coordinate in the image, or may be a row number / column number when the area is divided.

  The index DB 75 is a database that stores an image ID of image data stored in the search target image DB 90 and a feature vector (frequency of appearance of partial images for each visual keyword) generated from the image data in association with each other.

  The area management DB 80 is a database that manages the correspondence between visual keywords that map partial images in a search target image, and corresponds to the image ID, area ID, and VKID of the search target image as shown in FIG. Add and remember.

  The search target image DB 90 is a database for accumulating and storing image data collected from the Internet as a search target for similar images (referred to as “search target image”). As shown in FIG. 1, an image ID, image data, Are stored in association with each other. The image ID is identification information for uniquely identifying each image data, and is assigned when a keyword and image data are stored.

[Feature vector generation processing]
Here, the feature vector generation processing will be described with reference to the flowchart of FIG. 2 and the conceptual diagram of FIG. The feature vector generation process is performed on the query image by the feature vector generation unit 20 and the search target image by the indexing unit 70. In the following description, the case of the feature vector generation unit 20 will be described.

  First, a plurality of partial images are detected from the query image (step S11). As a method for detecting the partial image, there are a method for detecting a characteristic region (characteristic region) in the image and a method for detecting the partial image by dividing the image into predetermined regions.

As a technique for detecting feature regions,
・ Harris-affine
・ Hessian-affine
・ Maximally stable extremal regions (MSER)
・ Difference of Gaussians (DoG)
・ Laplacian of Gaussian (LoG)
・ Determinant of Hessian (DoH)
Etc.

  The feature region detection technology is published in “Local Invariant Feature Detectors: A Survey” (Foundations and Trends in Computer Graphics and Vision, Vol.3, No.3, pp.177-280, 2007.). Any known technique can be adopted as appropriate.

  In addition, as a method of detecting an image by dividing it into predetermined areas, for example, the image is divided into predetermined M × N blocks, or divided so that the size of the divided blocks becomes predetermined m × n pixels. There is a technique to do. For example, when an image is divided into 10 × 10 blocks, if the size of the image is 640 × 480 pixels, the size of one block is 64 × 48 pixels.

  3 shows an example in which an image is divided into predetermined areas. The 0001 image is divided into 7 × 6 blocks. No. For images of 0002, 5 × 7 blocks, No. The 0003 image is divided into 6 × 6 blocks. In the illustrated example, it is divided into several blocks for simplification of description, but it is divided into several hundred to several thousand blocks.

  Next, the feature amount of the detected partial image is calculated (step S12). When the feature region is extracted, a local feature amount resistant to affine transformation such as scale change, rotation, and angle change is extracted. Examples of the local feature amount include the following.

・ SIFT
・ Gradient location and orientation histogram
・ Shape context
・ PCA-SIFT
・ Spin images
・ Steerable filters
・ Differential inverters
・ Complex filters
・ Moment inviteants

  The extraction of local features is published in “A performance evaluation of local descriptors” (IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.27, No.10, pp.1615-1630, 2005.) A known technique can be adopted as appropriate.

  Since the feature vector generated based on the feature amount extracted from the feature region is generated from the feature region where the object (object) is highly likely to exist, it is effective as an index indicating the feature of the object in the image.

  Further, when partial images are extracted by area division, image feature amounts expressed by quantifying the features of each image such as the color scheme, texture, and shape of the image are used. Since the feature vector generated from the feature amount of the region image detected by the region division is generated from each part constituting the image, it is effective as an index indicating the overall configuration of the image.

  Then, the plurality of partial images detected from the image data are mapped (classified) to visual keywords based on the feature values of the partial images (step S13). The mapping to the visual keyword is performed by selecting the visual keyword having the closest distance based on the distance in the feature amount space between the center point of each visual keyword (cluster) and the feature amount of the region image.

  In the example of FIG. 3, the partial images T1, T3 to T6 detected from the image with the image ID “0001” are mapped to the visual keyword # 1, and the partial image T2 is mapped to the visual keyword # 2. Also, the partial images T12 to T14 detected from the image with the image ID “0002” are mapped to the visual keyword # 3. Also, the partial image T11 of the image with the image ID “0002” and the partial image T21 of the image with the image ID “0003” are mapped to the visual keyword # 4.

  When each partial image is mapped to a visual keyword (cluster), the feature vector generation unit 20 counts the appearance frequency of the region image for each visual keyword, and displays it in a multidimensional manner according to the appearance frequency of the region image for each visual keyword. A feature vector to be generated is generated (step S14). In the feature vector generation process of the indexing unit 70, the generated feature vector and the image ID are associated with each other and stored in the index DB 75.

  For example, in the case of the image “0001” in FIG. 3, the appearance frequency of the region image detected from the image is “5” for the visual keyword # 1, “1” for the visual keyword # 2, and “1” for the visual keyword # 3. 0 '. A feature vector generated by using the appearance frequency for the plurality of visual keywords as a vector element is stored in the index DB 75 as an image index.

  In the feature vector generation process of the indexing unit 70, a region ID is assigned to a region image detected from the search target image, and region management is performed by associating the VKID of the visual keyword mapping the region image with the image ID and region ID. Store in DB80. With this area management DB 80, it is possible to confirm to which visual keyword a partial image in each image is mapped.

[Cutting out comparison area and calculating similarity]
Next, the comparison area cut-out process and the similarity calculation process will be described with reference to the flowchart of FIG. 4 and the conceptual diagrams of FIGS. These processes are performed by the comparison area extraction unit 30 selecting image IDs one by one from the index DB 75 and the area management DB 80.

  First, the comparison area cutout unit 30 performs matching between the partial image extracted from the query image and the partial image extracted from the selected search target image, and selects a matching partial image, that is, a partial image with high similarity. Select (step S21). Specifically, matching can be performed based on the appearance frequency of the partial image for each visual keyword included in the feature vector of the query image and the feature vector of the search target image.

  For example, as shown in FIG. 5, the number of partial images belonging to the visual keyword # 1 among the feature vectors of the query image G3 is “4”, and the partial images belonging to the visual keyword # 1 of the feature vector of the search target image G5 are displayed. The number is '5'. Since this indicates the same visual keyword, that is, the number of partial images mapped to the cluster, it can be determined that these partial images have high similarity.

  In the example of FIG. 5, since the partial images G30 and G31 and the partial image G51 are mapped to the visual keyword # 1, the partial image mapped to the same visual keyword is searched by referring to the area management DB 80. Then, the matching partial image is selected. In this manner, partial images can be easily matched by using feature vectors generated from visual keywords.

  Next, the comparison area cutout unit 30 cuts out the comparison area based on the partial image in the search target image that matches the partial image of the query image (step S22). Specifically, a region including the matched partial image is cut out. In FIG. 5, it is determined that the partial images G30 to G33 in the query image match the partial images G51 to G53 in the search target image, and a rectangle including these is formed, and this is cut out as the comparison region R. .

  Next, the similarity calculation unit 40 calculates a similarity score between the query image and the image in the comparison area (step S23). Specifically, a feature vector for the image in the comparison area is generated, and the distance between the feature vector and the feature vector of the query image is calculated as a similarity score.

  For the feature vector of the image in the comparison area, the comparison area cut out in step S22, that is, the partial image in the area cut out in consideration of the position as a comparison target is searched based on the area ID of the area management DB 80. Since the visual keyword to which the partial image belongs can be determined from the VKID associated with the region ID, a feature vector is generated by newly calculating the appearance frequency of the partial image for each visual keyword.

  In this process, the area IDs and VKIDs of the partial images that have already been mapped to the visual keywords may be totaled, and the feature vector V7 for the comparison area R can be generated as shown in FIG. As for the calculation of the similarity score, the similarity between the partial image in the query image and the partial image in the comparison area may be calculated for each, and the total may be used as the similarity score.

  Next, the similarity calculation unit 40 calculates the ratio of the partial image that matches the partial image in the comparison area to the partial image extracted from the query image (step S24). For example, the number of partial images extracted in the query image G3 shown in FIG. 6 is “8”, and the number of partial images that match the partial images in the comparison region is “4”. The ratio of the matched partial image to the total number of partial images is ½.

  Next, the similarity calculation unit 40 calculates the ratio of the partial image that matches the partial image in the query image to the partial image extracted from the comparison region (step S25). For example, the number of partial images extracted in the comparison region R shown in FIG. 6 is “9”, and the number of partial images matching the partial image in the query image G3 is “3”. The ratio of the matched partial image to the total number of partial images is 1/3.

  The similarity calculation unit 40 calculates the similarity by multiplying the similarity score calculated in step S23 by weighting the ratio of the matched partial images in the query image and the comparison region as a weight (step S26).

The above calculation of the similarity is expressed as follows.
Similarity = Ratio of matched partial images in query image A × Ratio of matched partial images in comparison region B × Similarity score C
Ratio of matched partial images in query image A = number of matched partial images in query image / number of partial images in query image Ratio of matched partial images in comparison region B = matched portion in comparison region Number of images / number of partial images in comparison area

  For example, when the similarity between the query image G3 in FIG. 7 and the comparison region R is calculated, there are partial images G71 to G73 that match the upper and lower portions of the image in the comparison region R. In the conventional calculation of the similarity, the similarity is calculated based on the degree of similarity with the partial image in the query image. Therefore, a high similarity is obtained in the case of FIG.

  However, when there is no partial image that matches the central portion in the comparison region R, it is desirable that the similarity is low. Based on the above formula, by using the ratio of the number of matched partial images to the number of partial images in the comparison region, the degree of similarity can be lowered for an image having a region that does not match.

  In this way, it is possible to calculate the degree of similarity of a region in the search target image that matches the partial image in the query image, and further paying attention to the partial image that matches in the inner region. Therefore, it is possible to realize a high-accuracy similar image search that pays attention to a characteristic region of the query image.

  In the above-described embodiment, the similarity score is weighted by using both the ratio of the matched partial image in the query image and the ratio of the matched partial image in the comparison area. It is good also as weighting by.

  In addition, the comparison area has been cut out as an area including all the matched partial images.For example, for the set of matched partial images, the partial image located on the outermost side in each of the upper, lower, left, and right directions is displayed. An area including a set of partial images after removal may be cut out as a comparison area.

  For example, in FIG. 8, with respect to the set of partial images in the comparison region R, the partial image farthest from the center of the comparison region R is G52. Therefore, the comparison region R10 including G51 and G53, which are partial images matching the partial image of the query image G3, is extracted from the partial images from which the partial image G52 has been removed. The number of partial images to be removed may be determined by a constant (for example, 2) or may be determined by a ratio (for example, 5%) to the total number of extracted partial images.

  In this way, by removing the outer partial image from the matched partial images and cutting out the comparison region, the region of the search image for calculating the similarity can be further narrowed down. realizable.

  Further, the feature vectors may be weighted by TF / IDF (term frequency-inverse document frequency) which is a word weighting method in text search.

For reference materials on TF / IDF,
CDManning, P. Raghavan and H. Schutze: "Introduction to Information Retrieval", Cambridge University Press. 2008.
It has been known.

TF / IDF is an algorithm for extracting a characteristic word in a sentence, and is calculated by two indexes, ie, TF that is the appearance frequency of the word and IDF that is the reverse appearance frequency. Specifically, it is calculated | required by following Formula.
TF / IDF = TF (i, j) / T (i) * IDF (j)
IDF (i) = log (N / DF (i))

here,
TF (i, j) is the number of occurrences of keyword j in document i to be extracted, T (i) is the number of all words in document i N is the number of all documents DF (j) is The number of documents containing the keyword j.

  By treating the document as an image and the word as a partial image belonging to the same visual keyword, obtaining a TF / IDF value for each visual keyword of each image, and adding this TF / IDF value for each visual keyword, Generate a feature vector.

  At this time, assuming that the image ID is i and each visual keyword k, TF / IDF (i, k) that is a weight value of each visual keyword is calculated by the following equation.

TF / IDF (i, k) = TF (i, k) / T (i) * IDF (k)
IDF (k) = log (N / DF (k))

  TF (i, k) is obtained by weighting the number of partial images extracted from the image i that appear in the visual keyword k, and the partial images belonging to (appearing in) each visual keyword k and visual The weight value (0 to 1) described above based on the distance from the center point of the keyword k is obtained.

  T (i) is a value obtained by weighting the total number of partial images extracted from the image i based on the distance from the visual keyword, and is based on the distance from the cluster to which each partial image extracted from the image i belongs. This is the sum of the weight values.

  DF (k) is a value obtained by weighting the number of partial images classified into each visual keyword k appearing in each visual keyword k based on the distance from the visual keyword. N is the total number of images in the search target image DB 90.

  In this way, the document in TF / IDF is regarded as an image, and the words in the document are regarded as partial images belonging to the same visual keyword, and weighting is performed, so that the importance of partial images appearing in each image is reduced and specified. The scalar value of the feature vector can be weighted so as to increase the importance of the characteristic partial image that appears conspicuously in the image.

  Using the weighting by TF / IDF, a similarity score between the visual keyword to which the partial image in the query image belongs and the visual keyword to which the partial image in the search target image belongs may be obtained.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

DESCRIPTION OF SYMBOLS 1 Image search device 3 Comparison area extraction part 10 Query image reception part 20 Feature vector generation part 30 Comparison area extraction part 40 Similarity calculation part 50 Search result output part 60 Visual keyword generation part 70 Indexing part 75 Indexing part 65 Visual keyword DB
75 Index DB
80 Area management DB
90 Search target image DB
R comparison area

Claims (5)

In an image search device for calculating a similarity with a query image and searching for an image with a high similarity from search target images,
First extraction means for extracting a plurality of partial images from the query image;
Second extraction means for extracting a plurality of partial images from the search target image;
A region cutout unit that selects a partial image in the search target image similar to the partial image extracted from the query image, and cuts out a region including the selected partial image from the search target image;
When calculating the similarity between the query image and the image in the clipped region, the number of the partial images included in the clipped region and the partial image in the region Similarity calculation means for assigning, as a weight, a ratio between the number of partial images similar to the partial image extracted from the query image;
An image search apparatus comprising: The similarity calculation means includes:
A ratio weight between the number of the partial images included in the query image and the number of partial images similar to the partial images in the search target image among the partial images in the query image is further used as the weight of the similarity The image search apparatus according to claim 1, wherein the image search apparatus is assigned. The region cutting means is
When selecting a partial image in the search target image that is similar to the partial image extracted from the query image, a partial image located outside the set of partial images is excluded from the selection, and after the exclusion The image search apparatus according to claim 1, wherein an area including a partial image is cut out. In an image search method in which a computer calculates a similarity with a query image and searches an image with a high similarity from search target images,
A first extraction step of extracting a plurality of partial images from the query image;
A second extraction step of extracting a plurality of partial images from the search target image;
Selecting a partial image in the search target image similar to the partial image extracted from the query image, and cutting out a region including the selected partial image from the search target image;
When calculating the similarity between the query image and the image in the clipped region, the number of the partial images included in the clipped region and the partial image in the region A similarity calculation step of assigning, as a weight, a ratio between the number of partial images similar to the partial image extracted from the query image;
The image search method characterized in that the computer executes.   A program for causing a computer to execute the image search method according to claim 4.

Images