JP5121917B2 - Image search apparatus, image search method and program - Google Patents

Description

  The present invention relates to an image search device, an image search method, and a program.

  By inputting an image as a search key and comparing image feature amounts (represented by quantifying image features such as color scheme, texture, and shape), an image that is a search key (hereinafter referred to as a “query image”) A technique for searching for an image similar to is known. When the user inputs a query image, the feature amount is extracted from the query image, and the similarity with the feature amount of the search target image is calculated to search for a similar image (for example, Patent Document 1).

  Further, as a similar image search technique that focuses on a partial area in an image, a technique called a visual keyword is known. This is a technique devised by considering that one image is composed of a plurality of partial images, and a feature vector is generated by the following processing.

  That is, a plurality of partial images are extracted from an image, and the partial image is mapped based on the feature amount to a reference cluster (hereinafter referred to as “reference cluster” where appropriate) formed by clustering the images in advance ( And feature vectors are generated based on the number of reference clusters to which each partial image belongs. As described above, by using the visual keyword, it is possible to perform an image search with high accuracy not by the feature amount extracted from the entire image but by the feature amount obtained by capturing the image as a fine area.

  An image similar to the query image is output as a search result based on the similarity calculated as described above. When ranking and ranking are displayed based on the similarity, it is common to select an image having the similarity equal to or higher than a predetermined threshold and output it as a search result (for example, Patent Document 2). ).

JP 2001-52175 A JP 2005-148801 A

  By the way, when performing an image search, the query image input by the user includes an object (target object) that the user wants to search, and it is desirable that an image similar to the object is output as a search result.

  However, the feature amount and feature vector extracted from the image are obtained by mechanically quantifying the color scheme, texture, shape, and the like of the image, and thus do not consider the objects included in the query image. For this reason, even if the similarity is equal to or higher than a preset threshold value, an image that does not include an object similar to the query image may be selected. In addition, if the threshold value is set higher, an image including an object similar to the query image cannot be selected as a search result. As described above, when the threshold value is determined in advance, the accuracy of the selected search result is lowered.

  The present invention has been made in view of the above-described problems, and an object thereof is to improve the accuracy of selecting an image to be output as a result of a similar image search.

  To achieve the above object, according to a first aspect of the present invention, there is provided an image search apparatus for searching for and outputting an image similar to a query image from a plurality of search target images, and a plurality of search images extracted from the query image and the search target image. Feature vector generation means for generating a feature vector based on the number of classification of each partial image extracted from each image into each cluster by classifying each partial image into a plurality of clusters, and the feature vector of the query image And ranking means for calculating the similarity with the feature vectors of the plurality of search target images, ranking the search target images based on the similarity, and a cluster that classifies the partial images of the query image as a reference cluster The common classification for calculating the ratio of the number of the reference clusters obtained by classifying the partial images of the search target image in common with respect to the number of all the reference clusters A calculation unit, from the top of the said ranking search target image, wherein the ratio is characterized by and a selecting means for selecting as an output target search target image a predetermined value or more.

  According to the first aspect of the present invention, the ratio of the number of reference clusters in which the partial images of the search target image are commonly classified is calculated with respect to the reference clusters in which the partial images of the query image are classified. From the top, search target images having a ratio equal to or higher than a predetermined value are selected as output targets. That is, the selection condition of the output target is a ratio of the number of common reference clusters calculated based on the query image, and the selection condition dynamically changes based on the query image. Therefore, based on the query image, a search target image having a high degree of similarity and having a reference cluster in common with the query image is selected. Therefore, it is possible to improve the accuracy of selecting an image to be output as a result of similar image search.

  According to a second aspect of the present invention, there is provided a clustering apparatus that selects the query image one by one from the search target image and inputs the query image to the feature vector generation unit. A clustering unit that obtains a search target image selected by the selection unit and creates a set of the search target image and the query image is provided.

  According to the second invention, a set is created by collecting the search results selected for the query images selected from the search target images. For this reason, an image group is created by an image including an object similar to the query image, and high-precision image clustering can be realized.

  According to the present invention, the accuracy of selecting an image to be output as a result of a similar image search is improved.

The block diagram which shows the function structure of an image search device. The flowchart of a feature vector generation process. The flowchart of a search process. The figure for demonstrating an example of calculation of a common classification degree. The figure for demonstrating an example of selection of the image of the output object. The figure which shows an experimental result. The flowchart of a clustering process.

[Configuration of image search device]
Hereinafter, an embodiment when the image search apparatus of the present invention is applied to the image search apparatus 1 shown in FIG. 1 will be described with reference to the drawings.

  FIG. 1 is a functional block diagram of the image search apparatus 1. The image search device 1 is connected to the Internet connected via a communication network, and can collect image data from the web via the Internet. The collected data is accumulated in a database (DB) to create a search target image.

  The image search apparatus 1 receives a query image transmitted from a client terminal such as a personal computer or a mobile terminal connected via a communication network as a search request. Then, a similar image search is performed according to the search request, and search results ranked in the order of similarity are output to the client terminal.

The function of the image search device 1 may be used as a module. For example, an input of a query image may be received from another module, and an image of a search result may be returned to the module.

  The image search apparatus 1 according to the present embodiment generates and indexes an image feature vector using a visual keyword technique. An image search using a visual keyword represents a single image as a set of a plurality of image areas, and the image search is performed based on feature amounts obtained from image areas (hereinafter referred to as “partial images” as appropriate) constituting each image. This is a technique for generating an index (feature vector), and can be said to be an application of a text search technique for obtaining a feature amount of a sentence from a keyword in the text.

  For this reason, in image search using visual keywords, text search technology (transposition index, vector space model, word appearance frequency, etc.) is applied to image region search by treating the image region in the image as a keyword. Scale and high speed can be realized.

As reference technical literature on image search using visual keywords,
Sivic and Zisserman: “Efficient visual search for objects in videos”, Proceedings of the IEEE, Vol.96, No.4, pp.548-566, Apr 2008.
・ Yang and Hauptmann: “A text categorization approach to video scene classification using keypoint features”, Carnegie Mellon University Technical Report, pp. 25, Oct 2006.
・ Jiang and Ngo: “Bag-of-visual-words expansion using visual relatedness for video indexing”, Proc. 31 ^st ACM SIGIR Conf., Pp.769-770, Jul 2008.
・ Jiang, Ngo, and Yang: “Towards optimal bag-of-features for object categorization and semantic video retrieval”, Proc. 6th ACM CIVR Conf., Pp.494-501, Jul. 2007.
・ Yang, Jiang, Hauptmann, and Ngo: “Evaluating bag-of-visual-words representations in scene classification”, Proc. 15 ^th ACM MM Conf., Workshop on MMIR, pp.197-206, Sep. 2007.
Etc.

  In addition, by expressing a single image as a set of a plurality of partial images, unlike general similar image search, search using an image obtained by cutting a part of an image in an arbitrary size or position as a query image is possible. It becomes possible. For this reason, in order to obtain a desired search result, the user can express a query more directly and accurately by an operation such as designating a part of an image.

  As shown in FIG. 1, the image search apparatus 1 includes a query image receiving unit 10, a feature vector generating unit 20, a ranking unit 30, a common classification degree calculating unit 40, a search result selecting unit 50, a visual keyword generating unit 60, a visual keyword. A DB 65, an indexing unit 70, an index DB 75, a region management DB 80, and a search target image DB 90 are configured.

  These functional units are configured by so-called computers, and are associated with a CPU (Central Processing Unit) as an arithmetic / control device, a RAM (Random Access Memory) and a ROM (Read Only Memory) as a storage medium, a communication interface, and the like. It is realized with.

  The query image receiving unit 10 receives and receives a query image serving as a search key for similar image search from a client terminal or a module. This query image includes an image stored in the search target image DB 90, an image cut out by an operation for designating a partial area of the image data, and a newly received image. The query image may be a single image or a combination of a plurality of images.

  The feature vector generation unit 20 extracts a partial image from the query image, performs a feature vector generation process (see FIG. 2) that generates a feature vector based on the feature amount of the partial image, and generates a feature vector from the query image. To do. As shown in FIG. 1, the feature vector generation unit 20 includes a partial image extraction unit 22 and a mapping unit 24, and implements a feature vector generation process to be described later by cooperating with each other.

  The ranking unit 30 calculates the similarity between the feature vector for each search target image stored in the index DB 75 and the feature vector generated from the query image, and ranks the search target images based on the similarity (ranking). Append) For calculating the similarity, a known technique such as a cosine distance or a Bhattacharyya distance is used.

  The common classification degree calculation unit 40 calculates, as the common classification degree, the ratio of the common visual keyword that maps the partial image of the search target image to the visual keyword (cluster) that maps the partial image of the query image. That is, the common classification degree is a degree having a common visual keyword with the query image, and it can be said that the higher the common visual keyword, the higher the probability of having a common area.

  The search result selection unit 50 selects and outputs images having a common classification degree equal to or higher than a predetermined value from the top of the search target images ranked by the ranking unit 30. The data output by the search result selection unit 50 is, for example, data obtained by sorting the image IDs of search target images based on the similarity. An address (URL) for accessing the search target image DB 90 may be added to the image ID.

  When generating a feature vector of image data, the visual keyword generation unit 60 generates a classification destination (reference cluster) to which a partial image in the image is mapped. The visual keyword generation unit 60 extracts a plurality of partial images from images used for image search and image data prepared in advance for learning, and clusters them based on the feature amounts of the partial images. As a standard method of clustering, k-means, Hierarchical Agglomerative Clustering (HAC) or the like is used.

  The feature vector generation unit 20 generates a feature vector by mapping (classifying) the partial image detected from the image to a cluster formed by the clustering of the visual keyword generation unit 60. This cluster is referred to as a “visual keyword” as a feature amount space for expressing an image as a collection of visual keywords. Note that “classification” and “mapping” as used in the present invention refers to associating a partial image with a reference cluster (visual keyword), and does not add the partial image to the set of reference clusters.

  The visual keyword DB 65 includes a visual keyword ID (VKID) for identifying a cluster formed by clustering by the visual keyword generation unit 60, and a center coordinate that is a coordinate of a center point in the feature amount space (multidimensional space) of the cluster. , A database that stores the radius indicating the cluster range in association with each other.

  The center coordinate is a value indicating an average value of feature amounts of images belonging to each cluster, and is represented by multidimensional coordinates on the feature amount space. The radius is obtained, for example, by the distance from the image farthest from the center coordinate among the images belonging to the cluster.

  The indexing unit 70 generates a feature vector for the image data stored in the search target image DB 90 based on the feature vector generation process of FIG. 2, and associates the generated feature vector with the index DB 75 as an index of the image data. And remember. The indexing unit 70 includes a partial image extraction unit 72 and a mapping unit 74, and implements a feature vector generation process to be described later by cooperating with each other.

  Further, the indexing unit 70 assigns a region ID to the partial image detected from the image data, and stores the VKID of the visual keyword mapping the partial image in the region management DB 80 in association with the image ID and the region ID. The area ID may be an XY coordinate in the image, or may be a row number / column number when the area is divided.

  The index DB 75 is a database that stores an image ID of image data stored in the search target image DB 90 and a feature vector (frequency of appearance of partial images for each visual keyword) generated from the image data in association with each other.

  The area management DB 80 is a database that manages the correspondence between visual keywords that map partial images in a search target image, and corresponds to the image ID, area ID, and VKID of the search target image as shown in FIG. Add and remember.

  The search target image DB 90 is a database for accumulating and storing image data collected from the Internet as a search target for similar images (referred to as “search target image”). As shown in FIG. 1, an image ID, image data, Are stored in association with each other. The image ID is identification information for uniquely identifying each image data, and is assigned when a keyword and image data are stored.

[Feature vector generation processing]
Next, the feature vector generation process will be described with reference to the flowchart of FIG. 2 and the conceptual diagram of FIG. The feature vector generation process is performed on the query image by the feature vector generation unit 20 and the search target image by the indexing unit 70. In the following description, the case of the feature vector generation unit 20 will be described.

  First, the partial image extraction unit 22 extracts a plurality of partial images from the image data of the query image (step S11). As a method for extracting the partial image, there are a method for extracting a characteristic region (feature region) in the image and a method for extracting the image by dividing the image into predetermined regions.

As a technique for detecting feature regions,
・ Harris-affine
・ Hessian-affine
・ Maximally stable extremal regions (MSER)
・ Difference of Gaussians (DoG)
・ Laplacian of Gaussian (LoG)
・ Determinant of Hessian (DoH)
Etc.

  The feature region detection technology is published in “Local Invariant Feature Detectors: A Survey” (Foundations and Trends in Computer Graphics and Vision, Vol.3, No.3, pp.177-280, 2007.). Any known technique can be adopted as appropriate.

  In addition, as a method of detecting an image by dividing it into predetermined areas, for example, the image is divided into predetermined M × N blocks, or divided so that the size of the divided blocks becomes predetermined m × n pixels. There is a technique to do. For example, when an image is divided into 10 × 10 blocks, if the size of the image is 640 × 480 pixels, the size of one block is 64 × 48 pixels.

  Next, the feature amount of the extracted partial image is calculated (step S12). When the feature region is extracted, a local feature amount resistant to affine transformation such as scale change, rotation, and angle change is extracted. Examples of the local feature amount include the following.

・ SIFT
・ Gradient location and orientation histogram
・ Shape context
・ PCA-SIFT
・ Spin images
・ Steerable filters
・ Differential inverters
・ Complex filters
・ Moment inviteants

  The extraction of local features is published in “A performance evaluation of local descriptors” (IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.27, No.10, pp.1615-1630, 2005.) A known technique can be adopted as appropriate.

  Since the feature vector generated based on the feature amount extracted from the feature region is generated from the feature region where the object (object) is highly likely to exist, it is effective as an index indicating the feature of the object in the image.

  Further, when partial images are extracted by area division, image feature amounts expressed by quantifying the features of each image such as the color scheme, texture, and shape of the image are used. Since the feature vector generated from the feature amount of the region image detected by the region division is generated from each part constituting the image, it is effective as an index indicating the overall configuration of the image.

  Next, the mapping unit 24 selects one of the plurality of partial images extracted from the image (step S13), the partial image, and each visual keyword (cluster formed in advance by the visual keyword generation unit 60). Is calculated (step S14). This distance is obtained by the distance between the center coordinates of the partial image in the feature amount space and the center coordinates of the visual keyword. As the value used for the center coordinates of the cluster, in addition to the centroid as the average value of the data (features) belonging to the cluster, the medoid that adopts the data closest to the center, or the center position when the data are arranged in ascending order Median that employs data to be applied can be applied.

  Then, the selected partial image is mapped (classified) to the visual keyword having the closest calculated distance (step S15). At this time, the mapping unit 24 adds the number of partial images to the scalar value of each visual keyword of the feature vector in the index DB 75. Further, the VKID of the visual keyword that maps the partial image is assigned to the region ID of the partial image in the region management DB 80, and each is associated and stored.

  For example, in FIG. 4, it is assumed that as a result of calculating the distance between the partial image G11 and the visual keywords VK1 to VKN, it is determined that the visual keyword VK1 is the closest. In this case, the partial image G11 is mapped to the visual keyword VK1, and 1 is added to the scalar value of VK1 of the feature vector V.

  Next, the mapping unit 24 determines whether or not all partial images have been selected in step S13 (step S16). If there is an unselected partial image (step S16; No), the process proceeds to step S13. And the process up to step S15 is repeated.

  If it is determined that all partial images have been selected (step S16; Yes), a feature vector is generated based on the mapped visual keyword (step S17). That is, a feature vector expressed in multi-dimensions is generated according to the appearance frequency of partial images for each visual keyword as a result of mapping. Thereby, the feature vector V about the query image G1 is generated as shown in FIG.

  The feature vector generation unit 20 outputs the feature vector of the query image generated by the above processing to the ranking unit 30 and the common classification degree calculation unit 40. Further, when the indexing unit 70 generates each feature vector of the search target image by the feature vector generation process, the indexing unit 70 stores it in association with the index DB 75 in association with the image ID of the search target image.

  As described above, a feature vector represented by a distribution of local feature amounts in an image to visual keywords can be generated by feature vector generation processing.

[Search processing]
Next, search processing performed by the ranking unit 30, the common classification degree calculation unit 40, and the search result selection unit 50 will be described with reference to the flowchart of FIG. 3 and conceptual diagrams of FIGS.

  First, when the ranking unit 30 obtains the feature vector of the query image output from the feature vector generation unit 20, the similarity between the feature vector and the feature vectors of the plurality of search target images stored in the index DB 75 is obtained. Is calculated (step S31).

  Then, the search target images are ranked (ranked) based on the calculated similarity and rearranged in descending order of similarity (step S32). Thereby, as shown in FIG. 5, search target images similar to the query image G1 are obtained in the order of similarity.

  Further, the common classification degree calculation unit 40 uses the visual keyword (VK) to which the query image belongs as a reference cluster, and the ratio of the number of the reference clusters in which the partial images of the search target image are commonly classified with respect to the reference cluster. Is calculated as a common classification degree (step S33).

  This common classification is calculated by dividing the number of common visual keywords that map partial images of the search target image among the visual keywords that map partial images of the query image by the number of visual keywords that map partial images of the query image. It is the value.

  For example, in FIG. 4, there are three visual keywords VK1, VK2, and VK3 to which the partial images of the query image G1 are mapped, and among these visual keywords, the visual keywords to which the partial images of the search target image are mapped. Since there are two, VK1 and VK3, the common classification degree is calculated as 2/3.

  The search result selection unit 50 selects those having a common classification degree equal to or higher than a predetermined threshold from the top of the search target images ranked by the ranking unit 30 (step S34). For example, if the threshold is set to 10%, search target images G51 to G56 are selected. These search target images have a high degree of including partial images belonging to a common visual keyword on the basis of the query image.

  The search result selection unit 50 sets the search target images in the descending order of similarity selected based on the common classification degree as search results, and outputs the search results to the client terminal or module that has received the query image (step S35).

  As described above, when outputting the search result for the query image, the similar image based on the local feature amount is searched based on the similarity between the feature vector of the query image and the search target image, and the query image is further searched. A search target image having a common area is selected based on a common classification degree that is a relative value as a reference.

  Selection of the search target image based on the common classification degree is performed based on a predetermined threshold, and this threshold is a relative ratio based on the query image. FIG. 6 is an example of the result of an experiment performed by changing the query image for 10,200 search target images with the common classification degree threshold set to 10%. As shown in FIG. 6, even if the common classification degree threshold is set to 10%, the minimum value of the similarity of the search target images selected as a result of the experiment varies depending on the reference query image.

  For this reason, unlike setting a fixed (absolute) threshold value with respect to the conventional similarity, the similarity of the search target image actually selected is dynamically changed according to the query image, A search target image having a visual keyword common to the query image is selected. Therefore, based on the mapped visual keyword of the query image, it is possible to improve the accuracy of selecting an image to be output as a similar image search result.

[Modification]
The above-described embodiment is an example to which the present invention is applied, and the applicable range can be changed as appropriate.

  For example, the common classification degree may be weighted. Specifically, in the above-described embodiment, the common classification degree (for example, 2/3 = (1 + 1) / () is determined by the number of visual keywords that map query images and the number of visual keywords that map search target images in common. 1 + 1 + 1)), but the number of these visual keywords is weighted using the distance between the partial image and the visual keyword.

  In FIG. 4, partial images G11 to G15 of the query image are mapped to the visual keyword VK1, and an average value D1 of the distance between each mapped partial image and the visual keyword VK1 is obtained. For the visual keyword VK2, the average value D2 of the distance between the mapped partial image and the visual keyword VK2 is obtained.

  Based on this distance, by weighting the number of visual keywords to which the query image is mapped, this number becomes 1 × D1 + 1 × D2 + 1 × D3.

  Similarly, for the search target image, the average value of the distance from the partial image mapped to each visual keyword is calculated for each visual keyword, and the number of visual keywords mapped in common with the query image based on this distance Is weighted.

  For example, in FIG. 4, the average value of the distance to the partial image for the visual keyword VK1 to which the search target image is mapped is obtained as DA, and the average value of the distance to the partial image for the visual keyword VK3 is obtained as DC. For example, if the number of visual keywords mapped in common with the query image is weighted, 1 × DA + 1 × DC is obtained.

  When the common classification degree is obtained by the number of visual keywords weighted in this way, (DA + DC) / (D1 + D2 + D3) is obtained. By weighting based on the distance between the partial image and the visual keyword in this way, it is possible to consider the variation in the distance between the visual keyword and the partial image, so that the accuracy of the common classification degree can be improved. .

  Further, for example, the above-described embodiment has been described only for the configuration in which the search result is output for the query image. However, the search target image may be clustered using this. FIG. 7 is a flowchart for explaining the class ring processing.

  Although not shown, the image search apparatus 1 of FIG. 1 includes a clustering unit, and this clustering unit selects one search target image from the search target image DB 90 (step S51). And the search process mentioned above is performed by inputting the selected search object image into the query image reception part 10 as a query image (step S52).

  The clustering unit receives the search result output from the search result selection unit 50, forms a set (cluster) from the search target image and the query image, and stores the set in a cluster DB (not shown) for each set ( Step S53).

  Then, the registered query image and search target image are deleted from the search target image DB 90 (step S54). For this deletion, for example, it is not necessary to actually delete data, for example, by setting a deletion flag for each image.

  The clustering unit determines whether or not there is an unprocessed image in the search target image DB 90 (step S55), and if it is determined that there is an unprocessed image (step S55; Yes), the process proceeds to step S51. Move to let the search process.

  As described above, by performing the above-described search processing on the images stored in the search target image DB 90 and sequentially creating a set of images, an image including an object highly similar to the query image. Can be summarized (clustering).

  The plurality of sets created by the clustering process may be further summarized based on the distance between the sets (for example, the distance between the averages of the feature vectors of the image groups in the set). As a standard method of this clustering, k-means, Hierarchical Agglomerative Clustering (HAC), etc. are used. That is, a cluster including similar objects can be created by performing hierarchical clustering by bottom-up by collecting a plurality of clusters.

  Further, the feature vectors may be weighted by TF / IDF (term frequency-inverse document frequency) which is a word weighting method in text search.

For reference materials on TF / IDF,
CDManning, P. Raghavan and H. Schutze: "Introduction to Information Retrieval", Cambridge University Press. 2008.
It has been known.

TF / IDF is an algorithm for extracting a characteristic word in a sentence, and is calculated by two indexes, ie, TF that is the appearance frequency of the word and IDF that is the reverse appearance frequency. Specifically, it is calculated | required by following Formula.
TF / IDF = TF (i, j) / T (i) * IDF (j)
IDF (i) = log (N / DF (i))

here,
TF (i, j) is the number of occurrences of keyword j in document i to be extracted, T (i) is the number of all words in document i N is the number of all documents DF (j) is The number of documents containing the keyword j.

  This is regarded as a partial image belonging to the same visual keyword with a document as an image and a word, and a TF / IDF value is obtained for each visual keyword of each image, and this TF / IDF value is converted into a visual keyword mapping the partial image. A feature vector is generated by addition.

  At this time, assuming that the image ID is i and each visual keyword k, TF / IDF (i, k) that is a weight value of each visual keyword is calculated by the following equation.

TF / IDF (i, k) = TF (i, k) / T (i) * IDF (k)
IDF (k) = log (N / DF (k))

  TF (i, k) is obtained by weighting the number of partial images extracted from the image i that appear in the visual keyword k, and the partial images belonging to (appearing in) each visual keyword k and visual The weight value (0 to 1) described above based on the distance from the center point of the keyword k is obtained.

  T (i) is a value obtained by weighting the total number of partial images extracted from the image i based on the distance from the visual keyword, and is based on the distance from the cluster to which each partial image extracted from the image i belongs. This is the sum of the weight values.

  DF (k) is a value obtained by weighting the number of partial images classified into each visual keyword k appearing in each visual keyword k based on the distance from the visual keyword. N is the total number of images in the search target image DB 90.

  In this way, the document in TF / IDF is regarded as an image, and the words in the document are regarded as partial images belonging to the same visual keyword, and weighting is performed, so that the importance of partial images appearing in each image is reduced and specified. The scalar value of the feature vector can be weighted so as to increase the importance of the characteristic partial image that appears conspicuously in the image.

  Using the weighting by TF / IDF, a similarity score between the visual keyword to which the partial image in the query image belongs and the visual keyword to which the partial image in the search target image belongs may be obtained.

  Moreover, although the common classification degree calculation part 40 demonstrated as calculating with respect to all the search target images, for example, it is a predetermined number (for example, 1-20th rank) from the top among the search target images ranked by similarity. It is good also as calculating only about 20 search object images. Further, when the common classification degree of a predetermined number of search target images is all equal to or greater than the threshold value, the calculation target may be further expanded in order for lower search target images (for example, 20 items from 21 to 40). . Thereby, since it is sufficient to determine whether or not the calculated common classification degree is equal to or greater than the threshold value, it is possible to increase the processing speed.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

DESCRIPTION OF SYMBOLS 1 Image search device 10 Query image reception part 20 Feature vector generation part 22 Partial image extraction part 24 Mapping part 30 Ranking part 40 Common classification degree calculation part 50 Search result selection part 60 Visual keyword generation part 70 Indexing part 72 Partial image extraction part 74 Mapping unit 65 Visual keyword DB
75 Index DB
80 Area management DB
90 Search target image DB
VK1-VKN visual keywords

Claims (4)

In an image search device that searches and outputs an image similar to a query image from a plurality of search target images,
By classifying each of the partial images extracted from the query image and the search target image into one of a plurality of clusters, a feature vector based on the number of classifications of the partial images extracted from each image into each cluster is generated. Feature vector generation means;
Ranking means for calculating a similarity between the feature vector of the query image and the feature vectors of the plurality of search target images, and ranking the search target images based on the similarity;
A common classification degree calculation that calculates a ratio of the number of the reference clusters that commonly classify the partial images of the search target image with respect to the number of all the reference clusters, with a cluster obtained by classifying the partial images of the query image as a reference cluster Means,
Selecting means for selecting, as an output target, a search target image having the ratio equal to or higher than a predetermined value from the top of the ranked search target images;
An image search apparatus comprising: An image search device according to claim 1;
By selecting the query images one by one from the search target images and inputting them to the feature vector generation unit, the search target images selected by the selection unit are obtained, and the search target image, the query image, A clustering unit that creates a set of
A clustering apparatus comprising: In an image search method in which a computer searches and outputs an image similar to a query image from a plurality of search target images,
By classifying each of the plurality of partial images extracted from the query image and the search target image into one of a plurality of clusters, a feature vector is generated based on the number of classifications of the partial images extracted from each image into each cluster. A feature vector generation step;
A ranking step of calculating a similarity between the feature vector of the query image and the feature vectors of the plurality of search target images, and ranking the search target images based on the similarity;
A common classification degree calculation that calculates a ratio of the number of the reference clusters that commonly classify the partial images of the search target image with respect to the number of all the reference clusters, with a cluster obtained by classifying the partial images of the query image as a reference cluster Process,
From the top of the ranked search target images, a selection step of selecting, as an output target, a search target image whose ratio is a predetermined value or more
The image search method characterized in that the computer performs the above.   A program for causing a computer to execute the image search method according to claim 3.

Images