JP2014197412A - System and method for similarity search of images - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system and method for an efficient semantic similarity search of images with a classification structure.SOLUTION: A system and method comprises: building a semantic classification-search tree for a plurality of images (202), the classification tree including at least two image categories representing subsets of the plurality of images; receiving a query image (204); classifying the query image to select one of the at least two categories of images (208); and restricting the search for an image of interest by using the query image to the selected one of the at least two image categories (210).

Description

The present invention relates to computer graphic processing and display systems, and more particularly to image similarity retrieval.

  Detection and retrieval of images similar to query images is very useful for various applications in the real world. The techniques described in this disclosure address the problem of querying an image database, and preferably query images at a semantic level (ie, images that contain the same object and background, but sometimes include various variations). Is to find an image similar to. This issue arises in various applications, for example, a location recognition service on a mobile terminal, where the user takes an image of the landmark, and then the mobile terminal informs the user of the location and description of the landmark Is the case. In another application, a user can take images of multiple products in a store and then the mobile terminal can return a web page at a corresponding price for the same product offered at a different retail store. In the discovery of copyright infringement, it is possible to identify potential infringements by searching the internet for the use of unauthorized images. In multimedia content management, duplicates and the like can support links to stories in videos, reports, and articles on web pages from multiple sources.

  Although the techniques described in this disclosure can be applied to general image or video search, the present disclosure is more meaningful than visual search based on low-level features such as color and texture. Focus on searching images or videos at the level. Searching for images or videos based on low-level features has been well studied, and highly efficient search algorithms are available for large databases. Searching for images or videos at a semantic level is much more difficult than searching for low-level features. The reason is that a comparison of the objects contained in the image or video is included. For many real-world applications, such as those described above, low-level feature search is generally not sufficient because images containing different objects do not have similar colors or textures.

  Image or video retrieval at a semantic level requires a comparison of objects in the image. Similar images defined for this purpose should contain the same object and background, but may have some changes such as object motion, changes in light and dark. The problem is very difficult because it is difficult to understand or represent images at a level that is meaningful to computers, computing devices, and the like. There were several early studies that performed image and video searches at a semantic level. For example, "Detecting Image Near-Duplicate by Stochastic Attributed Relational Graph Matching with Learning" written by DQ Zhang and SF Chang, October 2004, ACM Multimedia (New York, USA) Partial type similarity measurements are described. Similarity measurements described by Zhang et al. Can actually compare objects in an image and obtain highly accurate results. However, this method is very slow compared to conventional search methods that use low level features (eg, color histograms) and is difficult to apply to real-world applications.

  Accordingly, there is a need for techniques for effectively retrieving images at a semantic level.

  Systems and methods are provided for effectively performing semantic similarity searches of images with classification structures. The system and method query an image database to find a semantic level, i.e., an image that contains the same objects and background as the query image and has several variations, similar to the query image. Enable. The techniques of this disclosure significantly reduce similarity calculations by restricting semantic similarity searches for images within a particular class or category. First, a classification search tree is constructed for all images in the database. Next, for each input query image, the query image is classified into one or more categories (usually semantic categories such as people, indoors, outdoors, etc.). A category represents the entire image space, ie a subset of a database of images. The image similarity calculation is then limited to the subset.

  According to one aspect of the present disclosure, a method for searching a plurality of images for an image of interest is provided. The method builds a classification structure for multiple images. The classification structure includes at least two image categories, each of which represents a subset of a plurality of images. The method then receives the query image and limits the search for the image of interest to a selected one of the at least two image categories.

  According to another aspect, a system for retrieving a plurality of images for an image of interest includes a database including a plurality of images structured into at least two semantic categories. Each semantic category represents a subset of multiple images. The system also retrieves at least one image, an image classification module that classifies the query image and selects one of at least two semantic categories, and retrieves the image of interest using the query image. And an image search module, wherein the search is limited to a selected one of at least two semantic categories.

  In accordance with yet another aspect, a machine-readable program media apparatus is provided for performing program-executable program instructions to perform a plurality of steps for retrieving a plurality of images for an image of interest. The method includes constructing a classification structure for the plurality of images. The classification structure includes at least two image categories, each of which represents a subset of a plurality of images. The method also receives a query image, classifies the query image into a selected one of at least two image categories, and searches for an image of interest in the selected one of the at least two image categories. Limit to.

The same reference numbers in the drawings indicate similar elements in the overall list.
FIG. 1 is an exemplary diagram of a system for similarity search of images in accordance with aspects of the present disclosure. FIG. 2 is a flow diagram of an exemplary method for similarity search of images in accordance with aspects of the present disclosure. FIG. 3 illustrates a classification search tree according to the present disclosure. FIG. 4 illustrates a simple search performed on a classified search tree according to the present disclosure. FIG. 5 illustrates a redundant search performed in a classified search tree according to the present disclosure. FIG. 6 illustrates a method for building or generating a classified search tree in accordance with aspects of the present disclosure. FIG. 7 shows feature vectors for images with tagged keywords. FIG. 8 illustrates a method for adding a new image to a classified search database according to an aspect of the present disclosure.

  These and other aspects, features, and advantages of the present disclosure will be set forth or apparent from the detailed description of the preferred embodiments when read in conjunction with the accompanying drawings.

  It will be understood that each element shown in the figures may be implemented in various forms by hardware, software, or a combination thereof. Each of these elements is implemented in combination with hardware and software on one or more appropriately programmed general purpose devices that can include a processor, memory, and input / output interfaces.

  The description herein sets forth the principles according to the present disclosure. Accordingly, it will be appreciated by those skilled in the art that various arrangements can be devised that embody the principles of the present disclosure and fall within the spirit and scope thereof, even if not explicitly described or shown herein.

  All examples and conditional terms listed here are intended for educational purposes to help readers understand the principles of the present disclosure and the inventor's concepts aimed at contributing to technology promotion. And is not to be construed as being limited to the specifically listed examples and conditions.

  Moreover, all statements herein reciting principles, aspects, embodiments and specific examples thereof are intended to encompass both structural and functional equivalents. These equivalents are also intended to include currently known equivalents and equivalents developed in the future, i.e., any element developed to perform the same function regardless of structure.

  Thus, for example, those skilled in the art will appreciate that the block diagrams shown herein represent illustrative conceptual circuits that embody the principles according to the present disclosure. Similarly, any flowcharts, flow diagrams, state transition diagrams, pseudocode, etc. may be substantially represented on a computer-readable medium and executed by a computer or processor with or without the computer or processor specified. Various processes are shown.

  Functions by various elements shown in the figure can be provided by using hardware capable of executing software in cooperation with dedicated hardware and optimum software. When provided by a processor, it can be provided by a single dedicated processor, a single shared processor, or multiple single processors (some can be shared). Furthermore, explicit uses such as the terms “processor” or “controller” should not be construed to refer exclusively to software-executable hardware, and without limitation, a DSP (Digital Signal Processor). It may include hardware, ROM (Read Only Memory) for storing software, RAM (Random Access Memory), and non-volatile storage.

  Other conventional or custom hardware is also included. Similarly, any switches shown in the figures are merely conceptual. These functions can be performed by dedicated logic, control program and logic program operation through interaction of dedicated logic, or manually, and the particular technique is selected by the executioner as is particularly understood by the context.

  With respect to the claims, any element expressed as a means for performing a particular function can be, for example, a) a combination of circuit elements that perform the function, or b) a suitable circuit that executes software that performs the function. And any method for performing functions including any form of software, including firmware, microcode, etc. This disclosure, as defined by the claims, includes the functions provided by the various listed means being combined and performing the functions together in the manner recited in the claims. Thus, means capable of providing these functions can be considered equivalent to those shown herein.

  The detection and retrieval of images similar to query images is very useful for various applications in the real world. The challenge is to effectively find images that are similar to the query image on a semantic basis (i.e., those taken in the same scene and having the same object). Some conventional techniques have proposed high-precision algorithms that search for semantic images at low speed. The efficiency issue is particularly important when the image database is huge. Usually, the time for searching the image database is linearly proportional to the size of the database. The system and method according to the present disclosure speeds up the search by utilizing the image database structure and the meaning of the image.

  Systems and methods are provided for effectively retrieving images or videos through hierarchical processing. Assuming that a high quality image / video similarity algorithm or function is already available, the speed of the algorithm is slower than the traditional feature type similarity calculation algorithm. Accordingly, the system and method according to the present disclosure provides a speed-up process that accelerates semantic searching of image or video databases. For purposes of omission, the present disclosure is applicable to video, ie, continuous images, but will focus on image retrieval. The system and method speed up the search algorithm by utilizing the structure of the content space of the image. The technique according to the present disclosure limits visual similarity searches for images within a particular class or category, and greatly reduces similarity calculations. Initially, a classification structure is built for every image in the database, but not limited to a classification tree. Next, for each input query image, the images are classified into one or more categories (typically semantic categories are people, indoors, outdoors, etc.) that represent a subset of the overall image space. Image similarity calculations are then limited within the subset.

  Referring now to the drawings, FIG. 1 describes an exemplary system component 100 in accordance with an embodiment according to the present disclosure. The scanning device 103 scans the film print 104. For example, a negative of a camera film is converted into a digital format such as a DPX (Digital Picture Exchange) file of Cineon-format or SMPTE (Society of Motion Picture and Television Engineers). To. The scanning device 103 can include a device that can generate video output from film, such as a telecine device, or Ari LocPro® for video output. Alternatively, a file from a post generation process or digital cinema 106 (eg, a file in computer readable form) can be used directly. The source of the computer readable file can be an AVID (registered trademark) editor, a DPX file, or a D5 tape.

  The digital image or the scanned film print is input to a post-processing device 102 such as a computer. The computer includes one or more central processing units (CPU), a memory 110 such as a RAM and a ROM, a keyboard, a cursor control device (for example, a mouse and a joystick), and an input / output (I / O) user interface 112 such as a display device. Or any other known computer form with hardware. The computer platform also includes an operating system and microinstruction code. The various processes and functions described herein can be part of the microinstruction code or part of a software application (or a combination thereof) that is executed via the operating system. In one embodiment, the software application program is clearly embodied on a program storage device. This software application program can be uploaded and executed by any suitable device, such as post-processing device 102. Also, various other peripheral devices can be connected to the computer platform by various interfaces and a bus structure such as a parallel port, a serial port, or a USB (Universal Serial Bus). Other peripheral devices may include additional storage device 124 and printer 128.

  Also, file / film prints in computer readable form 106 (eg, a digital cinema that can be stored on external hard drive 124) are input directly to computer 102.

  The software program includes a similarity search module 114 stored in the memory 110, and efficiently searches for an image of interest based on the query image. The similarity search module 114 further includes an image classification module 116 that generates a plurality of classifiers and sub-classifiers to classify the query images into at least one category. The feature extraction unit 118 extracts features from the image. The feature extraction unit is known to those skilled in the art and extracts features such as texture, line direction, and edge, but is not limited thereto. In one embodiment, the classifier includes a pattern recognition function that classifies the query image based on the extracted features.

  The similarity search module 114 further includes an image search module 119 that includes a plurality of image search units configured to search each of the image subsets of the image database 122. Each image search unit determines an image of interest from the query image using similarity measurements.

  The keyword tagging unit 120 tags features to each image in the database. In one embodiment, the keyword tagging unit 120 includes a dictionary of N keywords, and the keyword tagging unit 120 is used to generate a feature vector from the keywords. The tagged features can be used to store images in multiple subsets. Further, in one embodiment, the image classification module 116 generates a classifier using keywords.

  Furthermore, the similarity search module 114 includes an object recognition unit 121 for recognizing an object (target) in an image in the database. By using the recognized object, the image classification module 116 can learn from the object and build a classifier based on the object.

  FIG. 2 is a flow diagram of an exemplary method for similarity search of images using a classification data structure such as, but not limited to, a classification search tree, in accordance with an aspect according to the present disclosure. First, in step 202, a classification search tree described in detail below is constructed. Next, in step 204, the post-processing device 102 obtains at least one two-dimensional (2D) image, such as a query image. The post-processing device 102 can obtain a query image by obtaining a digital image file in a computer-readable format, for example via a consumer camera. Although the technique according to the present disclosure has been described with respect to images, successive images, for example moving images such as videos, can also utilize the technique according to the present disclosure. A digital video file can be obtained by capturing a series of temporary movies with a digital camera. Alternatively, a video sequence can be captured by a normal film type camera. In this case, the film is scanned via the scanning device 103. In step 206, the query image is classified by the classifier and subsequently classified by the sub-classifier in step 208 until it reaches the lowermost layer or branch of the tree. In step 210, a search by the search unit is performed within the image subset of the database 122 rather than against the entire image space or database. Details of building or generating a classified search tree and searching within the tree are described below.

  The system and method according to the present disclosure employs a tree-type search to limit image comparison within a small subset of the database. The tree-type search is based on the image classification described below. A classification tree is constructed by automatically or manually tagging images with keywords.

The system and method according to the present disclosure speeds up the search process by limiting the search for images of interest along the branches of the classified search tree. In performing the search, it is assumed that a high-precision similarity measure S (I _q , I _d ) is used, where I _q is a query image and I _d is an image in the database. A similarity measure is a number that indicates the similarity of two images, for example 1.0 means that the two images are the same, 0.0 means that the two images are completely different. . Usually distance is considered as a similar reciprocal. One similar example is the reciprocal of the distance between the color diagrams of two images. Similarity measurements are known to those skilled in the art, and image similarity measurements are “learnable” for an image category, and similarity searches can be optimized within that category. This similarity measure may be designed manually for a certain image category. In either case, a similar measurement is applied to image category C and is represented as S _C (I _q , I _d ).

  A classification search tree is a tree for each intermediate node in the tree to detect or classify one or more categories in the image using a classifier. Each branch in the tree represents a category. Only the branches of the detected category are then traversed to the tree. As shown in FIG. 3, each leaf node 302, 304, 306, 308, 310 in the tree represents an image corresponding to a particular category. A classification search tree can have multiple layers and levels. For example, the tree in FIG. 3 has three levels. Further, as shown in FIG. 3, the classification search tree includes a classifier and a searcher.

  The classifier is used to classify query images into categories. In one embodiment, the classifier is a function based on pattern recognition or device learning algorithms or automatically extracted features such as color and texture. The general procedure for classification is as follows. First, a feature vector is extracted from the image, then a pattern recognition algorithm or function obtains the feature vector, and one or more class labels are selected with a selected reliability score (eg, class ID) representing one or more certain image categories. And score). In general, the pattern recognition algorithm takes a feature vector as input and outputs an integer indicating a class ID, or the pattern recognition function compares the extracted vector with a stored vector. Other pattern recognition algorithms or functions are known to those skilled in the art. The classifier can also be binary. In this case, the classifier outputs a yes label or a no label indicating whether each image belongs to a certain category. The classifier can be automatically constructed from manual design or exemplary data.

  A searcher is a program used to calculate the similarity of images and find the image of interest that most closely resembles the query image.

In the case of simple classification search, if the query images are classified into one and the leaf category is category C, there is only one category at each level. After the classification is completed, i.e., after the query image reaches the bottom (leaf layer) of the classification search tree, a similarity measure S _C (I _q , I _d ) calculation is performed, as shown in FIG. Search for images that fall within the subset of the database corresponding to category C. In FIG. 4 and the remaining figures, branches or leaf nodes traversed during the search are indicated by solid lines, and classifiers and searchers that are not traversed are indicated by dashed lines. For example, in FIG. 4, a query image is received and output to the classifier 0. In the classifier 0, it is determined whether this image is further classified by the classifier 0.1, which is a sub-classifier, for example. From the classifier 0.1, the query image is output to the classifier 0.1.1.1, where the image subset 0.1.1.2 is searched by a searcher 0.0.1 to search for images similar to the query image. It is determined whether 1.1.2 is used. It will be appreciated that by limiting the search for images of interest to the image subset 0.1.1.2, the search is performed more efficiently and quickly.

  In this case, the output of the classifier can be binary or n-variable. In the case of a binary classifier, the output of this classifier indicates whether or not the query image belongs to a category. Similarly, in the case of an n-variable classifier, the output of this classifier can be an integer value indicating which category the query image belongs to. If all of the classifiers in the classification search tree are binary, the tree can be a binary tree, otherwise it can be a non-binary classification searcher.

  One challenge of a simple classification search is that if the classification is incorrect, the query image can go to a completely wrong category, resulting in an incorrect search result. This problem can be solved by a redundant search that allows multiple categories to be searched rather than a single category.

Referring to FIG. 5, in the case of redundant classification search, the query image is classified into one or more leaf categories, for example, classifier 0.1 and classifier 0.2. After classification of completion, namely, their category in the bottom (leaf layer) of the query image is classified search tree, after reaching the example classifier 0.1.1 and classifier 0.2, similar measurements S _{C (I q 1} , I _d ) calculation is performed, for example, as shown in FIG. 5, searcher 0.1.1.2 searches image subset 0.1.1.2, and searcher 0.2.2.1 The image is searched in the range of the subset of the database corresponding to the selected image category C by searching the image subset 0.2.2.1.

  In order to implement a redundant classification search, the output of the classifier needs to be a list of class labels and a list of floating values representing the reliability corresponding to the categories present in the query image. A thresholding procedure can then be used to obtain a list of categories for which the classifier output is greater than the threshold. The query image is made to belong to the resulting list of categories. After arriving at the bottom level of the tree, a similarity score for each image from the list of categories is determined, and then the image with the highest similarity score is selected as the image of interest.

  In order to enable efficient search for images, a classification search tree is constructed to structure the image space so that not every image is searched each time. Referring to FIG. 6, building or generating a classification search tree includes two stages. In the first stage, all branches of the tree are built, and if the classification search tree has multiple layers, all classifiers are built and the classifiers are organized into trees. In the second stage, the images in the database are grouped into categories to form a subset of the images in the database. In addition, a searcher is defined to search within each range of the subset of images.

  In order to build a classification search tree, a classifier for intermediate nodes in the tree must first be built. Each classifier corresponds to one semantic class (for example, outdoor scene, tree, human face, etc.). This semantic class can be determined manually by a human or can automatically use a clustering function or algorithm. The relationship between classifiers (ie, tree structure) can be defined by human design.

  Once a semantic class is defined, a semantic classifier is built for intermediate nodes, eg, sub-classifiers 304, 306, 308, 310. Each classifier or sub-classifier can be constructed in a different procedure, one at a time. In one embodiment, a “global” classifier is provided, and then the “global” classifier learns from the exemplary images for each image category. This procedure allows the system and method according to the present disclosure to build many semantic classifiers without having to design each classifier specifically. This type of classifier is called learning-based scene or object recognition. An exemplary learning scene or object recognition is disclosed in R. Fergus, P. Perona, and A. Zisserman in Object Class Recognition by Unsupervised Scale-Invariant Learning ", Proc. Of the IEEE Conf on Computer Vision and Pattern Recognition 2003. The article by Fergus et al. Describes how to learn from an unlabeled and unsegmented scattered scene and recognize an object class model in a scale-invariant manner. The object is modeled as a flexible set of parts, and a probabilistic representation is used for all object aspects, forms, appearances, closures and relative scales. Used to select the region and scale of the scale, and in learning, the scale invariant object model Is evaluated. This is in. The recognition performed by using the expectation maximization in the maximum likelihood setting, this method has been used for Beishian side to classify the images.

  Another way to define and build classifiers is to use “keyword tagging” by image users. Regarding “keyword tagging”, the image user manually assigns keywords such as “tree”, “face”, and “blue sky” to the image. These manually tagged keywords take into account the types of image features and can therefore be used for classification purposes. For example, once a keyword spot classifier is constructed and a keyword with a classifier is found, the images can be classified into a class. For higher performance, tagged keywords are treated as a kind of feature and converted to a feature vector. This is realized by a technique used for image retrieval called “term vector” (term vector). Basically, a dictionary with N keywords is built and an N-dimensional keyword feature vector is assigned to the image for each image tagged with the keyword. If the image is tagged with the i-th keyword in the dictionary, then the i-th element of the term vector is assigned 1 or 0. As a result, a term vector for each image is provided to represent the meaning of the image. This term vector can be concatenated with the feature vectors described above to form a new feature vector for image classification as shown in FIG.

  For each image subset, an image searcher is manually designed or learned. An image searcher is used to perform a similarity search within a subset of the database.

  After the classifier is defined and built, each image in the database is classified into a subset. The method of building a subset of images is very similar to the classification-retrieval process. When an image is entered into the database, it is automatically classified into a classification tree until it reaches the bottom level of the classification tree where the image is placed in the image pool corresponding to one of the bottom level classifiers as shown in FIG. Is done.

  A potential challenge is that the image contains more than one object, for example an image that contains people and trees. For example, if there are two semantic classes in the classification tree, such as “people” and “trees”, there may be ambiguity in classifying images into one class. This problem can be solved by the above-described redundancy classification. That is, the input image is classified into two subsets.

Although embodiments incorporating teachings according to the present disclosure are described in detail herein, those skilled in the art can readily devise many other various embodiments that also incorporate these teachings. Having described preferred embodiments of a system and method for efficient semantic similarity search of images in a categorized search tree (but not limited to those described), modifications and variations by those skilled in the art in light of the above teachings Note that is possible. Thus, it will be understood that changes may be made in the particular embodiments of the disclosed disclosure within the scope of the disclosure described by the appended claims.

Claims (26)

A method for searching a plurality of images for an image of interest,
Building a classification structure for the plurality of images including at least two image categories representing a subset of the plurality of images;
Receiving a query image;
Categorizing the query image to select one of the at least two image categories;
Limiting the search to a selected one of the at least two image categories with respect to an image of the image of interest.   The method of claim 1, wherein the classification structure is a semantic classification search tree. The step of classifying the query image includes:
Extracting features from the query image;
The method of claim 1, comprising identifying one of the at least two image categories based on extracted features.   The method of claim 1, wherein the step of classifying the query image is performed by a pattern recognition function. Building the classification structure includes determining a classifier for each of the image categories;
The method of claim 1, wherein the classifier classifies images into one of the at least two image categories.   The method of claim 5, wherein the step of determining the classifier is performed by using a clustering function for the plurality of images.   6. The method of claim 5, further comprising determining at least one sub-classifier for each of the determined classifiers. Classifying each of the plurality of images based on the determined classifier;
6. The method of claim 5, further comprising storing each of the plurality of images in at least one of the subset of the plurality of images. Building the classification structure comprises:
Tagging each image of the plurality of images with a feature keyword;
Storing each of the plurality of images in at least one of the subsets of the plurality of images based on the feature keyword.   The method of claim 9, further comprising determining a classifier for each of the image categories based on the feature keywords. Building the classification structure comprises:
Recognizing an object from each of the plurality of images of the at least two image categories;
Determining a classifier for each of the image categories based on the recognized object of each image;
Further including
The method of claim 1, wherein the classifier classifies images into one of the at least two categories.   The method of claim 1, wherein the search for the image of interest is performed by similarity measurement. Classifying the query image into at least two of the at least two image categories;
Searching the image of interest using the query image in the at least two image categories;
Determining a similarity score for each of the images found in each of the at least two categories;
The method of claim 1, further comprising: selecting the image with the highest similarity score as the image of interest. A system for searching a plurality of images related to an image of interest,
A database comprising the plurality of images structured into at least two of the image semantic categories representing a subset of the plurality of images;
Means for obtaining at least one query image;
An image classification module for classifying the query image and selecting one of the at least two image semantic categories;
An image search module for searching for the image of interest using the query image;
With
The system is characterized in that the search is limited to a selected one of at least two of the image semantic categories.   The image processing module further comprises a feature extraction unit that extracts features from the query image, and the image classification module determines one of at least two image semantic categories based on the extracted features. Item 15. The system according to Item 14.   The system of claim 14, wherein the image classification module includes a pattern recognition function.   Means for constructing a semantic classification search tree including a classifier for each of the image semantic categories, wherein the classifier classifies images into one of at least two of the image semantic categories. The system according to claim 14.   The system of claim 17, wherein the image classification module determines the classifier by applying a clustering function to the plurality of images.   The system of claim 17, wherein the image classification module determines a sub-classifier for each determined classifier.   The image classification module classifies each of the plurality of images based on the determined classifier, and stores each of the plurality of images in a subset of the plurality of images in the database. The system according to claim 17.   Further comprising a keyword tagging unit that tags each image of the plurality of images with a feature keyword and stores each of the plurality of images in a subset of the plurality of images of the database based on the feature keyword. The system of claim 17.   The system of claim 21, wherein the image classification module determines the classifier for each of the image semantic categories based on the feature keywords.   An object identification unit for identifying an object from each of the plurality of images of at least two image semantic categories, wherein the image classification module determines a classifier for the image semantic category based on the recognized objects of each image; 18. The system of claim 17, wherein:   The system of claim 14, wherein the image retrieval module includes a similarity measure.   The image classification module classifies the query image into at least two of the at least two image semantic categories, and the image search module uses the query image to determine the interest in at least two of the image semantic categories. Searching for images, determining a similarity score for each image found in each of at least two of the image semantic categories, and selecting the image with the highest similarity score as the image of interest The system according to claim 14. A machine readable program storage device that specifically executes program instructions executable by a machine to perform a plurality of method steps for retrieving a plurality of images for an image of interest,
The method
Building a classification structure for the plurality of images including at least two image categories representing a subset of the plurality of images;
Receiving a query image;
Classifying the query images and selecting one of at least two image categories;
Limiting the search for the image of interest to a selected one of the at least two image categories;
A method comprising the steps of:

Images