JP5598925B2 - Retrieval device and program for retrieving high-dimensional feature vectors with high accuracy - Google Patents

Description

  The present invention relates to a technique for accurately retrieving reference content similar to a query content (search key) similarly represented by a set of feature vectors from a set of reference content represented by a set of feature vectors. In particular, it is suitable for searching multimedia contents (for example, images) represented by a set of high-dimensional feature vectors.

  In recent years, not limited to online / offline, it has become possible to accumulate a large amount of content as the capacity of the storage increases. In addition, with the widespread use of information terminal devices typified by mobile phones and smartphones, digital content such as photograph data acquired by the user can be easily stored in a large amount in a database. Offline databases include storage devices such as HDD (Hard Disk Drive), DVD (Digital Versatile Disk), and Blu-ray disc. Online databases include social network services such as Flickr (registered trademark) and MySpace (registered trademark). According to these storage devices and services, a technique for searching for a large amount and various multimedia contents of individuals stored in a database becomes important.

  In order to search for multimedia contents, there is a technique for extracting a large number of feature vectors from these contents and outputting contents having a high degree of similarity between sets of feature vectors as search results. According to this technique, feature vectors of multimedia content are quantized and a histogram is created from the frequency of the quantized feature vectors. The similarity (distance) is calculated by the distance of the L1 norm or L2 norm between the histograms. The norm represents the distance between two points. The L1 norm means the sum of the absolute values of the dimensions of the two points, and the L2 norm means the sum of the squares of the values of the two points.

  There is also a technique for extracting a large amount of local feature vectors from image content, vector quantizing them, and calculating the similarity based on the number of local feature vectors vector-quantized to the same representative vector (for example, Non-Patent Document 1). reference).

  Furthermore, there is a technique for extracting a plurality of local invariant feature amounts from an image, making a histogram of the frequency of feature vectors, and calculating the similarity between the image and the category based on the overlapping ratio of the histograms (see, for example, Patent Document 1) . According to this technique, features (for example, background features) that are not necessary for pattern recognition of a subject can be removed based on the histogram. As a result, the feature of the object can be extracted without previously separating the object and the non-object from the image.

On the other hand, according to the prior art described above, the original feature vector information is lost due to quantization. The following problems arise due to the roughness of the quantization.
(Error detection) When roughly quantized, different feature vectors are likely to be quantized into representative vectors having the same identification value (ID (IDentifier)). This increases false detection of determining that different content is similar.
(Undetected) Conversely, when fine quantization is performed, there is a high possibility that similar feature vectors are quantized into representative vectors having different IDs. This increases non-detection that similar content is not determined to be similar.

  In order to solve such a problem, there is a technique for improving the trade-off between two types of errors that are erroneously detected and undetected (for example, Non-Patent Documents 2 and 3). According to this technique, the feature vector is encoded in advance into a compact code. Then, the distance between feature vectors having the same ID by vector quantization is approximated by the distance between codes. Only feature vectors whose distance is less than or equal to a predetermined threshold value contribute to the similarity.

JP 2010-282581 A JP 2009-020769A

J. Sivic et al., "Video Google: A Text Retrieval Approach to Object Matching in Videos," in Proc. ICCV, 2003. H. Jegou, M. Douze, and C. Schmid, "Improving bag-offeatures for large scale image search," in IJCV, vol.87, no.3, pp.316-336, 2010. Y. Uchida, M. Agrawal, and S. Sakazawa, "Accurate Content-Based Video Copy Detection with Efficient Feature Indexing," in Proc. Of ICMR, 2011. D. G. Lowe, "Distinctive Image Features from Scale-InvariantKeypoints", International Journal of Computer Vision, vol. 60, no. 2, pp.91-110, 2004. H. Jegou, M. Douze, and C. Schmid, "Product quantization fornearest neighbor search," in IEEE Trans. On PAMI, vol. 33, no. 1, pp117-128, 2011.

  However, according to the techniques described in Non-Patent Documents 2 and 3 described above, since it is determined whether or not all quantization IDs contribute to similarity by the same predetermined threshold, the quantization ID The density of each feature vector is not considered at all. For this reason, in one quantization ID, the feature vector contributes excessively to the similarity, while in another quantization ID, the feature vector contributing to the similarity decreases.

  Therefore, according to the present invention, when searching for a high-dimensional feature vector, by considering the density of feature vectors for each quantization ID, the unfairness in determining the similarity between the quantization IDs can be reduced. An object of the present invention is to provide a search device and a program that can be relaxed.

According to the present invention, there is provided a search device for searching reference content similar to query content represented by a set of feature vectors from a set of reference content represented by a set of feature vectors,
Reference information storage means for storing a plurality of combinations of reference content identification values and reference feature vectors for each reference hash identification value;
For query content, hashing means for deriving one or more unique query hash identification values using one or more hash functions for each query feature vector;
Distance calculating means for calculating a distance between each reference feature vector corresponding to the query hash identification value and the query feature vector using the reference information storage means;
Distance sorting means for sorting all reference content identification values corresponding to query hash identification values in ascending order of distance;
Reference content identification value extraction means for extracting only reference content identification values included in the upper threshold ratio (%) among the sorted reference content identification values;
It has a similar search means for searching for the most similar reference content from among a plurality of reference contents including a reference feature vector corresponding to the extracted reference content identification value.

According to another embodiment of the search device of the present invention,
The reference information storage means stores a reference code for the reference feature vector instead of the reference feature vector,
The hashing means further derives one or more unique reference hash identification values using one or more hash functions for each of a number of reference feature vectors;
Encoding means for encoding a reference feature vector into a reference code and storing it in a reference information storage means;
The distance calculation means preferably calculates the distance between each reference code corresponding to the query hash identification value and the query feature vector using the reference information storage means.

  According to another embodiment of the search device of the present invention, the similarity search means votes a plurality of reference content identification values derived for each of a plurality of query feature vectors included in the query content for each reference content identification value. It is also preferable to extract a reference content identification value corresponding to the number of votes.

According to another embodiment of the search device of the present invention,
The reference information accumulation means accumulates the Lp norm of the frequency of the reference hash identification value for each reference content,
The similarity search means preferably calculates, as the similarity, a value obtained by normalizing the number of votes obtained for each reference content with the Lp norm of the corresponding reference content.

According to another embodiment of the search device of the present invention,
The encoding means performs a projective transformation on the reference feature vector and the query feature vector, and generates a binary string reference code and query code by binarizing the value of each dimension after conversion with a threshold value,
It is also preferable that the distance calculation means calculates a Hamming distance between the reference code and the query code.

According to another embodiment of the search device of the present invention,
The encoding means is
The representative vector corresponding to the hash identification value is accumulated,
It is also preferable to calculate a reference difference vector between the reference feature vector and a representative vector corresponding to the reference hash identification value of the reference feature vector, and encode the reference difference vector into a reference code.

  According to another embodiment of the search device of the present invention, the representative vector is preferably calculated from an average vector or a median vector of a number of feature vectors having the hash identification value.

According to another embodiment of the search device of the present invention,
The encoding means is
A codebook for product quantization,
The reference difference vector is encoded into a reference code by direct product quantization using a codebook,
A query difference vector between the query feature vector and a representative vector corresponding to the query hash identification value,
It is also preferable that the distance calculation means calculates an Lp distance between the reference code and the query difference vector.

  According to another embodiment of the search device of the present invention, the hashing means further includes a codebook for vector quantization, and the hash function performs vector quantization to thereby obtain a reference hash identification value and a query hash identification value. It is also preferable to calculate.

  According to another embodiment of the search device of the present invention, the hashing unit discriminates the value of the inner product of the reference feature vector and the query feature vector and a predetermined vector of the same dimension with a predetermined threshold value, thereby making the reference It is also preferable to calculate a hash identification value and a query hash identification value.

  According to another embodiment of the search device of the present invention, the vector is preferably a local feature vector extracted from a local feature region of an image.

According to the present invention, a program that causes a computer to function so as to search for reference content similar to query content represented by a set of feature vectors from a set of reference content represented by a set of feature vectors,
Reference information storage means for storing a plurality of combinations of reference content identification values and reference feature vectors for each reference hash identification value;
For query content, hashing means for deriving one or more unique query hash identification values using one or more hash functions for each query feature vector;
Distance calculating means for calculating a distance between each reference feature vector corresponding to the query hash identification value and the query feature vector using the reference information storage means;
Distance sorting means for sorting all reference content identification values corresponding to query hash identification values in ascending order of distance;
Reference content identification value extraction means for extracting only reference content identification values included in the upper threshold ratio (%) among the sorted reference content identification values;
The computer is caused to function as a similarity search means for searching for the most similar reference content from among a plurality of reference contents including a reference feature vector corresponding to the extracted reference content identification value.

  According to the search device and program of the present invention, when searching for high-dimensional feature vectors, the density of the similarity between quantization IDs is determined by considering the density of feature vectors for each quantization ID. Fairness can be eased.

It is explanatory drawing showing the clustering with respect to the set of a reference feature vector. It is a 1st functional block diagram of the search device in this invention. It is the 1st explanatory view showing the data structure in the present invention. It is explanatory drawing showing the function structure of a hash part. It is a graph showing the vote of reference content ID. It is a 2nd function block diagram of the search device in this invention. It is the 2nd explanatory view showing the data structure in the present invention. It is a function block diagram of the encoding part in this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  According to the search device and program of the present invention, the reference content that is most similar to the query content (content of the search key) is searched from among a large number of reference contents (contents to be searched). For searching, a set of feature vectors is extracted from the multimedia content. That is, one set of feature vectors corresponds to one multimedia content. Specifically, the reference content includes a set of a plurality of reference feature vectors, and similarly, the query content includes a set of a plurality of query feature vectors. The reference feature vector and the query feature vector are vectors having the same number of dimensions.

  When the multimedia content is an image, the feature vector is a local feature vector extracted from the local feature region of the image.

  FIG. 1 is an explanatory diagram showing clustering for a set of reference feature vectors.

According to FIG. 1A, a large number of reference feature vectors are clustered into, for example, N clusters Fn (n = 1 to N). For clustering, an algorithm such as k-means or k-means ++ is used. Each cluster includes a number of reference feature vectors. When a query feature vector is included in the cluster, a reference feature vector belonging to a distance range of a predetermined threshold is derived from the query feature vector as a candidate. According to FIG. 1, it is expressed as follows.
×: Reference feature vector +: Representative vector in cluster (mean vector or median vector)
Black circle ●: Query feature vector Wavy line ○: Distance range from the query feature vector to a predetermined threshold

  For example, it is assumed that the query feature vector a is included in the cluster F1 and the query feature vector b is included in the cluster F3. According to the conventional technique, the similarity is calculated from the query feature vector by the number of reference feature vectors belonging to a predetermined threshold distance range.

However, depending on the cell range of the cluster, it is divided as follows.
When the cell range of the cluster is small, the feature vector distribution density is high. In this case, it is determined from the query feature vector that the number of reference feature vectors belonging to the predetermined threshold range is large and the degree of similarity is high.
When the cell range of the cluster is large, the distribution density of feature vectors is low. In this case, it is determined from the query feature vector that the number of reference feature vectors belonging to the predetermined threshold range is small and the similarity is low.

  On the other hand, according to the present invention based on FIG. 1B, only the reference feature vector included in the upper threshold ratio P (%) is extracted from the reference feature vectors in the cluster. For only the extracted reference feature vector, it is determined that the higher the number belonging to the predetermined threshold range from the query feature vector, the higher the similarity. That is, according to the present invention, the reference feature vector to be determined differs depending on the cell range of the cluster (reference feature vector distribution).

Below, two types of embodiment from which the comparison object for a similar search differs about query content and reference content is described.
[First Embodiment] The query feature vector of the query content and the reference feature vector of the reference content are compared to determine the similarity.
Second Embodiment A query code (or query difference vector) in a query feature vector of query content and a reference code in a reference feature vector of reference content are compared to determine the similarity.

[First Embodiment]
FIG. 2 is a first functional configuration diagram of the search device according to the present invention.
FIG. 3 is a first explanatory diagram showing a data structure in the present invention.

  According to FIG. 2, the search device 1 includes a reference information storage unit 100, a feature vector set extraction unit 111, a hashing unit 112, a distance calculation unit 113, a distance sorting unit 114, and a reference content ID extraction unit 115. And a similarity search unit 116. These functional components are realized by executing a program that causes a computer installed in the apparatus to function.

  The search device 1 inputs a large number of reference contents in advance, and stores information related to the reference contents in the reference information storage unit 100. In addition, the search device 1 inputs query content serving as a search key at the time of search, and uses the reference information storage unit 100 to search for reference content that is most similar to the query content.

The feature vector set extraction unit 111 extracts a set of feature vectors from the multimedia content. Here, a set of a plurality of reference feature vectors is extracted from the reference content, and similarly, a set of a plurality of query feature vectors is extracted from the query content (see 111 in FIG. 3). For example, according to the SIFT (Scale-Invariant Feature Transform) algorithm, a set of 128-dimensional (D = 128) feature vectors is extracted from one image (see, for example, Non-Patent Document 4). SIFT is a technique for analyzing a characteristic local region using a scale space and describing a feature vector that is invariant to scale change and rotation.
Feature vector f: f = (f ₁ , f ₂ ,..., F _D )
The extracted reference feature vector is output to the hashing unit 112 and the reference information storage unit 100. Further, the extracted query feature vector is output to the hashing unit 112 and the distance calculation unit 113.

  The hashing unit 112 derives one or more unique reference hash IDs (identification values) for the reference content for each reference feature vector using one or more hash functions. Similarly, for the query content, one or more unique query hash IDs are derived for each query feature vector using one or more hash functions (see 112 in FIG. 3).

  FIG. 4 is an explanatory diagram illustrating a functional configuration of the hashing unit.

  The hashing unit 112 uses one of the following two processes. The same hashing process is performed on the reference feature vector and the query feature vector.

(First Hash Process) The hash unit 112 has a vector book for vector quantization, and executes vector quantization on the feature vector. Thereby, a reference hash ID and a query hash ID are calculated.

Here, the codebook to quantize the feature vector f, a hash IDn (= 1 to N), in which associates the representative vector f _n.
Hash IDn: representative vector f _n
1: f ₁ = (f ₁₁ , f ₁₂ ,..., F _1D )
2: f ₂ = (f ₂₁ , f ₂₂ ,..., F _2D )
3: f ₃ = (f ₃₁ , f ₃₂ ,..., F _3D )
...
N: f _N = (f _N1 , f _N2 ,..., F _ND )

The code book is generated by performing the following steps (see, for example, FIG. 1).
(S1) A set of reference feature vectors is clustered into N clusters.
(S2) Next, a representative vector is derived for each cluster (average vector or median vector).
(S3) A code book in which a unique hash IDn (= 1 to N) is assigned to each representative vector is generated.

According to the first hashing process, a representative vector f _n ′ having the smallest distance from the input feature vector f is calculated, and an identification value ID _n ′ is assigned to the feature vector f.
n = q (f) = argmin _n || f−f _n || ²
(The quantization function q (f) of the registration vector f derives n where || f−f _n || ² is minimized)
q: R ^D- > N (means quantization)
f _n : representative vector

At this time, instead of assigning one IDn ′ having the smallest distance to the feature vector, a plurality of higher IDs in the representative vector having the smallest distance may be assigned. Alternatively, a plurality of different code books may be held, and an ID may be individually assigned to each code book. Through these processes, one or more hash IDs are assigned to each of the D-dimensional feature vector sets f _{1 to} f _K.

(Second Hashing Process) The hashing unit 112 discretizes the inner product value of the reference feature vector and the query feature vector and a predetermined vector of the same dimension with a predetermined threshold value, thereby obtaining the reference hash ID and the query hash. ID is calculated.

  The hashing unit 112 may use a hash function used in LSH (Locality-Sensitive Hashing), which is one of the approximate nearest neighbor algorithms. According to LSH, hash IDs are output so that the hash IDs of similar data match and the hash IDs of dissimilar data are likely to be different.

  The reference hash ID obtained by hashing the reference feature vector is output to the reference information storage unit 100. The query hash ID obtained by hashing the query feature vector is output to the distance calculation unit 113.

  The reference information storage unit 100 stores a plurality of combinations of reference content IDs and reference feature vectors for each reference hash ID (see 100 in FIG. 3). That is, the reference content ID and the reference feature vector are listed for each reference hash ID.

  Returning to FIG. 2 again, the distance calculation unit 113 refers to a list of reference feature vectors corresponding to the query hash IDik in the query feature vector when the reference information storage unit 100 stores the reference feature vector. Then, the distance between each reference feature vector and the query feature vector is calculated (see 113 in FIG. 3). Information on the calculated distance is output to the distance sorting unit 114.

  The distance sorting unit 114 sorts all reference content IDs corresponding to the query hash ID in ascending order of distance. Sorted for each pair of reference content ID and distance.

  The reference content ID extraction unit 115 extracts only the reference content ID group included in the upper threshold ratio P (%) from the entire sorted reference content ID (see 114 in FIG. 3).

  The similarity search unit 116 determines the similarity between the query content (a set of query feature vectors) and the reference content (a set of reference feature vectors). The most similar reference content is searched from among a plurality of reference contents including a reference feature vector corresponding to the reference content ID extracted by the reference content ID extraction unit 115.

  Here, the similarity search unit 116 votes, for each reference content ID, a plurality of reference content IDs derived for each of the plurality of query feature vectors included in the query content, and determines a reference content ID corresponding to the number of votes. Extract.

  FIG. 5 is a graph showing the vote of the reference content ID.

First, for the reference content ID group extracted by the reference content ID extraction unit 115, the number of each reference content ID is counted for each query hash ID of the query feature vector (see FIG. 5A). For example, assume that query hash IDs 1 to 3 are derived for each query feature vector for one query content. A reference content ID is extracted for each query hash ID.
Crie hash ID Reference content ID
1-> 5, 6
2-> 1, 4, 5
3-> 1,5
Here, the appearance frequency of the reference content ID is expressed as follows.
Reference content ID appearance frequency
1-> 2
2-> 0
3-> 0
4-> 1
5-> 3 (local maximum number of votes)
6-> 1

  Next, the total number for each reference content ID is normalized by a normalization term. Here, the normalization term is calculated as follows.

When the hash ID is ID1 to 4, the reference information storage unit 100 stores the appearance frequency of each hash ID for each reference content (see FIG. 5B). For example, the appearance frequency of the reference hash ID in the reference content ID1 is as follows.
[For reference content ID1]
Reference hash ID appearance frequency
1-> 1
2-> 3
3-> 0
4-> 1

Then, according to the appearance frequency for each reference hash ID, two types of normalization terms are calculated as follows. Hereinafter, only the reference hash ID1 will be described.
L1 norm of reference hash ID1 1 + 3 + 0 + 1 = 5
In the case of the L2 norm of the reference hash ID 1 √ (1 ² +3 ² +0 ² +1 ² ) = √ (1 + 9 + 0 + 1) = √11

Finally, the similarity of the reference content to the query content is expressed as follows (see FIG. 5C).
Reference content ID L1 norm similarity L2 norm similarity
1-> 2/5 2 / √11
2-> 0/9 0 / √9
3-> 0/11 0 / √11
4-> 1/8 1 / √8
5-> 3/9 3 / √9
6-> 1/8 1 / √8

  In this manner, the similarity search unit 116 calculates a value obtained by normalizing the number of votes for each reference content with the Lp norm of the corresponding reference content as the similarity. Then, the reference content ID having the highest similarity is output as a search result.

[Second Embodiment]
FIG. 6 is a second functional configuration diagram of the search device according to the present invention.
FIG. 7 is a second explanatory diagram showing a data structure in the present invention.

  According to the first embodiment described above, the reference information storage unit 100 stores a plurality of combinations of reference content IDs and reference feature vectors for each reference hash ID. On the other hand, according to the second embodiment, the reference information storage unit 100 stores a plurality of combinations of the reference content ID and the reference code for the reference feature vector for each reference hash ID. The second accumulation information accumulates a reference code instead of the reference feature vector of the first accumulation information.

  According to FIG. 6, compared with FIG. 2, it further has the encoding part 117 which encodes a reference feature vector and a query feature vector into a reference code and a query code (or query difference vector).

  According to FIG. 7, the reference information storage unit 100 stores a reference code for each reference content ID (see 100 in FIG. 7). The distance calculation unit 113 calculates a distance between the reference code and the query code (or query difference vector) (see 113 in FIG. 7). Except for these points, it is exactly the same as FIG. 3 described above.

  FIG. 8 is a functional configuration diagram of the encoding unit in the present invention.

  The encoding unit 117 executes either one of the two main types of encoding processes. The same encoding process is performed on the reference feature vector and the query feature vector.

(First encoding process)
The encoding unit 117 receives the reference feature vector and the query feature vector output from the feature vector set extraction unit 111. Next, for each feature vector fk, a difference vector rk from the representative vector gik corresponding to the assigned hash IDik is calculated (rk = fk−gik). Then, the projective transformation is executed on the difference vector rk by random orthogonal projection (Random Orthogonal Projection). Next, a binary string reference code and a query code are generated by binarizing the value of each dimension after conversion with a threshold value. In the case of direct product quantization, the difference vector rk is divided into a plurality of partial vectors, each of which is quantized with an individual codebook, and a sequence of quantization IDs is designated as code ck. The reference code is output to the reference information storage unit 100, and the query code is output to the distance calculation unit 113.

  For such an encoding process, the distance calculation unit 113 calculates a Hamming Embedding of the reference code and the query code (see, for example, Non-Patent Document 2).

(Second encoding process)
The encoding unit 117 stores a representative vector corresponding to the hash ID. The representative vector may be calculated from an average vector or median vector of a number of feature vectors having the hash ID. Then, the encoding unit 117 calculates a reference difference vector between the reference feature vector and a representative vector corresponding to the reference hash ID of the reference feature vector, and encodes the reference difference vector into a reference code.

  The encoding unit 117 further includes a code book for direct product quantization. Then, the reference difference vector is encoded into a reference code by direct product quantization using a code book (see, for example, Non-Patent Document 3). In addition, the encoding unit 117 calculates a query difference vector between the query feature vector and the representative vector corresponding to the query hash ID. The reference code is output to the reference information storage unit 100, and the query difference vector is output to the distance calculation unit 113.

  For such an encoding process, the distance calculation unit 113 calculates the Lp distance between the reference code and the query difference vector (see, for example, Non-Patent Documents 4 and 5).

  In both the first encoding process and the second encoding process, an average vector of reference feature vectors to which each hash ID is assigned by the hashing unit 112 is calculated in advance from a large amount of reference feature vectors.

  According to the search device and program of the present invention, when searching for high-dimensional feature vectors, the density of the similarity between quantization IDs is determined by considering the density of feature vectors for each quantization ID. Fairness can be eased.

  Various changes, modifications, and omissions of the above-described various embodiments of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

DESCRIPTION OF SYMBOLS 1 Search apparatus 100 Reference information storage part 111 Feature vector set extraction part 112 Hashing part 113 Distance calculation part 114 Distance sort part 115 Reference content ID extraction part 116 Similarity search part 117 Encoding part

Claims (12)

A search device for searching for reference content similar to query content represented by a set of feature vectors from a set of reference content represented by a set of feature vectors,
Reference information storage means for storing a plurality of combinations of reference content identification values and reference feature vectors for each reference hash identification value;
Hashing means for deriving one or more unique query hash identification values for each query feature vector using one or more hash functions for the query content;
Distance calculation means for calculating a distance between each reference feature vector corresponding to the query hash identification value and the query feature vector using the reference information storage means;
Distance sorting means for sorting all the reference content identification values corresponding to the query hash identification values in ascending order of the distance;
Reference content identification value extraction means for extracting only reference content identification values included in the upper threshold ratio (%) among the sorted reference content identification values;
A search apparatus comprising: similar search means for searching for the most similar reference content from among a plurality of reference contents including a reference feature vector corresponding to the extracted reference content identification value. The reference information storage means stores a reference code for the reference feature vector instead of the reference feature vector,
The hashing means further derives one or more unique reference hash identification values using one or more hash functions for each of the plurality of reference feature vectors;
Encoding means for encoding the reference feature vector into a reference code and storing it in the reference information storage means;
The distance calculation unit calculates a distance between each reference code corresponding to the query hash identification value and the query feature vector using the reference information storage unit. Search device.   The similarity search means votes a plurality of reference content identification values derived for each of a plurality of query feature vectors included in the query content for each reference content identification value, and a reference content identification value corresponding to the number of votes The search device according to claim 1, wherein the search device is extracted. The reference information storage means stores an Lp norm of the frequency of the reference hash identification value for each reference content,
4. The search apparatus according to claim 3, wherein the similarity search means calculates a value obtained by normalizing the number of votes obtained for each reference content by the Lp norm of the corresponding reference content as the similarity. The encoding means performs a projective transformation on the reference feature vector and the query feature vector, and generates a binary sequence reference code and query code by binarizing the values of each dimension after conversion with a threshold value. And
The search apparatus according to claim 4, wherein the distance calculation unit calculates a Hamming distance between the reference code and the query code. The encoding means includes
Storing representative vectors corresponding to the hash identification values;
5. A reference difference vector between the reference feature vector and a representative vector corresponding to a reference hash identification value of the reference feature vector is calculated, and the reference difference vector is encoded into a reference code. The search device described in 1.   The search apparatus according to claim 6, wherein the representative vector is calculated from an average vector or a median vector of a number of feature vectors having the hash identification value. The encoding means includes
A codebook for product quantization,
The reference difference vector is encoded into a reference code by direct product quantization using the codebook,
Calculating a query difference vector between the query feature vector and a representative vector corresponding to the query hash identification value;
The search apparatus according to claim 7, wherein the distance calculating unit calculates an Lp distance between the reference code and a query difference vector. The hashing unit further includes a codebook for vector quantization, and the hash function calculates a reference hash identification value and a query hash identification value by performing vector quantization. 9. The search device according to 8. The hashing means calculates a reference hash identification value and a query hash identification value by discretizing a value of an inner product of the reference feature vector and the query feature vector and a predetermined vector of the same dimension with a predetermined threshold. 9. The search device according to claim 5 or 8, wherein:   The search device according to claim 9 or 10, wherein the vector is a local feature vector extracted from a local feature region of an image. A program that causes a computer to function so as to search reference content similar to query content represented by a set of feature vectors from a set of reference content represented by a set of feature vectors,
Reference information storage means for storing a plurality of combinations of reference content identification values and reference feature vectors for each reference hash identification value;
Hashing means for deriving one or more unique query hash identification values for each query feature vector using one or more hash functions for the query content;
Distance calculation means for calculating a distance between each reference feature vector corresponding to the query hash identification value and the query feature vector using the reference information storage means;
Distance sorting means for sorting all the reference content identification values corresponding to the query hash identification values in ascending order of the distance;
Reference content identification value extraction means for extracting only reference content identification values included in the upper threshold ratio (%) among the sorted reference content identification values;
A search program characterized by causing a computer to function as a similarity search means for searching for the most similar reference content from among a plurality of reference contents including a reference feature vector corresponding to the extracted reference content identification value.

Images