JP2011221689A - Learning device, learning method, recognition device, recognition method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To raise recognition accuracy when object recognition using a local feature quantity is performed by LSH.SOLUTION: A hash function is applied to a model feature quantity vector representing the feature quantity of a model feature point extracted from a model image, and the model feature quantity vector is quantized. Quantization is performed so that an inner product value between the model feature quantity vector and a hash function vector is compared with a threshold, and is quantized to one value of two values of 1 and 0. In addition, when the two values are within a quantization error margin which is set to the hash function vector on a feature quantity space, the other value of the two values is also set as a quantization value. The model feature point which has the model feature quantity vector is clustered so as to belong to a plurality of clusters based on the quantization value of the model feature quantity vector. This invention is applicable to a device which performs an object recognition using the local feature quantity.

Description

  The present invention relates to a learning device, a learning method, a recognition device, a recognition method, and a program, and in particular, a learning device capable of improving recognition accuracy when recognition using a local feature amount is performed by LSH, and learning The present invention relates to a method, a recognition device, a recognition method, and a program.

  As an object recognition method for recognizing an object appearing in an image, there is a method in which an object is described with local feature values and matching between local feature values is performed. According to object recognition using local feature amounts, robustness can be ensured even when there is a change in viewpoint or partial occlusion of an object in an image.

  In object recognition using local features, many query feature points and model feature points are matched (nearest neighbor search), so a huge amount of computation is required to determine the similarity between all feature values. become. Here, the query feature point is a feature point extracted from a query image that is an image input as a query at the time of object recognition, and the model feature point is a model that is an image used at the time of learning performed before object recognition. It is a feature point extracted from the image.

  Therefore, in recent years, a method such as LSH (Locality Sensitive Hashing) has been used as a method for speeding up object recognition using local features.

  When LSH is used, at the time of learning, a hash value is calculated by applying a hash function to a model feature amount vector representing a feature amount of a model feature point, and the model feature amount vector is quantized by threshold processing. Each model feature point is clustered according to the value after quantization.

  FIG. 1 is a diagram illustrating an example of quantization of a feature vector.

In the example of FIG. 1, hash function vectors f _{1 to} f ₃ are prepared in advance. The hash function is represented as a vector having the same dimensionality as the feature vector to which the hash function is applied.

For example, as shown in FIG. 1, when a feature vector A is extracted as a model feature vector, an inner product of the feature vector A and each of the hash function vectors f _{1 to} f ₃ is calculated as a hash value. . Depending on whether the calculated inner product value is equal to or greater than the threshold value, the feature vector is quantized to one of 1 and 0.

In the example of FIG. 1, the feature vector A is quantized to a value of 0 by using the hash function vector f ₁ . Further, the feature vector A is quantized to a value 1 by using the hash function vectors f ₂ and f ₃ , respectively.

  In this case, the model feature point whose feature amount is represented by the feature amount vector A is a feature point belonging to a cluster (cluster identified by [0, 1, 1]) having [0, 1, 1] as a key. Clustered.

Similarly, when the feature vector B is extracted as the model feature vector, the inner product of the feature vector B and each of the hash function vectors f _{1 to} f ₃ is calculated.

In the example of FIG. 1, the feature vector B is quantized to a value 0 by using the hash function vector f ₁ . Further, by using the hash function vectors f ₂ and f ₃ , the feature vector B is quantized to a value 0 and a value 1, respectively.

  In this case, the model feature points whose feature amounts are represented by the feature amount vector B are clustered as feature points belonging to a cluster having [0, 0, 1] as a key.

  At the time of learning, all model feature quantity vectors are quantized in this way, and model feature points are clustered based on the quantization result.

  On the other hand, at the time of recognition, a hash value is calculated by applying a hash function to a query feature quantity vector representing a feature quantity of a query feature point by a process similar to the process described with reference to FIG. Quantization of the quantity vector is performed.

  Also, each query feature point is clustered according to the quantized value, the feature vector similarity is calculated with model feature points belonging to the same cluster, and the nearest model feature point is determined. .

  Since clustering is performed, according to LSH, it is possible to achieve a nearest neighbor search approximately with a smaller amount of computation than a perfect nearest neighbor search that calculates the similarity of feature quantities between all feature points. become.

‘Locality-sensitive hashing scheme based on p-stable distributions’, Mayur Datar, Piotr Indyk, Proceedings of the twentieth annual symposium on Computational geometry, pp. 253-262, 2004 Object Recognition Using Locality-Sensitive Hashing of Shape Contexts ’, Andrea Frome, Jitendra Malik, Nearest-Neighbor Methods in Learning and Vision, Eds. G. Shakhnarovich, T. Darrell, P. Indyk, 2005

  In a method of quantizing a feature vector into one of two values of 1 and 0 depending on whether or not a hash value obtained by applying a hash function is greater than or equal to a threshold value, a quantization error is generated. This is likely to occur, and the approximate nearest neighbor search may not be performed correctly.

  FIG. 2 is a diagram for explaining a quantization error of a feature vector.

  For example, the feature quantity vectors A and B in FIG. 2 are feature quantity vectors representing the feature quantities of model feature points extracted from the model image during learning. In the hash function vector f, a hyperplane orthogonal to the hash function vector f (or a straight line orthogonal when the feature amount space is expressed in two dimensions) is set as a threshold value.

  In the example of FIG. 2, the inner product value of the feature quantity vector A and the hash function vector f is 0 or more as a threshold value, and the feature quantity vector A is quantized to a value of 1. Further, the inner product value of the feature vector B and the hash function vector f is less than 0 as a threshold, and the feature vector B is quantized to a value of 0.

  The feature vectors A and B are quantized to different values even though they are close in the feature space. As a result, the model feature point whose feature amount is represented by the feature amount vector A and the model feature point whose feature amount is represented by the feature amount vector B are clustered as feature points belonging to different clusters.

  Since the hash value to be subjected to the threshold processing is the inner product value of the hash function vector and the feature quantity vector, the quantization as described above bisects the feature quantity space by a hyperplane orthogonal to the hash function vector. This corresponds to the process of setting one of the two values 0 and 0.

  Therefore, when there is a feature vector near the cut surface on both sides of the cut surface (when the inner product is close to 0), each feature vector is quantized to a different value even if they are close. As a result, in the feature space, feature points having close feature vectors are clustered as feature points belonging to different clusters, and compared with a complete nearest neighbor search for similarity between all feature points. As a result, matching performance deteriorates.

  The present invention has been made in view of such a situation, and makes it possible to improve recognition accuracy when object recognition using local features is performed by LSH.

  The learning device according to one aspect of the present invention determines a model feature point that is a feature point of input model data, and extracts the feature amount of the model feature point, and represents the feature amount of the model feature point Depending on whether or not the inner product value of the model feature vector and the LSH hash function vector having the same dimension as the model feature vector is equal to or greater than a threshold, the model feature vector is set to 1 and 0. When the model feature vector is quantized to one of binary values and the model feature vector is within the margin set in the hash function vector in the feature space, the other of the binary values of 1 and 0 is used. Quantization means for setting a value as a value after quantization of the model feature quantity vector, and the model feature points are obtained by using the model feature quantity vector obtained by using the respective hash function vectors. The combination of values after quantization Torr and a clustering unit for clustering the feature point belonging to the cluster identified.

  Storage means for storing the information of the model feature vector and the information of the cluster to which the model feature point belongs can be further provided.

  The margin may be set in the hash function vector so as to include a neighborhood of a hyperplane orthogonal to the hash function vector.

  A learning method according to one aspect of the present invention determines a model feature point that is a feature point of input model data, extracts a feature amount of the model feature point, and represents a feature amount of the model feature point Depending on whether the inner product value of the vector and the LSH hash function vector having the same number of dimensions as the model feature vector is equal to or greater than a threshold value, the model feature vector is a binary value of 1 and 0. If one of the values is quantized and the model feature vector is within the margin set in the hash function vector in the feature space, the other of the binary values 1 and 0 is A set of model feature vectors obtained by quantizing the model feature vectors obtained by using the hash function vectors. Comprising the step of clustering the feature point belonging to the cluster identified by Align.

  A program according to one aspect of the present invention determines a model feature point that is a feature point of input model data, extracts a feature amount of the model feature point, and a model feature amount vector representing the feature amount of the model feature point Depending on whether the inner product value of the model feature vector and the LSH hash function vector having the same number of dimensions is greater than or equal to a threshold value, the model feature vector is expressed as one of binary values 1 and 0. When the model feature vector is within the margin set in the hash function vector in the feature space, the other one of the binary values 1 and 0 is Set as a value after quantization of a model feature quantity vector, and the model feature point is a set of values after quantization of the model feature quantity vector obtained by using the respective hash function vectors To execute a process including the step of clustering the feature point belonging to the cluster identified in the computer by mating.

  According to another aspect of the present invention, there is provided a recognition apparatus that determines a query feature point, which is a feature point of data input as a query, and extracts a feature amount of the query feature point, and a feature amount of the query feature point Quantize the query feature vector into one of binary values 1 and 0 depending on whether or not the inner product value of the query feature vector representing the hash function vector is greater than or equal to a threshold value Clustering means for clustering the query feature points as feature points belonging to a cluster identified by a combination of values obtained by quantizing the query feature quantity vectors obtained by using the hash function vectors. , Calculating the similarity between the query feature point and each of the model feature points belonging to the same cluster as the query feature point, and based on the calculated similarity , And a recognition means for determining the model feature point serving as a nearest neighbor of the query feature points.

  A storage unit that stores information on the model feature vector generated by the learning device and information on the cluster to which the model feature point belongs can be further provided.

  According to another aspect of the present invention, there is provided a recognition method that determines a query feature point that is a feature point of data input as a query, extracts a feature amount of the query feature point, and represents a feature amount of the query feature point. The query feature vector is quantized into one of binary values 1 and 0 according to whether or not the inner product value of the feature vector and the hash function vector is greater than or equal to a threshold, and the query feature point As a feature point belonging to a cluster identified by a combination of values after quantization of the query feature vector obtained using the hash function vectors, and the query feature point and the query feature point The similarity between each model feature point belonging to the same cluster is calculated, and based on the calculated similarity, the model feature that is closest to the query feature point Including the step of determining the.

  A program according to another aspect of the present invention determines a query feature point that is a feature point of data input as a query, extracts a feature amount of the query feature point, and represents a feature amount of the query feature point Depending on whether or not the inner product value of the quantity vector and the hash function vector is greater than or equal to a threshold value, the query feature quantity vector is quantized into one of binary values of 1 and 0, and the query feature point is Clustering as feature points belonging to a cluster identified by a combination of values after quantization of the query feature vector obtained using the hash function vectors, the query feature points, and the query feature points, A similarity with each model feature point belonging to the same cluster is calculated, and the model feature that is the nearest neighbor of the query feature point is calculated based on the calculated similarity. To execute a process including the step of determining a point at the computer.

  In one aspect of the present invention, a model feature point that is a feature point of input model data is determined, a feature amount of the model feature point is extracted, and a model feature amount vector representing the feature amount of the model feature point; Depending on whether the inner product value of the LSH hash function vector having the same dimension number as the model feature vector is equal to or greater than a threshold value, the model feature vector is one of binary values of 1 and 0. Quantized to one value. Further, when the model feature vector is within a margin set in the hash function vector in the feature space, the other one of the binary values 1 and 0 is the quantum of the model feature vector. Clustered as feature points belonging to clusters that are set as values after quantization and the model feature points are identified by a combination of the quantized values of the model feature quantity vectors obtained using the respective hash function vectors Is done.

  In another aspect of the present invention, a query feature point that is a feature point of data input as a query is determined, a feature amount of the query feature point is extracted, and a query feature amount that represents the feature amount of the query feature point The query feature vector is quantized to one of binary values 1 and 0 depending on whether or not the inner product value of the vector and the hash function vector is equal to or greater than a threshold value. Further, the query feature points are clustered as feature points belonging to a cluster identified by a combination of values after quantization of the query feature quantity vectors obtained by using the hash function vectors, and the query feature points And the similarity between each of the model feature points belonging to the same cluster as the query feature point is calculated, and the model feature point that is closest to the query feature point is determined based on the calculated similarity. The

  According to the present invention, it is possible to improve the recognition accuracy when object recognition using local features is performed by LSH.

It is a figure which shows the example of quantization of a feature-value vector. It is a figure explaining a quantization error. It is a block diagram which shows the structural example of the learning apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the structural example of the index preparation part of FIG. It is a figure which shows the example of correction | amendment of a quantization error. It is a figure which shows the example of a quantization value. It is a block diagram which shows the structural example of the recognition apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the structural example of the recognition part of FIG. It is a flowchart explaining the learning process of a learning apparatus. 10 is a flowchart for describing index creation processing performed in step S4 of FIG. 9. It is a flowchart explaining the recognition process of a recognition apparatus. It is a flowchart explaining the nearest neighbor candidate selection process performed in step S34 of FIG. It is a block diagram which shows the structural example of an object recognition apparatus. It is a figure which shows the example of the determination of a feature point. It is a figure which shows the positional relationship between feature points. It is a figure explaining the matching of a feature point. It is a block diagram which shows the structural example of a computer.

[Configuration example of learning device]
FIG. 3 is a block diagram illustrating a configuration example of a learning device according to an embodiment of the present invention.

  The process of performing nearest neighbor search by LSH, such as KNN matching, includes a learning process as an offline phase and a recognition process as an online phase. The learning device 1 in FIG. 3 is a device that performs learning processing.

  The learning device 1 is configured by a computer. At least a part of the functional units shown in FIG. 3 is realized by executing a predetermined program by the CPU of the computer constituting the learning apparatus 1.

  As illustrated in FIG. 3, the learning device 1 includes an image processing unit 11, a feature amount extraction unit 12, an index creation unit 13, a hash function storage unit 14, and a model dictionary storage unit 15. A model image that is an image of an object to be recognized is input to the image processing unit 11.

  The image processing unit 11 performs initial processing such as processing for converting a model image into a grayscale image, processing for generating an edge image based on the grayscale model image, and multi-resolution processing. The image processing unit 11 outputs the model image data obtained by the initial processing to the feature amount extraction unit 12.

  The feature quantity extraction unit 12 determines each point on the edge of the model image as a feature point. For example, the feature amount extraction unit 12 extracts, as a feature amount (local feature amount), information on a pixel at a position corresponding to a feature point included in each image having different resolutions obtained by multi-resolution processing.

  The feature quantity extraction unit 12 stores the feature quantity vector information representing the extracted feature quantity in the model dictionary storage unit 15 and outputs the information to the index creation unit 13.

  Hereinafter, the feature points of the model image are appropriately referred to as model feature points, and the feature amount vectors representing the feature amounts of the model feature points are referred to as model feature amount vectors.

  The index creation unit 13 reads the LSH hash function stored in the hash function storage unit 14 and quantizes the model feature quantity vector using the read hash function.

  Further, the index creation unit 13 clusters the model feature points as feature points belonging to a cluster using a combination of quantized values obtained by applying a plurality of hash functions to the model feature quantity vector as keys.

  The index creation unit 13 creates index information that is information that associates each model feature point with a key that is identification information of a cluster to which the model feature point belongs, and stores the index information in the model dictionary storage unit 15. The quantization and clustering of the model feature vector by the index creation unit 13 will be described later.

  The hash function storage unit 14 stores a plurality of LSH hash functions generated in advance. The generation of the LSH hash function is described in Non-Patent Document 1, for example.

  Each hash function stored in the hash function storage unit 14 is set with a quantization error margin which is a margin for reducing the quantization error of the model feature vector. The hash function storage unit 14 stores information related to the quantization error margin in association with the hash function.

  The model dictionary storage unit 15 stores a model dictionary including information on the model feature quantity vector extracted by the feature quantity extraction unit 12 and index information created by the index creation unit 13. The model dictionary stored in the model dictionary storage unit 15 is provided to a recognition device that actually performs object recognition. The model dictionary is provided from the learning device 1 to the recognition device by wired or wireless communication or via a recording medium.

  FIG. 4 is a block diagram illustrating a configuration example of the index creating unit 13 in FIG.

  As shown in FIG. 4, the index creation unit 13 includes a quantization unit 21 and a clustering unit 22.

  The quantization unit 21 calculates a hash value by applying the hash function stored in the hash function storage unit 14 to the model feature quantity vector supplied from the feature quantity extraction unit 12.

  The hash function stored in the hash function storage unit 14 is expressed as a vector (hash function vector) having the same number of dimensions as the model feature vector. The hash value calculated by the quantization unit 21 is a real vector, and each term of the vector is an inner product of a model feature vector and a hash function vector.

  The quantization unit 21 compares the inner product value calculated as the hash value with a threshold value, and quantizes the model feature vector into one of binary values of 1 and 0. For example, the quantization unit 21 quantizes the model feature vector to a value 1 when the inner product value of the model feature vector and the hash function vector is 0 or more as a threshold value, and if the value is less than 0, the quantizer 21 Quantize to zero.

  Further, when the model feature vector is within the quantization error margin set in the hash function vector in the feature quantity space, the quantization unit 21 quantizes the model feature quantity vector to a value of 1. If it is, the value 0 is set as the quantization value, and if it is quantized to the value 0, 1 is set as the quantization value.

  FIG. 5 is a diagram illustrating an example of correcting the quantization error.

  Feature quantity vectors A and B in FIG. 5 are model feature quantity vectors. The hash function vector f is a vector representing one hash function among a plurality of prepared hash functions. For the hash function vector f, a hyperplane orthogonal to the hash function vector f is set as a threshold value that bisects the feature amount space.

  Further, with reference to the intersection of the hyperplane representing the threshold and the hash function vector f, the boundary A and the boundary B include the neighborhood on one side of the hyperplane representing the threshold and the neighborhood on the other side on the inside. Is set. The range inside the boundary A and the boundary B is a quantization error margin of the hash function vector f.

  For example, when the inner product value of the feature vector A and the hash function vector f is obtained as a value of 0 or more, the feature vector A is quantized to a value 1 as described above. In addition, since the feature quantity vector A is included in the quantization margin range of the hash function vector f in the feature quantity space, the value 0 is also set as the quantization value of the feature quantity vector A.

  When the hash function vector f is used, the feature vector A is quantized to both 0 and 1 values.

  Similarly, when the inner product value of the feature vector B and the hash function vector f is obtained as a value less than 0, the feature vector B is quantized to a value of 0. Since the feature vector B is included in the quantization margin range of the hash function vector f in the feature space, the value 1 is also set as the quantization value of the feature vector B.

  When the hash function vector f is used, the feature vector B is quantized to both 0 and 1 values.

  The quantization unit 21 performs the quantization of the model feature amount vector as described above using each hash function vector stored in the hash function storage unit 14. The quantized value obtained by the quantizing unit 21 is supplied to the clustering unit 22.

  The clustering unit 22 clusters each model feature point based on the quantized value obtained by the quantizing unit 21.

  FIG. 6 is a diagram illustrating an example of the quantized value of the model feature quantity vector.

FIG. 6 shows an example of quantized values of feature quantity vectors A and B, which are model feature quantity vectors, obtained using the hash function vectors f _{1 to} f ₃ , respectively.

As shown in FIG. 6, when the hash function vector f ₁ is used, the feature quantity vector A is quantized to a value of 0.

When the hash function vector f ₂ is used, the feature quantity vector A is quantized to a value of 1. Here, since the feature quantity vector A is a feature quantity vector included in the range of the quantization margin set in the hash function vector f ₂ in the feature quantity space, the value 0 is also quantized of the feature quantity vector A. Set as a value.

When using a hash function vector f _3, feature vector A is quantized to the value 1.

Similarly, when the hash function vector f ₁ is used, the feature vector B is quantized to a value of 0.

In the case of using a hash function vector f _2, feature vector B is quantized to the value 0. The feature amount vector B set on the feature space, since a feature vector included in the scope of the set quantization margin hash function vector f _2, as the value 1 is also quantized value of the feature vector B Is done.

When using a hash function vector f _3, feature vector B is quantized to the value 1.

  In this case, the clustering unit 22 uses [0, 1, 1], which is a combination of quantized values calculated using the respective hash function vectors for the model feature points represented by the feature amount vector A. Clustering is performed as feature points belonging to two clusters, a cluster having a key and a cluster having [0, 0, 1] as a key.

  In addition, the clustering unit 22 uses [0, 0, 1], which is a combination of quantized values calculated using the respective hash function vectors, for the model feature points represented by the feature vector B. And a feature point belonging to two clusters having [0, 1, 1] as a key.

  The clustering unit 22 creates index information in which [0, 1, 1] and [0, 0, 1] are associated with the ID of the model feature point whose feature quantity is represented by the feature vector A. Further, the clustering unit 22 creates index information in which [0, 0, 1] and [0, 1, 1] are associated with the ID of the model feature point whose feature value is represented by the feature value vector B.

  As described above, the index creation unit 13 calculates the inner product value of the model feature quantity vector and the hash function vector for each model feature quantity vector, and compares it with the threshold value, so that one of the binary values 1 and 0 can be obtained. The model feature vector is quantized to one value.

  Further, when the model feature vector is included in the range of the quantization error margin set in the hash function vector, the other value of the binary values 1 and 0 is also set as the quantization value, and is based on the quantization value. Model feature points are clustered.

  As a result, the model feature point whose feature value is represented by the model feature value vector in the vicinity of the hyperplane serving as the threshold value of the hash function vector is represented by both the cluster including the value 1 as the key and the cluster including the value 0 as the key. Can be clustered into clusters.

  That is, it is possible to perform clustering in a form having redundancy on model feature points having a model feature quantity vector in the vicinity of a hyperplane serving as a threshold for any hash function vector.

  The model dictionary generated by the learning device 1 as described above is provided to the recognition device, and object recognition using local features is performed.

[Configuration example of recognition device]
FIG. 7 is a block diagram illustrating a configuration example of a recognition apparatus according to an embodiment of the present invention.

  The recognition device 2 in FIG. 7 is a device that performs object recognition using local features by LSH.

  The recognition device 2 is also configured by a computer. At least a part of the functional units shown in FIG. 7 is realized by executing a predetermined program by the CPU of the computer constituting the recognition device 2. The learning device 1 and the recognition device 2 may be realized by the same computer, or may be realized by different computers.

  As illustrated in FIG. 7, the recognition device 2 includes an image processing unit 31, a feature amount extraction unit 32, a recognition unit 33, a hash function storage unit 34, and a model dictionary storage unit 35. A query image is input to the image processing unit 31.

  The image processing unit 31 performs the same processing as the image processing unit 11 in FIG. That is, the image processing unit 31 performs initial processing based on the query image. The image processing unit 31 outputs the query image data obtained by performing the initial processing to the feature amount extraction unit 32.

  Similar to the feature quantity extraction unit 12 of FIG. 3, the feature quantity extraction unit 32 determines the feature points of the query image and extracts the feature quantities of the respective feature points.

  Hereinafter, the feature points of the query image will be referred to as query feature points as appropriate, and the feature amount vectors representing the feature amounts of the query feature points will be referred to as query feature amount vectors.

  The recognizing unit 33 quantizes the query feature quantity vector based on the LSH hash function stored in the hash function storage unit 34 and clusters the query feature points. Further, the recognition unit 33 selects a model feature point belonging to the same cluster as the query feature point as the nearest neighbor based on the index information stored in the model dictionary storage unit 35, and selects the query feature point and the nearest neighbor candidate. The similarity of the feature quantity vector with the model feature point is calculated.

  The recognizing unit 33 selects a predetermined number of model feature points in descending order of the feature vector similarity, and selects the nearest model feature point.

  The recognition unit 33 recognizes which model image the object included in the query image is based on, for example, the number of nearest model feature points that match the query feature point, and outputs a recognition result. .

  The hash function storage unit 34 stores the same plurality of LSH hash functions as those stored in the hash function storage unit 14 of the learning device 1. No quantization error margin is set for the hash function stored in the hash function storage unit 34.

  The model dictionary storage unit 35 stores the model dictionary provided from the learning device 1.

  FIG. 8 is a block diagram illustrating a configuration example of the recognition unit 33 in FIG.

  As shown in FIG. 8, the recognition unit 33 includes a quantization unit 41, a clustering unit 42, a similarity calculation unit 43, and an output unit 44.

  The quantization unit 41 calculates a hash value by applying the hash function stored in the hash function storage unit 34 to the query feature quantity vector.

  The quantization unit 41 compares the inner product value calculated as the hash value with a threshold value, and quantizes the query feature vector into one of binary values of 1 and 0. For example, the quantization unit 41 quantizes the query feature quantity vector to a value 1 when the inner product value of the query feature quantity vector and the hash function vector is 0 or more as a threshold value. Quantize to zero.

  The quantization unit 41 quantizes the query feature quantity vector using each hash function vector stored in the hash function storage unit 34.

For example, as described with reference to FIG. 6, by using the hash function vectors f _{1 to} f ₃ , the feature quantity vector A that is a query feature quantity vector is quantized to values 0, _{1, and 1,} respectively. Since a quantization error margin is not set for the hash function used by the quantization unit 41 to quantize the query feature value vector, the value obtained as the quantization value of the query feature value vector is 0 for one hash function vector. Or one value of 1.

Further, by using the hash function vectors f _{1 to} f ₃ , the feature quantity vector B, which is a query feature quantity vector, is quantized to values 0, 0, and 1, respectively.

  The quantization unit 41 outputs the quantized value to the clustering unit 42. Since the index information generated by the learning device 1 already has redundancy and importance is placed on the execution speed, the quantization error correction as described above is not performed in the quantization performed by the quantization unit 41.

  Similar to the clustering unit 22 in FIG. 4, the clustering unit 42 clusters each query feature point based on the quantization value obtained by the quantization unit 41.

  For example, when the query feature quantity vector is quantized to values 0, 1, and 1, respectively, the clustering unit 42 sets [0, 1, 1] as the query feature point whose feature quantity is represented by the query feature quantity vector. Clustering is performed as feature points belonging to the cluster as a key.

  When the query feature quantity vector is quantized to values 0, 0, and 1, respectively, the clustering unit 42 determines the query feature point whose feature quantity is represented by the query feature quantity vector as [0, 0, 1]. Clustered as feature points belonging to a cluster using as a key.

  The clustering unit 42 outputs information on the key of the cluster to which the query feature point belongs to the similarity calculation unit 43.

  Based on the index information included in the model dictionary stored in the model dictionary storage unit 35, the similarity calculation unit 43 sets a model feature point belonging to the same cluster as the cluster to which the query feature point belongs as the feature point of the nearest neighbor candidate. Identify.

  The similarity calculation unit 43 reads information on the model feature vector of the nearest neighbor candidate from the model dictionary storage unit 35, and calculates the similarity of the feature vector between the query feature point and the model feature point of the nearest neighbor candidate. Information on the feature quantity vector of the query feature point is supplied from the feature quantity extraction unit 32.

  The similarity calculation unit 43 selects a predetermined number of model feature points in descending order of similarity, and performs processing such as generalized Hough transform and outlier removal processing to select the nearest model feature point. The model feature point with the highest degree of similarity may be selected as the nearest model feature point. The similarity calculation unit 43 outputs information on the nearest model feature point to the output unit 44.

  The output unit 44 recognizes which model image the object included in the query image is based on the number of nearest model feature points that match the query feature point, and outputs the recognition result. .

  Since model feature points are clustered in a redundant manner during learning, it is possible to reduce the possibility that query feature points will be clustered as feature points belonging to a cluster different from the nearest model feature point become.

  Therefore, it is highly possible that the nearest model feature point is included as a target for actually calculating the similarity of the feature quantity vector with the query feature point, and matching accuracy can be improved. It is also possible to increase the accuracy of the final recognition result.

[Device operation]
Here, the learning process of the learning device 1 will be described with reference to the flowchart of FIG.

  The process of FIG. 9 is started when a model image is input to the learning device 1.

  In step S1, the image processing unit 11 performs initial processing.

  In step S2, the feature amount extraction unit 12 determines each point of the edge image obtained by the initial processing as a model feature point.

  In step S <b> 3, the feature amount extraction unit 12 extracts the feature amounts of the respective model feature points, and stores the model feature amount vector information representing the extracted feature amounts in the model dictionary storage unit 15.

  In step S4, the index creation unit 13 performs index creation processing. In the index creation process, model feature vectors are quantized, and model feature points are clustered based on the quantized values. After the index creation process ends, the process ends.

  Next, the index creation process performed in step S4 of FIG. 9 will be described with reference to the flowchart of FIG.

  The process in FIG. 10 is performed every time information of a model feature quantity vector representing a feature quantity of one model feature point is supplied from the feature quantity extraction unit 12.

  In step S <b> 11, the quantization unit 21 of the index creation unit 13 selects one hash function stored in the hash function storage unit 14.

  In step S12, the quantization unit 21 calculates an inner product value between the model feature vector and the selected hash function vector.

  In step S13, the quantization unit 21 quantizes the model feature vector to a value 1 when the calculated inner product value is 0 or more as a threshold value, and quantizes the model feature vector to a value 0 when it is less than 0. To do.

  In step S14, the quantization unit 21 determines whether or not the model feature quantity vector is within the quantization error margin set in the hash function vector on the feature quantity space.

  If it is determined in step S14 that the model feature vector is within the quantization error margin, in step S15, the quantization unit 21 corrects the quantization error of the model feature vector.

  That is, the quantization unit 21 sets the value 0 which is the other value when the model feature vector is quantized in step S13 to the value 1 which is one of the binary values 1 and 0. To do. In addition, when the quantizing unit 21 quantizes the model feature vector to the value 0 which is one of the binary values 1 and 0 in step S13, the quantization unit 21 sets the value 1 which is the other value. To do.

  If it is determined in step S14 that the model feature vector is not within the quantization error margin, the process in step S15 is skipped.

  In step S <b> 16, the quantization unit 21 determines whether all hash functions stored in the hash function storage unit 14 have been selected. If it is determined in step S16 that there is a hash function that has not yet been selected, the quantization unit 21 returns to step S11, selects a different hash function, and repeats the above processing.

  On the other hand, if it is determined in step S16 that all hash functions have been selected, in step S17, the clustering unit 22 clusters the model feature points as feature points belonging to a cluster having a combination of quantized values as a key. The clustering unit 22 also creates and stores index information in which model feature point IDs are associated with keys.

  The above processing is performed on all model feature points extracted by the feature amount extraction unit 12. Thereafter, the process returns to step S4 in FIG. 9, and the subsequent processing is performed.

  With the above processing, the model feature points having the model feature quantity vector in the vicinity of the hyperplane serving as the threshold value for any one of the hash function vectors can be clustered with redundancy.

  Next, the recognition process of the recognition device 2 will be described with reference to the flowchart of FIG.

  The process of FIG. 11 is started when a query image is input to the recognition device 2.

  In step S31, the image processing unit 31 performs initial processing.

  In step S32, the feature amount extraction unit 32 determines each point of the edge image obtained by the initial processing as a query feature point.

  In step S33, the feature quantity extraction unit 32 extracts the feature quantities of the respective query feature points, and outputs query feature quantity vector information representing the extracted feature quantities.

  In step S34, the nearest neighbor candidate selection process is performed by the recognition unit 33. In the nearest neighbor selection process, the query feature vector is quantized and the query feature points are clustered. Further, the similarity of the feature quantity vector between the query feature point and each model feature point belonging to the same cluster is calculated.

  In step S <b> 35, the recognition unit 33 determines the nearest model feature point from the nearest neighbor candidates selected by the process in step S <b> 34 by performing processes such as generalized Hough transform and outlier removal process.

  In step S36, the recognition unit 33 recognizes which model image the object included in the query image is after the above processing has been performed on all query feature points. Thereafter, the process is terminated.

  Next, the nearest neighbor candidate selection process performed in step S34 in FIG. 11 will be described with reference to the flowchart in FIG.

  The process of FIG. 12 is performed every time information of a query feature quantity vector representing a feature quantity of one query feature point is supplied from the feature quantity extraction unit 32.

  In step S <b> 51, the quantization unit 41 of the recognition unit 33 selects one hash function stored in the hash function storage unit 34.

  In step S52, the quantization unit 41 calculates an inner product value between the query feature vector and the selected hash function vector.

  In step S53, the quantization unit 41 quantizes the query feature quantity vector to a value 1 when the calculated inner product value is 0 or more as a threshold, and quantizes the query feature quantity vector to a value 0 when it is less than 0. To do.

  In step S54, the quantization unit 41 determines whether all hash functions stored in the hash function storage unit 34 have been selected. If it is determined in step S54 that there is a hash function that has not yet been selected, the quantization unit 41 returns to step S51, selects a different hash function, and repeats the above processing.

  On the other hand, if it is determined in step S54 that all the hash functions have been selected, in step S55, the clustering unit 42 clusters the query feature points as feature points belonging to a cluster using the combination of quantized values as a key.

  In step S56, the similarity calculation unit 43 selects a model feature point belonging to the same cluster as the query feature point as a nearest neighbor candidate based on the index information. In addition, the similarity calculation unit 43 calculates the similarity of the feature quantity vector between the query feature point and the model feature point of the nearest neighbor candidate.

  In step S57, the similarity calculation unit 43 selects a predetermined number of model feature points in descending order of the feature vector similarity.

  The above processing is performed on all query feature points extracted by the feature amount extraction unit 32. Thereafter, the process returns to step S34 in FIG. 11, and the subsequent processing is performed.

  With the above processing, it is possible to reduce the possibility that query feature points will be clustered as feature points belonging to a different cluster from the nearest model feature point, and to improve the accuracy of matching and the accuracy of the final recognition result. It becomes possible to increase.

[Modification]
In the above, the correction of the quantization error of the feature vector is performed only at the time of learning, but it may be performed at the time of recognition. Further, it may be performed only at the time of recognition, not at the time of learning.

  In the above description, the case where matching of local feature values extracted from an image is performed by LSH has been described. However, information to be matched is not limited to feature values extracted from an image. For example, the present invention can be applied to matching of feature amounts extracted from audio data.

[Application example]
Performing object recognition using local features by LSH as described above is applicable to object recognition described in, for example, Japanese Patent Application Laid-Open Nos. 2008-77625 and 2008-243175.

  FIG. 13 is a block diagram illustrating a configuration example of the object recognition apparatus.

  The configuration of the object recognition device 111 shown in FIG. 13 is basically the same as the configuration of the object recognition device described in Japanese Patent Application Laid-Open No. 2008-243175.

  As illustrated in FIG. 13, the object recognition device 111 includes a model feature amount registration unit 131 and a query (target) image recognition unit 132.

  The model feature amount registration unit 131 includes a camera unit 151, a frame memory 152, an edge strength image generation unit 153, an edge image generation unit 154, a model feature amount extraction unit 155, and a model dictionary storage unit 156.

  The camera unit 151 captures an image of a subject and supplies the image obtained by the imaging to the frame memory 152 as a model image.

  The frame memory 152 stores the model image supplied from the camera unit 151.

  The edge strength image generation unit 153 generates an edge strength image based on the model image stored in the frame memory 152. The edge strength image is composed of edge strength indicating the degree of change in the pixel value with respect to the change in position in a predetermined region of the model image. In an edge strength image, the sharper and larger the change in pixel value, the stronger the edge strength, and the slower and smaller the change in pixel value, the weaker the edge strength.

  That is, the edge strength image generation unit 153 generates an edge strength image including edge strength indicating the degree of change in pixel value in a neighboring region including the target pixel in the model image. The edge strength image generation unit 153 supplies the generated edge strength image to the model feature amount extraction unit 155.

  The edge image generation unit 154 generates an edge image based on the model image stored in the frame memory 152. The edge image is an image that shows a magnitude boundary between pixel values of the pixels of the model image.

  For example, the edge image generation unit 154 generates an image having a value of 1 if the pixel value of the target pixel in the model image is equal to or greater than a predetermined threshold value, and generating 0 as the edge image. The edge image generation unit 154 supplies the generated edge image to the model feature amount extraction unit 155.

  The model feature quantity extraction unit 155 determines model feature points on the edge image supplied from the edge image generation unit 154.

  Further, the model feature quantity extraction unit 155 includes a base point that is a reference point among the model feature points, and a support point that is a model feature point other than the base point and is determined depending on the base point. The geometric positional relationship of is determined.

  Here, the geometric positional relationship is a relationship in which the position of one other point is expressed by a distance and an angle from the reference point when one of the two points of interest is used as a reference. That is, the model feature amount extraction unit 155 determines the relative position of the support point with respect to the base point.

  The model feature points are edge points that are points where a reference circle set in a local region on the edge image and the edge image intersect. The base point is one of the edge points on the reference circle, and the other points are the support points.

  Specifically, as illustrated in FIG. 14, the model feature amount extraction unit 155 determines a reference circle R as a local region of the edge image E, and determines an edge point on the edge image E that intersects the reference circle R as Let base point b and support points s1, s2, and s3.

  As shown in FIG. 15, the relative positions (distance and angle) of the support points s1 to s3 from the base point b are relative distances r1, r2, r3 and relative angles θ1, θ2, relative to the reference axis I. It is represented by θ3.

  In the model feature amount extraction unit 155, a plurality of sets of base points and support points are selected by changing the size and position of the reference circle R.

  The model feature amount extraction unit 155 extracts the edge strengths of model feature points (base points and support points) as model feature amounts based on the edge strength image supplied from the edge strength image generation unit 153. The model feature amount is represented as a model feature amount vector which is a feature amount vector having a predetermined number of dimensions.

  Also, the model feature quantity extraction unit 155 quantizes the model feature quantity vector using a previously provided LSH hash function in the same manner as the feature quantity extraction unit 12 of FIG. The model feature quantity extraction unit 155 also performs the process of correcting the quantization error as described above.

  The model feature quantity extraction unit 155 clusters the model feature points as feature points belonging to a cluster using a combination of quantized values obtained by applying a plurality of hash functions to the model feature quantity vector as keys.

  The model feature quantity extraction unit 155 creates index information that is information that associates each model feature point with a key that is identification information of a cluster to which the model feature point belongs, and stores the index information in the model dictionary storage unit 156.

  In addition, the model feature quantity extraction unit 155 causes the model dictionary storage unit 156 to store the geometric positional relationship between the base points and the support points and the model feature quantity vector information for each model image.

  The model dictionary storage unit 156 includes a storage device such as a hard disk drive.

  On the other hand, the query image recognition unit 132 in FIG. 13 includes a camera unit 161, a frame memory 162, an edge strength image generation unit 163, an edge image generation unit 164, a query feature quantity extraction unit 165, a matching unit 166, and an object identification unit 167. Composed.

  The camera unit 161, the frame memory 162, the edge strength image generation unit 163, and the edge image generation unit 164 are respectively a camera unit 151, a frame memory 152, an edge strength image generation unit 153, and an edge image generation unit of the model feature amount registration unit 131. The same as 154.

  That is, the camera unit 161 captures an image of a subject and supplies the image obtained by the imaging to the frame memory 162 as a query image.

  The frame memory 162 stores the query image supplied from the camera unit 161.

  The edge strength image generation unit 163 generates an edge strength image based on the query image stored in the frame memory 162.

  The edge image generation unit 164 generates an edge image based on the query image stored in the frame memory 162.

  The query feature quantity extraction unit 165 obtains the edge strength at the edge point of the query image corresponding to the model feature point of the model image, based on the edge image and the edge strength image of the query image, as a query feature quantity that is the feature quantity of the query image. Extract as The query feature quantity is represented as a query feature quantity vector that is a feature quantity vector having a predetermined number of dimensions.

  The query feature quantity extraction unit 165 supplies the query feature quantity vector information to the matching unit 166 together with key information which is cluster identification information.

  The matching unit 166 quantizes the query feature quantity vector using a previously provided LSH hash function in the same manner as the recognition unit 33 in FIG. 7.

  The matching unit 166 clusters the query feature points (base points and support points) as feature points belonging to a cluster using a combination of quantized values obtained by applying a plurality of hash functions to the query feature quantity vector as keys. To do.

  Based on the index information stored in the model dictionary storage unit 156, the matching unit 166 selects a model feature point belonging to the same cluster as the query feature point as a nearest neighbor candidate. For example, model feature points belonging to the same cluster are detected for each query feature point constituting a set of base points and support points of a query image.

  The matching unit 166 reads the model feature vector information of the detected model feature points from the model dictionary storage unit 156, and calculates the feature vector similarity between the query feature point and the nearest neighbor candidate model feature point. The matching unit 166 determines a model feature point that is closest to the query feature point based on the calculated similarity.

  The object identification unit 167 identifies an object included in the query image based on the number of nearest model feature points.

  The matching by the matching unit 166 will be described.

  For example, for the edge points that are P feature points of the model image and the points corresponding to the P feature points in the query image, the cost between the feature amounts is calculated, and the points with sufficiently high cost values are matched. Obtained as a pair.

  FIG. 16A is a diagram for explaining matching between the base point b1 of the model image (the edge image E of the model image) and the point p ′ of the query image (the edge image E ′ of the query image).

  First, the position correction of the support point s of the model image is performed in accordance with the reference axis of the point p ′ of the query image. Specifically, position correction is performed as shown in FIG. 16B using the reference axis direction nθ possessed by the base point b1 of the model image and the reference axis direction n′θ possessed by the point p ′ of the query image. That is, the reference axis direction nθ possessed by the base point b1 of the model image is tilted in the direction of the arrow by the difference dθ from the reference axis direction n′θ possessed by the point p ′ of the query image.

  Subsequently, the base b1 is aligned with the position of the point p ′, and the feature amount cost d (b1, p ′) between b1 and p ′ is calculated. For each support point, the feature amount within the search area shown in FIG. 16C. The point where the cost d (s1j, p'k) is minimum is searched.

Assuming that the model feature point is m and the query feature point is t, d (m, t) representing the distance between the local feature amounts is expressed by the following equation (1).

  In equation (1), hm (k) represents the feature quantity of the model feature point, and ht (k) represents the feature quantity of the query feature point.

The total matching cost including the base point and the support point is expressed by the following equation (2).

Α1j and β1j in equation (2) are the penalty costs of angle and distance, respectively, and are represented by the following equation (3).

  Θp′k and rp′k are the angle and distance from the base point bi of the point p′k that best matches each support point.

  The above calculation is performed for all the query feature points, and the point having the maximum cost is set as the corresponding point b'1 of the model base point b1.

  As described above, the learning process performed in the learning apparatus 1 in FIG. 3 and the recognition process performed in the recognition apparatus 2 in FIG. 7 are objects described in Japanese Patent Application Laid-Open Nos. 2008-77625 and 2008-243175. Applicable to recognition.

[Computer configuration example]
The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware or a general-purpose personal computer.

  FIG. 17 is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processing by a program.

  A CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, and a RAM (Random Access Memory) 203 are connected to each other via a bus 204.

  An input / output interface 205 is further connected to the bus 204. To the input / output interface 205, an input unit 206 such as a keyboard and a mouse and an output unit 207 such as a display and a speaker are connected. The input / output interface 205 is connected to a storage unit 208 made up of a hard disk, nonvolatile memory, etc., a communication unit 209 made up of a network interface, etc., and a drive 210 that drives the removable medium 211.

  In the computer configured as described above, for example, the CPU 201 loads the program stored in the storage unit 208 to the RAM 203 via the input / output interface 205 and the bus 204 and executes it, thereby executing the above-described series of processing. Is done.

  The program executed by the CPU 201 is recorded in the removable medium 211 or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and is installed in the storage unit 208.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  DESCRIPTION OF SYMBOLS 1 Learning apparatus, 2 Recognition apparatus, 11 Image processing part, 12 Feature-value extraction part, 13 Index creation part, 14 Hash function memory | storage part, 15 Model dictionary memory | storage part, 21 Quantization part, 22 Clustering part, 31 Image processing part, 32 feature quantity extraction unit, 33 recognition unit, 34 hash function storage unit, 35 model dictionary storage unit, 41 quantization unit, 42 clustering unit, 43 similarity calculation unit, 44 output unit

Claims (9)

Extracting means for determining a model feature point that is a feature point of the input model data and extracting a feature amount of the model feature point;
According to whether or not the inner product value of the model feature quantity vector representing the feature quantity of the model feature point and the hash function vector of the LSH hash function vector having the same dimension number as the model feature quantity vector is greater than or equal to a threshold value, If the model feature vector is quantized to one of binary values 1 and 0, and the model feature vector is within the margin set in the hash function vector in the feature space, 1 and Quantization means for setting the other of the binary values of 0 as a value after quantization of the model feature vector;
Clustering means for clustering the model feature points as feature points belonging to a cluster identified by a combination of values after quantization of the model feature quantity vectors obtained by using the hash function vectors. . The learning apparatus according to claim 1, further comprising a storage unit that stores information on the model feature quantity vector and information on a cluster to which the model feature point belongs in association with each other. The learning device according to claim 1, wherein the margin is set in the hash function vector so as to include a vicinity of a hyperplane orthogonal to the hash function vector. Determine model feature points that are the feature points of the input model data,
Extracting feature quantities of the model feature points;
According to whether or not the inner product value of the model feature quantity vector representing the feature quantity of the model feature point and the hash function vector of the LSH hash function vector having the same dimension number as the model feature quantity vector is greater than or equal to a threshold value, Quantize the model feature vector into one of the binary values 1 and 0,
When the model feature quantity vector is within the margin set in the hash function vector in the feature quantity space, the other one of the binary values 1 and 0 is quantized after the model feature quantity vector is quantized. As the value of
A learning method including a step of clustering the model feature points as feature points belonging to a cluster identified by a combination of values after quantization of the model feature vector obtained using the hash function vectors. Determine model feature points that are the feature points of the input model data,
Extracting feature quantities of the model feature points;
According to whether or not the inner product value of the model feature quantity vector representing the feature quantity of the model feature point and the hash function vector of the LSH hash function vector having the same dimension number as the model feature quantity vector is greater than or equal to a threshold value, Quantize the model feature vector into one of the binary values 1 and 0,
When the model feature quantity vector is within the margin set in the hash function vector in the feature quantity space, the other one of the binary values 1 and 0 is quantized after the model feature quantity vector is quantized. As the value of
Clustering the model feature points as feature points belonging to a cluster identified by a combination of values after quantization of the model feature vector obtained by using the hash function vectors. The program to be executed. Determine model feature points that are the feature points of the input model data,
Extracting feature quantities of the model feature points;
According to whether or not the inner product value of the model feature quantity vector representing the feature quantity of the model feature point and the hash function vector of the LSH hash function vector having the same dimension number as the model feature quantity vector is greater than or equal to a threshold value, Quantize the model feature vector into one of the binary values 1 and 0,
When the model feature quantity vector is within the margin set in the hash function vector in the feature quantity space, the other one of the binary values 1 and 0 is quantized after the model feature quantity vector is quantized. As the value of
Information generated by a learning device that clusters the model feature points as feature points belonging to a cluster identified by a combination of values after quantization of the model feature vector obtained using the hash function vectors In a recognition device that performs processing based on
Extracting means for determining a query feature point that is a feature point of data input as a query and extracting a feature amount of the query feature point;
Depending on whether or not the inner product value of the query feature quantity vector representing the feature quantity of the query feature point and the hash function vector is greater than or equal to a threshold value, the query feature quantity vector is one of binary values of 1 and 0. A quantization means for quantizing to one value;
Clustering means for clustering the query feature points as feature points belonging to a cluster identified by a combination of values after quantization of the query feature vector obtained using the hash function vectors;
A similarity between the query feature point and each of the model feature points belonging to the same cluster as the query feature point is calculated, and the model feature point that is closest to the query feature point based on the calculated similarity A recognition device comprising: recognition means for determining The recognition apparatus according to claim 6, further comprising a storage unit that stores information on the model feature vector generated by the learning device and information on a cluster to which the model feature point belongs in association with each other. Determine model feature points that are the feature points of the input model data,
Extracting feature quantities of the model feature points;
According to whether or not the inner product value of the model feature quantity vector representing the feature quantity of the model feature point and the hash function vector of the LSH hash function vector having the same dimension number as the model feature quantity vector is greater than or equal to a threshold value, Quantize the model feature vector into one of the binary values 1 and 0,
When the model feature quantity vector is within the margin set in the hash function vector in the feature quantity space, the other one of the binary values 1 and 0 is quantized after the model feature quantity vector is quantized. As the value of
Information generated by a learning device that clusters the model feature points as feature points belonging to a cluster identified by a combination of values after quantization of the model feature vector obtained using the hash function vectors In the recognition method performed based on
Determine the query feature points that are the feature points of the data entered as queries,
Extracting feature quantities of the query feature points;
Depending on whether or not the inner product value of the query feature quantity vector representing the feature quantity of the query feature point and the hash function vector is greater than or equal to a threshold value, the query feature quantity vector is one of binary values 1 and 0 Quantize to one value,
Clustering the query feature points as feature points belonging to a cluster identified by a combination of values after quantization of the query feature vector obtained using the hash function vectors,
A similarity between the query feature point and each of the model feature points belonging to the same cluster as the query feature point is calculated, and the model feature point that is closest to the query feature point based on the calculated similarity A recognition method including the step of determining. Determine model feature points that are the feature points of the input model data,
Extracting feature quantities of the model feature points;
According to whether or not the inner product value of the model feature quantity vector representing the feature quantity of the model feature point and the hash function vector of the LSH hash function vector having the same dimension number as the model feature quantity vector is greater than or equal to a threshold value, Quantize the model feature vector into one of the binary values 1 and 0,
When the model feature quantity vector is within the margin set in the hash function vector in the feature quantity space, the other one of the binary values 1 and 0 is quantized after the model feature quantity vector is quantized. As the value of
Information generated by a learning device that clusters the model feature points as feature points belonging to a cluster identified by a combination of values after quantization of the model feature vector obtained using the hash function vectors In a program for causing a computer to execute processing based on
Determine the query feature points that are the feature points of the data entered as queries,
Extracting feature quantities of the query feature points;
Depending on whether or not the inner product value of the query feature quantity vector representing the feature quantity of the query feature point and the hash function vector is greater than or equal to a threshold value, the query feature quantity vector is one of binary values of 1 and 0. Quantize to one value,
Clustering the query feature points as feature points belonging to a cluster identified by a combination of values after quantization of the query feature vector obtained using the hash function vectors,
A similarity between the query feature point and each of the model feature points belonging to the same cluster as the query feature point is calculated, and the model feature point that is closest to the query feature point based on the calculated similarity A program that causes a computer to execute a process including a step.

Images