JP2015230715A - Feature quantity arithmetic device, feature quantity arithmetic method, and feature quantity arithmetic program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a feature quantity arithmetic device appropriate for computing binary feature quantities.SOLUTION: A feature quantity arithmetic device comprises: a feature-quantity binarization unit binarizing feature quantities extracted from a pyramid image constituted by an input image and a plurality of resized images obtained by scaling up or down at a plurality of magnifications; and a feature quantity arithmetic unit applying a dictionary set constituted by a plurality of dictionaries at different sizes to the binarized feature quantities and determining the association between the input image and the dictionaries, the feature quantity arithmetic unit applying the identical dictionary set or similar dictionary sets to the images in the pyramid image.

Description

  The present invention relates to a feature amount calculation device, a feature amount calculation method, and a feature amount calculation program for calculating a feature amount extracted from an image, and in particular, a feature amount calculation device that calculates a binarized feature amount, a feature amount The present invention relates to a calculation method and a feature amount calculation program.

  Conventionally, feature quantities are used in many fields such as image search, voice recognition, sentence search, and pattern recognition. The feature amount is information obtained by converting information such as images, sounds, and sentences so as to be easily handled by a computer. The feature amount is represented by a D-dimensional vector (feature vector).

  By performing the feature amount calculation using the feature vector, for example, the similarity of content can be determined. That is, if the distance between the feature vector of the image α and the feature vector of the image β is small, it can be considered that α and β are similar. Similarly, if the distance between the feature vector of the speech waveform α and the feature vector of the speech waveform β is small, it can be considered that α and β are similar. As described above, in information processing such as speech recognition, sentence search, and pattern recognition, information is converted into feature vectors, the feature vectors are compared with each other, and the distance between them is determined to determine information similarity.

As a measure of the distance between feature vectors, an L1 norm, an L2 norm, an angle between vectors, or the like is used. These feature vectors x, for Y∈R ^D, can be calculated as follows.
L1 norm
L2 norm
Angle between vectors

Here, when the extracted feature vector is a real vector, there are the following problems. First, there is a problem that two feature vectors x, the calculation of the distance between the Y∈R ^D slows. For example, when the L2 norm square is used as a distance measure,
Therefore, for a real number, D subtractions, D multiplications, and D-1 additions are required. In particular, when the feature vector is expressed by a floating point number, the calculation load becomes very high. If the feature vector has a higher dimension, the calculation load becomes higher.

  Another problem is that a large amount of memory is consumed. When a feature vector is represented by a 4-byte single-precision real number, the D-dimensional feature vector consumes 4D bytes of memory. As the feature vector becomes higher in dimension, the memory consumption increases. When dealing with a large amount of feature vectors, the memory is consumed by the number of feature vectors to be handled.

  Therefore, in recent years, a method for solving these two problems by converting a feature vector into a binary code composed of a sequence of 0 and 1 has been proposed. Typical techniques include random projection (see random projection, Non-Patent Document 1), belly sparse random projection (see Non-Patent Document 2), and spectral hashing (Spectral Hashing, see Non-Patent Document 3). is there.

  In these methods, a D-dimensional feature vector is converted into a d-bit binary code. This conversion is performed so that the distance in the original space strongly correlates with the Hamming distance in the converted space (for the reason that the distance in the original space and the Hamming distance in the converted space are strongly correlated) (See Lemma 3.2 on page 1121 of Patent Document 1). As a result, the calculation of the distance between feature vectors can be replaced by the calculation of the Hamming distance between binary codes.

  The Hamming distance is obtained by counting the number of different bits in two binary codes. This calculation can be performed very quickly because it only counts the number of bits that are 1 after XORing the two codes. In many cases, binary code conversion can increase the speed by several tens to several hundred times. Further, by substituting the calculation of the distance between feature vectors with the calculation of the Hamming distance between binary codes, the required memory capacity, which was originally 4D bytes, can be reduced to d / 8 bytes. Thereby, memory capacity can be saved to tens to hundreds of times.

  By converting the extracted feature quantity into binary code and applying various algorithms, it becomes possible to search and recognize content. For example, when searching for similar content, all the feature quantities registered in the database in advance are converted into binary codes. Also, the feature amount of the content given as an input query is converted into a binary code. Then, by calculating the Hamming distance between the binary code of the input query and all the binary codes registered in the database, it is possible to search and output content similar to the input query.

  The binary code consists of a sequence of 0's and 1's of d bits. This can be considered as a d-dimensional vector in which each element takes only binary values of −1 and 1. In order to avoid confusion in the following description, the terms “binary code” and “binary vector” are distinguished as follows. A “binary code” is a data representation consisting of a sequence of 0s and 1s. For example, when a 128-bit binary code is stored in a memory in the C language, an array for 16 elements of an unsigned integer type may be prepared (8 bits × 16 = 128 bits).

On the other hand, a “binary vector” is a vector in which each element takes only binary values. For example, when a binary vector is a vector in which each element takes only −1 and 1, the binary vector corresponding to the binary code “01101110” is (−1, 1, 1, −1, 1, 1). , 1, -1) ^T. Of course, a vector in which each element takes only binary values of 0 and 1 is also a binary vector, and furthermore, a vector in which each element takes only binary values of arbitrary α and β (where α ≠ β) Is also a binary vector. However, the difference between the “binary code” and the “binary vector” relates to the expression of information, and there is no essential difference between the two.

Michel X. Goemans, avid P. Williamson, "Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming", Journal of the ACM Volume 42, Issue 6 (November 1995) Pages: 1115-1145 Ping Li, Trevor J. Hastie, Kenneth W. Church, "very sparse random projections", KDD '06 Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining (2006) Y. Weiss, A. Torralba, R. Fergus., "Spectral Hashing", Advances in Neural Information Processing Systems, 2008.

  In order to perform the calculation using the feature amount, it is necessary to extract the feature amount from the input content. In the following, the problem of the present invention will be described by taking as an example the case where the identification target included in the input image as the input content is identified as the feature amount calculation.

  In general, HOG (Histograms of Oriented Gradients) features are used in object recognition. First, an outline of identification using HOG feature values will be described. FIG. 49 is a diagram for explaining a method of extracting HOG feature values from an input image. In order to extract the HOG feature value, the identification device first divides the input image into M pixels × M pixels (M is a natural number), and then uses a gradient direction histogram in the direction of D types (D is a natural number). Ask. This small area of M pixels × M pixels is referred to as a “cell” as one unit. One cell is given a D-dimensional feature vector. Further, a group of N cells × N cells as one unit is called a “block”. Since a D-dimensional feature vector is given to each cell, an (N × N × D) -dimensional feature vector is given to one block.

  Usually, a (N × N × D) -dimensional vector given to a block is normalized so that the length becomes 1. This is a measure to make the lighting conditions more robust. Adjacent blocks are arranged to overlap. That is, the left and right adjacent blocks are arranged to share some cells.

  The discriminating apparatus cuts out an (N × N × D × H × V) -dimensional feature amount from here using a window of horizontal H block × vertical V block. The identification device considers this as a feature amount of the object and applies identification processing to determine whether or not the object shown in this window is a specific target (for example, a pedestrian).

  In the case of pedestrian recognition, it is known that M = 8, N = 2, D = 32, H = 8, and V = 16 are appropriate parameters. For example, when the HOG feature value is extracted from an input image having a width of 640 pixels and a height of 480 pixels with the standard parameters described above, the HOG feature value extracted using a window of width 79 blocks × length 59 blocks is extracted. Is done.

  When the position of the specific target included in the input image is unknown, the identification device slides the window of H block × V block in the input image (N × N × D × It is determined whether or not a specific target is included in the input image by cutting out the feature quantity of (H × V) dimension and applying the identification process each time. Furthermore, the size of the identification target in the input image may be unknown. As a technique for identifying an identification object when the size of the identification object is unknown, there are the following techniques.

(First method: feature pyramid method)
FIG. 1 is a diagram for explaining the feature pyramid method. In this method, the identification device deforms (resizes) the input image into L sizes, generates L images having different sizes, and extracts feature amounts for the respective images. The identification device performs a feature amount calculation for identification using a window W of the same size for each image.

  FIG. 2 is a diagram for explaining the identification process of the feature pyramid method. When the identification device obtains the input image 10, the identification device reduces the input image 10 at a plurality of reduction rates, and generates a plurality of resized (reduced) images 11. The identification device extracts feature amounts for each of the input image and the plurality of resized images (a total of L images) to generate a feature pyramid. That is, the identification device performs feature amount extraction processing L times. The identification device performs a feature amount calculation for identification using a plurality of stages of feature amounts extracted from images of each size. At this time, since the size of the window is fixed, it is sufficient to prepare a dictionary for identification corresponding to the size of the window.

  FIG. 3 is a diagram for explaining identification processing when the above-described technology for speeding up by binarizing feature quantities is applied to the feature pyramid method. As in the case of FIG. 2, when the input image 10 is obtained, the identification device reduces the image by a plurality of reduction ratios, generates a plurality of resized (reduced) images 11, and generates the input image and the plurality of resized images. A feature amount is extracted for each of the images (total L images) to generate a feature pyramid. The identification device binarizes each of the plurality of stages of feature amounts extracted from each size image. That is, the identification device performs the feature value binarization process L times. The identification device performs a feature value calculation for identification using the binary feature value of each stage. Since binary feature values are used, this feature value calculation is speeded up.

  However, first, in the feature pyramid method, since it is necessary to perform feature amount extraction processing for a plurality of images having different sizes, there is a problem that feature amount extraction is slow in this respect. In addition, when speeding up the feature amount calculation by binarizing the feature amount, the binarization process of the feature amount must be performed by the number of stages of resizing. In this respect, the binarization of the feature amount is performed. The benefits of speeding up the feature calculation cannot be fully received.

(Second method: Classifier pyramid method)
FIG. 4 is a diagram for explaining the classifier pyramid method. In this method, the identification device transforms (resizes) the cell size when extracting the feature value from the input image into 2 × 2 pixels, 3 × 3 pixels,... Extract feature values. The block size N and the number of vertical and horizontal blocks H and V of the object model (window) can be set to N = 2, H = 8, and V = 16 as described above, for example. The identification device performs feature value calculation for identification using L different sizes of windows W for the feature values of each stage. Although the vertical and horizontal pixel sizes of the window W change depending on the cell size, the number of dimensions of the feature amount does not change.

  FIG. 5 is a diagram for explaining identification processing by the classifier pyramid method. When the input image 10 is obtained, the identification device extracts feature amounts of the input image with L different cell sizes (for example, 2 × 2 pixels, 3 × 3 pixels,...). At this time, since the operation for obtaining the gradient histogram in the cell is redundant, the load of the feature quantity extraction process can be reduced by a technique such as using an integral image of the feature quantity, but in principle, the feature quantity is extracted. It is necessary to perform the processing (block configuration, normalization of the feature value given to the block) L times. The identification device performs a feature value calculation for identification using a plurality of stages of feature values extracted for each cell size.

  FIG. 6 is a diagram for explaining identification processing when the above-described technique for speeding up by binarizing feature quantities is applied to the classifier pyramid method. As in FIG. 3, when the input image 10 is obtained, the identification device is characterized by L different cell sizes (for example, 2 × 2 pixels, 3 × 3 pixels,...). Extract the amount. The identification device binarizes each feature amount of each cell size. That is, the identification device performs the feature value binarization process L times. The identification device performs a feature value calculation for identification using the binary feature value of each stage. Since binary feature values are used, this feature value calculation is speeded up.

  However, first, in the classifier pyramid method, when the feature quantity is not invariant to the scale, the discrimination performance deteriorates at a scale where the feature quantity is not good. For example, the classifier pyramid method is not suitable because the above-described HOG feature is not scale invariant. More specifically, it is known that the cell size of 8 × 8 pixels is appropriate for the HOG feature amount. However, in the classifier pyramid method, when an apparently large object is to be detected, the cell The size must be very large, and conversely, if an apparently small object is to be detected, the cell size must be very small, and the object recognition accuracy in that case can be significantly degraded. There is also a problem that a dictionary must be learned for each of a plurality of block sizes.

  The classifier pyramid method can speed up the feature extraction process using the redundancy of feature quantities of different block sizes, and can speed up the identification process by binarizing the feature quantities. Is just a collection of two technologies, not a synergistic effect.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a feature amount calculation apparatus suitable for calculating a binary feature amount.

  The feature value calculation apparatus according to one aspect of the present invention binarizes an feature value extracted from each of an input image and a pyramid image including a plurality of resized images obtained by enlarging or reducing the input image at a plurality of magnifications. A feature amount binarization unit, and a feature amount calculation for determining a relevance between the input image and the dictionary by applying a dictionary set including a plurality of dictionaries having different sizes to the binarized feature amount And the feature amount calculation unit uses the binarized feature amount in common with respect to the plurality of dictionaries for each of the pyramid images, and relates to the plurality of dictionaries. Is determined.

  With this configuration, feature values extracted from each of the pyramid images are binarized, and a plurality of dictionaries with different sizes are applied to each binary feature value. Therefore, as in the feature pyramid method, the pyramid Compared to extracting feature values from each image and using the same dictionary for each feature value, the number of feature extraction times can be reduced, reducing the load of feature value extraction processing. Speed up. Compared to the case of extracting feature quantities with different cell sizes from the input image without generating a pyramid image and performing calculations using different dictionaries for each feature quantity as in the classifier pyramid method. The number of binarization processes can be reduced, and the load of the binarization process can be reduced and the speed can be increased. That is, in the above configuration, since binarized feature values are commonly used for a plurality of dictionaries, it corresponds to a difference in size of an input image of a target (for example, a pedestrian) whose relevance is to be determined in the input image. By using a binarized feature value that is shared for each cell, a plurality of dictionaries with different numbers of cells can reduce the number of feature values that need to be calculated. Can be speeded up.

  The feature amount calculation device may further include a feature amount extraction unit that extracts the feature amount from each of the pyramid images, and the feature amount binarization unit is extracted by the feature amount extraction unit. The feature quantity may be binarized.

  With this configuration, it is possible to binarize the real number of feature quantities extracted from each of the pyramid images.

  In the feature amount calculation device, the feature amount calculation unit may perform identification using the dictionary for the input image.

  With this configuration, it is possible to speed up identification by binarizing feature amounts without increasing the processing load. For example, continuous input images (moving images) can be recognized in real time. Become.

  In the feature amount calculation device, the feature amount calculation unit may apply the dictionary set in which all or some of the plurality of dictionaries are the same for each of the pyramid images.

  With this configuration, the capacity of a dictionary set composed of a plurality of dictionaries can be reduced.

  The feature amount calculation device may further include a feature amount conversion unit that converts the feature amount so as to enhance the discrimination ability using the binarized co-occurrence element of the feature amount.

  With this configuration, it is possible to improve the accuracy of identification of the input image in the feature amount calculation unit.

  In the above feature quantity computing device, the real vector having a real number as an element is decomposed into a linear sum of a plurality of base vectors having elements composed only of binary or ternary discrete values. The image processing apparatus may further include a basis vector obtaining unit that obtains a basis vector, the dictionary may be generated using the plurality of basis vectors, and the feature amount calculation unit may include a feature vector indicating the feature amount and the feature amount The relevance between the real vector and the feature vector may be determined by sequentially performing inner product calculation with each of a plurality of basis vectors.

  With this configuration, it is possible to determine the relevance between the feature quantity and the real vector at high speed by speeding up the vector operation by discretization of the dictionary.

  The feature amount calculation device includes a feature amount conversion unit that converts the feature vector so as to enhance discrimination capability using a co-occurrence element of the feature vector determined to be relevant by the feature amount calculation unit; A feature vector converted by the feature quantity conversion unit is further subjected to inner product calculation with each of a plurality of base vectors, thereby determining a relevance between the real vector and the feature vector. And a feature amount calculation unit.

  With this configuration, a relevance determination is performed by a cascade process in which a relevance determination without using co-occurrence is performed and a relevance determination using a co-occurrence is performed for a feature vector determined to be related. Can be further increased in speed.

  In the feature amount calculation apparatus, the feature amount calculation unit extracts a feature amount while sliding a window for each of the pyramid images, and applies the dictionary set to the feature amount extracted from the window. Sex may be determined.

  With this configuration, there is a plurality of dictionaries to be applied, but the feature amount may be extracted once, so that the processing is simplified.

  The above-described feature quantity computing device includes a real matrix composed of a plurality of real vectors having real numbers as elements, a coefficient matrix, and a base matrix composed of a plurality of base vectors having only binary or ternary discrete values as elements. A real matrix decomposition unit that decomposes the product into a product, the dictionary may be generated using the plurality of base matrices, and the feature value calculation unit includes the feature vector indicating the feature value and the plurality of feature vectors. Calculating a product of the feature vector and the basis matrix, calculating a product of the product and the coefficient matrix, and using the result to calculate the inner product with each of the real vectors of The relationship between each real vector and the feature vector may be determined.

  With this configuration, it is possible to determine the relevance between the feature amount and each of the plurality of real vectors at high speed by speeding up the vector operation by discretization of the dictionary.

  The feature value calculation method according to one aspect of the present invention binarizes an feature value extracted from each of an input image and a pyramid image including a plurality of resized images obtained by enlarging or reducing the input image at a plurality of magnifications. A feature amount binarization step, and a feature amount calculation for determining a relevance between the input image and the dictionary by applying a dictionary set including a plurality of dictionaries having different sizes to the binarized feature amount In the feature amount calculation step, the dictionary set in which all or some of the plurality of dictionaries are the same is applied to each of the pyramid images.

  Even with this configuration, since the feature values extracted from each of the pyramid images are binarized and a plurality of dictionaries having different sizes are applied to each binary feature value, as in the feature pyramid method, Compared to extracting features from each pyramid image and performing calculations using different dictionaries for each feature, the number of feature extractions can be reduced, reducing the load of feature extraction processing. Speed up. Compared to the case of extracting feature quantities with different cell sizes from the input image without generating a pyramid image and performing calculations using different dictionaries for each feature quantity as in the classifier pyramid method. The number of binarization processes can be reduced, and the load of the binarization process can be reduced and the speed can be increased. That is, in the above configuration, since binarized feature values are commonly used for a plurality of dictionaries, it corresponds to a difference in size of an input image of a target (for example, a pedestrian) whose relevance is to be determined in the input image. By using a binarized feature value that is shared for each cell, a plurality of dictionaries with different numbers of cells can reduce the number of feature values that need to be calculated. Can be speeded up.

  The feature amount calculation program according to one aspect of the present invention stores a feature amount extracted from each of a pyramid image including a plurality of resized images obtained by enlarging or reducing the input image and the input image at a plurality of magnifications. Applying a binarizing feature value binarizing step and applying a dictionary set of a plurality of dictionaries having different sizes to the binarized feature value to determine relevance between the input image and the dictionary A feature amount calculation program for executing a feature amount calculation step, wherein in the feature amount calculation step, all or some of the plurality of dictionaries are the same for each of the pyramid images. The dictionary set is applied.

  Even with this configuration, since the feature values extracted from each of the pyramid images are binarized and a plurality of dictionaries having different sizes are applied to each binary feature value, as in the feature pyramid method, Compared to extracting features from each pyramid image and performing calculations using different dictionaries for each feature, the number of feature extractions can be reduced, reducing the load of feature extraction processing. Speed up. Compared to the case of extracting feature quantities with different cell sizes from the input image without generating a pyramid image and performing calculations using different dictionaries for each feature quantity as in the classifier pyramid method. The number of binarization processes can be reduced, and the load of the binarization process can be reduced and the speed can be increased. That is, in the above configuration, since binarized feature values are commonly used for a plurality of dictionaries, it corresponds to a difference in size of an input image of a target (for example, a pedestrian) whose relevance is to be determined in the input image. By using a binarized feature value that is shared for each cell, a plurality of dictionaries with different numbers of cells can reduce the number of feature values that need to be calculated. Can be speeded up.

  According to the present invention, the number of feature extractions and the number of binarization processes can be reduced, so the relevance determination process can be speeded up.

Illustration for explaining the feature pyramid method Diagram explaining the identification process of the feature pyramid method The figure explaining the identification process at the time of applying the speed-up technique by the above-mentioned binarization of the feature-value to the feature pyramid method Illustration for explaining the classifier pyramid method Diagram explaining the classification process of the classifier pyramid method The figure explaining the identification process at the time of applying the speed-up technique by the above-mentioned binarization of the feature-value to the classifier pyramid method Illustration for explaining the hybrid pyramid method Diagram explaining the identification process of the hybrid pyramid method The figure explaining the identification processing of embodiment of this invention which applied the speed-up technique by the binarization of the feature-value to the hybrid pyramid method The figure which shows the example of the element of the binary feature vector in the 1st example of the 2nd Embodiment of this invention The table | surface which shows the relationship between XOR and the harmonic mean in the 1st example of the 2nd Embodiment of this invention Table showing XOR of combinations of all elements of binary feature vector in the first example of the second exemplary embodiment of the present invention The figure which shows calculation of the co-occurrence element by the rotation shift without carry in the 1st example of the 2nd Embodiment of this invention Table showing XOR of combinations of all elements of binary feature vector in the first example of the second exemplary embodiment of the present invention The figure which shows calculation of the co-occurrence element by the rotation shift without carry in the 1st example of the 2nd Embodiment of this invention Table showing XOR of combinations of all elements of binary feature vector in the first example of the second exemplary embodiment of the present invention The figure which shows calculation of the co-occurrence element by the rotation shift without carry in the 1st example of the 2nd Embodiment of this invention Table showing XOR of combinations of all elements of binary feature vector in the first example of the second exemplary embodiment of the present invention The figure which shows calculation of the co-occurrence element by the rotation shift without carry in the 1st example of the 2nd Embodiment of this invention Table showing XOR of combinations of all elements of binary feature vector in the first example of the second exemplary embodiment of the present invention The block diagram which shows the structure of the feature-value conversion apparatus in the 1st example of the 2nd Embodiment of this invention. The figure which shows the HOG feature-value for 1 block of the image in the 2nd example of the 2nd Embodiment of this invention, and the result of binarizing it The figure explaining the enhancement of the feature description capability by multiple thresholds in the second example of the second embodiment of the present invention The figure explaining feature-value conversion in the 2nd example of the 2nd Embodiment of this invention The block diagram which shows the structure of the feature-value conversion apparatus in the 2nd example of the 2nd Embodiment of this invention. Program code for comparison example Example program code Graph showing the relationship between false detection and detection rate when recognition is performed by the recognition device after generating a recognition model by learning The block diagram which shows the structure of the feature-value calculating apparatus in the 1st example of the 3rd Embodiment of this invention. The figure which shows decomposition | disassembly of the real vector in the 1st example of the 3rd Embodiment of this invention The figure which shows the example of a calculation in the 2nd example of the 3rd Embodiment of this invention Flow diagram for speeding up threshold processing by cascade in the vector operation unit in the third example of the third embodiment of the present invention The block diagram which shows the structure of the object recognition apparatus in the 1st application example of the 3rd Embodiment of this invention. The block diagram which shows the structure of the k-means clustering apparatus in the 2nd application example of the 3rd Embodiment of this invention. The figure which shows the example of linear SVM in the case of identifying the person in an image with the some identification reference | standard in the 4th Embodiment of this invention. The figure which shows the example of linear SVM in the case of identifying the person in an image with the some identification reference | standard in the 4th Embodiment of this invention. The block diagram which shows the structure of the feature-value calculating apparatus in the 1st example of the 4th Embodiment of this invention. The figure which shows decomposition | disassembly of the real number matrix in the 1st example of the 4th Embodiment of this invention The figure for demonstrating the relationship between the real number matrix and basis matrix in the 1st example of the 4th Embodiment of this invention. The figure which shows the example of a calculation in the 2nd example of the 4th Embodiment of this invention The block diagram which shows the structure of the object recognition apparatus in the 1st application example of the 4th Embodiment of this invention. The figure which shows the dictionary and bias for every rotating road sign, rotation angle in the 1st application example of the 4th Embodiment of this invention. The figure which shows the property of the coefficient matrix in the 1st application example of the 4th Embodiment of this invention The graph which shows the example of the identification function in the 1st application example of the 4th Embodiment of this invention The block diagram which shows the structure of the k-means clustering apparatus in the 2nd application example of the 4th Embodiment of this invention. The block diagram which shows the process in the identification device of the 1st example of the 5th Embodiment of this invention The block diagram which shows the process in the identification device of the 2nd example of the 5th Embodiment of this invention The block diagram which shows the process in the identification device of the 3rd example of the 5th Embodiment of this invention The figure for demonstrating the method of extracting a HOG feature-value from an input image.

  Hereinafter, a feature value computing device according to an embodiment of the present invention will be described with reference to the drawings. The embodiment described below shows an example when the present invention is implemented, and the present invention is not limited to the specific configuration described below. In carrying out the present invention, a specific configuration according to the embodiment may be adopted as appropriate.

1. First Embodiment Prior to describing a feature amount computing device according to an embodiment of the present invention, feature amount extraction and features in a feature amount computing device according to an embodiment of the present invention will be described with reference to FIGS. An outline of the quantity calculation process will be described. In the following, a case will be described as an example where the feature quantity computing device of the present embodiment is an identification device, the feature quantity to be extracted is a HOG feature quantity, and the feature quantity computation process is an identification process. The identification device as the feature amount computing device of the present embodiment employs a hybrid pyramid method in which the feature pyramid method and the classifier pyramid method are fused.

  FIG. 7 is a diagram for explaining the hybrid pyramid method. The identification device deforms (resizes) the input image into L / K sizes, generates L / K different images, and extracts HOG feature values for each image. The identification device performs feature value calculation for identification using K different sizes of windows W for the feature values of each stage.

  FIG. 8 is a diagram for explaining the identification process of the hybrid pyramid method. When the identification device obtains the input image 10, the identification device reduces the input image 10 at a plurality of reduction rates, and generates a plurality of resized (reduced) images 11. Specifically, the identification device is called a 1/2 image obtained by reducing the input image 10 by 1/2, a 1/4 image reduced by 1/4, a 1/8 image reduced by 1/8, and so on. As described above, the pyramid image is generated by the resized image sequentially reduced by a power of 2.

  Next, the identification device extracts a HOG feature amount for each of the pyramid image including the input image and a plurality of resized images (total L / K images). That is, the identification device performs feature amount extraction processing L / K times. Here, K is an octave interval and indicates how many templates are prepared for each pyramid image (the number of frames superimposed on each image in FIG. 7). The identification device stores a dictionary set including a plurality of (K types) dictionaries having different model sizes, and for each HOG feature value, a dictionary corresponding to each size from which the feature value is cut out is used for identification. The feature amount calculation is performed. At this time, the identification device identifies each pyramid image using the same dictionary set.

  FIG. 9 is a diagram for explaining identification processing according to an embodiment of the present invention in which a technique for speeding up by binarizing feature values is applied to the hybrid pyramid method. When the identification device obtains the input image 10, the identification device reduces the input image 10 at a plurality of reduction rates, and generates a plurality of resized (reduced) images 11. Specifically, the identification device is called a 1/2 image obtained by reducing the input image 10 by 1/2, a 1/4 image reduced by 1/4, a 1/8 image reduced by 1/8, and so on. As described above, the pyramid image is generated by the resized image sequentially reduced by a power of 2. Note that the identification device may generate not only the resized image by reducing the input image but also the resized image by enlarging the input image.

  Next, the identification device extracts a HOG feature amount for each of the pyramid image including the input image and a plurality of resized images (total L / K images). That is, the identification device performs feature amount extraction processing L / K times. The identification device binarizes each of a plurality of stages of HOG feature values extracted from each size image. That is, the identification device performs the feature value binarization process L / K times. The identification device stores a dictionary set composed of a plurality of (K types) dictionaries with different model sizes, and identifies each binary HOG feature value by using a dictionary corresponding to each size from which the feature value is cut out. The feature amount calculation for is performed. At this time, the identification device identifies each pyramid image using the same dictionary set.

  As described above, the identification device according to the present embodiment needs only binarization processing for the layer (L / K) of the pyramid even when binarization processing is added. The extraction process can be accelerated. Moreover, although there are not many binarization processes, the benefits of speeding up the identification process by binarization can be sufficiently received. Further, the size of the feature amount is reduced in the hybrid pyramid and further binarized, so that the memory consumption by the feature amount can be reduced. Therefore, combining the hybrid pyramid method and technology for speeding up identification processing by binarizing feature values maximizes the benefits of speeding up identification processing by binarization while reducing the processing load of binarization. A synergistic effect is achieved.

  That is, taking an identification device that detects a pedestrian from an input image as an example, in the feature pyramid method, the input image is resized so that the pedestrian to be detected has a specific number of pixels. The number is M × M, and the number of dictionary cells is also constant. In the classifier pyramid method, the number of pixels in one cell is resized according to the pedestrian to be detected, and the number of cells in the dictionary is constant. In contrast, in the hybrid pyramid method of the present embodiment, the pyramid image is generated by resizing the input image to some extent according to the pedestrian to be detected, and a dictionary having a different number of cells is prepared for each pyramid image. Apply. As a result, the feature quantity used to identify a plurality of dictionaries can be shared, and the feature calculation time can be shortened.

  In the identification device described above, each image included in the pyramid image is identified using the same dictionary set. However, the dictionary set may not be completely the same, and is used for each image of the pyramid image. The dictionary sets may be similar to each other. That the dictionary sets are similar means that only some of the plurality of dictionaries included in the dictionary set are common.

2. Second embodiment
2-1. BACKGROUND Conventionally, a recognition apparatus for recognizing an object by machine learning has been put into practical use in many fields such as image search, voice recognition, and text search. For this recognition, feature amounts are extracted from information such as images, sounds, and sentences. When recognizing a specific target from an image, for example, HOG feature values can be used as image feature values (for example, Navneet Dalal and Bill Triggs, “Histograms of Oriented Gradients for Human Detection”, CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)-Volume 1-Volume 01, Pages 886-893). The feature quantity is handled in the form of a feature vector so that it can be easily handled by a computer. That is, information such as images, sounds, and sentences is converted into feature vectors for object recognition.

The recognition device recognizes an object by applying a feature vector to a recognition model. For example, the recognition model of the linear classifier is given by Equation (1).
Here, x is a feature vector, w is a weight vector, and b is a bias. The linear classifier performs binary classification according to whether f (x) is greater than or less than zero when a feature vector x is given.

  Such a recognition model is determined by performing learning using a large number of feature vectors prepared for learning. In the above linear classifier example, the weight vector w and the bias b are determined by using a large number of positive examples and negative examples as learning data. As a specific method, for example, learning by SVM (support vector machine) can be adopted.

  Linear discriminators are particularly useful because of the fast computation required for learning and discrimination. However, since the linear discriminator can only perform linear discrimination (binary classification), it has a drawback of poor discrimination ability. Therefore, an attempt has been made to improve the description ability of the feature quantity by applying nonlinear transformation to the feature quantity in advance. For example, attempts have been made to enhance the discrimination ability by using the co-occurrence of feature quantities. Specifically, FIND (Feature Interaction Descriptor) features correspond to this (for example, Hui CAO, Koichiro YAMAGUCHI, Mitsuhiko OHTA, Takashi NAITO, and Yoshiki NINOMIYA, "Feature Interaction Descriptor for Pedestrian Detection", IEICE TRANSACTIONS on Information and Systems Vol.E93-D No.9 pp.2656-2659).

The FIND feature value is a co-occurrence element by taking a harmonic average with respect to all combinations of each element of the feature vector, thereby enhancing the ability to identify the feature value. Specifically, when a D-dimensional feature vector x = (x ₁ , x ₂ ,..., X _D ) ^T is given, the nonlinearity of the following expression (2) is obtained for all combinations of elements. Perform simple calculations.
At this time, the FIND feature amount is given by y = (y ₁₁ , y ₁₂ ,..., Y _DD ) ^T.

  For example, when the feature vector x has 32 dimensions, the FIND feature value obtained by removing duplicate combinations is 528 dimensions. Note that y may be normalized so that the length becomes 1 as necessary.

2-2. SUMMARY However, in order to determine the FIND feature amount, it is necessary to calculate all combinations of elements of the feature vector, the amount of calculation on the order of the square against dimensionality. Moreover, since division occurs in the calculation of each element, there is a problem that it is extremely slow. Furthermore, since the feature quantity has a large number of dimensions, there is a problem that the amount of memory consumption increases.

  An object of the present embodiment is to provide a feature quantity conversion device that performs nonlinear transformation of a feature quantity at high speed when the feature quantity is binary in view of the above-described problem.

  Another object of the present embodiment is to provide a feature amount conversion device that converts a feature vector into binary even when the feature vector is not binary.

  The feature amount conversion apparatus according to the first aspect of the present embodiment includes a bit rearrangement unit that generates a plurality of rearranged bit strings obtained by rearranging elements of input binary feature vectors into different arrays, A logic operation unit that generates a plurality of logic operation bit strings by performing a logic operation on each of the rearranged bit strings and the input feature vector, and integrates the plurality of generated logic operation bit strings, And a feature integration unit that generates a transformed feature vector. With this configuration, the co-occurrence element of the input feature vector is calculated by rearrangement of the input feature vector and logical operation, so that the operation of the co-occurrence element can be performed at high speed.

  The feature integration unit may further integrate the elements of the input feature vector together with the generated plurality of logical operation bit strings. According to this configuration, by using the elements of the original feature vector, it is possible to obtain a non-linear transformation feature vector having a higher description capability without increasing the amount of calculation.

  The logic operation unit may calculate an exclusive OR of the rearranged bit string and the input feature vector. Since the exclusive OR is equivalent to the harmonic mean and the appearance probabilities of “+1” and “−1” are also the same, according to this configuration, a co-occurrence element having a high feature description capability equivalent to FIND is calculated. it can.

  The bit rearrangement unit may generate the rearranged bit string by performing a rotation shift without carry on the elements of the input feature vector. According to this configuration, co-occurrence elements with high feature description capability can be calculated efficiently.

  The feature quantity conversion device may include d / 2 bit rearrangement units when the inputted feature vector is d-dimensional. According to this configuration, all combinations of the elements of the input feature vector can be generated by the plurality of bit rearrangement units by performing a carry-less rotate shift in which each bit rearrangement unit is shifted by one bit.

  The bit rearrangement unit may perform random rearrangement on the elements of the input feature vector. Also with this configuration, co-occurrence elements with high feature description capability can be calculated.

  The feature amount conversion apparatus includes: a plurality of binarization units that binarize an input real number feature vector to generate the binary feature vector; and a plurality of binarization units corresponding to each of the plurality of binarization units. Each of the plurality of co-occurrence element generation units includes the plurality of bit rearrangement units and the plurality of logic operation units, and each of the plurality of co-occurrence element generation units. , The binary feature vector is input from the corresponding binarization unit, and the feature integration unit generates the logic generated by each of the plurality of logic operation units of the plurality of co-occurrence element generation units. All of the operation bit strings may be integrated to generate the nonlinear transformation vector. According to this configuration, even when the feature vector element is a real number, a binary feature vector with high feature description capability can be obtained at high speed.

  The binary feature vector may be a feature vector obtained by binarizing the HOG feature value.

  The feature amount conversion apparatus according to the second aspect of the present embodiment includes a bit rearrangement unit that rearranges elements of an input binary feature vector to generate a rearranged bit string, and the rearranged bit string is input. Further, a logical operation unit that performs a logical operation on the feature vector to generate a logical operation bit string, and a feature integration that generates a nonlinear transformation feature vector by integrating the element of the feature vector and the generated logical operation bit string The structure provided with the part. Also with this configuration, since the co-occurrence elements of the input feature vectors are calculated by rearranging the input feature vectors and logical operations, the calculation of the co-occurrence elements can be performed at high speed.

  The feature amount conversion apparatus according to the third aspect of the present embodiment includes a plurality of bit rearrangement units that generate rearranged bit strings obtained by rearranging elements of input binary feature vectors into different arrays, A logical operation unit that generates a logical operation bit string by performing a logical operation between each of the rearranged bit strings generated by the bit rearrangement unit, and a plurality of the logical operation bit strings generated by the elements of the feature vector And a feature integration unit that generates a nonlinear transformation feature vector. Also with this configuration, since the co-occurrence elements of the input feature vectors are calculated by rearranging the input feature vectors and logical operations, the calculation of the co-occurrence elements can be performed at high speed.

  The feature amount conversion apparatus according to the fourth aspect of the present embodiment includes a plurality of bit rearrangement units that generate rearranged bit strings in which elements of input binary feature vectors are rearranged in different arrays, A plurality of logical operation units that perform logical operations between the respective rearranged bit sequences generated by the bit rearrangement unit to generate logical operation bit sequences, and integrate the generated plurality of logical operation bit sequences. And a feature integration unit for generating a nonlinear transformation feature vector. Also with this configuration, since the co-occurrence elements of the input feature vectors are calculated by rearranging the input feature vectors and logical operations, the calculation of the co-occurrence elements can be performed at high speed.

  The learning device according to the present embodiment has a configuration including the above-described feature value conversion device and a learning unit that performs learning using the nonlinear conversion feature vector generated by the feature value conversion device. . Also with this configuration, since the co-occurrence elements of the input feature vectors are calculated by rearranging the input feature vectors and logical operations, the calculation of the co-occurrence elements can be performed at high speed.

  The recognition apparatus according to the present embodiment has a configuration including the above-described feature quantity conversion apparatus and a recognition unit that performs recognition using the nonlinear transformation feature vector generated by the feature quantity conversion apparatus. . Also with this configuration, since the co-occurrence elements of the input feature vectors are calculated by rearranging the input feature vectors and logical operations, the calculation of the co-occurrence elements can be performed at high speed.

  In the above recognition device, the recognition unit calculates the inner product of the weight vector in the recognition and the nonlinear transformation feature vector in the order of wide distribution or the highest entropy value, and the inner product is recognized. The calculation of the inner product may be terminated at a time when it can be determined that the value is larger or smaller than a predetermined threshold. With this configuration, the recognition process can be speeded up.

  The feature amount conversion program according to the present embodiment includes a plurality of bit rearrangement units that generate a rearranged bit string by rearranging the elements of input binary feature vectors into different arrays, respectively, A logical operation of each of the rearranged bit strings and the input feature vector is performed, and a plurality of logical operation units each generating a logical operation bit string and a plurality of the generated logical operation bit strings are integrated to be nonlinear It is made to function as a feature integration unit that generates a transformed feature vector. Also with this configuration, since the co-occurrence elements of the input feature vectors are calculated by rearranging the input feature vectors and logical operations, the calculation of the co-occurrence elements can be performed at high speed.

2-3. Effect According to the present embodiment, the co-occurrence elements of the input feature vector are calculated by the rearrangement of the input feature vector and the logical operation, so that the operation of the co-occurrence element can be performed at high speed.

2-4. First Example of Second Embodiment As described in the first embodiment, the feature amount conversion apparatus of the first example extracts the HOG feature amount by the hybrid pyramid method, and extracts the extracted HOG. Binarize feature values. The feature quantity conversion device of the first example performs a nonlinear transformation on the feature vector when a feature vector that is a binary HOG feature quantity is given, so that a feature vector with improved discrimination power (hereinafter referred to as a feature vector) , Referred to as “non-linear transformation feature vector”). For example, when an area having 8 pixels × 8 pixels as one unit is defined as a cell, the HOG feature value is obtained as a 32-dimensional vector for each block formed of 2 × 2 cells. In this example, it is assumed that the HOG feature value is obtained as a binarized vector. Before explaining the configuration of the feature quantity conversion apparatus of this example, the principle of obtaining a nonlinear transformation feature vector having a co-occurrence element equivalent to FIND by performing nonlinear transformation on a binary feature vector will be explained.

  FIG. 10 is a diagram illustrating an example of elements of a binary feature vector. Each element of the feature vector takes a value of “+1” or “−1”. In FIG. 10, the vertical axis indicates the value of each element, and the horizontal axis indicates the number of elements (number of dimensions). In the example of FIG. 10, the number of elements is 32.

When obtaining the FIND feature value, the harmonic average according to the equation (3) is calculated using these elements.
Here, a and b are values (“+1” or “−1”) of each element. Since a and b are either “+1” or “−1”, the number of combinations is limited to four. Therefore, when the element of the feature vector is binary of “+1” or “−1”, this harmonic average is equivalent to XOR.

  FIG. 11 is a table showing the relationship between XOR and harmonic average. As shown in FIG. 11, the relationship between XOR and the harmonic average is (−½) × XOR = harmonic average. Therefore, for the feature values binarized to “+1” and “−1”, instead of obtaining the harmonic average of all the combinations thereof, the FIND feature amount and the XOR of all the combinations are obtained. It can be converted into a feature quantity with improved discrimination power. Therefore, the feature quantity conversion apparatus of this example improves the discriminating power by taking the XOR of the combination of binary feature vectors having values of “+1” and “−1”.

  FIG. 12 is a table showing XORs of combinations of all elements of binary feature vectors having values of “1” and “−1”. FIG. 12 shows a case where the number of dimensions of a binary feature vector is 8 for simplification of the drawing. The number sequence in the first row and the number sequence in the first row are feature vectors. In the example of FIG. 12, the feature vector is (+1, +1, -1, -1, +1, +1, -1, -1).

  As is clear from the equation (3), since the harmonic mean does not change even if a and b are interchanged, the portion surrounded by the thick line in the table of FIG. 12 is the XOR of all combinations of the elements of this feature vector. It becomes a part except the duplication part. Therefore, in this example, this part is adopted as a co-occurrence element. In addition, since XOR by the same elements is always “−1”, these are not adopted as co-occurrence elements in this example.

  When the elements of the original feature vector in this example and the elements (co-occurrence elements) of the portion surrounded by the thick line in FIG. 12 are arranged, a feature amount equivalent to FIND is obtained. At this time, a co-occurrence element can be calculated at high speed by performing a rotation shift without carry on the original feature vector and calculating the XOR of each element.

  FIG. 13 is a diagram illustrating calculation of co-occurrence elements by a rotate shift without carry. The bit string 100 of the original feature vector is shifted to the right by 1 bit, and the rightmost bit is brought to the first bit (leftmost) to perform a rotation shift without carry to prepare the rearranged bit string 101. . When the bit string 100 and the rearranged bit string 101 are XORed, a logical operation bit string 102 is obtained. This logical operation bit string 102 becomes a co-occurrence element.

  FIG. 14 again shows XOR of combinations of all elements of the binary feature vector. The logical operation bit string 102 in FIG. 13 corresponds to a portion surrounded by a thick frame in FIG. Element E81 is the same as element E18.

  FIG. 15 is a diagram illustrating calculation of co-occurrence elements by a rotate shift without carry. The original feature vector bit string 100 is shifted to the right by 2 bits, and the rightmost 2 bits are shifted to the first and second bits to perform a carry-less rotate shift to prepare a rearranged bit string 201. . When the bit string 100 and the rearranged bit string 201 are XORed, a logical operation bit string 202 is obtained. This logical operation bit string 202 becomes a co-occurrence element.

  FIG. 16 shows XOR of combinations of all elements of the binary feature vector. The logical operation bit string 202 in FIG. 15 corresponds to a portion surrounded by a thick frame in FIG. Elements E71 and E82 are the same as elements E17 and E28, respectively.

  FIG. 17 is a diagram illustrating calculation of co-occurrence elements by a rotate shift without carry. The bit string 100 of the original feature vector is shifted to the right by 3 bits, and the rightmost 3 bits are shifted to the first bit, the second bit, and the third bit to perform a rotation shift without carry, and the rearranged bit string 301 is prepared. When the bit string 100 and the rearranged bit string 301 are XORed, a logical operation bit string 302 is obtained. This logical operation bit string 302 becomes a co-occurrence element.

  FIG. 18 shows XOR of combinations of all elements of the binary feature vector. The logical operation bit string 302 in FIG. 17 corresponds to a portion surrounded by a thick frame in FIG. Elements E61, E72, and E83 are the same as elements E16, E27, and E38, respectively.

  FIG. 19 is a diagram illustrating calculation of co-occurrence elements by a rotate shift without carry. The original feature vector bit string 100 is shifted 4 bits to the right, and the right 4 bits are shifted to the 1st bit, 2nd bit, 3rd bit, 4th bit to perform a rotation without carry, A rearranged bit string 401 is prepared. When the bit string 100 and the rearranged bit string 401 are XORed, a logical operation bit string 402 is obtained. This logical operation bit string 402 becomes a co-occurrence element.

  FIG. 20 shows XOR of combinations of all elements of the binary feature vector. The logical operation bit string 402 in FIG. 19 corresponds to a portion surrounded by a thick frame in FIG. The elements E51, E62, E73, and E81 are the same as the elements E15, E26, E37, and E48, respectively, and either one is not necessary, but this is used as it is for the convenience of calculation.

  By performing the calculations in FIGS. 13, 15, 17, and 19, all the elements in the portion surrounded by the thick line in FIG. 12 can be calculated. In other words, the calculation of the co-occurrence element of the feature vector having the number of bits of 8 can be obtained by performing the four rotations without carry and the XOR calculation. Similarly, when the number of bits (number of dimensions) of a binary feature vector is 32, the binary feature vector can be obtained by 16 rotations without carry and XOR calculation. When the number of bits (the number of dimensions) is d, it can be obtained by d / 2 rotation without carry rotation and calculation of XOR.

  The feature quantity conversion apparatus adds the element of the original feature vector to the co-occurrence element obtained as described above to obtain a nonlinear conversion feature vector. Therefore, when a 32-dimensional binary feature vector is transformed, the number of dimensions of the obtained nonlinear transformation feature vector is 32 × 16 + 32 = 544 dimensions. Below, the structure of the feature-value conversion apparatus which implement | achieves conversion of the above feature vectors is demonstrated.

  FIG. 21 is a block diagram showing the configuration of the feature quantity conversion apparatus of this example. The feature amount conversion apparatus 101 includes N bit rearrangers 111 to 11N, the same number (N) of logical operation units 121 to 12N as the bit rearrangers, and a feature amount integrator 130. A part or all of these bit rearranging units 111 to 11N, logical operation units 121 to 12N, and feature amount integrator 130 may be realized by a computer executing a feature amount conversion program, or by hardware. It may be realized.

  In this example, a binarized feature vector is input to the feature value conversion apparatus 101 as a feature value to be converted. The feature vectors are input to N bit rearrangers 111 to 11N and N logical operators 121 to 12N, respectively. The outputs of the corresponding bit arrayers 111 to 11N are further input to the N logical operation units 121 to 12N.

  The bit reordering units 111 to 11N reorder the input binary feature vectors by a carryless rotation shift to generate a rearranged bit string. Specifically, the bit reorderer 111 performs a 1-bit carryless rotate shift to the right of the feature vector, and the bit reorderer 112 performs a 2-bit carryless rotate shift to the right of the feature vector. The rearranger 113 performs a 3-bit carry-less rotate shift to the right of the feature vector, and the bit rearranger 11N performs an N-bit carry-less rotate shift to the right.

  In this example, if the input binary feature vector is d-dimensional, N = d / 2. Thereby, XOR can be calculated for all combinations of all elements of the feature vector.

  The logical operation units 121 to 12N calculate XOR between the rearranged bit sequence output from the corresponding bit rearrangement unit 111 to 11N and the bit sequence of the original feature vector. Specifically, the logical operator 121 calculates the XOR between the rearranged bit string output from the bit rearranger 111 and the bit string of the original feature vector (see FIG. 13), and the logical operator 122 The XOR of the rearranged bit string output from the arrayer 112 and the bit string of the original feature vector is calculated (see FIG. 15), and the logical operator 123 calculates the original value of the rearranged bit string output from the bit rearranger 113. The XOR with the bit string of the feature vector is calculated (see FIG. 17), and the logical operator 12N calculates the XOR with the bit string of the original feature vector and the rearranged bit string output from the bit rearranger 11N.

  The feature integrator 113 arranges the original feature vector and the outputs (logical operation bit strings) from the logical operation units 121 to 12N, and generates a non-linear transformation feature vector having them as elements. As described above, when the input feature vector has 32 dimensions, the nonlinear transformation feature vector generated by the feature integrator 113 has 544 dimensions.

  As described above, according to the feature value conversion apparatus 101 of this example, the dimension of the feature vector is increased by adding those co-occurrence elements (elements of logical operation bit strings) to the binarized feature vector elements. Thus, the discriminating power of feature vectors can be improved.

  In addition, since the feature vector conversion device 101 of this example has “+1” and “−1” as the elements of the original feature vector, the harmonic average thereof is used as a co-occurrence element like the FIND feature quantity, and Focusing on the fact that XOR of elements is equivalent to co-occurrence elements, XOR of all combinations of each element is calculated and they are used as co-occurrence elements. It can be carried out.

  Furthermore, the feature quantity conversion apparatus 101 of this example calculates the XOR between the bit string of the original feature vector and the bit string obtained by performing a rotation shift without carry on the original feature vector in order to calculate the XOR of each element. When the register width of the computer is less than or equal to the number of bits of the original feature vector (the number of XOR calculations), this XOR calculation can be performed simultaneously, and therefore the co-occurrence elements can be calculated at high speed. Can do.

2-5. Second Example of Second Embodiment Next, as a second example, when the HOG feature quantity is obtained as a real vector instead of a binary vector, it is changed to a binary vector with high discriminating power. A feature amount conversion device for conversion will be described.

  FIG. 22 is a diagram showing the HOG feature amount for one block of an image and the result of binarizing it. The HOG feature amount of this example is obtained as a 32-dimensional feature vector. The upper part of FIG. 22 shows each element of the feature vector, the vertical axis indicates the size of each element, and the horizontal axis indicates the number of elements.

  Each element is binarized to obtain the lower binarized feature vector. Specifically, a threshold for binarization is set at a predetermined position in the range of each element, and when the value of the element is equal to or greater than the set threshold, the element is set to “+1”, and the value of the element is If it is smaller than the set threshold, the element is set to “−1”. Since the range of each element is different, different threshold values (32 types) are set for each element. By binarizing each of the 32 real elements of the feature vector, it can be converted into a binarized feature vector (32 bits) having 32 elements.

  Here, the feature description capability of the feature vector can be enhanced (the amount of information increased) by using the multiple threshold. That is, by setting k different thresholds and performing binarization shown in FIG. 22 for each threshold, the number of dimensions of the binarized feature vector can be increased.

  FIG. 23 is a diagram for explaining the enhancement of feature description capability by multiple thresholds. In this example, binarization is performed using four types of threshold values. Each element of the 32-dimensional real vector is binarized using a 20% position in the range as a threshold value, and an element for 32 bits is generated. Similarly, each element of the 32-dimensional real vector is binarized using the 40% position, 60% position, and 80% position of the range as threshold values, and 32 bit elements are reproduced. When these elements are integrated, a binarized 128-dimensional feature vector (128 bits) is obtained.

  When a feature vector is given as a real vector, binarization with multiple thresholds is performed to improve the feature description capability of the feature vector as shown in FIG. The apparatus 10 can perform non-linear transformation to further increase the amount of information.

  Here, a device for speeding up the binarization of the HOG feature value will be described. In general, the length of the HOG feature must be normalized to 1 in block units. This is because the normalization makes it robust against the brightness.

32D real HOG features before normalization
far. Also, the normalized 32D real HOG feature value
far. At this time,
It is.

32D HOG features after binarization
And At this time,
It is.

This binarization is very slow because square root operations and division occur once. Therefore, paying attention to the fact that the HOG feature is non-negative, the above inequality
Is squared and the denominator of the left side is transferred to the right side to obtain the following expression.

By transforming in this way, the real HOG feature value can be binarized by the following formula without performing the calculation and division of the square root.

  Here, for example, an element determined to be “−1” (smaller than the threshold) as a result of binarization using the 20% position of the range as the threshold value is set to the 40% position, 60% position, and 80% position of the range as the threshold value. Even in the case of binarization, it is naturally “−1”. In this sense, the 128-bit binarization vector obtained by binarization with multiple thresholds includes redundant elements. Therefore, it is not efficient to obtain the co-occurrence element by applying the 128-bit binarized vector as it is to the feature amount conversion apparatus 10 of the first example. In view of this, in this example, a feature quantity conversion device capable of reducing the redundancy and obtaining the co-occurrence element more efficiently is provided.

  FIG. 24 is a diagram for explaining the feature amount conversion in this example. The feature amount conversion apparatus of this example binarizes a feature vector obtained as a real vector with k different thresholds. In the example of FIG. 24, each of 32 elements is binarized by binarizing a 32-dimensional real vector with four types of threshold values of 20% position, 40% position, 60% position, and 80% position of the range. Get a bit string with. The steps so far are the same as in the example of FIG.

  In the feature quantity conversion apparatus of this example, before the bit strings obtained by the respective threshold values are integrated, the co-occurrence elements are obtained using these bit strings. As a result, as shown in FIG. 24, a 544-bit bit string can be obtained from each 32-bit bit string. Eventually, these four bit sequences are integrated to obtain a 2176-bit binarized nonlinear transformation feature vector.

  FIG. 25 is a block diagram showing the configuration of the feature quantity conversion apparatus of this example. The feature quantity conversion apparatus 102 includes N binarizers 211 to 21N, the same number (N) of co-occurrence element generators 221 to 22N, and a feature quantity integrator 23. A part or all of the binarizers 211 to 21N, the co-occurrence element generators 221 to 22N, and the feature quantity integrator 23 may be realized by a computer executing a feature quantity conversion program, or hardware. It may be realized by wear.

  In this example, a real number feature vector is input to the feature quantity conversion apparatus 102. The feature vectors are input to N binarizers 211 to 21N, respectively. The binarizers 211 to 21N binarize real feature vectors with different threshold values. The binarized feature vectors are input to the corresponding co-occurrence element generators 221 to 22N, respectively.

  Each of the co-occurrence element generators 221 to 22N has the same configuration as the feature amount conversion apparatus 101 described in the first example. That is, each of the co-occurrence element generators 221 to 22N includes a plurality of bit reordering units 111 to 11N, a plurality of logical operation units 121 to 12N, and a feature integration unit 13, and performs co-rotation rotation without rotation and XOR operation. The starting elements are calculated, and these and the input bit string are integrated.

  When a 32-bit bit string is input to each of the co-occurrence element generators 221 to 22N, a 544-bit bit string is output from each of the co-occurrence element generators 221 to 22N. The feature integrator 23 arranges the outputs from the co-occurrence element generators 221 to 22N, and generates a nonlinear transformation feature vector having these as elements. As described above, when the input feature vector has 32 dimensions, the feature vector generated by the feature integrator 213 has 2176 dimensions (2176 bits).

  As described above, according to the feature value conversion apparatus 20 of the present example, even when a feature value is obtained as a real vector, it can be binarized and the information amount of the binarized vector can be increased. it can.

2-6. Modification of Second Embodiment The feature amount conversion device 101 of the first example and the feature amount conversion device 102 of the second example are used as learning data when determining a recognition model from a large number of learning data. The nonlinear transformation feature vector is obtained by performing the nonlinear transformation on the inputted feature vector. This non-linear transformation feature vector is used for learning processing by SVM or the like by the learning device, and the recognition model is determined. That is, the feature quantity conversion devices 101 and 102 can be used as a learning device. The feature quantity conversion apparatuses 101 and 102 also perform the above processing on the feature vector when the data to be recognized is input as the feature vector in the same format as the learning data after the recognition model is determined. Perform nonlinear transformation to obtain a nonlinear transformation feature vector. This nonlinear transformation feature vector is used for linear identification or the like by the recognition device, and a recognition result is obtained. That is, the feature amount conversion apparatuses 101 and 102 can be used as a recognition apparatus.

  Note that the logical operation units 121 to 12N do not necessarily calculate XOR as a logical operation, and may calculate AND or OR, for example. However, as described above, XOR is equivalent to the harmonic mean for obtaining the FIND feature value, and as is clear from the table of FIG. 11, when the feature vector is arbitrary, the value of XOR is “ Since “+1” and “−1” appear with equal probability, the entropy of the co-occurrence element is increased (the amount of information is increased), and the description capability of the nonlinear transformation feature vector is improved. It is advantageous to calculate XOR.

  Further, the feature quantity conversion device 101 and the co-occurrence element generators 221 to 22N include the d / 2 bit rearrangers 111 to 11N with respect to the dimension d of the feature vector. The number may be smaller than this (N = 1 may be sufficient) or larger. In addition, the number of logical operation units 121 to 12N may be smaller than d / 2 (N = 1 may be sufficient) or larger than d / 2.

  The bit rearrangers 111 to 11N each generate a new bit string by performing a carry-less rotate shift on the bit string of the original feature vector. Each of the rearrangers 111 to 11N, for example, A new bit string may be generated by randomly rearranging the bit strings of the feature vectors. However, carry-rotate without shift is advantageous in that all combinations can be covered with the minimum number of bits, and the logic is simple and the processing speed is high.

  In addition, the logical operation units 121 to 12N perform logical operations on the original feature vector bit sequence and the bit sequence rearranged by the bit rearrangement unit. However, some or all of the logical operation units perform bit rearrangement. A logical operation may be performed between the bit sequences rearranged by the unit. At this time, the dimension number of the bit vector obtained by the bit rearranger may differ from the dimension number of the original feature vector. Further, the dimensions may be different between the input and output of the binarizers 211 to 21N. Further, the feature integrator 13 generates the nonlinear transformation feature vector using the elements of the original feature vector, but the original feature vector may not be used.

  In the second example, each of the co-occurrence element generators 221 to 22N has the same configuration as that of the feature amount conversion apparatus 101 of the first example, that is, a plurality of bit rearrangers 111 to 11N, a plurality of However, the co-occurrence element generators 221 to 22N are not provided with the feature integrator 13 and are output from the plurality of logic operators 121 to 12N. A plurality of logical operation bit strings may be directly output to the feature integrator 23, and the feature integrator 23 may integrate these to generate a nonlinear transformation feature vector.

  In the first and second examples, the example in which the image is identified has been described. However, the identification target may be other data such as voice and text. Further, the recognition process may be another recognition process that is not linear identification.

  In the first and second examples described above, the plurality of bit rearrangers 111 to 11N generate a rearranged bit string by generating the rearranged bit strings, respectively, and the plurality of logical operation units 121 to 12N Each of the plurality of rearranged bit strings was subjected to a logical operation to calculate an XOR with each of the bit strings of the original feature vectors. The plurality of bit rearrangers 111 to 11N and the plurality of logical operation units 121 to 12N correspond to the bit rearrangement unit and the logical operation unit of the present embodiment, respectively. The bit rearrangement unit and the logical operation unit according to the present embodiment are not limited to the above example. For example, a plurality of rearrangement bits may be generated and a plurality of logical operations may be performed by software processing.

2-7. Example Next, an example using the feature value conversion apparatus of the present embodiment will be described. FIG. 26 shows the program code of the comparative example, and FIG. 27 shows the program code of the example. The comparative example is a program for converting a feature quantity having a 32-dimensional real number element into a FIND feature quantity. The embodiment is a program for performing non-linear transformation on a feature quantity having 32-dimensional binarized elements by the feature quantity conversion apparatus 10 of the first example. Hereinafter, for convenience of explanation, k is the number of steps of the binarization threshold.

  The same pseudo data was converted by the programs of the comparative example and the example. As a result, in the comparative example, the calculation time per block was 7212.71 nanoseconds. On the other hand, in the embodiment, the calculation time per block when the same pseudo data is converted is 22.04 nanoseconds (327.32 times the speed of the comparative example) when k = 1, k = 2 when 33.20 nanoseconds (217.22 times the speed of the comparative example), k = 3 when 42.14 nanoseconds (171.17 times the speed of the comparative example), when k = 4 To 53.76 nanoseconds (134.16 times the speed of the comparative example). Thus, the nonlinear transformation of the example was sufficiently fast compared with the comparative example.

  FIG. 28 is a graph showing the relationship between false detection and detection rate when recognition is performed by the recognition device after generating a recognition model by learning. The horizontal axis indicates erroneous detection, and the vertical axis indicates the detection rate. In the recognition device, it is desirable that the false detection is small and the detection rate is high. That is, in the graph of FIG. 28, the closer to the upper left corner, the higher the recognition performance.

  In FIG. 28, a broken line is a graph when learning and recognition are performed using the HOG feature amount as originally produced by Mr. Dalal as it is, and a one-dot chain line is a FIND feature obtained by optimally tuning the C parameter. This is a graph when learning and recognition are performed using a quantity, and the solid line indicates an example. Specifically, the non-linear transformation obtained by the second example of the present embodiment with k = 4. It is a graph at the time of learning and recognition using a feature vector.

  As is clear from FIG. 28, the FIND feature value and the example have higher recognition performance than the case where the HOG feature value is used as it is. In the embodiment, since the binarization is performed, the recognition performance is inferior to the FIND feature amount, but the deterioration is slight. From the above results, according to the present embodiment, it was confirmed that the processing speed is remarkably improved while the recognition performance is hardly inferior as compared with the FIND feature amount.

A further example of this embodiment will be described. In this example, recognition by the discriminator when the real number of feature values is binarized with k types of thresholds is accelerated by cascade processing. A vector obtained by binarizing a real feature quantity X with k types of threshold values,
far. For the purpose of identification or the like, an operation of calculating w ^T b in the following equation and comparing it with a threshold Th is performed. Here, w is a weight vector for identification.

For example, it is assumed that k = 4, b ₁ is 20%, b ₂ is 40%, b ₃ is 60%, and b ₄ is binarized at 80%. At this time, b ₂ and b ₃ clearly have higher entropy than b ₁ and b ₄ . Therefore, w ₂ ^T b ₂ and w ₃ ^T b ₃ have a wider distribution than w ₁ ^T b ₁ and w ₄ ^T b ₄ .

Focusing on this, in this example, w ₂ ^T b ₂ , w ₃ ^T b ₃ , w ₁ ^T b ₁ , and w ₄ ^T b ₄ are calculated in this order, and w ^T b is less than a predetermined threshold Th in the middle. If it can be determined that it will definitely increase or decrease, the process is terminated at that point. This can speed up the processing. In other words, the cascade order is arranged in the order of wide distribution of w _i ^T b _i or in descending order of entropy value.

3. Third embodiment
3-1. If the background feature vector is converted into a d-dimensional binary vector in which each element takes only binary values of -1 and 1, it can be used for various processes such as identification processing by SVM (support vector machine) and k-means clustering. Binary code can be applied. However, in these cases, it may not be possible to benefit from high-speed distance calculation by Hamming distance. That is, depending on the algorithm, there are cases where the benefits of high-speed distance calculation by binary code conversion cannot be obtained.

As an example in which the benefits of high-speed distance calculation by binary code conversion cannot be obtained, recognition (identification) processing by an identification device (Classifier) and k-means clustering will be described below. First, regarding the recognition processing by the identification device, for example, it is considered that a linear SVM (linear support vector machine) is applied to the problem of identifying binary vectors xε {−1,1} ^d into two classes. In the linear SVM, the following expression (4) is evaluated.
The identification device identifies that the input vector x belongs to class A if the evaluation function f (x) is positive, and that the input vector x belongs to class B if the evaluation function (x) is negative. w is a weight parameter, and wεR ^d . b is a bias parameter, and bεR ¹ . The parameters w and b are dictionaries that are automatically determined by a learning process using feature amounts prepared for learning.

Here, even if the feature quantity prepared for learning is a binary vector, wεR ^d is not a binary value but a real value. The calculation of f (x) includes w ^T x, but since x is binary and w is a real-valued vector, the calculation of w ^T x requires a floating-point operation. turn into. As described above, the recognition processing by the discriminator to which the SVM is applied cannot benefit from the speeding up of calculation by making the feature vector a binary vector.

  Next, when k-means clustering is applied to a binary vector, i.e., when n d-dimensional binary vectors are given, k clusters obtained by collecting binary vectors that are close to each other are obtained. Think about the problem you want. k-means is an algorithm for calculating k clusters and representative vectors according to the following procedure.

Step 1: k features are randomly selected from N feature amounts and set as cluster representative vectors.
Step 2: For each of the N feature values given as input, a representative vector having the closest distance is obtained.
Step 3: The average of the feature quantities belonging to each representative vector is calculated and set as a new representative vector.
Step 4: Repeat Step 2 and Step 3 until convergence.

  The problem in k-means clustering is that in step 3 a new representative vector is defined by the average of the binary vectors. Even if the data given as input is a binary vector, the representative vector becomes a real vector by the average calculation. Therefore, in the distance calculation in step 2, it is necessary to obtain the distance between the binary vector and the real vector. In other words, floating point arithmetic is required. As described above, even in k-means clustering, it is not possible to receive the benefit of speeding up the calculation by making the feature vector a binary vector.

As described above, the recognition processing by the classifier (Classifier) and k-means clustering cannot benefit from speeding up the calculation by making the feature vector a binary vector. The reason is that an inner product operation between the d-dimensional binary vector pε {−1,1} ^d and the d-dimensional real vector qεR ^d is necessary. Note that k-means clustering requires a “distance” between a d-bit binary vector pε {−1,1} ^d and a d-dimensional real vector qεR ^d, and this is also the result. now, it is reduced to the calculation of the inner product of p ^T q. This is because the square of the Euclidean distance between p and q is expressed by the following equation.

  Therefore, speeding up the calculation of the inner product of a binary vector and a d-dimensional real vector in both recognition processing by the identification device and k-means clustering leads to the solution of the problem.

3-2. Overview Therefore, the relevance determination apparatus according to the present embodiment, when a feature vector is a d-dimensional binary vector pε {−1,1} ^d , such a feature vector and a d-dimensional real vector qε. In order to perform an inner product (p ^T q or q ^T p) with R ^d at high speed, the following configuration is provided.

  The relevance determination device according to the first aspect of the present embodiment includes a feature vector acquisition unit that acquires a binarized feature vector, and an element that includes a real vector consisting only of binary or ternary discrete values. A basis vector acquisition unit for acquiring the plurality of basis vectors obtained by decomposing into a linear sum of a plurality of basis vectors, and sequentially calculating an inner product of the feature vector and each of the plurality of basis vectors; And a vector operation unit for determining the relevance between the real vector and the feature vector.

  The inner product calculation of the feature vector and the basis vector is an inner product calculation of a first binary vector having only -1 and 1 as elements and a plurality of second binary vectors having only -1 and 1 as elements. May be included.

  The first binary vector may be the feature vector or may be a vector obtained by dividing each element of the feature vector by a predetermined coefficient, and the feature vector is obtained by linearly transforming each element. Vector.

  The second binary vector may be the basis vector, may be a vector obtained by dividing each element of the basis vector by a predetermined coefficient, and the second binary vector may linearly represent each element. The basis vector may be obtained by conversion.

  The inner product calculation of the feature vector and the base vector includes an inner product calculation of a binary vector having only -1 and 1 as elements and a plurality of ternary vectors having only -1, 0 and 1 as elements. Good.

  The binary vector may be the feature vector, may be a vector obtained by dividing each element of the feature vector by a predetermined coefficient, and the vector that obtains the feature vector by linearly transforming each element. It may be.

  The plurality of ternary vectors may be the plurality of basis vectors, or may be a vector obtained by dividing each element of the plurality of basis vectors by a predetermined coefficient. It may be a vector from which a plurality of basis vectors are obtained.

  The vector calculation unit obtains an exclusive product of the first binary vector and the second binary vector, thereby obtaining an inner product of the first binary vector and the second binary vector. May be calculated.

The vector calculation unit generates a 0 permutation vector by substituting 0 elements of the ternary vector with any of -1 or 1 in the inner product calculation of the binary vector and the ternary vector, A filter vector is generated by replacing a zero element of the ternary vector with -1 and a non-zero element with 1, and generating an exclusive OR of the binary vector and the zero replacement vector and the filter By calculating the logical product with the vector, the number of elements D _{filterd_hamming} of non-zero and different elements between the binary vector and the ternary vector is obtained, and the number of elements D _{filterd_hamming} and the number of non-zero elements are calculated as the two elements. By subtracting from the number of elements of the value vector, the number of elements of the same element that is non-zero between the binary vector and the ternary vector is obtained, and non-zero between the binary vector and the ternary vector Of the same element Wherein said binary vector from prime by subtracting the number of elements of the non-zero at different elements between the ternary vector may determine the inner product of the three-value vector and the binary vector.

  The plurality of basis vectors may be obtained such that a difference between the real vector and a linear sum of the plurality of basis vectors is a decomposition error, and the decomposition error is minimized.

  The plurality of basis vectors are set such that the difference between the inner product of the real vector and the feature vector and the inner product of the linear sum of the plurality of basis vectors and the feature vector is set as a decomposition error so that the decomposition error is minimized. You may be asked for.

  The plurality of basis vectors includes a first update for fixing the elements of the plurality of basis vectors and updating a plurality of coefficients related to the plurality of basis vectors so that the decomposition error is minimized, and the plurality of basis vectors. And the second update for updating the element of the basis vector so as to minimize the decomposition error, and the coefficient may be obtained together with the plurality of coefficients.

  The plurality of basis vectors may be obtained by repeating the first update and the second update until a reduction amount of the decomposition error becomes a predetermined value or less.

  The plurality of basis vectors are obtained by changing the initial values of the plurality of basis vectors and the plurality of coefficients to obtain a plurality of the plurality of basis vectors and the plurality of coefficients, and the plurality of the plurality of basis vectors that minimize the decomposition error. It may be obtained by employing a basis vector and the plurality of coefficients.

  The plurality of coefficients related to the plurality of basis vectors may be discrete values.

  The plurality of basis vectors may be obtained by decomposing an offset real vector obtained by subtracting an average value of elements of the real vector from each element of the real vector into a linear sum of the base vectors.

  Whenever the inner product calculation of the feature vector and the base vector is performed, the vector calculation unit performs the integration when the sum of the inner product calculation results, the real vector, and the feature vector are related to each other. A range of possible values is obtained, and when the sum is outside the range of possible values, the inner product calculation is terminated, and a magnitude relationship between the inner product of the feature vector and the base vector and a predetermined threshold is determined. Good.

  The vector calculation unit, for each inner product calculation of the feature vector and each of the plurality of basis vectors, when the total result of the inner product calculation up to the basis vector is larger than a threshold value for maximum-side early determination, The inner product calculation may be terminated, and it may be determined that the inner product of the real vector and the feature vector is larger than the threshold.

  The vector calculation unit obtains, by learning, a minimum value that an inner product of the feature vector and each of the plurality of basis vectors can take, and calculates the inner product from the threshold value and the feature vector. The maximum side early determination threshold value may be obtained by subtracting the sum of the minimum values that the inner product can take.

  The minimum value that can be taken by the inner product of the feature vector and each of the plurality of basis vectors is a predetermined upper ratio on the lowest side of the values that can be taken by the inner product of the feature vector and each of the plurality of basis vectors. It may be a certain value.

  The vector calculation unit may determine that the feature vector and the base vector are not related when it determines that the inner product of the real vector and the feature vector is larger than the threshold.

  The vector calculation unit, for each inner product calculation of the feature vector and each of the plurality of basis vectors, when the sum of the inner product calculation results up to the basis vector is smaller than a minimum-side early determination threshold, The inner product calculation may be terminated, and it may be determined that the inner product of the real vector and the feature vector is smaller than the threshold value.

  The vector calculation unit obtains, by learning, a maximum value that an inner product of the feature vector and each of the plurality of basis vectors can take, and calculates the inner product from the threshold and the basis vector and the feature vector. The minimum early determination threshold may be obtained by subtracting the sum of the maximum values that the inner product can take.

  The maximum value that can be taken by the inner product of the feature vector and each of the plurality of basis vectors is a predetermined ratio on the upper side of the maximum side of the values that can be taken by the inner product of the feature vector and each of the plurality of basis vectors. It may be a certain value.

  The minimum-side early determination threshold value may be a minimum value of the sum of the inner product calculation results that can be taken when the real vector and the feature vector are related.

  The vector calculation unit may determine that the feature vector and the base vector are not related when it determines that the inner product of the real vector and the feature vector is smaller than the threshold.

  The vector calculation unit may perform the inner product calculation in order from the basis vector having the largest absolute value of the coefficient.

  The vector operation unit decomposes the feature vector and the decomposed real vector into a plurality of partial vectors, and an inner product of the feature vector decomposition vector and the decomposed partial vector of the real vector is the threshold value. It may be determined whether or not it becomes larger and / or smaller than the threshold value.

  The feature vector may be a HOG feature amount, the real vector may be a linear SVM weight vector, and the vector calculation unit may identify the feature vector by a linear SVM as the relevance determination.

  The feature vector is a vector to be clustered by k-means clustering, the real vector is a representative vector in k-means clustering, and the vector calculation unit is configured to determine the relevance as the feature vector. And a clustering process including calculation of the distance between the representative vector and the representative vector.

  The feature vector is a vector to be subjected to an approximate nearest neighbor search by k-means tree, the real vector is a representative vector registered in a node of a k-ary tree, and the vector operation unit As the determination of the relevance, a clustering process including a calculation of a distance between the feature vector and the representative vector may be performed.

  The feature vector may be a vector representing a feature amount of an image.

  The relevance determination program according to the present embodiment has a configuration that causes a computer to function as the relevance determination device.

  The relevance determination method according to the present embodiment includes a feature vector acquisition step of acquiring a binarized feature vector, and a plurality of basis vectors having elements composed of only binary or ternary discrete values as real vectors. A basis vector obtaining step for obtaining the plurality of basis vectors obtained by decomposing the plurality of basis vectors, and sequentially calculating an inner product of the feature vector and each of the plurality of basis vectors. And a vector calculation step for determining relevance with the feature vector.

3-3. Effect According to the present embodiment, a real vector is decomposed into a linear sum of binary base vectors, and an inner product calculation with the binarized feature vector is performed, so an inner product calculation of the feature vector and the real vector is performed. Can be speeded up.

3-4. First Example of Third Embodiment FIG. 29 is a block diagram showing a configuration of a feature quantity computing device 103 of the first example of the present embodiment. The feature amount calculation device 103 includes a content acquisition unit 131, a feature vector generation unit 132, a feature vector binarization unit 133, a real vector acquisition unit 134, a real vector decomposition unit 135, a vector calculation unit 136, a database 137.

  As will be described later, the feature quantity computing device 103 of this example determines the relevance between a feature vector and a real vector by a vector operation involving an inner product operation of the feature vector and a real vector stored in the database as dictionary data. It functions as an association determination device. That is, the feature calculation device 103 corresponds to the relevance determination device of the present embodiment.

  The feature amount calculation device 103 as the relevance determination device is realized by a computer executing the relevance determination program according to the present embodiment. The relevance determination program may be recorded on a recording medium and read from the recording medium by a computer, or may be downloaded to a computer through a network.

  The content acquisition unit 131 acquires content data such as image data, audio data, and character data. Such content data may be provided from an external device or may be generated by the content acquisition unit 131. For example, the content acquisition unit 131 may be a camera, and image data may be generated there as content data.

The feature vector generation unit 132 generates a D-dimensional feature vector from the content data acquired by the content acquisition unit 131. For example, when the content is an image, the feature vector generation unit 132 extracts the feature amount of the image. The feature vector binarization unit 133 binarizes the D-dimensional feature vector generated by the feature vector generation unit 132, and each element has only a binary value of -1 and 1, and a d-dimensional binary vector pε {-1, 1} ^d is generated.

  The configuration including the content acquisition unit 131, the feature vector generation unit 132, and the feature vector binarization unit 133 may be any configuration that can finally acquire a binarized feature vector. For example, the content acquisition unit 131 and the feature vector 132 may be provided, and the feature vector binarization unit 133 may acquire the feature vector from the external device and binarize the acquired feature vector. It may be configured to directly acquire a binarized feature vector.

The real vector acquisition unit 134 acquires a d-dimensional real vector qεR ^d . The real vector may be given from an external device, may be read from a storage device (not shown) of the feature quantity computing device 103, or may be generated by the real vector acquisition unit 134. Good. A real vector has a real number including floating-point numbers in its elements.

The real vector decomposition unit 135 decomposes the d-dimensional real vector qεR ^d into a linear sum of binary base vectors m _i ε {−1,1} ^d . Specifically, the real vector decomposition unit 135 decomposes the d-dimensional real vector qεR ^d into a base matrix M having binary elements and a coefficient vector c having real elements by the following equation (5). To do.
Here, M = (m ₁ , m ₂ ,..., M _k ) ∈ {−1, 1} ^dxk , and c = (c ₁ , c ₂ ,..., C _k ) ^T ∈R ^k . That is, the basis matrix M is composed of k basis vectors m _i , where the basis vector _mi is a d-dimensional binary vector having elements of only −1 and 1, and thus the basis matrix M is , Is a binary matrix of d rows and k columns in which elements take only -1 and 1. The coefficient vector c is a k-dimensional real vector having real coefficients related to k basis vectors as elements. Of course, q and Mc are preferably decomposed so as to coincide as much as possible, but may include an error.

3-4-1. The first decomposition method real vector decomposition unit 135 decomposes the real vector by error minimization. The procedure of the first decomposition method is as follows.
(1) The base matrix M and the coefficient vector c are initialized at random.
(2) The base matrix M is fixed and the error of decomposition
The coefficient vector c is updated so that is minimized. This can be obtained by the least square method.
(3) Decomposition error with the coefficient vector c fixed
The basis matrix M is updated so that is minimized. This minimization algorithm will be described in detail later.
(4) Repeat (2) and (3) until convergence. For example,
When the amount of decrease is less than or equal to a certain value, it is determined that it has converged.
(5) The solutions obtained in steps (1) to (4) are held as candidates.
(6) Repeat step (1) to step (5)
Candidate basis matrices M and c that can be reduced are adopted as final results. Note that the steps (1) to (5) need not be repeated, but the problem of initial value dependency can be avoided by repeating a plurality of times.

Next, the update process of the base matrix M in step (3) will be described. FIG. 30 is a schematic representation of Equation (5). As enclosed by the broken line frame in FIG. 30, the element of the i-th row vector of the base matrix M depends only on the i-th element of the real vector q. There are only 2 ^k row vectors in the i-th row of the base matrix M in the case of binary decomposition as in this example (note that there are only 3 ^{k in} the case of ternary decomposition in the second example described later). not exist). Therefore, the real vector decomposition unit 105 comprehensively checks all of them to determine the decomposition error.
Use a row vector that minimizes. This is applied to all the row vectors of the base matrix M to update the elements of the base matrix M.

3-4-2. Second decomposition method Next, the second decomposition method will be described. In the first decomposition method, the decomposition error is
We considered to minimize this decomposition error. However, what is actually desired to be approximated after approximating the real vector to the linear sum of the basis vectors is the inner product p ^T q of the feature vector and the real vector.

Therefore, in the second decomposition method, N feature vectors p are collected in advance, and the sum of ^these is ^defined as P∈R ^dxN . And the decomposition error
And minimize this. By doing so, the real vector is decomposed according to the actual data distribution, so that the approximation accuracy of the inner product is improved.

This approximate decomposition can be performed by sequentially obtaining m _i . The procedure of the second decomposition method is as follows.
(1) Substituting q into r (r ← q)
(2) Assign 1 to i (i ← 1)
(3) By the first decomposition method
To obtain m _i and c _i .

(4) m _i obtained in step (3), the c _i as the initial value, the following steps
Minimize.
(4-1) Fix m _i ,
Update c _i so that is minimized. This can be obtained by the least square method.
(4-2) securing the c _i,
Update m _i so that is minimized. Since _mi is a discrete value, this becomes a combinatorial optimization problem, for example minimization using algorithms such as Greedy algorithm, tabu search, simulated annealing, etc. It can be carried out. Since a good initial value is obtained in step (3), these algorithms can satisfactorily minimize the decomposition error.
(4-3) Repeat (4-1) and (4-2) until convergence. For example,
When the amount of decrease of becomes less than a certain value, it is determined that it has converged.

(5) Substituting r−m _i c _i for r (r ← r−m _i c _i ), substituting i + 1 for i (i ← i + 1), and if i ≦ k, return to step (3), If i> k, go to step (6).
(6) The solutions M and c obtained in steps (1) to (6) are held as candidates.
(7) Repeat steps (1) to (6)
Candidates M and c that can be reduced are adopted as final results. Note that step (7) need not be repeated, but the problem of initial value dependency can be reduced by repeating a plurality of times.

  In the first and second decomposition methods, the base matrix M does not always have to be binary (or the ternary in the second example), and the types of elements that the base matrix M can take are limited. Applicable if available. The coefficient vector c may also be a discrete value determined in advance as in the base matrix M. For example, the power may be restricted to a power of 2, so that the processing can be speeded up. Further, when the average value of the elements of the real vector q to be decomposed is remarkably large (or small), that is, when the average value is significantly different from 0, the average value is subtracted from each element of the real vector q in advance. If an offset real vector is generated, and the offset real vector is decomposed into a base matrix M and a coefficient vector c, the approximate decomposition of Expression (5) can be performed with fewer bases.

The vector calculation unit 136 performs a calculation using the feature vector. The specific contents of the calculation will be specifically described later together with an application example of the feature amount calculation device 103 of this example. For the calculation using the feature vector, the inner product of the binarized feature vector pε {−1,1} ^d and the real vector q decomposed into a linear sum of binary vectors by the real vector decomposition unit 135 is used. It includes the calculation of p ^T q. Hereinafter, calculation of the inner product p ^T q will be described first.

The inner product p ^T q can be transformed into the following equation (6).
Here, p ^T _mi is an inner product of binary vectors. Inner product p ^T m _i of the binary vector each other can be calculated very fast. The reason is as follows.

An inner product between binary vectors can be reduced to a Hamming distance calculation. The Hamming distance is obtained by counting bits having different values in two binary codes, and the Hamming distance between two binary vectors is obtained by counting the number of elements having different values. Here, p and m _i Hamming distance D _hamming (p, m _i) of the writing and the inner product p ^T m _i is a relationship of D _hamming (p, m _i) and the following equation (7).
Here, as described above, d is the number of bits of the binary code.

The calculation of the Hamming distance is extremely fast because it can be calculated by counting the bits in which 1 stands after applying XOR in two binary codes. If the binary vector is expressed by a binary code (bit string of 0 and 1), the Hamming distance can be calculated by the following equation (8).
Here, the XOR function is an operation of the exclusive OR when considering the p and m _i in binary code representation, BITCOUNT function is processing to enumerate the number of bits 1 of the binary code is standing .

In summary, the inner product p ^T q can be transformed as shown in the following equation (9).
That is, the d-bit Hamming distance calculation is performed k times, the weighted sum related to the coefficient vector c is calculated for k Hamming distances, and the sum of the constant terms is p ^T q. Thus, if k is sufficiently small, p ^T q can be calculated much faster than calculating floating point precision.

In the above inner product calculation, the binarized feature vector p corresponds to the "first binary vector", basis vectors m _i corresponds to a "second binary vector".

  The database 137 stores a plurality of real vectors decomposed by the real vector decomposition unit 135, that is, linear sums of a plurality of base vectors as dictionary data. The vector calculation unit 136 reads the linear sum of the basis vectors from the database 137 and performs the above calculation. This database 137 corresponds to a “basic vector acquisition unit”.

  As described above, according to the feature amount computing device 103 of this example, even when the computation processing using the feature vector includes the inner product computation of the feature vector and another real vector, the feature vector is binarized. In addition, since real vectors are also decomposed into linear sums of binary vectors, their inner product operations can be speeded up.

3-5. Expansion of the first example of the third embodiment In the first example described above, the binary vectors p and m _i are respectively expressed as pε {−1,1} ^d and m _i ε {−1,1. } It was defined as ^d, and it was explained that the inner product operation p ^T _mi becomes faster by decomposing a real vector into a linear sum of binary vectors. However, p, more general binary vector p'∈ a ^{_{m i {-a, a} d}} , m i'∈ {-a, a} as ^d, it is possible to their fast inner product operations. In this case, since p ′ ^T m _i ′ = a ² (p ^T m _i ), the inner product of the binary vectors defined by −1 and 1 may be multiplied by a ² . In this case, the binary vector p obtained by dividing the feature vector p ′ by the coefficient a corresponds to the “first binary vector”, and obtained by dividing the base vector m _i ′ by the coefficient a. It is a binary vector m _i corresponds to a "second binary vector".

Furthermore, even if the feature vector and the base vector are arbitrary binary vectors p ″ ε {α, β} ^d and m _{i ″} ε {γ, δ} ^d , high-speed inner product calculation is possible. Here, the coefficients α, β, γ, and δ are real numbers, and α ≠ β and γ ≠ δ. In this case, P'' and m _i'' is obtained by performing a linear transformation to each element of the binary vector p, and m _i is defined by -1 and 1, the following equation (10) and (11) It is expanded like this.
Note that the bold “1” in the equations (10) and (11) is a vector having a length of d and all elements being 1. Further, A, B, C, and D in the expressions (10) and (11) are real numbers, and may be calculated in advance so that the expressions (10) and (11) are established.

Inner product p'' ^T m _i'' can be expanded by the following equation (12).
The calculation in parentheses in the equation (12) is an inner product between binary vectors consisting of -1 and 1. Therefore, even when the feature vector is a binary vector having an arbitrary binary element and the real vector is expanded into a linear sum of binary vectors having an arbitrary binary element, high-speed calculation is possible. is there. In this case, the binary vector p from which the feature vector p ″ is obtained by linearly transforming each element corresponds to the “first binary vector”, and each element is linearly transformed. The binary vector m _{i from} which the basis vector m _{i ″} is obtained corresponds to the “second binary vector”.

3-6. Second Example of Third Embodiment Next, a feature value computing device of a second example will be described. The configuration of the feature quantity computing device of the second example is the same as that of the first example shown in FIG. In the first example, the real vector decomposition unit 135 decomposes the real vector into a linear sum of binary vectors using Equation (5). However, the real vector decomposition unit 135 of the feature amount computing device of this example converts the real vector into Decomposes a linear sum of ternary vectors.

The real vector decomposition unit 135 decomposes the d-dimensional real vector qεR ^d into a linear sum of ternary vectors. Specifically, the real vector decomposition unit 135 decomposes the d-dimensional real vector qεR ^d into a base matrix M having ternary elements and a coefficient vector c having real elements by the following equation (13). To do.
Here, M = (m ₁ , m ₂ ,..., M _k ) ∈ {−1, 0, 1} ^dxk , and c = (c ₁ , c ₂ ,..., C _k ) ^T ∈R ^k . . That is, the basis matrix M consists of k basis vectors m _i, where the basis vectors m _i, the element is a three-value vector of d-dimensional take -1,0, and 1 only, therefore, basal The matrix M is a ternary matrix of d rows and k columns having elements of only −1, 0, and 1. The coefficient vector c is a k-dimensional real vector having real coefficients related to k basis vectors as elements. Of course, q and Mc are preferably decomposed so as to coincide as much as possible, but may include an error. The real vector decomposition unit 135 decomposes the real vector by error minimization as in the first example.

The vector calculation unit 136 calculates the inner product p ^T q. Hereinafter, the vector calculation unit 136 for calculating the inner product p ^T q is also referred to as an inner product calculation unit 136. The inner product p ^T q can be transformed into the following equation (14).
Here, p ^T m _i is the inner product of the binary vector p and a three-value vector m _i. Here, the inner product calculation unit 106 uses, instead of the ternary vector m _i , a 0 permutation vector m _i ^bin , a filter vector m _i ^filter , and a 0 element number z _i defined below.

First, the inner product calculation unit 136 replaces the 0 element of m _i with −1 or 1. For each element of m _i, to replace it to -1, is either replaced by 1, it may be any. By this replacement, a 0 replacement vector m _i ^bin ε {-1, 1} ^d is generated. This 0 permutation vector m _i ^bin ε {-1, 1} ^d is a binary vector.

Further, the inner product calculation unit 136 replaces the 0 element of m _i -1, replace the elements other than 0 to 1. By this replacement, a filter vector m _i ^filter ε {-1, 1} ^d is generated. This filter vector m _i ^filter is also a binary vector.

Further, the inner product calculation unit 136 obtains the number of elements z _i of 0 of m _i . z _i is an integer. The inner product calculation unit 136 uses the binary vector m _i ^bin , the filter vector m _i ^filter , and the number of zero elements z _i to ^convert p ^T m _i in the equation (14) to the following equations (15) and Calculate according to (16).
Here, the AND function of Expression (15) is an operation of taking a logical product when a binary vector is considered in binary code expression.

Hereinafter, with reference to FIG. 31, the derivation of the equations (15) and (16) will be described using a specific example. FIG. 31 is a diagram illustrating a calculation example of this example. In the example of FIG. 31, p = {− 1,1, −1,1, −1,1} and m _i = {− 1,0,1,0,1,1}. In this ^{_{example, m i bin = {- 1}} , *, 1, *, 1,1} a. Here, “*” represents any one of −1 or 1. Further, m _i ^filter = {1, -1,1, -1,1,1}, and z _i = 2.

The exclusive OR of p and m _i ^bin in equation (15) is XOR (p, m _i ^bin ) = {− 1, *, 1, *, 1, −1}, that is, p and m _i Among the elements of, elements that are non-zero and different, that is, elements that are a pair of -1 and 1 or 1 and -1, are 1, and elements that are a pair of -1 and -1 or 1 and 1 are -1. .

Next, the logical product of the exclusive OR and m _i ^filter is AND (XOR (p, m _i ^bin ), m _i ^filter )) = {− 1, −1,1, −1,1, − 1}, and the elements of p and m _i, 1 is standing on elements that differ nonzero, is -1 otherwise. When this bit count is taken, the number of elements that are 1, that is, the number of non-zero and different elements is counted, and D _{filterd_hamming} (p, m _i ^bin , m _i ^filter ) = 2.

Here, among the elements p and m _i , the number of elements that are a set of 1 and 1 or −1 and −1 is the total number of elements d = 6, and the number of non-zero and different elements D _{filterd_hamming} = This is obtained by subtracting the number of elements z _i = 2 which are 2 and 0. That is, the number of elements that are a set of 1 and 1 or −1 and −1 = d−D _{filterd_hamming−} z _i = 6-2-2 = 2.

p and m _i are elements (a set of -1 and 1 or 1 and -1) from the number of elements (a set of elements whose product is 1) that is a set of 1 and 1 or -1 and -1. P ^T m _i = (d−D _{filterd_hamming} −z _i ) −D _{filterd_hamming} = d−z _i −2D _{filterd_hamming} ), And the value is 6-2-2 × 2 = 0. Of course, this result is as follows. P ^T m _i = {− 1,1, −1,1, −1,1} × {−1,0,1,0,1,1} = 1 + 0 + (− 1 ) +0 + (− 1) + 1 = 0.

Summarizing the equations (15) to (16), the inner product p ^T q can be transformed as the following equation (17).
The inner product calculation unit 106 calculates the inner product p ^T q by the equation (17).

The function D _{filterd_hamming} (p, m _i ^bin , m _i ^filter ) is very similar to the Hamming distance calculation, only an AND operation is added. Therefore, even when qεR ^d is decomposed into a linear sum of ternary vectors, p ^T q can be calculated much faster than p ^T q is calculated with floating-point precision.

As described above, the advantage of decomposing a d-dimensional real vector qεR ^d into a linear sum of ternary vectors instead of binary is that the approximation of Equation (13) is a linear sum of a smaller number of vectors. But it is to come true. That is, since the value of k can be kept small, the speed is further increased.

3-7. Expansion of the second example of the third embodiment In the second example described above, the binary vector p and the ternary vector m _i are respectively expressed as pε {−1,1} ^d and m _i ε {−. 1,0,1} is defined as ^d, the inner product calculating a real vector by decomposing a linear sum of the three value vector p ^T m _i has been described to be a high speed. However, even if p and m _i are more general binary vectors p′∈ {−a, a} ^d and ternary vectors m _i ∈ {−a, 0, a} ^d , their high-speed inner product operations can be performed. Is possible. In this case, since p ′ ^T m _i ′ = a ² (p ^T m _i ), the inner product of the binary vectors defined by −1 and 1 may be multiplied by a ² .

Furthermore, even if the binary vector p and the ternary vector m _i are generalized as pε {α, β} ^d and m _i ε {γ−δ, γ, γ + δ} ^d , high-speed inner product calculation is possible. Here, α, β, γ, and δ are real numbers, and α ≠ β and δ ≠ 0. In this case, by performing a linear transformation of the formula (18) and (19) to each element of p and m _i, respectively p'' and m _i'' is obtained.
Note that the bold “1” in the equations (18) and (19) is a vector having a length of d and all elements being 1. Further, A, B, C, and D in the expressions (18) and (19) are real numbers, and are calculated in advance so that the expressions (18) and (19) are satisfied.

The inner product p ″ ^T m _{i ″} can be expanded as shown in the following equation (20).
The calculation in parentheses in the equation (20) is an inner product of binary vectors consisting of -1 and 1, or an inner product of a binary vector consisting of -1 and 1, and a ternary vector consisting of -1, 0, 1. is there. Accordingly, even when the feature vector is an arbitrary binary vector and the real vector is expanded into a linear sum of the ternary vectors generalized as described above, high-speed calculation is possible.

3-8. Third Example of Third Embodiment In the first and second examples, the inner product calculation of the feature vector p and the real vector q performed in the calculation process in the vector calculation unit 136 has been described. An arithmetic process involving the inner product operation of the feature vector p and the real vector q will be described later as an application example, but as the arithmetic process, the inner product p ^T q may be compared with a certain threshold T. For example, when the feature vector is identified, the inner product p ^T q is compared with a certain threshold T.

In the first example and the second example, the inner product p ^T q is expressed by Equation (6) (when the real vector q is decomposed into a linear sum of binary vectors) and Equation (14) (the real vector q is converted into a ternary vector). As shown in the following (21).
Focusing on this point, the vector calculation unit 136 divides the threshold value processing into k stages when the calculation using the feature vector includes a comparison process (threshold value processing) between the inner product p ^T q and the threshold value. This (cascade) can speed up the threshold processing.

3-8-1. After the first cascade , the threshold processing by the first cascade is speeded up. In the following example, the threshold value is T, and p ^T q> T is determined. FIG. 32 is a flowchart for speeding up the threshold processing by cascade in the vector calculation unit 136. The vector calculation unit 136 first sets i = 1 and y = 0 (step S11). Next, y is updated to y + c _i (p ^T m _i ) (step S12). Next, it is determined whether y is larger than T _i ^min (step S13). Note that T _i ^min is a minimum early determination threshold value for determining p ^T q <T early, and the determination method will be described later. If y is smaller than T _i ^min (YES in step S13), it is determined that p ^T q <T (step S14), and the inner product calculation of feature vector p and real vector q is terminated there, The process ends.

If y is equal to or greater than T _i ^min (NO in step S13), it is next determined whether y is greater than T _i ^max (step S15). Note that T _i ^max is a maximum side early determination threshold value for determining p ^T q> T at an early stage, and a determination method thereof will be described later. If y is larger than T _i ^max (YES in step S15), it is determined that p ^T q> T (step S16), and the inner product operation of feature vector p and real vector q is terminated there, The process ends.

If y is less than or equal to T _i ^max (NO at step S15), and whether or not i = k, that is, whether there is a basal m _i which has not been calculated yet (step S17) If i = k is not satisfied (NO in step S17), i is incremented (step S18), and the process returns to step S12. If i = k, that is, if all bases m _i have been calculated (YES in step S17), the process proceeds to step S14, and it is determined that p ^T q <T and the process is terminated. To do.

As described above, when calculating the inner product of the binary vector p and the real vector q, the threshold value processing is divided into k stages by decomposing the real vector q into a linear sum of the binary vector or the ternary vector. As a result, when p ^T q <T is clearly established, and when p ^T q> T is clearly established, the result of such threshold processing can be obtained early. As a result, the inner product operation of the binary vector p and the real vector q can be suppressed to less than k times.

Although the determination result does not completely match the case where the cascade is not adopted (non-cascade), the determination result is matched as much as the case where the determination result is not cascaded by appropriately selecting T _i ^min and T _i ^max . be able to.

Note that by rearranging the magnitude of c _i, it is possible to enhance the effect of early decision. Furthermore, to decompose so take the difference in the magnitude of c _i, it is possible to enhance the effect of early decision.

Next, a method for determining the minimum side early determination threshold value T _i ^min and the maximum side early determination threshold value T _i ^max will be described. Considering the determination of the minimum-side early determination threshold T ₁ ^min and the maximum-side early determination threshold T ₁ ^max in the first stage of the cascade, c _i (p ^T m _i ) (i = 2, 3,..., K) It is necessary to determine a threshold value that can be safely judged in an unknown state. Therefore, the maximum value P _i ^max and the minimum value P _i ^min that can be taken by c _i (p ^T m _i ) are obtained in advance. This is obtained by substituting binary feature values p extracted from a plurality of data prepared for learning in advance into c _i (p ^T m _i ).

The minimum-side early determination threshold value T _i ^min can be determined as follows.
That is,
Is satisfied, p ^T q <T is satisfied regardless of what large value is entered after the second stage of the cascade. Therefore, it can be said that p ^T q <T is always satisfied at this point. Therefore, T _i ^min can be obtained by the following equation (22).
The T _i ^min after the second stage can be determined in the same manner.

The maximum side early determination threshold value T _i ^max can be determined as follows.
That is,
Is satisfied, p ^T q> T is satisfied regardless of what small value is entered after the second stage of the cascade. Therefore, it can be said that p ^T q> T is always satisfied at this point. Therefore, T _i ^max can be obtained by the following equation (23).
The T _i ^max after the second stage can be determined in the same manner. As is clear from the above description, T _k ^min and T _i ^max are T.

In Equation (23), P _i ^min may not be the minimum, but may be selected as a value within a sufficiently small upper few%. Also in Equation (22), P _i ^max may not be the maximum, but may be selected as a sufficiently large value of several%.

3-8-2. Second Cascade In the second cascade, a deeper cascade is implemented by decomposing a real vector into a plurality of subvectors, thereby speeding up the comparison process with the threshold. That is, in the first cascade,
Focusing on this, the threshold processing was divided into k stages. In the second cascade, this is expanded as follows.

First, as shown in the following equation (24), qεR ^d is decomposed into m partial vectors.
Similarly, pε {1,1} ^d is decomposed into m partial vectors as shown in the following equation (25).
Here, it is assumed that the number of dimensions of q _i and p _i is the same.

At this time, the inner product p ^T q can be written as in the following equation (26).
Since each of the m inner products p _i ^T q _i is an inner product of a binary vector and a real vector, the binary / ternary decomposition method can also be applied to each of them. For example, if each inner product p _i ^T q _i is decomposed into k pieces, p ^T q can be decomposed into mk stages of cascade processing. As a result, the number of cascade stages is increased, and the effect of early determination can be improved. Even in the second cascade, the effect of early determination can be further improved by selecting the order of the cascade processing in the order of the absolute value of the coefficients or the order in which erroneous identification is reduced.

In the third example, in order to determine p ^T q> T, whether p ^T q <T is clearly established for each base (that is, clearly p ^T q> T is not established). Whether or not p ^T q> T is clearly satisfied is determined, but only one of them may be determined.

  Next, calculation processing in the vector calculation unit 136 will be described. The vector calculation unit 136 in the first and second examples described above involves the inner product calculation of the binarized feature vector p and the real vector q, but there are various types of such calculation processing. That is, the above example of the present embodiment can be applied to various apparatuses that perform arithmetic processing using feature vectors. Note that the third example described above can be applied to various apparatuses that perform arithmetic processing involving processing for comparing a feature vector with a threshold, as described above. Therefore, application examples of the present embodiment will be described below.

3-9. First Application Example of Third Embodiment In this application example, this embodiment is applied to object recognition by HOG. FIG. 33 is a block diagram illustrating a configuration of the object recognition apparatus. The object recognition device 104 performs object recognition by HOG. The object recognition device 104 includes a pyramid image generation unit 141, an HOG feature amount extraction unit 142, a binary code conversion unit 143, a parameter determination unit 144, a parameter matrix decomposition unit 145, and a linear SVM identification unit 146. Yes.

  The pyramid image generation unit 141 acquires an image as an input query, and generates a pyramid image obtained by reducing the image at a plurality of scales. Thereby, it is possible to deal with objects of different sizes. This pyramid image generation unit 141 corresponds to the content acquisition unit 131 shown in FIG. The HOG feature amount extraction unit 142 divides the image at each stage of the pyramid image into cells having a size of 8 × 8 pixels, and extracts the HOG feature amount from each cell. The HOG feature amount extraction unit 142 extracts a D-dimensional feature amount from each cell. The HOG feature amount extraction unit 142 corresponds to the feature vector extraction unit 102 shown in FIG. The binary code conversion unit 143 converts the D-dimensional feature value given to each cell into a d-dimensional binary vector. This binary code conversion unit 143 corresponds to the feature vector binarization unit 133 shown in FIG.

  The parameter determination unit 144 determines the weight vector w and the real number bias b used in the linear SVM in the linear SVM identification unit 146. The parameter determination unit 144 determines the weight vector w and the bias b by learning processing using the feature amount prepared for learning. The parameter matrix decomposing unit 145 decomposes the weight vector w into a linear sum of discrete value vectors according to Expression (5) or Expression (13) described in the first or second example.

The linear SVM identifying unit 146 performs feature vector identification using the linear SVM. The linear SVM identification unit 146 first configures a window by collecting W × H cells. A feature vector extracted from one window is a W × H × d-dimensional vector. The linear SVM identification unit 146 applies the linear SVM of the following expression (27) to this feature vector.
Here, the inner product operation w ^T x in the linear SVM can be realized by the high-speed inner product operation of the real vector and the binary vector described as the first or second example.

3-10. Second Application Example of Third Embodiment In this application example, this embodiment is applied to k-means clustering. FIG. 34 is a block diagram illustrating a configuration of the k-means clustering apparatus. The k-means clustering device 105 includes a content acquisition unit 151, a feature vector generation unit 152, a feature vector binarization unit 153, a representative vector update unit 154, a convergence determination unit 155, a representative vector decomposition unit 156, And a nearest representative vector search unit 157.

  The content acquisition unit 151 acquires N contents to be clustered. The feature vector generation unit 152 extracts the feature amounts as feature vectors from each content acquired by the content acquisition unit 151. The feature vector binarization unit 153 binarizes each feature vector extracted by the feature vector extraction unit 152.

  First, the representative vector update unit 154 randomly selects k feature vectors from the N feature vectors binarized by the feature vector binarization unit 153 and sets them as representative vectors. The convergence determination unit 155 performs convergence determination every time the representative vector update unit 154 updates the representative vector. If the convergence determination unit 155 determines that the convergence has occurred, the k-means clustering apparatus 105 ends the clustering process. The representative vector decomposition unit 156 decomposes the representative vector updated by the representative vector update unit 154 into discrete value (binary or ternary) vectors.

  The nearest representative vector search unit 157 causes the N binary vectors input from the feature vector binarization unit 153 to belong to the nearest representative vector. The closest representative vector 157 outputs this result to the representative vector update unit 154. For each representative vector, the representative vector update unit 154 calculates an average vector of feature vectors (binarized) belonging to the representative vector, and sets this as a new representative vector. Since the representative vector updated by the representative vector update unit 154 in this way is calculated as the average of the binary vectors, it becomes a real vector.

  Therefore, if there is no representative vector decomposing unit 156, the nearest representative vector searching unit 157 calculates inner products in order to obtain the distance between the updated representative vector (real vector) and the feature vector (binary vector). Must. Therefore, in this application example, as described above, this representative vector (real vector) is converted into a discrete value (binary or ternary) vector by the representative vector decomposing unit 156 as described in the first or second example. Disassembled into As a result, the nearest representative vector search unit 157 can calculate the distance between each feature vector and each representative vector at high speed. Therefore, the representative vector to which each feature vector is closest (that is, the representative vector to which the feature vector belongs) can be calculated at high speed. To explore.

3-11. Third Application Example of Third Embodiment In this application example, the present embodiment is applied to an approximate nearest neighbor search by k-means tree. The approximate nearest neighbor search device of this example is a Marius Muja and David G. Lowe, "Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration", in International Conference, as an approximate nearest neighbor search method using k-trees using k-means. on Computer Vision Theory and Applications (VISAPP '09), 2009 (http://www.cs.ubc.ca/~mariusm/index.php/FLANN/FLANN, http: // people .cs.ubc.ca / ~ mariusm / uploads / FLANN / flann_visapp09.pdf).

  Specifically, the approximate nearest neighbor search apparatus of the present example constructs a k-ary tree by recursively applying k-means to N pieces of data, and approximates the proposed tree search principle. The nearest neighbor point is searched. This method is designed on the assumption that the data is a real vector and the representative vector registered in the node is a binary vector. However, even when the data is a binary vector and the representative vector registered in the node is a real vector, the tree search can be speeded up by adopting the first or second example.

3-12. In the modified feature quantity computing device 103 of the third exemplary embodiment , the content acquisition unit 131, the feature vector generation unit 132, the feature vector binarization unit 133, the real vector acquisition unit 134, the real vector decomposition unit 135, and the vector calculation Part of the part 136 and other parts may be configured as separate devices. In particular, the content acquisition unit 131, the feature vector generation unit 132, the feature vector binarization unit 133, and the vector calculation unit 136 are mounted on the feature calculation device 103, and the real vector acquisition unit 134 and the real vector decomposition unit 135 are different from each other. It may be mounted on the device. In this case, a plurality of real vectors (a combination of a plurality of coefficient vectors and base vectors) decomposed by the real vector decomposition unit 135 are stored in the database of the feature calculation device 103, and the vector calculation unit 136 decomposes from the database. Obtain a plurality of real vectors. At this time, the vector calculation unit 136 functions as a base vector acquisition unit (first and second examples), a binary vector acquisition unit (first example), and a ternary vector acquisition unit (second example). To do.

  Note that the content data acquired by the content acquisition unit 131 may be measurement data obtained from a vehicle. Furthermore, the measurement data obtained from the vehicle may be, for example, image data captured by a camera installed in the vehicle, or sensing data measured by a sensor installed in the vehicle. In this case, the vector calculation unit 136 of the feature calculation device 103 as the relevance determination device determines the relevance between the measurement data and the dictionary data. For example, when image data taken by a camera installed in a vehicle is acquired as measurement data, data of a plurality of person images are stored in a database as dictionary data, and the feature as a relevance determination device The vector calculation unit 136 of the calculation device 103 may determine whether or not a person is included in the image of the image data according to any of the fourth to sixth examples.

4). Fourth embodiment
4-1. Background In the third embodiment, speeding up the calculation of the inner product of a binary vector and a d-dimensional real vector in both recognition processing by a classifier and k-means clustering leads to the solution of the problem. When the feature vector is a d-dimensional binary vector pε {−1,1} ^d , the inner product between such a feature vector and the d-dimensional real vector qεR ^d ( The relevance determination device that performs the calculation of p ^T q or q ^T p) at high speed has been described.

  That is, the relevance determination apparatus according to the third embodiment includes a feature vector acquisition unit that acquires a binarized feature vector, and an element that includes a real vector only from binary or ternary discrete values. By sequentially performing an inner product calculation of each of the feature vector and each of the plurality of basis vectors, a basis vector acquisition unit that acquires the plurality of basis vectors obtained by decomposing into a linear sum of a plurality of basis vectors, A vector operation unit for determining the relevance between the real vector and the feature vector. With this configuration, the real vector is binarized after being decomposed into a linear sum of a plurality of binary basis vectors. Since the inner product calculation with the feature vector is performed, the inner product calculation between the feature vector and the real vector can be speeded up.

  Incidentally, it may be necessary to determine the relevance between the feature vector and each of the plurality of real vectors by calculating the inner product of the binarized feature vector and the plurality of real vectors. For example, as described above, in linear SVM, only whether a feature vector belongs to class A or class B, that is, whether or not the feature vector falls under a certain identification criterion, is determined. There are cases where such identification is desired for a plurality of criteria. As a specific example, I would like to determine whether the captured image is an adult, a child, a car, or a road sign. There is a case.

  In the k-means clustering described above, for each of the N feature vectors given as an input, a distance with an inner product calculation is calculated with k representative vectors. Here, each of the k representative vectors is a real vector because it is defined by the average of the binary vectors as described above. Therefore, k-means clustering also requires inner product calculation of binarized feature vectors and a plurality of real vectors.

4-2. Overview Therefore, this embodiment speeds up the inner product calculation of a binarized feature vector and a plurality of real vectors, thereby quickly determining the relevance between such feature vectors and a plurality of real vectors. The purpose is to do.

  The relevance determination apparatus according to the present embodiment includes a feature vector acquisition unit that acquires a binarized feature vector, a real matrix composed of a plurality of real vectors, a coefficient matrix, and binary or ternary discrete elements. A real number matrix decomposing unit that decomposes into a product of a base matrix composed of a plurality of basis vectors having only values; and calculation of an inner product of the feature vector and each of the plurality of real number vectors, the feature vector and the base matrix A vector operation unit that calculates a product of the product and the coefficient matrix, and uses the result to determine the relevance between each of the plurality of real vectors and the feature vector; It has the composition provided. With this configuration, in order to calculate the inner product of a feature vector and each of a plurality of real vectors, a real matrix composed of a plurality of real vectors is decomposed into a discrete base matrix and a coefficient matrix, and then the feature vector and the base matrix And the product with the coefficient matrix, the result of the inner product operation between the feature vector and each of multiple real vectors can be obtained at high speed, and the relationship between the feature vector and multiple real vectors can be obtained. Sex determination can be performed at high speed.

  The relevance determination device may further include a real matrix generation unit that generates the real matrix by arranging the plurality of real vectors. With this configuration, a real matrix can be easily generated from a plurality of real vectors.

  In the above relevance determination device, when the plurality of real vectors have predetermined parameters, the real matrix generation unit generates the real matrix by arranging the plurality of real vectors according to the order of the parameters. Good. With this configuration, since real vectors similar to each other in the real number matrix are adjacent to each other, adjacent coefficient matrices are also similar.

In the above relevance determination device, the real matrix decomposition unit includes:
May be decomposed by solving the cost function. Here, Q is the real matrix, M is the basis matrix, and C is the coefficient matrix. With this configuration, an error when a real matrix is decomposed into a product of a base matrix and a coefficient matrix is evaluated as a cost, and the real matrix is decomposed. Therefore, the real matrix can be decomposed easily and with high accuracy. Specifically, a real matrix can be decomposed with a base matrix and a coefficient matrix that minimize this cost function (which satisfies a predetermined convergence condition).

In the above-described relevance determination device, the real matrix decomposition unit fixes the elements of the base matrix and optimizes the elements of the coefficient matrix by a least square method, and fixes the elements of the coefficient matrix Then, the base matrix and the coefficient matrix may be obtained by repeating the second update that optimizes the elements of the base matrix by full search. With this configuration, a real matrix can be easily decomposed. Note that when fixing the elements of the coefficient matrix, the number of combinations to be searched when determining each row of the base matrix, 2 ^k as in the case of binary decomposition, since there are only 3 ^k as in the case of ternary decomposition, full search The amount of computation will not increase too much.

In the above relevance determination device, the real matrix decomposition unit includes:
May be decomposed by solving the cost function. Here, Q is the real matrix, M is the basis matrix, C is the coefficient matrix, and λ is a coefficient. Even with this configuration, an error when a real matrix is decomposed into a product of a base matrix and a coefficient matrix can be evaluated as a cost, and the real matrix can be decomposed easily and accurately, and the coefficient matrix can be sparse. Therefore, the product of the feature vector and the real matrix can be calculated at high speed. Specifically, a real matrix can be decomposed with a base matrix and a coefficient matrix that minimize this cost function (which satisfies a predetermined convergence condition).

In the above-described relevance determination device, the real matrix decomposition unit fixes the elements of the base matrix and optimizes the elements of the coefficient matrix by a proximity gradient method, and fixes the elements of the coefficient matrix Then, the base matrix and the coefficient matrix may be obtained by repeating the second update that optimizes the elements of the base matrix by full search. With this configuration, a real matrix can be easily decomposed. Note that when fixing the elements of the coefficient matrix, the number of combinations to be searched when determining each row of the base matrix, 2 ^k as in the case of binary decomposition, since there are only 3 ^k as in the case of ternary decomposition, full search The amount of calculation does not increase too much.

In the above relevance determination device, the real matrix decomposition unit includes:
May be decomposed by solving the cost function. Here, Q is the real matrix, M is the basis matrix, C is the coefficient matrix, and P is a set of a plurality of the feature vectors. With this configuration, since the error due to the decomposition of the product of the feature vector and the real matrix is evaluated as a cost using a plurality of feature vectors instead of the error of the decomposition of the real matrix (data-dependent decomposition), the feature vector and the real matrix Can be approximated with higher accuracy. Specifically, a real matrix can be decomposed with a base matrix and a coefficient matrix that minimize this cost function (which satisfies a predetermined convergence condition).

  In the above-described relevance determination device, the real matrix decomposition unit fixes the elements of the base matrix and optimizes the elements of the coefficient matrix by a least square method, and fixes the elements of the coefficient matrix Then, the base matrix and the coefficient matrix may be obtained by repeating the second update for optimizing the elements of the base matrix by solving the combined optimization problem. With this configuration, a real matrix can be easily decomposed. Note that the union optimization problem can be solved using an algorithm such as a greedy algorithm, tab search, or simulated annealing.

In the above relevance determination device, the real matrix decomposition unit includes:
May be decomposed by solving the cost function. Here, Q is the real matrix, M is the basis matrix, C is the coefficient matrix, P is a set of a plurality of feature vectors, and λ is a coefficient. With this configuration, since the error due to the decomposition of the product of the feature vector and the real matrix is evaluated as a cost using a plurality of feature vectors instead of the error of the decomposition of the real matrix (data-dependent decomposition), the feature vector and the real matrix The product of the feature vector and the real number matrix can be calculated at high speed by making the coefficient matrix sparse. Specifically, a real matrix can be decomposed with a base matrix and a coefficient matrix that minimize this cost function (which satisfies a predetermined convergence condition).

  In the above-described relevance determination device, the real matrix decomposition unit fixes the elements of the base matrix and optimizes the elements of the coefficient matrix by a proximity gradient method, and fixes the elements of the coefficient matrix Then, the base matrix and the coefficient matrix may be obtained by repeating the second update for optimizing the elements of the base matrix by solving the combined optimization problem. With this configuration, a real matrix can be easily decomposed. Note that the union optimization problem can be solved using algorithms such as a greedy algorithm, tabu search, and simulated annealing.

In the above relevance determination device, the real matrix decomposition unit includes:
As a cost function, by decomposing the real matrix by solving the cost function to obtain initial values of elements of the basis matrix and the coefficient matrix, or
May be used as a cost function, and the real matrix may be decomposed by solving the cost function to obtain initial values of elements of the base matrix and the coefficient matrix. With this configuration, the base matrix and coefficient matrix obtained by data-independent decomposition are used as initial values, so it is possible to start repetitive updating for data-dependent decomposition from a sufficiently good initial solution, thus effectively reducing costs. Can be reduced.

  In the above relevance determination device, the real number matrix decomposition unit obtains a plurality of types of the base matrix and the coefficient matrix by changing initial values of the elements of the base matrix and the coefficient matrix, and the cost function is minimized. The real matrix may be decomposed by adopting the basis matrix and the coefficient matrix. With this configuration, it is possible to reduce variations due to initial values and further reduce the error in decomposition.

  In the above relevance determination device, the feature vector may be a HOG feature, the plurality of real vectors may be a plurality of weight vectors corresponding to parameters of a plurality of linear classifiers, and the vector calculation The unit may identify the feature vector for each of the plurality of criteria by using an identification function of the plurality of linear classifiers as the determination of the relevance. With this configuration, it is possible to speed up feature vector identification by a plurality of linear classifiers.

  In the above-described relevance determination device, when the feature vector and the plurality of real vectors have one or a plurality of parameters, the real matrix generation unit arranges the plurality of real vectors according to the order of the parameters. A real number matrix is generated, and the vector calculation unit is a continuous function related to the parameter for each of a plurality of vectors constituting the coefficient matrix and having the same direction as the direction in which the plurality of real number vectors are arranged. The parameter that expresses and maximizes the discriminant function may be obtained as a parameter value of the feature vector. With this configuration, when a real matrix is generated by combining a plurality of real vectors, a real matrix is generated by arranging a plurality of real vectors in the order of parameters in which they change smoothly. Therefore, the parameter value of the feature vector can be obtained with high resolution.

  In the above-described relevance determination device, the feature vector may be a vector to be clustered by k-means clustering, the real vector may be a representative vector in k-means clustering, and the vector calculation unit May perform a clustering process including a calculation of a distance between the feature vector and the representative vector as the determination of the relevance. With this configuration, the calculation of the distance between the feature vector and the representative vector in k-means clustering can be speeded up.

  In the above-described relevance determination apparatus, the feature vector may be a vector to be subjected to an approximate nearest neighbor search by k-means tree, and the real vector is a representative vector registered in a node of a k-ary tree. The vector calculation unit may perform a clustering process including calculation of a distance between the feature vector and the representative vector as the determination of the relevance. With this configuration, it is possible to speed up the calculation of the distance between the feature vector in the approximate nearest neighbor search by k-means tree and the representative vector registered in the node of the k-ary tree.

  In the above-described relevance determination device, the feature vector may be a vector that represents a feature amount of an image. With this configuration, it is possible to speed up the inner product calculation of the feature vector and the plurality of real vectors in the calculation of the feature amount of the image.

  The relevance determination program according to the present embodiment is a relevance determination program for causing a computer to function as the above relevance determination device. Even with this configuration, in order to calculate the inner product of a feature vector and each of a plurality of real vectors, a real matrix composed of a plurality of real vectors is decomposed into a discrete value base matrix and a coefficient matrix, and then the feature vector and the base Since the product with the matrix is calculated and the product with the coefficient matrix is further calculated, the result of the inner product operation between the feature vector and each of the plurality of real vectors can be obtained at high speed, and thus the feature vector and the plurality of real vectors can be obtained. Relevance can be determined at high speed.

The relevance determination method according to the present embodiment includes a feature vector acquisition step for acquiring a binarized feature vector, a real matrix composed of a plurality of real vectors, a coefficient matrix, and binary or ternary discrete elements. A real matrix decomposing step for decomposing the product with a base matrix composed of a plurality of basis vectors having only values, and calculating an inner product of the feature vector and each of the plurality of real vectors, the feature vector and the base matrix A vector operation step of calculating the product of the product and the coefficient matrix, and using the result to determine the relationship between each of the plurality of real vectors and the feature vector, It has the composition which includes. Even with this configuration, in order to calculate the inner product of a feature vector and each of a plurality of real vectors, a real matrix composed of a plurality of real vectors is decomposed into a discrete value base matrix and a coefficient matrix, and then the feature vector and the base Since the product with the matrix is calculated and the product with the coefficient matrix is further calculated, the result of the inner product operation between the feature vector and each of the plurality of real vectors can be obtained at high speed, and thus the feature vector and the plurality of real vectors can be obtained. Relevance can be determined at high speed.
4-3. effect

  According to the present embodiment, it is possible to speed up the inner product calculation of the binarized feature vector and each of the plurality of real vectors, and to determine the relevance between such a feature vector and each of the plurality of real vectors. It can be done at high speed.

  Hereinafter, the feature amount calculation apparatus of the present embodiment will be described with reference to the drawings.

4-4. A situation where there are a plurality of real vectors First, a situation where there are a plurality of real vectors whose inner product with the feature vector is to be calculated will be described. FIG. 35 is a diagram illustrating an example of a linear SVM when a person in an image is identified based on a plurality of identification criteria. In this example, for an input feature vector, as shown in FIG. 35, it is not simply identification of whether or not there is a person in the image of the feature vector, but whether it is “adult (front)”. No, “adult (horizontal)”, and “child (front)” are identified. That is, there are a plurality of criteria for identifying feature vectors. In this case, as shown in FIG. 35, the weight parameter (hereinafter also referred to as “dictionary”) w of the evaluation formula f (x) of the wisdom line SVM has a plurality (w ₁ , w ₂ , w ₃ ) for each identification criterion. ,..., W _L ) and a plurality of bias b (b ₁ , b ₂ , b ₃ ,..., B _L ) must be prepared for each identification criterion.

FIG. 36 is a diagram illustrating an example of a linear SVM when a person in an image is identified by a plurality of identification criteria according to the distance to the subject. In this example, as shown in FIG. 36, for a certain feature vector input, the person identification is robust to the distance to the subject, that is, the scale change of the subject in the image. In addition to identifying whether there is an adult in the image of the feature vector, whether it is “adult (far)”, whether it is “adult (medium distance)”, “adult (near) ) ”Or not. That is, in this case as well, there are a plurality of criteria for identifying feature vectors. Therefore, as shown in FIG. 36, there are a plurality of linear SVM dictionaries w for each identification criterion (w ₁ , w ₂ , w ₃ ,..., W _L ) must be prepared, and a plurality of bias b (b ₁ , b ₂ , b ₃ ,..., B _L ) must be prepared for each identification criterion.

As described above, when a certain feature vector is identified by a plurality of criteria, the plurality of criteria are often similar to each other. FIG. 35 and FIG. 36 also show such an example, that is, in the example of FIG. 35, “adult (front)” and “adult (horizontal)” have the common point of adults, and “adult (front) ) ”And“ Children (front) ”have the common feature of people, and“ Adults (front) ”,“ Adults (horizontal) ”and“ Children (front) ”have the common features of people. Have. Also in the example of FIG. 36, “adult (far)”, “adult (medium distance)”, and “adult (near)” have a common point of “adult”. Therefore, the dictionaries (w ₁ , w ₂ , w ₃ ,..., W _L ) that are a plurality of real vectors in FIGS. 35 and 36 are similar to each other. In k-means clustering, representative vectors that are k real vectors are often similar to each other. The relevance determination apparatus according to the present embodiment speeds up processing by taking advantage of the property that a plurality of real vectors are similar to each other.

4-5. First Example of Fourth Embodiment FIG. 37 is a block diagram showing a configuration of a feature quantity computing device 106 of a first example of the fourth embodiment. The feature amount calculation device 106 includes a content acquisition unit 161, a feature vector generation unit 162, a feature vector binarization unit 163, a real number matrix acquisition unit 164, a real number matrix decomposition unit 165, a vector calculation unit 166, a database 167.

  As will be described later, the feature amount computing device 106 of this example relates the feature vector and the plurality of real vectors by a vector operation involving an inner product operation of the feature vector and a plurality of real vectors stored in the database as dictionary data. It functions as a relevance determination device that determines sex. That is, the feature calculation device 106 corresponds to the relevance determination device of the present embodiment.

  The feature amount computing device 106 as a relevance determination device is realized by a computer executing the relevance determination program according to the present embodiment. The relevance determination program may be recorded on a recording medium and read from the recording medium by a computer, or may be downloaded to a computer through a network.

  The content acquisition unit 161 acquires content data such as image data, audio data, and character data. These content data may be provided from an external device or may be generated by the content acquisition unit 161. For example, the content acquisition unit 161 may be a camera, and image data may be generated there as content data.

The feature vector generation unit 162 generates a D-dimensional feature vector from the content data acquired by the content acquisition unit 161. For example, when the content is an image, the feature vector generation unit 162 extracts the feature amount of the image. The feature vector binarization unit 163 binarizes the D-dimensional feature vector generated by the feature vector generation unit 162, and each element has only a binary value of -1 and 1, and a d-dimensional binary vector pε {-1, 1} ^d is generated. This feature vector binarization unit 163 corresponds to the “feature vector acquisition unit” of the present embodiment.

  The configuration including the content acquisition unit 161, the feature vector generation unit 162, and the feature vector binarization unit 163 may be any configuration that can finally acquire a binarized feature vector. For example, the content acquisition unit 161 and the feature vector generation unit 162 may be omitted, and the feature vector binarization unit 163 may acquire a feature vector from an external device and binarize the acquired feature vector. The vector binarization unit 163 may directly acquire a binarized feature vector from an external device.

The real matrix acquisition unit 164 acquires a plurality of d-dimensional real vector q _n ∈R ^d (n = 1, 2,..., L). The plurality of real number vectors q _n may be provided from an external device, may be read from a storage device (not shown) of the feature value computing device 106, and is generated by the real number matrix acquisition unit 164. It may be. Each real vector q _n has a real number including a floating-point number in its element. Here, the arrangement of a plurality of real number vectors q _n is expressed as a real number matrix Q = (q ₁ , q ₂ ,..., Q _L ) ∈R ^d x ^L.

When a real matrix Q in which a plurality of real vectors q _n are combined in this way, the plurality of linear SVMs in FIGS. 35 and 36 can be expressed together as in the following equation (28).

As shown in FIG. 38, the real matrix decomposition unit 105 decomposes the real matrix Q of d rows and L columns into a product of a binary base matrix Mε {−1,1} ^dxk and a coefficient matrix. Specifically, the real matrix decomposition unit 105 decomposes the real matrix Q of d rows and L columns into a base matrix M having binary elements and a coefficient matrix C having real elements by the following equation (29). .
Here, as shown in FIG. 38, M = (m ₁ , m ₂ ,..., M _k ) ∈ {−1, 1} ^dxk and C = (c ₁ , c ₂ ,..., C _L ) ^T ∈ R ^kxL .

That is, the basis matrix M is composed of k basis vectors m _i , where the basis vector _mi is a d-dimensional binary vector having elements of only −1 and 1, and thus the basis matrix M is , Is a binary matrix of d rows and k columns in which elements take only -1 and 1.

The coefficient matrix C includes L (L is the number of classes) coefficient vectors c _n , where the coefficient vector c _n is an element of real coefficients related to k (k is the basis number) basis vectors. As a k-dimensional real vector. Of course, it is preferable to decompose Q and MC so that they coincide as much as possible, but an error may be included. Hereinafter, a method in which the real matrix decomposition unit 105 decomposes the real matrix Q as shown in Expression (29) will be described.

4-5-1. First Decomposition Method As a first decomposition method, a data-independent decomposition method will be described. In the first decomposition method, the real matrix decomposition unit 105 performs decomposition by solving a cost function g ₁ of the following equation (30) representing a decomposition error.
However, the base matrix M is binary, and Mε {−1, 1} ^dxk .

The real matrix decomposition unit 105 solves the cost function g _{1 according} to the following procedure.
(1) The base matrix M and the coefficient matrix C are initialized at random.
(2) By fixing the elements of the base matrix M and optimizing the elements of the coefficient matrix C by the least square method, the elements of the coefficient matrix C are updated so that the cost function g ₁ is minimized.
(3) The elements of the coefficient matrix C are fixed, and the elements of the base matrix M are updated by a full search so that the cost function g ₁ is minimized. The full search which is this minimization algorithm will be described in detail later.
(4) Repeat (2) and (3) until convergence. For example, when the cost function g ₁ satisfies a predetermined convergence condition (for example, the amount of decrease is equal to or less than a certain value), it is determined that the cost function g ₁ has converged.
(5) The solutions obtained in steps (1) to (4) are held as candidates.
(6) Steps (1) to (5) are repeated, and the candidate base matrix M and candidate coefficient matrix C that have the smallest cost function g ₁ are adopted as the final results. Note that the steps (1) to (5) need not be repeated, but the problem of initial value dependency can be avoided by repeating a plurality of times.

Next, the update process of the base matrix M in step (3) will be described. As surrounded by the broken line frame in FIG. 39, the element of the row vector of the jth row of the base matrix M depends only on the element of the jth row of the real number matrix. Therefore, the value of each row vector of the base matrix M can be optimized independently of other rows, so that the base matrix M can perform an exhaustive search (full search) for each row. There are only 2 ^k row vectors in the j-th row of the base matrix M in the case of binary decomposition as in this example (note that there are only 3 ^{k in} the case of ternary decomposition in the second example described later). not exist). Therefore, the real number matrix decomposing unit 105 comprehensively checks them and employs a row vector that minimizes the cost function g ₁ . This is applied to all the row vectors of the base matrix M to update the elements of the base matrix M.

4-5-2. Second Decomposition Method As a second decomposition method, a data-independent decomposition method that makes the coefficient matrix C sparse will be described. In the second decomposition techniques, real matrix decomposition unit 105 performs degradation by solving the cost function g ₂ of the formula is an exploded error (31).
However, the base matrix M is binary, and Mε {−1, 1} ^dxk . Also, | C | ₁ is the L1 norm of the element of the coefficient matrix C, and λ is its coefficient.

The real matrix decomposition unit 105 solves the cost function g _{2 according} to the following procedure.
(1) The base matrix M and the coefficient matrix C are initialized at random.
(2) The elements of the base matrix M are fixed, and the elements of the coefficient matrix C are optimized by the proximity gradient method.
(3) The elements of the coefficient matrix C are fixed, and the elements of the base matrix M are updated by a full search so that the cost function g ₂ is minimized.
(4) Repeat (2) and (3) until convergence. For example, when the cost function g ₂ satisfies a predetermined convergence condition (for example, the amount of decrease is equal to or less than a certain value), it is determined that the cost function g ₂ has converged.
(5) The solutions obtained in steps (1) to (4) are held as candidates.
(6) Steps (1) to (5) are repeated, and the candidate base matrix M and candidate coefficient matrix C that have the smallest cost function g ₂ are adopted as the final results. Note that the steps (1) to (5) need not be repeated, but the problem of initial value dependency can be avoided by repeating a plurality of times.

  According to the second decomposition method, the coefficient matrix C can be made sparse. By making the coefficient matrix C sparse, in the calculation of the product MC, the portion related to the zero element of the coefficient matrix C can be omitted, and the inner product calculation can be performed at higher speed.

4-5-3. Third decomposition method Next, a third decomposition method will be described. In the first decomposition method, the decomposition error is expressed as the cost function g _1.
To minimize this decomposition error. However, what is actually desired to be approximated after approximating the real matrix to the product of the base matrix and the coefficient matrix is the product Q ^T p of the feature vector and the real matrix.

Therefore, in the third decomposition method, S feature vectors p are collected in advance, and the sum of ^these is ^defined as P∈R ^dxS . And the decomposition error
And minimize this. That is, in the third decomposition method, the real matrix decomposition unit 105 performs decomposition by solving the cost function g ₃ of the following equation (32).
According to this cost function g ₃ , the real matrix Q is decomposed according to the actual data distribution, so that the approximation accuracy at the time of decomposition is improved.

This approximation decomposition can be performed by obtaining the basis vectors m _i sequentially. The procedure of the third decomposition method is as follows.
(1) The base matrix M and the coefficient matrix C are obtained by the first or second decomposition method and set as initial values thereof.
(2) The elements of the base matrix M are fixed, and the elements of the coefficient matrix C are optimized by the least square method.
(3) The elements of the base matrix M are updated by fixing the elements of the coefficient matrix C and optimizing the elements of the base matrix M. The update process of the base matrix M will be described later.
(4) Repeat (2) and (3) until convergence, and hold the base matrix M and coefficient matrix C minimizing the cost function g ₃ as candidates.
(5) Steps (1) to (6) are repeated, and a base matrix M and a coefficient matrix C in which the cost function g ₃ is minimized are adopted as final results. In step (1), since the base matrix M and the coefficient matrix C are optimized again by the first or second decomposition method, the initial values are changed. In addition, although step (5) may not be repeated, the problem of initial value dependency can be reduced by repeating a plurality of times.

  Next, the update process of the base matrix M in step (3) will be described. In the case of data-dependent decomposition, the value of the row vector of the base matrix M is no longer independent of other rows and is dependent. Since the elements of the base matrix M are binary or ternary, that is, discrete values, the optimization of the base matrix M becomes a combinatorial optimization problem. Therefore, for optimization of the base matrix M, for example, an algorithm such as a greedy algorithm, a tab search, or a simulated annealing can be used. Since a good initial value is obtained in step (1), these algorithms can satisfactorily minimize the decomposition error.

For example, when the greedy algorithm is used, the base matrix M is optimized by the following procedure.
(3-1) T elements of the base matrix M are selected at random.
(3-2) 2 ^T combinations (3 ^{T in} the case of ternary decomposition described later) are tried, and the one that minimizes the cost function g ₃ is adopted.
(3-3) Repeat step (3-1) and step (3-2) until convergence.

4-5-4. Fourth Decomposition Method The fourth decomposition method is a combination of the second decomposition method and the third decomposition method. Specifically, the real matrix decomposition unit 105 performs decomposition by solving the cost function g ₄ of the following equation (33).
According to the cost function g ₄ , the real matrix Q is decomposed according to the actual data distribution, so that the approximation accuracy at the time of decomposition is improved and the coefficient matrix C can be made sparse. That is, both of the advantages of the second decomposition method and the third decomposition method can be obtained. The specific decomposition procedure is the same as that in the third decomposition method.

4-5-5. Modifications of First and Second Decomposition Methods The first and second data-independent decomposition methods described above are k ² ways (k ³ ways in the case of ternary decomposition), where k is the number of decompositions. Since search is necessary, application is difficult when k is large. In such a case, the mutual similarity of the real vectors q _n belonging to the real matrix Q is examined in advance, the similar real vectors are clustered, and the first or second decomposition method is applied to each cluster. do it.

The vector calculation unit 106 performs a calculation using a feature vector. The specific contents of the calculation will be specifically described later together with an application example of the feature amount calculation apparatus 100 of the present example. The calculation using the feature vector includes calculation of a product Q ^T p of the binarized feature vector pε {−1,1} ^d and the real matrix Q decomposed by the real matrix decomposition unit 105. It is. In the following, first, the calculation of the product Q ^T p will be described.

The product Q ^T p can be transformed into the following equation (34).
Here, m _i ^T p is an inner product of binary vectors. Further, c _{n, i} is the i th element of the coefficient vector c _n of the n th class, that is, the element of i row and n column of the coefficient matrix C. The inner product m _i ^T p between the binary vectors can be calculated extremely quickly. The reason is as follows.

An inner product between binary vectors can be reduced to a Hamming distance calculation. The Hamming distance is obtained by counting bits having different values in two binary codes, and the Hamming distance between two binary vectors is obtained by counting the number of elements having different values. Here, m _i and p Hamming distance D _hamming (m _i, p) of the writing, the inner product m _i ^T p is
D _hamming (m _i, p) and the following formula relation (35).
Here, as described above, d is the number of bits of the binary code.

The calculation of the Hamming distance is extremely fast because it can be calculated by counting the bits in which 1 stands after applying XOR in two binary codes. If the binary vector is expressed by a binary code (bit sequence of 0 and 1), the Hamming distance can be calculated by the following equation (36).
Here, XOR function is an operation of the exclusive OR when considering the m _i and p in binary code representation, BITCOUNT function is processing to enumerate the number of bits 1 of the binary code is standing .

In summary, the product Q ^T p can be transformed as shown in the following equation (37).
That is, the d-bit Hamming distance calculation is performed k times, the weighted sum related to the coefficient matrix C is calculated for k Hamming distances, and the sum of the constant terms is Q ^T p. Therefore, if k is sufficiently small, Q ^T p can be calculated much faster than calculating with floating point precision.

  The database 107 stores the product of the base matrix M and the coefficient matrix C as dictionary data for a plurality of real number matrices Q decomposed by the real number matrix decomposition unit 105. The vector calculation unit 106 reads the product of the base matrix M and the coefficient matrix C from the database 107 and performs the above calculation.

  As described above, according to the feature amount computing device 100 of this example, even when the computation processing using the feature vector includes the product computation of the feature vector and the real number matrix, the feature vector is binarized. Since the real matrix is also decomposed into the product of the base matrix and coefficient matrix that are binary matrices, the product of the feature vector and the base matrix is calculated in the product of the feature vector and the real matrix. By further calculating the product with the coefficient matrix, the product operation of the feature vector and the real number matrix can be speeded up.

  In addition, since a plurality of real vectors are combined into one real matrix and the real matrix is decomposed into a binary matrix, a base matrix and a coefficient matrix, each real vector is decomposed as in the prior application technique. In comparison, the number of basis vectors constituting the basis matrix, that is, the number of basis can be reduced. In principle, it is possible to set the number of bases to one or less per class (that is, base number k ≦ number of classes L).

4-6. Expansion of the first example of the fourth embodiment In the first example described above, the binary vectors m _i and p are changed to m _i ε {−1,1} ^d and pε {−1,1, respectively. } It is defined as ^d, and it is explained that the product operation Q ^T p becomes faster by decomposing a real matrix into a product of a binary base matrix and a real coefficient matrix. However, even if m _i and p are more general binary vectors m _{i ′} ε {−a, a} ^d and p′ε {−a, a} ^d , their high-speed product operation can be performed. In this case, since m _i ′ ^T p ′ = a ² (m _i ^T p), the inner product of binary vectors defined by −1 and 1 may be multiplied by a ² .

Further, even if the feature vector and the base vector are arbitrary binary vectors m _{i ″} ε {α, β} ^d , p ″ ε {γ, δ} ^d , high-speed inner product calculation is possible. Here, the coefficients α, β, γ, and δ are real numbers, and α ≠ β and γ ≠ δ. In this case, m _{i ″} and p ″ are obtained by performing linear transformation on the elements of the binary vectors m _i and p defined by −1 and 1, and the following equations (38) and (39) are obtained. It is expanded like this.
Note that the bold “1” in the equations (38) and (39) is a vector having a length of d and all elements being 1. Further, A, B, C, and D in the expressions (38) and (39) are real numbers, and may be calculated in advance so that the expressions (38) and (39) are satisfied.

The inner product m _{i ″} ^T p ″ can be expanded as in the following equation (40).
The calculation in parentheses in the equation (40) is the inner product of binary vectors consisting of -1 and 1. Therefore, even when the feature vector is a binary vector having an arbitrary binary element and the real matrix is expanded into a product of a binary base matrix and a real coefficient matrix, high-speed calculation is possible. .

4-7. Second Example of Fourth Embodiment Next, a feature value computing device of a second example will be described. The configuration of the feature quantity computing device of the second example is the same as that of the first example shown in FIG. In the first example, the real number matrix decomposition unit 105 decomposes the real number matrix Q into a binary base matrix and a real number coefficient matrix according to Equation (28), but the real number matrix decomposition unit of the feature quantity computing device 100 of this example. 105 decomposes the real matrix into a ternary basis matrix and a real coefficient matrix.

The real matrix decomposition unit 105 decomposes the real matrix QεR ^dxL of d rows and L columns into a product of a ternary basis matrix and a real coefficient matrix. Specifically, the real matrix decomposing unit 105 converts a d-row L-column real matrix QεR ^dxL into a base matrix M having ternary elements and a coefficient matrix C having real elements by the following equation (41). Disassembled into
Here, M = (m ₁ , m ₂ ,..., M _k ) ∈ {−1, 0, 1} ^dxk and C = (c ₁ , c ₂ ,..., C _L ) ^{T ∈R kxL} . . That is, the basis matrix M consists of k basis vectors m _i, where the basis vectors m _i, the element is a three-value vector of d-dimensional take -1,0, and 1 only, therefore, basal The matrix M is a ternary matrix of d rows and k columns having elements of only −1, 0, and 1.

The coefficient matrix C is composed of L (L is the number of classes) coefficient vectors c _n , where the coefficient vector c _n is a k-dimensional real number having real coefficients related to k basis vectors as elements. Is a vector. Of course, it is preferable to decompose Q and MC so that they coincide as much as possible, but an error may be included. The real number matrix decomposition unit 105 can decompose the real number matrix Q by the first to third decomposition methods in the same manner as in the first example.

Vector operation unit 106 calculates product Q ^T p. Hereinafter, the vector calculation unit 106 that calculates the product Q ^T p is also specifically referred to as a product calculation unit 106. The product Q ^T p can be transformed into the following equation (42).
Here, m _i ^T p is the inner product of the ternary vector m _i and binary vector p. Here, the product calculation unit 106 uses, instead of the ternary vector m _i , a 0 permutation vector m _i ^bin , a filter vector m _i ^filter , and a 0 element number z _i defined below.

First, the product calculation unit 106 replaces the 0 element of m _i with −1 or 1. For each element of m _i, to replace it to -1, is either replaced by 1, it may be any. By this replacement, a 0 replacement vector m _i ^bin ε {-1, 1} ^d is generated. This 0 permutation vector m _i ^bin ε {-1, 1} ^d is a binary vector.

Also, product unit 106 replaces the 0 element of m _i -1, replace the elements other than 0 to 1. By this replacement, a filter vector m _i ^filter ε {-1, 1} ^d is generated. This filter vector m _i ^filter is also a binary vector.

Further, the product calculation unit 106 obtains the number of elements z _i of 0 of m _i . z _i is an integer. The product calculation unit 106 uses these binary vectors m _i ^bin , filter vectors m _i ^filter , and 0 element number z _i to ^convert m _i ^T p in equation (42) to the following equations (43) and Calculate according to (44).
Here, the AND function of Expression (44) is an operation of taking a logical product when a binary vector is considered in binary code expression.

Hereinafter, the derivation of the equations (43) and (44) will be described using the specific example of FIG. FIG. 40 is a diagram illustrating a calculation example of this example. In the example of FIG. 40, p = {− 1,1, −1,1, −1,1} and m _i = {− 1,0,1,0,1,1}. In this ^{_{example, m i bin = {- 1}} , *, 1, *, 1,1} a. Here, “*” represents any one of −1 or 1. Further, m _i ^filter = {1, -1,1, -1,1,1}, and z _i = 2.

The exclusive OR of p and m _i ^bin in equation (44) is XOR (p, m _i ^bin ) = {− 1, *, 1, *, 1, −1}, that is, p and m _i Of the elements of, elements that are different from non-zero, that is, elements that are pairs of -1 and 1 or 1 and -1, are 1, and elements that are pairs of -1 and -1 or 1 and 1 are -1. .

Next, the logical product of the exclusive OR and m _i ^filter is AND (XOR (p, m _i ^bin ), m _i ^filter )) = {− 1, −1,1, −1,1, − 1}, and the elements of p and m _i, 1 is standing on elements that differ nonzero, is -1 otherwise. When this bit count is taken, the number of elements that are 1, that is, the number of non-zero and different elements is counted, and D _{filterd_hamming} (p, m _i ^bin , m _i ^filter ) = 2.

Here, among the elements p and m _i , the number of elements that are a set of 1 and 1 or −1 and −1 is the total number of elements d = 6, and the number of non-zero and different elements D _{filterd_hamming} = This is obtained by subtracting the number of elements z _i = 2 which are 2 and 0. That is, the number of elements that are a set of 1 and 1 or −1 and −1 = d−D _{filterd_hamming−} z _i = 6-2-2 = 2.

m _i ^T p is an element (a set of -1 and 1 or 1 and -1) from the number of elements (a set of elements whose product is 1) that is a set of 1 and 1 or -1 and -1. Since it is equal to a value obtained by subtracting the number of elements whose product is −1, m _i ^T p = (d−D _{filterd_hamming−} z _i ) −D _{filterd_hamming} = d−z _i −2D _{filterd_hamming} ), And the value is 6-2-2 × 2 = 0. Of course, this result is as follows. P ^T m _i = {− 1,1, −1,1, −1,1} × {−1,0,1,0,1,1} = 1 + 0 + (− 1 ) +0 + (− 1) + 1 = 0.

Summarizing the equations (42) to (44), the product Q ^T p can be transformed as the following equation (45).
The product calculation unit 106 calculates the product Q ^T p by this equation (45).

The function D _{filterd_hamming} (p, m _i ^bin , m _i ^filter ) is very similar to the Hamming distance calculation, only an AND operation is added. Therefore, the Q∈R ^d x ^L, even when decomposed into a product of the three value matrix and the coefficient matrix, rather than calculating the Q ^T p in floating point precision, so that it can calculate the Q ^T p much faster Become.

As described above, the advantage of decomposing a d-dimensional real matrix Q∈R ^dxL into a product of a ternary basis matrix and a coefficient matrix instead of a binary is that the approximation of Expression (37) is smaller in number. The basis matrix of the basis number of. That is, since the number of bases can be kept small, the speed is further increased.

4-8. Expansion of the second example of the fourth embodiment In the second example described above, the binary vector p and the ternary vector m _i are respectively expressed as pε {−1,1} ^d and m _i ε {−. 1,0,1} is defined as ^d, explain that inner product p ^T m _i is faster by decomposing the real matrix comprising a plurality of real vector the product of the basis matrix and a coefficient matrix of three-valued did. However, even if p and m _i are more general binary vectors p′∈ {−a, a} ^d and ternary vectors m _i ∈ {−a, 0, a} ^d , their high-speed inner product operations can be performed. Is possible. In this case, since p ′ ^T m _i ′ = a ² (p ^T m _i ), the inner product of the binary vectors defined by −1 and 1 may be multiplied by a ² .

Furthermore, even if the binary vector p and the ternary vector m _i are generalized as pε {α, β} ^d and m _i ε {γ−δ, γ, γ + δ} ^d , high-speed inner product calculation is possible. Here, α, β, γ, and δ are real numbers, and α ≠ β and δ ≠ 0. In this case, by performing a linear transformation of the formula (46) and (47) to each element of the m _i and p, m _i'' and p'' is obtained, respectively.
Note that the bold “1” in the equations (46) and (47) is a vector having a length d and all elements being 1. Further, A, B, C, and D in the equations (46) and (47) are real numbers, and are calculated in advance so that the equations (46) and (47) are satisfied.

The inner product m _{i ″} ^T p ″ can be expanded as shown in the following equation (48).
The calculation in parentheses in the equation (48) is an inner product of binary vectors consisting of -1 and 1, or an inner product of a binary vector consisting of -1 and 1 and a ternary vector consisting of -1, 0 and 1. is there. Therefore, even when the feature vector is an arbitrary binary vector and the real matrix is expanded using the generalized ternary matrix as described above, the product of such a feature vector and the real matrix can be increased at high speed. Can be calculated.

4-9. Application Example Next, calculation processing in the vector calculation unit 106 will be described. The vector calculation unit 106 in the first and second examples described above involves calculation of a product of the binarized feature vector p and a real matrix Q obtained by combining a plurality of real vectors q. There are various kinds of arithmetic processing. That is, the above example of the present embodiment can be applied to various apparatuses that perform arithmetic processing using feature vectors.

4-9-1. First Application Example of Fourth Embodiment In this application, the present embodiment is applied to an object recognition apparatus that recognizes a plurality of types of objects by SVM using HOG feature values. FIG. 41 is a block diagram illustrating a configuration of the object recognition apparatus. The object recognition device 107 includes a pyramid image generation unit 171, a HOG feature amount extraction unit 172, a binary code conversion unit 173, a parameter determination unit 174, a parameter matrix decomposition unit 175, a linear SVM identification unit 176, and a peak detection. Part 177.

  The pyramid image generation unit 171 acquires an image as an input query, and generates a G-stage pyramid image obtained by reducing the image at a plurality of scales. Thereby, it is possible to deal with objects of different sizes. The pyramid image generation unit 171 corresponds to the content acquisition unit 161 illustrated in FIG. The HOG feature amount extraction unit 172 divides the image at each stage of the pyramid image into blocks each having a size of 16 × 16 pixels, and extracts the HOG feature amount from each block. The HOG feature amount extraction unit 172 extracts a D-dimensional feature amount from each block. The HOG feature quantity extraction unit 172 corresponds to the feature vector extraction unit 162 shown in FIG. The binary code conversion unit 173 converts the D-dimensional feature value given to each cell into a d-dimensional binary vector. This binary code conversion unit 173 corresponds to the feature vector binarization unit 163 shown in FIG.

The parameter determination unit 174 uses the weight vector w _n used in the linear SVM in the linear SVM identification unit 176 for each type of object to be recognized (adult, child, car, motorcycle, etc., which is defined by parameters). (N = 1, 2,..., L) and a real bias b _n (n = 1, 2,..., L) are determined. Parameter determination unit 174, using the feature quantity prepared for the learning, and determining the L type of weight vector w _n and the bias b _n by the learning process to generate a weight matrix W summarizes the weight vector w _n . This parameter determination unit 174 corresponds to the real matrix acquisition unit 164 shown in FIG. The parameter matrix decomposition unit 175 decomposes the weight matrix W into a product of a discrete value base matrix and a coefficient matrix according to the equation (29) or the equation (41) described in the first or second example. The parameter matrix decomposition unit 175 corresponds to the real number matrix decomposition unit 165 shown in FIG.

The linear SVM identifying unit 176 identifies feature vectors using the linear SVM. The linear SVM identification unit 176 first configures a window by collecting s _x × _sy blocks as a group. A feature vector extracted from one window is an s _x × s _y × d-dimensional vector. The linear SVM discriminating unit 176 applies the linear SVM of the following expression (49) to this feature vector.
Here, the product operation W ^T x in the linear SVM can be realized by the high-speed product operation of the real matrix and the binary vector described as the first or second example.

  The detection result may be hardened in the vicinity of the detection position. Therefore, the peak detection unit 177 sets a position where the value of f (x) is maximized around as a representative detection position. The linear SVM identification unit 176 and the peak detection unit 177 are configured to perform processing using feature vectors, and correspond to the vector calculation unit 166 in FIG.

Next, an example will be described in which the object recognition apparatus 107 detects a rotatable object based on the HOG feature value. Figure 42, for road signs to rotate, shows a case of creating a dictionary w _n and the bias b _n at each rotation angle. In FIG. 42, the left-right direction indicates the rotation angle θ of the road sign.

The traditional approach to obtain a dictionary w _n and the bias b _n for each rotation angle by performing a learning process. Thereafter, the HOG feature amount is extracted from the input image, and this road sign is detected by applying a window (sliding window) L times. However, such a conventional method requires L times of inner product calculation per window, which increases the amount of calculation. Further, the angular resolution of detection is 2 pi / L, which is rough.

Therefore, in this application example, the parameter determination unit 174 as a matrix Q collectively dictionary w _n, SVM discrimination unit 176, together the inner product calculation of the plurality of dictionaries w _n and the feature vector p by the following equation (50) Do.
By decomposing into k integer bases in this way, processing can be performed by k times of inner product operation of binary and binary or inner product operation of binary and ternary per window. At this time, since adjacent dictionaries are similar, the number k of integer bases can be reduced, and in principle, one or less (k ≦ L) can be set per class.

In this application example, the peak detection unit 177 further increases the accuracy of the detection resolution focusing on the property of the coefficient matrix C. FIG. 43 is a diagram illustrating the properties of the coefficient matrix C. When the real vector q _n changes according to the rotation angle θ as a parameter, a plurality of real vectors q _n are combined to generate a real matrix Q as shown in FIG. of arranging the real vector q _n in the order parameter theta, as shown in FIG. 43, the vector of the real vector q _n is ordered in the same direction as the direction of the coefficient matrix C, i.e. the coefficient matrix C of each row vector elements The line direction changes smoothly.

Therefore, the peak detection unit 177 fits the row vector of the coefficient matrix C with a polynomial and expresses it with a continuous function as shown in the following equation (51).
Here, α _i is a fitting coefficient.

If the formula of the discriminant function is arranged using this, the discriminant function at the rotation angle θ can be expressed in the form of a continuous function related to the parameter θ as shown in the following formula (52).
The peak detection unit 177 performs peak detection using this discrimination function. Since c _i (θ) is a polynomial as shown in equation (51), fθ (p) is also a continuous function (continuous polynomial). FIG. 44 is a graph showing an example of fθ (p). In FIG. 44, the horizontal axis is the rotation angle θ, and the vertical axis is fθ (p). The peak detection unit 177 detects θ when fθ (p) has a positive maximum as the target rotation angle, that is, the parameter value of the feature vector p.

As described above, when generating a matrix Q together multiple dictionaries w _n, as a plurality of dictionaries w _n it changes smoothly, the parameter matrix arranged in the order of (the θ in the example of FIG. 42) Q Since the discriminant function can be expressed in the form of a polynomial related to the parameter, the parameter can be detected with high resolution.

In the above description, the parameter is described as the rotation angle, but the parameter may be a scale, for example. That is, as shown in FIG. 36, the size of the window is fixed, the classifier is separately learned for each person size (scale) in the window, the polynomial is fitted to the scale σ, and the scale σ is identified. By calculating the peak of the instrument, the scale can be estimated with high accuracy. Further, by devising in this way, generation of the pyramid image itself can be made unnecessary. Furthermore, there may be a plurality of parameters. For example, fitting to the above polynomial may be performed for both the rotation angle θ and the scale σ. In this case, the coefficient is a two-dimensional polynomial such as c _i (θ, σ).

The coefficient α _i can be obtained by first obtaining the coefficient matrix C and then fitting each row, but the coefficient α _i may be obtained directly without obtaining the individual elements c _{n, i} of the coefficient matrix C. . Furthermore, the function to be fitted may not be a polynomial, and may be fitted to a trigonometric function (sine, cosine), for example.

4-9-2. Second Application Example of Fourth Embodiment In this application example, this embodiment is applied to k-means clustering. FIG. 45 is a block diagram illustrating a configuration of the k-means clustering apparatus. The k-means clustering apparatus 108 includes a content acquisition unit 181, a feature vector generation unit 182, a feature vector binarization unit 183, a representative matrix update unit 184, a convergence determination unit 185, a representative matrix decomposition unit 186, A closest representative vector search unit 187.

  The content acquisition unit 181 acquires N contents to be clustered. The feature vector generation unit 182 extracts the feature amounts from the contents acquired by the content acquisition unit 181 as feature vectors p. The feature vector binarization unit 183 binarizes each feature vector extracted by the feature vector extraction unit 182.

First, the representative matrix update unit 184 randomly selects k (= L) from the N feature vectors binarized by the feature vector binarization unit 183, and selects the representative vector q _n (n = 1). , 2,..., L), and a matrix in which these representative vectors q _n are collected is a representative matrix Q. The convergence determination unit 185 performs convergence determination each time the representative matrix update unit 24 updates the representative matrix. If the convergence determination unit 185 determines that the convergence has occurred, the k-means clustering apparatus 108 ends the clustering process. The representative matrix decomposition unit 186 decomposes the representative matrix updated by the representative matrix update unit 184 into a discrete value (binary or ternary) matrix.

The closest representative vector search unit 187 causes the N binary vectors input from the feature vector binarization unit 183 to belong to the nearest representative vector q _n . The nearest representative vector search unit 187 outputs this result to the representative matrix update unit 184. For each representative vector q _n , the representative matrix update unit 184 calculates an average vector of feature vectors (binarized) belonging to the representative vector q _{n and} sets this as a new representative vector q _n . The representative vector q _n updated by the representative matrix update unit 184 in this way is calculated as the average of the binary vectors, and thus becomes a real vector.

Therefore, if there is no representative matrix decomposition unit 186, the nearest representative vector search unit 187 calculates the inner product of them to obtain the distance between the updated representative vector (real vector) and feature vector (binary vector). Must. Therefore, in this application example, as described above, the representative matrix Q, which is a set of the representative vectors q _n (real vector), is separated by the representative matrix decomposition unit 186 as described in the first or second example. Decomposes a product of a value (binary or ternary) matrix and a real coefficient matrix. As a result, the nearest representative vector search unit 187 can calculate the distance between each feature vector and each representative vector at high speed. Therefore, the representative vector to which each feature vector is closest (that is, the representative vector to which the feature vector belongs) can be calculated at high speed. To explore.

4-9-3. Third Application Example of Fourth Embodiment In this application example, the present embodiment is applied to an approximate nearest neighbor search by k-means tree. The approximate nearest neighbor search device of this application example is Marius Muja and David G. Lowe, "Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration", in International, as an approximate nearest neighbor search method using k-trees using k-means. Conference on Computer Vision Theory and Applications (VISAPP '09), 2009 (http://www.cs.ubc.ca/~mariusm/index.php/FLANN/FLANN, http: // people .cs.ubc.ca / ~ mariusm / uploads / FLANN / flann_visapp09.pdf)

Specifically, the approximate nearest neighbor search apparatus of this application example constructs a k-ary tree by recursively applying k-means to N pieces of data, and follows the above-described tree search principle. Approximately search for the nearest point. This method is designed on the assumption that the data is a real vector and the representative vector registered in the node is a binary vector. However, even when the data is a binary vector and the representative vector registered in the node is a real vector, the tree search can be speeded up by adopting the first or second example .

4-10. In the modified feature quantity computing device 106 of the fourth exemplary embodiment , the content acquisition unit 161, the feature vector generation unit 162, the feature vector binarization unit 163, the real matrix acquisition unit 164, the real number matrix decomposition unit 165, and the vector calculation A part of the unit 166 and other parts may be configured as separate devices. In particular, the content acquisition unit 161, the feature vector generation unit 162, the feature vector binarization unit 163, and the vector calculation unit 166 are mounted on the feature calculation device 106, and the real matrix acquisition unit 164 and the real number matrix decomposition unit 165 are different from each other. It may be mounted on the device. In this case, a plurality of real number matrices decomposed by the real number matrix decomposition unit 165 are stored in the database 167 of the feature calculation device 106, and the vector calculation unit 166 acquires the plurality of real number matrices decomposed from the database 167. .

  In the example of the above embodiment, the base matrix M is binary or ternary. However, the base matrix M may not be binary or ternary. If the number of types of elements that the base matrix M can take is a finite number, the real matrix can be decomposed by applying the above decomposition method. Also, the coefficient matrix C may be a discrete value determined in advance as in the base matrix M. For example, the elements of the coefficient matrix C may be constrained to a power of 2, and the processing can be speeded up by doing so. When the average value of the elements of the real number matrix Q to be decomposed is remarkably large (or small), that is, when the average value is significantly different from 0, the average value is subtracted from each element of the real number matrix Q in advance. When an offset real number matrix is generated and this offset real number matrix Q ′ is decomposed into a base matrix M and a coefficient matrix C, approximate decomposition of Expression (29) and Expression (41) can be performed with fewer bases.

  In the first and second examples, the content data acquired by the content acquisition unit 161 may be measurement data obtained from the vehicle. Furthermore, the measurement data obtained from the vehicle may be, for example, image data captured by a camera installed in the vehicle, or sensing data measured by a sensor installed in the vehicle. In this case, the vector calculation unit 166 of the feature calculation device 106 as the relevance determination device determines the relevance between the measurement data and the dictionary data. For example, when image data taken by a camera installed in a vehicle is acquired as measurement data, data of a plurality of person images are stored in a database as dictionary data, and the feature as a relevance determination device The vector calculation unit 166 of the calculation device 106 may determine whether or not a person is included in the image of the image data according to any of the application examples described above.

5. Fifth Embodiment The first to fourth embodiments described above can be implemented in combination. In particular, the first embodiment can be combined with the second to fourth embodiments. For example, the pyramid image generation unit 141, the HOG feature amount extraction unit 142, the binary conversion unit 143, and the linear SVM identification unit 146 in the object recognition apparatus 104 (FIG. 33) described as the first application example of the third embodiment. However, each processing may be performed according to the hybrid pyramid method (FIG. 9) described as the first embodiment. Further, the parameter decomposing unit 145 of the object recognizing device 104, like the real matrix decomposing unit 155 of the feature value computing device 105 (FIG. 37) of the fourth embodiment, converts a real matrix composed of a plurality of real vectors, By decomposing the coefficient matrix into a product of a base matrix made up of a plurality of base vectors having only binary or ternary discrete values as elements, a linear SVM identification unit 146 has a feature vector and each of a plurality of real vectors. For calculating the inner product of, calculate the product of the feature vector and the base matrix, calculate the product of the product and coefficient matrix, and use the result to associate each of the real vectors with the feature vector. Sex may be determined.

5-1. First Example of Fifth Embodiment FIG. 46 is a block diagram showing processing in the identification device of the first example of the fifth embodiment. The processing in the identification device 109 corresponds to the binary high-speed identification processing shown in FIG. In this example, the HOG feature amount is extracted from the input content according to the first embodiment and binarized. When the binary feature amount of the input content is obtained, the identification device 109 slides a plurality of types of windows having different sizes to cut out the feature amount from the window (step S21). This cutout process may be cut out while changing the size of the window (number of vertical and horizontal blocks) such as 4 horizontal blocks × 8 vertical blocks, 5 horizontal blocks × 10 vertical blocks,. Thereby, it is possible to cut out a plurality of types of windows having different sizes. In this case, the number of dimensions of each cut-out feature amount differs depending on the number of vertical and horizontal blocks when cut out.

  The ternary decomposition completed dictionary 119 stores a dictionary (identification model) corresponding to a plurality of types of windows having different sizes. This ternary decomposition completed dictionary 119 is obtained by decomposing a real vector into a product of a ternary base matrix and a coefficient according to the third embodiment.

  When the feature amount is cut out, recognition is performed by a cascade of ternary bases using the cut out binary feature amount and the ternary decomposed dictionary 119 according to the third embodiment (step S22). By performing cascade identification in this way, the identification process can be speeded up.

5-2. Second Example of Fifth Embodiment FIG. 47 is a block diagram illustrating processing in the identification device of the second example of the fifth embodiment. The processing in the identification device 110 corresponds to the binary high-speed identification processing shown in FIG. Also in this example, the HOG feature amount is extracted from the input content according to the first embodiment and binarized. When the binary feature value of the input content is obtained, a process of cutting out the feature value from the window is performed (step S31). In this example, the process of extracting the feature value (step S31) is performed only once for a plurality of dictionaries. Since the cut-out process is performed only once, the process is simplified.

  The ternary decomposed dictionary 1101 and the ternary decomposed dictionary 1102 may store a dictionary (identification model) learned separately for each size to be recognized regarding a recognition target to be recognized. In step S31, since the extraction process is performed only once, the size of the detection window is constant in any dictionary, but the size (magnification) of the recognition sample is changed as shown in FIG. By learning the dictionary, it becomes possible to recognize objects of different sizes. The ternary decomposed dictionaries 1101 and 1102 are obtained by decomposing a real vector into a product of a ternary basis matrix and coefficients according to the third embodiment.

  When the feature amount is cut out (step S31), the identification device 110 performs the identification process S30 for each of a plurality of (K types) windows having different sizes. In each identification process S30, identification is performed by a cascade of ternary bases according to the third embodiment using the extracted binary feature quantity and the ternary decomposition completed dictionary 1101 (step S32). If no detection is made in this cascade identification (step S32), the identification device 110 immediately outputs a non-detection result for the window of that size. The identification device 110 performs two-stage cascade identification. If detected in the first cascade identification (step S32), a FIND feature quantity is generated by taking the co-occurrence by XOR and bit shift according to the second embodiment for this binary feature quantity. (Step S33).

  When the FIND feature value is generated (step S33), the FIND feature value and the ternary decomposition completed dictionary 1102 are used to identify the ternary basis cascade according to the third embodiment (step S34). . In this way, it is possible to further increase the speed by performing two-stage cascade identification in which coarse identification is performed by cascade identification that does not use co-occurrence, and cascade detection using co-occurrence is performed on the detected one.

5-2. Third Example of Fifth Embodiment FIG. 48 is a block diagram illustrating processing in the identification device of the third example of the fifth embodiment. The processing in the identification device 120 corresponds to the binary high-speed identification processing shown in FIG. Also in this example, the HOG feature amount is extracted from the input content according to the first embodiment and binarized. When the binary feature amount of the input content is obtained, the identification device 120 cuts out the feature amount from the window while sliding the window (step S41). In this example, the process of extracting the feature amount (step S41) is performed only once for a plurality of dictionaries. Since the cut-out process is performed only once, the process is simplified.

  The ternary decomposition completed dictionary 1201 stores a dictionary (identification model) learned separately for each size to be recognized with respect to a recognition target to be recognized. In step S41, since the extraction process is performed only once, the size of the detection window is constant in any dictionary. However, the size (magnification) of the recognition sample is changed as shown in FIG. By learning the dictionary, it becomes possible to recognize objects of different sizes. Since the feature amount is cut out only once (step S41) and the number of dimensions is the same in any dictionary, the fourth embodiment can be applied. Therefore, it is assumed that the ternary decomposition completed dictionary 1201 is obtained by decomposing a real matrix obtained by collecting a plurality of real vectors into a product of a ternary base matrix and a coefficient matrix according to the fourth embodiment. .

  When the feature amount is cut out (step S41), the identification device 120 uses the cut out binary feature amount and the ternary decomposed dictionary 1201, for each of a plurality (K types) of windows having different sizes. Identification processing is performed (step S40). In this identification process, classification is performed by ternary basis cascade according to the fourth embodiment (step S42). At this time, linear identification functions corresponding to all dictionaries are calculated in a lump. Of the linear discriminant functions calculated in a lump, a window corresponding to a dictionary having a positive sign is output as a detection result (step S43). In this way, linear discrimination functions corresponding to all the dictionaries can be calculated in a lump, so that further speed-up is possible.

  The present invention can reduce the number of feature extractions and the number of binarization processes, and thus has the effect of speeding up the relevance determination process, and is a feature that calculates the feature extracted from an image It is useful as an arithmetic unit.

DESCRIPTION OF SYMBOLS 10 Input image 11 Resized image 101 Feature-value conversion apparatus 111-11N Bit rearrangement device 121-12N Logical operation unit 130 Feature integrator 102 Feature-value conversion apparatus 211-21N Binarizer 221-22N Co-occurrence element generator 230 Feature Integrator 103 Feature amount computing device 131 Content acquisition unit 132 Feature vector generation unit 133 Feature vector binarization unit 134 Real vector acquisition unit 135 Real vector decomposition unit 136 Vector calculation unit (inner product calculation unit)
105 k-means clustering device 151 content acquisition unit 152 feature vector generation unit 153 feature vector binarization unit 154 representative vector update unit 155 convergence determination unit 156 representative vector decomposition unit 157 nearest neighbor representative vector calculation unit 106 feature quantity calculation device 161 content Acquisition unit 162 Feature vector generation unit 163 Feature vector binarization unit 164 Real matrix acquisition unit 165 Real matrix decomposition unit 166 Vector operation unit (product operation unit)
DESCRIPTION OF SYMBOLS 107 Object recognition apparatus 171 Pyramid image generation part 172 HOG feature-value extraction part 173 Binary code conversion part 174 Parameter determination part 175 Parameter matrix decomposition part 176 Linear SVM identification part 177 Peak detection part 108 k-means clustering apparatus 181 Content acquisition part 182 Feature Vector generating unit 183 Feature vector binarizing unit 184 Representative matrix updating unit 185 Convergence determining unit 186 Representative matrix decomposing unit 187 Nearest representative vector calculating unit 109 Discriminating device 119 Tri-level decomposed dictionary 109, 110, 120 Discriminating device 1101, 1102 , 1201 Ternary decomposition dictionary

Claims (11)

A feature quantity binarization unit that binarizes a feature quantity extracted from each of a pyramid image made up of an input image and a plurality of resized images respectively enlarged or reduced at a plurality of magnifications;
A feature amount calculation unit that determines a relevance between the input image and the plurality of dictionaries by applying a dictionary set including a plurality of dictionaries having different sizes to the binarized feature amount;
With
The feature amount calculation unit determines, for each of the pyramid images, the relevance to the plurality of dictionaries by using the binarized feature amount in common for the plurality of dictionaries. A feature amount calculation device. A feature amount extraction unit that extracts the feature amount from each of the pyramid images;
The feature quantity calculation apparatus according to claim 1, wherein the feature quantity binarization unit binarizes the feature quantity extracted by the feature quantity extraction unit.   The feature quantity computing device according to claim 1, wherein the feature quantity computing unit performs identification using the dictionary for the input image.   4. The feature amount calculation unit applies the dictionary set in which all or some of the plurality of dictionaries are the same for each of the pyramid images. The feature amount calculation device according to one item.   The feature quantity calculation device according to claim 3, further comprising a feature quantity conversion unit that transforms the feature quantity so as to enhance discrimination ability using the binarized co-occurrence element of the feature quantity. . Obtaining a basis vector for obtaining the plurality of basis vectors obtained by decomposing a real vector having real numbers as elements into a linear sum of a plurality of basis vectors having elements composed only of binary or ternary discrete values Further comprising
The dictionary is generated using the plurality of basis vectors;
The feature amount calculation unit determines the relevance between the real vector and the feature vector by sequentially calculating an inner product of the feature vector indicating the feature amount and each of the plurality of base vectors. The feature-value calculating apparatus as described in any one of Claim 1 thru | or 5. A feature amount conversion unit that converts the feature vector so as to enhance identification ability using a co-occurrence element of the feature vector determined to be relevant by the feature amount calculation unit;
A feature vector converted by the feature quantity conversion unit is further subjected to inner product calculation with each of a plurality of base vectors, thereby determining a relevance between the real vector and the feature vector. A feature amount calculation unit;
The feature amount calculation apparatus according to claim 6, further comprising:   The feature amount calculation unit cuts out a feature amount while sliding a window for each of the pyramid images, and determines the relevance of the feature amount cut out from the window by applying the dictionary set. The feature-value calculating apparatus as described in any one of Claim 1 thru | or 7. Real matrix decomposition unit that decomposes a real matrix consisting of multiple real vectors with real elements into a product of a coefficient matrix and multiple base vectors with only binary or ternary discrete values as elements Further comprising
The dictionary is generated using the plurality of basis matrices;
The feature amount calculation unit calculates a product of the feature vector and the base matrix as a calculation of an inner product of the feature vector indicating the feature amount and each of the plurality of real vectors, and further calculates the product and the coefficient matrix. The feature quantity calculation apparatus according to claim 8, wherein the product is calculated, and the relevance between each of the plurality of real vectors and the feature vector is determined using the result. A feature quantity binarization step for binarizing a feature quantity extracted from each of a pyramid image made up of an input image and a plurality of resized images each enlarged or reduced at a plurality of magnifications;
A feature amount calculation step of determining a relevance between the input image and the plurality of dictionaries by applying a dictionary set including a plurality of dictionaries having different sizes with respect to the binarized feature amount;
Including
In the feature amount calculating step, for each of the pyramid images, the binarized feature amount is commonly used for the plurality of dictionaries to determine relevance to the plurality of dictionaries. Characteristic feature amount calculation method. On the computer,
A feature quantity binarization step for binarizing a feature quantity extracted from each of a pyramid image made up of an input image and a plurality of resized images each enlarged or reduced at a plurality of magnifications;
A feature amount calculation step of determining a relevance between the input image and the plurality of dictionaries by applying a dictionary set including a plurality of dictionaries having different sizes with respect to the binarized feature amount;
A feature amount calculation program for executing
In the feature amount calculating step, for each of the pyramid images, the binarized feature amount is commonly used for the plurality of dictionaries to determine relevance to the plurality of dictionaries. Feature feature calculation program.