JP6375420B2 - Relevance determination device and relevance determination program - Google Patents

Description

  The present invention relates to a relevance determination device, a relevance determination program, and a relevance determination method that perform arithmetic processing using feature vectors of content such as images, sounds, and characters, and more particularly to a plurality of real vectors and binary vectors. The present invention relates to a relevance determination device and a relevance determination program for determining relevance between a feature vector and each of a plurality of real vectors by a vector operation including calculation of an inner product with a converted feature vector.

  Conventionally, feature quantities are used in many fields such as image search, voice recognition, sentence search, and pattern recognition. The feature amount is information obtained by converting information such as images, sounds, and sentences so as to be easily handled by a computer. The feature amount is represented by a D-dimensional vector (feature vector).

  By performing the calculation using the feature vector, for example, the similarity of content can be determined. That is, if the distance between the feature vector of the image α and the feature vector of the image β is small, it can be considered that α and β are similar. Similarly, if the distance between the feature vector of the speech waveform α and the feature vector of the speech waveform β is small, it can be considered that α and β are similar. As described above, in information processing such as speech recognition, sentence search, and pattern recognition, information is converted into feature vectors, the feature vectors are compared with each other, and the distance between them is determined to determine information similarity.

Ｌ２ノルム

ベクトル間角度

As a measure of the distance between feature vectors, an L1 norm, an L2 norm, an angle between vectors, or the like is used. These feature vectors x, for Y∈R ^D, can be calculated as follows.
L1 norm

L2 norm

Angle between vectors

であるから、Ｄ回の引き算、Ｄ回の乗算、Ｄ−１回の加算が必要である。特に、特徴ベクトルが浮動小数で表現される場合には、この計算負荷は非常に高くなる。特徴ベクトルが高次元になれば、この計算負荷はさらに高くなる。 When the feature vector is a real vector, there are the following problems. First, there is a problem that two feature vectors x, the calculation of the distance between the Y∈R ^D slows. For example, when the L2 norm square is used as a distance measure,

Therefore, D subtractions, D multiplications, and D-1 additions are required. In particular, when the feature vector is expressed by a floating point number, the calculation load becomes very high. If the feature vector has a higher dimension, the calculation load becomes higher.

  Another problem is that a large amount of memory is consumed. When a feature vector is represented by a 4-byte single-precision real number, the D-dimensional feature vector consumes 4D bytes of memory. As the feature vector becomes higher in dimension, the memory consumption increases. When dealing with a large amount of feature vectors, the memory is consumed by the number of feature vectors to be handled.

  Therefore, in recent years, a method for solving these two problems by converting a feature vector into a binary code composed of a sequence of 0 and 1 has been proposed. Typical techniques include random projection (see random projection, Non-Patent Document 1), belly sparse random projection (see Non-Patent Document 2), and spectral hashing (Spectral Hashing, see Non-Patent Document 3). is there.

  In these methods, a D-dimensional feature vector is converted into a d-bit binary code. This conversion is performed so that the distance in the original space strongly correlates with the Hamming distance in the converted space (for the reason that the distance in the original space and the Hamming distance in the converted space are strongly correlated) (See Lemma 3.2 on page 1121 of Patent Document 1). As a result, the calculation of the distance between feature vectors can be replaced by the calculation of the Hamming distance between binary codes.

  The Hamming distance is obtained by counting the number of different bits in two binary codes. This calculation can be performed very quickly because it only counts the number of bits that are 1 after XORing the two codes. In many cases, binary code conversion can increase the speed by several tens to several hundred times. Further, by substituting the calculation of the distance between feature vectors with the calculation of the Hamming distance between binary codes, the required memory capacity, which was originally 4D bytes, can be reduced to d / 8 bytes. Thereby, memory capacity can be saved to tens to hundreds of times.

  By converting the extracted feature quantity into binary code and applying various algorithms, it becomes possible to search and recognize content. For example, when searching for similar content, all the feature quantities registered in the database in advance are converted into binary codes. Also, the feature amount of the content given as an input query is converted into a binary code. Then, by calculating the Hamming distance between the binary code of the input query and all the binary codes registered in the database, it is possible to search and output content similar to the input query.

Michel X. Goemans, avid P. Williamson, "Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming", Journal of the ACM Volume 42, Issue 6 (November 1995) Pages: 1115-1145 Ping Li, Trevor J. Hastie, Kenneth W. Church, "very sparse random projections", KDD '06 Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining (2006) Y. Weiss, A. Torralba, R. Fergus., "Spectral Hashing", Advances in Neural Information Processing Systems, 2008.

  The binary code consists of a sequence of 0's and 1's of d bits. This can be considered as a d-dimensional vector in which each element takes only binary values of −1 and 1. In order to avoid confusion in the following description, the terms “binary code” and “binary vector” are distinguished as follows. A “binary code” is a data representation consisting of a sequence of 0s and 1s. For example, when a 128-bit binary code is stored in a memory in the C language, an array for 16 elements of an unsigned integer type may be prepared (8 bits × 16 = 128 bits).

On the other hand, a “binary vector” is a vector in which each element takes only binary values. For example, when a binary vector is a vector in which each element takes only −1 and 1, the binary vector corresponding to the binary code “01101110” is (−1, 1, 1, −1, 1, 1). , 1, -1) ^T. Of course, a vector in which each element takes only binary values of 0 and 1 is also a binary vector, and furthermore, a vector in which each element takes only binary values of arbitrary α and β (where α ≠ β) Is also a binary vector. However, the difference between the “binary code” and the “binary vector” relates to the expression of information, and there is no essential difference between the two.

  If a feature vector is converted into a d-dimensional binary vector in which each element takes only binary values of -1 and 1, various processes such as identification processing by SVM (support vector machine) and k-means clustering are binary. It can also be applied to code. However, in these cases, it may not be possible to benefit from high-speed distance calculation by Hamming distance. That is, depending on the algorithm, there are cases where the benefits of high-speed distance calculation by binary code conversion cannot be obtained.

ｆ（ｘ）が正ならばｘはクラスＡに属し、ｆ（ｘ）が負ならばｘはクラスＢに属するものとして識別する。 ｗは、重みパラメータであって、ｗ∈Ｒ^ｄである。ｂは、バイアスパラメータであって、ｂ∈Ｒ^１である。パラメータｗ及びｂは、学習用に用意した特徴量を用いて、学習処理により自動的に決定される。 As an example in which the benefits of high-speed distance calculation by binary code conversion cannot be obtained, recognition processing by a classifier (Classifier) and k-means clustering will be described below. First, for recognition processing by a classifier, for example, a linear classifier such as a linear SVM (Linear Support Vector Machine) is applied to the problem of classifying a binary vector xε {−1,1} ^d into two classes. Think about what to do. In the linear SVM, the following expression is evaluated.

If f (x) is positive, x belongs to class A, and if f (x) is negative, x is identified as belonging to class B. w is a weight parameter, and wεR ^d . b is a bias parameter, and bεR ¹ . The parameters w and b are automatically determined by a learning process using feature amounts prepared for learning.

Here, even if the feature quantity prepared for learning is a binary vector, wεR ^d is not a binary value but a real value. The calculation of f (x) includes w ^T x, but since x is binary and w is a real-valued vector, the calculation of w ^T x requires a floating point operation. turn into. As described above, the recognition processing by the discriminator to which the SVM is applied cannot benefit from the speeding up of calculation by making the feature vector a binary vector.

  Next, when k-means clustering is applied to a binary vector, i.e., when n d-dimensional binary vectors are given, k clusters obtained by collecting binary vectors that are close to each other are obtained. Think about the problem you want. k-means is an algorithm for calculating k clusters and representative vectors according to the following procedure.

Step 1: k features are randomly selected from N feature amounts and set as cluster representative vectors.
Step 2: For each of the N feature values given as input, a representative vector having the closest distance is obtained.
Step 3: The average of the feature quantities belonging to each representative vector is calculated and set as a new representative vector.
Step 4: Repeat Step 2 and Step 3 until convergence.

  The problem in k-means clustering is that in step 3 a new representative vector is defined by the average of the binary vectors. Even if the data given as input is a binary vector, the representative vector becomes a real vector by the average calculation. Therefore, in the distance calculation in step 2, it is necessary to obtain the distance between the binary vector and the real vector. In other words, floating point arithmetic is required. As described above, even in k-means clustering, it is not possible to receive the benefit of speeding up the calculation by making the feature vector a binary vector.

As described above, recognition processing by a classifier (Classifier) and k-means clustering cannot benefit from speeding up computation by using a feature vector as a binary vector. The reason is that an inner product operation between the d-dimensional binary vector pε {−1,1} ^d and the d-dimensional real vector qεR ^d is necessary. Note that k-means clustering requires a “distance” between a d-bit binary vector pε {−1,1} ^d and a d-dimensional real vector qεR ^d , but this is also the result. However, this results in an inner product operation called p ^T q. This is because the square of the Euclidean distance between p and q is expressed by the following equation.

  Therefore, speeding up the calculation of the inner product of a binary vector and a d-dimensional real vector in both recognition processing by the classifier and k-means clustering leads to the solution of the problem.

Therefore, the present applicant has determined that the inner product between such a feature vector and a d-dimensional real vector qεR ^d when the feature vector is a d-dimensional binary vector pε {−1,1} ^d. (Japanese Patent Application No. 2013-214182, hereinafter referred to as “Prior Application”) has been proposed (Japanese Patent Application No. 2013-214182), which proposes a high-speed calculation of p ^T q or q ^T p.

  The relevance determination device of the prior application includes a feature vector acquisition unit that acquires a binarized feature vector, and a linear vector of a plurality of base vectors each having an element composed of only a binary or ternary discrete value as a real vector A basis vector acquisition unit that acquires the plurality of basis vectors obtained by decomposing into a sum, and sequentially performing inner product calculation of the feature vector and each of the plurality of basis vectors, thereby the real vector and the feature A vector operation unit for determining relevance with the vector. With this configuration, real vectors are decomposed into linear sums of multiple binary basis vectors, and the inner product calculation of the binarized feature vector is performed, so the inner product calculation of the feature vector and the real vector is accelerated. it can.

  Incidentally, it may be necessary to determine the relevance between the feature vector and each of the plurality of real vectors by calculating the inner product of the binarized feature vector and the plurality of real vectors. For example, as described above, in linear SVM, only whether a feature vector belongs to class A or class B, that is, whether or not the feature vector falls under a certain identification criterion, is determined. There are cases where such identification is desired for a plurality of criteria. As a specific example, I would like to determine whether the captured image is an adult, a child, a car, or a road sign. There is a case.

  In the k-means clustering described above, for each of the N feature vectors given as an input, a distance with an inner product calculation is calculated with k representative vectors. Here, each of the k representative vectors is a real vector because it is defined by the average of the binary vectors as described above. Therefore, k-means clustering also requires inner product calculation of binarized feature vectors and a plurality of real vectors.

  Therefore, the present invention speeds up the inner product calculation of a binarized feature vector and a plurality of real vectors, thereby quickly determining the relevance between such a feature vector and a plurality of real vectors. With the goal.

  The relevance determination device of the present invention includes a feature vector acquisition unit that acquires a binarized feature vector, a real matrix composed of a plurality of real vectors, a coefficient matrix, and only binary or ternary discrete values as elements. A database storing dictionary data including the vector and the coefficient matrix obtained by decomposing the product into a base matrix composed of a plurality of basis vectors having a plurality of basis vectors, and the feature vector and each of the plurality of real vectors As an inner product calculation, a product of the feature vector and the base matrix is calculated, a product of the product and the coefficient matrix is further calculated, and the result is used to calculate each of the plurality of real vectors and the feature. It has the structure provided with the vector calculating part which determines the relationship with a vector. With this configuration, in order to calculate the inner product of a feature vector and each of a plurality of real vectors, a real matrix made up of a plurality of real vectors is decomposed into a discrete value base matrix and a coefficient matrix. And the product with the coefficient matrix, the result of the inner product operation between the feature vector and each of multiple real vectors can be obtained at high speed, and the relationship between the feature vector and multiple real vectors can be obtained. Sex determination can be performed at high speed.

  The relevance determination device may further include a real matrix generation unit that generates the real matrix by arranging the plurality of real vectors. With this configuration, a real matrix can be easily generated from a plurality of real vectors.

  In the above relevance determination device, when the plurality of real vectors have predetermined parameters, the real matrix generation unit generates the real matrix by arranging the plurality of real vectors according to the order of the parameters. Good. With this configuration, since real vectors similar to each other in the real number matrix are adjacent to each other, adjacent coefficient matrices are also similar.

  In the above relevance determination device, the feature vector may be a HOG feature, the plurality of real vectors may be a plurality of weight vectors corresponding to parameters of a plurality of linear classifiers, and the vector calculation The unit may identify the feature vector for each of the plurality of criteria by using an identification function of the plurality of linear classifiers as the determination of the relevance. With this configuration, it is possible to speed up feature vector identification by a plurality of linear classifiers.

  In the above-described relevance determination device, when the feature vector and the plurality of real vectors have one or a plurality of parameters, the real matrix generation unit arranges the plurality of real vectors according to the order of the parameters. A real number matrix is generated, and the vector calculation unit is a continuous function related to the parameter for each of a plurality of vectors constituting the coefficient matrix and having the same direction as the direction in which the plurality of real number vectors are arranged. The parameter that expresses and maximizes the discriminant function may be obtained as a parameter value of the feature vector. With this configuration, when a real matrix is generated by combining a plurality of real vectors, a real matrix is generated by arranging a plurality of real vectors in the order of parameters in which they change smoothly. Therefore, the parameter value of the feature vector can be obtained with high resolution.

  In the above-described relevance determination device, the feature vector may be a vector to be clustered by k-means clustering, the real vector may be a representative vector in k-means clustering, and the vector calculation unit May perform a clustering process including a calculation of a distance between the feature vector and the representative vector as the determination of the relevance. With this configuration, the calculation of the distance between the feature vector and the representative vector in k-means clustering can be speeded up.

  In the above-described relevance determination apparatus, the feature vector may be a vector to be subjected to an approximate nearest neighbor search by k-means tree, and the real vector is a representative vector registered in a node of a k-ary tree. The vector calculation unit may perform a clustering process including calculation of a distance between the feature vector and the representative vector as the determination of the relevance. With this configuration, it is possible to speed up the calculation of the distance between the feature vector in the approximate nearest neighbor search by k-means tree and the representative vector registered in the node of the k-ary tree.

  In the above-described relevance determination device, the feature vector may be a vector that represents a feature amount of an image. With this configuration, it is possible to speed up the inner product calculation of the feature vector and the plurality of real vectors in the calculation of the feature amount of the image.

  The relevance determination program of the present invention is a relevance determination program for causing a computer to function as the above-described relevance determination device. Even with this configuration, in order to calculate the inner product of a feature vector and each of a plurality of real vectors, a real matrix composed of a plurality of real vectors is decomposed into a discrete value base matrix and a coefficient matrix, and then the feature vector and the base Since the product with the matrix is calculated and the product with the coefficient matrix is further calculated, the result of the inner product operation between the feature vector and each of the plurality of real vectors can be obtained at high speed, and thus the feature vector and the plurality of real vectors can be obtained. Relevance can be determined at high speed.

  According to the present invention, it is possible to speed up the inner product calculation of a binarized feature vector and each of a plurality of real vectors, and to quickly determine the relevance between such a feature vector and each of a plurality of real vectors. It can be carried out.

The figure which shows the example of linear SVM in the case of identifying the person in an image with a some identification reference | standard The figure which shows the example of linear SVM in the case of identifying the person in an image with a some identification reference | standard The block diagram which shows the structure of the feature-value calculating apparatus of the 1st Embodiment of this invention. The figure which shows decomposition | disassembly of the real number matrix in the 1st Embodiment of this invention The figure for demonstrating the relationship between the real number matrix and basis matrix in the 1st Embodiment of this invention The figure which shows the example of a calculation in the 2nd Embodiment of this invention The block diagram which shows the structure of the object recognition apparatus of the 1st application example of this invention. The figure which shows the dictionary and bias for every rotating road sign, rotation angle of the 1st application example of this invention The figure which shows the property of the coefficient matrix of the 1st application example of this invention The graph which shows the example of the discriminant function of the 1st application example of this invention The block diagram which shows the structure of the k-means clustering apparatus of the 2nd application example of this invention.

  Hereinafter, a feature value computing device according to an embodiment of the present invention will be described with reference to the drawings.

First, a case where there are a plurality of real vectors whose inner products with feature vectors are to be calculated will be described. FIG. 1 is a diagram illustrating an example of a linear SVM when a person in an image is identified based on a plurality of identification criteria. In this example, for an input feature vector, as shown in FIG. 1, it is not simply identification of whether or not there is a person in the image of the feature vector, but whether it is “adult (front)”. No, “adult (horizontal)”, and “child (front)” are identified. That is, there are a plurality of criteria for identifying feature vectors. In this case, as shown in FIG. 1, a plurality of (q ₁ , q ₂ , q ₃ ) weight parameters (hereinafter also referred to as “dictionaries”) q of the evaluation formula f (x) of the linac SVM are provided for each identification criterion. ,..., Q _L ) and a plurality of bias b (b ₁ , b ₂ , b ₃ ,..., B _L ) must be prepared for each identification criterion.

FIG. 2 is a diagram illustrating an example of a linear SVM when a person in an image is identified based on a plurality of identification criteria according to the distance to the subject. In this example, as shown in FIG. 2, for a certain feature vector input, the identification of a person is robust to the distance to the subject, that is, the change in the scale of the subject in the image. In addition to identifying whether there is an adult in the image of the feature vector, whether it is “adult (far)”, whether it is “adult (medium distance)”, “adult (near) ) ”Or not. That is, in this case as well, there are a plurality of criteria for identifying feature vectors. Therefore, as shown in FIG. 2, the linear SVM dictionary q is divided into a plurality (q ₁ , q ₂ , q ₃ ,. _L ) must be prepared, and a plurality of bias b (b ₁ , b ₂ , b ₃ ,..., B _L ) must be prepared for each identification criterion.

As described above, when a certain feature vector is identified by a plurality of criteria, the plurality of criteria are often similar to each other. FIG. 1 and FIG. 2 also show such an example, that is, in the example of FIG. 1, “adult (front)” and “adult (horizontal)” have the common point of adults. ) ”And“ Children (front) ”have the common feature of people, and“ Adults (front) ”,“ Adults (horizontal) ”and“ Children (front) ”have the common features of people. Have. Also in the example of FIG. 2, “adult (far)”, “adult (medium distance)”, and “adult (near)” have a common point of “adult”. Therefore, the dictionaries (q ₁ , q ₂ , q ₃ ,..., Q _L ) that are a plurality of real vectors in FIGS. 1 and 2 are similar to each other. In k-means clustering, representative vectors that are k real vectors are often similar to each other. The relevance determination apparatus according to the embodiment of the present invention speeds up processing by taking advantage of the property that a plurality of real vectors are similar to each other.

1. Embodiment 1-1. First Embodiment FIG. 3 is a block diagram showing a configuration of a feature quantity computing device 100 according to an embodiment of the present invention. The feature amount calculation apparatus 100 includes a content acquisition unit 101, a feature vector generation unit 102, a feature vector binarization unit 103, a real number matrix acquisition unit 104, a real number matrix decomposition unit 105, a vector calculation unit 106, a database 107.

  As will be described later, the feature quantity computing device 100 of the present embodiment performs a feature vector and a plurality of real vectors by a vector operation involving an inner product operation between the feature vector and a plurality of real vectors stored in a database as dictionary data. It functions as a relevance determination device that determines the relevance of each other. That is, the feature calculation device 100 corresponds to the relevance determination device of the present invention.

  The feature quantity computing device 100 as a relevance determination device is realized by a computer executing the relevance determination program according to the embodiment of the present invention. The relevance determination program may be recorded on a recording medium and read from the recording medium by a computer, or may be downloaded to a computer through a network.

  The content acquisition unit 101 acquires content data such as image data, audio data, and character data. These content data may be provided from an external device or may be generated by the content acquisition unit 101. For example, the content acquisition unit 101 may be a camera, and image data may be generated there as content data.

The feature vector generation unit 102 generates a D-dimensional feature vector from the content data acquired by the content acquisition unit 101. For example, when the content is an image, the feature vector generation unit 102 extracts the feature amount of the image. The feature vector binarization unit 103 binarizes the D-dimensional feature vector generated by the feature vector generation unit 102, and each element has only a binary value of -1 and 1, and a d-dimensional binary vector pε {-1, 1} ^d is generated. The feature vector binarization unit 103 corresponds to the “feature vector acquisition unit” of the present invention.

  The configuration including the content acquisition unit 101, the feature vector generation unit 102, and the feature vector binarization unit 103 may be any configuration that can finally acquire a binarized feature vector. For example, the content acquisition unit 101 and feature vector generation unit 102 are not provided, and feature vector binarization unit 103 may acquire a feature vector from an external device and binarize the acquired feature vector. The vector binarization unit 103 may be configured to directly obtain a binarized feature vector from an external device.

The real matrix acquisition unit 104 acquires a plurality of d-dimensional real vector q _n ∈R ^d (n = 1, 2,..., L). The plurality of real number vectors q _n may be given from an external device, may be read from a storage device (not shown) of the feature value computing device 100, and is generated by the real number matrix acquisition unit 104. It may be. Each real vector q _n has a real number including a floating point number in its element. Here, an array of a plurality of real number vectors q _n is expressed as a real number matrix Q = (q ₁ , q ₂ , q ₃ ,..., Q _L ) ∈R ^{d × L.}

When a real matrix Q in which a plurality of real vectors q _n are combined in this way, the plurality of linear SVMs in FIGS. 1 and 2 can be expressed collectively as in the following equation (1).

ここで、図４に示すように、Ｍ＝（ｍ_１，ｍ_２，…，ｍ_ｋ）∈｛−１，１｝^ｄ×ｋであり、Ｃ＝（ｃ_１，ｃ_２，…，ｃ_Ｌ）^Ｔ∈Ｒ^ｋ×Ｌである。 As shown in FIG. 4, the real matrix decomposition unit 105 decomposes the real matrix Q of d rows and L columns into a product of a binary base matrix M∈ {−1,1} ^{d × k} and a coefficient matrix. Specifically, the real number matrix decomposition unit 105 decomposes the real number matrix Q of d rows and L columns into a base matrix M having binary elements and a coefficient matrix C having real elements by the following equation (2). .

Here, as shown in FIG. 4, M = (m ₁ , m ₂ ,..., M _k ) ∈ {−1, 1} ^{d × k} , and C = (c ₁ , c ₂ ,..., C _L ) ^T ∈ R ^{k × L.}

That is, the basis matrix M is composed of k basis vectors m _i , where the basis vector _mi is a d-dimensional binary vector having elements of only −1 and 1, and thus the basis matrix M is , Is a binary matrix of d rows and k columns in which elements take only -1 and 1.

The coefficient matrix C, L number (L is the number of classes) a coefficient vector c _n of where the coefficient vector c _n, k pieces (k is the number of base) coefficients of real elements according to basis vectors As a k-dimensional real vector. Of course, it is preferable to decompose Q and MC so that they coincide as much as possible, but an error may be included. Hereinafter, a method in which the real matrix decomposition unit 105 decomposes the real matrix Q as shown in Expression (2) will be described.

ただし、基底行列Ｍは二値であり、Ｍ∈｛−１，１｝^ｄ×ｋである。 (First decomposition method)
As a first decomposition method, a data-independent decomposition method will be described. In the first decomposition method, a real matrix decomposition unit 105 performs degradation by solving the cost function g ₁ of the formula (3) representing the degradation error.

However, the base matrix M is binary and M∈ {−1,1} ^{d × k} .

The real matrix decomposition unit 105 solves the cost function g _{1 according} to the following procedure.
(1) The base matrix M and the coefficient matrix C are initialized at random.
(2) fixing the elements of a basis matrix M, the elements of the coefficient matrix C by optimizing the least squares method, the cost function g ₁ updates the elements of the coefficient matrix C to minimize.
(3) to fix the elements of the coefficient matrix C, the cost function g ₁ updates the elements of the basis matrix M at full search to minimize. The full search which is this minimization algorithm will be described in detail later.
(4) Repeat (2) and (3) until convergence. For example, when the cost function g ₁ satisfies a predetermined convergence condition (for example, the amount of decrease is a certain value or less), it is determined that the cost function g ₁ has converged.
(5) The solutions obtained in steps (1) to (4) are held as candidates.
(6) Step (1) to repeat steps (5), to adopt the most cost function g ₁ Decrease be candidate basis matrix M and the candidate coefficient matrix C as the final result. Note that the steps (1) to (5) need not be repeated, but the problem of initial value dependency can be avoided by repeating a plurality of times.

Next, the update process of the base matrix M in step (3) will be described. As surrounded by the broken line frame in FIG. 5, the element of the row vector of the jth row of the base matrix M depends only on the element of the jth row of the real number matrix. Therefore, the value of each row vector of the base matrix M can be optimized independently of other rows, so that the base matrix M can perform an exhaustive search (full search) for each row. Row vector of the j-th row of the base matrix M is absent only 2 ^k as in the case of binary decomposition as in the present embodiment (Also in the case of a three-value decomposition of the second embodiment described later There are only ^3k ways). Therefore, the real number matrix decomposing unit 105 comprehensively checks them and employs a row vector that minimizes the cost function g ₁ . This is applied to all the row vectors of the base matrix M to update the elements of the base matrix M.

ただし、基底行列Ｍは二値であり、Ｍ∈｛−１，１｝^ｄ×ｋである。また、｜Ｃ｜_１は、係数行列Ｃの要素のＬ１ノルムであり、λはその係数である。 (Second decomposition method)
As a second decomposition method, a data-independent decomposition method that makes the coefficient matrix C sparse will be described. In the second decomposition techniques, real matrix decomposition unit 105 performs degradation by solving the cost function g ₂ of the formula is an exploded error (4).

However, the base matrix M is binary and M∈ {−1,1} ^{d × k} . Further, | C | ₁ is the L1 norm of the element of the coefficient matrix C, and λ is its coefficient.

Real matrix decomposition unit 105 solves a cost function g ₂ of the following procedure.
(1) The base matrix M and the coefficient matrix C are initialized at random.
(2) The elements of the base matrix M are fixed, and the elements of the coefficient matrix C are optimized by the proximity gradient method.
(3) to fix the elements of the coefficient matrix C, the cost function g ₂ updates the elements of the basis matrix M at full search to minimize.
(4) Repeat (2) and (3) until convergence. For example, when the cost function g ₂ satisfies a predetermined convergence condition (e.g., decrease amount is less than a predetermined value), it is judged to have converged.
(5) The solutions obtained in steps (1) to (4) are held as candidates.
(6) Step (1) to repeat steps (5), adopts the highest cost function g ₂ Decrease be candidate basis matrix M and the candidate coefficient matrix C as the final result. Note that the steps (1) to (5) need not be repeated, but the problem of initial value dependency can be avoided by repeating a plurality of times.

  According to the second decomposition method, the coefficient matrix C can be made sparse. By making the coefficient matrix C sparse, in the calculation of the product MC, the portion related to the zero element of the coefficient matrix C can be omitted, and the inner product calculation can be performed at higher speed.

を定義し、この分解誤差を最小化することを考えた。しかしながら、実数行列を基底行列と係数行列との積に近似した後に実際に近似をしたいのは、特徴ベクトルと実数行列の積Ｑ^Ｔｐである。 (Third decomposition method)
Next, the third decomposition method will be described. In the first decomposition method, the decomposition error is expressed as the cost function g _1.

To minimize this decomposition error. However, what is actually desired to be approximated after the real matrix is approximated to the product of the base matrix and the coefficient matrix is the product Q ^T p of the feature vector and the real matrix.

と定義して、これを最小化する。即ち、第３の分解手法では、実数行列分解部１０５は、下式（５）のコスト関数ｇ_３を解くことで分解を行う。

このコスト関数ｇ_３によれば、実数行列Ｑは、実際のデータの分布に従って分解されることになるため、分解の際の近似精度が向上する。 Therefore, in the third decomposition method, S feature vectors p are collected in advance, and the sum of these is defined as P∈R ^{d × S.} And the decomposition error

And minimize this. That is, in the third decomposition techniques, real matrix decomposition unit 105 performs degradation by solving the cost function g ₃ of the following formula (5).

According to the cost function g _3, real matrix Q, since that would be resolved according to the distribution of the actual data, is improved approximation accuracy of the degradation.

This approximation decomposition can be performed by obtaining the basis vectors m _i sequentially. The procedure of the third decomposition method is as follows.
(1) The base matrix M and the coefficient matrix C are obtained by the first or second decomposition method and set as initial values thereof.
(2) The elements of the base matrix M are fixed, and the elements of the coefficient matrix C are optimized by the least square method.
(3) The elements of the base matrix M are updated by fixing the elements of the coefficient matrix C and optimizing the elements of the base matrix M. The update process of the base matrix M will be described later.
(4) until convergence Repeat (2) and (3), to hold the cost function g ₃ as a minimized basis matrix M and the candidate coefficient matrix C have.
(5) Step (1) repeatedly to (6), employs a cost function g ₃ as a final result a basis matrix M and the coefficient matrix C minimized. In step (1), since the base matrix M and the coefficient matrix C are optimized again by the first or second decomposition method, the initial values are changed. In addition, although step (5) may not be repeated, the problem of initial value dependency can be reduced by repeating a plurality of times.

  Next, the update process of the base matrix M in step (3) will be described. In the case of data-dependent decomposition, the value of the row vector of the base matrix M is no longer independent of other rows and is dependent. Since the elements of the base matrix M are binary or ternary, that is, discrete values, the optimization of the base matrix M becomes a combinatorial optimization problem. Thus, for example, an algorithm such as a greedy algorithm, a tabu search, or a simulated annealing can be used to optimize the base matrix M. Since a good initial value is obtained in step (1), these algorithms can satisfactorily minimize the decomposition error.

For example, when the greedy algorithm is used, the base matrix M is optimized by the following procedure.
(3-1) T elements of the base matrix M are selected at random.
(3-2) tried combination of street 2 ^T (3 ^T as if 3 value decomposition described later), it is taken from the one cost function g ₃ minimized.
(3-3) Repeat step (3-1) and step (3-2) until convergence.

このコスト関数ｇ_４によれば、実数行列Ｑは、実際のデータの分布に従って分解されることになるため、分解の際の近似精度が向上するとともに、係数行列Ｃを疎にすることができる。即ち、第２の分解手法のメリットと第３の分解手法のメリットをいずれも得ることができる。具体的な分解の手順は、第３の分解手法と同様である。 (Fourth decomposition method)
The fourth decomposition method is a combination of the second decomposition method and the third decomposition method. Specifically, real matrix decomposition unit 105 performs degradation by solving the cost function g ₄ of the formula (6).

According to the cost function g _4, real matrix Q, since that would be resolved according to the distribution of actual data, as well as improved accuracy of approximation during decomposition can be sparsely coefficient matrix C. That is, both of the advantages of the second decomposition method and the third decomposition method can be obtained. The specific decomposition procedure is the same as that in the third decomposition method.

(Modification of the first and second decomposition methods)
In the first and second data-independent decomposition methods, when the number of decompositions is k, k ² types of search (k ³ types in the case of ternary decomposition) are required. Is difficult to apply. In such a case, the similarities of the real vectors q _n belonging to the real matrix Q are examined in advance, the similar real vectors are clustered, and the first or second decomposition method is applied to each cluster. do it.

The vector calculation unit 106 performs a calculation using a feature vector. Specific contents of the calculation will be specifically described later together with an application example of the feature amount calculation apparatus 100 of the present embodiment. The calculation using the feature vector includes calculation of a product Q ^T p of the binarized feature vector pε {−1,1} ^d and the real matrix Q decomposed by the real matrix decomposition unit 105. It is. In the following, calculation of the product Q ^T p will be described first.

ここで、ｍ_ｉ ^Ｔｐは二値ベクトル同士の内積である。また、ｃ_ｎ,ｉは、ｎ番目のクラスの係数ベクトルｃ_ｎのｉ番目の要素、即ち係数行列Ｃのｉ行ｎ列の要素である。この二値ベクトル同士の内積ｍ_ｉ ^Ｔｐは、極めて高速に計算可能である。その理由は以下のとおりである。 The product Q ^T p can be transformed into the following equation (7).

Here, m _i ^T p is an inner product of binary vectors. Further, c _{n, i} is an i-th element of the coefficient vector c _n of the n-th class, that is, an element of i rows and n columns of the coefficient matrix C. The inner product m _i ^T p between the binary vectors can be calculated extremely quickly. The reason is as follows.

ここで、前述のとおり、ｄはバイナリコードのビット数である。 An inner product between binary vectors can be reduced to a Hamming distance calculation. The Hamming distance is obtained by counting bits having different values in two binary codes, and the Hamming distance between two binary vectors is obtained by counting the number of elements having different values. Here, _{m i} and Hamming distance _{p D hamming (m i, p} ) and the described, the inner product _m ^{i T} p _is a relationship of _{D hamming (m} i, p) and the following equation (8).

Here, as described above, d is the number of bits of the binary code.

ここで、ＸＯＲ関数はｍ_ｉとｐをバイナリコード表現で考えたときに排他的論理和を取る操作であり、ＢＩＴＣＯＵＮＴ関数はバイナリコードの１が立っているビット数を数えあげる処理のことである。 The calculation of the Hamming distance is extremely fast because it can be calculated by counting the bits in which 1 stands after applying XOR in two binary codes. If the binary vector is expressed by a binary code (bit sequence of 0 and 1), the Hamming distance can be calculated by the following equation (9).

Here, XOR function is an operation of the exclusive OR when considering the m _i and p in binary code representation, BITCOUNT function is processing to enumerate the number of bits 1 of the binary code is standing .

すなわち、ｄビットのハミング距離計算をｋ回行い、ｋ個のハミング距離について、係数行列Ｃに関する重み付け和を計算し、定数項を足したものがＱ^Ｔｐになる。よって、ｋが十分小さければ、Ｑ^Ｔｐを浮動小数点精度で計算するよりも、はるかに高速に計算できるようになる。 In summary, the product Q ^T p can be transformed as shown in the following equation (10).

That is, d-bit Hamming distance calculation is performed k times, a weighted sum related to the coefficient matrix C is calculated for k Hamming distances, and Q ^T p is obtained by adding a constant term. Thus, if k is sufficiently small, Q ^T p can be calculated much faster than calculating floating point precision.

  The database 107 stores the product of the base matrix M and the coefficient matrix C as dictionary data for a plurality of real number matrices Q decomposed by the real number matrix decomposition unit 105. The vector calculation unit 106 reads the product of the base matrix M and the coefficient matrix C from the database 107 and performs the above calculation.

  As described above, according to the feature amount computing apparatus 100 of the present embodiment, even when the computation processing using the feature vector includes the product computation of the feature vector and the real number matrix, the feature vector is binarized. Then, the real matrix is also decomposed into the product of the base matrix and coefficient matrix, which is a binary matrix, so the product of the feature vector and the base matrix is calculated when calculating the product of the feature vector and the real matrix. In addition, by further calculating the product of the coefficient matrix, the product operation of the feature vector and the real number matrix can be speeded up.

  In addition, since a plurality of real vectors are combined into one real matrix and the real matrix is decomposed into a binary matrix, a base matrix and a coefficient matrix, each real vector is decomposed as in the prior application technique. In comparison, the number of basis vectors constituting the basis matrix, that is, the number of basis can be reduced. In principle, it is possible to set the number of bases to one or less per class (that is, base number k ≦ number of classes L).

1-2. Extension of the First Embodiment In the first embodiment described above, the binary vectors m _i and p are converted to m _i ε {−1,1} ^d and pε {−1,1} ^d , respectively. By definition, it has been explained that the product operation Q ^T p becomes faster by decomposing a real matrix into a product of a binary base matrix and a real coefficient matrix. However, even if m _i and p are more general binary vectors m _{i ′} ε {−a, a} ^d and p′ε {−a, a} ^d , their high-speed product operations can be performed. In this case, since m _i ′ ^T p ′ = a ² (m _i ^T p), the inner product of binary vectors defined by −1 and 1 may be multiplied by a ² .

なお、式（１１）及び（１２）中の太字の「１」は、長さがｄですべての要素が１であるベクトルである。また、式（１１）及び（１２）中のＡ、Ｂ、Ｃ、Ｄは実数であり、式（１１）及び（１２）が成立するようにあらかじめ計算しておけばよい。 Furthermore, even if the feature vector and the base vector are arbitrary binary vectors m _{i ″} ε {α, β} ^d , p ″ ε {γ, δ} ^d , high-speed inner product calculation is possible. Here, the coefficients α, β, γ, and δ are real numbers, and α ≠ β and γ ≠ δ. In this case, _{m i''} and p'' is obtained by performing a linear transformation to each element of the binary vector _{m i} and p defined by -1 and 1, the following equation (11) and (12) It is expanded like this.

Note that the bold “1” in the equations (11) and (12) is a vector having a length of d and all elements being 1. Further, A, B, C, and D in the equations (11) and (12) are real numbers, and may be calculated in advance so that the equations (11) and (12) are satisfied.

式（１３）の括弧内の計算は、−１及び１からなる二値ベクトル同士の内積である。従って、特徴ベクトルが任意の二値の要素をもつ二値ベクトルにされ、かつ、実数行列を二値の基底行列と実数の係数行列との積に展開した場合にも、高速演算が可能である。 The inner product m _{i ″} ^T p ″ can be expanded as shown in the following equation (13).

The calculation in the parentheses in the equation (13) is an inner product of binary vectors consisting of -1 and 1. Therefore, even when the feature vector is a binary vector having an arbitrary binary element and the real matrix is expanded into a product of a binary base matrix and a real coefficient matrix, high-speed calculation is possible. .

1-3. Second Embodiment Next, a feature amount computing device according to a second embodiment will be described. The configuration of the feature quantity computing device of the second embodiment is the same as that of the first embodiment shown in FIG. In the first embodiment, the real number matrix decomposing unit 105 decomposes the real number matrix Q into a binary base matrix and a real coefficient matrix according to equation (1). The real matrix decomposition unit 105 decomposes the real matrix into a ternary basis matrix and a real coefficient matrix.

ここで、Ｍ＝（ｍ_１，ｍ_２，…，ｍ_ｋ）∈｛−１，０，１｝^ｄ×ｋであり、Ｃ＝（ｃ_１，ｃ_２，…，ｃ_Ｌ）^Ｔ∈Ｒ^ｋ×Ｌである。すなわち、基底行列Ｍは、ｋ個の基底ベクトルｍ_ｉからなり、ここで、基底ベクトルｍ_ｉは、要素が−１、０、及び１のみをとるｄ次元の三値ベクトルであり、従って、基底行列Ｍは、要素が−１、０、及び１のみをとるｄ行ｋ列の三値行列である。 The real matrix decomposition unit 105 decomposes the real matrix QεR ^{d × L} with d rows and L columns into a product of a ternary basis matrix and a real coefficient matrix. Specifically, the real matrix decomposing unit 105 converts the real matrix QεR ^{d × L} of d rows and L columns into a base matrix M having ternary elements and a coefficient having real elements by the following equation (14). Decompose into matrix C.

Here, M = (m ₁ , m ₂ ,..., M _k ) ∈ {−1, 0, 1} ^{d × k} , and C = (c ₁ , c ₂ ,..., C _L ) ^T ∈ R ^{k × L.} That is, the basis matrix M consists of k basis vectors m _i, where the basis vectors m _i, the element is a three-value vector of d-dimensional take -1,0, and 1 only, therefore, basal The matrix M is a ternary matrix of d rows and k columns having elements of only −1, 0, and 1.

The coefficient matrix C, L number (L is the number of classes) a coefficient vector c _n of where the coefficient vector c _n, k-dimensional real number having real coefficients according to the k basis vectors as elements Is a vector. Of course, it is preferable to decompose Q and MC so that they coincide as much as possible, but an error may be included. The real number matrix decomposition unit 105 can decompose the real number matrix Q by the first to third decomposition methods in the same manner as in the first embodiment.

ここで、ｍ_ｉ ^Ｔｐは、三値ベクトルｍ_ｉと二値ベクトルｐとの内積である。積演算部１０６は、ここで、三値ベクトルｍ_ｉの代わりに、以下に定義する０置換ベクトルｍ_ｉ ^ｂｉｎ、フィルタベクトルｍ_ｉ ^{ｆｉｌｔｅｒ}、及び０要素数ｚ_ｉを用いる。 Vector operation unit 106 calculates product Q ^T p. Hereinafter, the vector calculation unit 106 that calculates the product Q ^T p is also specifically referred to as a product calculation unit 106. The product Q ^T p can be transformed into the following equation (15).

_{Here, m} ^{i T} p is the inner product of the ternary vector _{m i} and binary vector p. Product unit 106, here, instead of the three-valued vector _{m i,} 0 permutation vector _m ^{i bin} as defined below, filter vector _m ^{i filter,} and using a 0 element number _{z i.}

First, product unit 106, a zero element of _{m i,} replaced by -1 The 1. For each element of m _i, to replace it to -1, is either replaced by 1, it may be any. By this replacement, a zero replacement vector m _i ^bin ε {−1, 1} ^d is generated. The 0 substitution vector m _i ^bin ε {−1, 1} ^d is a binary vector.

Also, product unit 106 replaces the 0 element of _{m i} -1, replace the elements other than 0 to 1. By this replacement, a filter vector m _i ^filter ε {−1, 1} ^d is generated. The filter vector _m ^{i filter} is also binary vector.

ここで、式（１７）のＡＮＤ関数は、二値ベクトルをバイナリコード表現で考えたときに、論理積を取る操作である。 Furthermore, product unit 106 calculates the number of elements _{z i} 0 of _{m i.} z _i is an integer. The product calculation unit 106 uses the binary vector m _i ^bin , the filter vector m _i ^filter , and the number of 0 elements z _i to calculate m _i ^T p in the equation (15) and the following equations (16) and Calculate according to (17).

Here, the AND function of Expression (17) is an operation for taking a logical product when a binary vector is considered in binary code expression.

Hereinafter, the derivation of Expressions (16) and (17) will be described using the specific example of FIG. FIG. 6 is a diagram illustrating a calculation example of the present embodiment. In the example of FIG. 6, p = {- 1,1, -1,1, -1,1} and _is, m i = - a {1,0,1,0,1,1}. In this ^{_{example, m i bin = {- 1}} , *, 1, *, 1,1} a. Here, “*” represents any one of −1 or 1. ^{_{Also, m i filter = {1,}} -1,1, -1,1,1} _{, and} becomes a _z i = 2.

The exclusive OR of p and m _i ^bin in equation (17) is XOR (p, m _i ^bin ) = {− 1, *, 1, *, 1, −1}, that is, p and _mi Among the elements of, elements that are non-zero and different, that is, elements that are a pair of -1 and 1 or 1 and -1, are 1, and elements that are a pair of -1 and -1 or 1 and 1 are -1. .

Next, the logical product of the exclusive OR and _m ^{i filter _{is, AND (XOR (p, m}} i bin), m i filter)) = {- 1, -1,1, -1,1, - 1}, and the elements of p and m _i, 1 is standing on elements that differ nonzero, is -1 otherwise. When this bit count is taken, the number of elements that are 1, that is, the number of non-zero and different elements is counted, and D _{filtered_hamming} (p, m _i ^bin , m _i ^filter ) = 2.

Among the elements of p and _{m i,} 1 and 1 or -1 and the number of sets to become elements of -1, the total number of elements d = 6, elements that differ nonzero number _{D filtered_hamming} = It is obtained by subtracting the number z _i = 2 of elements that are 2 and 0. That is, the number of elements that are a set of 1 and 1 or −1 and −1 = d−D _{filtered_hamming−} z _i = 6-2-2 = 2.

m _i ^T p is an element (a set of −1 and 1 or 1 and −1) from the number of elements (a set of elements whose product is 1) that is a set of 1 and 1 or −1 and −1. since the product is equal to the value obtained by subtracting the number of pairs) of the elements becomes ^{_{-1, m i T p = (}} d-D filtered_hamming -z i) -D filtered_hamming = d-z i -2D filtered_hamming next, formula (16 ), And the value is 6-2-2 × 2 = 0. It should be noted that, as a matter of course, p ^T m _i = {− 1,1, −1,1, −1,1} × {−1,0,1,0,1,1} = 1 + 0 + (− 1 ) +0 + (− 1) + 1 = 0.

積演算部１０６は、この式（１８）によって、積Ｑ^Tｐを計算する。 Summarizing the equations (15) to (17), the product Q ^T p can be transformed as the following equation (18).

The product calculation unit 106 calculates the product Q ^T p by this equation (18).

The function D _{filtered_hamming} (p, m _i ^bin , m _i ^filter ) is very similar to the Hamming distance calculation and only an AND operation is added. Therefore, the Q∈R d ^{× L,} even when decomposed into a product of the three value matrix and the coefficient matrix, rather than calculating the Q T ^p in floating point precision, so that it can calculate the Q ^{T p} much faster Become.

As described above, the advantage of decomposing a d-dimensional real matrix Q∈R ^{d × L} into a product of a ternary basis matrix and a coefficient matrix instead of binary is that the approximation of Equation (10) is more The reason is that even a basis matrix with a small number of basis numbers is established. That is, since the number of bases can be kept small, the speed is further increased.

1-4. In the second embodiment of the extension above second embodiment, the binary vector p and ternary vector _{m i, ^{respectively, p∈ {-1,1} d, m}} i ∈ {-1,0 , 1} ^d , it has been explained that the inner product operation p ^T m _i is accelerated by decomposing a real matrix composed of a plurality of real vectors into a product of a ternary basis matrix and a coefficient matrix. However, even if p and m _i are more general binary vectors p′∈ {−a, a} ^d and ternary vectors m _i ∈ {−a, 0, a} ^d , their high-speed inner product operations can be performed. Is possible. In this case, since p ′ ^T m _i ′ = a ² (p ^T m _i ), the inner product of binary vectors defined by −1 and 1 may be multiplied by a ² .

なお、式（１９）及び（２０）中の太字の「１」は、長さがｄですべての要素が１であるベクトルである。また、式（１９）及び（２０）中のＡ、Ｂ、Ｃ、Ｄは実数であり、式（１９）及び（２０）が成立するようにあらかじめ計算しておく。 Furthermore, the binary vector p and ternary vector m _i the ^{p∈ {α, β} d,} m i ∈ {γ-δ, γ, γ + δ} be ^d and generalized, which enables high-speed dot product operations. Here, α, β, γ, and δ are real numbers, and α ≠ β and δ ≠ 0. In this case, by performing a linear transformation of the formula (19) and (20) to each element of the _{m i} and p, _{m i''} and p'' is obtained, respectively.

Note that the bold “1” in the equations (19) and (20) is a vector having a length of d and all elements being 1. Further, A, B, C, and D in the equations (19) and (20) are real numbers, and are calculated in advance so that the equations (19) and (20) are satisfied.

式（２１）の括弧内の計算は、−１及び１からなる二値ベクトル同士の内積、又は−１及び１からなる二値ベクトルと−１、０、１からなる三値ベクトルとの内積である。従って、特徴ベクトルが任意の二値ベクトルにされ、かつ、実数行列を上記のとおり一般化した三値行列を用いて展開した場合にも、そのような特徴ベクトルと実数行列との積を高速に演算できる。 The inner product m _{i ″} ^T p ″ can be expanded as shown in the following equation (21).

The calculation in the parenthesis of the equation (21) is an inner product of binary vectors consisting of -1 and 1, or an inner product of a binary vector consisting of -1 and 1, and a ternary vector consisting of -1, 0, 1. is there. Therefore, even when the feature vector is an arbitrary binary vector and the real matrix is expanded using the generalized ternary matrix as described above, the product of such a feature vector and the real matrix can be increased at high speed. Can be calculated.

2. Application Example Next, calculation processing in the vector calculation unit 106 will be described. The vector calculation unit 106 of the first and second embodiments described above involves calculation of the product of the binarized feature vector p and a real matrix Q that is a collection of a plurality of real vectors q. There are various such arithmetic processes. That is, the above-described embodiment of the present invention can be applied to various devices that perform arithmetic processing using feature vectors.

2-1. First Application Example In this application, the present invention is applied to an object recognition apparatus that recognizes a plurality of types of objects by SVM using HOG feature values. FIG. 7 is a block diagram illustrating a configuration of the object recognition apparatus. The object recognition apparatus 10 includes a pyramid image generation unit 11, an HOG feature amount extraction unit 12, a binary code conversion unit 13, a parameter determination unit 14, a parameter matrix decomposition unit 15, a linear SVM identification unit 16, and a peak detection Part 17.

  The pyramid image generation unit 11 acquires an image as an input query, and generates a G-stage pyramid image obtained by reducing the image at a plurality of levels of magnification. Thereby, it is possible to deal with objects of different sizes. This pyramid image generation unit 11 corresponds to the content acquisition unit 101 shown in FIG. The HOG feature amount extraction unit 12 divides the image at each stage of the pyramid image into blocks each having a size of 16 × 16 pixels, and extracts the HOG feature amount from each block. The HOG feature quantity extraction unit 12 extracts a D-dimensional feature quantity from each block. The HOG feature amount extraction unit 12 corresponds to the feature vector extraction unit 102 shown in FIG. The binary code conversion unit 13 converts the D-dimensional feature value given to each cell into a d-dimensional binary vector. The binary code conversion unit 13 corresponds to the feature vector binarization unit 103 shown in FIG.

The parameter determination unit 14 uses the weight vector w _n used in the linear SVM in the linear SVM identification unit 16 for each type of object to be recognized (adult, child, car, motorcycle, etc., which is defined by parameters). (N = 1, 2,..., L) and a real bias b _n (n = 1, 2,..., L) are determined. Parameter determination unit 14 uses the feature quantity prepared for the learning, and determining the L type of weight vector w _n and the bias b _n by the learning process to generate a weight matrix W summarizes the weight vector w _n . The parameter determination unit 14 corresponds to the real number matrix acquisition unit 104 shown in FIG. The parameter matrix decomposition unit 15 decomposes the weight matrix W into a product of a discrete value base matrix and a coefficient matrix according to the equation (2) or the equation (14) described in the first or second embodiment. The parameter matrix decomposition unit 15 corresponds to the real number matrix decomposition unit 105 shown in FIG.

ここで、線形ＳＶＭにおける積演算Ｗ^Ｔｘは、第１又は第２の実施の形態として説明した実数行列と二値ベクトルの高速な積演算により実現できる。 The linear SVM identifying unit 16 identifies feature vectors using a linear SVM. First, the linear SVM identification unit 16 configures a detection window by collecting s _x × s _y blocks as a group. The feature vector extracted from one detection window is a vector of s _x × s _y × d dimensions. The linear SVM identification unit 16 applies the linear SVM of the following expression (22) to this feature vector.

Here, the product operation W ^T x in the linear SVM can be realized by the high-speed product operation of the real matrix and the binary vector described as the first or second embodiment.

  The detection result may be hardened in the vicinity of the detection position. Therefore, the peak detection unit 17 sets a position where the value of f (x) is maximized around as a representative detection position. The linear SVM identification unit 16 and the peak detection unit 17 are configured to perform processing using feature vectors, and correspond to the vector calculation unit 106 in FIG.

Next, an example will be described in which the object recognition apparatus 10 detects a rotatable object based on the HOG feature amount. FIG. 8 shows a case where a dictionary q _n and a bias b _n are created at each rotation angle for a rotating road sign. In FIG. 8, the left-right direction indicates the rotation angle θ of the road sign.

In the conventional approach, the learning process is performed for each rotation angle to obtain the dictionary q _n and the bias b _n . Thereafter, the HOG feature amount is extracted from the input image, and this road sign is detected by applying a detection window (sliding window) L times. However, such a conventional method requires L times of inner product calculation per detection window, which increases the amount of calculation. Further, the angular resolution of detection is 2 pi / L, which is rough.

このようにｋ個の整数基底に分解することにより、１検出ウィンドウあたり、ｋ回の二値と二値との内積演算又は二値と三値との内積演算で処理が可能となる。このとき、隣り合う辞書同士が似ているため、整数基底の数ｋを小さくすることができ、原理的には１クラスあたり１個以下（ｋ≦Ｌ）とすることも可能である。 Therefore, in this application example, the parameter determination unit 14 collects the dictionary q _n into the matrix Q, and the SVM identification unit 16 collects the inner product calculation of the plurality of dictionaries q _n and the feature vector p by the following equation (23). Do.

By decomposing into k integer bases in this way, processing can be performed by k inner product operations of binary and binary values or inner product operations of binary values and ternary values per detection window. At this time, since adjacent dictionaries are similar, the number k of integer bases can be reduced, and in principle, one or less (k ≦ L) can be set per class.

In this application example, the peak detection unit 17 further increases the accuracy of detection resolution focusing on the property of the coefficient matrix C. FIG. 9 is a diagram illustrating the properties of the coefficient matrix C. When the real vector q _n changes according to the rotation angle θ as a parameter, a plurality of real vectors q _n are combined to generate a real matrix Q as shown in FIG. of arranging the real vector q _n in the order parameter theta, as shown in FIG. 9, the vector of the real vector q _n is ordered in the same direction as the direction of the coefficient matrix C, i.e. the coefficient matrix C of each row vector elements The line direction changes smoothly.

ここで、α_ｉは、フィッティングの係数である。 Therefore, the peak detector 17 fits the row vector of the coefficient matrix C with a polynomial and expresses it with a continuous function as shown in the following equation (24).

Here, α _i is a fitting coefficient.

ピーク検出部１７は、この識別関数を用いてピークの検出を行う。ｃ_ｉ（θ）は式（２４）に示すように多項式であるから、ｆ_θ（ｐ）もまた連続関数（連続の多項式）となる。図１０は、ｆ_θ（ｐ）の例を示すグラフである。図１０において、横軸は回転角度θであり、縦軸はｆ_θ（ｐ）である。ピーク検出部１７は、ｆ_θ（ｐ）が正の最大をとるときのθを対象の回転角度、即ち特徴ベクトルｐのパラメータ値として検出する。 If the formula of the discriminant function is arranged using this, the discriminant function at the rotation angle θ can be expressed in the form of a continuous function related to the parameter θ as in the following formula (25).

The peak detection unit 17 detects a peak using this discrimination function. Since c _i (θ) is a polynomial as shown in equation (24), f _θ (p) is also a continuous function (continuous polynomial). FIG. 10 is a graph showing an example of f _θ (p). In FIG. 10, the horizontal axis represents the rotation angle θ, and the vertical axis represents f _θ (p). The peak detection unit 17 detects _θ when f _θ (p) takes a positive maximum as the target rotation angle, that is, the parameter value of the feature vector p.

As described above, when generating a matrix Q together multiple dictionaries q _n, as the multiple dictionaries q _n it changes smoothly, the parameter matrix arranged in the order of (the θ in the example of FIG. 8) Q Since the discriminant function can be expressed in the form of a polynomial related to the parameter, the parameter can be detected with high resolution.

In the above description, the parameter is described as the rotation angle, but the parameter may be a scale, for example. That is, as shown in FIG. 2, the size of the detection window is fixed, the classifier is separately learned for each person size (scale) in the detection window, the polynomial is fitted to the scale σ, and the scale σ By obtaining the peak of the discriminator for, the scale can be estimated with high accuracy. Further, by devising in this way, generation of the pyramid image itself can be made unnecessary. Furthermore, there may be a plurality of parameters. For example, fitting to the above polynomial may be performed for both the rotation angle θ and the scale σ. In this case, the coefficient is a two-dimensional polynomial such as c _i (θ, σ).

The coefficient α _i can be obtained by first obtaining the coefficient matrix C and then fitting each row. However, the coefficient α _i may be obtained directly without obtaining the individual elements cn _{, i} of the coefficient matrix C. . Furthermore, the function to be fitted may not be a polynomial, and may be fitted to a trigonometric function (sine, cosine), for example.

2-2. Second Application Example In the present embodiment, the present invention is applied to k-means clustering. FIG. 11 is a block diagram illustrating a configuration of the k-means clustering apparatus. The k-means clustering device 20 includes a content acquisition unit 21, a feature vector generation unit 22, a feature vector binarization unit 23, a representative matrix update unit 24, a convergence determination unit 25, a representative matrix decomposition unit 26, The closest representative vector search unit 27 is provided.

  The content acquisition unit 21 acquires N contents to be clustered. The feature vector generation unit 22 extracts the feature amount from each content acquired by the content acquisition unit 21 as a feature vector p. The feature vector binarization unit 23 binarizes each feature vector extracted by the feature vector extraction unit 22.

First, the representative matrix update unit 24 randomly selects k (= L) from the N feature vectors binarized by the feature vector binarization unit 23, and selects them as the representative vector q _n (n = 1). , 2,..., L), and a matrix in which these representative vectors q _n are collected is a representative matrix Q. The convergence determination unit 25 performs convergence determination every time the representative matrix update unit 24 updates the representative matrix. If the convergence determination unit 25 determines that the convergence has been achieved, the k-means clustering apparatus 20 ends the clustering process. The representative matrix decomposition unit 26 decomposes the representative matrix updated by the representative matrix update unit 24 into a discrete value (binary or ternary) matrix.

The closest representative vector search unit 27 causes the N binary vectors input from the feature vector binarization unit 23 to belong to the nearest representative vector q _n , respectively. The closest representative vector search unit 27 outputs this result to the representative matrix update unit 24. For each representative vector q _n , the representative matrix update unit 24 calculates an average vector of feature vectors (binarized) belonging to the representative vector q _{n and} sets this as a new representative vector q _n . The representative vector q _n updated by the representative matrix update unit 24 in this way is calculated as the average of the binary vectors, and thus becomes a real vector.

Accordingly, if there is no representative matrix decomposition unit 26, the nearest representative vector search unit 27 calculates the inner product of them to obtain the distance between the updated representative vector (real vector) and the feature vector (binary vector). Must. Therefore, in the present embodiment, as described above, the representative matrix Q that is a set of the representative vectors q _n (real vector) is described by the representative matrix decomposing unit 26 in the first or second embodiment. And a product of a discrete value (binary or ternary) matrix and a real coefficient matrix. As a result, the nearest representative vector search unit 27 can calculate the distance between each feature vector and each representative vector at high speed, so that the representative vector to which each feature vector is closest (that is, the representative vector to which the feature vector belongs) is fast. To explore.

2-3. Third Application Example In the present embodiment, the present invention is applied to an approximate nearest neighbor search by k-means tree. The approximate nearest neighbor search apparatus according to the present embodiment uses Marius Muja and David G. Lowe, “Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration”, in as an approximate nearest neighbor search method using a k-ary tree using k-means. International Conference on Computer Vision Theory and Applications (VISAPP '09), 2009 (http://www.cs.ubc.ca/~mariusm/index.php/FLANN/FLANN, http: // people .cs.ubc.ca /~mariusm/uploads/FLANN/flann_visapp09.pdf)

  Specifically, the approximate nearest neighbor search apparatus of the present embodiment constructs a k-ary tree by recursively applying k-means to N pieces of data, and the principle of the tree search proposed above. To search for the nearest neighbor point approximately. This method is designed on the assumption that the data is a real vector and the representative vector registered in the node is a binary vector. However, even when the data is a binary vector and the representative vector registered in the node is a real vector, the tree search can be speeded up by adopting the first or second embodiment.

3. Modified Example In the feature amount calculation device 100, a part of the content acquisition unit 101, the feature vector generation unit 102, the feature vector binarization unit 103, the real number matrix acquisition unit 104, the real number matrix decomposition unit 105, and the vector calculation unit 106 and others These parts may be configured as separate devices. In particular, the content acquisition unit 101, the feature vector generation unit 102, the feature vector binarization unit 103, and the vector calculation unit 106 are mounted on the feature calculation device 100, and the real matrix acquisition unit 104 and the real matrix decomposition unit 105 are different from each other. It may be mounted on the device. In this case, a plurality of real number matrices decomposed by the real number matrix decomposition unit 105 are stored in the database 107 of the feature calculation apparatus 100, and the vector calculation unit 106 acquires the plurality of real number matrices decomposed from the database 107. .

  In the above embodiment, the base matrix M is binary or ternary, but the base matrix M may not be binary or ternary. If the number of types of elements that the base matrix M can take is a finite number, the real matrix can be decomposed by applying the above decomposition method. Also, the coefficient matrix C may be a discrete value determined in advance as in the base matrix M. For example, the elements of the coefficient matrix C may be constrained to a power of 2, and the processing can be speeded up by doing so. When the average value of the elements of the real number matrix Q to be decomposed is remarkably large (or small), that is, when the average value is significantly different from 0, the average value is subtracted from each element of the real number matrix Q in advance. When an offset real number matrix is generated and the offset real number matrix Q ′ is decomposed into a base matrix M and a coefficient matrix C, approximate decomposition of Expression (2) and Expression (14) can be performed with fewer bases.

  In the first and second embodiments, the content data acquired by the content acquisition unit 101 may be measurement data obtained from a vehicle. Furthermore, the measurement data obtained from the vehicle may be, for example, image data captured by a camera installed in the vehicle, or sensing data measured by a sensor installed in the vehicle. In this case, the vector calculation unit 106 of the feature calculation device 100 as the relevance determination device determines the relevance between the measurement data and the dictionary data. For example, when image data taken by a camera installed in a vehicle is acquired as measurement data, data of a plurality of person images are stored in a database as dictionary data, and the feature as a relevance determination device The vector calculation unit 106 of the calculation device 100 may determine whether or not a person is included in the image of the image data according to any of the application examples described above.

  The present invention is capable of speeding up the inner product calculation of a binarized feature vector and each of a plurality of real vectors, and determining the relevance between such a feature vector and each of a plurality of real vectors at high speed. A relationship that determines the relationship between a feature vector and each of a plurality of real vectors by a vector operation including the calculation of the inner product of a plurality of real vectors and a feature vector converted to a binary vector. It is useful as a sex determination device.

DESCRIPTION OF SYMBOLS 100 Feature amount calculating device 101 Content acquisition part 102 Feature vector generation part 103 Feature vector binarization part 104 Real number matrix acquisition part 105 Real number matrix decomposition part 106 Vector calculation part (product operation part)
DESCRIPTION OF SYMBOLS 10 Object recognition apparatus 11 Pyramid image generation part 12 HOG feature-value extraction part 13 Binary code conversion part 14 Parameter determination part 15 Parameter matrix decomposition part 16 Linear SVM identification part 17 Peak detection part 20 k-means clustering apparatus 21 Content acquisition part 22 Feature Vector generation unit 23 Feature vector binarization unit 24 Representative matrix update unit 25 Convergence determination unit 26 Representative matrix decomposition unit 27 Nearest representative vector calculation unit

Claims (9)

A feature vector acquisition unit for acquiring a binarized feature vector;
The vector obtained by decomposing a real matrix composed of a plurality of real vectors into a product of a coefficient matrix and a base matrix composed of a plurality of base vectors having only binary or ternary discrete values as elements and the coefficients A database for storing dictionary data including a matrix;
As the calculation of the inner product of the feature vector and each of the plurality of real vectors, the product of the feature vector and the base matrix is calculated, the product of the product and the coefficient matrix is calculated, and the result is A vector operation unit for determining the relevance between each of the plurality of real vectors and the feature vector;
A relevance determination device characterized by comprising:   The relevance determination device according to claim 1, further comprising a real number matrix generation unit that generates the real number matrix by arranging the plurality of real number vectors.   The real number matrix generation unit generates the real number matrix by arranging the plurality of real number vectors according to the order of the parameters when the plurality of real number vectors have a predetermined parameter. The relevance determination device described. The feature vector is a HOG feature amount,
The plurality of real vectors are a plurality of weight vectors corresponding to parameters of a plurality of linear classifiers,
The relevance according to claim 2, wherein the vector calculation unit identifies the feature vector for each of a plurality of criteria by using an identification function of the plurality of linear classifiers as the relevance determination. Judgment device. When the feature vector and the plurality of real vectors have one or more parameters, the real matrix generation unit generates the real matrix by arranging the plurality of real vectors according to the order of the parameters,
The vector calculation unit represents each of a plurality of vectors constituting the coefficient matrix in the same direction as a direction in which the plurality of real vectors are arranged as a continuous function related to the parameter, and the identification function The relevance determination apparatus according to claim 4, wherein the parameter that maximizes the value is obtained as a parameter value of the feature vector. The feature vector is a vector to be clustered by k-means clustering,
The real vector is a representative vector in k-means clustering,
The said vector calculating part performs the clustering process including the calculation of the distance between the said feature vector and the said representative vector as determination of the said relationship. The Claim 1 thru | or 3 characterized by the above-mentioned. Relevance determination device. The feature vector is a vector to be subjected to an approximate nearest neighbor search by k-means tree,
The real vector is a representative vector registered in a node of the k-ary tree,
The said vector calculating part performs the clustering process including the calculation of the distance between the said feature vector and the said representative vector as determination of the said relationship. The Claim 1 thru | or 3 characterized by the above-mentioned. Relevance determination device.   The relevance determination apparatus according to claim 1, wherein the feature vector is a vector that represents a feature amount of an image.   A relevance determination program for causing a computer to function as the relevance determination device according to any one of claims 1 to 8.

Images