JP2010086466A - Data classification device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To classify a data by reduced memories in a short calculation time. <P>SOLUTION: An SVM classification device 34 classifies the propriety as a face image, as to a pick-up image of a feature vector generated as a test data by a feature vector extracting part 32, using an SVM classification expression f(x) shown in Fig.11 obtained by learning, based on a quadratic function k(x, z) shown in Fig.11 approximated with a kernel function expressed by an exponential function, and based on a plurality of feature vectors prepared preliminarily as a training data, where x and z represent the feature vectors, γ represents a parameter in the kernel function, a, c and q represents coefficients determined by the approximation of the kernel function, x<SB>j</SB>represents a j-dimensional feature amount of the feature vector x, x<SB>k</SB>represents a k-dimensional feature amount of the feature vector x, v<SB>i</SB>represents a support vector, v<SB>ij</SB>represents a j-dimensional feature amount of the support vector v<SB>i</SB>, v<SB>ik</SB>represents a k-dimensional feature amount of the support vector v<SB>i</SB>, y<SB>i</SB>represents a label of the support vector v<SB>i</SB>, n is the number of the support vectors, d represents the dimension number of the feature vectors, and α<SB>1</SB>an b represent coefficients determined by the learning. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a data classification apparatus and program, and more particularly, to a data classification apparatus and program for classifying test data using an SVM classification formula.

Conventionally, a method of classifying whether a test image is a face image using an SVM classifier is known (Patent Document 1).
JP 2007-238090 A

  However, in the technique of the above-mentioned Patent Document 1, when the problem to be handled is complicated, the number of support vectors used in the SVM classifier becomes very large, so that a necessary amount of memory becomes enormous and a long calculation time is required. There is a problem that.

  The present invention has been made to solve the above problems, and an object of the present invention is to provide a data classification apparatus and program capable of classifying data with a small amount of memory and a short calculation time.

  In order to achieve the above object, a data classification apparatus according to the first invention includes a quadratic function k (x, z) represented by the following equation that approximates a kernel function represented by an exponential function, and training data: Classifying feature vectors input as test data using an SVM classification formula f (x) expressed by the following formula obtained by learning based on a plurality of feature vectors prepared in advance It is.

Further, x and z are feature vectors, γ is a parameter in a kernel function, and a, c, and q are coefficients determined by approximation of the kernel function. x _j is a j-dimensional feature quantity of the feature vector x, and x _k is a k-dimensional feature quantity of the feature vector x. v _i is the support _{vector, v ij} is the feature quantity of j dimension of support vectors _{v i, v ik} is the characteristic of k-dimensional support vector _{v i.} y _i is the label of the support vector v _i , n is the number of support vectors, and d is the number of dimensions of the feature vector. α _i and b are coefficients determined by the learning.

  The program according to the second aspect of the present invention provides a computer having a quadratic function k (x, z) represented by the following formula approximating a kernel function represented by an exponential function and a plurality of features prepared in advance as training data: A program for functioning as a means for classifying feature vectors input as test data using the SVM classification formula f (x) represented by the following formula, obtained by learning based on vectors: is there.

  According to the first invention and the second invention, a quadratic function k (x, z) represented by the above equation approximating a kernel function represented by an exponential function, and a plurality of prepared beforehand as training data The feature vectors input as test data are classified using the SVM classification formula f (x) expressed by the above formula obtained by learning based on the feature vectors.

  As described above, by classifying test data using the SVM classification formula having a small number of parameters, the data can be classified with a small amount of memory and a short calculation time.

  According to a third aspect of the present invention, there is provided a data classification device including a cubic function k (x, z) represented by the following expression approximating a kernel function represented by an exponential function, and a plurality of feature vectors prepared in advance as training data: The feature vectors input as test data are classified using the SVM classification formula f (x) expressed by the following formula obtained by learning based on the above.

Further, x and z are feature vectors, γ is a parameter in the kernel function, and a, c, q, and h are coefficients determined by approximation of the kernel function. x _j is a j-dimensional feature quantity of the feature vector x, x _k is a k-dimensional feature quantity of the feature vector x, and x _s is an s-dimensional feature quantity of the feature vector x. v _i is the support _{vector, v ij} is the feature quantity of j dimension of support vectors _{v i, v ik} is the characteristic of k-dimensional support vector _{v i, v IS} the s-dimensional support vector _{v i} It is a feature amount. y _i is the label of the support vector v _i , n is the number of support vectors, and d is the number of dimensions of the feature vector. α _i and b are coefficients determined by the learning.

  According to a fourth aspect of the present invention, there is provided a program comprising a computer, a cubic function k (x, z) represented by the following formula approximating a kernel function represented by an exponential function, and a plurality of feature vectors prepared in advance as training data: Is a program for functioning as a means for classifying feature vectors input as test data using the SVM classification formula f (x) represented by the following formula obtained by learning based on .

  According to the third and fourth aspects of the invention, learning is performed based on a cubic function represented by the above equation approximating a kernel function represented by an exponential function, and a plurality of feature vectors prepared in advance as training data. The feature vectors input as test data are classified using the SVM classification formula represented by the above formula obtained by the above.

  As described above, by classifying test data using the SVM classification formula having a small number of parameters, the data can be classified with a small amount of memory and a short calculation time.

  The above quadratic function can be a quadratic function obtained by approximating a kernel function by Tiller expansion. As a result, the kernel function can be approximated to a quadratic function with high accuracy.

  The above cubic function can be a cubic function obtained by approximating a kernel function by Tiller expansion. As a result, the kernel function can be approximated to the cubic function with high accuracy.

  The above quadratic function can be a quadratic function obtained by approximating a plurality of sampling data generated from an exponential function. As a result, the kernel function can be approximated to a quadratic function with high accuracy.

  Further, the quadratic function is approximated to sampling data that can be obtained when each of a plurality of discrete values is applied to the parameter γ in the kernel function among the plurality of sampling data generated from the exponential function. Thus, a plurality of quadratic functions obtained can be obtained.

  The above cubic function can be a cubic function obtained by approximating a plurality of sampling data generated from an exponential function. As a result, the kernel function can be approximated to the cubic function with high accuracy.

  Further, the above cubic function is approximated to sampling data that can be obtained when each of a plurality of discrete values is applied to a parameter γ in a kernel function among a plurality of sampling data generated from an exponential function. A plurality of cubic functions obtained by the above can be obtained.

  The plurality of sampling data described above can be generated according to the empirical distribution of the value of the exponent part of the exponential function obtained from the inner product value of a plurality of feature vectors prepared as training data.

  As the feature vector, a normalized feature vector can be used.

  As described above, according to the data classification apparatus and program of the present invention, data is classified with a small memory amount and a short calculation time by classifying test data using an SVM classification formula with few parameters. The effect that it can be obtained.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a case where the present invention is applied to an image classification apparatus that classifies whether an image captured by an imaging apparatus is a face image representing a face will be described as an example.

  As shown in FIG. 1, an image classification device 10 according to the first embodiment includes an imaging device 12 provided to capture a predetermined area, and a face image in which an image captured by the imaging device 12 represents a face. And a computer 14 for causing the display device 16 to display the classification result.

  The computer 14 includes a CPU, a ROM that stores programs for an SVM model learning processing routine and a face image classification processing routine, which will be described later, a RAM that stores data, and a bus that connects these. When the computer 14 is described in terms of functional blocks divided for each function realizing means determined based on hardware and software, as shown in FIG. 1, the kernel function of a support vector machine (Support Vector Machine, SVM) is quadratic. A kernel function approximation unit 20 that approximates a function, a training data storage unit 22 that stores a large number of feature vectors of face images and non-face images as training data, a quadratic function and training data that approximate a kernel function, The SVM learning unit 24 for learning the SVM classification formula, the SVM model conversion unit 26 for converting the formula by expanding the SVM classification formula obtained by learning, and the converted SVM classification formula are stored. And an SVM model storage unit 28.

  Here, a classification method using the SVM model will be described.

  In the classification method using the SVM model, when a set of training data is given, the training data is nonlinearly mapped to a higher-dimensional feature space, and a separation hyperplane having a maximum margin is constructed in the feature space.

In the SVM model, the calculation required for learning and classification involves only the internal integration of the features φ (x ₁ ) and φ (x ₂ ) in the nonlinear mapping space, so that instead of calculating the features φ (x ₁ ) explicitly Then, the kernel function K (x ₁ , x ₂ ) = φ (x ₁ ) ^T φ (x ₂ ) is implicitly calculated.

  As the kernel function, a polynomial kernel, an RBF (Radial Basis Function) kernel, a sigmoid kernel, or the like is often used. In the SVM model, only the vectors in the vicinity of the separation hyperplane need to be considered when calculating the kernel function, and these vectors are called support vectors. A general SVM classification formula is represented by the following formula (1).

Here, v _i represents a support vector, α _i represents a Lagrange coefficient, and y _i represents a label (1, −1) of each support vector v _i .

  In this embodiment, an RBF kernel is used as a kernel function. The classification performance by SVM using the RBF kernel is generally high performance and stable.

  The kernel function approximation unit 20 approximates the RBF kernel with a second-order polynomial function as described below. In the following description, it is assumed that the feature vector is normalized by the L2 norm without loss of universality.

  First, the RBF kernel can be expanded as shown in the following equation (2).

  Here, γ (> 0) is a parameter called kernel width. In the above equation (2), since the exponential function of the first term is always a fixed value, even if ignored, the kernel is not affected. That is, the RBF kernel is represented by the exponential function of the second term of the above equation (2).

  Further, the RBF kernel represented by the exponential function of the second term of the above equation (2) is approximated by a quadratic polynomial function represented by the following equation (3).

  The coefficients a, c, and q in the above equation (3) are determined as follows.

First, since the feature vectors x and z are normalized by the L2 norm, the inner product value x ^T z is always in the range [0, 1]. Therefore, the range of the exponent part 2γx ^T z of the RBF kernel is [0, 2γ].

In addition, the RBF kernel represented by an exponential function can be approximated by a tiller expansion. When the parameter γ takes a small value, since the range of the exponent part 2γx ^T z of the RBF kernel is a small interval, the RBF kernel can be approximated by the second-order Tiller expansion. = 1, c = 1, q = 1/2.

As will be described below, the SVM learning unit 24 learns γ, α _i , and b to determine the SVM classification formula.

First, as the kernel function of the SVM classification formula f (x) of the above formula (1), an approximated second order polynomial function of the above formula (3) is used, and each of a plurality of discrete values is set as the kernel function parameter γ. applying, for each discrete value of gamma, based on the feature vector of the training data, the following equation (4) that maximizes and with learning the alpha _i, when applying learned alpha _i Each SVM classification formula is determined by selecting b satisfying the following formula (5) for any i.

Here, N is the number of training data, x _i is a feature vector of training data, and y _i is a teacher label (1, −1) of the feature vector x _i . C is a constant.

Next, for each of the SVM classification formulas using α _i and b determined for the discrete values of γ, the calculation result when the feature vector of the training data is applied is compared with the teacher label of the training data. To do.

From each of the comparison results obtained for the discrete values of each γ, γ to be an optimal SVM classification formula is determined, and α _i and b determined for this γ, The SVM classification formula using is used as a learning result.

The SVM model conversion unit 26 converts the SVM classification expression f ₂ (x) obtained by learning into an expression with fewer parameters by expanding the expression as shown in the following expression (6), and stores the SVM model. Stored in the unit 28.

X _j is a j-dimensional feature quantity of the feature vector x, and x _k is a k-dimensional feature quantity of the feature vector x. v _i is the support _{vector, v ij} is the feature quantity of j dimension of support vectors _{v i, v ik} is the characteristic of k-dimensional support vector _{v i.} y _i is the label of the support vector v _i , n is the number of support vectors, and d is the number of dimensions of the feature vector. α _i and b are coefficients determined by learning.

The parameters β _j and ω _jk used in the above SVM classification formula (6) are calculated in advance from the weighted addition result between the support vectors and stored in advance in a memory (not shown).

  The computer 14 also includes an image input unit 30 that inputs a captured image output from the imaging device 12, and a feature vector extraction unit that extracts a feature vector composed of a plurality of types of feature amounts as test data from the captured image. 32 and the SVM classification unit stored in the SVM model storage unit 28, the test data is classified as to whether it is a face image, and the classification result is displayed on the display device 16. 34.

  The image input unit 30 includes, for example, an A / D converter and an image memory that stores image data for one screen.

  The feature vector extraction unit 32 extracts a plurality of types of feature amounts from the captured image, and generates a feature vector including a plurality of types of feature amounts as test data. Note that the feature vector of the training data and the feature vector of the test data are vectors composed of the same type of feature quantity.

  The SVM classification unit 34 calculates the SVM classification formula expressed by the above formula (6) for the feature vector of the test data. If the calculation result is a positive value (including 0), the captured image is a face image. If the calculation result is a negative value, the captured image is classified as a face image.

  Next, the operation of the image classification device 10 according to the first embodiment will be described.

  First, a large number of feature vectors of face images and feature vectors of non-face images are prepared as training data, and are stored in the training data storage unit 22 of the computer 14 together with a teacher label. Further, the SVM model learning processing routine shown in FIG.

  First, in step 100, the RBF kernel that is a kernel function is approximated by a quadratic polynomial function by quadratic tiller expansion. Then, in step 102, the training data is read. In step 104, the training data is applied to the quadratic polynomial function approximated in step 100 to learn, thereby determining the coefficient of the SVM classification formula. obtain.

  In the next step 106, the SVM classification formula obtained in the step 104 is converted into a mathematical formula represented by the above formula (6) by expanding the mathematical formula, and in the step 108, the SVM model storage unit 28 converts the formula into the mathematical formula. The SVM classification formula is stored, and the SVM model learning processing routine is terminated.

  Further, when an image of a predetermined area is picked up by the image pickup device 12, the face image classification processing routine shown in FIG.

  First, in step 120, a captured image is acquired from the imaging device 12, and in step 122, a plurality of types of feature amounts are extracted from the captured image acquired in step 120, thereby generating a feature vector as test data.

In step 124, the feature vector generated in step 122 is applied to the SVM classification formula stored in the SVM model storage unit 28 for calculation. In the next step 126, it is determined whether or not the calculated value obtained in step 124 is a positive value. If the calculated value is a positive value, in step 128, the classification result that the captured image is a face image is determined. Is displayed on the display device 16, and the face image classification processing routine is terminated. On the other hand, if it is determined in step 126 that the calculated value obtained in step 124 is a negative value, a classification result indicating that the captured image is not a face image is displayed on the display device 16 in step 130. Then, the face image classification processing routine ends.
Since the ω _kj = ω _jk in the equation (6), in the calculation of the face image classification processing routine, used in the SVM classifier formula {beta _j}, the total amount of memory related to {omega _jk} is ^d 2/2 (D represents the number of dimensions of the feature quantity). On the other hand, the total memory amount related to the original SVM model that is not approximated is nd (number of support vectors n × dimension number d of feature amount), and the efficiency of memory amount by the method of this embodiment is 2n / d. Become. Here, in general, the number of dimensions d of the feature amount is smaller than the number n of support vectors, so that the total memory amount is reduced. For example, when the feature quantity dimension d is 100 and the support vector number n is 4000, calculation is performed using the SVM model before conversion by calculating using the above equation (6). In comparison, the total memory capacity is about 80 times more efficient. Also, the calculation time can be improved by the method of the present embodiment in the same manner as the memory amount.

  As described above, according to the image classification device according to the first embodiment, the test data is classified by using the SVM classification formula having a small number of parameters based on the RBF kernel approximated to the quadratic function, thereby reducing the number of test data. Data can be classified by the amount of memory and in a short calculation time.

  Since it is not necessary to reduce the number of support vectors in order to reduce the amount of memory and calculation time, it can be processed at high speed without degrading the classification performance as compared with the classification based on the SVM model using the RBF kernel of the conventional method, and The amount of memory can be reduced and the cost can be reduced.

  Further, the RBF kernel can be approximated with a quadratic function with high accuracy by the quadratic tiller expansion.

  Next, a second embodiment will be described. Note that the configuration of the image classification apparatus according to the second embodiment is the same as that of the first embodiment, and thus the same reference numerals are given and description thereof is omitted.

  The second embodiment is mainly different from the first embodiment in that the RBF kernel is approximated to a cubic polynomial function.

  In the image classification device according to the second embodiment, the kernel function approximating unit 20 represents the RBF kernel represented by the exponential function of the second term of the above equation (2) by the following equation (7). Approximate with cubic polynomial function.

  The coefficients a, c, q, and h in the equation (7) are determined as follows.

When the parameter γ takes a small value, the range of the exponent part 2γx ^T z of the RBF kernel is a small interval, so that the RBF kernel can be approximated by the third-order Tiller expansion, and a = 1, c = 1, q = 1/2, and h = 1/6.

The SVM learning unit 24 uses the approximated third-order polynomial function of the equation (7) as the kernel function of the SVM classification equation f (x) of the equation (1), and uses a plurality of parameters γ for the kernel function. Applying each of the discrete values to learn α _i that maximizes the above equation (4) based on the feature vector of the training data for each discrete value of γ, (5) b satisfying the equation is selected to determine each of the SVM classification equations.

Further, for each SVM classification formula using α _i and b determined for each discrete value of γ, the calculation result when the feature vector of the training data is applied is compared with the teacher label of the training data. , Γ for determining the optimum SVM classification formula is determined from each of the comparison results obtained for the discrete values of γ, α _i and b determined for the γ, and determined γ The SVM classification formula using is used as a learning result.

The SVM model conversion unit 26 converts the SVM classification expression f ₃ (x) obtained by learning into an expression with fewer parameters by expanding the expression as shown in the following expression (8), and stores the SVM model. Stored in the unit 28.

Further, x _j is a j-dimensional feature quantity of the feature vector x, x _k is a k-dimensional feature quantity of the feature vector x, and x _s is an s-dimensional feature quantity of the feature vector x. v _i is the support _{vector, v ij} is the feature quantity of j dimension of support vectors _{v i, v ik} is the characteristic of k-dimensional support vector _{v i, v IS} the s-dimensional support vector _{v i} It is a feature amount. y _i is the label of the support vector v _i , n is the number of support vectors, and d is the number of dimensions of the feature vector. α _i and b are coefficients determined by the learning.

The parameters β _j , ω _jk , θ _jks used in the SVM classification formula of the above formula (8) are calculated in advance from the weighted addition result between the support vectors and stored in advance in a memory (not shown).

  Note that other configurations and operations of the image classification device according to the second embodiment are the same as those of the first embodiment, and thus the description thereof is omitted.

Here, in the calculation using the above equation (8), the total memory amount related to the parameters {β _j }, {ω _jk }, {θ _jks } used is proportional to d (d ² + 6d) / 2. Compared with the original SVM model that is not approximated, the memory efficiency is 2n / (d ² + 6d) according to the method of the present embodiment. Therefore, when the number of dimensions of the feature quantity is small (for example, less than 100), the efficiency of the total memory quantity can be realized. Also, the calculation time can be improved in the same manner as the memory amount.

  In this way, by classifying test data using the SVM classification formula having a small number of parameters based on the RBF kernel approximated to a cubic function, the data can be classified with a small memory amount and a short calculation time. In addition, by the cubic Tiller expansion, the RBF kernel can be approximated with a cubic function with high accuracy.

  Next, a third embodiment will be described. Note that the configuration of the image classification apparatus according to the third embodiment is the same as that of the first embodiment, and thus the same reference numerals are given and description thereof is omitted.

  In the third embodiment, a second-order polynomial function that approximates the RBF kernel is obtained by fitting the sampling data generated from the exponential function of the RBF kernel. And mainly different.

  In the image classification device according to the third embodiment, the kernel function approximating unit 20 approximates the RBF kernel with a quadratic polynomial function expressed by the above equation (3) as described below. Here, description will be made assuming that the feature vector is normalized by the L2 norm without loss of universality.

First, in the range of the exponent part 2γx ^T z of the RBF kernel, sampling data is generated at equal intervals from the exponential function of the second term of the above equation (2) as shown in FIG. Generate a set of distributed sampling data.

Further, since it is unknown which value is applied to the parameter γ of the RBF kernel, each value is approximated to a quadratic polynomial function within a range that can be taken when each discrete value is applied. For example, if the discrete values applied to γ are every 0.5 and [0.5, 1, 1.5...], The possible range of 2γx ^T z is {[0, 1], [0, 2 ], [0, 3] ...}. As shown in FIG. 4B, a quadratic polynomial function that performs fitting by a least square method on a set of sampling data included in each range (section) to minimize the sum of squares of the residuals. The coefficients a, c, and q of the above equation (6) are determined so as to approximate to ## EQU3 ## to obtain a second order polynomial function for each discrete value of the parameter γ. Note that FIG. 4B shows a fitting state in a section that can be taken when γ = 1.

  As described below, the SVM learning unit 24 learns γ, αi, and b, and determines the SVM classification formula.

First, using the quadratic polynomial function approximated for each discrete value of γ as the kernel function of the SVM classification formula f (x) of the above equation (1), for each discrete value of γ, the above (4 ) Learning α _i that maximizes the expression, and selecting b satisfying the above expression (5) to determine the SVM classification expressions.

Next, for each of the SVM classification formulas using α _i and b determined for the discrete values of γ, the calculation result when the feature vector of the training data is applied is compared with the teacher label of the training data. To do.

From each of the comparison results obtained for the discrete values of each γ, γ when an optimum SVM classification formula is obtained is determined, and α _i and b determined for this γ, and for this γ An SVM classification formula using the approximated second-order polynomial function and the determined γ is adopted as a learning result.

  Similar to the first embodiment, the SVM model conversion unit 26 develops mathematical expressions of the SVM classification expressions obtained by learning and converts them into mathematical expressions as shown in the above equation (6).

  Next, the SVM model learning process routine according to the third embodiment will be described with reference to FIG. In addition, about the process similar to 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  First, in step 300, sampling data is generated at regular intervals using the exponential function of the RBF kernel. In step 302, the RBF kernel is calculated based on the sampling data in the corresponding range for each discrete value of the parameter γ. Is approximated by a quadratic polynomial function.

  Then, in step 102, the training data is read, and in step 304, the training data is applied to the quadratic polynomial function approximated for each discrete value of γ in step 302, thereby learning the parameters γ and RBF kernel. The coefficient of the SVM classification formula is determined, and the SVM classification formula is obtained.

  In the next step 106, the SVM classification formula obtained in the above step 304 is converted into a mathematical formula represented by the above formula (6) by expanding the mathematical formula, and in the step 108, it is converted into the SVM model storage unit 28. The SVM classification formula is stored, and the SVM model learning processing routine is terminated.

  Further, when an image of a predetermined area is picked up by the image pickup device 12, a face image classification processing routine is executed in the computer 14 as in the first embodiment.

  In this way, by classifying test data using the SVM classification formula having a small number of parameters based on the RBF kernel approximated to a quadratic function, the data can be classified with a small amount of memory and a short calculation time. .

  In addition, by fitting the sampling data, the RBF kernel can be approximated with a quadratic function with high accuracy.

  Next, a fourth embodiment will be described. Note that the configuration of the image classification apparatus according to the fourth embodiment is the same as that of the first embodiment, and thus the same reference numerals are given and description thereof is omitted.

  The fourth embodiment is mainly different from the third embodiment in that the RBF kernel is approximated to a cubic polynomial function.

  In the image classification device according to the fourth embodiment, the kernel function approximating unit 20 approximates the RBF kernel with the cubic polynomial function expressed by the above equation (7) as described below.

First, in the range of the exponent part 2γx ^T z of the RBF kernel, sampling data is generated at equal intervals from the exponent function of the second term of the above equation (2), and a set of uniformly distributed sampling data is generated.

In addition, each is approximated to a cubic polynomial function in a range that can be taken when each discrete value is applied to the parameter γ of the RBF kernel. By fitting the set of sampling data included in the range (section) of 2γx ^T z that can be obtained when each discrete value of γ is applied, fitting is performed by the method of least squares so as to minimize the sum of squares of the residuals. The coefficients a, c, q, and h in the above equation (7) are determined so as to approximate a cubic polynomial function to obtain a cubic polynomial function for each discrete value of the parameter γ.

As in the third embodiment, the SVM learning unit 24 learns γ, α _i , and b and determines the SVM classification formula.

  Note that the other configuration and operation of the image classification device according to the fourth embodiment are the same as those of the third embodiment, and thus the description thereof is omitted.

  In this way, by classifying test data using the SVM classification formula having a small number of parameters based on the RBF kernel approximated to a cubic function, the data can be classified with a small memory amount and a short calculation time.

  In addition, by fitting the sampling data, the RBF kernel can be approximated with a cubic function with high accuracy.

  Next, a fifth embodiment will be described. Note that the configuration of the image classification apparatus according to the fifth embodiment is the same as that of the first embodiment, and thus the same reference numerals are given and description thereof is omitted.

  The fifth embodiment is different from the third embodiment in that sampling data is created in accordance with the histogram of the exponent part calculated from the training data and the RBF kernel is approximated.

  In the image classification device according to the fifth embodiment, the kernel function approximating unit 20 approximates the RBF kernel with a quadratic polynomial function expressed by the above equation (3) as described below.

First, as shown in FIG. 6, an empirical distribution (histogram) of inner product values x ^T z between feature vectors is calculated from the training data, and an empirical distribution (histogram) of the value of the exponent part 2γx ^T z is calculated from the calculated histogram. ) Then, in the range of the exponent part 2γx ^T z of the RBF kernel, from the exponent function of the second term of the above equation (2), as shown in FIG. 7A, according to the histogram of the value of the exponent part 2γx ^T z , Generate sampling data, and generate a set of sampling data.

In addition, each is approximated to a quadratic polynomial function within a range that can be taken when each discrete value is applied to the parameter γ of the RBF kernel. As shown in FIG. 7B, fitting is performed by a least square method to a set of sampling data included in the range (section) of 2γx ^T z that can be obtained when each discrete value of γ is applied, The coefficients a, c, q in the above equation (6) are determined so as to approximate a quadratic polynomial function that minimizes the sum of squares of the residuals, and a quadratic polynomial function for each discrete value of the parameter γ is obtained. . Note that FIG. 7B shows a state of fitting in the section of the exponent value that can be taken when γ = 1.

As in the third embodiment, the SVM learning unit 24 learns γ, α _i , and b and determines the SVM classification formula.

  Next, an SVM model learning process routine according to the fifth embodiment will be described with reference to FIG. In addition, about the process similar to 1st Embodiment and 3rd Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  First, in step 102, the training data is read. In step 500, the inner product values between the feature vectors of the training data read in step 102 are calculated to calculate the inner product value histogram, and the calculated inner product value histogram is calculated. To generate a histogram of the exponent part.

  In step 502, a plurality of sampling data is generated according to the histogram of the exponent part generated in step 500 above using the exponential function of the RBF kernel. In step 302, for each discrete value of the parameter γ, Then, based on the sampling data in the corresponding range, the RBF kernel represented by the exponential function is approximated by a second-order polynomial function.

  In step 304, the training data is learned by applying it to the quadratic polynomial function approximating the RBF kernel for each discrete value of γ in step 302, so that the parameter γ of the RBF kernel and the coefficient of the SVM classification equation are obtained. To determine the SVM classification formula.

  In the next step 106, the SVM classification formula obtained in the above step 304 is converted into a mathematical formula represented by the above formula (6) by expanding the mathematical formula, and in the step 108, it is converted into the SVM model storage unit 28. The SVM classification formula is stored, and the SVM model learning processing routine is terminated.

  Further, when an image of a predetermined area is picked up by the image pickup device 12, a face image classification processing routine is executed in the computer 14 as in the first embodiment.

  In this way, by classifying test data using the SVM classification formula having a small number of parameters based on the RBF kernel approximated to a quadratic function, the data can be classified with a small amount of memory and a short calculation time. .

  In addition, the RBF kernel can be approximated with a quadratic function with high accuracy by fitting the sampling data according to the experience distribution.

  Next, a sixth embodiment will be described. Note that the configuration of the image classification apparatus according to the sixth embodiment is the same as that of the first embodiment, and thus the same reference numerals are given and description thereof is omitted.

  The sixth embodiment is mainly different from the fifth embodiment in that the RBF kernel is approximated to a cubic polynomial function.

  In the image classification device according to the sixth embodiment, the kernel function approximating unit 20 approximates the RBF kernel with the cubic polynomial function expressed by the above equation (7) as described below.

First, the empirical distribution (histogram) of the inner product value x ^T z is calculated from the training data, and the empirical distribution (histogram) of the value of the exponent part 2γx ^T z is obtained from the calculated histogram. Then, in the range of the exponent part 2γx ^T z of the RBF kernel, a plurality of sampling data is generated according to the empirical distribution of the value of the exponent part 2γx ^T z from the exponent function of the second term of the above equation (2). Generate a collection of sampling data.

In addition, each is approximated to a cubic polynomial function in a range that can be taken when each discrete value is applied to the parameter γ of the RBF kernel. By fitting the set of sampling data included in the range (section) of 2γx ^T z that can be obtained when each discrete value of γ is applied, fitting is performed by the method of least squares so as to minimize the sum of squares of the residuals. The coefficients a, c, q, and h in the above equation (7) are determined so as to approximate a cubic polynomial function, and a cubic polynomial function for each discrete value of the parameter γ is obtained.

As in the third embodiment, the SVM learning unit 24 learns γ, α _i , and b and determines the SVM classification formula.

  Note that other configurations and operations of the image classification device according to the sixth embodiment are the same as those of the fifth embodiment, and thus description thereof is omitted.

  In this way, by classifying test data using the SVM classification formula having a small number of parameters based on the RBF kernel approximated to a cubic function, the data can be classified with a small memory amount and a short calculation time.

  In addition, the RBF kernel can be approximated with a cubic function with high accuracy by fitting the sampling data according to the experience distribution.

  Next, when the dimension number of the feature quantity and the number of support vectors are changed, classification by SVM using the approximate RBF kernel according to the above embodiment, and SVM using the conventional strict RBF kernel The result of comparing the amount of memory and the calculation time in the classification according to will be described.

  As shown in FIG. 9, the RBF kernel approximated by the second-order polynomial function or the third-order polynomial function described in the above embodiment is compared with the memory amount and the calculation time in the classification by the SVM using the strict RBF kernel. It can be seen that the calculation time and the amount of memory are reduced by the classification by the used SVM.

  Moreover, the result of a pedestrian identification experiment is demonstrated. A feature vector made up of 336-dimensional image features extracted from the captured image is used as test data, and an SVM classifier is used to classify whether the image represents a pedestrian. As described in the above embodiment, the classification by SVM in which the RBF kernel is approximated by a quadratic polynomial function is performed. As a comparative example, classification by SVM using the original strict RBF kernel (number of support vectors = 6685) was performed.

  As shown in FIG. 10, when the RBF kernel is approximated by a quadratic polynomial function, the calculation time can be reduced by 30 times or more and the amount of memory is about 40 times compared to the case where the original exact RBF kernel is used. We were able to reduce it. Further, when the RBF kernel was approximated by a quadratic polynomial function, a classification accuracy close to that obtained when the strict RBF kernel was used was obtained.

  In the above embodiment, the case where the captured image is classified as a face image has been described as an example. However, the present invention is not limited to this, and the data classification apparatus divides other test data into other classifications. The present invention may be applied to.

  Further, the program of the present invention may be provided by being stored in a storage medium.

It is a block diagram which shows the image classification device which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the SVM model learning process routine in the image identification device which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the face image classification process routine in the image identification device which concerns on the 1st Embodiment of this invention. (A) A graph showing a set of uniformly generated sampling data, and (B) a graph showing a quadratic function approximated to sampling data. It is a flowchart which shows the content of the SVM model learning process routine in the image identification device which concerns on the 3rd Embodiment of this invention. It is a graph which shows the experience distribution of the inner product value between the feature vectors of training data. (A) A graph showing a set of sampling data generated according to the experience distribution, and (B) a graph showing a quadratic function approximated to the sampling data. It is a flowchart which shows the content of the SVM model learning process routine in the image identification device which concerns on the 5th Embodiment of this invention. It is a graph which shows the relationship between the dimension number of the feature-value of a feature vector, calculation time, and memory amount. It is a figure which shows the calculation time and memory amount at the time of using a strict RBF kernel and the case of using the RBF kernel approximated with each approximation method.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image classification apparatus 14 Computer 20 Kernel function approximation part 22 Training data storage part 24 SVM learning part 26 SVM model conversion part 28 SVM model storage part 32 Feature vector extraction part 34 SVM classification part

Claims (12)

Obtained by learning based on a quadratic function k (x, z) represented by the following equation that approximates a kernel function represented by an exponential function and a plurality of feature vectors prepared in advance as training data A data classification device for classifying feature vectors input as test data using an SVM classification formula f (x) represented by the following formula.

Further, x and z are feature vectors, γ is a parameter in a kernel function, and a, c, and q are coefficients determined by approximation of the kernel function. x _j is a j-dimensional feature quantity of the feature vector x, and x _k is a k-dimensional feature quantity of the feature vector x. v _i is the support _{vector, v ij} is the feature quantity of j dimension of support vectors _{v i, v ik} is the characteristic of k-dimensional support vector _{v i.} y _i is the label of the support vector v _i , n is the number of support vectors, and d is the number of dimensions of the feature vector. α _i and b are coefficients determined by the learning. Obtained by learning based on a cubic function k (x, z) represented by the following expression that approximates a kernel function represented by an exponential function, and a plurality of feature vectors prepared in advance as training data, A data classification device that classifies feature vectors input as test data using an SVM classification formula f (x) represented by the following formula.

Further, x and z are feature vectors, γ is a parameter in the kernel function, and a, c, q, and h are coefficients determined by approximation of the kernel function. x _j is a j-dimensional feature quantity of the feature vector x, x _k is a k-dimensional feature quantity of the feature vector x, and x _s is an s-dimensional feature quantity of the feature vector x. v _i is the support _{vector, v ij} is the feature quantity of j dimension of support vectors _{v i, v ik} is the characteristic of k-dimensional support vector _{v i, v IS} the s-dimensional support vector _{v i} It is a feature amount. y _i is the label of the support vector v _i , n is the number of support vectors, and d is the number of dimensions of the feature vector. α _i and b are coefficients determined by the learning.   The data classification apparatus according to claim 1, wherein the quadratic function is a quadratic function obtained by approximating the kernel function by Tiller expansion.   3. The data classification device according to claim 2, wherein the cubic function is a cubic function obtained by approximating the kernel function by Tiller expansion.   The data classification apparatus according to claim 1, wherein the quadratic function is a quadratic function obtained by approximating a plurality of sampling data generated from the exponential function.   The quadratic function is approximated to the sampling data that can be taken when each of a plurality of discrete values is applied to the parameter γ in the kernel function among the plurality of sampling data generated from the exponential function. The data classification device according to claim 5, wherein a plurality of quadratic functions obtained as described above are used.   The data classification device according to claim 2, wherein the cubic function is a cubic function obtained by approximating a plurality of sampling data generated from the exponential function.   Approximating the cubic function with respect to the sampling data that can be obtained when each of a plurality of discrete values is applied to the parameter γ in the kernel function among the plurality of sampling data generated from the exponential function. The data classification apparatus according to claim 7, wherein a plurality of cubic functions obtained by the step are used.   9. The method according to claim 5, wherein the plurality of sampling data is generated according to an empirical distribution of values of exponent parts of the exponential function obtained from inner product values of the plurality of feature vectors prepared as the training data. The data classification device according to claim 1.   The data classification device according to claim 1, wherein a normalized feature vector is used as the feature vector. Computer
Obtained by learning based on a quadratic function k (x, z) represented by the following equation that approximates a kernel function represented by an exponential function and a plurality of feature vectors prepared in advance as training data A program for functioning as a means for classifying feature vectors input as test data using the SVM classification formula f (x) represented by the following formula.

Further, x and z are feature vectors, γ is a parameter in a kernel function, and a, c, and q are coefficients determined by approximation of the kernel function. x _k is a k-dimensional feature quantity of the feature vector x. v _i is the support _{vector, v ij} is the feature quantity of j dimension of support vectors _{v i, v ik} is the characteristic of k-dimensional support vector _{v i.} y _i is the label of the support vector v _i , n is the number of support vectors, and d is the number of dimensions of the feature vector. α _i and b are coefficients determined by the learning. Computer
Obtained by learning based on a cubic function k (x, z) represented by the following expression that approximates a kernel function represented by an exponential function, and a plurality of feature vectors prepared in advance as training data, A program for functioning as a means for classifying feature vectors input as test data using the SVM classification formula f (x) represented by the following formula.

Further, x and z are feature vectors, γ is a parameter in the kernel function, and a, c, q, and h are coefficients determined by approximation of the kernel function. x _k is a k-dimensional feature quantity of the feature vector x, and x _s is an s-dimensional feature quantity of the feature vector x. v _i is the support _{vector, v ij} is the feature quantity of j dimension of support vectors _{v i, v ik} is the characteristic of k-dimensional support vector _{v i, v IS} the s-dimensional support vector _{v i} It is a feature amount. y _i is the label of the support vector v _i , n is the number of support vectors, and d is the number of dimensions of the feature vector. α _i and b are coefficients determined by the learning.

Images