JP6676894B2 - Information processing apparatus, information processing method and program - Google Patents

Description

  The present invention relates to an information processing device, an information processing method, and a program.

  A method has been devised in which a discriminant function is constructed (learned) based on learning data composed of a set of an input vector and a label of a class, and a class to which an unknown input vector belongs is predicted. One of them is a support vector machine (hereinafter abbreviated as SVM). In SVM, it is known that high generalization performance can be obtained by maximizing a margin between a sample and an identification plane in learning data. In addition, since learning of the discriminant function is a quadratic programming problem, global optimality of the solution is guaranteed, and there is an advantage that there is no problem of convergence to a local solution unlike a neural network.

  On the other hand, the SVM has a problem that the processing time required for identification increases in proportion to the number of support vectors acquired by learning. Therefore, a technique has been devised for reducing the calculation cost at the time of identifying the SVM. For example, an approach has been devised in which the discrimination function is approximated using a smaller number of reference vectors after learning, so that the deterioration of the discrimination performance is suppressed as much as possible and the processing time is reduced.

  Non-Patent Document 1 proposes a method based on “integration” in which two support vectors are integrated and replaced with a new support vector. Non-Patent Document 2 proposes a method of performing “deletion”, “projection”, and “integration” of a support vector for an SVM of a multi-class classification.

  The “deletion” of the support vector may significantly change the discriminant function when there is no support vector with a small weight, and there is a problem in stably maintaining the discrimination accuracy. In “projection”, processing time is required for approximating the identification function because all weights of the remaining support vectors are updated. Further, when the support vectors are separated from each other in the feature space, there is a limit in absorbing a change in the discrimination function due to the projection. Since “integration” changes only two support vectors, the calculation cost is lower than that of “projection”, and the possibility that the discriminant function is significantly deteriorated is low. It can be said that the approach of "integration" has a good balance from the viewpoint of reducing the processing time and maintaining the identification accuracy.

  The “integration” approach in the prior art uses an evaluation criterion of minimizing the amount of change of a weight vector in a high-dimensional feature space. However, this method has a problem that the distribution of the input vector is not taken into consideration and the approximation accuracy is likely to deteriorate.

  The present invention has been made in view of the above, and has as its object to improve the approximation accuracy of a discriminant function.

In order to solve the above-described problem and achieve the object, the present invention is an information processing apparatus that processes an identification function represented by a sum of values of a kernel function having a plurality of reference vectors and input vectors as arguments. An acquisition unit that acquires the plurality of reference vectors; a generation unit that integrates a first reference vector and a second reference vector included in the plurality of reference vectors to generate an integrated vector; An evaluation value calculation unit that calculates an evaluation value representing an expected value of an approximation error based on a probability distribution of the input vector between an approximation function obtained by replacing a second reference vector with the integrated vector and the identification function; An output control unit that outputs the first reference vector and the second reference vector that minimize the evaluation value.

  According to the present invention, there is an effect that the approximation accuracy of the identification function can be improved.

FIG. 1 is a block diagram illustrating a configuration of the information processing apparatus according to the first embodiment. FIG. 2 is a flowchart illustrating an example of the integration processing according to the first embodiment. FIG. 3 is a block diagram illustrating a configuration of the information processing apparatus according to the second embodiment. FIG. 4 is a block diagram illustrating a configuration of an information processing apparatus according to the third embodiment. FIG. 5 is a block diagram illustrating a configuration of an information processing apparatus according to the fourth embodiment. FIG. 6 is a flowchart illustrating an example of the integration processing according to the fourth embodiment. FIG. 7 is a block diagram illustrating a configuration of an information processing apparatus according to the fifth embodiment. FIG. 8 is a flowchart illustrating an example of the identification processing according to the fifth embodiment. FIG. 9 is an explanatory diagram illustrating a hardware configuration example of the information processing apparatus according to each embodiment.

  Exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

  The following embodiments relate to an information processing apparatus that processes information indicating a discriminant function by the sum (linear sum) of a plurality of reference vectors represented by SVM and a kernel function with an input vector as an argument. The information processing apparatus according to each embodiment integrates reference vectors so as to minimize an expected approximation error of an identification function based on a probability distribution of an input vector. As described above, the processing speed can be improved by configuring and approximating the identification function with a smaller number of reference vectors. The information processing apparatus according to each of the embodiments includes an identification apparatus (an application that identifies a pattern while learning in real time, such as object detection from an image such as a moving image, object tracking using the image, character recognition, and voice recognition. ).

(First embodiment)
In the first embodiment, an example will be described in which an SVM identification function is targeted. In this case, the support vector becomes the reference vector. In this embodiment, an example in which a general kernel function is used will be described. The applicable identification method is not limited to SVM.

First, the principle will be described. When learning is performed by the SVM of the two-class classification, M (M is a positive integer) support vectors s ⁽ⁱ⁾ { ^RD (i = 1, 2,..., M) and corresponding weights α _i } R (i = 1,2, ···, M), and, Motomari offset B∈R, discriminant function f shown in the following equation (1) for the input vector x∈R ^D (x) is obtained. D represents the number of dimensions of the input space and the feature space.

Here, K (,) is a kernel function ( ^RD × ^RD → R). The class to which the input vector x belongs is predicted by the sign of f (x). As shown in the equation (1), the amount of calculation of the identification function f (x) is proportional to the number M of support vectors. If f (x) is approximated by the L (<M) support vectors in which M is reduced as in equation (2), the calculation amount can be reduced.

In the present embodiment, the number of support vectors is reduced by selecting two support vectors and integrating the selected two support vectors into one support vector. Such an integration approach is also used in Non-Patent Documents 1 and 2, for example. In Non-Patent Documents 1 and 2, support vectors are integrated on the basis of “minimizing the amount of change in weight in a feature space in which input vectors are mapped in a high dimension”. Specifically, it is as follows. Let φ be a mapping from the input space corresponding to the kernel K (,) to the feature space. At this time, the expression (1) can be rewritten as the following expression (3).

Here, w is a weight in the high-dimensional feature space, and is represented by the following equation (4).

When the weight expressed by the following equation (5) is defined, the equation (2) can be expressed as the following equation (6).

In Non-Patent Documents 1 and 2, the L2 norm of the difference between these weights is used, and J expressed by the following equation (7) is used as the evaluation function.

On the other hand, from the equations (3) and (6), the approximation error of the discriminant function is expressed by the following equation (8). As can be seen from this equation, even if equation (7) is minimized, the approximation error is not minimized for the distribution of the input vector x. However, the upper bound of the approximation error is suppressed (see Non-Patent Document 2).

For example, if φ (x) is a normal distribution with mean 0 and covariance Σ, the expected value of Expression (8) is expressed by Expression (9) below.

If Σ is not a unit matrix, the minimization of equation (7) is different from the minimization of this expected value. In the present embodiment, in view of this point, the support vectors are integrated so as to minimize the following expression (10), which is the expected value of the approximation error for the distribution p (x) of the input vector x.

The i-th and j-th (j = 1, 2,..., M) support vectors are integrated to generate a new support vector s and weight α. At this time, the evaluation function of the expression (10) can be expressed by the following expression (11). Hereinafter, all the expected values E [] are calculated for p (x).

When J (s, α, i, j) is partially differentiated by α, it can be expressed as the following equation (12).

S and α that minimize J (s, α, i, j) from the extreme value condition satisfy the following expression (13).

Further, when J (s, α, i, j) is partially differentiated with respect to s, it can be expressed as the following equation (14).

S and α that minimize J (s, α, i, j) from the extreme value condition satisfy the following equation (15).

S and α obtained by simultaneously solving equations (13) and (15) minimize J (s, α, i, j) for fixed i, j. However, it is not easy to solve equations (13) and (15) for a general kernel function K (the solution in a special case will be described later). Therefore, the following heuristics are used. First, s is represented by a linear combination of s ⁽ⁱ⁾ and S ^(j) as in the following equation (16).

  Since the support vector in the high-dimensional feature space is such a linear combination, it is assumed that a similar relationship is approximately established in the input space. The approximation accuracy depends on the kernel function. Α is eliminated from the expressions (13) and (15), and u that minimizes the square error of the conditional expression is obtained by a one-dimensional search. From u obtained in this manner, s is obtained from Expression (16), and α is obtained from Expression (13).

  Depending on the kernel function, the expected value in the expressions (13) and (15) may not be obtained analytically. In this case, the expected value can be approximately obtained by sampling a plurality of x from p (x) using a Monte Carlo method (for example, a Markov chain Monte Carlo method).

  The remaining problem is the selection of the support vectors to be integrated, i.e., i, j. Simply, all combinations are tried for support vectors belonging to the same class, and i, j that minimizes J (s, α, i, j) can be obtained. If the number of support vectors is extremely large, such a full search takes time. For this reason, minimization may be performed by simulated annealing, tabu search, and a meta-heuristic algorithm represented by a genetic algorithm. As a result, it is possible to avoid searching for all sets of reference vectors, and to reduce the calculation cost for approximating the discriminant function.

  The details of the present embodiment based on the above principle will be described below. FIG. 1 is a block diagram illustrating a configuration of the information processing apparatus 100 according to the first embodiment.

  As illustrated in FIG. 1, the information processing apparatus 100 according to the present embodiment includes a storage unit 20, an acquisition unit 101, a selection unit 102, a generation unit 103, a weight calculation unit 104, an evaluation value calculation unit 105, And an output control unit 106.

  The storage unit 20 stores various information used for processing. For example, the storage unit 20 stores information indicating a discriminant function such as a support vector, a weight, and an offset obtained by learning. The storage unit 20 may be provided outside the information processing apparatus 100. The storage unit 20 can be configured by any generally used storage medium such as a hard disk drive (HDD), an optical disk, a memory card, and a random access memory (RAM).

  The acquisition unit 101 acquires information to be processed. For example, the acquisition unit 101 acquires information such as a support vector and a weight by reading from the storage unit 20.

  The selection unit 102 selects a support vector to be processed from the acquired support vectors. For example, the selection unit 102 obtains an i-th support vector (first reference vector) and a j-th support vector (second reference vector) belonging to the same class among the obtained support vectors.

  The generation unit 103 generates an integrated vector obtained by integrating a plurality of selected support vectors. For example, the generation unit 103 generates an integrated vector s obtained by integrating the i-th support vector and the j-th support vector according to the above-described principle.

  The weight calculator 104 calculates the weight of the generated integrated vector s. For example, the weight calculator 104 calculates the weight α of the integrated vector s according to the above principle.

The evaluation value calculation unit 105 calculates an approximation error between an identification function, which is an identification function obtained by replacing a plurality of support vectors (i-th support vector, j-th support vector) before integration with an integration vector, and an identification function. Is calculated. For example, the evaluation value calculation unit 105 replaces s ⁽ⁱ⁾ and s ^(j) with s, and replaces α _i and α _j with α, the value of the evaluation function (expression (11)) (evaluation function value). Is obtained as an evaluation value.

  The output control unit 106 outputs various information such as a result of information processing. For example, the output control unit 106 outputs a first reference vector and a second reference vector having the smallest evaluation value (evaluation function value).

  The acquisition unit 101, the selection unit 102, the generation unit 103, the weight calculation unit 104, the evaluation value calculation unit 105, and the output control unit 106 allow a processing device such as a CPU (Central Processing Unit) to execute a program. That is, it may be realized by software, by hardware such as an IC (Integrated Circuit), or by using both software and hardware.

  Next, a support vector integration process performed by the information processing apparatus 100 according to the first embodiment thus configured will be described with reference to FIG. FIG. 2 is a flowchart illustrating an example of the integration processing according to the first embodiment.

The acquisition unit 101 determines the support vector s ⁽ⁱ⁾ (i = 1, 2,..., M) obtained from the storage unit 20 by learning the SVM and the corresponding weight α _i (i = 1, 2,. .., M) are acquired (step S101).

The selecting unit 102 selects the i-th and j-th support vectors belonging to the same class from the acquired support vectors (step S102). The selection unit 102 starts with, for example, i = 1 and j = 1, increments i and j in order, skips when s ⁽ⁱ⁾ and s ^(j) belong to different classes, and selects the next combination. .

The generation unit 103 integrates the selected support vectors s ⁽ⁱ⁾ and s ^(j) to generate an integrated vector s (step S103). The weight calculator 104 calculates the weight α of the generated integrated vector s (step S104).

The evaluation value calculation unit 105 calculates an evaluation function value that is a value of an evaluation function (Equation (11)) when s ⁽ⁱ⁾ and s ^(j) are replaced with s, and α _i and α _j are replaced with α. It is determined (step S105). When the obtained evaluation function value is smaller than the minimum value of the evaluation function values obtained before that, the evaluation value calculation unit 105 determines the indexes i, j, s, α of the support vectors to be integrated, and the evaluation function value. Is stored in the storage unit 20, for example.

  The selection unit 102 makes an end determination (step S106). For example, the selection unit 102 determines that the process has been completed if all the sets of support vectors belonging to the same class have been selected. If the processing has not been completed (Step S106: No), the process returns to Step S102 and the processing is repeated.

  When the processing is completed (Step S106: Yes), the output control unit 106 outputs the support vector indexes i, j, s, and α that minimize the evaluation function value (Step S107).

  Through the above processing, integration of the support vectors that minimizes the expected value of the approximation error based on the distribution of the input vectors is obtained. As a result, the processing time required for identification can be reduced while maintaining the approximation accuracy of the identification function as much as possible.

  In the above processing, two support vectors are integrated into one, but by repeating this multiple times, the number of support vectors can be reduced to an arbitrary number. At the time of repetition, the integrated support vector may be included in the integration target, or a non-integrated support vector may be included in the integration target. When the integration is repeated, the evaluation function value for each pair of support vectors may be stored at the first iteration. This eliminates the need to calculate the evaluation function value in the second and subsequent iterations, and can speed up the processing. In the case of reducing the number of support vectors by B, it is sufficient to sequentially select, from the set of support vectors that have not been selected until then, the number of support vectors having the smallest evaluation function value.

(Modification)
A meta-heuristic algorithm (simulated annealing, tabu search, genetic algorithm) may be used to optimize the support vectors to be integrated. In this case, in the first iteration in step S102, the selection unit 102 randomly selects i and j as initial values. In the second and subsequent iterations, the selection unit 102 selects i and j based on the metaheuristic algorithm using the identification function values obtained in the previous iteration. In such a modified example, since it is not necessary to obtain the evaluation function values for all combinations of i and j, the processing of support vector integration can be speeded up.

Further, the repetition in step S106 may not be performed, and a heuristic criterion may be used when selecting i and j in step S102. When α _i and α _j are small and the distance between s ⁽ⁱ⁾ and s ^(j) is small, the evaluation function value (equation (11)) tends to be small. Therefore, the selecting unit 102 determines that s ⁽ⁱ⁾ , s ^(j) (K (s ⁽ⁱ⁾ , s ^(j) )) in which α _i and α _j are equal to or less than the specific threshold value and the distance is minimum are maximum. S ⁽ⁱ⁾ and s ^(j) ). For example, when the number of support vectors is reduced by B, the selection unit 102 selects B non-overlapping s ⁽ⁱ⁾ and s ^(j) from a set in which α _i and α _j are equal to or less than a specific threshold and the distance is small. do it.

When generating the integrated vector in step S103, u may be fixed at a weight ratio as shown in the following equation (17) without performing one-dimensional search for u.

  Although the approximation accuracy of the discriminant function may be degraded by simplifying the calculation in this way, the processing time required for integrating the support vectors can be reduced. For example, when online learning is performed for each frame of a moving image, the processing time for integrating support vectors also contributes to real-time performance. Therefore, such high speed is significant.

  Although the example of the two-class classification has been described above, the present invention can be easily extended to the multi-class classification. In the case of multi-class classification, as many discriminating functions as the number of classes are prepared, and approximation of each discriminant function may be performed in the same manner.

(Second embodiment)
In the second embodiment, a case where an RBF (Radial Basis Function) kernel is used as a kernel function will be described. The RBF kernel is defined by the following equation (18). γ is a parameter.

When the RBF kernel is used, the expected value E [K (a, x) K (b, x)] is expressed by the following equation (19).

Assuming a uniform distribution as a distribution p of the input vector x∈R ^D (x), below it is established (20). In the present embodiment, the expected value E [f (x)] based on the uniform distribution is defined as the cumulative value (integral value) of f (x) over the entire area.

Therefore, E [K (a, x) K (b, x)] can be expressed by the following equation (21).

Further, assuming a normal distribution of mean μ and covariance と し て as distribution p (x) of input vector x, the following equation (22) is established.

Therefore, E [K (a, x) K (b, x)] can be expressed by the following equation (23).

In any case, E [K (a, x) K (b, x)] can be expressed as in the following equation (24) using a constant C independent of a and b. However, equation (24) does not hold for any p (x).

Using this result, equation (13) can be rewritten as equation (25) below.

Also, the following equation (26) holds.

Assuming a uniform distribution for the distribution p (x) of the input vector x, Equation (26) can be expressed as Equation (27) below. C ′ is a constant independent of a and b.

Using this result, equation (15) can be rewritten as equation (28) below.

Here, u is represented by the following equation (29).

Using this u, the expression (28) is expressed by the following expression (30). In the first embodiment, s is artificially represented by the linear combination of s ⁽ⁱ⁾ and s ^(j) in Expression (16). However, the RBF kernel naturally provides a linear combination.

By substituting equation (30) into equation (29) and eliminating s, the following equation (31) is obtained.

  Assuming that the right side of equation (31) is g (u), finding u is a problem of finding a fixed point of the function g (u). If a solution exists in the region of g ′ (u) <1 (eg, g ′ (u) <1 in the closed section [0 1]), u ← g starts from an appropriate initial value (eg, equation (17)). A solution can be obtained by repeating (u) (substitution of g (u) into u). If there is no solution in the region where g '(u) <1, it can be solved by Newton's method or by one-dimensional search.

When u is found, s is found from equation (30), and α is found from equation (25). The evaluation function value at this time is expressed by the following equation (32).

  The index i, j of the support vector that minimizes this evaluation function value is obtained.

  The details of the present embodiment based on the above principle will be described below. FIG. 3 is a block diagram illustrating a configuration of an information processing apparatus 200 according to the second embodiment.

  As illustrated in FIG. 3, the information processing device 200 according to the present embodiment includes a storage unit 20, an acquisition unit 101, a selection unit 102, a generation unit 203, a weight calculation unit 204, an evaluation value calculation unit 205, And an output control unit 106.

  In the second embodiment, the functions of the generation unit 203, the weight calculation unit 204, and the evaluation value calculation unit 205 are different from those of the first embodiment. Other configurations and functions are the same as those in FIG. 1 which is a block diagram of the information processing apparatus 100 according to the first embodiment. Further, the overall flow of the integration processing of the second embodiment is the same as that of FIG. 2 showing the integration processing of the first embodiment, and thus the description is omitted. In the following, differences from the first embodiment will be described.

As processing corresponding to step S103 in FIG. 2, the generation unit 203 integrates the selected support vectors s ⁽ⁱ⁾ and s ^(j) to generate an integrated vector s. The generation unit 103 according to the first embodiment generates the integrated vector s using, for example, Expressions (13), (15), and (16). The generation unit 203 of the present embodiment generates the integrated vector s using, for example, Expressions (30) and (31).

  As processing corresponding to step S104 in FIG. 2, the weight calculation unit 204 calculates the weight α of the generated integrated vector s. The weight calculation unit 204 calculates the weight α using, for example, Equation (25).

As a process corresponding to step S105 in FIG. 2, the evaluation value calculation unit 205 replaces s ⁽ⁱ⁾ and s ^(j) with s, and replaces α _i and α _j with α. The evaluation function value which is the value of the expression is calculated. When the obtained evaluation function value is smaller than the minimum value of the evaluation function values obtained before that, the evaluation value calculation unit 205 determines the indexes i, j, s, α of the support vectors to be integrated, and the evaluation function value. Is stored in the storage unit 20, for example.

  According to the present embodiment, even when the RBF kernel is used as the kernel function, integration of the support vectors that minimizes the expected value of the approximation error based on the distribution of the input vector can be obtained. As a result, the processing time required for identification can be reduced while maintaining the approximation accuracy of the identification function as much as possible. By using a uniform distribution or a normal distribution as the probability distribution of the input vector, it is possible to approximate the discriminant function while reflecting the uniformity of the input vector or the bias of the input vector.

(Third embodiment)
In the third embodiment, a case will be described in which an exponential Bhattacharyya kernel is used for a kernel function. The exponent Bhattacharyya kernel is defined by the following equation (33). β and γ are parameters. Note that the input vector x is a real number in which each element is 0 or more, and is normalized so that the sum is 1. The exponential Bhattacharyya kernel is particularly effective when dealing with histogram features.

ここで、入力ベクトルｘの分布として領域［０ １］^Ｄ内での一様分布を考える（厳密には入力ベクトルｘはＤ次元単体上に存在するが近似をする）。このとき、以下の（３５）式が成り立つ。

When the exponential Bhattacharyya kernel is used, the expected value E [K (a, x) K (b, x)] is expressed by the following equation (34).

Here, a uniform distribution in the region [0 1] ^D is considered as the distribution of the input vector x (strictly speaking, the input vector x exists on a D-dimensional simple substance but is approximated). At this time, the following equation (35) holds.

Here, if exp is expanded to a second-order term by Taylor expansion, the following equation (36) is obtained.

From the equations (34) and (36), the following equation (37) is obtained.

Using this equation (37), equation (13) can be rewritten as equation (38) below.

Also, the following equation (39) holds.

Here, considering the uniform distribution in the region [0 1] ^D as the distribution of the input vector x, and performing the Taylor expansion of exp to a quadratic term, the following equation (40) is obtained.

Using this result, equation (15) can be rewritten as equation (41) below. Here, t = 1, 2,..., D.

Although s and α can be obtained from the constraint equations (38) and (41), it is not easy to solve these equations. Therefore, the same approximate solution as in the first embodiment is used. First, s is represented by a linear combination of s ⁽ⁱ⁾ and S ^(j) as in the following equation (42).

The equation (38) is substituted into the equation (41) to eliminate α, and u that minimizes the square error of the conditional expression is obtained by a one-dimensional search. From u obtained in this manner, s is obtained from equation (41), and α is obtained from equation (38). The evaluation function value at this time is expressed by the following equation (43).

  Details of the present embodiment based on the above principle will be described below. FIG. 4 is a block diagram illustrating a configuration of an information processing device 300 according to the third embodiment.

  As illustrated in FIG. 4, the information processing apparatus 300 of the present embodiment includes a storage unit 20, an acquisition unit 101, a selection unit 102, a generation unit 303, a weight calculation unit 304, an evaluation value calculation unit 305, And an output control unit 106.

  In the third embodiment, the functions of the generation unit 303, the weight calculation unit 304, and the evaluation value calculation unit 305 are different from those of the first embodiment. Other configurations and functions are the same as those in FIG. 1 which is a block diagram of the information processing apparatus 100 according to the first embodiment. The overall flow of the integration process according to the third embodiment is the same as that in FIG. 2 showing the integration process according to the first embodiment, and thus the description is omitted. In the following, differences from the first embodiment will be described.

As processing corresponding to step S103 in FIG. 2, the generation unit 303 integrates the selected support vectors s ⁽ⁱ⁾ and s ^(j) to generate an integrated vector s. The generation unit 303 of the present embodiment generates the integrated vector s using, for example, Expressions (42) and (31).

  As processing corresponding to step S104 in FIG. 2, the weight calculation unit 304 calculates the weight α of the generated integrated vector s. The weight calculation unit 304 calculates the weight α using, for example, Expression (38).

As a process corresponding to step S105 in FIG. 2, the evaluation value calculation unit 305 replaces s ⁽ⁱ⁾ and s ^(j) with s and replaces α _i and α _j with α. The evaluation function value which is the value of the expression is calculated. When the obtained evaluation function value is smaller than the minimum value of the evaluation function values obtained before that, the evaluation value calculation unit 305 determines the indexes i, j, s, α of the support vectors to be integrated, and the evaluation function value. Is stored in the storage unit 20, for example.

  According to the present embodiment, even when the exponential Bhattacharyya kernel is used for the kernel function, integration of support vectors that minimizes the expected value of the approximation error based on the distribution of the input vector can be obtained. As a result, the processing time required for identification can be reduced while maintaining the approximation accuracy of the identification function as much as possible.

(Fourth embodiment)
In the fourth embodiment, an example will be described in which the distribution of input vectors is estimated from learning data, and the support vectors are integrated based on the estimated distribution so as to minimize the approximation error of the discriminant function. FIG. 5 is a block diagram illustrating a configuration of an information processing device 400 according to the fourth embodiment.

  As shown in FIG. 5, the information processing apparatus 400 according to the present embodiment includes a storage unit 20, an estimation unit 401, an acquisition unit 101, a selection unit 102, a generation unit 103, a weight calculation unit 104, an evaluation value A calculation unit 105 and an output control unit 106 are provided.

  The fourth embodiment differs from the first embodiment in that an estimation unit 401 is added. Other configurations and functions are the same as those in FIG. 1 which is a block diagram of the information processing apparatus 100 according to the first embodiment.

The estimating unit 401 determines an input vector distribution p (x) based on N input vectors s ⁽ⁱ⁾ (i = 1, 2,..., N) included in the learning data stored in the storage unit 20 or the like. Is estimated. The details of the estimation process of the input vector distribution by the estimation unit 401 will be described later.

  Next, a support vector integration process performed by the information processing apparatus 400 according to the fourth embodiment thus configured will be described with reference to FIG. FIG. 6 is a flowchart illustrating an example of the integration processing according to the fourth embodiment.

The estimation unit 401 reads the learning data from the storage unit 20, and based on N input vectors s ⁽ⁱ⁾ (i = 1, 2,..., N) included in the learning data, an input vector distribution p (x ) Is estimated (step S201).

  Steps S202 to S208 are the same as steps S101 to S107 in the information processing apparatus 100 according to the first embodiment, and a description thereof will be omitted.

Next, the process of estimating the input vector distribution will be described. The estimating unit 401 can use, for example, an empirical distribution represented by the following equation (44). Here, δ is a delta function.

  The method of estimating the input vector distribution p (x) is not limited to this. For example, the estimating unit 401 may assume a parametric model for p (x) and obtain its parameters by maximum likelihood estimation. For example, a normal distribution, a Laplace distribution, a uniform distribution (region limited), a Poisson distribution, a beta distribution, a gamma distribution, and a mixed distribution thereof can be considered for p (x). The estimating unit 401 may further estimate the distribution by MAP estimation and Bayes estimation, assuming a prior distribution as a parameter.

As another approach, the estimation unit 401 may estimate an input vector distribution based on a discriminant function f (x) obtained from learning data. For example, the estimating unit 401 sets p (x) as in the following equation (45).

  Strictly speaking, the distribution of Expression (45) is not an estimated distribution of the input vector but a distribution for the purpose of weighting the approximation error. The purpose of the SVM is class classification, and the approximation accuracy of the identification function near the identification boundary is important. Therefore, it is considered that the input vector is concentrated near the discrimination boundary where f (x) = 0 as in the equation (45), and the expected value of the approximation error is taken to improve the approximation accuracy near the discrimination boundary. Can be. The calculation of the expected value of the approximation error based on the equation (45) can be performed by, for example, the Markov chain Monte Carlo method.

  The distribution of Expression (45) is an example of a distribution in which the value of the probability density decreases as the distance from the identification boundary increases. By using such a probability distribution, it is possible to approximate the discriminant function with emphasis on the vicinity of the discriminating boundary that greatly affects the discriminating accuracy.

  The present embodiment can be applied not only to a general kernel, but also to the RBF kernel of the second embodiment and the exponential Bhattacharya kernel of the third embodiment.

  According to the present embodiment, the estimation accuracy of the identification function can be improved by estimating the distribution of the input vector based on the learning data and minimizing the expected value of the approximation error of the identification function based on the distribution. . Further, the identification function can be approximated while reflecting the distribution of the learning data.

(Fifth embodiment)
In the present embodiment, an example will be described in which an information processing apparatus is realized as an identification apparatus that performs pattern recognition and the like using an identification function obtained by integrating reference vectors. FIG. 7 is a block diagram illustrating a configuration of an information processing device 500 according to the fifth embodiment.

  As illustrated in FIG. 7, the information processing apparatus 500 according to the present embodiment includes a storage unit 20, an acquisition unit 101, a selection unit 102, a generation unit 103, a weight calculation unit 104, an evaluation value calculation unit 105, An output control unit 106 and a recognition unit 501 are provided.

  The fifth embodiment is different from the first embodiment in that a recognition unit 501 is added. Other configurations and functions are the same as those in FIG. 1 which is a block diagram of the information processing apparatus 100 according to the first embodiment.

  The recognizing unit 501 executes a recognizing (recognizing) process using the SVM whose discriminant function is approximated by the method of the first embodiment. For example, the recognizing unit 501 uses SVM to identify to which class the feature vector to be identified belongs. The feature amount vector is, for example, a vector representing a feature of an image, a feature of a character, and a feature of a sound. As described above, the identification device of the present embodiment can be applied to any identification processing such as image recognition, character recognition, and voice recognition.

  Next, an identification process performed by the information processing apparatus 500 according to the fifth embodiment thus configured will be described with reference to FIG. FIG. 8 is a flowchart illustrating an example of the identification processing according to the fifth embodiment.

  The recognition unit 501 inputs a feature amount vector to be identified from the storage unit 20 or an external device (step S301). The recognizing unit 501 identifies to which class the feature vector belongs using the SVM (step S302). The recognition unit 501 outputs the identification result (Step S303), and ends the identification processing.

  This embodiment can be applied not only to the first embodiment but also to the second to fourth embodiments. According to the present embodiment, since an identification function approximated by using a smaller number of reference vectors can be used, it is possible to reduce processing time while minimizing deterioration of identification performance.

  Next, a hardware configuration of the information processing apparatus according to each of the above embodiments will be described with reference to FIG. FIG. 9 is an explanatory diagram illustrating a hardware configuration example of the information processing apparatus according to each embodiment.

  The information processing apparatus according to each embodiment communicates with a control device such as a CPU (Central Processing Unit) 51 and a storage device such as a ROM (Read Only Memory) 52 and a RAM (Random Access Memory) 53 by connecting to a network. And a bus 61 for connecting each unit.

  The program executed by the information processing apparatus according to the present embodiment is provided by being incorporated in a ROM or the like in advance.

  The program executed by the information processing apparatus according to the present embodiment is a file in an installable format or an executable format in a computer such as a CD-ROM, a flexible disk (FD), a CD-R, and a DVD (Digital Versatile Disk). You may comprise so that it may be recorded on a readable recording medium and provided as a computer program product.

  Furthermore, the program executed by the information processing apparatus of the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Further, the program executed by the information processing apparatus of the present embodiment may be provided or distributed via a network such as the Internet.

  The program executed by the information processing apparatus according to the present embodiment has a module configuration including the above-described units. As actual hardware, the CPU (processor) reads out the program from the ROM and executes the program. Is loaded on the main storage device, and each unit is generated on the main storage device.

Reference Signs List 20 storage unit 100, 200, 300, 400, 500 information processing device 101 acquisition unit 102 selection unit 103, 203, 303 generation unit 104, 204, 304 weight calculation unit 105, 205, 305 evaluation value calculation unit 106 output control unit 401 Estimation unit 501 Recognition unit

D. Nguyen, T. Ho, “An efficient method for simplifying support vector machines”, ICML, 617-624, 2005. Z. Wang, K. Crammer, S. Vucetic, "Multi-Class Pegasos ON a Budget", In Proc. ICML, 2010.

Claims (9)

An information processing apparatus for processing an identification function represented by a sum of values of a kernel function having a plurality of reference vectors and an input vector as arguments,
An acquisition unit that acquires a plurality of the reference vectors,
A generation unit that generates an integrated vector obtained by integrating a first reference vector and a second reference vector included in the plurality of reference vectors;
Calculating an evaluation value representing an expected value of an approximation error based on a probability distribution of the input vector between an approximation function obtained by replacing the first reference vector and the second reference vector with the integrated vector and the discrimination function. An evaluation value calculation unit to perform
An output control unit that outputs the first reference vector and the second reference vector that minimize the evaluation value;
An information processing apparatus comprising: The reference vector is a support vector obtained by learning a support vector machine,
The information processing device according to claim 1. The probability distribution of the input vector is a uniform distribution or a normal distribution,
The information processing device according to claim 1. The probability distribution of the input vector is estimated from learning data,
The information processing device according to claim 1. The probability distribution of the input vector is a distribution in which the value of the probability density decreases as the distance from the identification boundary increases.
The information processing device according to claim 1. The output control unit selects and selects the first reference vector and the second reference vector with the minimum evaluation value by a simulated annealing, a taboo search, and a meta-heuristic algorithm including a genetic algorithm. Outputting the first reference vector and the second reference vector;
The information processing device according to claim 1. The acquisition unit acquires one or more identification functions for identifying one or more classes, respectively.
The information processing device according to claim 1. An information processing method executed by an information processing apparatus that processes an identification function represented by a sum of values of a kernel function having a plurality of reference vectors and input vectors as arguments,
An obtaining step of obtaining a plurality of the reference vectors;
A generation step of generating an integrated vector obtained by integrating a first reference vector and a second reference vector included in the plurality of reference vectors;
Calculating an evaluation value representing an expected value of an approximation error based on a probability distribution of the input vector between an approximation function obtained by replacing the first reference vector and the second reference vector with the integrated vector and the discrimination function. Evaluation value calculating step to be performed;
An output control step of outputting the first reference vector and the second reference vector that minimize the evaluation value;
An information processing method comprising: An information processing apparatus that processes an identification function represented by a sum of values of a kernel function having a plurality of reference vectors and input vectors as arguments,
An acquisition unit that acquires a plurality of the reference vectors,
A generation unit that generates an integrated vector obtained by integrating a first reference vector and a second reference vector included in the plurality of reference vectors;
Calculating an evaluation value representing an expected value of an approximation error based on a probability distribution of the input vector between an approximation function obtained by replacing the first reference vector and the second reference vector with the integrated vector and the discrimination function. An evaluation value calculation unit to perform
An output control unit that outputs the first reference vector and the second reference vector that minimize the evaluation value;
Program to function as

Images