JP6402607B2 - Information processing apparatus, information processing method, and information processing program - Google Patents

Description

  The present invention relates to a technique for learning a model using a learning device.

  In a certain document, for each repeated round, learn multiple weak hypotheses that update the weights for each learning case and output a binary classification result, and learn a model that combines them (combined model) A boosting technique is disclosed. According to this technique, the certainty factor for the weak hypothesis learned in each round is calculated. The final model is expressed in a combined form based on the certainty of each weak hypothesis.

  In this way, when the weak hypothesis is realized using a classifier that outputs a classification result expressed in binary, since the certainty factor can be calculated analytically, the processing load related to the certainty factor calculation Is relatively small.

Quinlan, J.R .: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993) Rosenblatt, F .: The perceptron: A probabilistic model for information storage and organization in the brain. 65 (6) (1958) 386-408 Freund, Y., Schapire, R.E .: A decision-theoretic generalization of on-line learning and an application to boosting.Journal of computer and system sciences 55 (1) (1997) Schapire, R.E., Singer, Y .: Improved boosting algorithms using confidence-rated predictions. Machine Learning 37 (3) (1999) 297-336

  In one aspect, an object of the present invention is to reduce a processing load for calculating a certainty factor of a hypothesis model that outputs a classification result including a plurality of values.

  An information processing apparatus according to an aspect is based on (A) a plurality of sets including a learning case and a label indicating one of predetermined two values for the learning case, and a coefficient value for each learning case. Based on the learning unit that learns the hypothesis model, and (B) (b1) the hypothesis model and each learning case, a classification result indicating the tendency corresponding to one of the above two values and the degree of the tendency for each learning case (B2) Among the learning cases, the tendency in the classification result identifies the learning case corresponding to the corresponding label, and for each identified learning case, the product of the corresponding coefficient value and the above degree in the classification result (B3) Among the learning cases, the above-mentioned tendency in the classification result identifies a learning case that does not correspond to the corresponding label, and for each identified learning case, the corresponding relation The product of the value and the above-mentioned degree in the classification result is calculated, and the second sum of the product is calculated. And a calculation unit that calculates the certainty factor of the hypothesis model by dividing by a value that is twice the maximum value.

  As one aspect, it is possible to reduce the processing load for calculating the certainty factor of a hypothesis model that outputs a classification result including a plurality of values.

FIG. 1 is a diagram showing a main processing flow of AdaBoost. FIG. 2 is a diagram illustrating functional blocks of the model learning device. FIG. 3 is a diagram illustrating a configuration of the model learning unit. FIG. 4 is a diagram illustrating functional blocks of the model learning device and the model application device. FIG. 5 is a diagram showing a model learning process flow in the present embodiment. FIG. 6 is a diagram illustrating an example of learning case data. FIG. 7 is a diagram illustrating an example of label data. FIG. 8 is a diagram illustrating an example of the coupled model data in the initial state. FIG. 9 is a diagram showing weight data in the first round. FIG. 10 is a diagram showing an overview of weak learning processing in the first round. FIG. 11 is a diagram illustrating a certainty factor calculation processing flow. FIG. 12 is a diagram showing an outline of classification based on the first weak hypothesis. FIG. 13 is a diagram showing an outline of the certainty factor calculation in the first round. FIG. 14 is a diagram showing an outline of the update of the combined model in the first round. FIG. 15 is a diagram showing a weight update processing flow. FIG. 16 is a diagram showing an outline of weight update in the first round. FIG. 17 is a diagram showing an outline of weak learning processing in the second round. FIG. 18 is a diagram showing an outline of classification based on the second weak hypothesis. FIG. 19 is a diagram showing an outline of certainty calculation in the second round. FIG. 20 is a diagram illustrating an outline of the update of the combined model in the second round. FIG. 21 is a diagram showing a model application processing flow. FIG. 22 is a diagram illustrating an example of model application using the combined model data in the first round. FIG. 23 is a diagram illustrating an example of model application using the combined model data in the second round. FIG. 24 is a diagram illustrating functional blocks of a computer.

  First, AdaBoost (Non-Patent Documents 1 and 2) will be described as an example of a boosting technique. The purpose of learning is to derive a mapping F: X-> Y from a given learning case X to a label set Y. In this example, it is assumed that the label indicates one of binary values of {−1, +1}.

FIG. 1 shows a main processing flow of AdaBoost. The learning system receives the learning data S and the number of repetitions T for boosting (S1). The learning data S includes m sets of learning examples and labels, and {(x ₁ , y ₁ ),. . . (X _m , y _m )}.

x _i εX represents a learning case related to the i-th set, and y _i εY represents a label related to the i-th set. The number of repetitions T is the number of times boosting is repeated.

The learning system initializes the weights (w _1,1 ,... W _{1, m} ) of the learning cases (S3). The learning case weight w _{1, i} is the weight of the learning case x _i . An initial value 1 / m is set to the weight w _{1, i} (1 ≦ i ≦ m) of each learning case. That is, the initial weight of each learning case is assumed to be equal.

  Then, 1 is set to the counter t (S5).

Learning system causes seeking weak hypothesis h _t the weak learners (S7). Weak learner learns the weak hypothesis h _t using the weight of the learning data S and training example described above _{(w 1,1, ... w 1,} m). For example, a classifier such as a decision tree learner (Non-Patent Document 1) or a perceptron (Non-Patent Document 2) is used as a weak learner. h _t represents the weak hypothesis obtained in the t-th round.

Next, the learning system calculates a certainty factor α _t for the obtained weak hypothesis h _t (S9).

Further, the learning system updates the weight of the learning case according to the following formula (S11).

そして、（２）式は、以下のような正規化を行うための係数である。

H _t included in the formula (1) (x _i) is the classification result by _{h t} for training example x _i, e is Napier's constant. Further, the denominator of the formula (1) is expressed as follows.

Equation (2) is a coefficient for performing normalization as follows.

  The learning system adds 1 to the counter t (S13), and determines whether the counter t has exceeded the number of repetitions T (S15). If it is determined that the counter t does not exceed the number of repetitions T, the learning system returns to S7 and repeats the series of processes described above.

When it is determined that the counter t has exceeded the number of repetitions T, the learning system obtains a final hypothesis F (S17). The learning system obtains the final hypothesis F by combining the T types of weak hypotheses obtained by the loop processing described above according to the following equation.

  Sign in the expression is a function that returns +1 if the input value is positive, and -1 otherwise.

  FIG. 1 shows an example of a procedure for calculating a final hypothesis F that integrates weak hypotheses after completing the iterative processing. However, as will be described later with reference to FIG. The final hypothesis F may be obtained by integrating the model.

[[π]] は、ある命題πが成り立つ場合に１となり、ある命題πが成り立たない場合に０となることを意味している。 Note that AdaBoost has proved that there is an upper limit on the number of learning errors in the final hypothesis F consisting of T types of weak hypotheses, as shown in the following equation.

[[π]] means 1 when a proposition π holds, and 0 when a proposition π does not hold.

As long obtained satisfies confidence alpha _t the following equation with respect to the weak hypothesis h _t in each round t, that learning by AdaBoost converges, guided by the formula (5).
Z _t (α _t ) <1 (6)

However, even if there is a round in which the coefficient Z _t (α _t ) = 1, the round does not affect the above-described upper limit value.

Next, a method for calculating the certainty factor α _t will be described. First, a method for calculating the certainty factor α _t when a classifier (for example, a decision tree learner) that outputs a binary classification result is used as a weak learner will be described.

According to Non-Patent Document 4, when a classifier that outputs a binary classification result is used as a weak learner, the certainty factor α _t is analytically calculated.

The differentiation of Z _t (α _t ) shown in Equation (2) by α _t is obtained as follows.

Then, alpha _t in the case of a zero derivative with alpha _t the Z _{t (α t)} is determined by the following equation.

これは、t 番目の弱仮説によって正しい分類が行われた各学習事例ｘ_Otに対する重みの合計である。

これは、t 番目の弱仮説によって誤った分類が行われた各学習事例ｘ_Ntに対する重みの合計である。 The base of the logarithm is e.

This is the sum of the weights for each learning case x _Ot correctly classified by the t th weak hypothesis.

This is the sum of the weights for each learning case x _Nt that has been incorrectly classified by the t th weak hypothesis.

As shown in the following formula, Ot corresponds to the ID of a learning case that is correctly classified by the t-th weak hypothesis among IDs 1 to m for identifying a learning case.

As shown in the following equation, Nt corresponds to the ID of a learning case that is erroneously classified by the t-th weak hypothesis among IDs 1 to m for identifying a learning case.

Next, a method of calculating the certainty factor α _t when a classifier (for example, a perceptron) that outputs a multilevel classification result is used as a weak learner will be described.

  In the calculation method of Equation (11) in Non-Patent Document 4 described above, it is not assumed that a classifier that outputs a classification result represented by a multivalued real value is used.

When a classifier that outputs a classification result represented by a multi-valued real value is used as a weak learner, it is not appropriate to obtain the certainty factor α _t using the equation (8). This is because if the equation (8) is used, the multivalue is not correctly reflected in the certainty factor α _t .

As a suboptimal method instead of analytical calculation, it is conceivable to obtain the certainty factor α _t by the bisection method. According to the bisection method, a solution is derived by narrowing down a search range corresponding to a section specified by two points. Since the bisection method is a known technique, the calculation of the certainty factor α _t by the bisection method will be simply described.

As shown in Non-Patent Document 4, the second-order differentiation by the certainty factor α _t with respect to Z _t is positive. That is, the graph of Z _t (α _t ) shows a downward convex shape. Dichotomy in this example is equivalent to searching a minimum value of Z _t trial basis.

However, when the certainty factor α _t is obtained by the bisection method, the processing load tends to increase.

In the present embodiment, the certainty factor α _t is approximately calculated in order to reduce the processing load. Hereinafter, an approximate expression for obtaining the certainty factor α _t that satisfies the condition shown in Expression (6) will be described.

First, equation (2) can be converted as follows.

The first term in Equation (11) is the total weight of the learning cases in which the classification result based on the t-th weak hypothesis is 0, as shown in the following equation.

  Ot included in the second term in Equation (11) corresponds to the ID of the learning case that has been correctly classified by the t-th weak hypothesis, as described above using Equation (9). That is, the second term in Equation (11) relates to a learning case in which correct classification is performed by the t-th weak hypothesis.

  In addition, Nt included in the third term in Expression (11) corresponds to the ID of the learning case in which classification is erroneously performed by the t-th weak hypothesis, as described above using Expression (10). That is, the third term in Equation (11) relates to a learning case in which an incorrect classification is performed by the t-th weak hypothesis.

Note that U _t included in the second term and the third term in Equation (11) is the maximum value among the absolute values of the classification results for each learning case based on the t-th weak hypothesis, as shown in the following equation. Means.

And the Bernoulli inequality shown below is applied to the second term and the third term in the equation (11). It is known that if 0 <x, the following inequality holds for any r where 0 <r ≦ 1.
x ^r ≦ r (x−1) +1

Therefore, the following inequality holds for the second term in equation (11).

Similarly, the following inequality holds for the third term in equation (11).

Z ^~ _t (α _t ) ( ^~ means a hat on Z), Z _t (α _t ), where the second and third terms in equation (11) are replaced with the right side of the above inequality. The following formula is established between:
Z _t (α _t ) ≦ Z ^~ _t (α _t ) (12)

Z ^~ _t (α _t ) is expressed by the following equation.

従って、上述したＺ^~ _t（α_t）は、以下のように書き換えられる。

On the other hand, as shown in Expression (3), since the total weight of each learning case based on the t-th weak hypothesis is 1, the following expression is established.

Accordingly, the above-described Z ^~ _t (α _t ) can be rewritten as follows.

Then, the differentiation by α _t with respect ^to Z ^~ _t is expressed by the following equation.

本実施の形態では、この式に従って確信度α_tを算出する。 Thus, alpha _t the time derivative with alpha _t for Z ^~ _{t (α t)} is 0, the alpha _t of words when Z _{t (α t)} is the minimum value, represented by the following equation.

In the present embodiment, the certainty factor α _t is calculated according to this equation.

Note that even if the certainty factor α _t is calculated using this equation, the equation (4) for calculating the final hypothesis does not change.

Next, it is shown that learning by AdaBoost converges if the certainty factor α _t satisfying Expression (13) is obtained in each round t.

By substituting α _t shown in Expression (13) into Z ^to _t , the following expression is obtained.

From this equation, it can be seen that Z ^~ _t (α _t ) ≦ 1. Then, considering equation (12), the following is obtained.
Z _t (α _t ) ≦ Z ^~ _t (α _t ) ≦ 1

Therefore, the certainty factor α _t calculated by the equation (13) satisfies the condition of Z _t (α _t ) <1 shown in the equation (6) or the condition of Z _t (α _t ) = 1. . In other words, since the upper limit value of the learning error is reduced or maintained, the upper limit value of the learning error is not increased at least by boosting.

  Hereinafter, a specific configuration and operation of the apparatus will be described. FIG. 2 shows functional blocks of the model learning device. The model learning device includes a learning data input unit 101 for inputting learning data (including label data and combined model data), and a learning data storage unit for storing learning data input by the learning data input unit 101 103, and a model learning unit 105 that performs model learning using data stored in the learning data storage unit 103.

  Further, the model learning apparatus includes a parameter storage unit 106 that stores parameters used for processing in the model learning unit 105, a weight data storage unit 107 that stores weight data calculated by the model learning unit 105, and a model learning unit 105. A certainty factor storage unit 108 that stores the certainty factor data to be calculated, and a model data storage unit 109 that stores model data (for example, weak hypothesis data and combined model data) that is a processing result of the model learning unit 105 are included.

  Further, the model learning device includes a target data input unit 111 for inputting processing target data for specifying a target to be classified by applying the generated model data, and processing target data input by the target data input unit 111. A target data storage unit 113 for storing the model data, and a model application unit 115 for performing classification by applying the model data stored in the model data storage unit 109 to the processing target data stored in the target data storage unit 113. A classification result storage unit 117 that stores the classification result of the model application unit 115, and an output unit 119 that outputs the classification result stored in the classification result storage unit 117.

  FIG. 3 shows the configuration of the model learning unit. The model learning unit 105 includes a boosting unit 201 and a weak learning unit 203. The boosting unit 201 performs a boosting process. The weak learning unit 203 executes weak learning processing. The weak learning unit 203 in this example uses a perceptron that outputs a multi-valued score as a classification result as a classifier. The perceptron in this example outputs a positive or negative real value as a score. However, the weak learning unit 203 may use another classifier.

  The boosting unit 201 includes an initialization unit 205, a certainty factor calculation unit 207, a combined model update unit 209, and a weight update unit 211. The initialization unit 205 initializes data. The certainty factor calculation unit 207 calculates the certainty factor for the weak hypothesis. The combined model update unit 209 updates the combined model. The weight update unit 211 updates the weight of the learning case.

  The model learning apparatus shown in FIG. 2 performs model learning processing and model application processing. However, the model application process may be performed by an apparatus different from the apparatus that performed the model learning process.

  FIG. 4 shows an example in which a model application device is provided separately from the model learning device. The model learning device has a model data storage unit 109a similar to the model data storage unit 109 shown in FIG. Further, an output unit 119a that outputs model data (for example, final combined model data) stored in the model data storage unit 109a is provided.

  On the other hand, the model application device includes a reception unit 121 that receives model data output from the model learning device. The model application apparatus further includes a model data storage unit 109b for storing the model data received by the reception unit 121. The target data input unit 111 to the classification result storage unit 117 included in the model application device are the same as those of the model learning device in FIG. The model application apparatus further includes an output unit 119b similar to the output unit 119 in the model learning apparatus in FIG.

  Learning data input unit 101, model learning unit 105, target data input unit 111, model application unit 115, output unit 119, boosting unit 201, weak learning unit 203, initialization unit 205, certainty factor calculation unit 207, combination The model update unit 209 and the weight update unit 211 are realized using hardware resources (for example, FIG. 24) and a program that causes a processor to execute the processing described below.

  The learning data storage unit 103, the parameter storage unit 106, the weight data storage unit 107, the certainty degree storage unit 108, the model data storage unit 109, the target data storage unit 113, and the classification result storage unit 117 described above are hardware resources (for example, 24).

  Next, processing contents of the model learning unit 105 and the like in the present embodiment will be described with reference to FIGS.

  FIG. 5 shows a model learning process flow in the present embodiment. As described above, the certainty factor calculation process in the present embodiment is different from the conventional technique.

  First, the learning data input unit 101 accepts input regarding the learning data S and the number of repetitions T, for example, in accordance with an instruction from the user (S101). The learning data S includes learning case data and label data.

FIG. 6 shows an example of learning case data 601. The learning case data 601 includes a feature corresponding to the learning case ID. In this example, the learning case identified by x ₁ is composed of a feature a, a feature b, and a feature c. Similarly, training example identified by x ₂ is composed of a feature a, feature b and feature d. training example identified by x ₃ is composed of feature a and feature b. The number m of learning examples in this example is 3.

FIG. 7 shows an example of label data 701. The label ID corresponds to the learning case ID. y ₁ corresponds to x ₁ . y ₂ corresponds to x ₂ . Furthermore, y ₃ corresponds to x ₃ . The label data 701 includes a label corresponding to the label ID. In this example, the label identified by y ₁ is “+1”. Similarly, the label identified by y ₂ is “−1”. Furthermore, the label identified by y ₃ is “+1”.

  You may make it use the learning data which put together the learning example and the label.

  The initialization unit 205 sets an initial combined model in the model data storage unit 109 (S103). FIG. 8 shows an example of the coupled model data 801a in the initial state. The combined model data 801 includes a third score for each feature. In this example, three types of scores are used. The first score is included in the weak hypothesis data. The second score is included in the classification result data. The first score and the second score will be described later. The initialization unit 205 sets 0 to the third score corresponding to each feature. In this example, 0 is set to the third score corresponding to each of the features a to d.

Next, the initialization unit 205 initializes the weights (w _1,1 ,... W _{1, m} ) of the learning cases stored in the weight data storage unit 107 (S105). Specifically, a value of 1 / m is set to each weight equally.

FIG. 9 shows the weight data 901a in the first round. The weight ID corresponds to the learning case ID. w _1,1 corresponds to x ₁ . w _1,2 corresponds to x ₂ . Furthermore, w _1,3 corresponds to x ₃ . The weight data 901 includes a weight corresponding to the weight ID. In this example, “0.33333” is set for each of the weight identified by w ₁ , the weight identified by w ₂ , and the weight identified by w ₃ .

  The boosting unit 201 sets 1 to the parameter t (S107). The parameter t is a variable for counting the number of executions of the routine from S109 to S117, and the round is specified by t.

  The weak learning unit 203 executes weak learning processing (S109). In the weak learning process in this example, a perceptron is used as a classifier. In this example, perceptron learning is repeated only once.

  FIG. 10 shows an overview of weak learning processing in the first round. FIG. 10 shows how weak hypothesis data 1001a is generated as a result of performing weak learning processing using the above-described learning case data 601, label data 701, and weight data 901a in the first round.

  The weak hypothesis generated in the first round is referred to as the first weak hypothesis. The weak hypothesis data 1001a is data of the first weak hypothesis. The weak hypothesis data 1001 includes a first score for each feature. The first score that is a positive value indicates that the label of the learning case including the feature tends to be “+1”. The first score which is a negative value indicates that the label of the learning case including the feature tends to be “−1”.

  The details other than the model update portion using the case weights are the same as those in the prior art, and the details of the weak learning processing using the perceptron will be omitted.

The certainty factor calculation unit 207 executes a certainty factor calculation process (S111). In the certainty factor calculation process, the certainty factor calculation unit 207 calculates the certainty factor α _t according to the above-described equation (13).

  FIG. 11 shows a certainty factor calculation processing flow. The certainty factor calculation unit 207 first executes classification processing based on the weak hypothesis obtained in S109 (S201).

  FIG. 12 shows an overview of classification based on the first weak hypothesis. For each learning case included in the learning case data 601, classification is performed by applying the weak hypothesis data 1001a related to the first weak hypothesis, and a classification result is obtained. The classification result data 1201a indicates the classification result based on the first weak hypothesis. In this example, the classification result is evaluated. When the classification result and the label have the same sign, it is evaluated as “correct”. On the other hand, if the classification result and the label do not coincide with each other, it is evaluated as “error”. That is, it is evaluated as “correct” when the classification result shows a tendency to match the label, and is evaluated as “error” when the classification result shows a tendency not to match the label.

The classification result for each learning case is obtained as the second score. In this example, the second score h ₁ (x ₁ ), which is the classification result obtained by applying the first weak hypothesis to the learning example x ₁ , is “1”, and the evaluation is “correct”. Similarly, the second score h ₁ (x ₂ ), which is a classification result obtained by applying the first weak hypothesis to the learning case x ₂ , is “0.33333”, and the evaluation is “error”. Furthermore, the second score h ₁ (x ₃ ), which is the classification result obtained by applying the first weak hypothesis to the learning case x ₃ , is “0.66667”, and the evaluation is “correct”.

  In this example, evaluation is given to the classification result for the purpose of explanation, but the evaluation may be omitted.

Confidence factor computing unit 207 obtains the maximum value U _t of the absolute value of the second score is a classification result (S203). In the example of the classification result data 1201a illustrated in FIG. 12, the absolute value “1” of the second score h ₁ (x ₃ ) corresponds to the maximum value U _t .

  The certainty factor calculation unit 207 sets the parameter i to 1 (S205). The parameter i is a variable for counting the number of executions of the routine from S207 to S215, and the learning case is specified by i.

The certainty factor calculation unit 207 determines whether the value obtained by multiplying the label y _i specified by the parameter i by the classification result h _t (x _i ) is positive (S207). The positive value corresponds to the evaluation of the classification result h _t (x _i ) being “correct”.

Label y _i to the classification result when the value obtained by multiplying h _t (x _i) is determined to be positive, the parameter t and weights w _t specified by the parameter _i, the classification result to the _i h _t (x _i) A value obtained by multiplying the absolute value of is added to the parameter for calculating the sum of positive values (S209). Then, the process proceeds to S215.

On the other hand, when it is determined that the value obtained by multiplying the label y _i by the classification result h _t (x _i ) is not positive, the certainty factor calculation unit 207 determines whether or not the value is negative (S211). ). The negative value corresponds to the evaluation of the classification result h _t (x _i ) being “error”.

Label y _i to the classification result when the value obtained by multiplying h _t (x _i) is determined to be negative, the parameter t and weights w _t specified by the parameter _i, the classification result to the _i h _t (x _i) A value obtained by multiplying the absolute value of is added to the parameter for calculating the total sum relating to negative (S213). Then, the process proceeds to S215.

On the other hand, if it is determined that the value obtained by multiplying the label y _i by the classification result h _t (x _i ) is not negative, that is, if the value is 0, the process proceeds to S215 as it is.

  The certainty factor calculation unit 207 adds 1 to the parameter i (S215), and determines whether the parameter i exceeds the number m of learning cases (S217). If it is determined that the parameter i does not exceed the number m of learning cases, the process returns to S207 and a series of processing is repeated.

When it is determined that the parameter i has exceeded the number m of learning examples, the certainty factor calculation unit 207 obtains the logarithm of (total for positive / total for negative) (S219). Then, the certainty factor calculation unit 207 multiplies the obtained logarithm by 1 / (2 × U _t ) (S221). As a result, a certainty factor α _t for the t-th hypothesis is obtained. The certainty factor α _t for the t-th hypothesis is stored in the certainty factor storage unit 108.

FIG. 13 shows an outline of the certainty factor calculation in the first round. In this example, the sum for positive is 0.33333 × 1 + 0.33333 × 0.666667. Moreover, the sum total regarding negative is 0.33333 * 0.33333. The certainty factor α ₁ for the first weak hypothesis is “0.80472”.

When the certainty calculation process is completed, the process returns to the process of S113 shown in FIG. The combined model update unit 209 calculates the combined model data 801 in the current round based on the combined model data 801 in the previous round, the weak hypothesis data 1001 obtained in the current round, and the certainty factor α _t (S113). Specifically, for each feature, a value obtained by multiplying the first score of the current weak hypothesis data 1001 by the certainty factor α _t is added to the third score of the combined model data 801 in the previous round. Then, the obtained sum is set as the third score of the combined model in the current round.

  FIG. 14 shows an outline of the update of the combined model in the first round. The third score “0.26824” for the feature a of the combined model data 801b in the first round is obtained by 0.80472 × 0.333333 + 0. Similarly, the third score “0.26824” for the feature b is obtained by 0.80472 × 0.33333 + 0. Similarly, the third score “0.26824” for the feature c is obtained by 0.80472 × 0.33333 + 0. Further, the third score “−0.26824” for the feature d is obtained by 0.80472 × (−0.33333) +0.

  Subsequently, the weight update unit 211 executes a weight update process (S115). In the weight update process, the weight update unit 211 updates the weight data 901 stored in the weight data storage unit 107.

FIG. 15 shows a weight update processing flow. The weight update unit 211 calculates a coefficient Z _t (α _t ) for normalization according to the above-described equation (2) (S301). The weight update unit 211 sets 1 to the parameter i (S303). The parameter i is a variable for counting the number of executions of the routines up to S305 and S307, and the learning case is specified by i.

The weight update unit 211 calculates the next weight w _{t + 1, i} according to the above-described equation (1) (S305).

  The weight updating unit 211 adds 1 to the parameter i (S307), and determines whether the parameter i exceeds the number m of learning cases (S309). If it is determined that the parameter i does not exceed the number m of learning cases, the process returns to S305 and the above-described processing is repeated.

  If it is determined that the parameter i has exceeded the number m of learning cases, the weight update process is terminated, and the process returns to S117 shown in FIG.

FIG. 16 shows an outline of weight update in the first round. 16, the classification result data 1201a according to the first weak hypothesis, a label data 701, the weight data 901a in the first round, based on the confidence alpha ₁ for the first weak hypothesis, weight data 901b in the second round Is shown.

The weight corresponding to the learning example x ₁ that has obtained the “correct” evaluation is reduced from “0.33333” to “0.19114”. Similarly, the weight corresponding to the learning example x ₃ is also decreased from “0.33333” to “0.24995”. On the other hand, the weight corresponding to the learning example x ₂ that has been evaluated as “error” has increased from “0.33333” to “0.55891”. In this way, by reducing the weight for learning cases that have obtained a “correct” evaluation and increasing the weight for learning cases that have received an “error” evaluation, it is possible to obtain a weak hypothesis that is corrected in the next weak learning process. become.

  Returning to the description of FIG. The boosting unit 201 adds 1 to the parameter t (S117). Then, the boosting unit 201 determines whether or not the parameter t has exceeded the number of repetitions T (S119). If it is determined that the parameter t does not exceed the number of repetitions T, the process returns to S109 and a series of processing is repeated. When it is determined that the parameter t exceeds the number of repetitions T, the output unit 119b of the model application apparatus illustrated in FIG. 4 outputs the final combined model data 801 (S121). In the case of the model learning apparatus shown in FIG. 2, the process of S121 may be omitted.

  17 to 21 show an outline of processing in the second round. First, FIG. 17 shows an outline of weak learning processing in the second round. In the second round, weak learning processing is performed based on the weight data 901b updated in the first round.

  The weak hypothesis generated in the first round is referred to as the second weak hypothesis. The weak hypothesis data 1001b is data of the second weak hypothesis.

FIG. 18 shows an overview of classification based on the second weak hypothesis. In the second round, classification using the weak hypothesis data 1001b related to the second weak hypothesis is performed, and a classification result is obtained. The classification result data 1201b indicates the classification result based on the second weak hypothesis. In the second round, the evaluation for the learning example x ₂ is changed to “correct”.

FIG. 19 shows an outline of the certainty factor calculation in the second round. Α ₂ for the second weak hypothesis is “1.18647”, which is larger than the certainty factor α ₁ for the first weak hypothesis.

FIG. 20 shows an outline of the update of the combined model in the second round. FIG. 20 shows how the combined model data 801b in the first round is updated to the combined model data 801c in the second round based on the weak hypothesis data 1001b related to the second weak hypothesis and α ₂ for the second weak hypothesis. Show.

  Finally, the model application process in the model application unit 115 will be described. FIG. 21 shows a model application processing flow.

  The target data input unit 111 inputs processing target data (S401). The processing target data is stored in the target data storage unit 113. The processing target data includes a combination of features as in the learning case data 601. An unknown combination other than the combination of features in the learning case included in the learning case data 601 may be processed.

  The model application unit 115 executes classification processing based on the combined model (S403). That is, classification by the classifier is performed using the combined model data 801 generated by model learning. The classification result is stored in the classification result storage unit 117. Then, the output unit 119 outputs the classification result stored in the classification result storage unit 117 (S405).

  In many cases, the finally generated combined model data 801 is used. However, in the following, in order to compare the suitability of the combined model data 801b in the first round and the combined model data 801c in the second round, an example of model application using the combined model data 801b and 801c is shown. .

  In many cases, unknown combinations other than the combination of features in the learning case are processed, but in the following, the same combination as the combination in the learning case is processed to make it easier to explain the improvement. And

FIG. 22 shows an example of model application using the combined model data 801b in the first round. The combination of features in the processing target x ₁₁ included in the processing target data 2201 is the same as the combination of features related to the learning example x ₁ . Therefore, the processing target x ₁₁ should be classified into the label “+1”. Similarly, the combination of features in the processing target x ₁₂ is the same as the combination of features related to the learning example x ₂ . Therefore, the processing target x ₁₂ should be classified into the label “−1”. Further, the combination of features in the processing target x ₁₃ is the same as the combination of features related to the learning example x ₃ . Therefore, the processing target x ₁₃ should be classified into the label “+1”.

The second score H ₁ (x ₁₁ ), which is the classification result for the processing target x ₁₁ , is positive as is the label to be classified. Therefore, the evaluation is “correct”. Further, the second score H ₁ (x ₁₃ ), which is the classification result for the processing target x ₁₃ , is also positive like the label to be classified. Therefore, the evaluation is “correct”. On the other hand, the second score H ₁ (x ₁₂ ), which is the classification result for the processing target x _12, is not an expected negative value. Therefore, the evaluation is “error”. That is, one of the processing targets is not correctly classified.

FIG. 23 shows an example of model application using the combined model data 801c in the second round. The processing target data 2201 is the same as in the case of FIG. When using a binding model data 801c in the second round, evaluation of the classification results for the processed x ₁₁ to be processed x ₁₃ is "correct". In other words, all the processing objects remain and are correctly classified. This example shows that the suitability of the combined model increases with increasing rounds.

  According to the present embodiment, the tendency (in this example, positive or negative in the second score) that matches one of the predetermined two values indicated by the label and the degree of the tendency (in this example, the second score) ) (The absolute value of the second hypothesis) can be reduced.

  Furthermore, it is possible to secure the convergence of learning in boosting in which a tendency that matches one of the predetermined two values indicated by the label and a weak hypothesis for obtaining a classification result indicating the degree of the tendency are combined.

  Although one embodiment of the present invention has been described above, the present invention is not limited to this. For example, the functional block configuration described above may not match the program module configuration.

  Further, the configuration of each storage area described above is an example, and the above configuration is not necessarily required. Further, in the processing flow, if the processing result does not change, the processing order may be changed or a plurality of processes may be executed in parallel.

  The model learning device described above is a computer device, and as shown in FIG. 24, a memory 2501, a CPU (Central Processing Unit) 2503, a hard disk drive (HDD: Hard Disk Drive) 2505, and a display device 2509. A display control unit 2507 connected to the computer, a drive device 2513 for a removable disk 2511, an input device 2515, and a communication control unit 2517 for connecting to a network are connected by a bus 2519. An operating system (OS) and an application program for executing the processing in this embodiment are stored in the HDD 2505, and are read from the HDD 2505 to the memory 2501 when executed by the CPU 2503. The CPU 2503 controls the display control unit 2507, the communication control unit 2517, and the drive device 2513 according to the processing content of the application program, and performs a predetermined operation. Further, data in the middle of processing is mainly stored in the memory 2501, but may be stored in the HDD 2505. In the embodiment of the present invention, an application program for performing the above-described processing is stored in a computer-readable removable disk 2511 and distributed, and installed in the HDD 2505 from the drive device 2513. In some cases, the HDD 2505 may be installed via a network such as the Internet and the communication control unit 2517. Such a computer apparatus realizes various functions as described above by organically cooperating hardware such as the CPU 2503 and the memory 2501 described above and programs such as the OS and application programs. .

  The embodiment of the present invention described above is summarized as follows.

  The information processing apparatus according to the present embodiment includes (A) a plurality of sets including a learning case and a label indicating one of predetermined two values for the learning case, and a coefficient value for each learning case. Based on the learning unit that learns the hypothesis model, and (B) (b1) the classification indicating the tendency corresponding to one of the above two values and the degree of the tendency for each learning case based on the hypothesis model and each learning case (B2) Among the learning cases, the trend in the classification result identifies the learning case corresponding to the corresponding label, and for each identified learning case, the corresponding coefficient value and the above degree in the classification result (B3) Among the learning cases, the above-mentioned tendency in the classification result identifies a learning case that does not correspond to the corresponding label, and corresponds to each identified learning case. (B4) The logarithm of the ratio of the first sum to the second sum is calculated as the absolute value of the above degree in each classification result. A calculation unit that calculates the certainty factor of the hypothesis model by dividing by a value that is twice the maximum value among the values.

  In this way, it is possible to reduce the processing load for calculating the certainty factor of the hypothesis model for obtaining the classification result indicating the tendency corresponding to one of the predetermined two values indicated by the label and the degree of the tendency.

  Further, the calculation unit may calculate a hypothesis model and a certainty factor of the hypothesis model in each round using a coefficient value for each learning case in the round in each round repeated a plurality of times. Moreover, you may make it include the 1st update part which updates a learning model by combining the hypothesis model in the said round based on the certainty in each round. Moreover, you may make it include the 2nd update part which updates the coefficient value with respect to each learning example to the coefficient value for the next round based on reliability.

  In this way, it is possible to guarantee the convergence of learning in boosting that combines a tendency corresponding to one of the predetermined two values indicated by the label and a hypothesis model for obtaining a classification result indicating the degree of the tendency.

  A program for causing a computer to perform the processing in the information processing apparatus described above can be created. The program can be read by a computer such as a flexible disk, a CD-ROM, a magneto-optical disk, a semiconductor memory, and a hard disk It may be stored in a possible storage medium or storage device. Note that intermediate processing results are generally temporarily stored in a storage device such as a main memory.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Appendix 1)
A learning unit for learning a hypothesis model based on a plurality of sets including a learning case and a label indicating any one of predetermined two values for the learning case, and a coefficient value for each learning case;
Based on the hypothesis model and each learning case, obtain a classification result indicating the tendency corresponding to any of the two values and the degree of the tendency for each learning case,
Among the learning cases, the tendency in the classification result identifies a learning case corresponding to the corresponding label, and for each learning case specified, the product of the corresponding coefficient value and the degree in the classification result And calculate the first sum of the products,
Among the learning cases, the tendency in the classification result identifies a learning case that does not correspond to the corresponding label, and for each learning case specified, the product of the corresponding coefficient value and the degree in the classification result And calculate the second sum of the products,
The confidence of the hypothesis model is calculated by dividing the logarithm of the ratio of the first sum to the second sum by a value twice the maximum of the absolute values of the degree in each classification result. An information processing apparatus including a calculation unit.

(Appendix 2)
The calculation unit calculates a hypothesis model and a certainty factor of the hypothesis model in the round using a coefficient value for each learning case in the round in each round repeated a plurality of times,
A first update unit that updates the learning model by combining the hypothesis model in the round based on the certainty factor in each round;
The information processing apparatus according to appendix 1, further comprising: a second updating unit that updates a coefficient value for each learning case to a coefficient value for the next round based on the certainty factor.

(Appendix 3)
A first step of learning a hypothesis model based on a plurality of sets including a learning case and a label indicating one of predetermined two values for the learning case, and a coefficient value for each learning case;
A second step of obtaining a classification result indicating a tendency corresponding to one of the two values and a degree of the tendency for each learning case based on the hypothesis model and each learning case;
Among the learning cases, the tendency in the classification result identifies a learning case corresponding to the corresponding label, and for each learning case specified, the product of the corresponding coefficient value and the degree in the classification result And calculating a first sum of the products,
Among the learning cases, the tendency in the classification result identifies a learning case that does not correspond to the corresponding label, and for each learning case specified, the product of the corresponding coefficient value and the degree in the classification result And calculating a second sum of the products,
The confidence of the hypothesis model is calculated by dividing the logarithm of the ratio of the first sum to the second sum by a value twice the maximum of the absolute values of the degree in each classification result. An information processing method including a fifth step and executed by a computer.

(Appendix 4)
In each round of repeating the first step to the fifth step, using the coefficient value for each learning case in the round, calculate the hypothesis model in the round and the certainty of the hypothesis model,
Furthermore,
A sixth step of updating the learning model by combining the hypothesis model in the round based on the certainty factor in each round;
The information processing method according to supplementary note 3, including a seventh step of updating a coefficient value for each learning case to a coefficient value for the next round based on the certainty factor.

(Appendix 5)
A first step of learning a hypothesis model based on a plurality of sets including a learning case and a label indicating one of predetermined two values for the learning case, and a coefficient value for each learning case;
A second step of obtaining a classification result indicating a tendency corresponding to one of the two values and a degree of the tendency for each learning case based on the hypothesis model and each learning case;
Among the learning cases, the tendency in the classification result identifies a learning case corresponding to the corresponding label, and for each learning case specified, the product of the corresponding coefficient value and the degree in the classification result And calculating a first sum of the products,
Among the learning cases, the tendency in the classification result identifies a learning case that does not correspond to the corresponding label, and for each learning case specified, the product of the corresponding coefficient value and the degree in the classification result And calculating a second sum of the products,
The confidence of the hypothesis model is calculated by dividing the logarithm of the ratio of the first sum to the second sum by a value twice the maximum of the absolute values of the degree in each classification result. An information processing program for causing a computer to execute the fifth step.

DESCRIPTION OF SYMBOLS 101 Learning data input part 103 Learning data storage part 105 Model learning part 106 Parameter storage part 107 Weight data storage part 108 Certainty degree storage part 109 Model data storage part 111 Target data input part 113 Target data storage part 115 Model application part 117 Classification result Storage unit 119 Output unit 121 Reception unit 201 Boosting unit 203 Weak learning unit 205 Initialization unit 207 Certainty factor calculation unit 209 Combined model update unit 211 Weight update unit 601 Learning example data 701 Label data 801 Combined model data 901 Weight data 1001 Weak Hypothesis data 1201 Classification result data 2201 Processing target data

Claims (4)

A learning unit for learning a hypothesis model based on a plurality of sets including a learning case and a label indicating any one of predetermined two values for the learning case, and a coefficient value for each learning case;
Based on the hypothesis model and each learning case, obtain a classification result indicating the tendency corresponding to any of the two values and the degree of the tendency for each learning case,
Among the learning cases, the tendency in the classification result identifies a learning case corresponding to the corresponding label, and for each learning case specified, the product of the corresponding coefficient value and the degree in the classification result And calculate the first sum of the products,
Among the learning cases, the tendency in the classification result identifies a learning case that does not correspond to the corresponding label, and for each learning case specified, the product of the corresponding coefficient value and the degree in the classification result And calculate the second sum of the products,
The confidence of the hypothesis model is calculated by dividing the logarithm of the ratio of the first sum to the second sum by a value twice the maximum of the absolute values of the degree in each classification result. An information processing apparatus including a calculation unit. The calculation unit calculates a hypothesis model and a certainty factor of the hypothesis model in the round using a coefficient value for each learning case in the round in each round repeated a plurality of times,
A first update unit that updates the learning model by combining the hypothesis model in the round based on the certainty factor in each round;
The information processing apparatus according to claim 1, further comprising: a second updating unit that updates a coefficient value for each learning case to a coefficient value for the next round based on the certainty factor. A first step of learning a hypothesis model based on a plurality of sets including a learning case and a label indicating one of predetermined two values for the learning case, and a coefficient value for each learning case;
A second step of obtaining a classification result indicating a tendency corresponding to one of the two values and a degree of the tendency for each learning case based on the hypothesis model and each learning case;
Among the learning cases, the tendency in the classification result identifies a learning case corresponding to the corresponding label, and for each learning case specified, the product of the corresponding coefficient value and the degree in the classification result And calculating a first sum of the products,
Among the learning cases, the tendency in the classification result identifies a learning case that does not correspond to the corresponding label, and for each learning case specified, the product of the corresponding coefficient value and the degree in the classification result And calculating a second sum of the products,
The confidence of the hypothesis model is calculated by dividing the logarithm of the ratio of the first sum to the second sum by a value twice the maximum of the absolute values of the degree in each classification result. An information processing method including a fifth step and executed by a computer. A first step of learning a hypothesis model based on a plurality of sets including a learning case and a label indicating one of predetermined two values for the learning case, and a coefficient value for each learning case;
A second step of obtaining a classification result indicating a tendency corresponding to one of the two values and a degree of the tendency for each learning case based on the hypothesis model and each learning case;
Among the learning cases, the tendency in the classification result identifies a learning case corresponding to the corresponding label, and for each learning case specified, the product of the corresponding coefficient value and the degree in the classification result And calculating a first sum of the products,
Among the learning cases, the tendency in the classification result identifies a learning case that does not correspond to the corresponding label, and for each learning case specified, the product of the corresponding coefficient value and the degree in the classification result And calculating a second sum of the products,
The confidence of the hypothesis model is calculated by dividing the logarithm of the ratio of the first sum to the second sum by a value twice the maximum of the absolute values of the degree in each classification result. An information processing program for causing a computer to execute the fifth step.

Images