JPH07311754A - Learning machine - Google Patents

Abstract

PURPOSE:To efficiently and precisely perform a learning by generating optimum learning data which are effective to the learning. CONSTITUTION:A learning data generation part 4 generates input data xnu according to a probable rule and generates the learning data (xnu, ynu) by using the real response of an unknown system to the input data as tutor data ynu, and an initial learning data generation part 4a which generates initial learning data and an update learning data generation part 4b which generates learning data for update are provided corresponding to that a conditioned probability estimation part 2 carries out the learning in two stages of learning for initial learning and subsequent parameter update.

Description

Detailed Description of the Invention

[0001]

FIELD OF THE INVENTION This invention relates to learning machines that can be used to obtain a desired output from a given input, such as system identification, pattern recognition and control problems.

[0002]

2. Description of the Related Art Conventionally, in a statistical learning machine such as a neural network, for example, back propagation (error back propagation algorithm) is known as a learning mechanism. As shown in FIG. 2, the statistical learning machine includes an input layer 51, an intermediate layer 52, and an output layer 53.
In the case of a layered perceptron (hierarchical network), the learning mechanism of the backpropagation is that the weight of the coupling between the unit of the input layer 51 and the unit of the intermediate layer 52, the unit of the intermediate layer 52 and the unit of the output layer 53 The weight of the coupling of is included in the parameter, and learning of the square error minimization is performed. More specifically, considering an m-layer network, the sum of the inputs to the i-th unit of the k-layer is i
^k _i, the output and o ^k _i, the weight of binding to the i-th unit of k layer and w ^k-1 _i ^k _j from the i unit k-1 layer, a function that gives the input-output relationship of each unit Given f, the relationship between these variables is:

[0003]

[Equation 1]

Taking the loss function r as the square of the error, the loss function r when a certain set of input / output patterns (x, y) is given is given by the following equation.

[0005]

[Equation 2]

[0006] Here, w is a collection of all the connection weights of the network to which the association is given, and in order to learn the connection weight w (that is, to find the correction amount of the connection weight w). ), The gradient (gradient) ∂r / ∂w ^k-1 _i ^k _j for the weight w of the connection of the loss function r may be calculated. That is, when the parameter that determines the magnitude of one modification is ε, the modification amount Δw ^k−1 _i ^k _{j of the} connection weight w is
Is given by the following equation.

[0007]

[Equation 3]

[0008] Number 3 the second, from the third equation, the signal d ^k _j used to modify the weights w ^k-1 _i ^k _j of coupling k = m k =
It can be seen that it is calculated recursively toward 2. Also,
From the third equation of the equation 3, the process of calculating d ^k _j is that the error d ^{k + 1} _h between the ideal output and the actual output in the output layer is used as an input, and the direction from the output layer to the input layer is changed. , And propagates while taking the sum weighted by w ^k _i ^{k + 1} _h in the opposite direction to the signal propagation. This is the backpropagation (error backpropagation) learning algorithm.

[0009]

However, in the conventional learning machine as described above, the learning data is given only passively, and the learning data effective for learning is not generated. For example, in a learning machine using the above-described learning algorithm of back propagation, as learning data (a pair of input / output patterns (x, y)), one that is passively generated from a true probability distribution is usually used. But,
There is no guarantee that learning with a sample that follows the true probability distribution is optimal, so the backpropagation learning algorithm described above actually uses the following to reduce oscillations and accelerate learning convergence: I had to make corrections like expressions.

[0010]

[Equation 4]

In Equation 4, α represents a small positive constant and t represents the number of corrections.

As described above, the conventional learning machine has not been designed to generate the optimum learning data effective for learning to perform the learning, so that the learning cannot be performed efficiently and accurately. was there.

It is an object of the present invention to provide a learning machine capable of generating optimum learning data effective for learning and performing learning efficiently and accurately.

[0014]

In order to achieve the above object, the invention according to claim 1 is an input means for receiving an input vector x from an input vector space X, an input data for learning and the above-mentioned input data for it. Using the learning data that is a set of the teacher data that is the response of the unknown system, the learning of the predetermined parameter θ is performed, and the true conditional probability p
(y | x) is estimated, and the conditional probability estimating means for estimating the unknown system by this, and the parameter <θ> learned by the conditional probability estimating means are used to obtain an unlearned value given by the input means. The output means for calculating the output y for the input vector x as a sample according to the conditional probability p (y | x; <θ>), and the input data generated according to the stochastic rule, the learning data used by the conditional probability estimation means Learning data creating means is created from the response of the unknown system to the input data. With this configuration, it is possible to generate the optimum learning data effective for learning by the learning data creating means, and thereby the learning can be performed efficiently and accurately.

Further, in the inventions according to claims 2 to 6, the conditional probability estimating means updates the parameters with new learning data after the initial learning means for learning the initial parameters and the initial parameters are determined. Parameter learning means, and the learning data creating means, the initial learning data creating means for creating the initial learning data used by the initial learning means of the conditional probability estimating means, and the initial learning ends in the initial learning data creating means. After that, it has an update learning data creating means for creating the update learning data used by the parameter updating means. As a result, optimum learning data effective for learning can be generated according to the operating status of the system.

[0016]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of an embodiment of a learning machine according to the present invention. The learning machine according to the present embodiment outputs an M-dimensional output according to a true conditional probability p (y | x) for an input vector x = (x ₁ , ..., X _L ) from an L-dimensional input vector space X. A learning machine (statistical learning machine) that estimates an unknown system that generates an output vector y = (y ₁ , ..., y _M ) on a vector space Y
Therefore, referring to FIG. 1, the learning machine includes an input unit 1 that receives an input vector x from an L-dimensional input vector space X, learning input data x _ν (1 ≦ ν ≦ N), and Teacher data y which is the response of the unknown system
_Using N learning data {x _ν , y _ν | 1 ≤ ν ≤ N} consisting of a set of _ν (1 ≤ ν ≤ N), learning of a predetermined parameter θ is performed, and an unknown system, that is, true The conditional probability estimator 2 for estimating the conditional probability p (y | x), and the parameter <θ> learned by the conditional probability estimator 2.
Using, the output unit 3 for calculating the output y for the unlearned input vector x given from the input unit 1 as a sample according to the conditional probability p (y | x; <θ>), and the conditional probability estimation unit 2 And a learning data creating unit 4 for creating learning data {x _ν , y _ν } (ν = 1 to N).

Here, the conditional probability estimation unit 2 uses an initial learning unit 2a that performs initial learning of the parameter θ using the initial learning data, and the learning data for updating after the initial parameter θ is determined. A parameter updating unit 2b for updating the parameter θ is provided. In the present embodiment, the conditional probability estimator 2 performs learning as described above,
The initial learning and the subsequent parameter updating are performed in two stages. In both the initial learning unit 2a and the parameter updating unit 2b, the estimated amount (learning result) <θ> of the parameter θ is, for example, a parameter having the parameter θ. Using the attached probability density function family {p (y | x; θ)}, the maximum likelihood estimation method well known in the field of statistics is used to obtain the following equation.

[0018]

[Equation 5]

That is, the parameter θ that maximizes S is obtained as the estimated amount <θ>.

Further, the learning data creation unit 4 creates the input data x _ν according to the stochastic rule, and creates the learning data {x _ν , y _ν } with the response of the true unknown system to them as the teacher data y _ν. However, the conditional probability estimation unit 2
Initial learning data creation unit 4a that creates initial learning data and update learning data creation that creates update learning data in correspondence with performing learning in two stages of initial learning and subsequent parameter update And a section 4b.

When the distribution of the actual input data is known, the initial learning data creating section 4a can create the initial learning data according to the distribution of the actual input data.
For example, the initial learning data {x _ν , y _ν } may be generated by generating the input data x _ν of the initial learning data from the estimated amount of the generation distribution of the input data generated when the learning machine is used (operating). it can. On the other hand, when the actual distribution of the input data is unknown, the initial learning data can be created according to the uniform distribution or the normal distribution that is determined in advance based on prior knowledge. For example, the learning machine can generate the initial learning data {x _ν , y _ν } by generating the input data x _ν of the initial learning data according to the probability distribution on the constant input space prepared in advance.

The update learning data creating section 4b is adapted to create update learning data by a predetermined probability density function r (x; v). That is, the update learning data creation unit 4b uses the parameters in the input vector space X
(v) holds the probability density function family {r (x; v)},
For example, the estimated amount <q (x)> of the probability density function q (x) of the occurrence distribution of the actual input data that occurs when this learning machine is used (operating) and the initial learning unit of the conditional probability estimation unit 2 The estimated value E (v) of the learning error when the input data x _ν of the learning data is generated according to {r (x; v)} using the estimated value <θ> of the parameter θ obtained by 2a. The value of the parameter v to be minimized is calculated as the optimum value, and the input data x _v of the learning data for updating is generated by the probability density function r (x; v) based on the optimized parameter v, and the learning data for updating { x _ν , y _ν } is created.

The estimated amount <q (x)> of the probability density function q (x) of the distribution of the actual input data generated when the learning machine is used (when operating) is, for example, Generate N'input data in the same situation as the actual operation,
The distribution can be estimated by a parametric model, or represented by a delta function as in the following equation, and prepared in advance.

[0024]

[Equation 6]

The calculation of the value of the parameter v that minimizes the estimated value E (v) of the learning error is performed by the learning error minimization circuit 5 in the update learning data creation unit 4b. More specifically, the learning error minimization circuit 5 optimizes the parameter v so as to minimize the learning error estimated value E (v) of the following equation.

[0026]

[Equation 7]

Here, <I> and <J (v)> are matrices, which are calculated as follows.

[0028]

[Equation 8]

In general, it is often difficult to analytically solve v that gives the minimum value, so v may be optimized by sequentially updating it by, for example, the steepest descent method. The parameter thus obtained will be written as <v>. The update learning data creating means is r (x;
According to <v>), a certain number of input data are generated, and learning data composed of the input data and teacher data which is an output result of a truly unknown system for them is created as update learning data. The parameter updating unit 2b
Using this update learning data, the parameter θ of p (y | x; θ) is learned again by the maximum likelihood estimation method.

It will be described below that the method described above is effective in reducing the learning error.

Now, the true conditional probability p (y | x) is assumed to be included in the set model {p (y | x; θ)}, and p (y | x) = p (y | x ; Θ ₀ ). Where θ ₀
Is a true parameter. Further, the density function of the input distribution that generates the learning data is r (x), and the parameter obtained by the maximum likelihood estimation from the N learning data thus created is <θ>. At this time, the learning loss amount R is measured by the following equation.

[0032]

[Equation 9]

How to measure this loss is Kullback-Leibler Div
It is based on ergence and is often used in the field of statistics. Taylor expansion of this amount R around θ ₀ gives the following equation.

[0034]

[Equation 10]

In the above equation, it is understood that the first-order differential term (second term) becomes "0" by integrating with y. Also,
The first term in the above equation is a term that does not depend on the estimator <θ>. Therefore, it can be seen that the third term of the above equation can be seen to determine the goodness of learning. Letting E ₀ be the expected value of the value of this term depending on the way the learning data appears, E ₀ is obtained by a simple calculation as in the following equation.

[0036]

[Equation 11]

Here, as well known in the field of statistical estimation, the following equation holds.

[0038]

[Equation 12]

When I _ab and J _ab are defined as follows,

[0040]

[Equation 13]

The following approximate expression can be derived from the equation (12).

[0042]

[Equation 14]

The estimated value of the learning error used by the update learning data creating unit 4b of the present invention is calculated by using the estimated value of each matrix for the value E ₀ , and it is necessary to reduce the amount of learning. It can be seen that this leads to reducing the error.

Next, the operation of the learning machine having such a configuration will be described. The learning machine of this embodiment is used to estimate an unknown system capable of returning a sample according to p (y | x) for an arbitrary input x given by this learning machine. True systems can often return outputs for arbitrary inputs, such as in system identification problems. The learning machine of this embodiment can be used to estimate such a system.

The estimation (learning) of the unknown system is performed by the conditional probability estimator 2, but in order to generate learning data for the conditional probability estimator 2, a learning data generator 4 is provided in this embodiment. ing. The learning data creation unit 4
In order to provide learning data to the conditional probability estimation unit 2, input data is generated according to the stochastic rule, and learning data is created by using the response of the true unknown system to them as the teacher data.

At this time, first, the initial learning data creating unit 4a of the learning data creating unit 4 creates the initial learning data {x _ν , y _ν } according to a predetermined input data distribution. When the initial learning data {x _ν , y _ν } is created, the initial learning unit 2a of the conditional probability estimation unit 2 uses this initial learning data {x _ν , y _ν } to initialize the predetermined parameter θ. Learning is performed to generate an initial learning result <θ>.

After the initial learning is performed in this way,
The update learning data creation unit 4b of the learning data creation unit 4 is
The estimated amount <q (x)> of the probability density function q (x) of the occurrence distribution of the actual input data that occurs when the learning machine is operating, and the parameter θ obtained by the initial learning unit 2a of the conditional probability estimation unit 2. And the estimated value <θ> of the learning data are used to minimize the estimated value E (v) of the learning error when the input data x _ν of the learning data is generated according to the probability density function family {r (x; v)}. The value of the parameter v is calculated as the optimum value, and the input data x _ν of the learning data for update is generated by the probability density function r (x; v) based on the optimized parameter v, and the learning data for update {x _ν , Create y _ν }. When the update learning data {x _ν , y _ν } is created, the parameter updating unit 2b of the conditional probability estimation unit 2 updates the update learning data {x _ν , y.
_ν } is used to perform the update learning of the parameter θ, and the update learning result <θ> is generated. Here, the update learning data {x _ν , y _ν } is optimum learning data effective for learning, and therefore, the parameter updating unit 2b of the conditional probability estimating unit 2 efficiently and accurately performs learning. By performing well, a highly accurate (small learning error) learning result <θ> can be quickly obtained.

In this way, the parameter probability θ is learned in the conditional probability estimator 2, and after the update learning result <θ> is generated, when the input vector x is given from the input unit 1, the output unit 3 Is a sample that follows the conditional probability p (y | x; <θ>) of the output y for the given input vector x using the parameter (update learning result) <θ> learned by the conditional probability estimator 2. Can be calculated as

As described above, in the present embodiment, the problem that the unknown system gives the teacher data to the input data prepared by the learning machine, such as the system identification problem,
When learning is performed by a statistical learning machine such as a neural network, it is possible to design how to prepare the learning data optimally, and improve the estimation accuracy after learning.

[0050]

As described above, according to the first aspect of the present invention, the learning data used by the conditional probability estimating means is the input data generated according to the stochastic rule and the unknown system for the input data. Since it is created from the response, the optimum learning data effective for learning can be generated and learning can be performed efficiently and accurately.

According to the invention described in claims 2 to 6, the conditional probability estimating means determines the parameters by the initial learning means for learning the initial parameters and the new learning data after the initial parameters are determined. The learning data creating means includes an initial learning data creating means for creating initial learning data used by the initial learning means of the conditional probability estimating means, and an initial learning in the initial learning data creating means. After the completion of the above, since it has an update learning data creating means for creating the update learning data used by the parameter updating means, the optimum learning data effective for learning is generated, and the learning is efficiently and accurately performed. Can be done.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an embodiment of a learning machine according to the present invention.

FIG. 2 is a diagram showing a perceptron having a three-layer structure as a statistical learning machine.

[Explanation of symbols]

 1 Input Part 2 Conditional Probability Estimating Part 3 Output Part 4 Learning Data Creating Part 2a Initial Learning Part 2b Parameter Updating Part 4a Initial Learning Data Creating Part 4b Updating Learning Data Creating Part 5 Learning Error Minimization Circuit

Claims (6)

[Claims] 1. A learning machine for estimating an unknown system that produces an output vector y according to a true conditional probability p (y | x) with respect to an input vector x from an input vector space X. Using the input means for receiving the input vector x from and the learning data consisting of the learning input data and the teacher data which is the response of the unknown system to the learning data, learning of the predetermined parameter θ is performed,
From the input means, the true conditional probability p (y | x) is estimated and the conditional probability estimating means for estimating the unknown system by this, and the parameter <θ> learned by the conditional probability estimating means are used. Given unlearned input vector x
Output means for calculating the output y for the sample as a sample according to the conditional probability p (y | x; <θ>), the learning data used by the conditional probability estimation means, the input data generated according to the stochastic rule, and the input data. A learning machine for creating learning data from the response of the unknown system to the learning machine. 2. The learning machine according to claim 1, wherein the conditional probability estimation means updates the parameters with new learning data after the initial learning means for learning the initial parameters and the initial parameters are determined. A parameter updating means, and the learning data creating means, the initial learning data creating means for creating the initial learning data used by the initial learning means of the conditional probability estimating means, and the initial learning data creating means After learning,
A learning machine comprising: update learning data creating means for creating update learning data used by the parameter updating means. 3. The learning machine according to claim 2, wherein the initial learning data creating means generates the input data of the initial learning data from an estimated amount of an input occurrence distribution generated when the learning machine is used. Learning machine to do. 4. The learning machine according to claim 2, wherein the initial learning data creating means generates the input data of the initial learning data according to a probability distribution on a constant input space prepared in advance by the learning machine. Characteristic learning machine. 5. The learning machine according to claim 2, wherein the learning data creating means for updating holds a parameterized probability density function {r (x; v)} on an input vector space, The input data of the learning data is set to {r by using the estimated amount <q (x)> of the probability density function of the occurrence distribution of the input data generated at the time of operation and the estimated value <θ> of the parameter θ obtained by the initial learning means. (x; v)} The value of the parameter v that reduces the estimated value E (v) of the learning error when generated according to (x; v)} is calculated as an optimum value, and the probability density function r (x; v) by the optimized parameter v is calculated. ) Is used to generate input data of update learning data. 6. The learning machine according to claim 5, wherein the optimum parameter v is sequentially updated using the steepest descent method so as to minimize the estimated value E (v) of the learning error. A learning machine characterized by being obtained by.