JPH09138786A - Learning device for neural network - Google Patents

Abstract

PROBLEM TO BE SOLVED: To uniformly learn respective pieces of data at high speed in spite of the learning easiness of learning data, the classification of a data pattern and the number of data for respective classifications. SOLUTION: A category storage means 3 divides and stores learning data for respective categories, and a neural network (N. N) operation means 1 calculates N. N outputs for the respective categories. A weighting factor update means 4 changes and stores weighting factors for the respective categories in accordance with a square error between the N. N outputs of the categories and a teacher signal and the number of data for the respective categories. A weighting means 5 weights the sum of the square errors of the respective categories by means of the weighting factor. A coupling coefficient update means 2 updates a coupling coefficient by a back propagation method with the sum of the square errors as an error function.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a learning device for a neural network (hereinafter simply referred to as "NN"), and in particular, a characteristic of a nonlinear object whose parameters are unknown is calculated from the input / output data of the object. The present invention relates to a device for making N learn.

[0002]

2. Description of the Related Art A conventional N.P. N
The learning device will be described. FIG. 6 shows a conventional N.M. 1 shows an outline of a learning device of N. N calculation means,
Reference numeral 2 is a coupling coefficient updating means.

FIG. It is a figure explaining the content of the N calculation means 1. This example is a three-layer N.M. N, the number of elements in the intermediate layer is n, the output function of the intermediate layer is a sigmoid function, and the output function of the input / output layer is a linear function. Less than,
3-layer N.V. N will be described as an example.

Referring to FIG. Normalized learning data is input to the N calculation means 1, and N. N output is calculated. First, N. P-th learning data X which is an input to N
When (p) is X1 (p), X2 (p), ..., Xm (p), the input Uj (p) to the j-th element of the intermediate layer has a threshold value of θwj and an i-th input layer element. Coupling coefficient Wij from the element to the jth intermediate layer element
As

[0005]

(Equation 1)

Is calculated as Where W0j = θwj, X0
(p) = 1. Output Hj of the j-th element in the intermediate layer
(p) is the output function of the hidden layer is f

[0007]

## EQU2 ## Hj (p) = f (Uj (p)).

Next, the input Or to the r-th output layer is a threshold value θvr, and the coupling coefficient Vjr from the j-th intermediate layer element to the r-th output layer element is

[0009]

(Equation 3)

Is calculated as Here, V0r = θvr, H0
(p) = 1. Output of r-th output layer element Yr (p)
Is

[0011]

## EQU4 ## Yr (p) = Or (p) is calculated.

Next, in the coupling coefficient updating means 2, N. N sum of squared errors between the output of the computing means 1 and the teacher signal E

[0013]

(Equation 5)

Is an error function, and N.V. is calculated by the backpropagation method. Calculate the coupling coefficient for N. Where dr
(p) is a teacher signal for the output Yr (p) of the r-th output layer element.

The coupling coefficient Wij from the i-th input layer element to the j-th intermediate layer element is given by (Equation 6).

[0016]

(Equation 6)

The coupling coefficient Vjr from the j-th intermediate layer element to the r-th output layer element is updated as shown in (Equation 7).

[0018]

(Equation 7)

And is updated. In the above (Equation 6) and (Equation 7), n is the number of learnings, p is the total number of learning data, and μ is a learning coefficient.
(0 <μ) and λ are moment coefficients (0 <λ <1), which are usually set at the start of learning.

In pattern recognition and the like, the learning coefficient is changed according to the learning degree for each learning data. A learning method of N is disclosed in Japanese Patent Laid-Open No. 3-18967. The learning degree is a value obtained by multiplying the recognition result for each learning data by the value of the element that is most fired in the output for the learning data.

For the learning data X (p), the learning coefficient μ ′
(p) as in (Equation 8)

[0022]

8 ′ μ ′ (p) = {0, 1, C, 2 · C, 4 · C, 6 ·
C} C is a positive constant, and if X (p) is correctly recognized and the learning degree of X (p) is high, the learning coefficient μ ′ (p) = 0. When (p) = 1 is low, the learning coefficient μ ′ (p) = C X (p) is erroneously recognized, and when the learning degree of X (p) is low, the learning coefficient μ ′ (p) = 2. In the case of medium C, the learning coefficient μ ′ (p) = 4 · C. When the learning coefficient is high, the learning coefficient μ ′ (p) = 6 · C is changed, and the coupling coefficient is changed to (Equation 6) or (Equation 7) for

[0023]

(Equation 9)

[0024]

(Equation 10)

By updating to, the weighting of the learning coefficient of the data that is difficult to learn is increased, and more of the data is learned in a pseudo manner so that each learning data can be learned uniformly.

[0026]

However, in the above-mentioned conventional apparatus, when the learning coefficient is weighted and the learning is performed, the same learning is performed when learning the dynamic characteristic of the object in which the magnitude of the teacher signal continuously changes. There is a problem that the weight changes depending on the size of the teacher signal even with an error.

If the conventional weighting is not performed,
Since the easiness of learning depends on the classification of the data patterns to be learned and the number of the data, in order to uniformly learn each data, the ratio of the number of data for each pattern in all learning data and the learning There was a problem that data had to be created in consideration of ease.

The present invention solves the above-mentioned problems of the prior art, and an object of the present invention is to provide a learning device of a neural network capable of uniformly learning each data at high speed.

[0029]

In order to achieve the above object, a learning device for a neural network according to the present invention comprises (1) a category memory for storing learning data, which is a learning target given as an input, for each of a plurality of categories. Means, a neural network operation means for performing an operation of a neural network on learning data divided into a plurality of categories, a square error between the output of the neural network operation means and the teacher signal, and the number of data in each category. Weighting coefficient updating means for changing and storing the weighting coefficient for each category, weighting means for weighting the squared error according to the weighting coefficient, and connection of the neural network from the weighted squared error by the back propagation method. A configuration including a coupling coefficient updating unit that obtains a coefficient To.

(2) Further, in addition to the above-mentioned configuration, there is provided a configuration having category dividing means for dividing the learning data, which is a learning target given as an input, into a plurality of categories according to the degree of learning.

According to the above configurations (1) and (2), in pattern recognition, dynamic characteristic learning of a target whose parameters are unknown due to nonlinearity, etc., the learning is performed by classification of the data pattern of each learning data or the number of data for each classification. Regardless of how easy it is, each data can be learned uniformly and at high speed. Also, when creating learning data, the ease of learning each data,
It is possible to save the trouble of creating learning data in consideration of the ratio of the number of data for each classification.

[0032]

Embodiments of the present invention will be described below in detail. (Embodiment 1) FIG. It is the figure which showed the outline of the N learning device, The same code | symbol is attached | subjected to the same function part as FIG. That is, 1 is N. N calculation means, 2 is a coupling coefficient updating means. Further, 3 is a category storage unit, 4 is a weighting coefficient updating unit, and 5 is a weighting unit. In addition, here, as in the case of the conventional example, FIG.
The three-layer N.M. A description will be given using N. As shown in FIG. N is a three-layer N.N. with m inputs and s outputs. N,
The number of elements in the intermediate layer is n, the output function of the intermediate layer is a sigmoid function, and the output function of the input / output layer is a linear function.

In FIG. 1, the category storage means 3 stores data patterns and N. A normalized learning data group, which is divided into a plurality of categories for each classification according to the size of N inputs, is stored for each category. The learning data stored in the category storage means 3 is N. N calculation means 1
Is input to the N.P. for each category as in the conventional example. N output is calculated. In the weighting coefficient updating means 4, the N.V. The weighting coefficient is updated for each category according to the squared error between the N output and the teacher signal and the number of data in each category.

First, the N.V. Sum of squared error between N output and teacher signal Ek (pk)
Is calculated as (Equation 11),

[0035]

[Equation 11]

When the number of data items of category k is Nk, the weighting coefficient αk for each category is calculated as shown in (Equation 12).

[0037]

(Equation 12)

Here, Yrk (pk) is the output of the r-th element of the output layer for the pk-th data of category k, drk (p
k) is a teacher signal for Yrk (pk).

Eavk is the average of the squared error sum Ek of each category, Eav is the average of the squared error sum E of all learning data, N is the total number of learning data, and Ek (pk) is the pk-th element of Ek. , E (p) represents the p-th element of E, and K represents the number of categories. The weighting coefficient αk may be changed as in (Equation 13).

[0040]

Αk = N / Nk Next, in the weighting means 5, the weighting coefficient αk is used to weight the sum Ek of the squared errors of each category as shown in (Equation 14).

[0041]

[Equation 14] Ek = αk · Ek The coupling coefficient updating means 2 updates the coupling coefficient by the back propagation method using the squared error sum Ek of each category calculated in (Equation 14) as an error function.

The coupling coefficient Wij from the i-th input layer element to the j-th intermediate layer element is given by (Equation 15).

[0043]

(Equation 15)

Is updated, and from the j-th intermediate layer element, r
The coupling coefficient Vjr to the th output layer element is as shown in (Equation 16).

[0045]

(Equation 16)

Is updated. In the above (Equation 15) and (Equation 16), μ is a learning coefficient and λ is a moment coefficient (0 <λ <1), which is usually set at the start of learning.

As described above, the number of data in each category and the N.V. Depending on the magnitude of the squared error between the N output and the teacher signal, weighting the sum of squared errors of the category with a small number of data or the category with a large error mean, It is possible to uniformly and rapidly learn each data regardless of the number of data for each classification such as patterns.

FIG. 2 shows the flow of the learning process. This process is repeated until the sum Esum of the squared errors of all the learning data becomes smaller than the predetermined value.

First, in STEP 21, a learning data group which is divided into a plurality of categories for each classification such as a data pattern in advance and is normalized is stored for each category.

Next, in STEP 22, the N.V. N is calculated for each category, and N. Calculate N output.

In STEP 23, N. The N.D. for all learning data, which is the sum of the squared error Ek between the N output and the teacher signal and the squared error Ek for each category. N
The sum Esum of the squared errors between the output and the teacher signal is calculated as in (Equation 17).

[0052]

[Equation 17]

Here, Ek (pk) represents the pk-th element of the vector Ek.

At STEP 24, the sum of squared errors Esum is compared with the learning end error ε set at the start of learning. If the sum of squared errors Esum is less than or equal to the learning end error ε, the learning is ended. If the learning end condition is not reached in STEP24, ST
Proceeding to EP25, the weighting coefficient is updated for each category according to the sum of squared errors Ek for each category and the number of data Nk for each category.

The weighting coefficient αk for data of a certain category k is calculated as in (Equation 12). The weighting coefficient αk may be changed as in (Equation 12).

At STEP 26, the squared error Ek of each category is weighted by the weighting coefficient αk for each category as shown in (Equation 14).

In STEP 27, the coupling coefficient Wij from the i-th input layer element to the j-th intermediate layer element is updated by the back propagation method as shown in (Equation 15), and the j-th intermediate layer element r The coupling coefficient Vjr to the th output layer element is
It is updated like (Equation 16). After finishing STEP27, return to STEP22.

In the present invention, the multi-input multi-output or multi-input one-output hierarchical N.M. It can be similarly used for learning N, and can be used when the intermediate layer output function is a nonlinear continuous function other than the sigmoid type and when the output layer output function is a nonlinear function. Is.

By performing the above-mentioned processing, each data can be processed uniformly and at a high speed regardless of the ease of learning of each learning data. N can be learned.

(Embodiment 2) FIG. It is the figure which showed the outline of the N learning apparatus, The same code | symbol is attached | subjected to the same function part as FIG. 6 is a category dividing means.

In FIG. 3, the category storage means 3 stores data patterns and N. A normalized learning data group, which is divided into a plurality of categories for each classification according to the size of N inputs, is stored for each category. The learning data stored in the category storage means 3 is N. The N.N. N output is calculated. In the weighting coefficient updating means 4, as in the first embodiment, the N.V. for each category is calculated by (Equation 12) or (Equation 13).
The weighting coefficient is updated for each category according to the sum Ek of the squared errors between the N output and the teacher signal and the number of data Nk in each category. The weighting means 5 weights the squared error of each category using the weighting coefficient according to (Equation 14) in the same manner as in the first embodiment, and the coupling coefficient updating means 2 weights the squared error by the weighting means. By using the square error of each category as an error function, the coupling coefficient is updated by (Equation 15) and (Equation 16) by the backpropagation method.

In the category dividing means 6, the types of categories to be divided, the division method, the hierarchical order, etc. are set in advance as shown in FIG. In addition, as a method of dividing the categories, the data pattern of the learning data and the type of division,
Division according to the size of the learning data input to each input layer element, or N.I. There are divisions according to the magnitude of the square error between the N output and the teacher signal.

For each category, the average of the sum Ek of the squared errors is compared with the reference value βk set in advance for each category. If Ek is less than or equal to the reference value βk, the data of that category is stored in the hierarchy of FIG. It is divided according to the above, and is stored again in the category storage means 3.

As described above, each category is set to N. It is divided according to the average size of the sum of the squared errors of the N output and the teacher signal, and according to the number of data of each category and the size of the squared error of each category, 2 of each category is divided.
By weighting the multiplication error, each data can be learned uniformly and at high speed regardless of the ease of learning of each learning data and the number of data for each classification such as a data pattern.

FIG. 5 shows a flow of learning processing. This process is repeated until the sum Esum of the squared errors of all the learning data becomes smaller than the predetermined value.

Since STEP 51 to STEP 54 are the same as STEP 21 to STEP 24 in the learning processing flow of FIG. 2, description thereof will be omitted. In STEP55, the average of the sum Ek of the squared errors for each category is compared with the reference value βk preset for each category. If Ek is equal to or smaller than the reference value βk, the data of the category is divided in STEP56 according to the hierarchy and division method of FIG. 4, and the procedure is returned to STEP51 to re-store the category.
If Ek of each category is more than the standard value, STEP5
Go to 7.

Steps 57 to 59 are the same as steps 25 to 27 in the learning processing flow of FIG. In STEP57,
The weighting coefficient αk is updated by (Equation 12) or (Equation 13) for each category according to the sum Ek of squared errors for each category and the number of data Nk in each category. In STEP58, the squared error sum Ek of each category is weighted as in (Equation 14) by the weighting coefficient αk for each category, and in STEP59, by the backpropagation method,
The coupling coefficient is updated by (Equation 15) and (Equation 16). After STEP59 ends,
Return to STEP52.

By performing the above-described processing, each data can be uniformly and rapidly learned regardless of the easiness of learning of each learning data and the number of data for each classification such as a data pattern.

[0069]

As described above, according to the first aspect of the present invention, in the pattern recognition or the learning of the object whose parameters are nonlinear and whose parameters are unknown, the learning data is preliminarily set to the data pattern or N. It is divided into a plurality of categories by classification according to the size of N inputs, the number of data in each category, and N. Weighting the squared error of the category with a small number of data or the category with a large error mean according to the magnitude of the squared error between the N output and the teacher signal makes it easier to learn each learning data and classifies the data pattern etc. Each data can be learned uniformly and at high speed regardless of the number of data for each.

Further, when creating the learning data, it is possible to save the trouble of creating the learning data in consideration of the easiness of learning each data and the ratio of the number of data such as each data pattern.

According to the invention of claim 2, by subdividing each category according to the degree of learning,
The learning can be performed more uniformly and at high speed.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of a neural network learning device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a learning processing flow according to the first embodiment of the present invention.

FIG. 3 is a schematic block diagram in Embodiment 2 of the neural network learning device of the invention.

FIG. 4 is a category hierarchy diagram of learning data according to the second embodiment of the present invention.

FIG. 5 is a diagram showing a learning processing flow according to the second embodiment of the present invention.

FIG. 6 is a schematic block diagram of a conventional neural network learning device.

FIG. 7 is an explanatory diagram of a neural network.

[Explanation of symbols]

1 ... N. N calculation means, 2 ... Coupling coefficient updating means, 3 ...
Category storage means, 4 ... Weighting coefficient updating means, 5
... Weighting means, 6 ... Category dividing means.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Masuo Takigawa 1006 Kadoma, Kadoma City, Osaka Prefecture Matsushita Electric Industrial Co., Ltd.

Claims (2)

[Claims] 1. A learning device for causing a neural network to learn a characteristic of a learning target, wherein the learning data, which is a learning target given as an input, is stored for each of a plurality of categories, and a category storage means is divided into a plurality of categories. A neural network computing means for computing a neural network on the learned data, and an output of the neural network computing means and a teacher signal.
Weighting coefficient updating means for changing and storing the weighting coefficient for each category according to the squared error and the number of data of each category, and weighting means for weighting the squared error according to the weighting coefficient, A neural network learning device, comprising: a coupling coefficient updating unit that obtains a coupling coefficient of a neural network from a squared error by a back propagation method. 2. A learning device for allowing a neural network to learn the characteristics of a learning target, wherein the learning data, which is a learning target given as an input, is divided into a plurality of categories according to the learning degree. Category storing means for storing learning data for each divided category, neural network calculating means for calculating a neural network for learning data divided into a plurality of categories, output of the neural network calculating means and teacher signal Weighting coefficient updating means for changing and storing the weighting coefficient for each category according to the squared error between and and the number of data in each category,
It further comprises weighting means for weighting the squared error in accordance with the weighting coefficient, and coupling coefficient updating means for obtaining a coupling coefficient of the neural network from the weighted squared error by a backpropagation method. Neural network learning device.