JPH07160660A - Neural network system - Google Patents

Abstract

PURPOSE:To accelerate the processing of additional learning and to easily construct a system by deciding a coupling coefficient between an additional output neuron and an intermediate layer based on a difference between the output of the additional output neuron and an additional teaching signal. CONSTITUTION:A learned neural network system 10, the additional output neuron (C4, C5) 11 and an arithmetic part 12 are provided and only additional data are supplied to the system 10 as input data. Then, the output of the additional output neuron 11 is supplied to the arithmetic part 12, the difference from the additional teaching signal is obtained, the difference is fed back and an additional coupling coefficient is decided. That is, the leaned neural network system 10 is provided with an input layer 2 (A1-A3), the intermediate layer 3 (B1-B4) and an output layer 4 (C1-C3), the coupling coefficient between the intermediate layer 3 and the output layer 4 is learned and established as shown by a solid line and the additional coupling coefficient indicated by a dotted line between the additional output neuron 11 (C4, C5) and the intermediate layer 3 (B1-B4) is decided.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network system which has a hierarchical structure and changes a coupling coefficient by learning to perform input / output conversion according to learning data, and particularly to speed up additional learning processing. Regarding the improvement of.

[0002]

2. Description of the Related Art A conventional neural network system is disclosed, for example, in Japanese Patent Laid-Open No. 4-180153. This neural network system has a hierarchical structure, and the coupling coefficient between layers is corrected by the backpropagation (BP) method. This B
The P method is to correct the coupling coefficient between the output layer and the intermediate layer one step before by the difference between the output from the output layer of the neural network system and the instruction value judged and instructed by a human, and to successively change the layers. It is a method of correcting the coupling coefficient between each layer by returning the difference to.

FIG. 4 shows a conventional neural network system 1 applied to temperature and humidity control by an air conditioner or the like. This system, for example, room temperature, room humidity,
Signals from the input layer 2 and the neurons A _i (i = 1 to 4) of the input layer 2 to which environmental conditions such as outdoor temperature and outdoor humidity are input are weighted by the coupling coefficient W _ij (j = 1 to 5). An intermediate layer 3 composed of neurons B _j that have been input
The signal from each neuron B _j of the intermediate layer 3 is a coupling coefficient V _ik.
The output layer 4 is composed of neurons C _k that are weighted and input with (k = 1 to 3). The output of each neuron C _k of the output layer 4 controls, for example, the wind direction, the air volume, and the blowing temperature of the air conditioner. When it is desired to change the inference output by the network, if the switch 5 on the key operation unit of the air conditioner is manually operated, the manual output signal has priority, and the output signal operates based on this. Then, during the manual operation, the difference calculation device 6 samples and outputs the difference between the inference output of the neural network system 1 (for example, the blast temperature that is the output of the neuron C ₂ ) and the manual setting at a fixed time,
It is stored in the register 7.

After the power is turned off, the difference stored in the register 7 is averaged by the averaging circuit 8. The output of the averaging circuit 8 is added by the adding means 9 to the output layer 4 stored in the RAM 10.
R in addition to each coupling coefficient V _j3 related to the neuron C ₂ of
Reset to AM10. That is, the coupling coefficient V _j3 is modified. On the other hand, each coupling coefficient W _ij of the intermediate layer 3 is written in the ROM 11 as a preset value.

[0005]

In the BP method of the above-mentioned conventional neural network system, when it is necessary to make the already learned neural network system learn another data again, the data learned so far. , And new additional data, and had to retrain all the data. For this reason, there is a drawback in that the additional learning process takes a very long time.

Explaining this point in detail, in the neural network system shown in FIG. 5, for example, the input of the neuron C ₂ of the output layer 4 = (the output of B ₁ × V ₁₂ ) + (B ₂ ×
Output × V ₂₂ ) + (B ₃ output × V ₃₂ ) C ₂ output = C ₂ input> threshold value → 1 C ₂ input ≦ threshold value → 0 or 1 / (1 + exp (−C ₂ output)

The BP method is a method for modifying the coupling coefficients W _ij and V _ij in such a neural network system so that the desired output is obtained with respect to the input, and this is learning. Therefore, for example, in the system of FIG. 5, when the output O ₃ is newly required for the two-dimensional inputs I ₁ and I ₂ , the conventional BP method uses O as the output for the inputs I ₁ and I ₂ . ₁ , O ₂ , O ₃ are prepared, all of them are relearned, and the coupling coefficients W _ij , V _ij and the threshold value θ _i ,
Correcting γ _i .

FIG. 6 shows an additional learning method in such a conventional neural network system based on the BP method. Reference numeral 5 denotes additional output neurons C ₄ and C ₅ .
Conventional data + additional data are given as input data, and all differences between each output of all output neurons and the teaching signal are fed back to correct all coupling coefficients.

An object of the present invention is to speed up the processing of additional learning and facilitate system construction in a neural network system based on the BP method.

[0010]

In order to achieve the above object, the present invention is a back propagation type neural network system having an input layer, an intermediate layer, and an output layer. A neural network system, wherein an additional output neuron corresponding to additional data input to the system is provided, and a coupling coefficient between the additional output neuron and the intermediate layer is defined as a difference between an output of the additional output neuron and an additional teaching signal. The summary is that it was decided based on.

[0011]

In the additional learning of the present invention, only the additional data is given as the input data to the learned neural network system, and the difference between the output of the additional output neuron and the additional teaching signal corresponding thereto is fed back to the intermediate layer. And the coupling coefficient between the and the additional output neuron is determined.

[0012]

EXAMPLES Examples of the present invention will be described below. Figure 1 is BP
In the conceptual diagram for explaining the additional learning of the present invention in the neural network system which adopts the learning method based on the method, the already learned output neurons O _{1 to} O _m are not considered.

FIG. 2 is a flow chart for explaining the additional learning of the present invention. In the figure, in step S1, the coupling coefficient V and the threshold γ is initialized, obtaining the output of each neuron B ₁ .about.B _k intermediate layer 3 by the input of additional data I ₁ ~I _n in step S2. Then step S3
So the desired additional output of the additional output neuron C _{m + 1} O _{m + 1}
The error between the (additional teaching signal) and the actual output _{O'm + 1} is obtained, and in step S4 it is determined whether or not this error is less than or equal to the target value. Then, based on the above error, the coupling coefficients V _{m + 1,1 to} V _{m + 1, k}
And the threshold value are corrected, and it is further determined in step S6 whether or not the number of times of learning described above has reached a set value (several thousands to tens of thousands). If not, the process returns to step S2, If so, the process ends.

FIG. 3 shows an embodiment of the additional learning method according to the present invention, in which 10 is a learned neural network system, 11 is an additional output neuron (C ₄ , C ₅ ), and 12 is an arithmetic unit. Only additional data is given as data. In response to this, the additional output neuron 1
The output of 1 is given to the calculation unit 12, and the difference from the additional teaching signal is obtained by this, and this difference is fed back,
The additional coupling coefficient is determined based on this.

In the embodiment of FIG. 3 described above, the learned neural network system 10 uses the input layer 2 (A ₁ ,
A ₂ , A ₃ ), the intermediate layer 3 (B _{1 to} B ₄ ) and the output layer 4
(C ₁ , C ₂ , C ₃ ), the coupling coefficient between the intermediate layer 3 and the output layer 4 is also learned and determined as shown by the solid line,
Additional output neuron 11 (C ₄ , C ₅ ) and intermediate layer 3 (B ₁
Although the determining additional coupling coefficient indicated by the dotted line between .about.B _4), the present invention is not limited to such embodiments, it can be applied to Examples as follows.

FIG. 11 shows another embodiment of the present invention. For example, when the neuron C ₁ of the output layer 4 is output as good, and when C ₂ is output as bad, the discrimination is performed, and when a completely new pattern that has not been learned until now is input, C Both ₁ and C ₂ may be output. In that case, an additional output neuron 11 (C ₄ ) is newly provided and the coupling coefficient between C ₄ and B _{1 to} B ₃ of the intermediate layer 3 is determined so that C ₄ outputs as in FIG. C is fixed
All of _{1 to} C ₄ are output and cannot be identified.

Therefore, in the embodiment shown in FIG. 11, all the neurons C _{1 to} C ₄ and the intermediate layer 3 of the output layer 4 are indicated by the dotted lines.
Is modified so as to determine the coupling coefficient between each neuron B _{1 to} B ₃ . Although this embodiment requires more labor than the embodiment of FIG. 3, it is more effective than the prior art because it is sufficient to modify the neurons in the intermediate layer 3 and the output layer instead of modifying all the neurons as in the prior art. .

12 and 13 show another embodiment of the present invention. When a new pattern which has not been learned so far is input, it may be output to either C ₁ or C ₂ . In that case, the neuron C ₁ which does not modify the output
Alternatively, the coupling coefficient between C ₂ and C ₂ is not changed, but the coupling coefficient between another output neuron C ₁ , C ₄ or C ₂ , C ₄ and the intermediate layer 3 is determined as shown by the dotted line. To do. In the present invention, in order to further reduce the learning time, it is preferable to perform the KL conversion of the input data and compress the input data.

7 and 8 show K- for this data compression.
FIG. 9A is a flowchart showing processing including L conversion, and FIG. 9A is a flowchart showing processing for K-L conversion.

FIG. 10 shows an example of a hardware configuration for realizing the neural network system of the present invention including the above-mentioned K-L conversion, which is not special hardware itself but a general processor such as a workstation. May be included. In FIG. 10, reference numeral 20 is input means for inputting data, 21 is memory for storing KL conversion matrix data, 22 is CPU, 23 is memory for storing data after KL conversion, and 24 is weighting (coupling coefficient) data. A memory for storage, 25 is output means, 26 is output data input means for difference calculation, and 27 is a data bus. With these, a KL conversion matrix data calculation section and a KL conversion execution section 28, and a neural network. A learning unit and an execution unit 29 are configured. These execution units 28 and 29 execute special learning of the neural network system of the present invention and calculations such as data compression by KL conversion.
In addition, you may perform the statistical process of input data before the above-mentioned K-L conversion.

[0021]

As described above, according to the present invention, in the neural network system, when the additional output neuron is provided and the additional learning is executed, the amount of calculation is smaller than that of the conventional additional learning. The processing speed becomes faster. In addition, since the operation principle of the additional learning of the present invention is simple, the system (program) can be easily constructed.

[Brief description of drawings]

FIG. 1 is a conceptual diagram for explaining additional learning of the present invention in a neural network system that adopts a learning method based on a BP method.

FIG. 2 is a flowchart for explaining additional learning of the present invention.

FIG. 3 is a block diagram showing an embodiment of the present invention.

FIG. 4 is a block diagram showing an example of a conventional neural network system.

FIG. 5 is a conceptual diagram of a neural network system.

FIG. 6 is an explanatory diagram of an additional learning method in a conventional neural network system.

FIG. 7 is a flowchart showing processing including KL conversion for data compression.

FIG. 8 is a flowchart showing a process including K-L conversion for data compression.

FIG. 9 is a flowchart showing a KL conversion process.

FIG. 10 is a block diagram showing an example of a hardware configuration for realizing the neural network system of the present invention.

FIG. 11 is a block diagram showing another embodiment of the present invention.

FIG. 12 is a block diagram showing still another embodiment of the present invention.

FIG. 13 is a block diagram showing still another embodiment of the present invention.

[Explanation of symbols]

 1 Neural Network System 2 Input Layer 3 Intermediate Layer 4 Output Layer 10 Learned Neural Network System 11 Additional Output Neuron 12 Operation Unit

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] July 1, 1994

[Procedure Amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0006

[Correction method] Change

[Correction content]

To explain this point in detail, in the neural network system shown in FIG. 5, for example, the input to the neuron C _{2 in the} output layer 4 = (the output of B ₁ × V ₁₂ ) +
(B ₂ × output × V ₂₂ ) + (B ₃ output × V ₃₂ ) C ₂ output = C ₂ input> threshold value → 1 C ₂ input ≦ threshold value → 0 or == 1 / (1 + exp (output -C ₂₎₎

[Procedure Amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0019

[Correction method] Change

[Correction content]

7 and 8 show K for this data compression.
FIG. 9 is a flow chart showing a process including -L conversion .
It is a flowchart which shows the process of L conversion.

[Procedure 3]

[Document name to be corrected] Drawing

[Correction target item name] Figure 9

[Correction method] Change

[Correction content]

[Figure 9]

Claims (4)

[Claims] 1. A back-propagation type neural network system having an input layer, an intermediate layer, and an output layer, wherein the system is a learned neural network system, and additional data input to the system. And a coupling coefficient between the additional output neuron and the intermediate layer is determined based on a difference between the output of the additional output neuron and the additional teaching signal. 2. A KL conversion means for compressing input data and inputting the compressed data to the input layer.
The neural network system described in. 3. A backpropagation type neural network system having an input layer, an intermediate layer and an output layer, wherein a neuron corresponding to additional data input to the system is provided and the additional output neuron is provided. Coupling coefficients between the intermediate layer and between the output layer and the intermediate layer,
A neural network system, which is determined based on a difference between an output of each additional output neuron and each neuron of an output layer and a teaching signal. 4. A backpropagation type neural network system having an input layer, an intermediate layer, and an output layer, wherein the system requires modification of outputs of some neurons with respect to the output layer. In the system, an additional output neuron corresponding to additional data input to the system is provided, and a coupling coefficient between the additional output neuron and the intermediate layer and an output layer neuron and an intermediate layer requiring an output modification are provided. A neural network system characterized in that a coupling coefficient is determined based on a difference between an output of a non-learned output neuron of an additional output neuron and an output layer and a teaching signal.