JP3359074B2 - Learning method of neural network - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for learning a neural network (hereinafter referred to as NN).

[0002]

2. Description of the Related Art Neurocomputers focus on neurons (neural cells), which are basic elements of the brain.
The idea is to achieve a function similar to that of the brain, using as a hint the NN that results from the combination of these. The feature of these NNs lies in the parallel information processing between the neurons and the ability to learn. The NN has a hierarchical structure composed of several layers, each layer is composed of an appropriate number of cells, there is no coupling in the same layer, and the coupling between each layer is from the input layer (first layer) to the output layer ( The connection is made in one direction toward the final layer). The cells of each layer except the input layer receive weighted inputs from the cells of the previous layer, calculate the sum thereof, and multiply the sum by an appropriate function f to output. Cell input / output functions used in the NN include a threshold function, a piecewise linear function, a logistic function, an identity function, and the like (references, published by Sangyo Tosho, Hideki Aso, "Neural Network Information Processing", page 13).

[0003]

If an interval linear function is used as an input / output function of cells constituting the NN, a membership function composed of straight lines can be created by the NN. The membership function composed of straight lines has an advantage that hardware can be easily realized. SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and has been developed for fuzzy inference having a cell for outputting a limit output of an addition result of an input signal.
It is intended to provide.

[0004]

According to a first aspect of the present invention, there is provided a neural network learning method, comprising connecting a cell for outputting an input signal and a signal of a cell for outputting the input signal via a predetermined coupling coefficient. A plurality of intermediate cells and a neural network in which these intermediate cells have a hierarchical structure and are connected to a plurality of final output cells, wherein the intermediate cells have a limit area where the output is constant at a predetermined value in a certain input section. A cell that outputs a piecewise linear function having at least one of the following: a means for outputting an addition result of a signal coupled to an input end in accordance with the piecewise linear function, and updating a coupling coefficient using an error back propagation method At this time, when the output signal of the intermediate cell is within the limit area of the piecewise linear function, the error is not propagated. Further, the neural network learning method according to claim 2 includes a cell for outputting an input signal, a plurality of intermediate cells for connecting the signal of the cell for outputting the input signal via a predetermined coupling coefficient, and In a neural network in which intermediate cells have a hierarchical structure and are connected to a plurality of final output cells,
The intermediate cell is a cell that outputs a piecewise linear function having the following three regions, and has means for outputting an addition result of a signal coupled to an input terminal according to the piecewise linear function,
When the coupling coefficient is updated using the error backpropagation method, when the output signal of the intermediate cell is in any region of the piecewise linear function, the error is propagated at a predetermined ratio. Note 1. A low limit region in which the cell input is less than or equal to a first predetermined value and the cell output is limited by the first predetermined value. 2. A cell input is between the first predetermined value and a second predetermined value greater than the first predetermined value, and a cell output is a second power greater than the first predetermined value and the second predetermined value. A linear region that increases linearly with a predetermined slope to a predetermined value. 3. A high limit region in which the cell input is equal to or greater than a second predetermined value and the cell output is limited by the second predetermined value.

[0005]

An embodiment will be described below with reference to the drawings. FIG.
1 is a configuration diagram of an embodiment of an NN according to the present invention. In FIG. 1, a is a cell that outputs a piecewise linear function, O1 to O
_jmx is a cell b _j (j = 1 to 2) connected to the input terminal of the cell a.
Output of _mx), W1~W _jmx is the coupling coefficient. At this time, the cell a outputs a value represented by the following equation according to the value of the input sum I.

(Equation 1) The input / output function of this cell is shown in FIG. As shown in FIG. 3, three sections of the cell input I, I <x1, x1 ≦ I ≦
x2 and x2 <I are referred to as low limit, linear, and high limit areas, respectively (when y1 ≦ y2, the low limit and the high limit are switched when y1> y2). In addition, in order to further clarify each of the above-mentioned regions, the regions were defined as words as follows. (A) Low limit area An area where the horizontal axis is a cell input, and when the cell input is divided into three by x1 and x2, the cell input is equal to or less than x1 and the cell output is limited by a predetermined value y1. (B) Linear region A region where the cell input is between x1 and x2 and the cell output increases linearly with a predetermined slope from y1 to y2 on the vertical axis. (C) High limit area An area where the cell input is equal to or larger than x2 and the cell output is limited by a predetermined value y2 on the vertical axis.

Next, a learning method using this cell will be described with reference to FIG. Here, Wi (k-1), j (k) is k-
The coupling coefficient between the i-th cell in the first layer and the j-th cell in the k-th layer is indicated.
Oj (k) indicates the output of the j-th cell in the k-th layer. Coupling coefficient Wij
The following equation is used for updating.

(Equation 2) A normal cell (sum x of the input to the cell and y of the cell is
A cell having a relation of a differentiable input / output function y = f (x) is represented by
Let's call it a normal cell. Specifically, reference (1)
The following is a description of the difference between the cell backpropagation method and the cell error backpropagation method according to the present invention.

(Equation 3) dj (k) is divided in the following cases.

(I) When the k-th layer is the output layer

(Equation 4)

(Ii) When the k-layer is an intermediate layer

(Equation 5) Dm (k + 1) in the above equation is obtained in the same manner as dj (k) in order from the output side.

(Equation 6) Therefore, learning is possible even in the NN including the cell according to the present invention.

[0009]

As described above, according to the present invention, N
Since the function of outputting the addition result of the input signal according to the piecewise linear function is added to the cells constituting N, it is effective to directly assign a software neural net to hardware.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an embodiment of an NN according to the present invention.

FIG. 2 is a view for explaining the learning method of FIG. 1;

FIG. 3 is a diagram showing an input / output function.

[Explanation of symbols]

a, b _{1 to} b _jmx cell Output of O _{1 to} O _jmx cell b _j W _{1 to} W _jmx coupling coefficient

Claims (2)

(57) [Claims] And 1. A cell for outputting an input signal, a plurality of intermediate cells of the cell signal and outputting the input signal is connected through a predetermined coupling coefficient, these intermediate cells has a hierarchical structure in neural network connected to a plurality of final output cell, you said intermediate cell is input section
And the limit area where the output is constant at a predetermined value
A cell which outputs a piecewise linear function having at least one, and has means for outputting the addition result of the signal coupled to the input terminal in accordance with the piecewise linear function, and updates the coupling coefficient using the error back propagation method At this time, the output signal of the intermediate cell is
A method for learning a neural network, wherein an error is not propagated when the error is within a limit area of a piecewise linear function . 2. A cell for outputting an input signal, a plurality of intermediate cells of the cell signal and outputting the input signal is connected through a predetermined coupling coefficient, these intermediate cells has a hierarchical structure in connection neural network to a plurality of final output cell, the intermediate cell following three regions
A cell that outputs a piecewise linear function having a means for outputting the addition result of the signal coupled to the input terminal in accordance with the piecewise linear function, and when updating a coupling coefficient using an error back propagation method, The output signal of the intermediate cell is
A method for learning a neural network, wherein an error is propagated at a fixed ratio when it is in any region of a shape function . Note 1. The cell input is less than or equal to the first predetermined value and the cell output is
Low limit area limited by a predetermined value of 1. 2. The cell input is different from the first predetermined value and the first predetermined value.
Cell output is greater than the second predetermined value, and the cell output is
A predetermined value of 1 and a second predetermined value larger than the first predetermined value
Up to a linear function with a predetermined slope
region. 3. Cell input is greater than or equal to a second predetermined value and cell output is
The high limit area limited by the second predetermined value.