JP3304476B2 - Neural network construction method, learning method, and generalization ability evaluation method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention has a hierarchical structure including an input layer neuron unit, an intermediate layer neuron unit, and an output layer neuron unit, and constitutes a neural network in which learning on recognition categories is performed. The present invention relates to a method, a method for performing the learning, and a method for evaluating the generalization ability.

[0002]

2. Description of the Related Art Neural networks are structurally classified into those belonging to a hierarchical type and those belonging to a mutual connection type. Among them, a typical one belonging to the hierarchical type has a three-layer structure including an input layer neuron unit, an intermediate layer neuron unit and an output layer neuron unit.

In a neural network having a three-layer structure, usually, a plurality of input layer neuron units, one or more intermediate layer neuron units connected to each of them, and an intermediate layer neuron unit And one or more output layer neuron units coupled to the input layer neuron unit, and the synapse transmission efficiency between each of the input layer neuron units and the intermediate layer neuron unit coupled thereto (hereinafter referred to as input / intermediate interlayer synaptic transmission). Efficiency), and the synaptic transmission efficiency between one intermediate layer neuron unit or a plurality of intermediate layer neuron units and the output layer neuron unit connected thereto (hereinafter referred to as the intermediate / output interlayer synaptic transmission efficiency). ) Takes a predetermined value. When an input signal related to a predetermined recognition category is supplied to each of the input layer neuron units, an output signal responsive to the recognition category is obtained from the output layer neuron unit.

[0004] Initial values for each of the input and intermediate interlayer synapse transmission efficiencies and the intermediate and output interlayer synapse transmission efficiencies are first set, and the initial values are set. The learning is performed and determined using a plurality of types of input signals related to the recognition category. Such learning is performed, for example, when an input signal relating to a specific recognition category is supplied to an input layer neuron unit, an output signal obtained in response to the input layer neuron unit is previously stored in correspondence with the supplied input signal. This is performed by changing the values of the input and intermediate interlayer synapse transmission efficiencies and the intermediate and output interlayer synapse transmission efficiencies so as to approach the proper signal (teacher signal) that has been set. At this time, when the input signal is supplied to the input layer neuron unit, the output signal obtained from the output layer neuron unit is made closer to the teacher signal so that the synapse transmission efficiency between the input and intermediate layers and the intermediate and
Learning to change each value of the output inter-layer synaptic transmission efficiency is usually repeated a plurality of times, and as a result of the finite number of learnings, the output layer neuron unit responds to the input signal supplied to the input layer neuron unit. The state in which the output signal obtained in step (1) is sufficiently close to the teacher signal is called a state in which learning converges (learning convergence state).

[0005]

In a neural network having a three-layer structure as described above, in the course of its construction,
Prior to learning with multiple types of input signals related to the recognition category according to a method such as back propagation, the input / intermediate interlayer synapse transmission efficiency and intermediate /
The setting of each initial value of the output interlayer synaptic transmission efficiency affects the generalization ability of the neural network. In the conventional neural network configuration method, each initial value of the input / intermediate interlayer synapse transmission efficiency and the intermediate / output interlayer synapse transmission efficiency is, for example, the input value regardless of the specific structure of the neural network.・ The initial values of the synaptic transmission efficiency between the intermediate layers and the synapse transmission efficiency between the intermediate and output layers are W
_{It is set such that IJ, the} threshold value is _Vth, and a random number from 0 to 1 is rand, and _WIJ = (rand-0.5) + _Vth .

Therefore, there are a plurality of types of neural network internal structures capable of obtaining a learning convergence state, and learning according to a method such as back propagation proceeds in a direction of recognizing a recognition category related to an input signal. There are cases where convergence takes place with a neural network internal structure and cases where convergence takes place with a different neural network internal structure.In the latter case, the reliability of the output signal obtained in the output layer neuron unit is reduced. A low, unpractical neural network will be formed. That is, the case where the neural network after learning has a high generalization ability to appropriately identify the recognized recognition category and the other recognition categories,
There is a case where the recognition category that has undergone learning is hardly distinguished from the other recognition categories, and the generalization ability is extremely low.

In view of the above, the present invention belongs to a hierarchical type having an input layer neuron unit, an intermediate layer neuron unit and an output layer neuron unit, and as a result of learning with a plurality of types of input signals related to a recognition category, always obtains. To provide a method of constructing a neural network capable of obtaining a neural network having an internal structure having excellent generalization ability, and further comprising an input layer neuron unit and an intermediate layer neuron unit And a learning method that makes it possible to relatively easily evaluate the generalization ability of a neural network belonging to a hierarchical type having an output layer neuron unit, and a neural circuit under which the learning method is executed It is another object of the present invention to provide a method for evaluating network generalization ability.

[0008]

In order to achieve the above object, a first aspect of the method for constructing a neural network according to the present invention is to connect a plurality of input layer neuron units to each of a plurality of input layer neuron units. The input or output intermediate neuron units and the output neuron units connected to the intermediate layer neuron unit are expressed by an inequality: 0 < f (x) ≦ 1 is established,
The initial value of the input / interlayer synaptic transmission efficiency is set so that when all the inputs to the input layer neuron unit are at the preset minimum level, the output of the intermediate layer neuron unit is set to the preset minimum level, When the output of one intermediate layer neuron unit or a plurality of intermediate layer neuron units is the preset minimum level, the output of the output layer neuron unit is set in advance. Under the setting of the highest level set, learning by backpropagation method is used to determine the value of the input / middle layer synaptic transmission efficiency and the value of the intermediate / output layer synaptic transmission efficiency. It is said.

A second aspect of the method for constructing a neural network according to the present invention is a method for constructing a neural network, comprising the steps of:
One intermediate-layer neuron unit connected to each of the input-layer neuron units and a plurality of output-layer neuron units connected to the intermediate-layer neuron unit are expressed by an inequality of an input / output function f (x) for each: 0 <f
(x) A plurality of unit three-layer neural networks having conditions satisfying ≦ 1 are prepared in correspondence with a plurality of recognition categories, respectively, and for each of the unit three-layer neural networks, the input / intermediate interlayer synaptic transmission efficiency is provided. Is set so that when the inputs to the input layer neuron units are all at the preset minimum level, the output of the intermediate layer neuron unit is set to the preset minimum level.
The initial value of the output inter-layer synaptic transmission efficiency is set so that when the output of the intermediate layer neuron unit is at the preset minimum level, the output of the output layer neuron unit is set to the preset maximum level. Learning about the recognition category to be performed by the method of back propagation, determine the value of the synaptic transmission efficiency between the input and intermediate layers and the value of the synaptic transmission efficiency between the intermediate and output layers,
For a plurality of unit three-layer neural networks on which learning has been performed, a plurality of input layer neuron units in each of them are shared, and a plurality of output layer neuron units in each of them are shared, thereby forming one three-layer neural network. It is supposed to get a net.

On the other hand, a method for learning a neural network according to the present invention comprises a plurality of input layer neuron units, and a plurality of intermediate layer neurons respectively connected to a plurality of recognition categories and connected to the input layer neuron units. Unit, and a plurality of output layer neuron units coupled to each of the hidden layer neuron units, and an input / output function f (x) for each of the input layer neuron unit, the hidden layer neuron unit and the output layer neuron unit Is an inequality: For a three-layer neural network that satisfies 0 <f (x) ≦ 1, the initial value of the input / interlayer synaptic transmission efficiency is set to the minimum value in which all inputs to the input layer neuron unit are preset Level, the output of the hidden neuron unit is set to be a preset minimum level. The initial value of the synapse transmission efficiency between the intermediate and output layers is set so that the output of the output layer neuron unit becomes the predetermined maximum level when the output of the intermediate layer neuron unit is the predetermined minimum level. Then, learning about each of the plurality of recognition categories is performed by a back-propagation method using only the synaptic transmission efficiency relating to one intermediate layer neuron unit corresponding to each of the plurality of recognition categories as a learning target. -The value of the intermediate layer synapse transmission efficiency and the value of the intermediate and output layer synapse transmission efficiency are determined.

Further, the method for evaluating the generalization ability of a neural network according to the present invention includes a plurality of input-layer neuron units and a plurality of input-layer neuron units respectively corresponding to a plurality of recognition categories. An intermediate layer neuron unit, and a plurality of output layer neuron units coupled to each of the intermediate layer neuron units, and an input / output function f for each of the input layer neuron unit, the intermediate layer neuron unit, and the output layer neuron unit For a three-layer neural network in which (x) satisfies the inequality: 0 <f (x) ≦ 1, the initial value of the input / interlayer synaptic transmission efficiency is preset for all inputs to the input layer neuron unit. Set the output of the hidden neuron unit to the preset minimum level In addition, the initial value of the intermediate / output interlayer synaptic transmission efficiency is set so that the output of the output layer neuron unit becomes the preset maximum level when the output of the intermediate layer neuron unit is the preset minimum level. Originally, learning about each of a plurality of recognition categories is performed by a back propagation method with only the synaptic transmission efficiency relating to one intermediate layer neuron unit corresponding to each of the plurality of recognition categories as a learning target. , The value of the input / middle layer synaptic transmission efficiency and the value of the intermediate / output layer synapse transmission efficiency are determined, and after the learning is established, each of the intermediate layer neuron units takes the on-output state. The ratio between the ON output and the OFF output is calculated, and the three-layered god is determined based on the calculated ratio. The method is characterized in that a generalization ability of a network is determined.

[0012]

In the first aspect of the above-described neural network configuration method according to the present invention, the initial value of the input / intermediate interlayer synaptic transmission efficiency is set to a minimum value at which all inputs to the input layer neuron unit are preset. When the level is at the level, the output of the intermediate layer neuron unit is set to be a predetermined minimum level, and the initial value of the intermediate / output interlayer synaptic transmission efficiency is set to one intermediate layer neuron unit or a plurality of intermediate layer neuron units. When the output of each of the layer neuron units is at the preset lowest level, the output of the output layer neuron unit is set to be at the preset maximum level. Then, based on the setting of the initial value of the input / intermediate interlayer synapse transmission efficiency and the initial value of the intermediate / output interlayer synapse transmission efficiency, learning by the back propagation method is performed, and the input / intermediate interlayer synapse transmission efficiency is determined. The value of the efficiency and the value of the intermediate / output interlayer synaptic transmission efficiency are determined. Therefore, the configured neural network always has an internal structure with excellent generalization ability.

In a second aspect of the method for constructing a neural network according to the present invention, for each unit three-layer neural network, the initial value of the input / interlayer synaptic transmission efficiency is set to the input layer neuron unit. Are set to bring the output of the hidden neuron unit to the preset minimum level when all the inputs to are at the preset minimum level,
Also, the initial value of the transmission efficiency between the intermediate and output layers is 1
When the outputs of the intermediate layer neuron units are at the preset lowest level, the learning for the corresponding recognition category is back-proportional under the condition that the output of the output layer neuron unit is set to the preset maximum level. The value of the synapse transmission efficiency between the input and intermediate layers and the value of the synapse transmission efficiency between the intermediate and output layers are determined by the gating method. Then, for a plurality of unit three-layer neural networks on which such learning has been performed, a plurality of input layer neuron units are shared in each of the plurality of output layer neuron units, and a plurality of output layer neuron units in each are shared. Is constructed. Therefore,
A three-layer neural network, which always has an internal structure with excellent generalization ability, can be relatively easily obtained with rapid learning.

According to the neural network learning method of the present invention, a plurality of input layer neuron units, a plurality of intermediate layer neuron units respectively corresponding to a plurality of recognition categories, and a plurality of output layers are provided. When the three-layer neural network having the neuron unit has an initial value of the input / intermediate interlayer synaptic transmission efficiency, all the inputs to the input layer neuron unit are at the preset minimum level,
When the output of the intermediate layer neuron unit is set to be the preset minimum level, and the initial value of the intermediate / output interlayer synaptic transmission efficiency is the preset minimum level of the output of the intermediate layer neuron unit. ,
Under the condition that the output of the output layer neuron unit is set to be a preset highest level, learning for each of the plurality of recognition categories involves one intermediate layer neuron unit corresponding to each of the plurality of recognition categories. The synapse transmission efficiency is learned only by the back propagation method, and the value of the input / middle layer synapse transmission efficiency and the value of the intermediate / output layer synapse transmission efficiency are determined. Therefore, the value of the input / interlayer synaptic transmission efficiency after learning and the
The generalization ability of the three-layer neural network can be relatively easily evaluated based on the value of the output interlayer synaptic transmission efficiency.

Further, according to the method for evaluating the generalization ability of a neural network according to the present invention, a plurality of input layer neuron units, a plurality of intermediate layer neuron units respectively corresponding to a plurality of recognition categories, and a plurality of When the initial value of the input / intermediate interlayer synaptic transmission efficiency is the minimum level of all the inputs to the input layer neuron unit, the three-layer neural network having the output layer neuron unit The output of the output layer neuron is set when the output of the intermediate layer neuron unit is set to the preset minimum level, and the initial value of the intermediate / output interlayer synaptic transmission efficiency is the preset minimum level. Each of the multiple recognition categories, with the unit output set to the highest preset level Learning of regarding, performed by a technique of back propagation plurality of recognition categories each only synaptic transmission efficiency involved in one hidden neuron unit corresponding to a learning target, whereby,
The values of the input and intermediate interlayer synapse transmission efficiencies and the values of the intermediate and output interlayer synapse transmission efficiencies are determined, and furthermore, after learning is completed, each of the intermediate layer neuron units will be in the on-output state. The ratio between the output and the off output is determined, and the generalization ability is determined based on the ratio. Therefore,
Evaluation of the generalization ability of the three-layer neural network will be performed quickly and relatively easily.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A case where a three-layer neural network is constructed according to a first example of a neural network constructing method according to the present invention will be described below.

First, as shown in FIG. 1, 64 input layer neuron units SA1 to SA64, five intermediate layer neuron units AA1 to AA5 each connected to each of the input layer neuron units SA1 to SA64,
And each of the intermediate layer neuron units AA1 to AA5
And 16 output-layer neuron units RA1 to RA16 connected to each of them. At this time, the input / output function f (x) for each of the input layer neuron units SA1 to SA64, the intermediate layer neuron units AA1 to AA5, and the output layer neuron units RA1 to RA16.
For the inequality: 0 <f (x) ≦ 1 holds. Specifically, for example, f (x) = 1 / (1 + exp (−x−50)) (1), where 1 / (1 + exp (−x−50))> 0.9. Sometimes f (x) = 0.9 and 1 / (1 + exp (−x−5
0)) <0.1, f (x) = 0.1.

Subsequently, the input layer neuron unit SA1
To SA64, and the initial value W _IJ (IH) of the input / interlayer synaptic transmission efficiency wij (IH) between any one of the intermediate layer neuron units AA1 to AA5 coupled thereto. Are all at the lowest level, that is, 0.1, the intermediate layer neuron unit AA1
To AA5 at the lowest level, that is,
0.1, and the initial value of the intermediate / output interlayer synaptic transmission efficiency wij (H0) between each of the intermediate layer neuron units AA1 to AA5 and any of the output layer neuron units RA1 to RA16 coupled thereto. The value W _IJ (HO) is converted to the intermediate layer neuron units AA1 to AA.
When all the output signals of No. 5 are at the lowest level, that is, 0.1, the output signals DO1 to DO16 from the output layer neuron units RA1 to RA16 are set to the highest level, that is, 0.9.

Specifically, for example, the reference value for the input / intermediate interlayer synapse transmission efficiency wij (IH) is wo (IH), and the reference value for the intermediate / output interlayer synapse transmission efficiency wij (HO) is wo (HO). ), OFF output of each neuron unit is OL = 0.1, number of input layer neuron units is NI (= 64), number of hidden layer neuron units is NH = (5), inside of hidden layer neuron unit in off output state The potential is mpL (H), the internal potential of the output layer neuron unit in the ON output state is mpH (O), 0 to 1
A random number to rand (0 ≦ rand ≦ 1) , as, wo (IH) · OL · NI <mpL (H), W IJ (IH) = wo (IH) - rand ··· (2) wo (HO ) · OL · NH> mpH ( O), W IJ (HO) = wo (HO) - form as relationship rand ··· (3) is satisfied. Then, mpL (H) and mpH (O) are calculated from equation (1), and mpL (H) = − 52.1972 and mpH (O) = − 47.8028, so that equations (2) and (3) ), Wo (IH) · OL · NI = wo (IH) · 0.1 · 64 <mpL (H) = − 52.1972, wo (HO) · OL · NH = wo (HO) · 0.1 · 5> mpH (O ) = − 47.8028. Therefore, wo (IH) =-50, wo (HO) =-50
From this, W _IJ (IH) = − 50−rand and
_WIJ (HO) = − 50−rand is set.

In this way, the initial value W _IJ (IH) of the input / middle layer synaptic transmission efficiency wij (IH) and the initial value W _IJ (HO) of the intermediate / output layer synapse transmission efficiency wij (H0) are set. Then, learning by the back propagation method is performed, and thereby, the input / intermediate interlayer synaptic transmission efficiency wij
(IH) value and intermediate / output interlayer synaptic transmission efficiency wij (H0)
Determine the value of. The learning by the back propagation method is performed, for example, assuming that the formed three-layer neural network functions as an alphabet pattern recognition device. The input signals DI1 to DI64 input to the input layer neuron units SA1 to SA64 are For example, the data signal is a 64-bit data signal representing an alphabet pattern, and a 16-bit teacher signal is set for the data signal. The learning algorithm is, for example, δW ^k−1 _i ^k _j (t + l) = −0.2 d ^k _j O ^k−1 _i + 0.2δW ^k−1 _i ^k _j (t). Where W ^k-1 _i ^k
_j is the synaptic transmission efficiency value from the i-th neuron unit in the k-1 layer to the j-th neuron unit in the k layer, where δW ^k-1 _i ^k _j (t) and δW
^k-1 _i ^k _j (t + l) is W at times t 1 and t + 1, respectively.
represents the correction amount of ^k-1 _i ^k _j, O ^k-1 _i represents the output of the i-th neuron units in k-1 layer, d ^k _j is the modification of W ^k-1 _i ^k _j Represents the learning signal used.

At the time of learning, the input signals DI1 to DI6
4, when arranged as shown in FIG. 2, a signal group GDI1 which takes "0" and "1" as shown in FIG. 3A and represents "A" without including noise, and 9, "B", "B", "C", "D" and "E" as shown in part, "0" and "1" are taken to represent "A" including two bits of noise. Signal group GDI2
GDI 10 is selected. Nine types of signal groups GDI2 to GDI10 representing "A" including noise of 2 bits
Are the two-bit noise in each of the input signals DI1
Of two to DI64 selected according to random numbers, the place that should be "0" is replaced with "1", or
It is caused by inverting what should be "1" to "0". Further, as the teacher signal, for example, “A”
16-bit EUC (Enhanced Unix Code): 0 0 1 0 0 0 1 1 0 1 0 0 0 0 0 1
Is used.

In each of the signal groups GDI1 to GDI10, "0" represents the lowest level: 0.1, and "1" represents the highest level: 0.9. The convergence of the learning is determined, for example, by determining that the learning has converged when the total bit error is equal to or less than 0.01. In the embodiment of learning, one of the signal groups GDI1 to GDI10 is determined. The learning under supply is repeated until convergence, and after convergence, the learning under supply of another one of the signal groups GDI1 to GDI10 is repeated until convergence. Learning for each of the GDIs 10 is sequentially performed.

According to the input of the present invention,
The initial value W _IJ (IH) of the intermediate layer synaptic transmission efficiency wij (IH) and the initial value W of the intermediate / output layer synapse transmission efficiency wij (H0)
The setting of _IJ (HO) and the learning using the signal groups GDI1 to GDI10 as the input signals DI1 to DI64 were actually performed. As a result, the initial value W _IJ (IH) of the input / middle layer synaptic transmission efficiency wij (IH) is determined as shown in FIG. W _IJ (HO) is set as shown in FIG. 5, the value of the synapse transmission efficiency wij (IH) between the input and intermediate layers is determined as shown in FIG. The value of synaptic transmission efficiency wij (H0) is
The determination was made as shown in FIG.

In FIG. 4, 1 to 64 of i represent input layer neuron units SA1 to SA64, respectively, and 1 to 5 of j represent intermediate layer neuron units AA1 to AA5, respectively.
To represent. In FIG. 5, 1 to 5 of i represent intermediate layer neuron units AA1 to AA5, respectively, and j
Are output layer neuron units RA1 to R, respectively.
Represents A16. That is, for example, the initial value W of the input / interlayer synaptic transmission efficiency wij (IH) between the input layer neuron unit SA1 and the intermediate layer neuron unit AA1.
_IJ (IH) is represented by i = 1, j = 1 in FIG. 4 and is -50.51, and the intermediate / output interlayer synapse transmission efficiency wij between the intermediate layer neuron unit AA5 and the output layer neuron unit RA16. (HO) initial value W _IJ
(HO) is represented by i = 5, j = 16 in FIG. 5 and is -50.41.

In FIG. 6, 1 to 64 of i represent input layer neuron units SA1 to SA64, respectively, and 1 to 5 of j represent intermediate layer neuron units AA1 respectively.
~ AA5. Further, in FIG.
Denote intermediate layer neuron units AA1 to AA5, respectively, and 1 to 16 of j denote output layer neuron units R, respectively.
A1 to RA16 are shown. That is, for example, the input layer neuron unit SA1 and the intermediate layer neuron unit AA
The value of the input / interlayer synaptic transmission efficiency wij (IH) between 1 and −1 is expressed by i = 1 and j = 1 in FIG. 6 and is −95.17, and the intermediate layer neuron unit AA5 and the output layer neuron The value of the intermediate / output inter-layer synaptic transmission efficiency wij (HO) with the unit RA16 is represented by i = 5, j = 16 in FIG.
It is.

Input layer neuron units SA1 to SA6
4, having intermediate layer neuron units AA1 to AA5 and output layer neuron units RA1 to RA16;
The value of the intermediate interlayer synaptic transmission efficiency wij (IH) was determined as shown in FIG. 6, and the value of the intermediate / output interlayer synapse transmission efficiency wij (H0) was determined as shown in FIG. The inventors of the present application performed a generalization ability test on the three-layer neural network obtained according to the first example of the method for constructing a neural network according to the present invention. Such a generalization ability test shows that when the input signals DI1 to DI64 are arranged as shown in FIG. 2, "0" and "1" as shown partially in FIGS. 8A and 8B respectively. To represent "A" including noise of 2 bits.
Type signal groups GDI11 to GDI20 and A in FIG.
And B respectively take on "0" and "1" as shown in part, and "B" to "B"
25 signal groups GDI21-G each representing "Z"
DI45 is selected and these signal groups GDI11 to GD
Each of I45 is an input layer neuron unit SA1 to SA
64, the output layer neuron unit R
Output signals DO1 to DO16 obtained from A1 to RA16
Was tested. The signal group GDI
In each of 21 to GDI 45, "0" represents the lowest level: 0.1, and "1" represents the highest level: 0.9.

As a result of the generalization ability test, ten kinds of signal groups G representing "A" including noise of 2 bits
DI11 to GDI20, and 25 types of signal groups GDI each representing “B” to “Z” without including noise
21 to GDI 45 are correctly recognized, and in particular, the output signals DO1 to DO16 are input under the input of each of the 25 types of signal groups GDI21 to GDI45 representing “B” to “Z” without noise. All achieved the highest level: 0.9. Thus, it has been experimentally confirmed that the three-layer neural network obtained according to the above-described first example of the method for configuring a neural network according to the present invention has high generalization ability. .

The second embodiment of the method for constructing a neural network according to the present invention.
In the case of forming a three-layer neural network according to the example of FIG. 1, for example, as shown in FIG. 1, 64 input layer neuron units SA1 to SA64 are connected to each of the input layer neuron units SA1 to SA64. Five intermediate layer neuron units AA1 to AA5 and 16 output layer neuron units RA1 to RA each connected to each of the intermediate layer neuron units AA1 to AA5.
16 and the input layer neuron units SA1 to SA
64, the input / output function f (x) for each of the intermediate layer neuron units AA1 to AA5 and the output layer neuron units RA1 to RA16 is based on the inequality: 0 <f (x) ≦ 1. , The initial value W _IJ (IH) of the synaptic transmission efficiency w ij (IH) between the input and intermediate layers is converted to the input signal D input to the input layer neuron units SA1 to SA64, respectively.
When I1 to DI64 are all at the lowest level: 0.1, the output signals in each of the intermediate layer neuron units AA1 to AA5 are set to the lowest level: 0.1, and the initial synapse transmission efficiency wij (H0) between the intermediate and output layers is set. The value W _IJ (HO) is converted to the hidden neuron units AA1 to AA5.
When all the output signals are at the lowest level: 0.1,
In setting the output signals DO1 to DO16 from the output layer neuron units RA1 to RA16 to have the highest level: 0.9, specifically, W _IJ (IH) = − 40−2 · (rand−0.5), W _IJ (HO) = -40
− Set to 2 · (rand−0.5).

Then, in the same manner as in the case of forming the three-layer neural network according to the above-described example of the neural network configuring method according to the present invention, learning by the back propagation method is performed as the input signals DI1 to DI64. When arranged as shown in FIG. 2, a signal group GDI1 which takes "0" and "1" as shown in FIG. 3A and represents "A" without including noise, and FIG. B,
Nine types of signal groups GDI2 to GDI1 which take "0" and "1" as shown by C, D and E, respectively, and represent "A" including noise of 2 bits
0 is supplied, thereby determining the value of the synaptic transmission efficiency wij (IH) between the input and intermediate layers and the value of the synaptic transmission efficiency wij (H0) between the intermediate and output layers.

According to the second example of the method for constructing a neural network according to the present invention, an initial value W _IJ ( _IJ ) of the input / interlayer synaptic transmission efficiency wij (IH) is set by the inventor of the present application.
H) and the initial value W _IJ (HO) of the intermediate and output interlayer synaptic transmission efficiency wij (H0), and learning using the signal groups GDI1 to GDI10 as the input signals DI1 to DI64 is actually performed. I was As a result, the initial value W _IJ (IH) of the input / intermediate interlayer synaptic transmission efficiency wij (IH) is determined as shown in FIG. 10 and the initial value of the intermediate / output interlayer synapse transmission efficiency wij (H0) is determined. W _IJ (HO) is set as shown in FIG. 11, and the value of the input / intermediate interlayer synaptic transmission efficiency w ij (IH) after learning is determined as shown in FIG. The value of the later intermediate / output interlayer synapse transmission efficiency wij (H0) was determined as shown in FIG.

10, 11, 12, and 13 are the same as i and j in FIGS. 4, 5, 6, and 7, and for example, the input layer neuron unit SA1
The initial value W _IJ (IH) of the input / interlayer synaptic transmission efficiency wij (IH) between the input and intermediate layer neuron unit AA1 is represented by i = 1 and j = 1 in FIG.
0.03 and the intermediate layer neuron unit AA5
The initial value W _IJ (HO) of the intermediate and output interlayer synaptic transmission efficiency wij (HO) between the output and the output layer neuron unit RA16 is
It is represented by i = 5, j = 16 in FIG.
−39.82. Further, after learning, for example, the input / intermediate interlayer synapse transmission efficiency w between the input layer neuron unit SA1 and the intermediate layer neuron unit AA1
The value of ij (IH) is represented by i = 1, j = 1 in FIG. 12 and is −34.89, and the intermediate / output interlayer synapse transmission between the intermediate layer neuron unit AA5 and the output layer neuron unit RA16. The value of the efficiency wij (HO) is represented by i = 5 and j = 16 in FIG. 13 and is -35.74.

Input layer neuron units SA1 to SA6
4, having intermediate layer neuron units AA1 to AA5 and output layer neuron units RA1 to RA16;
The value of the intermediate interlayer synaptic transmission efficiency wij (IH) was determined as shown in FIG. 12, and the value of the intermediate / output interlayer synapse transmission efficiency wij (H0) was determined as shown in FIG. Second, a method for configuring a neural network according to the present invention
The generalization ability test was also performed by the inventors of the present application on the three-layer neural network obtained according to the example of (1). In such a generalization ability test, when the input signals DI1 to DI64 are arranged as shown in FIG. 2, ten kinds of signal groups GD as shown in FIG.
I11 to GDI20, and 25 types of signal groups GDI21 as shown partially in each of FIGS.
To GDI 45 are input layer neuron units SA1
Output signals DO1 to DO16 obtained from the output layer neuron units RA1 to RA16 while being supplied to
This was done by assaying O16. As a result, 1
All of the 0 types of signal groups GDI11 to GDI20 and the 25 types of signal groups GDI21 to GDI45 are correctly recognized, and in particular, the 25 types of signal groups GDI21 to GDI45.
Are input, the output signals DO1 to DO1
6 were all at the highest level: 0.9. Thus, it was experimentally confirmed that the three-layer neural network obtained according to the above-described second example of the neural network configuration method according to the present invention also has high generalization ability. .

The third method for constructing a neural network according to the present invention.
In the case of forming a three-layer neural network according to the example of FIG. 1, for example, as shown in FIG. 1, 64 input layer neuron units SA1 to SA64 are connected to each of the input layer neuron units SA1 to SA64. Five intermediate layer neuron units AA1 to AA5 and 16 output layer neuron units RA1 to RA each connected to each of the intermediate layer neuron units AA1 to AA5.
16 and the input layer neuron units SA1 to SA
64, the input / output function f (x) for each of the intermediate layer neuron units AA1 to AA5 and the output layer neuron units RA1 to RA16 is based on the inequality: 0 <f (x) ≦ 1. , The initial value W _IJ (IH) of the synaptic transmission efficiency w ij (IH) between the input and intermediate layers is converted to the input signal D input to the input layer neuron units SA1 to SA64, respectively.
When I1 to DI64 are all at the lowest level: 0.1, the output signals in each of the intermediate layer neuron units AA1 to AA5 are set to the lowest level: 0.1, and the initial synapse transmission efficiency wij (H0) between the intermediate and output layers is set. The value W _IJ (HO) is converted to the hidden neuron units AA1 to AA5.
When all the output signals are at the lowest level: 0.1,
In setting the output signals DO1 to DO16 from the output layer neuron units RA1 to RA16 to have the highest level: 0.9, specifically, W _IJ (IH) = − 30−rand, W _IJ (HO) = − 20−2 ・ (rand
−0.5) is set.

Then, learning by the back propagation method is performed as input signals DI1 to DI64 in the same manner as in the case of configuring a three-layer neural network according to the above-described example of the neural network configuring method according to the present invention. When arranged as shown in FIG. 2, a signal group GDI1 representing "A" without noise by taking "0" and "1" as shown in FIG. 3A, and FIG. B,
Nine types of signal groups GDI2 to GDI1 which take "0" and "1" as shown by C, D and E, respectively, and represent "A" including noise of 2 bits
0 is supplied, thereby determining the value of the synaptic transmission efficiency wij (IH) between the input and intermediate layers and the value of the synaptic transmission efficiency wij (H0) between the intermediate and output layers.

According to the third example of the method for constructing the neural network according to the present invention, the inventor of the present application has provided an initial value W _IJ (IH) of the input / interlayer synaptic transmission efficiency wij (IH).
H) and the initial value W _IJ (HO) of the intermediate and output interlayer synaptic transmission efficiency wij (H0), and learning using the signal groups GDI1 to GDI10 as the input signals DI1 to DI64 is actually performed. I was As a result, the initial value W _IJ (IH) of the input / intermediate interlayer synapse transmission efficiency wij (IH) is determined as shown in FIG. 14, and the initial value of the intermediate / output interlayer synapse transmission efficiency wij (H0) is obtained. W _IJ (HO) is set as shown in FIG. 15, and the value of the input / intermediate interlayer synaptic transmission efficiency w ij (IH) after learning is determined as shown in FIG. The value of the later intermediate / output interlayer synapse transmission efficiency wij (H0) was determined as shown in FIG.

14, 15, 16 and 17 are the same as i and j in FIGS. 4, 5, 6 and 7, and for example, the input layer neuron unit SA1
The initial value W _IJ (IH) of the input / interlayer synaptic transmission efficiency wij (IH) between the input and intermediate layer neuron unit AA1 is represented by i = 1 and j = 1 in FIG.
0.51 and the intermediate layer neuron unit AA5
The initial value W _IJ (HO) of the intermediate and output interlayer synaptic transmission efficiency wij (HO) between the output and the output layer neuron unit RA16 is
It is represented by i = 5, j = 16 in FIG.
−19.82. Further, after learning, for example, the input / intermediate interlayer synapse transmission efficiency w between the input layer neuron unit SA1 and the intermediate layer neuron unit AA1
The value of ij (IH) is represented by i = 1, j = 1 in FIG. 16 and is -25.10, and the intermediate / output interlayer synapse transmission between the intermediate layer neuron unit AA5 and the output layer neuron unit RA16. The value of the efficiency wij (HO) is represented by i = 5, j = 16 in FIG. 17, and is -16.46.

Input layer neuron units SA1 to SA6
4, having intermediate layer neuron units AA1 to AA5 and output layer neuron units RA1 to RA16;
The value of the intermediate interlayer synapse transmission efficiency wij (IH) was determined as shown in FIG. 16, and the value of the intermediate / output interlayer synapse transmission efficiency wij (H0) was determined as shown in FIG. Third, a method for configuring a neural network according to the present invention
The generalization ability test was also performed by the inventors of the present application on the three-layer neural network obtained according to the example of (1). In such a generalization ability test, when the input signals DI1 to DI64 are arranged as shown in FIG. 2, ten kinds of signal groups GD as shown in FIG.
I11 to GDI20, and 25 types of signal groups GDI21 as shown partially in each of FIGS.
To GDI 45 are input layer neuron units SA1
Output signals DO1 to DO16 obtained from the output layer neuron units RA1 to RA16 while being supplied to
This was done by assaying O16. As a result, 1
All of the 0 types of signal groups GDI11 to GDI20 and the 25 types of signal groups GDI21 to GDI45 are correctly recognized, and in particular, the 25 types of signal groups GDI21 to GDI45.
Are input, the output signals DO1 to DO1
6 were all at the highest level: 0.9. Thus, it was experimentally confirmed that the three-layer neural network obtained according to the above-described third example of the method for configuring a neural network according to the present invention also has high generalization ability. .

Next, a case where a three-layer neural network is configured according to the fourth example of the method for configuring a neural network according to the present invention will be described below. The three-layer neural network configured according to the fourth example has a function of identifying the content represented by the input signal.
Recognition categories, such as six characters, are identified.

In constructing a three-layer neural network according to the fourth example, first, a plurality of input layer neuron units, one intermediate layer neuron unit connected to each of them, and each of the input layer neuron units, A unit three-layer neural network in which a plurality of output layer neuron units connected to the intermediate layer neuron unit are arranged is prepared in a number corresponding to the number of recognition categories. Specifically, for example, based on each of the 26 letters of the alphabet as 26 recognition categories, as shown in FIG. 18, 64 input layer neuron units SBa1 to SBa64 and 64 input layer neuron units SBa1 to SBa64. , And a unit three-layer neural network CC1 in which 16 output-layer neuron units RBa1 to RBa16 each connected to the intermediate-layer neuron unit AB1 are arranged. , 64 input layer neuron units SBb1 to SBb64, one intermediate layer neuron unit AB2 coupled to each of the input layer neuron units SBb1 to SBb64, and 16 intermediate layer neuron units AB2 each coupled to the intermediate layer neuron unit AB2 Output layer neuron units RBb1 to RBb1 Unit three layer neural network CC2 formed by placing, ..., 6
Four input layer neuron units SBz1 to SBz6
4, one intermediate layer neuron unit AB connected to each of the input layer neuron units SBz1 to SBz64
26 and each of the intermediate layer neuron units AB26
Output layer neuron units RBz coupled to
1 to RBz16 arranged unit three-layer neural network CC
A total of 26 unit trilayer neural networks of 26 are prepared corresponding to the 26 letters of the alphabet, which is the 26 recognition categories, respectively.

At this time, the unit three-layer neural network CC1-CC
The input / output function f (x) for each of the input layer neuron unit, the intermediate layer neuron unit, and the output layer neuron unit in each of the 26 units is represented by an inequality: 0 <f (x)
≦ 1 holds. Specifically, for example, f (x) = 1 / (1 + exp (−x−50)) as represented by the above-described equation (1), where 1 / (1 + exp (−x−50)) > 0.9, f (x) = 0.9, and 1 / (1 + exp (-x-5
0)) <0.1, f (x) = 0.1.

Subsequently, in the unit three-layer neural network CC1, the initial value W _IJ of the input / interlayer synaptic transmission efficiency wij (IH) between each of the input layer neuron units SBa1 to SBa64 and the intermediate layer neuron unit AB1. IH),
When the input signals DIa1 to DIa64 input to the input layer neuron units SBa1 to SBa64 are all at the lowest level, that is, 0.1, the output signal of the intermediate layer neuron unit AB1 is set to have the lowest level, that is, 0.1. In addition, the unit three-layer neural network CC2-C
Similarly to the unit three-layer neural network CC1, each of the C25s also has one of each of the 64 input layer neuron units.
The initial value W _IJ (IH) of the input / interlayer synaptic transmission efficiency w ij (IH) between the intermediate layer neuron units is set to the lowest level, That is, when it is 0.1, the output signal of the hidden neuron unit is at the lowest level, that is,
0.1 and the unit three-layer neural network C
Input layer neuron units SBz1 to SBz in C26
The initial value W _IJ (IH) of the input / interlayer synaptic transmission efficiency wij (IH) between each of the Bz64 and the intermediate layer neuron unit AB26 is determined by the input layer neuron units SBz1-S
Input signals DIz1 to DIz6 respectively input to Bz64
When all the signals 4 are at the lowest level, that is, 0.1, the output signal of the intermediate layer neuron unit AB26 is set to the lowest level, that is, 0.1.

Further, the initial value W _IJ (HO) of the intermediate / output interlayer synaptic transmission efficiency wij (HO) between the intermediate layer neuron unit AB1 and each of the output layer neuron units RBa1 to RBa16 in the unit three-layer neural network CC1. ),
When the output signal from the intermediate layer neuron unit AB1 is at the lowest level, that is, 0.1, the output signal DOa from each of the output layer neuron units RBa1 to RBa16
1 to DOa16 are set to the highest level, that is, 0.9. In addition, each of the unit three-layer neural networks CC2 to CC25 also has 1 as in the unit three-layer neural network CC1.
The initial value W _IJ (0H) of the intermediate / output interlayer synaptic transmission efficiency w ij (OH) between each of the intermediate layer neuron units and each of the 16 output layer neuron units is calculated from one intermediate layer neuron unit. The output signal is at the lowest level, i.e., 0.
When it is 1, the output signal from each of the 16 output layer neuron units is set to the highest level, that is, 0.9, and further, the intermediate layer neuron unit AB26 and the output layer neuron in the unit three-layer neural network CC26. The initial value W _IJ (HO) of the intermediate / output interlayer synaptic transmission efficiency wij (HO) between each of the units RBz1 to RBz16 is determined when the output signal from the intermediate layer neuron unit AB26 is at the lowest level, that is, 0.1. , Output signals DOz from each of output layer neuron units RBz1 to RBz16
1 to DOz16 are set to the highest level, that is, 0.9.

Specifically, for example, the above formula (2) and
(3) as shown in, wo (IH) · OL · NI <mpL (H), W IJ (IH) = wo (IH) - rand, wo (HO) · OL · NH> mpH (O), The relationship of W _IJ (HO) = wo (HO) −rand is established. Then, mpL (H) and mpH (O) are calculated from equation (1), and mpL (H) = − 52.1972 and mpH (O) = − 47.8028, so that equations (2) and (3) ), Wo (IH) · OL · NI = wo (IH) · 0.1 · 64 <mpL (H) = − 52.1972, wo (HO) · OL · NH = wo (HO) · 0.1> mpH (O ) = − 47.8028. Therefore, wo (IH) =-50, wo (HO) =-50
From this, W _IJ (IH) = − 50−rand and
_WIJ (HO) = − 50−rand is set.

Subsequently, the input / intermediate interlayer
Initial value W of naps transmission efficiency wij (IH) _IJ(IH) and middle / out
Initial value W of synaptic transmission efficiency wij (H0)_IJ(HO)
For each of the specified unit three-layer neural networks CC1 to CC26
Learning using backpropagation techniques
And thereby the unit three-layer neural network CC1-CC26
Input and intermediate interlayer synaptic transmission efficiency wij (I
H) and the intermediate and output interlayer synapse transmission efficiency wij (H0)
Determine the value.

The learning by the back propagation method for the unit three-layer neural network CC1 is performed when the input signals DIa1 to DIa64 supplied to the input layer neuron units SBa1 to SBa64 are arranged as shown in FIG. 20. As shown in FIG. 20A, the signal group GD representing the pattern of the alphabet "A" by taking "0" and "1" without including noise.
As shown in Ia1 and some of B, C, D and E in FIG. 20, "0" and "1" are taken and 4
Based on the selection of nine types of signal groups GDIa2 to GDIa10 representing the pattern of “A” including the noise of bits, and selecting, for example, a 16-bit EUC representing “A” as the teacher signal, Learning algorithm
For example, δW ^k−1 _i ^k _j (t + l) = − 0.2 d ^k _j O ^k−1 _i
+ 0.2δW ^k−1 _i ^k _j (t). The noise of four bits in each of the nine types of signal groups GDIa2 to GDIa10 including the noise of four bits and representing the pattern of “A” is two of the input signals DIa1 to DIa64 selected according to a random number. This is generated by inverting a portion that should be “0” to “1” and inverting two portions that should be “1” selected according to random numbers to “0”.

In each of the signal groups GDIa1 to GDIa10, “0” represents the lowest level: 0.1, and “1”.
Represents the highest level: 0.9. Then, the convergence of the learning is determined, for example, by determining that the learning has converged when the total bit error is equal to or less than 0.01,
In the embodiment of learning, learning under supply of one of the signal groups GDIa1 to GDIa10 is repeated until convergence, and after convergence, another one of the signal groups GDIa1 to GDIa10 is supplied. The signal groups GDIa1 to GDIa10 are repeated until the learning with
Are sequentially learned.

Similarly, the unit three-layer neural network CC2
The learning by the back propagation method for each of the CC25s is performed by taking "0" and "1" as input signals supplied to the 64 input layer neuron units, respectively, without including noise, and without using noise. “B”
~ Signal group representing each pattern of "Y", and
Taking "0" and "1", nine types of signal groups each representing a pattern of "B" to "Y" including noise of 4 bits are selected, and as a teacher signal, for example,
Based on the selection of the 16-bit EUC representing each of "B" to "Y", the learning algorithm is performed in the same manner as in the case of the unit three-layer neural network CC1. "B" ~
In a signal group representing each pattern of “Y”,
“0” indicates the lowest level: 0.1, and “1” indicates the highest level: 0.9.

Further, learning by the back propagation method for the unit three-layer neural network CC26 is performed by converting the input signals DIz1 to DIz64 supplied to the input layer neuron units SBz1 to SBz64 into the signals shown in FIG.
21A, the signal group GDIz1 representing the pattern of the alphabet "Z" without including noise by taking "0" and "1", and B, C, and C in FIG.
As shown in a part of each of D and E, "0" and "1" are taken and "Z" including noise for 4 bits is taken.
Signal groups GDIz2 to GD representing the pattern of
Iz10 is selected, and as a teacher signal, for example,
Based on the selection of the 16-bit EUC representing "Z", the learning algorithm is performed in the same manner as in the case of the unit three-layer neural network CC1. It should be noted that nine types of signal groups GDI representing a "Z" pattern including noise of 4 bits
The noise of 4 bits in each of z2 to GDIz10 is obtained by inverting the two input signals DIz1 to DIz64, which should be two “0” selected according to the random numbers, to “1” and the two noises selected according to the random numbers. “1”
Is caused by inverting "0" to "0". In each of the signal groups GDIz1 to GDIz10, “0” represents the lowest level: 0.1, and “1”.
Represents the highest level: 0.9.

According to the inventors of the present application, the initial value W _IJ (IH) of the input / intermediate interlayer synaptic transmission efficiency wij (IH) for each of the unit three-layer neural networks CC1 to CC26 as described above.
And setting of an initial value W _IJ (HO) of the intermediate and output interlayer synapse transmission efficiency wij (H0), and signal groups GDIa1 to GDIa1 to G64 as input signals supplied to 64 input layer neuron units.
Learning was actually performed using each of DIa10,..., GDIz1 to GDIz10. As a result, the value of the input / intermediate interlayer synapse transmission efficiency wij (IH) for the unit three-layer neural network CC1 is determined as shown in FIG. 22, and the intermediate / output interlayer synapse transmission efficiency wij (H0) is determined. Is determined as shown in FIG. 23, and the input / intermediate interlayer synaptic transmission efficiency wij (IH) for each of the unit three-layer neural networks CC2 to CC25.
And the value of the intermediate / output interlayer synapse transmission efficiency wij (H0) are determined in the same manner as in the case of the unit three-layer neural network CC1, and further, the input / intermediate interlayer synapse transmission for the unit three-layer neural network CC26. The value of the efficiency wij (IH) is shown in FIG.
, And the value of the intermediate / output interlayer synaptic transmission efficiency wij (H0) was determined as shown in FIG.

In FIGS. 22 and 24, 1 to 64 of i correspond to 64 input layer neuron units, respectively.
1 to 16 of j in 3 and 25 correspond to 16 output layer neuron units. That is, for example, in the case of the unit three-layer neural network CC1, the value of the input / interlayer synaptic transmission efficiency wij (IH) between the input layer neuron unit SBa1 and the intermediate layer neuron unit AB1 is as shown in FIG.
At -43.90, represented by i = 1 at
Further, the value of the intermediate / output interlayer synaptic transmission efficiency wij (HO) between the intermediate layer neuron unit AB1 and the output layer neuron unit RBa16 is represented by j = 16 in FIG. 23 and is -50.40. Further, for example,
In the case of the unit three-layer neural network CC26, the input layer neuron unit SBz64 and the intermediate layer neuron unit AB2
The value of the input / intermediate interlayer synaptic transmission efficiency wij (IH) between 6 and 6 is represented by i = 64 in FIG.
42.75 and the intermediate layer neuron unit AB2
6 and the output layer neuron unit RBz1
The value of the output interlayer synapse transmission efficiency wij (HO) is represented by j = 1 in FIG. 25 and is -57.90.

In this way, the value of the input / middle layer synaptic transmission efficiency wij (IH) and the value of the intermediate / output layer synapse transmission efficiency wij (H0) are determined.
1 to CC26, each of the 64 input layer neuron units SBa1 to SBa64,.
Sharing of SBz1 to SBz64 and 16 output layer neuron units RBa1 to RB of each
a16,... RBz1 to RBz16 are shared, and as shown in FIG. 26, 64 input layer neuron units SB1 to SB64, 26 intermediate layer neuron units AB1 to AB26 and 16 A three-layer neural network CC having output layer neuron units RB1 to RB16 is obtained.

According to the fourth example of the method for constructing a neural network according to the present invention, the value of the input / middle layer synaptic transmission efficiency wij (IH) and the intermediate / output layer synapse transmission efficiency wij
The number of learning iterations for each input signal group to determine the value of (H0) is relatively small, and the number of learnings becomes enormous as the number of input signal groups to be learned increases. The problem in learning by the propagation method is overcome.

As described above, a three-layer neural network as shown in FIG. 26 obtained according to the fourth example of the method of constructing a neural network according to the present invention is generalized by the present inventor. Ability test was conducted. In such a generalization ability test, 100 random number seeds are used, and 10
The initial value W _IJ (IH) of the input / intermediate interlayer synapse transmission efficiency wij (IH) and the intermediate / output interlayer synapse transmission efficiency wij (H
0), a group of initial values W _IJ (HO) is set, and under each of them, for a three-layer neural network obtained according to the fourth example of the method for configuring a neural network according to the present invention, Input signals D supplied to the input layer neuron units SB1 to SB64, respectively.
When arranged as shown in FIG. 27 as IB1 to DIB64, "0" and "1" are taken like the signal group GDIa1 shown in FIG. Signal groups representing patterns, and signal groups GDIa2, GDIa4, and GD shown in BE of FIG.
As shown in Ia8, GDIa10, etc., “0” and “1” are taken to represent “A” including noise of 4 bits.
A total of 20 input signal groups for "A" including 9 signal groups and similar input signal groups for each of 20 "B" to "Z" were selected. . Each of the 520 signal groups in total is connected to the input layer neuron units SB1 to SB of each three-layer neural network.
64, the output layer neuron unit R
Output signals DOB1 to DOB obtained from B1 to RB16
It was performed by determining whether 16 was appropriate or not and calculating the generalization ability G.

The generalization ability G was calculated as G = the number of input signal groups from which appropriate output signals DOB1 to DOB16 were obtained / the total number of input signal groups (= 520). Therefore, the generalization ability G has a value of 1
The closer to, the higher it is. The relationship between the resulting random number seed RNS and the generalization ability G is as shown in the graph of FIG. 28 (horizontal axis: random number seed RNS, vertical axis: generalization ability G). , The maximum value is 1 (the number of error input signal groups is 0), and the minimum value is 0.976923 (the number of error input signal groups is 12)
And the average value is 0.998139 (the number of error input signal groups is 1)
Met. Thus, it was experimentally confirmed that the three-layer neural network obtained according to the fourth example of the neural network configuration method according to the present invention described above has high generalization ability. .

A plurality of input-layer neuron units, such as the three-layer neural network shown in FIG. 26, and a plurality of intermediate-layer neuron units respectively associated with a plurality of recognition categories and connected thereto. , And a plurality of output layer neuron units coupled to each of the hidden layer neuron units, and the input / output function f (x) for each of the input layer neuron unit, the hidden layer neuron unit and the output layer neuron unit is Inequality: 0 <f
The generalization ability of a three-layer neural network that satisfies (x) ≦ 1 can be evaluated by the neural network generalization ability evaluation method according to the present invention.

It has a plurality of input layer neuron units, a plurality of intermediate layer neuron units respectively corresponding to a plurality of recognition categories, and a plurality of output layer neuron units. The generalization ability of the three-layer neural network in which the input / output function f (x) for each of the output layer neuron units satisfies the inequality 0 <f (x) ≦ 1 is determined by the neural circuit according to the present invention. When evaluating according to the network generalization ability evaluation method, first, the initial and intermediate layer synaptic transmission efficiencies wij (IH) between each of the plurality of input layer neuron units and the hidden layer neuron unit connected thereto are determined. the value W _IJ (IH), the lowest level of the input signal is set every advance for a plurality of input layer neuron units, for example, when having 0.1, double The output signals of each of the intermediate layer neuron units are set to have a predetermined minimum level, for example, 0.1, and between each intermediate layer neuron unit and the output layer neuron unit connected thereto. The initial value W _IJ (HO) of the intermediate / output interlayer synaptic transmission efficiency w ij (H 0) is set to a plurality of values when the output signals of the plurality of intermediate layer neurons all have a preset minimum level, for example, 0.1. The output signal of each of the output layer neuron units is set to have a preset highest level, eg, 0.9.

More specifically, for example, supposing that the contents represented by the respective symbols are the same as those in the above equations (2) and (3), wo (IH) .OL.NI <mpL (H) _{, W IJ (IH) = wo} (IH) + (rand-0.5) ·· (4) wo (HO) · OL · NH> mpH (O), W IJ (HO) = wo (HO) + (rand- 0.5) .. (5) is established.

Subsequently, the input / intermediate interlayer
Initial value W of naps transmission efficiency wij (IH) _IJ(IH) and middle / out
Initial value W of synaptic transmission efficiency wij (H0)_IJ(HO)
The three-layer neural network
Each of the corresponding multiple hidden neuron units
Input-to-interlayer synaptic transmission efficiency wij (IH) and
Learning target synaptic transmission efficiency wij (H0)
Learning by the back propagation method
Perform this for each hidden neuron unit. Each such intermediate layer
Learning by neuron unit consists of multiple input layer neurons.
Input unit for each of multiple recognition categories
Groups are supplied individually and sequentially, multiple inputs
Input to the neuron unit for a specific recognition category
When force signals are supplied individually, their specific perceptions
One hidden neuron unit corresponding to a category
Related input / interlayer synaptic transmission efficiency wij (IH) and
Only the intermediate and output interlayer synapse transmission efficiency wij (H0)
The value is changed to an appropriate value, and
Input / interlayer interlayer related to interneuron unit
Transfer efficiency wij (IH) and synapse transfer between middle and output layers
Each value of the efficiency wij (H0) shall not be changed
It is done.

More specifically, for example, the number of neurons in the input layer is 64, the number of neurons in the intermediate layer is 26, and the number of neurons in the output layer is 16; F (x) = 1 / (1 + exp (−x−50)), where f (x) = 0.9 when 1 / (1 + exp (−x−50))> 0.9. , And 1 / (1 + exp (−x−
50)) If <0.1, perform under the condition that f (x) = 0.1. In such a case, mpL (H) = − 52.1972, m
Since pH (O) = − 47.8028, W _IJ (IH) = − 50−rand and W _IJ (HO) = − 50−rand from the relations of equations (4) and (5).
Is set.

First, the input / middle layer synapse transmission efficiency wij (IH) and the middle / output layer synapse transmission efficiency wij (H0) related to one of the 26 intermediate layer neuron units were set as learning targets. The learning by the back propagation method is performed by using the input / interlayer synaptic transmission efficiency wij (IH) related to the one intermediate layer neuron unit.
Only the synapse transmission efficiency wij (H0) between the intermediate and output layers can be changed, and the learning algorithm is, for example, δW ^k−1 _i ^k _j (t + l) = − 0.2 d ^k _j O ^k−1 _i + Under the condition of 0.2δW ^k−1 _i ^k _j (t), an input signal is supplied to the 64 input layer neuron units, whereby the input / intermediate signal associated with the ^one hidden layer neuron unit is obtained. The respective values of the interlayer synapse transmission efficiency wij (IH) and the intermediate / output interlayer synapse transmission efficiency wij (H0) are determined.

The input signal in such a case is, for example, a signal group GDIa1, which represents the pattern of the alphabet "A" without noise as shown in FIG.
Also, nine types of signal groups GDIa2 to GDIa10, each of which includes a part of each of B, C, D and E in FIG.
And a teacher signal corresponding thereto, for example, 1 representing "A".
6-bit EUC is used.

Thereafter, similarly, the input / intermediate interlayer synapse transmission efficiency wij (IH) and the intermediate / output interlayer synapse transmission efficiency wij (H0) relating to each of the remaining 25 intermediate layer neuron units are set as learning targets. Learning by the back-propagation technique described above is performed by using the input / interlayer synaptic transmission efficiency w associated with the one intermediate layer neuron unit.
It is assumed that only ij (IH) and the intermediate / output interlayer synaptic transmission efficiency wij (H0) can be changed, and the learning algorithm is, for example, δW ^k−1 _i ^k _j (t + l) = − 0.2 d ^k _j O ^{k −1} _i + 0.2δW ^k−1 _i ^k _j (t), input signals are sequentially supplied to the 64 input layer neuron units, whereby the input signals associated with each hidden layer neuron unit are obtained.・ Interlayer synapse transmission efficiency wij (IH) and intermediate / output layer synapse transmission efficiency w
Determine each value of ij (H0).

The input signal in this case is, for example, the alphabet “B” to “B” in the same manner as the signal group GDIa1 shown in FIG. 20B, a signal group representing each pattern of Z ", and B,
"B" to "Z" are displayed in the same manner as nine types of signal groups GDIa2 to GDIa10, each of which includes a part of each of C, D, and E, and represents a pattern of "A" including 4-bit noise. , Each of a total of 250 types of signal groups consisting of 9 types of signal groups representing the respective patterns of "", and the teacher signals respectively corresponding to the signal groups representing the respective patterns of the alphabet "B" to "Z". For example, a 16-bit EU representing each of “B” to “Z”
C is used.

As described above, as a result of learning by the back propagation method, the input / interlayer synaptic transmission efficiency wij (IH) and the intermediate / output layer synaptic transmission efficiency wij associated with each of the 26 intermediate layer neuron units are obtained. (H
In the three-layer neural network (referred to as three-layer neural network X) in which the respective values of (0) are determined, the input / intermediate interlayer synaptic transmission efficiency associated with each of the 26 intermediate layer neuron units The value of wij (IH) is binarized into a positive value and a negative value. When each of the 26 intermediate-layer neuron units is turned on, the output signals from the 16 output-layer neuron units become teacher signals.

Accordingly, the average value ap of the input / intermediate interlayer synapse transmission efficiency wij (IH) as a positive value and the average value of the input / intermediate interlayer synapse transmission efficiency wij (IH) as a negative value
An is calculated, and the ratio between the ON output and the OFF output in the 64 input layer neuron units is obtained as follows using the average value ap and the average value an.

Maxh is the number of input / intermediate interlayer synapse transmission efficiencies wij (IH) whose positive value is assumed, maxl is the number of input / intermediate interlayer synapse transmission efficiencies wij (IH) whose negative value is h, and h is 0 to 1 The value up to maxh that increases by one, l is the maximum that increases by one from 0
Let l be the value up to l, oh be the number of input layer neuron units in the on-output state, and ol be the number of input layer neuron units in the off-output state.
The sum mp of inputs in the input layer neuron unit is
Formula: mp = oh · (ap · h + an · (maxl−l)) + ol · (ap · (maxh−h) + an · l) where h is a value from 0 to maxh. And from 1 to maxl corresponding to each value of h
The values are repeatedly calculated so as to take the respective values up to. Then, the sum mp of inputs in the input layer neuron unit obtained by each calculation is compared with a threshold mpth for the sum of inputs in the input layer neuron unit for the intermediate layer neuron unit to be in the ON output state.

When the sum mp of inputs obtained by each calculation is larger than the threshold value mpth, that is, when mp> mpth, the intermediate layer neuron unit is in the ON output state. For the intermediate layer neuron unit to take on output state,
Based on the result of comparison of the sum of inputs in the input layer neuron unit with the threshold mpth, the number of calculations and the threshold at which a sum mp of inputs greater than the threshold mpth is obtained
The ratio between the ON output and the OFF output in the 64 input layer neuron units is obtained as the ratio to the number of calculations at which the sum mp of inputs equal to or less than mpth is obtained. Then, based on the obtained ratio, the generalization ability of the three-layer neural network X is determined and evaluated.

The results of an experiment conducted by the present inventor on the three-layer neural network X show that, for example, the recognition category:
Regarding “B”, the number (maxh) of the input / intermediate interlayer synapse transmission efficiencies wij (IH) as positive values is 25, and the average value of the input / intermediate interlayer synapse transmission efficiencies wij (IH) as positive values. Value (ap)
Is 10.664, the number of input / intermediate interlayer synapse transmission efficiencies wij (IH) as negative values (maxl) is 39, and the average value of input / interlayer synaptic transmission efficiencies wij (IH) as negative values (an ) Is −43.4
172. Even if 13 of the 64 input layer neuron units which should take the on output state are inverted to take the off output state, the output signals from the 16 output layer neuron units will be As a result, the recognition category: "B" is properly recognized.

[0069]

As is clear from the above description, the first aspect of the method for constructing a neural network according to the present invention (the first, second or third aspect of the method for constructing a neural network according to the present invention described above).
), A plurality of input-layer neurons, one or more intermediate-layer neurons, and a plurality or one output-layer neuron unit include an input / output function f ( x) is arranged so as to satisfy the inequality: 0 <f (x) ≦ 1, and the initial value of the input-intermediate interlayer synaptic transmission efficiency is set to the minimum level at which all inputs to the input layer neuron unit are preset. At one time, the output of the intermediate layer neuron unit is set to be a predetermined minimum level, and the initial value of the intermediate / output interlayer synaptic transmission efficiency is set to one intermediate layer neuron unit or a plurality of intermediate layer neurons. When the output of each of the units is at the preset lowest level, the output of the output layer neuron unit is set to the preset minimum level. It is set to make the level. Then, based on the setting of the initial value of the input / intermediate interlayer synapse transmission efficiency and the initial value of the intermediate / output interlayer synapse transmission efficiency, learning by the back propagation method is performed to perform the input / intermediate interlayer synapse transmission efficiency. It is assumed that the value of the efficiency and the value of the intermediate / output inter-layer synaptic transmission efficiency are determined, thereby forming a three-layer neural network. Thereby, the configured neural network can always be obtained as having an internal structure having excellent generalization ability.

Further, the second aspect of the method for constructing a neural network according to the present invention (exemplified by the first, second or third example of the method for constructing a neural network according to the present invention described above)
According to the above, the input / output function f (x) for each of a plurality of input layer neuron units, one intermediate layer neuron unit and a plurality of output layer neuron units is inequality: 0 <f (x) ≦ 1 A plurality of unit three-layer neural networks having the following conditions are prepared corresponding to a plurality of recognition categories, respectively, and for each unit three-layer neural network, the initial value of the input / intermediate interlayer synaptic transmission efficiency is set to the input value. When all inputs to the layer neuron unit are at the preset lowest level, the output of the hidden layer neuron unit is set to be the preset lowest level, and
When the initial value of the intermediate and output interlayer synapse transmission efficiency is the lowest level of each output of one intermediate layer neuron unit or a plurality of intermediate layer neuron units, the output of the output layer neuron unit is set in advance. Under the set maximum level, learning on the corresponding recognition category is performed by the backpropagation method, and the value of the synaptic transmission efficiency between the input and intermediate layers and the synapse transmission efficiency between the intermediate and output layers. Is determined. Then, for the plurality of unit three-layer neural networks on which the learning has been performed, the plurality of input layer neuron units are shared in each of the plurality of unit three-layer neural networks, and the plurality of output layer neuron units in each of the plurality are shared.
A three-layer neural network is configured. Thereby, always
A three-layer neural network having an internal structure with excellent generalization ability can be relatively easily obtained with rapid learning.

According to the neural network learning method of the present invention, a plurality of input layer neuron units, a plurality of intermediate layer neuron units respectively corresponding to a plurality of recognition categories, and a plurality of output layer neurons. When the three-layer neural network having the neuron unit has an initial value of the input / intermediate interlayer synaptic transmission efficiency, all the inputs to the input layer neuron unit are at the preset minimum level,
When the output of the intermediate layer neuron unit is set to be the preset minimum level, and the initial value of the intermediate / output interlayer synaptic transmission efficiency is the preset minimum level of the output of the intermediate layer neuron unit. ,
Under the condition that the output of the output layer neuron unit is set to be a preset highest level, learning for each of the plurality of recognition categories involves one intermediate layer neuron unit corresponding to each of the plurality of recognition categories. The synapse transmission efficiency is learned only by the back propagation method, and the value of the input / middle layer synapse transmission efficiency and the value of the intermediate / output layer synapse transmission efficiency are determined. as a result,
The generalization ability of the three-layer neural network can be relatively easily evaluated based on the value of the input / intermediate interlayer synapse transmission efficiency and the value of the intermediate / output interlayer synapse transmission efficiency after learning.

Further, according to the method for evaluating generalization ability of a neural network according to the present invention, a plurality of input layer neuron units, a plurality of intermediate layer neuron units respectively corresponding to a plurality of recognition categories, and a plurality of When the initial value of the input / intermediate interlayer synaptic transmission efficiency is the minimum level of all the inputs to the input layer neuron unit, the three-layer neural network having the output layer neuron unit The output of the output layer neuron is set when the output of the intermediate layer neuron unit is set to the preset minimum level, and the initial value of the intermediate / output interlayer synaptic transmission efficiency is the preset minimum level. Each of the multiple recognition categories, with the unit output set to the highest preset level Learning of regarding, performed by a technique of back propagation plurality of recognition categories each only synaptic transmission efficiency involved in one hidden neuron unit corresponding to a learning target, whereby,
The values of the input and intermediate interlayer synapse transmission efficiencies and the values of the intermediate and output interlayer synapse transmission efficiencies are determined, and furthermore, after learning is completed, each of the intermediate layer neuron units will be in the on-output state. The ratio between the output and the off output is determined, and the generalization ability is determined based on the ratio. As a result, the generalization ability of the three-layer neural network can be quickly and relatively easily evaluated.

[Brief description of the drawings]

FIG. 1 is a connection configuration diagram illustrating a three-layer neural network configured by a first example of a method of configuring a neural network according to the present invention.

FIG. 2 is a conceptual diagram explaining a process of configuring a three-layer neural network according to a first example of a neural network configuring method according to the present invention.

FIG. 3 is a conceptual diagram for explaining a process of configuring a three-layer neural network according to a first example of a neural network configuring method according to the present invention.

FIG. 4 is a diagram for explaining a process of configuring a three-layer neural network according to a first example of a neural network configuring method according to the present invention.

FIG. 5 is a diagram for explaining a process of configuring a three-layer neural network according to a first example of a method of configuring a neural network according to the present invention.

FIG. 6 is a diagram provided for describing a three-layer neural network configured according to a first example of a method of configuring a neural network according to the present invention.

FIG. 7 is a diagram provided for explanation of a three-layer neural network configured according to a first example of a method of configuring a neural network according to the present invention.

FIG. 8 is a diagram provided for describing a three-layer neural network configured according to a first example of a method of configuring a neural network according to the present invention.

FIG. 9 is a diagram provided for explanation of a three-layer neural network configured according to a first example of a method of configuring a neural network according to the present invention.

FIG. 10 is a diagram for explaining a process of configuring a three-layer neural network according to a second example of the method of configuring a neural network according to the present invention.

FIG. 11 is a diagram for explaining a process of configuring a three-layer neural network according to a second example of the neural network configuration method according to the present invention.

FIG. 12 is a diagram provided for describing a three-layer neural network configured according to a second example of the method of configuring a neural network according to the present invention.

FIG. 13 is a diagram provided for explanation of a three-layer neural network configured according to a second example of the method for configuring a neural network according to the present invention.

FIG. 14 is a diagram provided to explain a process of configuring a three-layer neural network according to a third example of the method of configuring a neural network according to the present invention.

FIG. 15 is a diagram for explaining a process of configuring a three-layer neural network according to a third example of the method of configuring a neural network according to the present invention.

FIG. 16 is a diagram provided for describing a three-layer neural network configured according to a third example of the method of configuring a neural network according to the present invention.

FIG. 17 is a diagram provided for describing a three-layer neural network configured according to a third example of the method of configuring a neural network according to the present invention.

FIG. 18 is a diagram provided to explain a process of configuring a three-layer neural network according to a fourth example of the method of configuring a neural network according to the present invention.

FIG. 19 is a conceptual diagram explaining a process of configuring a three-layer neural network according to a fourth example of a neural network configuring method according to the present invention.

FIG. 20 is a diagram provided to explain a process of configuring a three-layer neural network according to a fourth example of the method of configuring a neural network according to the present invention.

FIG. 21 is a diagram for explaining a process of configuring a three-layer neural network according to a fourth example of a neural network configuring method according to the present invention.

FIG. 22 is a diagram provided for describing a process of configuring a three-layer neural network according to a fourth example of a method of configuring a neural network according to the present invention.

FIG. 23 is a diagram provided to explain a process of configuring a three-layer neural network according to a fourth example of a method of configuring a neural network according to the present invention.

FIG. 24 is a diagram provided to explain a process of configuring a three-layer neural network according to a fourth example of a method of configuring a neural network according to the present invention.

FIG. 25 is a diagram for explaining a process of configuring a three-layer neural network according to a fourth example of the method of configuring a neural network according to the present invention.

FIG. 26 is a connection configuration diagram showing a three-layer neural network configured by a fourth example of the method for configuring a neural network according to the present invention.

FIG. 27 is a conceptual diagram serving to illustrate a three-layer neural network configured according to a fourth example of a method for configuring a neural network according to the present invention.

FIG. 28 is a diagram provided for explanation of a three-layer neural network configured according to a fourth example of the method for configuring a neural network according to the present invention.

[Explanation of symbols]

SA1 to SA3, SA62 to SA64 Input layer neuron unit SBa1 to SBa3, SBa62 to SBa64 Input layer neuron unit SBz1 to SBz3, SBz62 to SBz64 Input layer neuron unit SB1 to SB3, SB62 to SB64 Input layer neuron unit AA1 to AA5 Middle layer neuron Units AB1 to AB3, AB24 to AB26 Intermediate layer neuron units RA1, RA2, RA15, RA16 Output layer neuron units RBa1, RBa2, RBa15, RBa16 Output layer neuron units RBz1, RBz2, RBz15, RBz16 Output layer neuron units RAB, RB2, RB16 , RB16 output layer neuron unit

Continuation of the front page (56) References JP-A-3-201159 (JP, A) Toru Niki, "Method of Determining Initial Value of Backpropagation Algorithm in Identification Problem", IEICE Transactions C, Japan, Japan Corporation Published by The Institute of Electrical Engineers of Japan, March 20, 1990, Vol. 110, no. 3, pp. 119-125, JPO CSDB Document No .: CSN T199900303002 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06N 1/00-7/08 G06G 7/60 JST file (JOIS) CSDB (Japanese patent Agency) WPI / L (DIALOG) INSPEC (DIALOG)

Claims (6)

(57) [Claims] A plurality of input layer neuron units, one or more intermediate layer neuron units connected to each of the input layer neuron units, and a plurality of one or more neuron units connected to the intermediate layer neuron unit The output layer neuron units are arranged such that the input / output function f (x) for each of them satisfies the inequality: 0 <f (x) ≦ 1, and each of the input layer neuron units is connected to the input layer neuron unit. The initial value of the synaptic transmission efficiency between the hidden layer neuron unit and the output of the hidden layer neuron unit is set to a predetermined minimum level when all the inputs to the input layer neuron unit are at the preset minimum level. And the intermediate layer neuron unit or the plurality of intermediate layer neuron units. The initial value of the synaptic transmission efficiency between each of the knits and the output layer neuron unit connected thereto is set in advance by the output of each of the one intermediate layer neuron unit or the plurality of intermediate layer neuron units. When the output level is at the lowest level, the output layer neuron unit is set to be at the preset highest level, and learning by the back propagation method is performed, and each of the input layer neuron units and the The value of the synaptic transmission efficiency between the coupled intermediate layer neuron unit and the intermediate layer neuron unit or the plurality of intermediate layer neuron units and the output layer neuron unit coupled thereto; Neural network for determining the value of synaptic transmission efficiency during Configuration method. 2. A plurality of intermediate layer neuron units each coupled to each of the input layer neuron units, and a plurality of output layers each coupled to each of the plurality of intermediate layer neuron units. 2. The method according to claim 1, further comprising disposing a neuron unit. 3. A plurality of input layer neuron units, one hidden layer neuron unit connected to each of the input layer neuron units, and a plurality of output layer neuron units connected to the hidden layer neuron unit. And the input / output function f (x) for each is inequality: 0 <f
(x) A plurality of unit three-layer neural networks having a condition that satisfies ≦ 1 are prepared in correspondence with a plurality of recognition categories, respectively, and for each of the unit three-layer neural networks, The initial value of the synaptic transmission efficiency between each of them and the above-mentioned hidden neuron unit is set to the output of the hidden neuron unit when all the inputs to the above-mentioned input neuron unit are at a preset minimum level. The output of the intermediate layer neuron unit sets an initial value of the synaptic transmission efficiency between the intermediate layer neuron unit and the output layer neuron unit coupled thereto, in addition to setting the level to be a predetermined minimum level. When the output level is the lowest level, the output of the output layer neuron unit is set to the preset highest level. Under the setting, the learning of the corresponding recognition category is performed by the back propagation method, and the synaptic transmission between each of the input layer neuron units and the intermediate layer neuron unit connected thereto is performed. Determining the value of the efficiency and the value of the synaptic transmission efficiency between the intermediate layer neuron unit and the output layer neuron unit coupled thereto, for a plurality of unit three-layer neural networks on which the learning has been performed, A method for constructing a neural network, characterized in that a plurality of input layer neuron units in each of them are shared and a plurality of output layer neuron units in each of them are shared to obtain one three-layer neural network. 4. A plurality of input-layer neuron units, a plurality of intermediate-layer neuron units respectively connected to a plurality of recognition categories, respectively connected to each of the input-layer neuron units, and each of the intermediate-layer neuron units. And the input / output function f (x) for each of the input layer neuron unit, the intermediate layer neuron unit and the output layer neuron unit is inequality: 0 <f (x) For a three-layer neural network that satisfies ≦ 1, the initial value of the synaptic transmission efficiency between each of the input-layer neuron units and the intermediate-layer neuron unit coupled thereto is defined as: When all inputs to are at the preset minimum level, the output of the above hidden neuron unit is predicted. The initial value of the synaptic transmission efficiency between the intermediate layer neuron unit and the output layer neuron unit connected thereto is set in advance to set the lowest level, and the output of the intermediate layer neuron unit is preset. When the output level is the lowest level, the learning of each of the plurality of recognition categories is performed for each of the plurality of recognition categories under the condition that the output of the output layer neuron unit is set to be the highest level set in advance. By performing only the synaptic transmission efficiency relating to one corresponding intermediate layer neuron unit as a learning target by the back propagation method, the communication between each of the input layer neuron units and the intermediate layer neuron unit coupled thereto is performed. The value of the synaptic transmission efficiency, Learning neural network, characterized in that to determine the value of synaptic transmission efficiency between the bets each and coupled the output layer neuron units to it. 5. A plurality of input-layer neuron units, a plurality of intermediate-layer neuron units respectively connected to a plurality of recognition categories, each of which is connected to each of the input-layer neuron units, and each of the intermediate-layer neuron units. And the input / output function f (x) for each of the input layer neuron unit, the intermediate layer neuron unit and the output layer neuron unit is inequality: 0 <f (x) For a three-layer neural network that satisfies ≦ 1, the initial value of the synaptic transmission efficiency between each of the input-layer neuron units and the intermediate-layer neuron unit coupled thereto is defined as: When all inputs to are at the preset minimum level, the output of the above hidden neuron unit is predicted. The initial value of the synaptic transmission efficiency between the intermediate layer neuron unit and the output layer neuron unit connected thereto is set in advance to set the lowest level, and the output of the intermediate layer neuron unit is preset. When the output level is the lowest level, the learning of each of the plurality of recognition categories is performed for each of the plurality of recognition categories under the condition that the output of the output layer neuron unit is set to be the highest level set in advance. By performing only the synaptic transmission efficiency relating to one corresponding intermediate layer neuron unit as a learning target by the back propagation method, the communication between each of the input layer neuron units and the intermediate layer neuron unit coupled thereto is performed. The value of the synaptic transmission efficiency, And determining the value of the synaptic transmission efficiency between each of the input layer neurons and the output layer neuron unit connected to the input layer neuron. A generalization capability evaluation method for a neural network, comprising: determining a ratio between an ON output and an OFF output in a unit; and determining a generalization capability of the three-layer neural network based on the determined ratio. 6. The ratio between the on output and the off output of the input layer neuron unit, in which each of the intermediate layer neuron units takes the on output state after learning is established, is connected to each of the input layer neuron units. 6. The neural network according to claim 5, wherein the average value of the synaptic transmission efficiencies with the intermediate layer neuron unit is calculated using an average value of those having a positive value and an average value of those having a negative value. A method for evaluating the generalization ability of a network.