JPH04199259A - Neural network and methods or device for remembering and learning neural network - Google Patents

Abstract

PURPOSE:To make the learning speed of a neural network faster by removing such synapses that the absolute value of the weighting factor becomes smaller than a prescribed value from the neural network. CONSTITUTION:A neural network learning device which removes an asphyxial synapse/neuron is constituted of an input network 1, error reversely propagating weight modifying means 2, asphyxial synapse/neuron removing means 3, and control means 4. In order to optimize the neural network, such synapses that the absolute value of the weighting factor is close to zero and smaller than a fixed value and neurons from which all synapses are disconnected are disconnected from the neural network. Therefore, the neural network learning speed can be improved.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to neural networks, and particularly to a method and apparatus for performing learning or recall thereof at high speed.

[Conventional technology]

Neural networks recognize shapes and characters.

In the analysis of audio and image signals + m-separate processing, these processes are made possible by repeating learning without using complex algorithms. Regarding the configuration and operation of this neural network, please refer to Example Eva's "Hatern Recognition and Learning Algorithm" (written by Yoshinori Uesaka and Kazuhiko Ozeki).
General Publishing).

One of the problems in applying neural networks to real devices is that it takes a long time to perform product and nonlinear function operations during learning and recall.

One idea to solve this problem is described in Japanese Patent Application Laid-Open No. 1-201764. According to this, a new neuron is generated by multiplying the state (output value) of a neuron in multiple input neurons and multiple connection coefficients (synaps weights), and by a nonlinear operation based on the sum of these product values. It has been proposed to speed up the process of determining the state (output value) by performing arithmetic processing on each neuron in parallel.

Also, as a solution to this problem from another perspective,
Japanese Unexamined Patent Publication No. 64-82133 describes that the learning ability of a neural network can be improved by reinitializing neurons that have become quiet during the learning process.

[Problem to be solved by the invention]

However, the above-mentioned conventional technology has the following problems: (1) the neural network is not optimized for the target problem; and (2) it is difficult to apply it to targets that require enormous arithmetic processing or real-time processing. still remains. This makes it difficult to apply the method to prediction-1 diagnosis or control of processes that require a large amount of processing for learning and recall.

An object of the present invention is to overcome the above-mentioned problems and provide a neural network and a method for learning or recalling the same, which realize faster learning and recall by optimizing the neural network for the target.

Another object of the present invention is to normalize the input information of the neural network to speed up learning and recall.
The object of the present invention is to provide a neural network that can be applied to process diagnosis and control, and a method for learning or recalling the neural network.

[Means to solve the problem]

In order to achieve the above object, the present invention is characterized by the following points.

First, an input layer that inputs information, an output layer that outputs the final result, one or more intermediate layers between these two layers, multiple neurons in each layer, and connections between neurons in adjacent layers. In the neural network having synapses with weighting coefficients, there is provided means for removing synapses for which the absolute value of the weighting coefficient is less than or equal to a predetermined value from the neural network.

Second, an input layer that inputs input information, an output layer that outputs a final result, one or more intermediate layers between the two layers, a plurality of neurons provided for each layer, and the neurons between adjacent layers. In a neural network having synapses with weighting coefficients for connecting , there is provided means for normalizing the input information to a predetermined range and inputting it to the input layer, thereby determining the dynamic range of neuron output values.

[Effect]

The first feature of the present invention is that the forward neural network is able to perform synapse/synapse in a state of suspended animation to the extent that it performs one-minute learning.
This is based on the property that the number of neurons increases, and that once synapses/neurons are suspended, they cannot be regenerated as they are.

That is, synapses whose absolute value of weight is below a certain value close to O are separated from the neural network, and neurons from which all synapses have been removed are also separated to optimize the neural network. As a result, redundant product operations and nonlinear function value operations in learning and recall can be eliminated, and processing performance is improved.

The second feature mentioned above is that input information and weighting coefficients in actual systems in the industrial and information fields are sufficiently effective within a finite range;
Therefore, by limiting the numerical space range used within the neural network to a predetermined range,
It is based on the technical idea that product operations and nonlinear function operations that require a large amount of time can be simplified.

For this reason, the multiple input values xi (i = 1 to n) input to the input layer neurons are normalized to a predetermined range, and the range of the weight values of the calculations and synapses from the input to the output neuron is determined in advance. By doing this, all necessary results of products, quotients, exponentiation operations, etc. are created in advance into a table, and by replacing this with direct reference processing that searches for addresses according to input, the huge amount of calculation processing in learning and recall can be dramatically reduced. can be accelerated to

In addition, since the processing within the neural network is performed not by the absolute value of input information as in a linear model, but by extracting its feature quantities, each input value x, (i = 1 to n) of the 7 input layer neurons is The normalization of is based on the idea that it is not necessary to scale by the same dimension (unit dimension).

This is particularly useful in process diagnosis and control, which is the preferred application target of the present invention, and allows for fast and highly accurate learning by normalizing various input information using linear and nonlinear processing. / Recollection becomes possible.

Furthermore, by combining the first feature and the second feature, a faster neural network can be realized.

〔Example〕

Hereinafter, preferred embodiments of the present invention will be described based on the drawings. 1 to 5 are embodiments of the first invention according to the first feature of the invention. FIG. 1 shows the overall configuration of a neural network learning device for removing asphyxial synapses/neurons, and shows the neural network 1. It is composed of error backpropagation weight correction means 2, asphyxia synapse/neuron removal means 3, and control means 4.

In this device, a set x of learning patterns is input, and a known teacher pattern set d(x) for X is given. The correction means 2 calculates and resets the weight of each synapse so as to minimize the output of the forward neural network for the input x, that is, the error between the recall patterns h(x) and d(x). After this process is repeated and the correction is completed, the removing means 3 is activated to remove the synapses neurons in the suspended animation state. The control device 4 is constituted by a computer and controls the above processing.

FIG. 2 conceptually shows the configuration of the forward neural network 1. Although a forward-facing three-layer model is illustrated, the present invention is equally applicable to a multi-layer structure of four or more layers.

Neural net 1 has n input layer neurons x 1 (
x = 1. + 2 t..., n), m middle f'JiN (filter layer) neurons yj (j=1.2
．． ..., rn) and Q output layer neurons zb(
k=1.2. . Weight W lj, V Jk (7) range is -1
,0-1,01'. Each neuron has spatial additivity and nonlinearity, and performs the following processing (recall).

■ Intermediate neuron output value calculation After calculating the input layer neuron output value, that is, the product of the input value of the learning pattern or input information and the synapse weight W I J, calculate the sum of them, and calculate the nonlinear function (here, the sigmoid function) for the sum. ) to find the output value of the hidden layer neuron.

y=δ(woa+w, ax1+w, ax, -1-+wn
4xll)F=1y2+...+n) Here, yJ: Output value of hidden layer neuron δ: Sigmoid function δ(s)''1/(1+e-s)Xl: Output value of input layer neuron (xE 'x) ■ Output layer neuron power value calculation After calculating the product of the intermediate layer neuron output value obtained in (■) above and the weight VJk, calculate their sum, and then calculate a nonlinear function (sigmoid function) for the sum. Find the output layer neuron output value.

Zk=δ(vok+VLSI y1+Vzk'J
z + °゛+ V nk3' n) (k=1.2.
...+m) The set of output values Zk of the neural network 1 obtained by the above processing, that is, the recall pattern h(X), is weighted by the error backpropagation weight correction means 2 shown in FIG. 1 by performing the following processing. By repeating this process, the neural network is trained.

(2) Calculation of synapse weight correction amounts Calculate synapse weight correction amounts δ, -(x) and δ, , (X) that minimize the error between known teacher patterns d(X) and h(x).

δah(x)=α(x)(hb(x) dk(x))
hk(x) (1hk(x)) (j=1. , m) Here, α(X) is the degree of importance regarding the closeness of the output vector h(x) to the input vector x, and is a known item. .

f, (x) is a function of m n variables, f, (x) = δ(wo, ++w, +・x, -1−+wn
Jxj (j=1....+n) ■Weight correction Current synapse weight v? “ , O:+1 is corrected to a new weight V4 HWiJ using the result of ① above.

(j=o,...,m; to=1...,Q) (i=o
,...,n:j=1. ...+m) Here, lxl is the number of learning pattern elements Δ is the differential width and is a known item.

After completing the learning through the above procedure, the modified weight of each synapse is determined as follows by the suspended synapse/neuron removal means 3, and the suspended synapse/neuron is removed from the neural network.

In addition, removal of asphyxial synapses/neurons is performed by control device 4.
This is done by a method such as passing the processing procedure by.

FIG. 3 shows the processing flow of the asphyxia synapse/neuron removal means. Processing A-D removes asphyxia synapses between input and middle layers, and processing E-H removes synapses between middle and middle layers.
Asphyxia synapses between output layers are removed in processes 1 to 1 to remove asphyxic neurons in the middle layer, and asphyxic neurons in the output layer are removed in process LN.

(Process C) The judgment condition for asphyxia synapses is I wiJ!
<α(i=1....t n ; J=1 +...
・+m), it is considered asphyxia.

(Processing D) From forward neural network 1 to W I
Remove neurons with J.

(Processing G) The judgment condition for asphyxia synapses is 1 vtbl
If <α (j=1.-, m; k=1.., Q), it is assumed to be asphyxia.

(Processing H) From forward neural network 1 to Vik
Remove neurons with .

(Processing J) The conditions for determining asphyxia neurons are y, +=(W
i, +=φ for i=1 to n), then it is assumed to be asphyxia.

(Processing K) Remove y from the forward neural network 1.

(Processing M) The judgment condition for asphyxia neurons is zk=(VJ
k=φ for j = 1 to m) (Processing N) Remove Zk from forward neural network 1.

FIG. 4 is a diagram illustrating the operation of the asphyxia synaps/neuron removal means 3. Before the operation of means 3, all neurons are connected by synapses as shown in the figure.

Here, if the asphyxia determination constant α is α=O, O13, after 3 movements, I Wt-I J+□I =O, OO8l WI
J-, l = 0.0091 w l-s, j+1
Since t = o and o i 2 meet the asphyxia condition, the synapses corresponding to these are removed and counted. As a result, the middle layer neuron yJix also enters a state of suspended animation, so yJ
. , the output power Vj□k(k, 1...+m
) along with neuron y, 41 are removed.

In this way, the neural network 1 is optimized by removing asphyxial synapses/two corons at the time of completion of learning, and the number of times to calculate the product of neuron output values and synapses weights during virtual activation is reduced, resulting in faster operation. is achieved.

The above is a method for optimizing a forward neural network immediately after learning. In contrast, an example will be shown in which a neural network with fixed weights after completion of learning is transplanted to an actual system and recalled. Recall here refers to the operation of a neural network in a real system. The recollection operation is the same as in stages ① and ② above. That is, after calculating the sum of the products of input information and synapse weights, the output value of the intermediate layer neuron is determined by a sigmoid function value operation p'. Next, in a similar manner, output layer neuron values, that is, recall results, are obtained using the intermediate layer neuron output values and synapse weights.

FIG. 5 shows an example of recollection by applying the present invention to a forward neural network in which suspended synapses/neurons are not removed. - It eliminates the need for calculations for asphyxial synapses between input and intermediate layer neurons in processing a-f, and for the asphyxial synapses between intermediate and output layer neurons in processing g-α. As a result, high-speed recall can be achieved using this method even with a neural network that is not optimized.

Next, a second invention related to the second feature of the invention will be explained based on FIGS. 6 to 16. FIG. 6 shows the configuration of a learning device 10 to which the present invention is applied. The input information normalization means 13 is a 6-high speed neural network that normalizes the amount of information of the learning pattern X]1. Memory 14. The error backpropagation weight correction means 12 and the control means 15 are the same as those shown in FIG. In the above configuration, the weights of synapses in the high speed neural network 11 are determined using the teacher pattern set d(x) for the learning pattern X.

FIG. 7 similarly shows the configuration of a recollection device 20 to which the second invention is applied. The recall device 20 includes the input information normalization means 13. High speed neural network 11. It consists of a memory 14 and a control device 15, and outputs a recall result h(x) for input information x=(xyu, X2...xn).

The same symbols in FIG. 7 and FIG. 6 have the same functions. Furthermore, the error backpropagation weight correction means 12 in FIG.
performs the same operation as the first invention described above. Therefore, the following explanation will be made using the recollection device 20 shown in FIG.

FIG. 8 shows the configuration of the input information normalization means 13. The normalization means 13 receives an input x, (j = 1.

2, . . +n) into a digital value consisting of b bits. The input information xl is an analog quantity or a digital quantity. Below, input information X consisting of a bits.

The normalization operation for is explained.

Figure 9 shows input information xi having a range of O~(2a-1).
This is a conversion graph when linearly processing the range from O to (2"l).Here, a > b, and a=12.b=8
Then the range of O~(212-1=4095) is O~
It is linearly transformed into a range of (28-1=255). As a result, input information X having various ranges is normalized to Xi' having a range of all b bits or less.

FIG. 10 shows an example of the normalization means when the input XI includes uncertain elements and has nonlinear characteristics. Here, we introduce the concept of fuzzy set and evaluate three types of qualitative evaluation XIBI with a range of 21'-1) for input information X.
X+M, XiS is the output. In the same figure, the horizontal axis is X+
Define the range O~(2a-1,). The vertical axis is X+'
Define the range 0 to (2b-1). As types of qualitative evaluation, membership functions are defined as S (small), M (medium), and B (large). This results in X consisting of a bits, xts consisting of bits (smaller degree),
'Converted to 1M (medium degree) * Xta (large degree). According to this method, stable learning and recall is possible even when input information has large uncertainties and strong nonlinear elements.

FIG. 11 shows the imagination operation of the neural network 11 of FIG. 7. In processing A-E, the output value of the intermediate layer neuron is obtained, and in processing FJ, the output value of the output layer neuron, that is, the output value of the high-speed neural network 11 is obtained. The features of the present invention reside in processes C, E, H, and J. Processing C and H1 processing E and J perform the same operation, so processing C and processing E will be explained below.

(Processing C) Input value xI' (consisting of b bits) with normalized information amount
and the synaps weight W I J (consisting of C bits), the product xt'XwiJ is calculated. Conventionally, since the range of the input value Xl was undefined, a product operation had to be performed. However, in the present invention, since Xl is normalized to b bits by the input information normalization means 13, by storing the calculation result in a storage device in advance, it is possible to directly obtain it by address search.

FIG. 12(a) shows that (0 to (2b-1)
))

(Process E) Find the sigmoid function value for the sum of the product values found above.

The range of S of the sigmoid function is (p x 2 b + C) ~ (p
x 2b+C). Therefore, Fig. 12(b)
By defining a function value in advance as in the example below, the calculation result can be retrieved by address search. This eliminates the need for nonlinear function (sigmoid function) calculations.

The above is an example of the general formula of the product value and the function value, but it will be explained using more specific numerical values.

FIG. 13 shows an example in which the bit numbers (a) and (c) are both 11 bits, and the effective range is set to -1000 to 1000.

In the product value table in Figure (a), the vertical axis is the neuron output value -
10oO to 100o, synaps load value -10 on the horizontal axis
00 to 1000 are defined, and each product value is stored. For example, the neuron output product is 998 and the synapses weight value is 9.
In the case of 99, 997002 is directly searched as the product value.

The sigmoid function table in the same figure (, b) is a one-dimensional table, and the area is secured for the maximum number of neurons P in each layer in the target neural network, the neuron load value range, and the integral of the synapse load value range. be done. In this example, the maximum number of neurons is set to 10, so there is an area from -10,000,000 to 10,000,000. Letting the above value be S, the following is stored in the storage area. Dividing S by 1,000.000 is
This is to align the dimensions, and the reason why it is multiplied by 1,000 is because the range of the target neural network is -
This is to set it to 1000-1000.

By configuring the table in the memory 14 in this manner, product calculations and nonlinear function calculations, which conventionally require a large amount of time, become unnecessary.

The degree of speedup in the second invention described above will be quantitatively evaluated using recall time as an example. As a premise of the evaluation, ta=1 (μ5) 21 address search times taa*
=5(μs): Time for one sum operation tm =50(μs)
s): Assuming the time for one multiplication and division operation, the number of operations and processing time in each process are as shown in FIG.

Assuming that the number of neurons in each layer n, m, and Q in a general neural network is n=m=ffi=100, the one-time recall time ′r^01 in the conventional method is
TAo"= t adiX (2X (m+Q))
+ t mX((n+2)Xrn, +(rn+2)xf
f)=5X(2X(loo+100))+50X((1
00+2)X100+(loO+2)X100)=1,
022,000 (μ5) = 1.022 C seconds) On the other hand, the recall time Tx N of the recall device according to the present invention
E' lma, TANE11= taX(n Xm+mX D,)+
taitX(m+11)=IX(100XL00+1
00X100)+5X(100+100) =21,000 (μ5) =0.021 (seconds) The processing time of the conventional method is 1750.

FIG. 15 is a block diagram of a retrieval device 50 in which the above second method is processed in parallel using a synchronization signal to achieve higher speed. The recall device 50 includes a memory 14, a neuron output value and a synapses weight product address retrieval means 53. Sum calculation means 52
．． It consists of a sigmoid function value search means 51.

FIG. 16A does not show the correspondence with the neural network 11, but shows the detailed configuration of the recollection device 50. In the figure, output neuron 2! and intermediate neuron y, ~
The configuration of the recall device W50 is explained by taking out the 3/n bond.

The address search means 53 inputs the output value y of the intermediate neuron and the synapse value wi, refers to the product value table mentioned above, and determines the product value y□*w1 without performing a product operation. takes the output values of all intermediate neurons as input and outputs the sum value. The sigmoid function value search means 51, which receives the total sum value, refers to the sigmoid function value table described above and determines the sigmoid function value without performing nonlinear calculations.

The recall time TAPara port 0 of this device is TAParaf
fitell=t, (% force value x synapses weight processing time) + t2 (summation processing time) +t, (sigmoid function processing time) +t4 (middle layer neuron output value x synapses weight processing time) + (summation processing time) )+tc (sigmoid function processing time) ”ta+mXtaaa+ta+ta+QXtaad+t
a=1004 (μS) Mi1. (ms) Achieved 1/1000 of the processing time of the conventional method.

17 and 18 are examples of a learning device 30 and a recall device 40 that combine the first invention and the second invention described above.6 The learning device 130 includes the input information normalization means 13, a high-speed neural network 11. Memory 14° Control device 15
．． It consists of the asphyxial synapse/neuron removal means 3, and combines the operations according to the first and second inventions described above to achieve both effects, thereby achieving further speeding up of the learning device.

The recall device 40 performs a product operation with the input information normalization means 13° and the asphyxia synapse/neuron removal means.

It consists of a high-speed neural network 11 having a configuration that directly determines nonlinear operations by address search,
The effects of both inventions can be achieved together.

〔Effect of the invention〕

Since the present invention is configured as described above, (1) redundant synapses/neurons that do not contribute to recall are removed by the suspended synapse/neuron removal means, and the neural network is optimized (2) input information By using the normalization means, the product operation of the neuron output value and the synapse weight value and the nonlinear function (sigmoid function) operation can be replaced with address search processing, which has the effect of dramatically speeding up the learning and recall of the neural network. . This has the effect of making it possible to apply neural networks to real-time process control and diagnosis, which were previously not applicable.

[Brief explanation of the drawing]

1 to 5 explain the first embodiment of the present invention, in which FIG. 1 is a configuration diagram of a neural network learning device, FIG. 2 is a conceptual explanatory diagram of a neural network, and FIG. 3 and 4 are flowcharts showing the processing procedure of the asphyxia synaps/neuron removal means and an explanatory diagram showing the state of the neural network before and after removal, and FIG. S is the processing procedure of the asphyxia synapse/neuron removal means in recalling the neural network. It is a flowchart which shows. 6 to 16 explain a second embodiment of the present invention, and FIGS. 6 and 7 are configuration diagrams of a neural network learning device provided with input information normalization means and a neural network, Figures 8 to 10 are diagrams for having the function of a normalization means, Figure 11 is a flowchart explaining the imaginative operation of the neural network, and Figures 12 and 13 are diagrams showing the contents of the product value and function value file. FIG. 14 is a diagram for explaining the effect of the second embodiment. FIGS. 15 and 16 are specific configuration diagrams of the neural network of the second embodiment and their explanatory diagrams. It is. 17th
18A and 18B are configuration diagrams of a learning device and a recall device for explaining a third embodiment of the present invention. 1. 10, 20, 30, 40, 50... neural network, 2, 12... error back propagation weight correction means, 3
... Asphyxia synapses/neuron removal means, 4.15
...control device, 13..input information normalization means, 14.
·memory. Agent Patent Attorney Katsuma Ogawa 16' ＼U Figure 2 Figure 4 Asphyxia Removal of two corons Figure 8 Figure 9 Figure 10 Rumors of the beak of Low N N Low -----------------------1 -QQ6-

Claims (14)

[Claims] 1. An input layer that inputs information, an output layer that outputs the final result, one or more intermediate layers between the two layers, a plurality of neurons in each layer, and a synapse that connects neurons in adjacent layers using weight coefficients. In the neural network learning method, the weighting coefficient is set so that an error between output information of the neural network obtained by learning information input to the input layer and predetermined teacher information is equal to or less than a predetermined value. A method for learning a neural network, characterized in that synapses for which the absolute value of the weighting coefficient becomes equal to or less than a certain value are not used in the operation of the neural network. 2. In claim 1, the neuron is removed from the neural network when the absolute value of the weighting coefficients of all synapses connected to an arbitrary neuron in the next stage is less than or equal to the certain value. How to learn neural networks. 3. An input layer that inputs learning information, an output layer that outputs the final result, one or more intermediate layers between these two layers, one or more neurons that input/output and process information for each layer, and adjacent layers. a synapse that connects the neurons of the synapses using weighting coefficients, an output value calculation means that determines the output value of the next-stage neuron based on the product of the output value of the neuron and the weighting coefficient, and a storage means that stores the weighting coefficient of the synapse. a network; a weighting coefficient correcting means for calculating a corrected value of the weighting coefficient so as to calculate an error between predetermined teacher information and the output information of the neural network and make the error less than a predetermined value; and an absolute value of the weighting coefficient. the synapse is determined to be below a certain value and removes the synapse from the neural network; A neural network learning device characterized in that the neural network is removed from the neural network. 4. An input layer that inputs input information, an output layer that outputs the final result, one or more intermediate layers between the two layers, one or more neurons that input and output information and process information for each layer, and between adjacent layers. a synapse that connects the neurons of the synapses using weighting coefficients, an output value calculation means that determines the output value of the next-stage neuron based on the product of the output value of the neuron and the weighting coefficient, and a storage means that stores the weighting coefficient of the synapse. What is claimed is: 1. A neural network comprising: a synapse removal determining means for determining whether the absolute value of the weighting coefficient is less than or equal to a predetermined value, and cutting off the connection of the synapses when the absolute value of the weighting coefficient is less than or equal to the predetermined value. 5. An input layer that inputs one or more input information, an output layer that outputs one or more final information, one or more intermediate layers between the two layers, one or more neurons provided for each layer, and adjacent In a neural network having a synapse with a weighting coefficient that connects the neurons between layers, a normalization means for normalizing the input information to a predetermined range is provided, and the normalized information is provided to the neurons of the input layer. A neural network characterized by the following. 6. Neural network according to claim 5, characterized in that said input information normalization means includes linear and/or non-linear processing of input information. 7. 7. The neural network according to claim 6, wherein said nonlinear processing performs fuzzy evaluation using a membership function that evaluates said input information in advance and is stored. 8. an input layer for inputting information, an output layer for outputting a final result, one or more intermediate layers between the two layers, one or more neurons for each layer, and a weighting coefficient for connecting the neurons between adjacent layers. In a learning method for a neural network having synapses, one or more pieces of learning information are normalized to a predetermined range and given to a neuron in the input layer, which neuron uses it as an output value, and transmits the output value and this to the next stage. The product values of the plurality of synapses with the weighting coefficients are obtained from a product value table stored in advance, and the output value of each neuron in the next layer is a predetermined value determined according to the sum of the product values for each neuron. each of the nonlinear function values stored in advance in the predetermined range from the function value table, and the output value of the neural network obtained by sequentially repeating the process of determining the output values of these neurons for each layer and the plurality of learning information. A method for learning a neural network, characterized in that the weighting coefficients of the synapses are modified so that an error with teacher information given in advance corresponding to the synapses is equal to or less than a predetermined value. 9. an input information normalization means for normalizing learning information to a predetermined range; and an input layer for inputting the normalized learning information;
An output layer that outputs the final result, one or more intermediate layers between the two layers, neurons that perform information conversion processing for each layer, synapses that connect neurons between adjacent layers using weight coefficients in a predetermined range, and Neuron output value calculating means for determining the output value of the next stage neuron based on the product value of the output value of the neuron and the weighting coefficient, a set of the product value of the output value of the neuron and the weighting coefficient, and/or a neuron unit of the product value a neural network having a storage means for storing in advance a set of predetermined nonlinear function values representing output values of neurons using the sum total of Weighting coefficient correction means is provided for calculating a correction value of the weighting coefficient so that it is equal to or less than a predetermined value, and the output value calculation means determines the output value of each neuron for the learning information by searching the address from the storage means. A learning device for a neural network, characterized in that the weighting coefficient is modified so that the error between the output information of the neural network obtained by sequentially performing the processing for each layer and the teacher information is equal to or less than a predetermined value. 10. An input layer that inputs information, an output layer that outputs the final result, one or more intermediate layers between the two layers, one or more neurons for each layer, and synapses with weighting coefficients that connect neurons between adjacent layers. In a method for recollecting a neural network, one or more pieces of input information are normalized to a predetermined range and given to a neuron in the input layer, the neuron uses this as an output value, and a plurality of input information that transmits the output value and this to the next stage are provided. What is the weighting coefficient of the synapses? The product values are obtained from a product value table stored in advance, and the output value of each neuron in the next layer is a predetermined nonlinear weight coefficient that is obtained according to the sum of the product values for each neuron. A method for recollecting a neural network, characterized in that each function value is determined from a function value table stored in advance, and the process of determining the output values of these neurons is repeated for each layer in sequence to obtain the output value of the neural network. . 11. an input information normalization means for normalizing input information to a predetermined range; an input layer for inputting the normalized information; an output layer for outputting a final result; one or more intermediate layers between the two layers; One or more neurons input/output and process information for each layer; synapses with weighting coefficients in a predetermined range that connect neurons between adjacent layers; an output value calculation means for determining an output value, and a set of product values of the output information of the neuron and the weighting coefficient, and/or a sum total of the product value in units of the next stage neuron as a variable to indicate the output value of the next stage neuron. A storage means is provided for storing in advance a set of predetermined nonlinear function values, and the output value calculation means sequentially performs processing for each layer to search the address from the storage means and determine the output value of each neuron corresponding to the input information. A neural network characterized in that output information of the neural network is obtained. 12. In claim 11, the predetermined range for normalizing the input information is set to a maximum of b bits (b=1, 2, . . . , b), and the range of the weighting coefficient is set to a maximum of c bits (c=1, 2, . . . ,
c), the product value of the output information of the neuron and the weighting coefficient is a set of (2^b-1) x (2^c-1) elements. 13. 12. The neural network according to claim 11, wherein the nonlinear function value is given by a sigmoid function whose variable is the sum of the product values. 14. An input layer that inputs information, an output layer that outputs the final result, one or more intermediate layers between the two layers, one or more neurons for each layer, and synapses that connect neurons between adjacent layers using weight coefficients. In the neural network learning method, the learning information is normalized to a predetermined range and applied to each neuron in the input layer as an output value of the neuron, and a plurality of synapses transmitting the output value and this to the next stage. The product value of each neuron with the weighting coefficient is obtained from a product value table stored in advance, and the output value of each neuron in the next layer is a predetermined nonlinear function value obtained according to the sum of the product values for each neuron. The deviation between the output value of the neural network obtained by sequentially repeating the process of calculating the output values of these neurons for each layer by calculating each from a function value table stored in advance and the teacher information given in advance is less than a predetermined value. The method is characterized in that the weighting coefficient of the synapse is modified as follows, and when it is determined that the absolute value of the modified weighting coefficient is less than or equal to a certain value, the synapse is removed from the connection of the neural network. How to learn neural networks.