JPH05298277A - Method and device for learning neural network - Google Patents

Abstract

PURPOSE:To improve the versatility of the neural network in solving problems. CONSTITUTION:The neural network is provided with a neural network arithmetic means 102 where some neurons are connected to multilayer (108, 109, and 110) and learning the relation between input and output signals with teacher signals 106 and 107 based on the learning algorithm, storing the learning result, and performing an arithmetic operation based on the learning result for input signals. The system is provided with a learning algorithm group 104 having several types of learning algorithm. From among several types of learning algorithm 1-n, algorithm to be used for the actual learning is designed to be selected by a learning algorithm selection means 105 according to the problems solved by the neural network arithmetic means 102. Thus, one neural network arithmetic means 102 can perform learning according to problems, resulting in improving versatility.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for learning a multi-layer neural net, and a learning method for various applied apparatuses (devices for performing recognition, prediction, estimation, function approximation, control, etc.) of the multi-layer neural net.

[0002]

2. Description of the Related Art In general, learning of a neural net outputs a network obtained by inputting a certain signal (teacher input signal) into the network and a desired output (teacher output) corresponding to the teacher input signal. Signal), and the synaptic weight between neurons is corrected in the direction of decreasing the deviation. This learning is backpropagation
It is called "Shon" and can be referred to in many documents. For example, the flowchart is described in "Introduction and Learning Neurocomputer" (Technical Review Co., Ltd.) P47.

On the other hand, as a means for improving the learning method aimed at improving the generalization ability (inference ability, cognitive ability) of the neural network, the International Neural Network Soc is used.
Society, "Neural Networks" Vol.4, Number 1, P67-P79 (19
There is a method described in 91). In this method, random numbers of constant amplitude are superimposed on the teacher input signal.

[0004] In addition, there is also a similar statement in P29 of the Acoustical Society of Japan Spring National Convention Lectures (1993, Paper No. 1-5-13), which is superimposed on the teacher input signal in the generalization environment. It is said that a network having the highest generalization ability can be obtained by adding noise having a constant amplitude corresponding to the noise to the teacher input data in advance for learning. In addition, a technical report from the University of California, Los Angeles (Technology from the University of California, Los Angeles
Cal Report) and CSD-910068 (September 1991) describe a method of uniformly reducing the value of random numbers to improve the generalization ability and finally reaching 0.

[0005]

The learning methods that have been variously studied in the above-mentioned prior art do not universally improve the performance of the neural net computing means for all problems. For example, let us compare back propagation with a technique of superimposing a random number on a teacher input signal to improve it. If the problem learned by the network is an approximation of a non-linear function, simple backpropagation gives the network a higher generalization ability. However, in the case of clustering or pattern recognition, on the contrary, the generalization ability may be improved by superposing an appropriate random number on the teacher input signal. Even in the clustering problem, if the teacher signal is sufficiently prepared to include each cluster, a high generalization ability can be obtained by simple back propagation, and the superposition of random numbers complicates the processing. Only. On the other hand, when the teacher signal is insufficient, it has a great effect on improving the generalization ability. Furthermore, random number superposition improves generalization ability when the number of intermediate layer neurons is large and the degree of freedom of the intermediate layer is large, but when the number of intermediate layer neurons is small, the network is limited. The addition of random numbers may adversely reduce the generalization ability because it prevents the solution from being converged.

Therefore, the conventional neural network learning apparatus is prepared as a dedicated apparatus for each problem to be solved by the neural network learning apparatus, and a general-purpose neural network learning apparatus has not been considered.

When a neural net is used for clustering or pattern recognition, the conventional learning method, backpropagation, involves an operation of minimizing an error (output error) from the teacher output of the network output. , Since the operation of bringing the boundary closer to the ideal position (for example, the intermediate position of the cluster) does not directly correspond, even if the learning is further advanced and the error is reduced after reaching a certain output error, the generalization ability may not be improved. ..

In order to solve this problem, the prior art for superimposing a random number on a teacher input signal, in the case where the noise of the environment for generalization is known in advance, has the amplitude of the noise added to the teacher input signal and the amplitude of the noise. Just do it. But,
No consideration is given to the case where the noise is unknown, or the case where the data to which noise with various amplitudes is added is subject to generalization, and the amplitude of the random number to be added to the teacher input signal at this time is not considered. There is a problem that it cannot be easily specified. Furthermore, although the optimal random number superimposing program for increasing generalization ability differs depending on the problem, this is not taken into consideration. Therefore, superimposing a random number decreases generalization ability. There is a risk. For example, since the relationship between the similarity between the teacher input signals and the amplitude of the random numbers to be added is not taken into consideration, when similar teacher input signals belong to different clusters, these are reversed by the addition of excessive random numbers. There is a problem that cannot be identified. Furthermore, even in the method of gradually reducing the amplitude of random numbers as learning progresses, this is taken into consideration even though the optimum reduction program differs depending on the characteristics of the teacher input / output signals and the number of middle-layer neurons in the network. The problem is that a good network cannot always be obtained because it does not exist.

A first object of the present invention is to provide a neural network learning apparatus for learning a network with an algorithm suitable for a problem to be learned and constructing a neural network having a high generalization ability. It is in.

A second object of the present invention is clustering (pattern recognition, which is difficult to improve generalization ability by a conventional learning method).
It is an object of the present invention to provide a neural network learning device that can solve the identification problem and give the network the maximum generalization ability. That is, no matter how different the generalization target data is from the teacher input data, or when similar teacher input signals give different outputs, they do not depend on them, An object of the present invention is to provide a neural network learning device capable of constructing a network having a maximum generalization ability.

[0011]

A first object of the present invention is to provide a neural network learning apparatus with a plurality of types of learning algorithms, and to provide an appropriate algorithm corresponding to the network configuration, the problem to be handled, and the nature of the teacher signal. This is achieved by providing learning algorithm selection means for selection and learning.

The second object is to provide spatial distance detecting means for detecting the distance (spatial distance) between each teacher signal and the boundary line of the cluster to which they belong, and the detected distance is a small value for a particular teacher signal. It is achieved by learning with the goal of not becoming. More preferably, the random number generating means and the data processing means for correcting the amplitude of the generated random number and adding the corrected random number to the teacher input signal are provided.
Achieved by providing the data processing means with the function of analyzing the nature of the teacher signal, the function of determining the initial value of the random number amplitude from the analysis result, and the function of correcting the random number amplitude to a desired value based on the progress of learning. To be done.

[0013]

As a network learning algorithm, a conventional backpropagation that repeatedly learns a teacher signal, and a modified backpropagation that newly learns by superimposing appropriate noise on the teacher signal as a new teacher input signal.
Since a plurality of types such as an application and a vector quantization method are prepared, the learning algorithm selecting means selects an algorithm to be actually used among these. This selection may be performed by the operator from the outside, or the learning algorithm selection means itself may automatically make the selection based on the quality of the expected learning result (the generalization ability of the network). good. Further, the operator may designate an item to be prioritized from the generalization ability of the network, the speed of completion of learning, etc., and the learning algorithm selecting means may select the desired algorithm based on this. In any case, a mechanism for selecting the best algorithm according to the problem handled by the network and the nature of the teacher signal is provided.

The spatial distance detecting means detects the spatial distance between the teacher input signal and the boundary line of the cluster to which the signal belongs. This is performed for all teacher input signals, and the internal state of the neural net is updated so that this value does not become small for a particular teacher input signal, and learning is performed.

For the teacher signal, the data processing means determines the initial value of the random number amplitude to be superimposed on the teacher input signal from the number of teacher signals belonging to each cluster (pattern) and the spatial distance of the teacher signals belonging to different clusters. To do. In addition, the degree of completion of the network is evaluated from the cumulative learning number and the value of the synapse weight, and the random number amplitude value is optimally corrected based on this.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a configuration diagram of a neural net learning apparatus according to an embodiment of the present invention. This neural net learning device 101 includes a neural net computing means 102, a teacher signal memory 103, a learning algorithm group 104 prepared with a plurality of learning algorithms, and a learning algorithm selecting means for selecting an algorithm to be actually used for learning from these. It consists of 105. Neural net computing means 102
As an example, in the illustrated embodiment, a three-layered Lamelhard type network is used. Each layer (the input layer 108, the intermediate layer 109, and the output layer 110) is composed of an appropriate number of neurons, and the neurons between the layers are connected by synapses 111. For details of signal flow in networks and commonly used learning algorithms (backpropagation), refer to, for example, “Introduction and practical training
It can be referred to in many documents such as P15 (Signal flow) and P47 (Learning algorithm) of the "lo computer".

The teacher signal memory 103 stores data used for learning of the neural network computing means 102. Specifically, a plurality of pairs of an input (teacher input signal 106) to the neural net operation means 102 and a desired output of the operation unit (teacher output signal 107) corresponding to the teacher input signal are stored.

The learning algorithm group 104 is provided with a plurality of types of independent algorithms for learning the neural net computing means 102 using the contents of the teacher signal memory 103. The algorithms are (a) normal back propagation and (b)
Technical R at University of California, Los Angeles
eport, CSD-910068 (September 1991), a method of superimposing a uniform random number on the teacher input signal and inputting it to the network and learning by backpropagation, (c) Springer-Ver
lag's “Complex System and Cognitive Processes”
Kohonen's Learning Vector Quantiza, described on P135-137
Various algorithms such as (d) a learning method newly developed in the present embodiment shown in FIG. 8 can be considered as necessary. Note that these do not necessarily have to be provided independently, and a sharable part may have a shared configuration.

The learning algorithm selection means 105 selects the algorithm used for learning by switching.

The neural net learning apparatus 101 is shown in FIG.
As described above, it is generally realized as one of the application software in the computing device (workstation, personal computer, or dedicated computing device) 201. As an implementation mode, a learned and arithmetic integrated type structure in which the learned neural network arithmetic unit 102 outputs an arithmetic result to the I / O 203 of the computer 201 to drive the target 202, and FIG. As the learning result, for example, only the value of the weight of the synapse 111 is calculated in the arithmetic unit 301.
It is conceivable that the "learning / computation separated type" structure in which the actual operation is performed by the arithmetic unit 301 equipped with the neural network operation means 102 is downloaded. The target of calculation is various, such as control, recognition, prediction, diagnosis, etc., and the arithmetic unit 301 may be a controller for control (PID controller, PID controller,
Programmable controller, process controller)
And an image recognition device, and the target 202 is a control plant or various recognized systems.

FIG. 4 shows the learning algorithm selection means 102.
Is an example of a neural network learning device configured to be operated from an external switch 401. When the neural network learning apparatus 101 is realized by a workstation or the like, the structure is such that the desired algorithm is selected from the algorithm selection menu screen using the keyboard and the algorithm selection means is switched. In general, a dedicated learning device may be provided with a dedicated external changeover switch. In this case, the learning algorithm selection means 105 selects the algorithm set by the operator using the external changeover switch 401. Notification means 402
Informs the operator which algorithm is used for learning. As a notifying means, a workstation display may be used to display to the operator, or a dedicated display means using an LED or the like may be displayed.
It may be installed in a Larnet learning device. It is also possible to inform the operator by voice.

FIG. 5 shows the learning algorithm selection means 105.
FIG. 9 is an explanatory diagram of an embodiment in which the contents of the teacher signal memory 103 and the neural network calculation means 102 are loaded, the contents are analyzed, and the algorithm is automatically selected. Now, it is assumed that the learning algorithm group 104 has normal back propagation (SBP) and University of California, Los Ange.
les Technical Report, CSD-910068 (September 1991)
Network by superimposing uniform random numbers on the teacher input signal
It is assumed that a method of learning by back propagation (DBP) input to the client is provided. These backpropagation SBPs and DBPs have a superiority relationship as shown in FIG. That is, when the problem learned by the network is a non-linear function approximation, the SBP has a better generalization ability (the ability of the network to output a value close to a true value). On the other hand, in the case of the clustering problem, the University of
California, Los Angeles Technical Report, CSD-910
As described in P12 of September 1991, DBP generally has a higher generalization ability (ability to correctly recognize letters with noise, etc.), but for example, the number of intermediate-layer neurons is sufficient. If it is not equipped with S, conversely S
The generalization ability of BP is increased. Therefore, when the number of neurons in the intermediate layer is restricted due to the problem of the responsiveness of the neural net computing means 102 and the memory capacity, it is better to select SBP as the learning algorithm. Therefore, the learning algorithm to be used is selected in consideration of this superiority / inferiority relationship.

As an example of the superiority / inferiority relationship, FIG. 7 shows the number of intermediate layer neurons in the vehicle number recognition system described in IEEJ Transactions D, Vol111-D, No.1.P36-P44 (Jan., 1991). Shows the result of evaluation of the recognition rate by the network learned by SBP and DBP. In Fig. 7, the number of intermediate layers is 6
The recognition rate of the DBP is superior when the number is equal to or more than the number, but the recognition rate of the SBP is conversely exceeded when the number of neurons is four. The learning algorithm selection means 105 holds the superiority or inferiority relation of FIG. 6 as data, and the number 1 and the degree of variation of the teacher signals stored in the teacher signal memory 103 by the path 1 and the neural net operation means 102 by the path 2.
The algorithm to be selected is determined after detecting the number of intermediate layer neurons. The determination criterion may be set based on the result of the simulation shown in FIG. Even when another learning algorithm is added as a selection branch, basically, the superiority or inferiority according to the conditions can be clarified by simulation or the like, and the selection criterion can be set by the same idea. Also in this case, it is possible to equip the neural network learning device with the notifying means 402 as shown in FIG.

FIG. 8 shows an example in which the learning algorithm group 104 includes an algorithm for evenly separating the boundaries of clusters (categories) from the clusters. In the embodiment, an example provided as Algorithm 1 is shown. When each cluster is defined by a plurality of teacher signals as in the input space 801, the distance between the cluster and the boundary line can be represented by the distance between the signal closest to the boundary line and the boundary line. Further, the boundary line can be calculated from the output of the neural net calculation means 102 corresponding to each input. The detection means 802 detects distances (a, b, c, d, ...
*) And transfer them to Algorithm 1. In response to this, the algorithm 1 sets the weights of the synapses of the neural net computing means 102 to (a, b, c, d, ...
・ ・) Update so that the values become equal.

FIG. 9 shows an example in which the learning algorithm group 104 is provided with an algorithm for appropriately correcting the signal fetched from the teacher signal memory 103 and performing learning using the corrected signal. Here, the random number generation means 901 generates a random number having a predetermined specification at the timing when the pair of teacher signals is extracted. When the teacher input signal is a vector having a plurality of elements, a random number corresponding to the number of elements is generated at one time. In this embodiment, the learning monitoring means 9
02 monitors the number of repetitions for learning. That is, the number of times the pair of teacher signals has been presented to the neural net computing means 102 is stored. This is, for example, the learning monitoring means 9
02 is provided with a counter, and this counter is incremented each time a pair is extracted from the teacher signal memory 103, so that it can be easily realized (path 1). The teacher input (path 2) extracted from the teacher signal memory 103 and the value of the random number output from the random number generating means 901 corresponding to this are input to the data processing means 903 (path 3). Then, the value of the random number is corrected based on the content of the learning monitoring means 902 (pass 4) and added to the teacher input. If the random number generator 901 generates a random number with a constant amplitude as the content of the correction, the amplitude value of the random number is uniformly reduced in proportion to the number of learning times, for example, as in the following first equation.

[0026]

## EQU1 ## Rm = R.times. (1-Nt / Ntotal) (1) Rm ..... Modified random number value R ........ Output from the random number generating means 103. Random number value Nt ··· Current learning count Ntotal · Number of learning cutoffs.

If the amplitude of the random number is reduced in proportion to the square of the number of learnings, the following second equation is obtained.

[0028]

## EQU2 ## Rm = R.times. [1- (Nt / Ntotal) .multidot. (Nt / Ntotal)] (2). In addition to this, as a method of correcting the teacher signal, a number of methods of giving fluctuation to reduce the teacher signal can be considered. Further, as the monitoring content of the learning monitoring means 902, the update amount of the weight of the network, the increment of the weight with respect to the number of times of learning, or the like may be detected. Further, the random number generating means 103 may be provided with a mechanism for monitoring learning, and the amplitude of the random number may be directly corrected and generated. When the teacher input signal includes a plurality of elements, the content different for each element may be modified as necessary. Finally, the neural net computing means 10
2, the teacher input signal (path 5) to which Rm is added,
The teacher output signal (path 6) is presented as a pair, and learning is performed according to the pair. Here, the teacher signal analysis means 90
4 detects the number of signals stored in the teacher signal memory 103 and the extent of their spread (path 7), and determines the value R of the random number generated by the random number generation means 901 from this value (path 8). For example, when the distance between the signals closest to each other in the input space 801 of the teacher input signals belonging to the cluster 1 and the cluster 2 is r, the amplitude value of the optimum random number for identifying both clusters is r / 2. Therefore, the teacher signal analysis unit 902 calculates the value of the random number generated by the random number generation unit 901 as
Set R = r / 2. As in the first and second equations,
The data processing means 903 starts the initial value of correction for the teacher input signal from R / 2, and is optimized in terms of generalization ability.

[0029]

According to the present invention, since a plurality of neural net learning algorithms are provided and a function of selecting them for learning is provided, the characteristics of the arithmetic operations and teacher signals executed by the network, the network, and the like. The optimal learning algorithm can be used depending on its own configuration. Therefore, the generalization ability of the network can always be maximized. Further, means for detecting the spatial distance between the cluster boundary and the teacher input signal is provided so that this value can be reflected in the weight update of the network synapse. Therefore, it is possible to prevent the spatial distance between the boundary of the cluster and the specific teacher input signal from becoming small, and it is possible to improve the generalization ability in the clustering. Furthermore, the content of the teacher signal is analyzed for a learning algorithm that adds random numbers to the teacher input signal and is input to the network, and the initial value of the random number amplitude is set to a value that maximizes the generalization ability of the network. And the progress of learning,
A means for arbitrarily changing the amplitude value is provided. This can enhance the generalization ability in clustering.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a neural network learning apparatus according to an embodiment of the present invention.

FIG. 2 is a configuration diagram of a learning / calculation integrated system.

FIG. 3 is a configuration diagram of a learning / operation separation system.

FIG. 4 is a configuration diagram of a neural network learning device configured to switch the learning algorithm selection means with an external changeover switch.

FIG. 5 is a configuration diagram of a neural network learning device configured to automatically select a learning algorithm.

FIG. 6 is a diagram showing the superiority and inferiority of generalization ability according to a learning method.

FIG. 7 is a graph showing an example of a simulation result.

FIG. 8: Clusters (category) in the learning algorithm group
FIG. 6 is an explanatory diagram of an example including an algorithm that evenly separates the boundary line of from each cluster.

FIG. 9 is an operation explanatory diagram of the neural network learning apparatus according to the embodiment of the present invention.

[Explanation of symbols]

101 ... Neural net learning device, 102 ... Neural net computing means, 103 ... Teacher signal memory, 104 ...
Learning algorithm group, 105 ... Learning algorithm selecting means, 106 ... Teacher input signal, 107 ... Teacher output signal, 1
08 ... Input layer, 109 ... Intermediate layer, 110 ... Output layer, 11
1 ... Synapse, 201 ... Computing device, 202 ... Target, 20
3 ... I / O, 301 ... Device, 401 ... External switch, 4
02 ... Notification means, 801 ... Input space, 802 ... Detecting means, 901 ... Random number generating means, 902 ... Learning monitoring means, 9
03 ... Data processing means, 904 ... Teacher signal analysis means.

Claims (18)

[Claims] 1. A structure in which several neurons are connected in multiple layers, the relationship between an input signal and an output signal is learned in advance by a teacher signal based on a learning algorithm, the learning result is stored, and the input signal is stored. A neural network having a neural net computing means for computing and outputting the learning result for
The laural net learning device is provided with a plurality of types of learning algorithms, and a learning algorithm selecting means for selecting an algorithm to be used for actual learning from among the plurality of learning algorithms in accordance with a problem to be solved by the neural network computing means. -Lalnet learning device. 2. The neural net learning apparatus according to claim 1, further comprising means for an operator to input a selection command to the learning algorithm selection means. 3. The learning algorithm selecting means according to claim 1, wherein the learning signal and / or the new learning signal is selected.
A neural net learning device comprising means for analyzing a structure of a laural net calculation means and automatically determining an algorithm to be selected from an analysis result. 4. The neural net learning apparatus according to claim 3, wherein the means for automatically determining is to analyze the number and / or variation of the teacher signals. 5. The neural net learning according to claim 3, wherein the means for automatically determining is to analyze the number of neurons existing in the intermediate layer of the neural net computing means. apparatus. 6. The neural net learning apparatus according to claim 1, further comprising a notifying unit for notifying which algorithm is currently selected to the outside. 7. The cluster or pattern stored in the neural net computing means according to any one of claims 1 to 6, when the neural net computing means performs clustering or pattern recognition. A neural net learning device comprising a detecting means for detecting a spatial distance between each boundary and the teacher signal. 8. The teacher according to claim 1, wherein at least one of the plurality of types of learning algorithms is a teacher whose boundary between the clusters and patterns is specified according to the detection result of the detecting means. A neural net learning device, characterized in that learning is performed with the goal of being present at a position not close to a signal. 9. The random number generating means having a random number generating function according to claim 1, and the new node.
Learning monitoring means for monitoring the progress of learning of the Larnet computing means, and the value of the random number generated by the random number generating means according to the result of the monitoring is corrected to be the actual neural network computing means from the corrected random number and the teacher signal. A neural network learning apparatus according to any one of claims 1 to 6, further comprising data processing means for creating an input signal to be input. 10. The random number generating means according to claim 9, wherein the random number generating means generates a predetermined number of random numbers at a timing when a combination of teacher signals used for learning is extracted.
A neural net learning device, characterized in that it is transmitted to a data processing means. 11. The method according to claim 9 or 10,
The neural network learning apparatus, wherein the data processing means also comprises means for analyzing the content of the teacher signal and the progress of learning and determining an initial value of random numbers and a correction method from the analysis result. 12. A structure in which several neurons are connected in multiple layers, the relationship between an input signal and an output signal is learned in advance by a teacher signal based on a learning algorithm, the learning result is stored, and the input signal is stored. In a neural net learning device equipped with a neural net operation means for performing an operation based on the learning result and outputting it, a menu screen is displayed which allows an operator to select a learning program to be used from a plurality of kinds of prepared learning programs. A neural net learning device characterized in that it is provided with a display device. 13. A structure in which several neurons are connected in multiple layers, the relationship between the input signal and the output signal is learned in advance by a teacher signal based on a learning algorithm, and the learning result is stored, and the input signal is stored. In a neural net learning device equipped with a neural net computing means for performing an operation based on a learning result and outputting the learning program, a learning program used by the device to automatically select from among a plurality of prepared learning programs A neural net learning device characterized in that it is provided with a display device for displaying to the operator. 14. A structure in which several neurons are connected in multiple layers, the relationship between an input signal and an output signal is learned in advance by a teacher signal based on a learning algorithm, and the learning result is stored, and the input signal is stored. In a neural net provided with a neural net computing means for performing an operation based on a learning result and outputting the learning algorithm, a plurality of kinds of learning algorithms are prepared in a memory, and an algorithm used for actual learning is selected from the above. A new feature that the selection is made according to the problem to be solved by the neural network computing means.
Larnet learning method. 15. A structure in which several neurons are connected in multiple layers, the relationship between an input signal and an output signal is learned in advance by a teacher signal based on a learning algorithm, the learning result is stored, and the input signal is stored. For a neural net equipped with a neural net operation means for performing an operation based on the learning result and outputting the result, a menu screen for allowing an operator to select a learning program to be used from a plurality of types of learning programs prepared is displayed. A method for learning a neural net, characterized by: 16. A structure in which several neurons are connected in multiple layers, the relationship between an input signal and an output signal is learned in advance by a teacher signal based on a learning algorithm, the learning result is stored, and the input signal is stored. In a neural net equipped with a neural net computing means for performing an operation based on the learning result and outputting the learning program, the operator selects the learning program automatically selected by the device side from a plurality of prepared learning programs. A method for learning a neural net, characterized by being displayed on. 17. The neural network learning apparatus according to claim 1, wherein the neural network learning device incorporates the neural network learning device, which is built in a learned neural network for any one of recognition, prediction, estimation, function approximation, and control. A computer system comprising means for learning a net. 18. A computer system for performing any one of recognition, prediction, estimation, function approximation, and control by a learned neural network, wherein the neural network learning apparatus according to any one of claims 1 to 13 is provided. A computer system comprising means for learning a neural network.