AU1467992A - Neural network - Google Patents
Neural networkInfo
- Publication number
- AU1467992A AU1467992A AU14679/92A AU1467992A AU1467992A AU 1467992 A AU1467992 A AU 1467992A AU 14679/92 A AU14679/92 A AU 14679/92A AU 1467992 A AU1467992 A AU 1467992A AU 1467992 A AU1467992 A AU 1467992A
- Authority
- AU
- Australia
- Prior art keywords
- input
- output
- neural network
- factor
- gain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Saccharide Compounds (AREA)
- Magnetic Heads (AREA)
- Feedback Control In General (AREA)
Description
"A NEURAL NETWORK"
TECHNICAL FIELD
This invention concerns a neural network and, in a second aspect, concerns a method of training the neural network.
Neural networks can in general be considered to be a type of computer with architecture inspired by the neuron and synapse structure of the brain. These computers are capable of being trained to make decisions. For instance, they can be trained by "showing" them a series of
classified patterns. Once trained the computer will be able, when confronted by a pattern to decide which class it belongs to.
Neural networks are made up of neurons connected to each other by synapses. The network may be layered, in which case only the input and output layers are visible to the outside world, and intermediate layers are "hidden".
The function of the synapses (also referred to as weights) is to amplify the signals passing through them, from one neuron to another, by a strength factor.
The function of the neurons is to produce a
non-linear (squashed) value of the sum of the amplified signals received from other neurons.
"Training" (also referred to as teaching) a network involves finding a set of strength factor values for the synapses to enable the network to produce the correct output patterns from given input pattern. BACKGROUND ART
Many researchers have recently proposed architectures for very large scale integration (VLSI) implementation of
a type of neural network called a multi-layer perceptron in which the "training" is performed on-chip. Both digital and analog implementations have used a technique known as "back-propagation" to train the network because of its efficiency and popularity.
Back-propagation is a "supervised" training
technique. This means that to train a network to
recognise an input pattern, the "expected" output of the network associated with the input pattern needs to be known.
Back-propagation trains a network by calculating modifications in the strength factors of individual synapses in order to minimise the value of: half of the sum of the square of the differences between the network output and the "expected" output (the total mean squared error or TMSE). The minimisation process is performed using the gradient of the TMSE with respect to the
strength factor being modified (gradient descent).
Although the gradient with respect to the strength factors in the output layer (synapses connected to the output neurons) can be easily calculated, the gradient with respect to strength factors in the hidden layers is more difficult to evaluate. Back-propagation offers an
analytical technique that basically propagates the error backward through the network from the output in order to evaluate the gradient, and therefore to calculate the required strength factor modifications.
Analog implementation of back propagation requires bi-directional synapses, which are expensive, and the generation of the derivative of neuron transfer functions with respect to their input, which is difficult.
The Madaline Rule III has been suggested as a less expensive alternative to back-propagation for analog implementation. This rule evaluates the required
derivatives using "node perturbation". This means that each neuron is perturbated by an amount Δneti, which produces a corresponding change in the TMSE . The change in value of the input strength required Δwij is estimated by the following equation:
where
ΔE = Epert -E, the difference between the mean squared errors produced at the output of the network for a given pair of input and training signals when a node is
perturbated (Epert) and when it is not (E);
net- = ∑jWijxj;
xj = f (netj) with f being the non-linear squashing function; and
η is a constant.
In addition to the hardware needed for the operation of the network, the implementation of the Madaline Rule III training for a neural network having N neurons in analog VLSI requires an addressing module and wires routed to select and deliver the perturbations to each of the N neurons; multiplication hardware to compute the term
N times (if one multiplier is used then additional multiplexing hardware is required); and an addressing module and wires routed to select and read the xj terms.
If off-chip access to the gradient values is
required, then the states of the neurons (xj) need to be made available off-chip as well, and this will require a multiplexing scheme and N chip pads.
DISCLOSURE OF THE INVENTION
According to the present invention, there is provided a neural network of the type including an input port comprising one or more neurons (or neurodes) and an output port comprising one or more neurons.
The neurons of the input port are connected to the neurons of the output port by one or more paths, each of which comprises an alternating series of synapses
(weights) and neurons. The weights amplify passing signals by a strength factor. A strength factor
perturbating and refresh means applies perturbations to the strength factors of weights in the network, and updates the values of the strength factors depending on the difference between signals appearing at the output port, for a given pair of input and training patterns, when the weight is perturbated, and when it is not.
The output port is preferably connected to a
differencing means to provide an error signal, which represents the error produced at the output port, for a given pair of input and training patterns, when the strength factor of a weight is perturbated and when it is not. The output of the differencing means is preferably connected to a multiplier to multiply the error signal by a factor inversely proportional to the perturbation applied to a strength factor of a weight, to produce a signal representing an updated value for the strength factor of that weight. The strength factor perturbating
and refresh means preferably updates the values of the strength factors of each weight in accordance with the signal representing the update value for that strength factor received from the multiplying means.
Advantageously the neural network further comprises an input gain perturbating and refresh means to apply perturbations to input gains of neurons in the network and to update the value of each input gain depending on the difference between the signals appearing at the output port, for a given pair of input and training patterns, when that input gain is perturbated and when it is not .
Advantageously the neural network further comprises an output gain perturbating and refresh means to apply perturbations to output gains of neurons in the network and to update the value of each output gain depending on the difference between the signals appearing at the output port, for a given pair of input and training patterns, when that output gain is perturbated and when it is not.
According to a second aspect of the present
invention, as currently envisaged, during training the value of the strength factor of each weight is
perturbated, and then updated by an amount depending on the signal representing the update value for that strength factor.
This technique, called 'weight perturbation', relies on its ability to approximate the required derivative, that is the change in the value of each strength factor reσuirεd to update it, according to the following
equation:
where
ΔE is Epert -E, the difference between the mean squared errors produced at the output of the network for a given pair of input and training patterns when a weight is perturbated ( Epert) and when it is not (E);
ΔpertWij is the Perturbation applied to the strength factor at weight wij; and
Δwij is the update amount for the strength factor of w...
The gradient with respect to the strength factor may be evaluated by the forward difference approximation:
if the perturbation ΔpertWij is small enough the strength factor update amount becomes:
where
E () is the total mean square error produced at the output of the network for a given pair of input and training patterns and a given value of the strength factors of the weights.
The order of the error of the forward difference approximation can be improved by using the central
difference method so that:
if the perturbation Δpertwij is again small enough the strength factor update rule becomes:
however, the number of forward relaxations of a network of N neurons is of the order N3, rather than N2 for the forward difference method. Thus either method can be selected on the basis of a speed/accuracy trade-off.
Note, that as η and pertij are both constants, the analog implementation version can simply be written as:
A Wu - Gipertf) A E( wij,pert!j) '
with
and
Δ E(wij,pertij) = E( w- + pertij) - E(wij)
The strength factor update hardware involves only the evaluation of the error with a perturbated and an
unperturbated strength factor and then the multiplication by a constant.
This technique is advantageous for analog VLSI implementation for the following reasons:
As the gradient
is approximated to
(where ΔpertWij is the perturbation applied at weight Wij) , no back-propagation path is needed and only forward paths are required. This means, in terms of analog VLSI
implementations, no bidirectional circuits and hardware for the back-propagation are needed. Also, the hardware used for the operation of the network is used for the training. Only single simple circuits to implement the weight update are required. This simplifies the
implementation considerably.
Compared to Madaline Rule III, weight perturbation does not require the two neuron addressing modules, routing and extra multiplication.
Weight perturbation does not require any overheads in routing and addressing connections to every neuron to deliver the perturbations since the same wires used to access the weights are used to deliver weight
perturbations. Furthermore, Madaline Rule III requires extra routing to access the output state of each neuron and extra multiplication hardware is needed for the terms, which is not the case with weight perturbation. Finally, with weight perturbation, the approximated gradient values can be made available if needed at a comparatively low cost since if the mean square error is required off-chip then only one single extra pad is required. Otherwise, if approximated gradient values are to be calculated
off-chip, no extra chip area or pads are required, since the output of the network would be accessible anyway.
In summary weight perturbation is less expensive to implement in analog VLSI. The hardware cost in terms of VLSI chip area, programming complexity, hardware design time, and as a result the size of the neural network that can be implemented, is less than that required to
implemented Madaline Rule III.
The weight perturbation technique may also be used to train multi-layer recurrent networks, and many artificial neural network models with feedback, including:
multi-layer neural networks; simple recurrent networks like Elman networks; and recurrent networks training to recognise temporal sequences (like Williams and Zipser networks). For all these networks, the hardware
implementation of the weight perturbation technique is very similar. The weight perturbation technique can also be applied to fully interconnected multi-layer
perceptrons.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention will now be described by way of example only with reference to the accompanying drawings, in which:
figure 1 is a schematic diagram of a neural network embodying the present invention; and
figure 2 is a schematic diagram of a multi-layer recurrent exclusive-OR network embodying the present invention.
(Similar reference numerals have been used throughout both figures to identify corresponding elements.)
BEST MODES FOR CARRYING OUT THE INVENTION
Turning now to Figure 1, neural network 1 comprises an array of neurons, or neurodes, 2 only some of which are shown for the sake of clarity. The neurons 2 are
interconnected by synapses, or weights 3, to form an array depending on the desired function. Feed forward arrays have the flow of signals, shown by arrows 4, from the nodes of the input port 5 to the nodes of the input port 6, that is in the upwards direction only. Recurrent networks include reverse flows indicated by arrows 7 directed downwards. The weights are identifiable
throughout the array by coordinates x and y, of which i and j are taken to be any particular case.
The neurons at output port 6 are connected to a differencing means 8. Differencing means 8 calculates the difference between the mean squared error produced at the output of the network for a given pair of input and training signals applied at input port 4 when the strength factors of the weights are perturbated Epe rt and when they are not perturbated E.
The output of differencing means 8, ΔE, is fed to a multiplying means 9 where it is multiplied by a factor proportional to the reciprocal of the perturbation applied to the strength factor at weight Wij.
The output of the multiplier is the update rule for the strength factor at that weight, Δwij, and this is applied to an input of the weight perturbation and weight refreshing system 10. The output of the weight
perturbation and refreshing system 10 alternates between a perturbation ΔpertWij and the update values for the strength factors wij.
When the strength factors of the weights are
perturbated the error gradient and magnitude are
identified by the differencing means 8, and the value of the strength factor of each weight is then refreshed. The strength factors may be updated repeatedly until the error is within preset limits. At this point training is finished and the network is ready for operation.
The application of the technique to a network
performing an exclusive-OR function will now be described.
In order to perform exclusive-OR the network must recognise each of the four possible pairs of inputs, and output the correct one of two outputs, as shown in the following table:
The learning procedure may be implemented as shown in the following table:
Line 1 above applies the training loop (lines
enclosed in the outermost curly brackets) for each pattern that we wish the network to recognise.
Line 2 performs a forward pass through the network. This means that the effect of the application of the current input pattern p is propagated through the network. Propagation takes place through all neurodes and synapses whether they are forward connections or recurrent
connections (if recurrent connections exist, then a relaxation process is used to relax the network towards a stable state). Following the propagation of the current input pattern, the total mean squared error is computed using the actual output of the network and the expected output. The error is returned in E.
Line 3 clears previous weight modifications.
Line 4 iterates on ail the weights the instructions stated in lines 5 to 8.
Line 5 applies a perturbation to the weight currently being considered for modification and repropagates the input pattern the same way as in Line 2 and calculates the perturbated total mean squared error which is returned in Epert .
Line 6 calculates the DeltaError, the difference between the perturbated and unperturbated errors.
Line 7 computes the modification of the weight currently being considered using DeltaError, the strength of the perturbation that has been applied to the weight and the learning rate η . If batch training mode is used, then weight modifications are accumulated to the previous modifications computed for that weight. Otherwise
previous modification are cleared and the current one is stored (this is done by the = assignment).
Line 8 removes the perturbation that has been applied to the current weight and restores its previous value so the next weight can be considered for modification.
Line 9 is the bound on the weight iteration loop.
Line 10 updates the weights of the network according to the compared modifications if batch mode is not being used.
Line 11 is the bound of the iteration on all patterns to be taught to the network.
Line 12 updates the weights according to the computed modifications if batch mode is being used.
As indicated by the procedure above, either batch mode or on-line mode (not batch) can be used. The
difference is: in batch mode the strength factors of the weights are updated after their modifications are
accumulated over all patterns, whereas in on-line mode the strength factors of the weights are updated after the presentation of each pattern.
A recurrent exclusive-OR function 11 network will now be described with reference to figure 2. This network has two neurons 12 in the input port 5 connected by respective weights 13 paths to a single intermediate neuron 14. The two neurons 12 are also connected, via weights 15 to a single neuron 16 in the output port 6. The intermediate neuron 14 is connected by a feedback weight 17 to itself and by a feedforward weight 18 to the neuron 16 in the output port. The neuron 16 in the output port is
connected by feedback weight 19 to itself and by a
feedback weight 20 to the intermediate neuron 14. An offset neuron 21 is connected to each of the other neurons by respective weights. Typical training parameters are given in the following table:
Neurons with programmable gain may also be trained using the weight perturbation method by simply treating the programmable gain as an additional weight in the network. The neuron gain may either be applied to the net input of the neuron or the output of the neuron, and may attenuate or amplify the net input to a neuron, or the output value of a neuron. The advantages of such a feature, are
● increased learning speed;
● weight normalisation;
● automatic normalisation of the training set;
● the facilitation of pruning as the gain reflects the extent of the participation of a neuron in the network,
The node activation function is given by, yi = Gi. fi(gi. neti) where net- is, neti =∑ wij yj fi ( ) is the transfer function of node i , gi is the net input gain of node i and Gi is the output gain of node i.
The gradient with respect to the input gain is
The approximated gradient is then,
where,
Δ E⋎i (g) = E⋎i (g) - E
and⋎i (g) is the perturbation applied to the input gam gi , E⋎i(g) is the Mean Square Error of the network output with the perturbation applied, E is the Mean Square Error without the perturbation applied. The input gain update rules then become:
where η is the learning rate.
The gradient with reference to the output gain is
The approximated gradient is then,
wnere,
Δ E⋎i (g) = E⋎i (g) - E
and⋎i (G) is the perturbation applied to the input gain Gi, and E⋎i(G) is the Mean Square Error of the network output with the perturbation applied, E is the Mean Square Error without the perturbation applied. The gain update rules then become:
where η is the learning rate.
Gain Perturbation can then be implemented using an algorithm that is very similar to that for weight
perturbation. In fact a direct substitution of gain for weight can be made.
As there is a limit of two gains per node, (one for each of the net input and output of the node), the
computational complexity only increases linearly with the number of nodes in the network. Thus the cost of
processing a single training epoch will only increase as the ratio of the number of nodes over the number of interconnections. In a fully interconnected recurrent net this is:
where Cn is the increase in cost of processing the gains, N is the number of nodes.
For a fully connected feedforward network it is:
where NT is the total number of nodes, Nj is the number of nodes in layer j and L is the total number of layers in the network which are numbered (0, 1, 2, ...., L-2, L-1).
The gain perturbation algorithm was tested on several problems using non-recurrent feed forward networks in combination with weight perturbation and using weight perturbation only, as a control. These results show a faster convergence to the same error threshold using gain perturbation in combination with weight perturbation.
It was found that the method was insensitive to the size of the gain perturbation factor, but very sensitive to the rate of convergence and the permissible gain range.
The convergence rate was 0.3 and the perturbation
magnitude was 10-5 for Weight Perturbation.
The techniques require an input vector and a weight vector to be presented to the network, and an output vector is generated. It is not necessary to known about the internal architecture or activity of the network in order to perform optimisation of the weight space. Access to the synaptic weights, input neurons and output neurons is sufficient. Gain perturbation performs an
approximation of the gradient descent algorithms from which it is derived. As such it can solve any problems that can be solved by true gradient descent provided the perturbation signal magnitude is relatively small, with respect to the gain magnitude, and the convergence factor is small. This algorithm is particularly suited for VLSI or other hardware implementations of artificial neural networks due to its minimal hardware requirements.
Claims (29)
1. A neural network of the type including an input port comprising one or more neurons and an output port comprising one or more neurons, and in which the neurons of the input port are connected to the neurons of the output port by one or more paths, each of which comprises an alternating series of weights and neurons, the weights serving to amplify passing signals by a strength factor; the network further comprising a strength factor perturbating and refresh means to apply perturbations to the strength factors of weights in the network and to update the value of each strength factor depending on the difference between the signals appearing at the output port, for a given pair of input and training patterns, when that weight is perturbated and when it is not.
2. A neural network according to claim 1, wherein the output port is connected to a differencing means to provide an error signal which represents the error
produced at the output port for a given pair of input and training patterns when the value of the strength factor of a weight is perturbated and when it is not.
3. A neural network according to claim 2, wherein the output of the differencing means is connected to a multiplier to multiply the error signal by a factor inversely proportional to the perturbation applied to the strength factor of a weight, to produce a signal
representing an updated value for the strength factor of that weight.
4. A neural network according to claim 3 , wherein the strength factor perturbating and refresh means updates the value of the strength factors of each weight in accordance with the signal representing the updated value for that strength factor received from the multiplying means.
5. A neural network according to claim 2, wherein the error signal is the total mean square error produced at the output of the network for a given pair of input and training patterns when the strength factors are
perturbated and when they are not.
6. A neural network according to any preceding claim further comprising an input gain perturbating and refresh means to apply perturbations to input gains of neurons in the network and to update the value of each input gain depending on the difference between the signals appearing at the output port, for a given pair of input and training patterns, when that input gain is perturbated and when it is not.
7. A neural network according to claim 6, wherein the differencing means also operates to provide a second error signal which represents the error produced at the output port for a given pair of input and training
patterns when the value of the input gain of a neuron is perturbated and when it is not.
8. A neural network according to claim 7, wherein the output of the differencing means is connected to a multiplier to multiply the second error signal by a factor inversely proportional to the perturbation applied to the input gain of a neuron, to produce a signal representing an updated value for the input gain of that neuron.
9. A neural network according to claim 8, wherein the input gain perturbating and refresh means updates the value of the input gain of each neuron in accordance with the signal representing the update value for that input gain received from the multiplying means.
10. A neural network according to claim 7, wherein the second error signal is the total mean square error produced at the output of the network for a given pair of input and training patterns when the input gains are perturbated and when they are not.
11. A neural network according to any preceding claim, further comprising an output gain perturbating and refresh means to apply perturbations to output gains of neurons in the network and to update the value of each output gain depending on the difference between the signals appearing at the output port, for a given pair of input and training patterns, when that output gain is perturbated and when it is not.
12. A neural network according to claim 11, wherein the output port is connected to a differencing means to provide a third error signal which represents the error produced at the output port for a given pair of input and training patterns when the value of the output gain of a neuron is perturbated and when it is not.
13. A neural network according to claim 12, wherein the output of the differencing means is connected to a multiplier to multiply the third error signal by a factor inversely proportional to the perturbation applied to the output gain of a neuron, to produce a signal representing an updated value for the output gain of that neuron.
14. A neural network according to claim 13, wherein the output gain perturbating and refresh means updates the value of the output gain of each neuron in accordance with the signal representing the updated value for that output gain received from the multiplying means.
15. A neural network according to claim 12, wherein the third error signal is the total mean square error produced at the output of the network for a given pair of input and training patterns when the output gains are perturbated and when they are not.
16. A neural network substantially as herein
described with reference to the accompanying drawings.
17. A method of training a neural network of the type including an input port comprising one or more neurons and an output port comprising one or more neurons, wherein the neurons of the input port are connected to the neurons of the output port by one or more paths, each of which comprises an alternating series of weights and neurons; the method including the steps of:
(a) amplifying the signals passing through the weights by a strength factor; (b) perturbating the value of the strength factor of each weight; and then
(c) updating the value of the strength factor of each weight by an amount depending on the difference between the signal appearing at the output port, for a given pair of input and training patterns, when that weight is perturbated and when it is not .
18. A method of training a neural network according to claim 17, wherein the amount by which the strength factor of each weight is updated is proportional to the total mean square error produced at the output of the network for a given pair of input and training patterns, when that weight is perturbated and when it is not.
19. A method of training a neural network according to claim 18, wherein the amount by which the strength factor of each weight is updated is inversely proportional to the perturbation applied to the strength factor of that weight.
20. A method of training a neural network according to claims 17, 18 or 19, wherein step (b) and step (c) are repeated.
21. A method of training a neural network according to any one of claims 17 to 20 further including the steps of:
(d) amplifying the signals passing through the inputs of the neurons by an input gain factor;
(e) perturbating the value of the input gain factor of each neuron; and then (f) updating the value of the input gain factor of each neuron by an amount depending on the difference between the signal appearing at the output port, for a given pair of input and training patterns, when that input gain factor is perturbated and when it is not.
22. A method of training a neural network according to claim 21, wherein the amount by which the input gain factor of each neuron is updated is proportional to the total mean square error produced at the output of the network for a given pair of input and training patterns, when that input gain factor is perturbated and when it is not.
23. A method of training a neural network according to claim 22, wherein the amount by which the input gain factor of each neuron is updated is inversely proportional to the perturbation applied to the input gain factor of that neuron.
24. A method of training a neural network according to claims 21, 22, 23, wherein step (e) and step (f) are repeated.
25. A method of training a neural network according to any one of claims 17 to 24 including the further steps of:
(g) amplifying the signals passing through the outputs of the neurons by an output gain factor;
(h) perturbating the value of the output gain factor of each neuron; and then (i) updating the value of the output gain factor of each neuron by an amount depending on the difference between the signal appearing at the output port, for a given pair of input and training patterns, when that output gain factor is perturbated and when it is not.
26. A method of training a neural network according to claim 25, wherein the amount by which the output gain factor of each neuron is updated is proportional to the total mean square error produced at the output of the network for a given pair of input and training patterns, when that output gain factor is perturbated and when it is not.
27. A method of training a neural network according to claim 26, wherein the amount by which the output gain factor of each neuron is updated is inversely proportional to the perturbation applied to the output gain factor of that neuron.
28. A method of training a neural network according to claims 25, 26 or 27, wherein step (h) and step (i) are repeated.
29. A method of training a neural network
substantially as herein described with reference to the accompanying drawings .
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| AU14679/92A AU657523B2 (en) | 1991-03-28 | 1992-03-27 | Neural network |
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| AUPK535591 | 1991-03-28 | ||
| AUPK5355 | 1991-03-28 | ||
| AU14679/92A AU657523B2 (en) | 1991-03-28 | 1992-03-27 | Neural network |
| PCT/AU1992/000133 WO1992017855A1 (en) | 1991-03-28 | 1992-03-27 | A neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| AU1467992A true AU1467992A (en) | 1992-11-02 |
| AU657523B2 AU657523B2 (en) | 1995-03-16 |
Family
ID=25615499
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| AU14679/92A Ceased AU657523B2 (en) | 1991-03-28 | 1992-03-27 | Neural network |
Country Status (1)
| Country | Link |
|---|---|
| AU (1) | AU657523B2 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116055386A (en) * | 2023-03-07 | 2023-05-02 | 燧原智能科技(成都)有限公司 | Port weight updating method, device, chip and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5083285A (en) * | 1988-10-11 | 1992-01-21 | Kabushiki Kaisha Toshiba | Matrix-structured neural network with learning circuitry |
| US5075868A (en) * | 1989-09-18 | 1991-12-24 | The United States Of America As Represented By The Secretary Of The Navy | Memory modification of artificial neural networks |
-
1992
- 1992-03-27 AU AU14679/92A patent/AU657523B2/en not_active Ceased
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116055386A (en) * | 2023-03-07 | 2023-05-02 | 燧原智能科技(成都)有限公司 | Port weight updating method, device, chip and storage medium |
| CN116055386B (en) * | 2023-03-07 | 2023-06-02 | 燧原智能科技(成都)有限公司 | Port weight updating method, device, chip and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| AU657523B2 (en) | 1995-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US5640494A (en) | Neural network with training by perturbation | |
| Bishop | Exact calculation of the Hessian matrix for the multilayer perceptron | |
| Bishop | Neural networks and their applications | |
| Delashmit et al. | Recent developments in multilayer perceptron neural networks | |
| US4951239A (en) | Artificial neural network implementation | |
| US5704016A (en) | Temporal learning neural network | |
| US4912655A (en) | Adjusting neural networks | |
| US4914603A (en) | Training neural networks | |
| US4912649A (en) | Accelerating learning in neural networks | |
| JP2022509618A (en) | Resilient neural network | |
| Ponnapalli et al. | A formal selection and pruning algorithm for feedforward artificial neural network optimization | |
| Jabri et al. | Weight perturbation: An optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayer networks | |
| EP0484506A1 (en) | A learning machine synapse processor system apparatus | |
| US5299285A (en) | Neural network with dynamically adaptable neurons | |
| US5592589A (en) | Tree-like perceptron and a method for parallel distributed training of such perceptrons | |
| US4912654A (en) | Neural networks learning method | |
| US4912652A (en) | Fast neural network training | |
| US4912651A (en) | Speeding learning in neural networks | |
| Faceli et al. | Combining intelligent techniques for sensor fusion | |
| US4912653A (en) | Trainable neural network | |
| Kung et al. | Digital VLSI architectures for neural networks | |
| EP0577694B1 (en) | A neural network and training method | |
| JP2023502204A (en) | Neuromorphic device with crossbar array structure | |
| De Jesus et al. | Forward perturbation algorithm for a general class of recurrent network | |
| AU1467992A (en) | Neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MK14 | Patent ceased section 143(a) (annual fees not paid) or expired |