JPH07121494A - Parallel neural network - Google Patents

Abstract

PURPOSE:To obtain a required output signal for a learning input signal or a non-learning input signal by outputting an output unit signal from a main and compensation neural networks as against the input signal through the use of an output state signal from a last stage compensation neural network. CONSTITUTION:The main neural network 21 is connected to more than one compensation neural networks 22 in parallel for an input and they are successively executed learning. Then, through the use of the output state signal from the last stage compensation neural network 22, the output unit signal from the main and the compensation neural networks 21 and 22 is directly outputted or outputted with an addition operating processing, moreover, with an adding method operation processing at the time of a multilevel operation processing as against the input signal. Thus, the required output signal is outputted for the learning input signal and the multiple required outputs can be obtained for the non-learning input signal.

Description

Detailed Description of the Invention

[0001]

The present invention relates to a large-scale neural network applicable to the fields of pattern recognition, associative memory, data conversion, image processing, large-scale multi-valued logic circuit, etc., with high speed and stability by simple learning. The present invention relates to a parallel neural network capable of obtaining a desired output signal.

[0002]

2. Description of the Related Art Conventional neural networks include multi-layered (hierarchical) neural networks and interconnected neural networks, as shown in the document Hideki Aso, "Neural network information processing", industrial book publishing, etc. .

FIG. 1 shows a configuration example of an execution process in an execution process mode of a three-layer neural network 1 having one intermediate layer as a multilayer neural network. N
It is composed of an input layer 4 composed of P units, an intermediate layer 5 composed of P units, and an output layer 6 composed of M units. The number of units N of the input layer corresponds to the total number of elements of the input signal I of the input terminal 2, and the number of units M of the output layer is M.
Corresponds to the total number of elements of the output signal O of the output terminal 3.

The input signal I to the multilayer neural network is I ₁ , I ₂ ,. ． I _N composed of vectors having input signal elements, input to corresponding units of the input layer 4 via the input terminals 2, and then further weighted and output to the units (hidden units) of the intermediate layer 5 respectively. To be done. In the intermediate layer 5, the sum of weighted outputs from each unit of the input layer 4 is used as an input, the threshold value is subtracted, and then the unit output signal is output through a function having an input / output characteristic called a sigmoid function. It In the output layer 6, after the same input / output processing as that of the intermediate layer 5 is performed, the output unit signals are output from the respective output units via the output terminal 3 and the elements O ₁ , O ₂ ,. ． It is sent in correspondence with O _M. Here, the unit output signal output from the output layer 6 is referred to as an output unit signal, and is distinguished from outputs from other layers.

In the case of three or more layers, similarly, each unit in each layer weights the output signal of each unit in the adjacent layer on the input side, and then obtains the sum of them as an input. After subtracting the threshold value, the unit output signal is sent to the adjacent layer on the output layer side via the sigmoid function.

Next, FIG. 2 shows a configuration example of a learning process in the learning mode of the three-layer neural network. As a learning method for such a multilayer neural network, for example, there is a back propagation algorithm as described in the above document. In the learning process using this algorithm, after the connection weights are initialized, a learning input signal prepared in advance is input to the input layer 4 via the terminal 2. The teacher signal T prepared in advance is subtracted from the output unit signal of the output layer 6 by the subtracter 8 and is input to the connection weight controller 9 as an error signal for updating connection weight. The connection weight controller 9 obtains a correction value of the connection weight between the layers so as to minimize the error power based on the unit output signal of each layer input through the terminal 10 and the error signal for updating the connection weight, It is newly set as the connection weight of the three-layer neural network 1 via the terminal 10. This learning process is repeated for all learning input signals.

When the learning process is completely converged, the output unit signal for the learning input signal becomes the same as the teacher signal T. However, once the error power falls to the minimum, there is a problem that the learning does not proceed and the output is not minimized and an output unit signal different from the teacher signal T is output.
In particular, in a three-layer or multi-layer neural network with a large number of input units, a design method for surely converging on the teacher signal T has not been clarified, and the initial value of the connection weight may be changed or a hidden unit (intermediate unit) We are conducting trial and error, such as increasing the number or even more intermediate layers.

Also, the paper "Parallel, Self-Organizing,
Hierarchical Neural Networks "IEEE Trans on Neural
In the parallel neural network 20 shown in Networks, Vol.1, No.2, June 1990, a plurality of neural networks are connected in parallel, and for example, as shown in the execution process of FIG. The connection weights obtained by sequentially learning the neural networks 12, 1
3 and 14, and the error detection areas for the output unit signals obtained by the learning processing are set to error detectors 15 and 1, respectively.
Set to 6 and 17 to operate. The output unit signals of the neural networks 12, 13, 14 and their error detection areas are compared by the error detectors 15, 16, 17, and the output unit signals which do not fall into the error detection area are output by the output selection controller 18. One of them is determined and the output unit signal is taken out by the output selection switch 19 and sent out from the terminal 3 as a desired output signal O via the binary threshold circuit 41.

Further, in the learning processing, the first neural network 12 is made to perform learning a predetermined number of times on the learning input signal, and the output unit signal of 2 is output.
A learning input signal in which the output of the value threshold circuit and the binary teacher signal T do not match is detected, and the second neural network 13 at the next stage is similarly learned with respect to the learning input signal for a predetermined number of times. I am letting you. In this way, if there is a learning input signal in which the output of the binary threshold circuit of the output unit signal and the teacher signal T do not match, the neural network 14 of the next stage is further provided with respect to the learning input signal for a prescribed number of times. Have students learn sequentially. As a result, when the binary threshold circuit outputs of the output unit signals corresponding to all the learning input signals and the teacher signal T match, the addition of the parallel neural network is stopped and it is considered that the learning has converged.

Further, in order to select one correct output unit signal in the execution process, in this learning process, when a binary output that does not match the teacher signal T is generated in each output unit signal of each neural network. The error is subtracted from the output unit signal to find the error, the maximum error is detected, and the error detection area for each output unit signal is defined.

In the execution process, the error detectors 15, 16, 1
Whether or not there is an error in the output unit signal is detected based on the error detection area set in 7, and the output selection controller 18 determines the selection of one output unit signal from the neural network. With this method, it is not possible to output correctly the learning input signal as reported in the paper. Moreover, since a certain independent error area is provided for each output unit signal, the output unit signal can be correctly selected for the non-learning input signal to some extent, but there are many malfunctions depending on the input signal. . In particular, since the error detection area has no generalization property, if the input signal has little correlation or the number of output signal classifications (the number of teacher signals) increases, the selection will be greatly erroneous. Has the drawback of.

[0012]

As described in the conventional learning processing of the multilayer neural network 1, when the connection weight is learned using the teacher signal T in the multilayer neural network, depending on the setting of the initial value of the connection weight. , There are some drawbacks such as when the number of learning times to reach the converged state to send the desired output signal becomes very large, or when the state falls into the state of local minimum (local minimum) and it does not converge even if the learning is continued. is doing. Further, in the conventional parallel neural network 20, even in the method of providing an error detection region and selecting and outputting one correct output unit signal, the learning operation is not complete because the detection operations of the error detectors 15, 16 and 17 are incomplete. There is a case where a desired output signal cannot be selected even for the input signal for use, or a large erroneous selection is made for an input signal other than learning. Particularly, when the number of units in the input / output layer or the number of teacher signals T is large, it is very difficult to send a desired output signal, and a method for freely designing a satisfying neural network has not been established. .

An object of the present invention is to solve the above-mentioned problems, to equivalently and stably converge by simple learning as compared with a conventional neural network or the like, and to obtain a desired output signal even for an unlearned input signal. It is to provide a parallel neural network that is easy to obtain.

[0014]

In order to solve the above problems, in the parallel neural network of the present invention, the main neural network and at least one or more correction neural networks are connected in parallel to the input, and The main neural network learned from the prepared learning input signal and the learning input signal in which the output unit signal of the preceding neural network has an error exceeding the allowable range compared with the teacher signal prepared in advance On the other hand, the correction neural network sequentially learned with respect to the output neural network and the presence or absence of an error in the output unit signal of the preceding neural network exceeding the allowable range are used as the output status teaching signal, and at least the final learning is performed so that the output status signal can be transmitted. The correction neural network of the stage and the output state signal And at least output selection control means for determining the selection of the output unit signal of the neural network, and output selection switch means for selecting and transmitting the output unit signal based on the control signal from the output selection control means. Get the output signal.

Alternatively, a main neural network and at least one or more correction neural networks are connected in parallel with respect to the input, and the main neural network trained on a prepared input signal for learning and the neural network of the preceding stage. The correction neural network for outputting a correction error obtained by sequentially learning an input unit signal for learning in which an error exceeding an allowable range is generated in an output unit signal of the network as compared with the used teacher signal as the teacher signal. And a correction neural network at least in the final stage, which is further learned so that an output state signal can be transmitted with the presence or absence of an error in the output unit signal of the preceding neural network exceeding the allowable range as an output state teacher signal, and the output state The output unit of the neural network is based on the signal. Output selection control means for determining signal selective addition processing, and an output unit signal of the neural network is selected based on a control signal from the output selection control means to obtain an output unit signal having an error exceeding an allowable range. On the other hand, at least output selecting and adding means for adding, correcting and transmitting the output unit signals of the subsequent stage correction neural network, which is a correction error, is provided, and an output signal is obtained.

[0016]

In the parallel neural network of the present invention, as described above, the output unit signal from the main neural network is connected in parallel to the learning input signal which causes an error exceeding the allowable range as compared with the teacher signal T. Learning using the corrected neural network,
The output state signal indicating which neural network outputs the output unit signal within the allowable error range is learned so as to be output from at least the final stage correction neural network. Therefore, in the execution process, the output unit signal within the allowable error range is directly selected from the output unit signals output from the correction neural network at the final stage among the output unit signals from the main and correction neural networks. , Or an output unit signal having an error exceeding the allowable range is corrected and then selected, and a desired output signal is transmitted.

In the correction neural network in the subsequent stage connected in parallel, the number of learning input signals that generate an error exceeding the allowable range gradually decreases with the number of stages, and in the correction neural network in the final stage, the output unit signal is changed. It can be reliably converged within an allowable error range, a desired output signal can be output for the learning input signal, and the non-learning input signal can be output from the generalization property of the correction neural network. It is easy to obtain a desired output signal.

Further, the correction neural network outputs the correction error to the learning input signal by using the error exceeding the allowable range of the output unit signal of the main or correction neural network for the teacher signal T as the correction teacher signal. Are sequentially learned, and in the learning process of the final stage correction neural network, the output unit signal is converged so as to be within the allowable error range with the correction teacher signal. However, the correction teacher signal has many zero components. However, since it is sufficient to converge at least a small number of learning input signals, the final stage correction neural network can be easily converged even under the condition of a small number of learning times or a small number of hidden units. Also, in the correction neural network in the middle stage, the number of learning input signals that cause an error is smaller than that in the preceding stage, so that the number of times of learning and the number of hidden units can be significantly reduced.

When the correction neural network is trained to output the output state signal indicating whether the output unit signal of each neural network is within the allowable error range, the output state teacher signal has many zero components and easily converges. It can be done. Further, due to the generalization of the main and correction neural networks, it is possible to send the output state signal and the output unit signal with few errors even for the non-learning input signal.

When the final stage neural network does not converge within the prescribed number of learning due to the complicated teacher signal T, the number of hidden units is increased, or the number of learning of the preceding stage neural network is increased or hidden. By increasing the number of units and performing learning again, and reducing the number of learning input signals that cause an error from the correction teacher signal of the neural network in the middle stage, the correction neural network in the final stage can be reliably converged.

[0021]

EXAMPLES Examples 1 to 5 of the parallel neural network of the present invention will be given below, and the configuration and operation thereof will be described in detail. The parallel neural network of the present invention is a learning processing mode for learning a connection weight for a learning input signal prepared in advance, and an execution processing for obtaining an output signal for an input signal using the learned connection weight. The description will be made assuming that the operation is performed in two modes, ie, the mode. Here, the input signal, the output signal, the teacher signal, the corrected teacher signal, and the like are treated as vectors.

[0022]

First Embodiment FIG. 4 shows a configuration example of the execution processing in the execution processing mode in the parallel neural network 30 of the present invention as the first embodiment. The parallel neural network 30 of the present invention includes a main neural network 21, a correction neural network 22, an output selection controller 23, an output selection switch 19, and main and correction neural networks 21 and 2 which are connected in parallel to an input.
2 and an operation mode controller 24 for controlling the above operation modes.

Next, the operation will be described. Operation mode controller 2
Reference numeral 4 has a function of switching between the learning processing mode and the execution processing mode, and is set to the execution processing mode by the control of the operation mode control circuit 24 to start the operation.
The main and correction neural networks 21 and 22 are
The connection weights obtained by the learning processing in the learning processing mode are set respectively. For the input signals input to the main neural network 21 and the correction neural network 22, one output unit signal within the allowable error range is selected from the output unit signals from the main neural network 21 and the correction neural network 22. The output selection controller 23 determines based on the output state signal from the correction neural network 22, and switches the switch in the output selection switch 19 according to the output control signal to output either of the output units from the neural networks 21 and 22. Output signal O via terminal 3
Output as.

Here, the output state signal is a signal indicating whether the output unit signal from the main neural network 21 is within the allowable error range with respect to the teacher signal T, and the output state signal is a unit in the output layer of the correction neural network 22. Is added, learning is performed using the output state teacher signal obtained by the output state teacher signal generator 26 in the learning processing described later, and the learning signal is transmitted.

In the case where the main neural network 21 does not always give a desired output unit signal to the learning input signal but outputs an output unit signal having an error exceeding the allowable range with respect to the teacher signal T. There is. Therefore, the correction neural network 22 performs learning so that at least an output unit signal within the allowable error range is transmitted to the learning input signal having this error. The output state signal indicates whether or not there is an error in the output unit signal of the main neural network 21 that exceeds the allowable range with respect to the teacher signal T. Based on this signal, the selection of the output unit signal within the allowable error range is performed. Is done. When the main neural network 21 outputs the output unit signal within the allowable error range, even if the correction neural network 22 outputs the output signal unit exceeding the allowable error range, the output signal is not affected. Therefore, the correction neural network 22 can be easily learned.

Next, FIG. 5 shows the structure of the learning process in the learning process mode in the parallel neural network 30 of the present invention. In the learning processing mode, first, the initial parameters of the main neural network 21 are set, and the learning of the connection weight is started using the teacher signal T for the learning input signal. Here, for example, when a prescribed learning condition such as the number of times of learning is satisfied, learning is terminated, an error is detected by the error detector 25 from the output unit signal and the teacher signal T, and the output state teacher signal generator 26 The presence / absence of an error exceeding the allowable range is stored as the output state teacher signal of the main neural network, for example, 1 if there is an error exceeding the allowable range, and 0 if there is no error that exceeds the allowable range, and is stored in association with the learning input signal. At this time, most of the components of the output state teacher signal become zero.

Next, the initial parameters of the correction neural network 22 are set, and the learning input signal and the teacher signal T are set.
And learning is started using the output state teacher signal. In the learning process of the main neural network 21, as in the conventional case, the subtractor 210 subtracts the teacher signal T from the output unit signal and inputs it to the connection weight controller 211 as an error signal for updating connection weight, and each layer from the terminal 212. The unit weight is updated based on the unit output signal of (1) to minimize the error power, and the setting of a new unit weight is repeated via the terminal 212.

Here, it is difficult to learn the connection weights until the output unit signal from the main neural network 21 matches the teacher signal T within the allowable error range, as has been made clear in the problems of the prior art. In many cases Therefore, in the connection weight controller 211, for example, a method of comparing the number of times of learning with a specified value, a method of using a timer to compare with the set value in the case of real-time processing, or the size of the error signal for updating the connection weight The learning is ended by any method such as a method of comparing with the specified value, and the operation mode controller 24 is instructed.

In the learning process of the correction neural network 22, the subtractor 220 subtracts the teacher signal T from the output unit signal as an error signal for updating the connection weight, and
The subtractor 223 subtracts the output state teacher signal from the output state signal and inputs it to the connection weight controller 221 as an error signal for updating the connection weight for the output state signal.
The connection weight is updated based on the unit output signals of the respective layers from 22 to minimize the error power, and the terminal 222 is updated.
The setting of a new connection weight is repeated via.

Here, in the convergence determiner 27, at least the output unit signal corresponding to the learning input signal having an error in the main neural network 21 matches the corresponding teacher signal T within the allowable error range, and further, all The learning is repeated until a state in which the output state signal and the output state teacher signal match the learning input signal within the allowable error range is detected. If they match, it is considered that the correction neural network 22 has converged, and learning is terminated. Then, the operation mode controller 24 switches the learning mode to the execution processing mode.

At this time, the correction neural network 22
In the case where the learning does not converge under the specified learning condition, the number of learning times may be increased only for the learning input signal in which the main neural network 21 has an error exceeding the allowable range. It may be considered that the number of learning input signals that have occurred is too large, the number of hidden units in the correction neural network 22 is increased under the control of the operation mode controller 24, learning is performed again, and the same processing may be performed. . Alternatively, the main neural network 21 is re-learned to reduce the number of learning input signals that cause an error exceeding the allowable range, the output state teacher signal is regenerated, and then the correction neural network 22 is learned to completely converge. You may let me.

In the execution processing mode, the respective connection weights obtained by the above-mentioned learning processing are set in the main and correction neural networks 21 and 22, and at least the learning input signal is within the allowable error range. Signal T
The same output unit signal as the above is obtained through the output selection switch 19, and the desired output signal O can be obtained from the terminal 3 regardless of the error caused by the incomplete learning of the main neural network 21.

In the neural network 30 of this embodiment, as described above, the correction neural network 22 uses the teacher signal T for at least the limited learning input signal which has an error exceeding the allowable range. Also, the output state signal may be converged by using the output state teacher signal so that the output state signal can be correctly obtained, and the main neural network 21 does not necessarily have to converge. For example, of the output unit signals from the main neural network, learning to output about 90% of the output unit signals within the allowable error range can be realized with a small number of learning times. If more desired output signals are obtained from the main neural network 21 by converging more than this, the number of learning increases exponentially, so that the number of learning can be significantly reduced compared to the conventional method. Is clear.

On the other hand, the correction neural network 22 has only to converge to the remaining about 10% of the learning input signal and the output state teacher signal having many zero components. It is easy to completely converge within the error range. Therefore, it is not necessary to completely converge the main neural network 21 so as to output a desired output signal during learning, and the correction neural network 22 also easily converges during learning.
The number of hidden layers and the number of hidden units can be reduced, and the scale of operations can be reduced.

With respect to the unlearned input signal, an output state signal with few errors can be obtained from the generalization of the correction neural network 22, and the main and correction neural networks 21 and 22 having the generalization can be similarly obtained. The output unit signal within the allowable error range is output from any one of the output selection controllers 23
A desired output signal can be easily obtained by selecting and outputting under the control of.

Although not described here, a plurality of correction neural networks are further inserted in parallel between the main neural network 21 and the correction neural network 22, and the error exceeding the allowable range in the preceding neural network is eliminated. The learning input signals that have occurred are sequentially learned, and at least the final stage correction neural network 22 is learned with respect to the learning input signals that have errors simultaneously in the main neural network 21 and all the correction neural networks in the previous stage. You may let me. A configuration may be adopted in which the respective connection weights obtained by these learnings are set and executed, and one output unit signal of any neural network is selected and output via the output selection switch 19.

Further, not only the correction neural network 22 at the final stage but also all correction neural networks are caused to output respective output state signals, and the output selection controller 23 determines the output unit signal to be selected. A desired output unit signal may be selected and output via the output selection switch 19. Here, configuration examples of the execution process and the learning process of these embodiments are omitted.

As described above, since the parallel neural network 30 of the present invention can be simply and surely learned in a very short time, a large-scale neural network which is extremely difficult to design by the conventional neural network can be easily realized. it can. Also, by preparing a plurality of sets of learned connection weights in the parallel neural network 30 of the present invention and switching and setting them, it is possible to easily realize a plurality of large-scale neural networks with the same configuration. Also, since it is possible to re-learn in a short time according to the situation, it is possible to easily realize a large-scale neural network which gives a new input / output relationship on the same configuration. Further, the conventional neural network can be replaced with the parallel neural network 30 of the present invention, and the learning can be easily performed to use for pattern recognition, associative memory, data compression, data conversion and the like.

[0039]

Second Embodiment A parallel neural network 40 of the present invention as a second embodiment is configured such that a main neural network 21 and a correction neural network 22 are connected in parallel to an input, and an output unit signal of the main neural network 21 has an allowable range. If there is an error that exceeds, an output unit signal as a correction error is sent from the correction neural network 22, and this error is corrected by the addition calculation process and output.

In the configuration of the execution process in the execution process mode,
Similar to the first embodiment, the connection weight obtained by the learning process in the learning mode is set in each neural network. As shown in FIG. 6, the output unit signal within the allowable error range from the main neural network 21 with respect to the teacher signal T is output to the terminal 3 via the output selection adder 28 under the control of the output selection controller 23. The output unit signal output from the correction neural network 22 as an error for correction is directly output from the correction neural network 22 to the output unit signal, and the output selection adder 28 performs addition calculation to correct the error. Output to. Similar to the first embodiment, the output state signal indicating the presence or absence of an error exceeding the allowable range of the output unit signal from the main neural network 21 is sent from the correction neural network 22, and the output unit signal from the main neural network 21 is directly output. The output selection controller 23 determines whether to output or correct and output, and controls the output selection adder 28 to obtain a desired output signal from the terminal 3.

FIG. 7 shows a structural example of the learning processing in the learning processing mode. Similar to the above-described embodiments, first, the main neural network 21 is started to be learned by the control signal from the operation mode controller 24. After the learning is completed, the error detector 25 subtracts the output unit signal from the teacher signal T to obtain an error, and the error exceeding the allowable range is stored in the corrected teacher signal generator 29 as the corrected teacher signal.
The output state teacher signal generator 26 stores the output state teacher signal indicating the presence or absence of an error exceeding the allowable range.

Next, the learning of the correction neural network 22 is started by the control signal from the operation mode controller 24, the correction teacher signal is read from the correction teacher signal generator 29, and the subtraction unit 220 updates the connection weight updating error. A signal is obtained, the output state teacher signal is read from the output state teacher signal generator 26, the error signal for updating the connection weight is calculated by the subtractor 223, and the error signal for updating the connection weight is input to the connection weight processor 221 to correct the connection weight. Then, the correction neural network 22 is set and learned through the terminal 222. In this learning process, in the convergence determiner 27, the output unit signal is within the allowable error range with respect to the correction teacher signal in which the error exceeds the allowable range in at least the main neural network 21, and the output state teacher If all the output state signals of the signals are within the allowable error range, it is considered that the signals have converged, and learning is terminated.

If the convergence does not occur under the specified learning conditions, the same processing as that of the first embodiment, or the learning of the main neural network 21 is restarted, and the error exceeding the allowable range in the main neural network 21 is restarted. It is also possible to reduce the number of learning input signals that cause the above, regenerate the corrected teacher signal, and train the corrected neural network 22 again. When the learning process ends, the operation mode controller 24 switches the learning process mode to the execution process mode.

The corrected teacher signal has a zero component except where there is a correction error, and has a completely different component from the teacher signal T. Further, at least in the main neural network 21, an error exceeding an allowable range is generated. Since the learned input signal may be learned, it is possible to easily converge the correction neural network 22 quickly.

[0045]

Third Embodiment In a parallel neural network 50 of the present invention as a third embodiment, a correction neural network 31 is further provided in parallel between the main neural network 21 and the correction neural network 22 in order to more easily perform convergence by learning. Is added, and a configuration example in which the selection of the output unit signal within the allowable error range and the correction of the error are combined is shown. FIG. 8 shows a configuration example of the execution process in this case. First, the main neural network 21, the correction neural network 31, the correction neural network 22, and the connection weighting factors sequentially learned as in the first or second embodiment are set. In the output selection adder 28, for an error exceeding the allowable range of the main neural network 21, a process of switching the output unit signal from the correction neural network 31 to be used or an addition process as a correction error is performed. The processing used for correction and the like are executed by controlling internal switches under the control signal from the output selection controller 32.

Further, the correction neural network 31
When the output unit signal from the output error signal exceeds the allowable range, the output selection adder 28 uses the output unit signal from the correction neural network 22 by switching or the correction neural network 31. A process of correcting an error that exceeds the allowable range by an addition process using a correction error from the correction neural network 22 is also realized by controlling an internal switch.

In this way, the output selection controller 32 determines which output unit signal from the neural network is used directly or used as the correction error based on the output state signals from the correction neural networks 22 and 31. , And controls the output selection adder 28.

In each neural network, the learning method differs depending on the execution processing method. When the output unit signal is selected and directly output, the learning is performed using the teacher signal T, and the output unit signal as the correction error is output. Will be learned using the corrected teacher signal. Therefore, it is possible to select the learning method of using the teacher signal T or the correction teacher depending on how the error exceeds the allowable range of the neural network in the previous stage.
The description of these learning processes is omitted here.

[0049]

Fourth Embodiment As a parallel neural network 60 according to the fourth embodiment of the present invention, a main neural network 21 and
An output state in which the correction neural network 22 and the output state neural network 33 are connected in parallel to the input to indicate which of the main and correction neural networks 21 and 22 outputs the output unit signal within the allowable error range. Unlike the first embodiment, an output state neural network 33 is independently prepared in order to obtain a signal.

FIG. 9 shows an example of the configuration of the execution process in the execution process mode. The correction neural network 22 outputs the output unit signal within the allowable error range to the learning input signal which causes an error exceeding the allowable range in the output unit signal from the main neural network 21 due to the learning process in the learning mode. The learned connection weight is set. Further, in the output state neural network 33, a connection weight learned so as to independently output an output state signal indicating the presence / absence of an error exceeding the allowable range of the main neural network 21 is set. Based on this output state signal, the output selection controller 23 determines which neural network output unit signal to select, operates the output selection switch 19, and outputs the desired output signal O from the terminal 3. .

As the configuration of the learning process in the learning process mode of FIG. 10, the main neural network 21 is first learned as in the first embodiment. When the learning is completed under the specified learning condition, an error is obtained from the teacher signal T and the output unit signal by the error detector 25, and the output state teacher signal generator 26 determines whether or not there is an error exceeding the allowable range. Store as a teacher signal. Next, the correction neural network 22 is trained using the teacher signal T. The output unit signal from the correction neural network 22 and the teacher signal T coincide with each other within the allowable error range with respect to the learning input signal in which the output unit signal from the main neural network 21 outputs an error exceeding the allowable range. When this is detected by the convergence determiner 39, it is considered that the convergence has occurred, and learning is terminated.

Further, the output state neural network 3
In 3, the learning is performed based on the output state teacher signal from the output state teacher signal generator 26. When the convergence determiner 35 detects that the output unit signal and the output state teacher signal match within the allowable error range, it is regarded as converged and learning is terminated.

If they do not match within the specified number of learnings,
Under the control of the operation mode controller 34, the number of hidden units of the correction neural network 22 and the output state neural network 33 is increased, and these neural networks are learned again. If it does not converge further, the main neural network 21 is re-learned to reduce the number of learning input signals that cause an error exceeding the permissible range, the output state teacher signal is calculated again, and then the correction neural network 22 and the output state are obtained. The neural network 33 may be retrained.

Although a detailed structure is omitted here,
In FIG. 9, an output selection adder 28 is used in place of the output selection switch 19, and the correction neural network 22 is trained using a correction teacher signal having an error exceeding the allowable range in the main neural network 21 as in the second embodiment. Then, the correction error may be output, and the output selection adder 28 may perform addition processing to correct the error.

The fundamental difference between this embodiment and the previous embodiments is that the output state neural network 33 is independently used to output the output state signal. Correction and output state neural network 2
Since 2 and 33 have a higher degree of freedom and can be converged easily and surely, it is possible to reduce the number of hidden units and the number of learning of each neural network.

In this embodiment, the main neural network 2 is used.
1. The correction neural network 22 and the output state neural network 33 can increase the degree of freedom in design, and can realize a large-scale neural network that cannot perform stable convergence at high speed by the parallel neural network of the present invention.

[0057]

Fifth Embodiment As a parallel neural network 70 of the fifth embodiment of the present invention, a main neural network 21 and
The correction neural network 22 and the output state neural network 33 are connected in parallel to the input, and an error exceeding the allowable error of the output state signal from the correction neural network 22 is output state neural network 3
3 shows an example of a configuration of a method of correcting with an output unit signal from 3.

In the configuration of the execution processing in the execution processing mode, as shown in FIG. 11, the output unit signal within the allowable error range from the main neural network 21 with respect to the teacher signal T is output selected as in the second embodiment. Under the control of the controller 36, the output unit signal output from the terminal 3 via the output selection adder 28 and exceeding the permissible error range is output as the correction error from the correction neural network 22. Is added to the output selection adder 28 to correct the error and then output to the terminal 3 to obtain a desired output signal.

The output selection controller 36 causes the correction neural network 22 to output an output state signal indicating the presence or absence of an error exceeding the allowable range of the output unit signal from the main neural network 21, and causes the error to exceed the allowable range. If there is, an output unit signal as an output state correction error from the output state neural network 33 is added, the error is corrected, and a correct output state signal is obtained, and then a control signal is output to the output selection adder 28. .

FIG. 12 shows a configuration example of the learning process in the learning process mode. Similar to the third embodiment, first, learning of the main neural network 21 is started by the control signal from the operation mode controller 34. After learning, error detector 2
In step 5, the output unit signal is subtracted from the teacher signal T to obtain an error, and the error exceeding the allowable range is corrected.
The output state teacher signal generator 26 stores the corrected teacher signal as a correction signal 9, and the output state teacher signal generator 26 stores an output state teacher signal indicating the presence or absence of an error exceeding the allowable range.

Next, the learning of the correction neural network 22 is started using the correction teacher signal and the output state teacher signal by the control signal from the operation mode controller 34, and the correction teacher signal is read from the correction teacher signal generator 29. But let's learn. In this learning process, if the output signal of the output unit signal is within the allowable error range with respect to the learning input signal in which an error exceeding the allowable range has occurred in the main neural network 21 in the convergence determiner 39, it is considered to have converged. To end learning. At this time, the error detector 37 detects the error by subtracting the output state signal from the output state teacher signal from the output teacher signal generator 26, and detects an error. It is stored in the output state correction teacher signal generator 38.

Next, learning of the output state neural network 33 is started using the output state correction teacher signal in accordance with the control signal from the operation mode controller 34. In the learning process, if the output state correction signal is within the allowable error range with respect to the output state correction teacher signal in the convergence determiner 35,
Learning is considered to have converged, and learning is terminated.

If the convergence does not occur under the specified learning condition, the number of hidden units of the output state neural network 33 may be increased and the learning of the output state neural network 33 may be repeated again. If convergence is not detected by the convergence determiner 39 of the correction neural network 22, the number of hidden units may be increased and learning of the correction neural network 22 may be repeated. Alternatively, the learning of the main neural network 21 is started again to reduce the number of learning input signals that cause an error exceeding the allowable range in the main neural network 21, and the correction teacher signal generator 29 corrects the correction teacher signal. May be regenerated and the correction neural network 22 may be learned again. When the learning process ends, the operation mode controller 34 switches the learning process mode to the execution process mode.

As described above, when the correct output state signal cannot be obtained by the correction neural network 22, the output state correction signal is output by using the output state neural network 33, and the output selection controller 36 performs addition processing. After the correction and the correct output state signal are obtained, the control signal is output to the output selection adder 28. As a result, it becomes easy to obtain a desired output signal even for a non-learning input signal.

Although details are omitted here, the embodiment 1-
In 5, the multi-valued output unit signals are respectively obtained for the output unit signals of the main and the correction neural network through the multi-valued threshold circuit, and the teacher signal T or the corrected teacher signal and the multi-valued output unit signal are obtained in the convergence determiner. It is judged that they have converged by detecting that they match each other, and the multi-valued threshold circuit is inserted between each neural network and the output selection switch in the execution processing, or between the output selection adder. You may comprise a neural network.

Further, a multi-valued logic neural network may be constructed by using an output selection additive arithmetic unit instead of the output selection adder.

In the first to fifth embodiments described above, a neural network other than the above-mentioned multilayer neural network may be used as long as it is a neural network that can be trained by using the teacher signal as the correction neural network as shown. Further, as the main neural network 21, a neural network that does not use a teacher signal can be applied.

Further, in the parallel neural network, when the correction neural network at the final stage is learned, the number of hidden units of the correction neural network or the main neural network can be increased and the learning can be made to converge again.

In the parallel neural network, the output selection controller may be omitted and the function thereof may be performed by the correction neural network or the output state neural network.

[0070]

As described above, according to the parallel neural network of the present invention, the main neural network and at least one or more correction neural networks are connected in parallel to the input, and each of them is sequentially learned. Then, using the output state signal from at least the final stage of the correction neural network, the output unit signal from the main and correction neural network is directly output with respect to the input signal, or via the addition operation processing,
Further, in the case of multi-valued arithmetic processing, a desired output signal is output to the learning input signal by outputting through the additive arithmetic processing, and more desired output signals are output to the non-learning input signal. Can be easily obtained.

Compared with the conventional neural network, a small number of hidden layers units or hidden units can be used to equivalently and stably converge with a small number of learning operations, and a desired output signal can be transmitted. It is also possible to realize a parallel neural network having a large number of units having a large number of input signal elements and a large number of teacher signals T.

From these facts, it is necessary to freely design and realize a large-scale neural network, which is difficult to be realized by the conventional technique, in a short time, and artificial learning that requires rapid learning and complete convergence is required. It has a very wide range of effects such as a wide range of applications such as intelligent systems, search systems, data conversion, data compression, multi-valued image processing, and communication systems.

If a plurality of sets of connection weights learned in the multi-valued logic neural network of the present invention are prepared and the connection weights are switched and set, a programmable large-scale variable multi-valued logic circuit with a fixed delay can be easily provided. It can be realized, and a new large-scale multi-valued logic circuit can be realized on the same hardware by learning again in a short time according to the situation. In particular, the parallel neural network of the present invention is applied to a plurality of identical neuro L's having low internal calculation accuracy.
By using the SI chip, it is possible to effectively use the parallel processing function and the flexible multivalued logic operation processing. Also, since the learning input signal can be completely equivalently converged, it is possible to easily construct a large-scale neural network in which the parallel neural networks of the present invention are complicatedly combined.

[Brief description of drawings]

FIG. 1 is an example of a configuration of execution processing of a three-layer neural network according to a conventional method.

FIG. 2 is an example of a configuration of a learning process in a conventional three-layer neural network.

FIG. 3 is an example of a configuration of execution processing of a parallel neural network according to a conventional method.

FIG. 4 is a configuration example of an execution process of the parallel neural network of the present invention in the first embodiment.

FIG. 5 is a configuration example of a learning process of the parallel neural network of the present invention in the first embodiment.

FIG. 6 is a configuration example of an execution process of the parallel neural network of the present invention in the second embodiment.

FIG. 7 is a configuration example of a learning process of the parallel neural network of the present invention in the second embodiment.

FIG. 8 is a configuration example of an execution process of a parallel neural network of the present invention in a third exemplary embodiment.

FIG. 9 is a configuration example of an execution process of the parallel neural network of the present invention in the fourth embodiment.

FIG. 10 is a configuration example of a learning process of the parallel neural network of the present invention in the fourth embodiment.

FIG. 11 is a configuration example of an execution process of the parallel neural network of the present invention in the fifth embodiment.

FIG. 12 is a configuration example of a learning process of the parallel neural network of the present invention in the fifth embodiment.

[Explanation of symbols]

1 3 layer neural network 2 Neural network input signal input terminal 2 ₁ Neural network input signal unit input terminal 2 ₂ Neural network input signal unit input terminal 2 _N Neural network input signal unit input terminal 3 Neural network output signal output terminal 3 ₁ Neural network Output signal unit output terminal 3 ₂ Neural network output signal unit output terminal 3 _M Neural network output signal unit output terminal 4 Input layer 4 ₁ Input layer unit 4 ₂ Input layer unit 4 _N Input layer unit 5 Intermediate layer 5 ₁ Intermediate layer unit 5 ₂ middle layer unit 5 _P middle layer unit 6 output layer 6 ₁ output layer unit 6 ₂ output layer unit 6 _M output layer unit 7 teacher signal input terminal 7 ₁ unit teacher signal input terminal 7 ₂ unit teacher signal No. Input terminal 7 _M unit Teacher signal input terminal 8 Subtractor 8 ₁ Subtractor 8 ₂ Subtractor 8 _M Subtractor 9 Coupling weight controller 10 Coupling weight control input / output terminal 11 Binary threshold circuit 12 First neural network 13th 2 neural network 14 3rd neural network 15 error detector 16 error detector 17 error detector 18 output selection controller 19 output selection switch 20 parallel neural network 21 according to prior art 21 main neural network 210 subtractor 211 combination weight controller 212 coupling weight control input / output terminal 22 correction neural network 220 subtractor 221 coupling weight controller 222 coupling weight control input / output terminal 223 subtractor 23 output selection controller 24 operation mode controller 25 error detector 26 output state teacher signal generator 7 Convergence Determinator 28 Output Selection Adder 29 Corrected Teacher Signal Generator 30 Parallel Neural Network of the Present Invention of First Embodiment 31 Correction Neural Network 32 Output Selection Controller 33 Output State Neural Network 331 Connection Weight Controller 332 Connection Weight Control Input Output terminal 34 Operation mode controller 35 Convergence determiner 36 Output selection controller 37 Error detector 38 Output state correction teacher signal generator 39 Convergence determiner 40 Parallel neural network 41 of the present invention of Example 2 41 Binary threshold circuit 50 Implementation Example 3 parallel neural network of the present invention 60 Example 4 parallel neural network of the present invention 70 Example 5 parallel neural network of the present invention

Claims (13)

[Claims] 1. A main neural network in which at least one correction neural network is connected in parallel, and the main neural network trained on a learning input signal and an output unit signal of the preceding neural network are prepared in advance. The correction neural network that sequentially learns the learning input signal that has an error that exceeds the allowable range compared to the generated teacher signal, and whether there is an error that exceeds the allowable range in the output unit signal of the preceding neural network. Is used as an output state teaching signal, and at least the final stage of the correction neural network further learned so that the output state signal can be transmitted, and output selection for determining the selection of the output unit signal of the neural network based on the output state signal Control means and control signal from the output selection control means A parallel neural network comprising at least output selecting switch means for selecting and transmitting an output unit signal based on the above, and obtaining an output signal. 2. A main neural network in which at least one correction neural network is connected in parallel to the main neural network, and the main neural network learned for a learning input signal and the output unit signal of the preceding neural network are used. Of the learning input signal having an error exceeding the allowable range as compared with the learned teacher signal, and outputting the correction error obtained by sequentially learning the learning input signal using the error as a teacher signal, and the neural network of the preceding stage. A correction neural network at least in the final stage, which is further learned so as to output an output state signal with the presence or absence of an error in the output unit signal exceeding an allowable range as an output state teacher signal, and the neural network based on the output state signal Output that determines the selective addition processing of the unit output signal The output unit signal of the neural network is selected based on the selection control means and the control signal from the output selection control means, and the output unit signal having an error exceeding the allowable range is a correction error. A parallel neural network, which comprises at least output selecting and adding means for adding, correcting and transmitting the output unit signals of the correction neural network, and obtaining an output signal. 3. A main neural network in which at least two correction neural networks are connected in parallel to the main neural network, and the main neural network trained on a learning input signal and the output unit signal of the preceding neural network are used. Compared with the corrected neural network in which the learning input signal having an error exceeding the allowable range is successively learned, and the teacher signal used as the output unit signal of the preceding neural network. A correction neural network that outputs a correction error in which a learning input signal having an error exceeding the allowable range is sequentially learned using the error as a teacher signal, and the allowable range in the output unit signal of the neural network is exceeded. The output status signal with or without the error At least the final stage of the correction neural network that has been further learned so that it can be transmitted, output selection control means that determines the selection or selective addition processing of the output unit signal of the neural network based on the output state signal, and the output selection The output unit signal of the neural network is selected on the basis of the control signal from the control means, and the output unit signal of the correction neural network for outputting the correction error is added to the output unit of the correction neural network of the preceding stage to correct it. A neural network comprising at least an output selecting and adding means for sending out and obtaining an output signal. 4. A parallel neural network according to claim 2, wherein the output unit signal of said neural network is multi-valued by a multi-valued threshold circuit to obtain a multi-valued output unit signal, and the output selective addition means is replaced with an output selective addition means. network. 5. The parallel neural according to claim 3, wherein the output unit signal of the neural network is multi-valued by a multi-valued threshold circuit to obtain a multi-valued output unit signal, and the output selective addition means is replaced with the output selective addition means. network. 6. The main state neural network and at least one output state neural network are connected in parallel to the correction neural network, and the output state teacher signal is learned by the output state neural network,
The output state signal is output to the output selection control means.
The described parallel neural network. 7. The main state neural network and the correction neural network are connected in parallel with at least one output state neural network, and the output state teacher signal is learned by the output state neural network,
3. The output state signal is output to output selection control means.
The described parallel neural network. 8. The main neural network and the correction neural network are connected in parallel with at least one output state neural network, and the output state teacher signal is learned by the output state neural network,
The output state signal is output to the output selection control means.
The described parallel neural network. 9. The neural network according to claim 6, wherein at least the final stage of the correction neural network outputs an output state teacher signal indicating whether or not there is an error in an output unit signal of the neural network that exceeds an allowable range. Further learning is performed so that the state signal can be transmitted, and the output selection control means selects the output unit signal of the neural network based on the output state signals from the output state neural network and at least the final correction neural network, or A parallel neural network characterized by deciding selective addition processing. 10. A neural network as claimed in any one of claims 1-3 and 6-9, comprising a multi-valued threshold circuit for obtaining a multi-valued output unit signal from the output unit signals of the main and correction neural networks. A parallel neural network inserted at least between the neural network and the output selection switch means or the output selection addition means. 11. The parallel neural network according to claim 7, wherein the output unit signal of the neural network is multivalued by a multivalued threshold circuit to obtain a multivalued output unit signal, and the output selection / addition means is replaced with an output selection / addition means. network. 12. A parallel neural network according to claim 8, wherein the output unit signal of said neural network is multi-valued by a multi-valued threshold circuit to obtain a multi-valued output unit signal, and the output selection and addition means is replaced with the output selection and addition means. network. 13. The neural network according to claim 11, wherein at least the final stage of the correction neural network is an output state signal in which the presence or absence of an error in a multilevel output unit signal of the neural network is used as an output state teacher signal. Learn further to be able to send,
In the output selection control means, the selection of the multivalued output unit signal of the neural network or the determination of the selective addition processing is performed based on the output state signals from the output state neural network and at least the final stage correction neural network. A featured parallel neural network.