JPH06124274A - Signal processor - Google Patents

Abstract

PURPOSE:To arbitrarily designate and learn an output value when the pulse density of an input signal is an intermediate value between 0.0 and 1.0 regardless of the output value at the time of inputting the signal whose pulse density is 1.0 and the output value at the time of inputting the signal whose pulse density is 0.0, in a pulse density type neural network. CONSTITUTION:A signal processor in which a network-shaped neural network 1 is formed of plural signal processing units 2, 3, and 4, and signal lines 5 connecting those signal processing units 2, 3, and 4, is equipped with an input signal converting means 6 which inputs the input signal expressed by pulse columns to the neural network 1 by converting it into the combination of the plural pulse columns whose pulse density is 1.0 or 0.0.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a parallel distributed information processing apparatus aiming at artificially realizing the information processing function of a neural network, including character / image recognition, voice recognition, robot control, and image processing. The present invention relates to a signal processing device called a so-called neurocomputer, which is used for processing, natural language processing, associative memory, and the like.

[0002]

2. Description of the Related Art Recently, many attempts have been made to realize a neural network by hardware. Such hardware implementation is roughly classified into an analog type and a digital type. Among the digital type, there is a pulse density type neural network that expresses a signal by a pulse train. Such a pulse density type neural network is based on Japanese Patent Application Laid-Open No. 4-549 as a forward process, and Japanese Patent Application Laid-Open No. 4-1 as a learning process.
Various improved products have been proposed by the present applicant based on 11185. A pulse density type neural network other than the one by the present applicant is disclosed in, for example, Japanese Patent Application Laid-Open No. 1-244567 or US Pat.
Some of them are shown in the specification of 893255,
The method of signal processing differs depending on each type.

Here, signal processing in the pulse density type neural network proposed by the present applicant will be described. First, a bit train is actually used as the pulse train. That is, a pulse train is represented by a sequence of bits that take "0" or "1" as a value, one pulse is represented by a bit having a value of "1", and there is no pulse at a bit having a value of "0". Represent

The value of the signal is represented by the pulse density in the pulse train, that is, the density of bits having the value "1" in the bit train. For example, assuming that the bit string has a length of 10 bits, when a signal having a value of 0.6 is input to this neural network, 6 out of 10 bits are input to this neural network.
Bit strings of which the value is “1” and the remaining four are of the value “0” are input. Since the density never exceeds 1.0, the maximum value of the signal is 1.0.

Each neuron unit in the neural network receives as an input signal a bit string sent from outside the neural network or from some other neuron unit in the neural network. The coupling coefficient representing the strength of coupling between the neuron unit and another neuron unit is also represented by a bit string, and weighting by the coupling coefficient of the input signal is performed by taking the logical product of the input bit string and the coupling coefficient bit string. The neuron unit divides the weighted input signal into several groups, and individually calculates the logical sum of these weighted input signals for each group. Then, a predetermined logical operation is performed on the bit string of the logical sum result obtained for each group to generate a new bit string, and the new bit string is output to the outside of the neuron unit as an output signal. The output signal from such a neuron unit is sent to some other neuron unit in the neural network to be an input signal to that neuron unit, or is output outside the neural network and output from the neural network itself. Become a signal.

Next, learning in such a pulse density type neural network will be described. The pulse density type neural network proposed by the applicant is based on a hierarchical neural network configuration, and learning is performed according to the backpropagation method which is widely used for the hierarchical neural network.
First, an error signal to be given to the neuron unit of the output layer is generated from the output signal output from the neuron unit of the output layer and the teacher signal, and this error signal is given to the neuron unit of the output layer. Then, the neuron unit in the output layer generates an error signal to be given to the neuron unit in the intermediate layer based on the received error signal, and gives this error signal to the neuron unit in the intermediate layer. The same applies hereinafter. In this way, from the output layer to the input layer, the error signal is sequentially given to the previous layer while being generated by the neuron unit of each layer, and at the same time, each layer is based on the received error signal and its own neuron unit. This is a learning method in which the coupling coefficient between the neuron unit in the layer and is updated. Like the input signal and the output signal, the teacher signal and the error signal are represented by a bit string and are represented by the pulse density of the bit string.

In such a pulse density type neural network, consider a signal having a value of 0.0 or 1.0 and a signal having a value in between. As described above, the value of the signal is represented by the density of bits having the value "1" in the bit string. Therefore, the bit takes only two values of "0" and "1", but the signal itself takes a value of 0 or more and 1 or less. Here, in order to avoid confusion with the bit value, the signal value 0 is expressed as 0.0, and the signal value 1 is expressed as 1.0. The values 0.0 and 1.0 of such a signal have a special meaning compared to other values (for example, 0.5). That is,
A signal having a value of 1.0 means that the signal has a pulse density of 1.0. Therefore, all the bits in the pulse train constituting the signal have a value of "1". Which means that there is no bit of value "0".
Similarly, a signal value of 0.0 means that the signal consists only of bits of value "0". On the other hand, when the value of the signal is between 0.0 and 1.0, the bit train having the value "1" and the bit having the value "0" are mixed in the pulse train forming the signal. And
This means that signals having the same value can have a plurality of different bit arrangements. For example, when the pulse length of the signal is 10 and the value of the signal is 0.8, different bit arrangements such as bit arrangement 1) 1111111100 bit arrangement 2) 10101111111 bit arrangement 3) 1110111011 can be taken. However, the signals whose values are 0.0 and 1.0 are, as described above,
Since all the bits are “0” or “1”, there is only one bit arrangement.

In the case of the pulse density type neural network according to the method proposed by the present applicant, signal processing is performed by logically operating signals. Therefore, even if the signals have the same pulse density, the result of the logical operation for those signals will be different if the bit arrangement is different. It is difficult to use if the output of the neural network is different due to the difference in bit arrangement, so in the improved proposal example by the applicant, signal processing is performed while randomly changing the bit arrangement of the input signal and the coupling coefficient, and the average value of those results. Is adopted as the value of the result of signal processing.

When an input signal composed of a pulse train is input to the pulse density type neural network, if the value of the input signal is between 0.0 and 1.0, the value "1" is obtained at a certain moment. Bit is entered and the value is "0" at another moment
Will be entered. The output of the neural network for the bit of the value "1" fluctuates under the influence of the fluctuation of the bit arrangement of the bit string such as the coupling coefficient which is logically operated with the bit in the neural network. In order to know the average value, it suffices to input the bit of the value "1" into the neural network many times and take the average value of the outputs. This is the same as looking at the output of the neural network for a signal of value 1.0. Similarly, the average value of the output of the neural network for a bit of value "0" is equal to the output of the neural network for a signal of value 0.0.

Therefore, for example, when a signal of value 0.8 is input to the neural network, bits of value "1" and bits of value "0" are input to the neural network at a ratio of 0.8: 0.2. Therefore, the output at that time is the output when the signal of the value 1.0 is input and the value 0.0
And the output when the signal is input at a ratio of 0.8: 0.2. For example, when a signal with a value of 1.0 is input, the output is 1.0, when a signal with a value of 0.0 is input, the output is 0.5, and when a signal with a value of 0.8 is input. Output is 1.0 x 0.8 + 0.5 x
0.2 = 0.8 + 0.1 = 0.9. Further, at this time, when the signal of the value 0.6 is input, the output when the signal of the value 1.0 is input and the output when the signal of the value 0.0 is input are 0.6: 0. The total value is 0.8 at a ratio of 0.4.

[0011]

As described above, when the value is 0.
The output when a signal in the middle of 0 and 1.0 is input is determined from the output when a signal of value 1.0 is input and the output when a signal of value 0.0 is input. Conversely, this means that when a signal with a value between 0.0 and 1.0 is input, the output value and the value of 0.0 when the signal with a value of 1.0 is input. This means that it is not possible to learn to output an arbitrary value regardless of the output value when a signal is input. For example, when a value 0.8 signal is input to a neural network having only one input signal, 0.9 is output, and when a value 0.6 signal is input, 0.8 is output. , The output when a signal with a value of 1.0 is automatically input is 1.0, and the output when a signal with a value of 0.0 is input is 0.5. , When a signal with a value of 0.4 is input,
It cannot be learned to output a value other than 0.3, for example 0.2. Thus, the values are 0.0 and 1.0
If you want to learn to input a signal in the middle of and output a desired output value, what output value is output when a signal of value 1.0 is input, and a signal of value 0.0 is output. When inputting, the output value cannot be learned regardless of what output value is learned.

However, if the neural network is designed so that a signal having a value between 0.0 and 1.0 can be directly input and learned as it is, the input data that comes into contact with this constraint is A case may occur in which the user fails to learn after inserting it.

[0013]

A logical product calculating means for calculating a logical product of an input signal expressed by a pulse train and a coupling coefficient expressed by a pulse train, and calculation results by the logical product calculating means are divided into a plurality of groups. And a logical sum calculation means for calculating a logical sum of the calculation results for each group,
By a plurality of signal processing units provided with a logical operation means for performing an logical operation on the operation result by these logical sum operation means to generate an output signal, and a signal line connecting these signal processing units. An input signal for converting a signal input to the neural network represented by a pulse train into a combination of a plurality of pulse trains having a pulse density of 1.0 or 0.0 and inputting the signal in a signal processing device having a mesh-like neural network. A conversion means was provided.

[0014]

When a signal having a value between 0.0 and 1.0 is input and it is desired to learn to output a desired output value, the input signal is instead converted by the input signal converting means. If input as a signal converted into a combination of multiple pulse trains with a value (pulse density) of 1.0 or 0.0 and learning, input an input signal with a value between 0.0 and 1.0. As the output value when the value is set, specify an arbitrary value to be output regardless of the output value when a signal with a value of 1.0 is input and the output value when a signal with a value of 0.0 is input. It becomes possible to learn by doing.

That is, the output value when an input signal having a value between 0.0 and 1.0 is input is a value when a signal having a value of 1.0 and a signal having a value of 0.0 are input. Since it is decided from the output signal value, secure the number of inputs of the neural network so that the same number of pulse train signals as 1.0 or 0.0 can be input to the neural network as the number of input / output pairs to be learned at the same time. If this is done, any number of pairs of intermediate value input and output can be learned by such a learning method.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described with reference to the drawings. First, prior to the description of the embodiments, an outline of a processing example according to the basic idea of the present invention will be described. For example, the following 3
Let us consider a case where a pair of input / output pair input output 0.2 0.7 0.5 0.3 0.7 0.5 are simultaneously learned by a pulse density neural network.

In this case, the input signal is converted as follows, and the converted input signal and the teacher signal are converted into a neural network having three neuron units in the input layer and one neuron unit in the output layer. Give a pair to learn. However, (1.0,0.0,0.0) is the value of the input signal to the first neuron unit of the input layer is 1.0, the value of the input signal to the second neuron unit is 0.0, and the third neuron unit It means that the value of the input signal to is 0.0. The same applies to other converted input signals.

With such conversion of the input signal, instead of inputting a signal having a value between 0.0 and 1.0, a signal having a value of 1.0 or a signal having a value of 0.0 is input. You can learn by doing.

When inputting an input signal to the learned neural network to obtain an output value, the value of the input signal may be converted and input in the same manner as when learning.

A value of 1.0 or 0.
In order to convert into a combination of a plurality of pulse trains of 0, it is sufficient to create a correspondence table between the original input signal and the converted input signal actually input to the neural network.

FIG. 1 shows a structural example based on such an idea. Reference numeral 1 is a pulse density type neural network (neural network) 1 according to the method already proposed by the present applicant and has, for example, a three-layer hierarchical structure. A number of neuron units (signal processing units) 2 are provided, and an appropriate number (here, 5) of neuron units 3 are provided in the intermediate layer.
Are provided, and one neuron unit 4 is provided in the output layer, and each neuron unit 4 is connected by a signal line 5. However, the number of neuron units in each layer is not limited to such a number, and may be set appropriately. Then, an input data conversion unit (input signal conversion means) 6 is provided at the input unit of such a pulse density neural network 1, and an input signal input to the pulse density neural network 1 from the signal line 7 is input. Based on the conversion table stored in the conversion table storage unit 8, the converted input signal is given to the neuron unit 2 in the input layer.
As described above, the conversion table has a value of 0.0 or more and 1.
An input signal of 0 or less is associated with data formed by combining a plurality of pulse trains having a value (density) of 0.0 and a pulse train having a value of 1.0.

Two examples of the configuration of such a conversion table are shown in Tables 1 and 2.

[0023]

[Table 1]

This is the value 1.0 in the converted input signal.
Where the pulse train exists, the values are associated so as to represent the numerical value of the original input signal.

[0025]

[Table 2]

This corresponds to the numerical values of the original input signal in binary notation.

Of course, the correspondence is not limited to these, but the correspondence can be made as appropriate. The point is that the original input signal value and the converted input signal actually input to the neural network are 1: 1. It is sufficient if it corresponds.

[0028]

As described above, the present invention converts the input signal represented by a pulse train into a combination of a plurality of pulse trains having a pulse density of 1.0 or 0.0 for the pulse density type neural network. Since the input signal converting means for inputting the signal is provided, a signal having a pulse density of 1.0 is output as an output value when an input signal having a pulse density between 0.0 and 1.0 is input. It becomes possible to specify and output an arbitrary value regardless of the output value when inputting and the output value when inputting a signal with a pulse density of 0.0. Even for intermediate values The learning function of the pulse density type neural network can be fully exerted.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

[Explanation of symbols]

 1 Neural network 2-4 Signal processing unit 5 Signal line 6 Input signal conversion means

Claims (1)

[Claims] 1. A logical product calculating means for calculating a logical product of an input signal expressed by a pulse train and a coupling coefficient expressed by a pulse train, and a calculation result by the logical product calculating means are divided into a plurality of groups, and A plurality of logical sum calculation means for calculating the logical sum of the calculation results for each group, and a logical calculation means for performing a logical calculation on the calculation results by these logical sum calculation means to generate an output signal In a signal processing device in which a net-like neural network is formed by a signal processing unit of No. 1 and a signal line connecting these signal processing units, the input signal to the neural network expressed by a pulse train has a pulse density of 1.
A signal processing device comprising an input signal converting means for converting into a combination of a plurality of pulse trains of 0 or 0.0 and inputting.