JPH0784984A - Neural network circuit - Google Patents

Abstract

PURPOSE:To obtain a circuit which not only easily expands a network scale, but also obtains a high processing speed by providing a 1st select signal for selecting an input and an output and a 2nd select signal for selecting an input and an output differently from the 1st select signal. CONSTITUTION:When input data are supplied, the 1st select signal 103 is asserted and feature data are inputted to the neural network circuit 100. When the input ends, the 1st select signal 103 is negated. To obtain a process result corresponding to the feature data, the 2nd select signal 104 is asserted and the process result 102 is outputted from the neural network 100. Thus, the input and output of the data are performed with the two kind of select signals 103 and 104 and, for example, while the 1st select signal 103 is made common to all neural network circuits 100, the 2nd select signal 104 is made to characteristic signals to easily expand the network scale and load the input data to the respective neural network circuits 100 at the same time.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network circuit for recognizing an image.

[0002]

2. Description of the Related Art In recent years, much attention has been paid to the field of neural networks for information processing, and these neural networks are realized by imitating neural signal processing of the human body. At present, various neural networks have been proposed, but one of them is image recognition by a network of quantized neurons. National Conference Proceedings 77-8
Page 0, or "Multi-Functional Layered Networkusin
g Quantizer Neurons 』Computer World '90, November
1990) as an example.

FIG. 5 shows a network of quantized neurons. In this network, from the bottom to the first layer, the second layer, and the third layer, the quantized neurons indicated by circles in the figure are connected in a multilayer network, and the fourth layer is the teacher input layer and the fifth layer It has a structure in which normal artificial nerve cells with multiple inputs and one output are connected to the layer (the last layer of the uppermost stage).

As an example of the characteristic data given to each layer of this network, pixel values obtained by dividing an image into eight vertically and eight horizontally, and a horizontal difference between the 8 × 8 pixel values are divided. Value and the difference value in the vertical direction of the 8 × 8 pixel values.

As shown in the figure, the first layer consists of 64 quantized neurons corresponding to the number of 8 × 8 pixels, and each of these quantized neurons has a quantized signal input terminal S (see FIG. 6), the pixel value as the characteristic data is input to each quantized signal terminal S as shown in the figure.

A horizontal difference value of pixel values is input as feature data to the quantized signal input terminal S of the quantized neuron of the second layer, and the quantized signal input terminal S of the quantized neuron of the third layer is input. A vertical difference value of pixel values is input as the feature data. The teacher input layer, which is the fourth layer, changes the synaptic coupling coefficient of each neuron in the final layer according to the teacher signal.

The final layer is an output layer of the answer, and for example,
6 if the answer is for 62 alphanumeric characters
It consists of two normal multi-input, one-output artificial nerve cells. Here, a normal multi-input / single-output artificial nerve cell is one that takes the product sum of each input and the synaptic weight of that input and outputs the sum.

As shown in FIG. 9, each neuron in the output layer multiplies each input data by a weight Wn (n is an input number) determined by learning, and then outputs the sum of them. .

The quantized neuron, as shown in FIG.
In addition to the quantized signal input terminal S, a selection signal input terminal R,
And a plurality of outputs. The signal input from the selection signal input terminal R is the input value x from the quantized signal input terminal S.
Then, for example, the weight τ according to the equation (1) is applied and the result is output to each output.

Τj = 1-β (| j−x |) (1) where | j−x | represents the absolute value of j−x, and j represents the output number of the quantized neuron.

In the quantized neuron of FIG. 7, the feature data input from the quantized signal input terminal S has eight input levels and eight output levels. For example,
Assuming that the function of 1-β is shown in FIG. 10, when the value of the input from the quantized signal input terminal S is 0,
The value appearing at the eight outputs of the quantized neuron is the value given to the selection signal input terminal R multiplied by the coupling coefficient shown in FIG.

Regardless of the value given to the selection signal input terminal R, when the input value at the quantized signal input terminal S is 3, the respective coupling coefficients are, as shown in FIG. This is the same as when the coupling coefficient shown in the figure is slid to the right by three. That is, by adding 3 to the output number in FIG. 7A, the coupling coefficient when the input value of the quantized signal is 3 can be obtained. However, when it exceeds 7, the coupling coefficient is set to 0.

The maximum value is given to the input given to the selection signal input terminal R of the quantization neuron of the first layer. For example, if the operation in the quantized neuron is to be performed by an 8-bit operation, the coupling coefficient in the figure is used for the first layer,
When the value given to the quantized signal input terminal S is level 0, selection signal input = FF output number 0 = FF output number 1 = 7F output number 2 = 0 output number 3 = FF output number 4 = 0 output number 5 = 0 output number 6 = FF output number 7 = 0 (hexadecimal notation).

As shown in FIG. 6, when a network is constituted by quantized neurons having 8 input levels and 8 output levels, the quantized signal input terminal S is connected to the quantized neuron of the first layer. The selection signal input terminal R of the second layer quantization neuron is connected to each of the eight outputs, and the selection signal input of the third layer quantization neuron is connected to each of the eight outputs of the second layer quantization neuron. Terminal R
Are connected. That is, in this network, the total number of outputs of the quantized neurons in the third layer, which branches from one quantized neuron in the first layer to each quantized neuron in the second layer in a tree shape, is 512.

Connected to the quantized signal inputs of the eight quantized neurons of the second layer, which are connected to the eight outputs of the quantized neurons of the first layer, and to the eight outputs of the quantized neurons of the second layer. The quantized signal inputs of the eight third-layer quantized neurons are the horizontal difference at the same position as the pixel value as the feature data input to the quantized signal input terminal S of the first-layer quantized neuron. A value and a vertical difference value are given respectively.

The signals input to the selection signal input terminals of the quantized neurons of the second and third layers are the outputs of the quantized neurons of the first and second layers, respectively.

Therefore, the output of the quantized neuron of the third layer has three coupling coefficients, that is, the coupling coefficient of the neurons of the first layer, the coupling coefficient of the neurons of the second layer, and the coupling coefficient of the neurons of the third layer. It is the product of the selection signal input values of the quantized neurons of the first layer.

One multi-input one-output neuron in the final layer is
Output of the number of quantized neurons 64 × 512 in the third layer,
An input is obtained by multiplying each of them with the coupling coefficient of the fourth layer (teacher input layer).

Next, regarding learning by this neural network, the Hebb's learning rule will be described here. In Hebb's learning rule, the amount of change in the synaptic weight of a neuron is given in proportion to the magnitude of the input signal. That is, as a result of learning, a path having a large input signal can more easily pass the input signal. In the above neural network, the Hebb's learning rule is applied to the final input multi-input single-output neuron.

When the above-described neural network is implemented by hardware, for example, by an LSI or a breadboard, the number of inputs / outputs is determined from physical constraints. Here, as shown in FIG. 5, 64 × 3 inputs, 62
The output neural network is a unit of neural network circuit.

FIG. 11 shows an example of the configuration of a conventional neural network circuit. Reference numeral 200 is a neural network circuit configured by a multilayer neural network model using quantized neurons as shown in FIG. 20
1 is characteristic data to be input to the neural network circuit 200, 202 is an output result of processing obtained from the neural network circuit 200, 203 is a selection signal for controlling data input / output of the neural network circuit 200,
Reference numeral 250 is a weight memory for storing synapse weights of neurons of the neural network circuit 200. In the figure, the same numbers are the same.

When supplying input data to this neural network circuit, the selection signal is asserted to input the characteristic data to the neural network circuit. When the input is completed, the selection signal is negated. When the processing result for the characteristic data is obtained, the selection signal is asserted again and the processing result is output from the neural network circuit. In this way, input / output control is performed with a single select signal.

FIG. 12 shows an example of a conventional neural network circuit expansion method. 200A and 200B
Is the same as the neural network circuit 200. Reference numerals 204 and 205 are selection signals for controlling data input / output of the neural network circuits 200A and 200B, and 250A and 250B are the same as 250.

This is the number of inputs doubled.
Since the selection signal is unique to each neural network circuit, it is applied to only one neural network circuit selected by the selection signal for both input and output.

In the neural network circuit using the quantized neuron, when expanding the number of inputs, different input data are input to different neural network circuits, and the output value of the output neuron having the same number in each neural network circuit. Is added and the output neuron with the maximum value is taken as the answer.

On the other hand, when expanding the number of outputs, the same input data is input to all neural network circuits, the output neurons having the maximum output value of each neural network circuit are compared, and the larger one is answered. To do.

[0027]

When a neural network is to be realized by hardware, the biggest problem is the network scale due to its physical limitation. The more complex the processing target of the neural network, the larger the required network scale.

On the other hand, since there is always a limit to the network scale that can be realized by hardware, in order to solve this, it is usually necessary to arrange a plurality of identical neural network circuits to expand the network scale and perform parallel processing. To improve performance.

However, the conventional neural network circuit has only one selection signal, and when a plurality of neural network circuits are arranged and expanded, the same data is input to all the expanded neural network circuits, or all the expanded neural network circuits are input. Since it is impossible to simultaneously output from the neural network circuit, it is necessary to select the neural network circuits one by one and then input / output, which causes a problem such as deterioration of processing performance.

The present invention has been made as a result of researching the drawbacks of the conventional neural network circuit and its expansion method as described above. The network scale of the neural network can be easily expanded, and the processing capacity is not limited. However, it is to provide a neural network circuit that can achieve a higher processing speed than ever before.

[0031]

In order to achieve the above object, a neural network circuit according to claim 1 of the present application has a multi-layered structure that outputs a result of recognition processing of given feature data by executing a network operation. A neural network circuit, comprising: a first selection signal for selecting an input / output of the neural network circuit; and a second selection signal for selecting an input / output of the neural network circuit different from the first selection signal. It is characterized by.

A neural network circuit according to a second aspect is an extension of the neural network circuit according to the first aspect, and is provided with a plurality of neural network circuits, and the first of all neural network circuits is provided.
The number of inputs and outputs is expanded by making the selection signal common and making the second selection signal of each neural network circuit unique.

A neural network circuit according to a third aspect is an extension of the neural network circuit according to the first aspect, and is provided with a plurality of neural network circuits, and the first of all neural network circuits is provided.
The selection signal is common, the second selection signal of each neural network circuit is unique, and the number of inputs is expanded by having addition means for adding the outputs of the respective neural network circuits. It is a thing.

A neural network circuit according to a fourth aspect is an extension of the neural network circuit according to the first aspect, wherein a plurality of neural network circuits are provided, and the first selection signal of all neural network circuits is common. The second selection signal of each neural network circuit is unique, and the number of outputs is expanded by having a comparison means for comparing the outputs of the neural network circuits.

[0035]

In the neural network circuit according to the first aspect of the present invention, with the above-mentioned configuration, the first selection signal and the second selection signal are respectively supplied with signals for controlling different input / output, so that the network scale can be easily expanded. Can be planned.

According to the neural network circuit of the second aspect, by using the neural network circuit of the first aspect with the above configuration, the first selection signal is common to all the neural network circuits, and the second selection signal is used.
By supplying a selection signal specific to each neural network circuit, the network scale can be easily expanded.

According to the neural network circuit of the third aspect, the number of inputs of the network can be easily expanded by using the neural network circuit of the first aspect with the above configuration.

According to the neural network circuit of the fourth aspect, the number of outputs of the network can be easily expanded by using the neural network circuit of the first aspect with the above configuration.

[0039]

【Example】

(Embodiment 1) FIG. 1 shows the configuration of an embodiment of the neural network circuit according to claim 1 of the present invention. 100 is
For example, it is a neural network circuit composed of a multilayer neural network model using quantized neurons as shown in FIG. 101 is characteristic data input to the neural network circuit 100, 102 is an output result of processing obtained from the neural network circuit 100,
103 is a first selection signal for controlling data input / output of the neural network circuit 100, 104 is a second selection signal of the neural network circuit 100 different from 103,
A weight memory 150 stores the synapse weights of the neurons of the neural network circuit 100. In the drawings, the same reference numerals have the same functions.

When supplying the input data to the neural network circuit 100, the first selection signal 103 is asserted to input the characteristic data to the neural network circuit. When the input is completed, the first selection signal 103 is negated. When the processing result for the characteristic data is obtained, the neural network circuit 100 outputs the processing result 102 by asserting the second selection signal 104.

When the neural network is to be realized by hardware, the biggest problem is the network scale due to its physical limitation. The more complex the processing target of the neural network, the larger the required network scale.

On the other hand, since the network scale that can be realized by hardware is always limited, in order to solve it, it is usually necessary to arrange a plurality of the same neural network circuits to expand the network scale and perform parallel processing. To improve performance.

Therefore, as described above, by controlling the input / output of data with two kinds of selection signals, for example, the first selection signal of all neural network circuits is made common and the second selection signal is made unique. If it is set to "1", not only the network scale can be easily realized, but also input data can be loaded into each neural network circuit all at the same time, so that the processing performance is improved.

The neural network circuit of the present invention can be applied not only to the quantized neuron, but also to a neural network using a simple perceptron for outputting the sum of sums of products of inputs and their synapse weights as shown in FIG. .

(Embodiment 2) FIG. 2 shows an embodiment of the neural network circuit according to claim 2 of the present application. Here, a method for expanding the number of outputs to double the network scale will be described.

Reference numerals 100A and 100B are the same as those of the neural network circuit 100, 102 is an output of the neural network circuits 100A and 100B, 105 is a first selection signal common to the neural network circuits 100A and 100B, and 106 is a second of the neural network circuit 100A. The selection signal, 107 is the second selection signal of the neural network circuit 100B, 150A and 15
OB is the same as 150.

First, the input of characteristic data to the neural network will be described. 100A and 100B
Both neural network circuits are selected by the common first selection signal 105, and the feature data are simultaneously input to the neural network circuits 100A and 100B.

Next, regarding the output, after recognition processing, 10
Using the second selection signal 106 of 0A, the neural network circuit 10A
The recognition result 102 at 0A is output. Continued 100B
From the neural network circuit 100B using the second selection signal 107 of
The recognition result 102 is output. These two values are compared and the answer as the recognition result of the entire network is obtained.

If the result is wrong, only the neural network circuit of the neural networks 100A and 100B having the neuron having the output number to be fired is trained using the Hebb's learning rule, and the other neural network circuit is trained. Will not be done. With this method, the number of outputs can be doubled.

Although the case of doubling the number of outputs has been described here, the present invention can be easily applied to the case of k times by using k neural network circuits of the first embodiment. it can. The present invention can also be applied to learning by using not only Hebb's learning rule but also the steepest descent method.

(Embodiment 3) FIG. 3 shows an embodiment of the neural network circuit according to claim 3 of the present application. A method for expanding an N-input M-output neural network circuit to a 2N-input M-output network scale will be described.

Reference numerals 100A and 100B are the same as those of the neural network circuit 100, and 105 is the first selection signal of the neural network circuits 100A and 100B.
Reference numeral 06 is the second selection signal of the neural network circuit 100A, 107 is the second selection signal of the neural network circuit 100B, and 108 is the neural network circuit 1
The output of 00A, 109 is the neural network circuit 1
00B is an output, 110 is an adder that inputs 108 and 109, and adds them, and 111 is an output of the adder 110.

First, the input of feature data to the neural network will be described. The neural network circuit 100A is selected by the second selection signal 106 of 100A, and half of the input data of 2N is input to 100A. Subsequently, the neural network circuit 100B is selected by the second selection signal 107 of 100B, and the remaining half of the 2N input data is supplied to 100B.

Next, regarding the output, the numbers of the output neurons of the neural network circuits 100A and 100B are regarded as the same number. After the recognition processing is executed, the same numbers of the output neurons of the neural network circuits 100A and 100B are added by the adder 110, and the number of the output neuron having the largest value is used as the recognition result. If the result is wrong, learning is performed using the Hebb's learning rule for the neurons of the same output number to be fired in both the neural network circuits 100A and 100B. With this method, the number of inputs can be set to 2N.

Although the case of doubling the number of inputs has been described here, the present invention can be easily applied to the case of k times by using k neural network circuits of the first embodiment. it can. The present invention can also be applied to learning by using not only Hebb's learning rule but also the steepest descent method.

(Embodiment 4) FIG. 4 shows an embodiment of the neural network circuit according to claim 4 of the present application. A method for expanding the N-input M-output neural network circuit to the N-input 2M-output network scale will be described.

Reference numerals 100A and 100B are the same as those of the neural network circuit 100, reference numeral 105 is the first selection signal of the neural network circuits 100A and 100B, and 1
Reference numeral 06 is the second selection signal of the neural network circuit 100A, 107 is the second selection signal of the neural network circuit 100B, and 108 is the neural network circuit 1
The output of 00A, 109 is the neural network circuit 1
00B is an output, 110 is a comparator that receives 108 and 109 as inputs, and 113 is an output of the comparator 112.

First, the input of feature data to the neural network will be described. 100A and 100B
Both of the neural network circuits are selected by the common first selection signal 105, and the feature data are simultaneously input to the neural network circuits 100A and 100B.

Next, regarding the output, after the recognition processing is executed, the neural network circuits 100A and 100
The numbers of the output neurons having the largest value among the output neurons of B are input to the comparator 112 as the outputs 108 and 109, respectively. Then, the larger one of the two values is output as the recognition processing result 113. If the result is wrong, only the neural network circuit of the neural network circuits 100A and 100B having the output neuron of the output number to be fired is used by using Heb's learning rule, and the other neural network circuit is Not performed. With this method, the number of outputs can be set to 2M.

Although the case of doubling the number of outputs has been described here, the present invention can be easily applied to the case of k times by using k neural network circuits of the first embodiment. it can. The present invention can also be applied to learning by using not only Hebb's learning rule but also the steepest descent method.

[0061]

As described above in detail, according to the present invention, when a number of inputs larger than N is handled in a N-input M-output neural network circuit, N
By using a plurality of input M-output neural network circuits, it is possible to easily expand the network scale of the neural network and realize a neural network circuit that has a processing speed faster than the conventional one. It is possible.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a neural network circuit according to a first embodiment of the present invention.

FIG. 2 is a configuration diagram of a neural network circuit according to a second embodiment of the present invention.

FIG. 3 is a configuration diagram of a neural network circuit according to a third embodiment of the present invention.

FIG. 4 is a configuration diagram of a neural network circuit according to a fourth embodiment of the present invention.

FIG. 5 is a block diagram of a multilayer neural network with quantized neurons.

FIG. 6 is a block diagram of a quantized neuron.

FIG. 7 is an explanatory diagram of a setting example of a coupling coefficient.

FIG. 8 is an explanatory diagram of a setting example of a coupling coefficient.

FIG. 9 is a block diagram of a normal multi-input / single-output neuron.

FIG. 10 is an explanatory diagram of an example of setting a coupling coefficient.

FIG. 11 is a configuration diagram of a conventional neural network circuit.

FIG. 12 is a configuration diagram of a conventional neural network circuit expansion method.

[Explanation of symbols]

100, 100A, 100B Neural network circuit 101 Input of feature data 102 Output of processing result 103, 105 First selection signal 104, 106, 107 Second selection signal 108, 109 Output 110 Adder 111 Output of adder 112 Comparator 113 Output of comparator 150, 150A, 150B Weight memory 200, 200A, 200B Neural network circuit 201 Input of feature data 202 Output of processing result 203, 204 Selection signal 250, 250A, 250B Weight memory

Claims (4)

[Claims] 1. A multi-layer neural network circuit for outputting a result of recognition processing of given feature data by executing a network operation, wherein the input / output of the neural network circuit is selected.
A neural network circuit, comprising: a selection signal; and a second selection signal that is different from the first selection signal and that selects an input / output of the neural network circuit. 2. A plurality of neural network circuits according to claim 1, wherein the plurality of neural network circuits share a first selection signal, and each of the plurality of neural network circuits has a respective second selection signal. Neural network circuit characterized by expanding the number of inputs and outputs by making it unique. 3. A plurality of neural network circuits according to claim 1, wherein the plurality of neural network circuits share a first selection signal, and the plurality of neural network circuits each have a second selection signal. A neural network circuit, which is unique and has the number of inputs expanded by including an adding means for adding the outputs of the respective neural network circuits. 4. A plurality of neural network circuits according to claim 1, wherein the plurality of neural network circuits share a first selection signal, and each of the plurality of neural network circuits has a respective second selection signal. A neural network circuit, which is unique and has an expanded number of outputs by including a comparison means for comparing the outputs of the respective neural network circuits.