JPH04295976A - Neural network circuit - Google Patents

Abstract

PURPOSE:To obtain a function equivalent to the one using a multiplication circuit and to reduce a circuit scale and power consumption by performing a logical operation based on a size decision result between an input signal and a load coefficient. CONSTITUTION:When an identification area of arbitrary shape is formed in each circuit, input signals are inputted from (n)(n : integer >=1) input terminals 1. A size decision means 2 is provided with proper first load coefficient and second load coefficient for respective input terminal 1. The size decision means 2 performs size decision on the (n) input signal and the first load coefficient and the second load coefficient in accordance with those input signals, and outputs prescribed decision results. A logical operation means 3 performs a prescribed logical operation based on (n) size decision results from the size decision means 2. The computed result of the logical operation means 3 is outputted from an output terminal 4. In such a way, the identification area can be formed without using the multiplication circuit using large quantity of hardware.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network circuit, and more particularly to a neural network circuit that can achieve high performance with a small-scale neuron circuit suitable for LSI implementation.

[0002] Neural network circuits model biological neural networks to perform pattern recognition processes such as character recognition and voice recognition, which were difficult to perform with conventional Neumann type computers.
It realizes optimization problems, robot control, etc. Conventional Neumann-type computers perform sequential processing according to a program, which takes an enormous amount of calculation time, whereas neural network circuits have neuron circuits that can perform operations in parallel, resulting in extremely high processing speeds. .

[0003]Furthermore, the functions of neural network circuits are realized by changing connection information between neurons through learning. For this reason, it has the characteristic that even problems whose processing procedures are difficult to formulate into rules can be implemented as long as there are learning materials. If the system is operated while constantly learning, a flexible system can be constructed that can follow changes in desirable functions over time due to changes in the environment.

[0004] Furthermore, since neural network circuits are configured by connecting many identical neuron circuits to form a network, even if there is a faulty circuit, it can be easily replaced with another normal circuit and operated, so it is implemented as an LSI. This makes it possible to achieve high defect tolerance in many cases.

[0005]

2. Description of the Related Art A neural network circuit is constructed by connecting a large number of neuron circuits, each having a neuron circuit corresponding to a nerve cell as a unit. FIG. 6 shows the symbol of a neuron circuit. One neuron circuit 51 receives input signals from a plurality of input terminals x1, x2, xn, has a weighting coefficient for each input signal, and changes the strength of coupling according to the weighting coefficient. The error with the input is calculated, all the results are added together to determine the output, and the output is output from the output terminal y. The structure of the neural network circuit is determined by the connections of these neuron circuits.

FIG. 7 shows a two-layer neural network circuit. This structure is the simplest structure. In the figure, the layer of input terminals x1, x2, and xn is called an input layer, or a first layer, and the layer of neuron circuits is called a second layer 71, or an output layer. Signals from each input terminal are input to all neuron circuits in parallel, so that each neuron circuit can process input signals in parallel. When an input signal is applied, specific neuron circuits respond to perform processing such as recognition. However, since the processing capacity of this two-layer neural network circuit is not very large, a three-layer neural network circuit is generally used.

FIG. 8 shows a three-layer neural network circuit. In the case of a neural network circuit with a three-layer structure, the neuron circuit layer in the second layer 71 in FIG. 7 is called an intermediate layer or hidden layer, and the neuron circuit layer in the third layer 72 in FIG. 8 is called an output layer. . This third layer 72 uses the output of the neuron circuit of the second layer 71 as input, and the second layer 71
It may have exactly the same structure as , or it may have a different structure. First, when the second layer 71 and the third layer 72 have exactly the same structure, the output signal of each intermediate layer 71 is input to the neuron circuits of all the output layers 72. On the other hand, the second
If the layer 71 and the third layer 72 have different structures, the neuron circuit of the output layer 72 is simply an OR circuit (in this example, O
R circuit). In this case, since the output of the intermediate layer 71 is only connected to one neuron circuit of the output layer 72, the circuit scale is significantly reduced, and at the same time, sufficient performance is maintained for use in pattern recognition, etc. There is.

FIG. 9 shows the basic configuration of an ellipsoidal discrimination type of a conventional neuron circuit. Further, FIG. 10 shows the basic configuration of a polyhedral discrimination type of a conventional neuron circuit. Furthermore, Figure 11
represents the transfer characteristics of the threshold circuits 85 and 95. The configuration of FIG. 9 includes a plurality of input terminals, an arithmetic circuit 81, a square circuit 82, an arithmetic circuit 83, an adder circuit 84, a threshold circuit 85, and an output terminal 86.
The configuration shown in FIG. 10 includes a plurality of input terminals and a subtraction circuit 9.
1, an absolute value circuit 92, a multiplication circuit 93, an addition circuit 94, a threshold circuit 95, and an output terminal 96.

The configurations shown in FIGS. 9 and 10 both have n (n is 1)
(integer greater than or equal to) input terminals. 1st for each input terminal
There are a weighting factor (wi) and a second weighting factor (whi), and there are a total of 2n weighting factors in the entire neuron circuit. The subtraction circuits 81 and 91 input the input signal x1 and the first weighting coefficient (wi
), and the subtraction result is converted into a positive number by the square circuit 82 or the absolute value circuit 92. Next, multiplier circuits 83 and 93 add a second weight coefficient (
multiply by wh). Addition circuits 84 and 94 accumulate calculation results for the respective inputs and weight coefficients, apply the results to threshold circuits 85 and 95 to determine the output, and output the results from output terminals 85 and 95.

Further, the threshold circuits 85 and 95 have transfer characteristics as shown in FIG. For example, a case where the number of inputs is two will be explained. Figure 12 shows two conventional neuron circuits.
Indicates the identification area for input. The same figure (A) shows the identification area of the ellipsoidal identification shape shown in FIG.
This is the identification area of the polyhedral identification shape shown in . Load factor w1
The vector w created by and w2 represents the center of the identification area. Depending on whether the characteristics of the threshold circuits 85 and 95 are as shown in (A) or (B) in FIG. 10, the identification area will be the closed area (shaded area) in FIG. 12 or the outside area. is decided. In the circuit of FIG. 9, the shape of the identification area is (
It is an ellipse as shown in A), and the threshold of the threshold circuit 85 is h.
Then, its radius is in the direction of input x1

[Math 1] Also, in the direction of input x2

[Math. 2] When the number of inputs is four or more, the identification region becomes a superellipsoid, so this method is called an ellipsoid identification type.

On the other hand, in the circuit of FIG. 10, the shape of the identification area is a rhombus as shown in FIG. 11(B), and its dimensions are h/wh1 in the input x1 direction and h/wh2 in the input x2 direction. When the number of inputs is four or more, the identification area becomes a superpolyhedron, so this method is called a superpolyhedron identification form.

Furthermore, FIG. 13 shows a discrimination area in a case where the second weight coefficients of a conventional neuron circuit are equal and there are two inputs. 3A shows a spherical discriminant, and FIG. 2B shows a regular polyhedral discriminant. When the second weighting coefficients have the same value (wh) for n inputs of the neuron circuit, the discrimination area of the circuit in FIG. 9 becomes a spherical discrimination type, and the circuit in FIG. 10 becomes a regular polyhedral discrimination type. As a result, if the number of inputs is two, the shape of the identification area in FIG. 9 becomes as shown in FIG. 13(A), and
The shape of the identification area of 0 is as shown in (B).

[0013]

[Problem to be solved by the invention] However, FIG.
When constructing a neural network circuit by combining a large number of neuron unit circuits shown in FIG. 10, the more neuron circuits there are, the more complex identification regions can be set. In order to express an arbitrary-shaped identification region, the identification region is formed as a collection of a plurality of hyperellipsoids or hyperpolyhedra by performing logical sum processing in the output layer 72 as shown in FIG. FIG. 14 shows an example of forming an identification area in the case of two inputs using a plurality of conventional neuron circuits. In the figure, the part indicated by the solid line is the target identification area a.
The part indicated by the dotted line is the identification area b of the neuron circuit.
It is. (A) and (B) of the figure each show an example of approximating a target identification region of a given shape using a plurality of neuron circuits. 3A shows an ellipsoid identification shape, and FIG. 2B shows a polyhedral identification shape. From the same figure, in order to accurately match the identification area b of the neuron circuit to the target identification area a of an arbitrary general shape, it is necessary to eliminate the gap.
It can be seen that an extremely large number of neuron circuits are required. For this reason, the circuit scale (amount of hardware) required to configure a neural network circuit becomes extremely large.
As a result, problems arise in that the device becomes larger and the power consumption becomes extremely large. Therefore, it is essential to reduce the circuit scale of the neuron circuit as a constituent unit as much as possible. However, in conventional neuron circuits,
As is clear from the figure, a large number of subtraction circuits 81, 91, squaring circuits 82, absolute value circuits 92, multiplication circuits 83, 93, and addition circuits 84, 94 are required corresponding to the number of inputs. In particular, when implementing a neural network circuit using a digital circuit, the circuit scale of the multiplier circuit 83 and the squaring circuit 82 that uses it is large, resulting in an enormous amount of hardware, and there is a problem that a large number of neuron circuits cannot be integrated into LSI. . In addition, since the multiplication circuit consumes a large amount of power,
There is also the problem that the overall power consumption becomes extremely large due to the simultaneous operation of a large number of multiplier circuits.

The present invention has been made in view of the above points, and
It has the same functionality as a neural network circuit composed of neuron circuits that used conventional multiplication circuits,
The purpose is to provide a neural network circuit that reduces circuit size and power consumption.

[0015]

[Means for Solving the Problems] FIG. 1 shows a diagram of the basic configuration of the present invention. Each neuron circuit executes calculations in response to multiple input signals input to the neural network circuit, which is constructed by connecting the input and output terminals of a large number of neuron circuits, and all or part of the neural network circuit is In a neural network circuit that uses the output value of the neuron circuit as the output signal of the neural network circuit, and controls the function of the neural network circuit by changing the shape of the discrimination region depending on the size of the weighting coefficient of each neuron circuit, each of the neuron circuits Inputting an input signal when forming an arbitrary-shaped identification area
(n is an integer greater than or equal to 1) input terminals (1) and each input terminal (1) has its own first weighting coefficient and second weighting coefficient, and n input signals. and a magnitude determination means (2) for determining the magnitude of the first load coefficient and the second load coefficient corresponding to the input signal and outputting a predetermined determination result.
), a logical operation means (3) that performs a predetermined logical operation based on the n size determination results from the size determination means (2), and an output terminal (4) that outputs the operation result of the logical operation means (3). has.

[0016]

[Operation] In a plurality of neuron circuits constituting a neural network circuit, each neuron circuit has an input signal and a corresponding weighting coefficient, determines the magnitude relationship between the weighting coefficient and the input signal, and uses the result to determine a predetermined value. By configuring a neuron circuit that performs logical operations and outputs the results of logical operations, it is possible to form an identification area without using a multiplication circuit that requires an enormous amount of hardware. When configuring a neural network circuit, the larger the number of neuron circuits, the closer the target identification area will be. Therefore, since no multiplication circuit is used, the amount of hardware does not increase even if many neuron circuits are used, and power consumption, which increases in proportion to the amount of hardware, is also reduced.

[0017]

Embodiment FIG. 2 shows the basic configuration of a neuron circuit according to an embodiment of the present invention. The neuron circuit in the figure has multiple input terminals, a subtraction circuit 11, an absolute value circuit 12, and a magnitude determination circuit 1.
3. Consists of an AND circuit 14 and an output terminal 15. This neuron circuit has n inputs (x1, x2,...xn
) for n first loading coefficients (w1, w2,...wn)
and n second load coefficients (wh1, wh2,...whn). First, the subtraction circuit 11 receives input signals (x1, x2,
...xn) and the weighting coefficient, and input the result to the absolute value circuit 12. The absolute value circuit 12 converts the result of the subtraction circuit 11 into a positive number and inputs it to the magnitude determination circuit 13. The magnitude determination circuit 13 determines the magnitude of the input positive number and the value of the second weighting coefficient, and outputs the result to the output terminal 15.

The above will be explained with respect to the input signal x1, for example. Assuming that the input is x1, first, x1 is input to the subtraction circuit 11. The subtraction circuit 11 subtracts x1-w1 and outputs the result to the absolute value circuit 12. The absolute value circuit 12 sets the input subtraction result to |x1-w1|, and outputs it to the magnitude determination circuit 13. The magnitude judgment circuit 13 outputs the output of the absolute value circuit 12: 1 when |x1-w1|≦wh1; 0 when |x1-w1|>wh1
…
(1) Finally, the outputs of all the magnitude judgment circuits 13 are ANDed in the AND circuit 14, and the result is taken as the output of the neuron circuit.

[0019] Also, the output of the magnitude determination circuit 13 is expressed as |x
1 - w1 | 0 when ≦ wh1 | x1 - w1 | 1 when > wh1
…
(2) The following configuration is also possible. Also, instead of taking the logical product of the outputs of the magnitude determination circuit 13, the logical sum NOR,NA
You may use logic such as ND. This is related to what kind of identification region range the neuron circuit should have.

FIG. 3 shows an identification area of a neuron circuit according to an embodiment of the present invention. FIG. 5A shows a case where the input of a general neuron circuit and the value of the second weighting coefficient are different. In this case, when the number of inputs is two (w1, w2), the shape of the identification area formed by the inneuron circuit becomes a rectangle as shown in (A). Also, when the number of inputs is 3 (w1, w2,
w3) becomes a rectangular parallelepiped, and when the number of inputs is 4 or more, it becomes a hypercuboid. Taking the AND of the outputs of the magnitude determination circuit 13,
When the output is the output of a neuron circuit (FIG. 3), the identification area range is a closed area surrounded by a hypercuboid.

Further, if the output of the magnitude determination circuit 13 is subjected to NAND logic in the AND circuit 14, the identification area range is outside the closed area surrounded by the hypercuboid.

If the output of the magnitude determination circuit 13 is the opposite of (1) as shown in (2) above, and the logic of the AND circuit 14 is NOR of the output of the magnitude determination circuit 13,
If logic is followed, the identification area range becomes a closed area surrounded by a hypercuboid. Therefore, this example can be called a rectangular parallelepiped discrimination form.

Further, when the second weighting coefficient has the same value for all inputs of the neuron circuit, it becomes what can be called a cube discriminative type, and if the number of inputs is two (w1, w2), the result shown in FIG.
The identification area has a square shape as shown in (B). If the number of inputs is three (w1, w2, w3), it becomes a cube, and if the number of inputs is four or more, it becomes a hypercube. This cubic discrimination type has the advantage that the circuit scale is smaller than that of the rectangular parallelepiped discrimination type.

Comparing the neuron circuit of the present invention with the conventional neuron circuit, the multiplier circuit 83 shown in FIG. Not set.

FIG. 4 shows the basic configuration of a neuron circuit according to another embodiment of the present invention. The neuron circuit in the figure is composed of a first magnitude determination circuit 31, a second magnitude determination circuit 32, an AND circuit 33, and an output terminal 34. 1st
The output of the magnitude determination circuit 31 is 1 when xi≦w1+wh1, and 0 when xi>w1+wh1 (however, i=1, 2,
3...n), and the output of the second magnitude determination circuit 32 is x1≧
1 when wi-whi, 0 when xi<wi-whi (
However, i=1, 2, 3, . . . n), and the determined value is input to the AND circuit 33. The AND circuit 33 performs an AND operation based on the determination results of the first magnitude determination circuit 31 and the second magnitude determination circuit 32, and outputs the result.

The above will be explained regarding the input signal x1. When the input signal x1 is input, the first magnitude determination circuit 31 determines the first weight coefficient (w1) and the second weight coefficient (wh1).
) is compared with the input signal x1 (w1+wh1). Here, if the added value is x1≦w1+wh1, the output of the first magnitude determination circuit 31 is 1. Next, the second magnitude determination circuit 32 calculates the first weight coefficient (w1).
The difference (w1-wh1) between the second load coefficient (wh1) and the input signal x1 are compared. Here, the difference is x1≧w1−w
If it is h1, the output of the second magnitude determination circuit 32 will be 1. The outputs of the first magnitude determination circuit 31 and the second magnitude determination circuit 32 for each of these inputs are outputted to an AND circuit 33.
and outputs it from the output terminal 34 as the output of the neuron circuit. It can be seen that the method in this embodiment operates similarly to the previously described embodiment.

When comparing the identification functions of the neuron circuit of the present invention and the conventional neuron circuit, it is found that they have exactly the same functions except for the difference in the shape of the identification area. Figure 5
1 shows an example of forming an identification area in the case of two inputs using a plurality of neuron circuits of the present invention. The solid line indicates the target identification area a, and the dotted line indicates the identification area b of each neuron circuit. As shown in FIG. 5, the present invention is compatible with identification areas of arbitrary shapes.
The use of multiple neuron circuits to correspond to an arbitrary-shaped identification region is the same as in the conventional method.

In both the conventional neuron circuit and the neuron circuit of the present invention, if the identification areas of each neuron are arranged so as to be in contact with each other, gaps will be created, so it is necessary to create mutual overlap. In the overlapping portion, there is little difference in the shapes of the identification regions of both neuron circuits. In other words, there is almost no difference in performance due to the difference in the shape of the identification area of the two types of neuron circuits, since only the irregular shapes of the arbitrary shapes are different. Further, the neural network circuit using the neuron circuit of the present invention has the same function as the conventional neuron circuit.

[0029]

As described above, according to the present invention, the amount of hardware for constructing the neuron circuit is small. Next, this will be explained using a numerical example.

Here, it is known that if the number of bits of a digital signal is N, the number of transistors required to configure each circuit can be approximately estimated by the following equation. First, the number of transistors Na in the subtraction circuit can be roughly estimated using the following equation. A two-input addition circuit, a magnitude determination circuit, and a threshold circuit may also be considered equivalent. Subtraction circuit Na=28N The number of transistors Nm in the multiplication circuit can be roughly estimated by the following formula. 2
The number of transistors Ns in the multiplier circuit is twice that number. Multiplier circuit Nm=26N2 Square circuit Ns=2 (26N2) The number of transistors Nz in the absolute value circuit can be approximately estimated by the following equation. Absolute value circuit Nz=16N The number of transistors required is 8
As an example, the case of 16 bits is as follows.

[0031] In the case of N=8 (8 bits),
Na=224, Nm=1664, Ns=3
328, Nz=128 When N=16 (16 bits) Na=448, Nm=6656,
Ns=13312, Nz=256 It is clear that as the number of bits increases, the circuit scale of the multiplication circuit becomes extremely large compared to the addition circuit and the absolute value circuit. Here, assuming that the digital signal is 8 bits and the number of inputs is 100,
The number of transistors required to configure the conventional neuron circuit and the neuron circuit of the present invention will be estimated and compared. Conventional neuron circuit (ellipsoid discrimination form in Figure 9)
544000 Conventional neuron circuit (polyhedral discrimination form in Figure 10)
224000 Neuron circuit of the present invention (rectangular parallelepiped type in Figure 2)
57600 As described above, the neuron circuit of the present invention, which does not use a multiplication circuit, can be configured with a considerably smaller number of transistors than the conventional neuron circuit, and thus can significantly reduce the circuit scale. Furthermore, since power consumption also increases approximately in proportion to the circuit scale, according to the present invention, power consumption can be reduced by significantly reducing the circuit scale.

Furthermore, when making a neural network circuit hardware, the number of necessary neuron circuits varies depending on the application, but in general, the larger the number of neuron circuits, the better the processing power. Therefore, the processing power improves as the number of neuron circuits increases due to LSI integration. As a result, it is expected that a neural network circuit equipped with a large number of neuron circuits will be realized through LSI integration. However, due to chip size limitations, the circuit scale that can be mounted on one chip is limited. Furthermore, there is a limit to the power that can be consumed in one chip due to heat dissipation and mounting issues. Therefore, while LSI neural network circuits exhibit practical performance, the use of neuron circuits has an extremely large effect in improving the performance of neural network circuits to a practical level.

[Brief explanation of drawings]

FIG. 1 is a diagram showing the principle configuration of the present invention.

FIG. 2 is a basic configuration diagram of a neuron circuit according to an embodiment of the present invention.

FIG. 3 is a diagram showing identification areas of a neuron circuit according to an embodiment of the present invention.

FIG. 4 is a basic configuration diagram of a neuron circuit according to another embodiment of the present invention.

FIG. 5 is a diagram illustrating an example of forming an identification area in the case of two inputs using a plurality of neuron circuits of the present invention.

FIG. 6 is a diagram showing symbols of neuron circuits.

FIG. 7 is a diagram showing a neural network circuit with a two-layer structure.

FIG. 8 is a diagram showing a neural network circuit with a three-layer structure.

FIG. 9 is a diagram showing the basic configuration of an ellipsoid discrimination type of a conventional neuron circuit.

FIG. 10 is a basic configuration diagram of a polyhedral discrimination type of a conventional neuron circuit.

FIG. 11 is a diagram showing transfer characteristics of a threshold circuit.

FIG. 12 is a diagram showing identification areas in the case of two inputs of a conventional neuron circuit.

FIG. 13 is a diagram illustrating identification regions in a conventional neuron circuit in which the second weighting coefficients are equal and there are two inputs.

FIG. 14 is a diagram showing an example of forming an identification area in the case of two inputs using a plurality of conventional neuron circuits.

[Explanation of symbols]

1 Input terminal 2 Size determination means 3 Logical operation means 4 Output terminal 11 Subtraction circuit 12 Absolute value circuit 13 Size determination circuit 14. AND circuit 15 Output terminal 31 First size determination circuit 32 Second size determination circuit 33. AND circuit

Claims (1)

[Claims] Claim 1: Each neuron circuit executes calculations in response to a plurality of input signals input to a neural network circuit configured by connecting input and output terminals of a large number of neuron circuits, and In a neural network circuit, the output values of all or some of the neuron circuits are used as output signals of the neural network circuit, and the shape of the identification region is varied depending on the size of the weighting coefficient of each neuron circuit to control the function of the neural network circuit. , each of the neuron circuits has n (n is an integer of 1 or more) input terminals into which input signals are input when forming an arbitrary-shaped identification region, and a unique terminal corresponding to each of the input terminals. 1 and a second load coefficient, and determines the magnitude of the first load coefficient and the second load coefficient corresponding to the n input signals and the input signal, and obtains a predetermined determination result. It is characterized by having a magnitude determination means for outputting an output, a logic operation means for performing a predetermined logical operation based on the n magnitude determination results from the magnitude determination means, and an output terminal for outputting the operation result of the logic operation means. neural network circuit.