KR102309013B1 - An efficient neuromorphic circuit system of realizing negative weight - Google Patents

Abstract

It relates to a neuromorphic circuit system capable of efficiently implementing negative weights, expressing negative weights of synapses, comprising: a plurality of free neurons; multiple post neurons; a plurality of synapses disposed on intersections of a plurality of row lines extending in a row direction from each of the pre-neurons and a plurality of column lines corresponding to each of the post neurons to form a synapse array; a shift circuit for adding and summing shift weights to the inputs of the plurality of free neurons and outputting the summed result; and a subtraction circuit for subtracting the output of the shift circuit from the output of each of the plurality of column lines, and outputting the subtracted output to each of the post neurons, wherein each of the plurality of synapses is converted from the original weight to the shift weight. By providing a configuration with shifted weights and implementing negative weights using weight shifts in a neuromorphic system, one array is different from the conventional method using two arrays. Using only (array), it is not only efficient in terms of layout space and power consumption, but also makes hardware easy and efficient.

Description

An efficient neuromorphic circuit system of realizing negative weight }

The present invention relates to a neuromorphic circuit system capable of efficiently implementing negative weights, expressing negative weights of synapses, in a system implementing a neuromorphic system as a synaptic crossbar array structure.

In addition, the present invention proposes a crossbar-array structure of an efficient synapse that can implement a negative weight without using an additional synapse array, a neuromorphic circuit capable of implementing an efficient negative weight It's about the system.

As one of the next-generation computing methods for superior processing of vast amounts of information and atypical information, a neuromorphic system similar to the way the brain operates is in the spotlight as a next-generation method.

Such a neuromorphic system may be implemented using various hardware such as a CPU, a GPU, an analog circuit, and the like. Among them, the hardware of a crossbar-array structure including many synapse devices and passive devices is highly efficient in terms of cost as well as high integration of the devices. Therefore, the hardware of this structure is spotlighted as a hardware structure for a neuromorphic system.

However, unlike software that can express negative (-1 to 1) synapse weights, in the case of hardware, it is impossible to express negative weights through a single synapse element. . This is due to the fact that the conductance (the element representing the weight) of the actual element cannot be negative.

Therefore, the prior art implements negative weights by using two synapse arrays each having a positive polarity and a negative polarity.

As shown in Figure 1, the synapse weight (synapse weight) is generally implemented with two synaptic arrays. The two synaptic arrays M+ and M- represent positive and negative polarities of synapse weights, respectively. Each synaptic array is represented by a conductance (1/R). And a voltage (V _IN,j ) of a predetermined level is supplied as an input to the two synaptic arrays M+, M-, and the outputs (output current or output voltage) of the two synaptic arrays are subtracted from each other, and a negative weight ( negative weight).

2 shows a circuit for outputting negative weights composed of two synaptic arrays M+ and M-.

However, the circuit configuration with negative weights of the prior art as described above requires unnecessary power and area due to an additional array from a system point of view. Therefore, considering that the neuromorphic system pursues low power and high integration performance, the configuration of the prior art has a problem that is not suitable for application to the neuromorphic system.

Truong, SN, &Min, KS “New Memristor-Based Crossbar Array Architecture with 50-% Area Reduction and 48-% Power Saving for Matrix-Vector Multiplication of Analog Neuromorphic Computing”, Journal of semiconductor technology and science, 14 (3), 356-363 (2014)

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-described problems, and to present an efficient synapse crossbar-array structure that can implement negative weights without using an additional synaptic array. It is to provide a neuromorphic circuit system capable of efficiently implementing negative weights.

In order to achieve the above object, the present invention relates to a neuromorphic circuit system capable of efficiently implementing negative weights, comprising: a plurality of free neurons; multiple post neurons; a plurality of synapses disposed on intersections of a plurality of row lines extending in a row direction from each of the pre-neurons and a plurality of column lines corresponding to each of the post neurons to form a synapse array; a shift circuit for adding and summing shift weights to the inputs of the plurality of free neurons and outputting the summed result; and a subtraction circuit for subtracting the output of the shift circuit from the output of each of the plurality of column lines, and outputting the subtracted output to each of the post neurons, wherein each of the plurality of synapses is converted from the original weight to the shift weight. It is characterized in that it has a shifted weight.

Further, in the present invention, in a neuromorphic circuit system capable of efficiently implementing negative weights, the shift circuit connects all of the shift weight elements and a plurality of shift weight elements each having the shift weight and connected to the pre-neuron, and the plurality of shift weight elements. It is characterized in that it is composed of a shift line.

In addition, the present invention is characterized in that in the neuromorphic circuit system capable of efficiently implementing negative weights, the shift weight element is composed of a resistive element.

In addition, the present invention is a neuromorphic circuit system that can effectively implement negative weights, characterized in that the subtraction circuit is provided with a load resistor.

In addition, the present invention is characterized in that in the neuromorphic circuit system capable of efficiently implementing negative weights, the resistance R _shift of the resistance element is set to the resistance according to Equation 1.

[Formula 1]

However, w _shift is the shift weight.

In addition, the present invention is characterized in that in the neuromorphic circuit system capable of efficiently implementing negative weights, the shift weight is a value of 1/2 of the maximum width of the weight indicated by the synapse.

In addition, the present invention is characterized in that in the neuromorphic circuit system capable of efficiently implementing negative weights, the shift weight element and the synapse are implemented by the same element.

As described above, according to the neuromorphic circuit system capable of efficiently implementing negative weights according to the present invention, by implementing negative weights using a weight shift in a neuromorphic system, the existing 2 Unlike the method using multiple arrays, using only one array is effective in terms of layout space and power consumption, and the effect of easily and efficiently manufacturing hardware is obtained.

Therefore, the present invention can be used more efficiently in consideration of a neuromorphic system requiring high integration and low power operation characteristics. For example, the present invention can be applied to systems such as facial recognition and voice recognition based on a neuromorphic system, or a system for processing or converting data based on a neuromorphic system.

1 is a configuration diagram for implementing a negative weight according to the prior art;
FIG. 2 is a hardware-based detailed circuit diagram of the system for implementing negative weights of FIG. 1;
3 is a block diagram of a configuration of a neuromorphic circuit system capable of efficiently implementing negative weights according to an embodiment of the present invention.
4 is a diagram for explaining an operation method of a weight shift in a neuromorphic circuit system capable of efficiently implementing negative weights according to an embodiment of the present invention;
5 is an exemplary implementation diagram of a neuromorphic circuit system capable of efficiently implementing negative weights according to an embodiment of the present invention.
6 is a table showing experimental results of python to which the system according to the present invention is applied and pspice according to the prior art.

Hereinafter, specific contents for carrying out the present invention will be described with reference to the drawings.

In addition, in demonstrating this invention, the same part is attached|subjected with the same code|symbol, and the repetition description is abbreviate|omitted.

First, a configuration of a neuromorphic circuit system capable of efficiently implementing negative weights according to an embodiment of the present invention will be described with reference to FIG. 3 .

As shown in FIG. 3 , the neuromorphic system 100 according to an embodiment of the present invention includes a plurality of pre-neurons 10 and a plurality of row lines extending from the pre-neurons 10 in the row direction ( R), a plurality of post neurons 20, a plurality of column lines C extending from the plurality of post neurons 20 in the column direction, the intersection of the row lines R and the column lines C A subtraction circuit for subtracting a shift-weighted value from the output of a plurality of synapses 30 disposed on the synapses, a shift weight element 40 connected to each pre-neuron 10, and synapses of each column line C It consists of (50).

First, the pre-neuron 10 may transmit an electrical signal to the synapse 30 through the row line R in the learning mode or the read mode. The free neuron 10 is composed of N neurons. In addition, the row line R is also composed of N lines.

Meanwhile, preferably, each pre-neuron i may receive an input voltage V _i as an input value (here, i = 1, 2, ..., N). Here, a row line R extending from the pre-neuron i is referred to as a row line i.

Next, the post neuron 20 may transmit an electrical signal to the synapse 30 or receive an electrical signal from the synapse 30 through the column line C in the learning mode or the read mode. The post neurons 20 are composed of M neurons.

In addition, the column line C is a line corresponding to the post neuron 20, respectively, and is composed of M lines. A column line C corresponding to the post neuron j is referred to as a column line j. j = 1, 2, ..., M.

Each column line j is connected to a synapse (1,j), (2,j), ..., (N,j) provided at a point crossing the row lines 1 to N. Therefore, the output I _j of each column line j is the sum of the outputs of all synapses of the column line j.

Next, each synapse 30 is a device capable of storing multiple levels of conductance, in particular, positive level conductance. In the case of a synaptic device, various types of memory devices may be used. For example, a phase change random access memory, a resistive random access memory, a ferroelectric random access memory (FRAM), a flash memory, or the like may be used. In a hardware-based neuromorphic system, a synaptic weight is used as a weight based on conductance of a device.

On the other hand, each synapse (i, j) is provided at the intersection of the row line i and the column line j, the synapses 30 constitute a synapse array (synapse array). Preferably, each synapse (i, j) is connected to the row line i at one end, and the column line j is connected to the other end.

In addition, each synapse (i, j) represents a weight w _ij . In particular, the synapse (i, j) is a device that stores or stores the shifted weight. That is, the synapse (i, j) stores or stores the shifted weight as a conductance value.

That is, the shifted weight w _ij of the synapse (i, j) may be expressed as the following equation.

[Equation 1]

In the above equation, w _ij is the weight of the device and always has a positive value. In addition, w ⁰ _ij is a weight to be actually used, and includes a negative weight.

And w _shift is the shift weight. Therefore, the above equation assumes that the weight of the device is shifted by _{w shift from the weight to be used originally.} That is, this is because the weight of the device always has a positive value.

[Equation 1-2]

Therefore, in order to make a weight to be used, it can be implemented by subtracting the shift weight from the weight of the device. That is, the weight (or weight before shift) to be originally used should have positive and negative values, but since the synaptic element has only a positive conductance value physically, it is necessary to shift it in a negative direction to use a negative weight. .

For example, the weight of a synaptic element having a range of 9 to 11 always has a positive value, but if it is shifted by 10, a weight in the range of -1 to 1 (ie, including a negative weight) can be used. As an example, for the shift weight w _shift , a value of 1/2 of the maximum width of the synaptic element weight should be used. Therefore, the shift weight w _shift is a shift value that makes the positive weight that the device always has into the weight including the negative range.

Next, the shift weight element 40 stores (stores) the _{shift weight w shift as an element having a shift weight.} The shift weight element 40 is an element connected to each of the free neurons 10 or an element connected to the row line R, and is composed of N pieces. Therefore, each shift weight element 40 weights the input value of the corresponding pre-neuron with a shift weight. Also, each of the shift weight elements 40 is connected to the shift line S to sum the outputs of each element.

That is, the shift weight elements 40 constitute a shift circuit that weights and sums the shift weights to the inputs of a plurality of free neurons, and outputs the summed result. Preferably, the shift circuit is composed of a shift weight element 40 connected to each free neuron 10 and a shift line S connecting the shift weight elements in a column direction.

Preferably, the shift weight element 40 may be constituted by a resistive element. That is, it can be implemented as a resistance element having a resistance _{R shift} as shown in the following equation.

[Equation 2]

Here, w _shift is the shift weight.

Meanwhile, the shift weight element 40 is not limited to a resistance element, and any element having a conductance (reciprocal of resistance) of the _{shift weight w shift may be used.} For example, for the efficiency of the neuromorphic circuit manufacturing process, the same device may be implemented, such as a memristor constituting a synapse.

In addition, a shift line S in the column direction connecting all the shift weight elements 40 is provided. Accordingly, all output values weighted by the shift weight elements 40 may be summed and output to the shift line S.

_{The output I shift} of the shift line S is as follows.

[Equation 3]

Here, w _shift is the shift weight, and V _i is the input value of the pre-neuron i.

In addition, when the shift weight element 40 is implemented as a resistance element, the output I _shift of the shift line S may be expressed as the following equation.

[Equation 4]

Here, R _shift is the resistance value of the shift weight element 40 .

Next, the subtraction circuit 50 subtracts the shift-weighted values from the output of the synapses 30 of each column line (C). That is, the output value of the shift line (S) is subtracted from the output value of each column line (C).

As described above, the output I _j of each column line j is the sum of the outputs of all synapses of the column line j. Therefore, the output I _j of each column line j can be expressed as the following equation.

[Equation 5]

Here, w _ij represents the weight of the synapse (i, j), and V _i is the input value of the pre-neuron i.

^{The final output I *} _j of the subtraction circuit 50 is a value obtained by subtracting _{the output I shift} of the shift line S from _{the output value I j} of the column line C . Therefore, the final output I ^* _j is as follows.

[Equation 6]

In addition, when the shift weight element 40 is implemented as a resistance element, the final output I ^* _j is expressed by the following equation.

[Equation 7]

Meanwhile, the subtraction circuit 50 may add a _{load resistance R Load} to calculate the output of each column line C as a voltage.

Next, an example of the configuration of the neuromorphic circuit system according to the present invention will be described with reference to FIGS. 4 and 5 .

The neuromorphic circuit system according to the present invention is a simple system for implementing negative weights. It operates by shifting the conductance (conductance) of the synaptic element used in the array, that is, a weight value, in a negative direction.

As shown in Fig. 4, the shift weight is set to 10. The reason is that it is assumed that the weight values of the synaptic elements are 9 to 11. That is, it is assumed that the -1 to 1 weights to be used are shifted by 10. In addition, a 1/2 value of the weight range of the synaptic element may be calculated as a shift value of 10.

Therefore, as shown in FIG. 4, a value obtained by multiplying a weight of 10 by an input signal with respect to a value output through an element having a weight of 9 to 11 is subtracted. That is, it performs a re-shift process.

In other words, an output value obtained by multiplying an input signal by a weight having a value of 10 from an output value passed through an array composed of elements having a weight of 9 to 11 is subtracted. Through this, it is possible to obtain a result in which a negative weight is used. That is, a weight of 10, which is an average value of the minimum value (9) and the maximum value (11) of the element weights used, is used.

As shown in Figure 5, the input voltage (input voltage) is the value of V, and synapse weight (conductance), a device capable of having a weight in the range of 9 to 11 was used.

As shown in FIG. 5 , when a signal voltage is applied from an input, that is, a pre-neuron to a synapse array, an output current output from the synapse array to a post neuron direction is the same as in Equation 5. That is, it is expressed as the sum of products of conductance, which is a weight of the element in each synaptic array, and an applied voltage. At this time, the output is output through an array composed of elements (with weights of 9 to 11) that do not take into account negative weights. Therefore, it is only a positive output.

Therefore, to account for negative weights, the shift circuit shown in Fig. 5 is used. At this time, the shift weight element in the shift circuit, that is, the resistor for shift, must use the inverse of the weights having a value of 10 among the weight values of elements having a weight of 9 to 11.

For example, if a device with a weight value of 9 to 11 and a conductance value of 0.001 to 0.005 is used, the _{value of R shift} is 1/0.003; Since resistance is the reciprocal of conductance, the reciprocal value of the average value of 0.001 and 0.005 is used.

Therefore, by subtracting the value output from the array and the value output from the shift circuit, an output with a negative weight taken into account can be obtained. In this example, a voltage output was used by attaching a load resistor, and the voltage output to the first output unit (or post neuron) is (i ₁ × R _load ) - (i _shift × R) _load ).

Next, the effects of the present invention will be described in more detail through experiments.

In order to experimentally prove the system according to the present invention, an experiment was conducted using 16 input signals and 5 post neurons (output units). First, using Python-based software, training was performed through the operation method shown in FIG. 4, and synapse weight values to be used in the circuit were obtained (16 x 5 = 80 weight values).

And the obtained synapse weight (synapse weight) was compared and analyzed with software after designing a circuit through pspice, one of the programs for circuit simulation. Post neuron output value on python and output voltage value on neuromorphic circuit designed with pspice were compared and analyzed.

6 is a comparison of five output values expressed in a diagram in python and pspice. As a result, the output that matches the software output on python was obtained on pspice.

In the above, the invention made by the present inventors has been described in detail according to the above embodiments, but the present invention is not limited to the above embodiments, and various modifications can be made without departing from the gist of the present invention.

10: pre neuron 20: post neuron
30: synapse 40: shift weight element
50: subtraction circuit

Claims (7)

In a neuromorphic circuit system that can efficiently implement negative weights,
a large number of free neurons;
multiple post neurons;
a plurality of synapses disposed on intersections of a plurality of row lines extending in a row direction from each of the pre-neurons and a plurality of column lines corresponding to each of the post neurons to form a synapse array;
a shift circuit that weights shift weights to the inputs of the plurality of pre-neurons and outputs them to the plurality of synapses; and,
and a subtraction circuit for subtracting a value weighted by a shift weight from the output of each of the plurality of column lines, and outputting the subtracted output to each of the post neurons;
Each of the plurality of synapses has a weight shifted from the original weight to the shift weight,
the shift circuit includes a plurality of shift weight elements each having the shift weight and connected to the pre-neuron, and a shift line connecting all of the plurality of shift weight elements;
For each of the plurality of row lines, a shift weight element is connected to the row line between the pre-neuron connected to the row line and the synapse on the row line, and the shift weight element uses its shift weight to synapse the synapse on the row line. weighted by shifting the weight of
the shift circuit sums up shift weights of all shift weight elements connected to the shift line and outputs the summed result to the subtraction circuit;
the subtraction circuit subtracts the summed result of the shift circuit from the output of each of the plurality of column lines;
The synapse is a neuromorphic circuit system capable of efficiently implementing negative weights, characterized in that it is a device that stores positive weights.
delete According to claim 1,
The shift weight element is a neuromorphic circuit system capable of effectively implementing negative weights, characterized in that it is composed of a resistive element.
4. The method of claim 3,
A neuromorphic circuit system capable of efficiently implementing negative weights, characterized in that a load resistor is provided in the subtraction circuit.
4. The method of claim 3,
A neuromorphic circuit system capable of effectively implementing negative weights, characterized in that _{the resistance R shift} of the resistive element is set to the resistance according to Equation 1.
[Formula 1]

However, w _shift is the shift weight.
According to claim 1,
The shift weight is a neuromorphic circuit system capable of effectively implementing negative weights, characterized in that it is a value of 1/2 of the maximum width of the weight represented by the synapse.
According to claim 1,
The neuromorphic circuit system capable of effectively implementing negative weights, characterized in that the shift weight element and the synapse are implemented by the same element.

Images