KR20200100286A - An efficient neuromorphic circuit system of realizing negative weight - Google Patents

Abstract

The present invention relates to a neuromorphic circuit system capable of efficient negative weight realization which expresses a negative weight of a synapse. The neuromorphic circuit system capable of efficient negative weight realization comprises: a plurality of pre-neurons; a plurality of post-neurons; a plurality of synapses arranged on intersection points of a plurality of row lines extended from the pre-neurons in a row direction and a plurality of column lines corresponding to the post-neurons to form a synapse array; a shift circuit to add a shift weight to the input of the plurality of pre-neurons to sum the shift weight added to the input and output a summed result; a subtraction circuit to subtract the output of the shift circuit from the output of the plurality of column lines to output the subtracted output to the post-neurons. Each of the plurality of synapses has a weight shifted from the original weight to the shift weight, a negative weight is realized by using a weight shift in a neuromorphic system, and only one array unlike a conventional method of using two arrays is used to not only be efficient in terms of an arrangement space or power consumption, but to easily and efficiently manufacture hardware.

Description

{An efficient neuromorphic circuit system of realizing negative weight}

The present invention relates to a neuromorphic circuit system capable of efficiently implementing negative weights, expressing negative weights of synapses, in a system implementing a neuromorphic system as a crossbar array structure of synapses.

In addition, the present invention proposes an efficient synapse crossbar-array structure capable of implementing negative weights without using an additional synapse array, a neuromorphic circuit capable of implementing an efficient negative weight. It's about the system.

As one of the next-generation computing methods for superior processing of vast amounts of information and unstructured information, a neuromorphic system similar to the way the brain works is in the spotlight as a next-generation method.

Such a neuromorphic system may be implemented using various hardware such as a CPU, a GPU, and an analog circuit. Among them, hardware having a crossbar-array structure including many synapse devices and passive devices not only enables high integration of devices, but is also very efficient in terms of cost. Therefore, hardware of this structure is in the spotlight as a hardware structure for a neuromorphic system.

However, unlike software that can express the synapse weight of negative numbers (-1 to 1), in the case of hardware, it is impossible to express the weight of negative numbers through a single synapse element. . This is due to the fact that the conductance (a device representing a weight) cannot be negative.

Accordingly, the prior art implements negative weights by using two synapse arrays each having a positive polarity and a negative polarity.

As shown in FIG. 1, the synapse weight is generally implemented as two synaptic arrays. The two synaptic arrays M+ and M- represent the positive and negative polarities of synapse weights, respectively. Each synaptic array is expressed as conductance (1/R). And a voltage of a predetermined level (V _IN,j ) is supplied as input to the two synaptic arrays M+ and M-, and the outputs (output current or output voltage) of the two synaptic arrays are subtracted from each other, and a negative weight ( negative weight).

2 shows a circuit configured with two synaptic arrays M+ and M- to output negative weights.

However, the conventional negative-weighted circuit configuration as described above requires unnecessary power and area due to an additional array from the system point of view. Therefore, when considering that a neuromorphic system pursues low power and high integration performance, the configuration of the prior art has a problem that is not suitable for application to a neuromorphic system.

Truong, SN, &Min, KS “New Memristor-Based Crossbar Array Architecture with 50-% Area Reduction and 48-% Power Saving for Matrix-Vector Multiplication of Analog Neuromorphic Computing”, Journal of semiconductor technology and science, 14 (3), 356-363 (2014)

An object of the present invention is to solve the above-described problems, and propose an efficient synapse crossbar-array structure capable of implementing negative weights without using an additional synaptic array. It is to provide a neuromorphic circuit system capable of efficiently implementing negative weights.

In order to achieve the above object, the present invention relates to a neuromorphic circuit system capable of implementing an efficient negative weight, comprising: a plurality of free neurons; Multiple post neurons; A plurality of synapses disposed on intersections of a plurality of row lines extending in a row direction from each of the free neurons and a plurality of column lines corresponding to each of the post neurons to form a synaptic array; A shift circuit that weights and adds shift weights to the inputs of the plurality of free neurons, and outputs the summed result; And a subtraction circuit for subtracting an output of the shift circuit from the output of each of the plurality of column lines and outputting the subtracted output to each of the post neurons, wherein each of the plurality of synapses is converted from an original weight to the shift weight. It is characterized by having a shifted weight.

In addition, the present invention is a neuromorphic circuit system capable of efficiently implementing negative weights, wherein the shift circuit has the shift weight and connects all of the plurality of shift weight elements and a plurality of shift weight elements connected to each free neuron. It characterized in that it is composed of a shift line.

In addition, the present invention is a neuromorphic circuit system capable of efficiently implementing negative weights, wherein the shift weighting element is formed of a resistance element.

In addition, the present invention is a neuromorphic circuit system capable of efficiently implementing negative weights, characterized in that the subtraction circuit has a load resistance.

In addition, the present invention is characterized in that in a neuromorphic circuit system capable of efficiently implementing negative weights, the resistance R _shift of the resistance element is set to a resistance according to Equation 1.

[Equation 1]

However, w _shift is the shift weight.

In addition, in a neuromorphic circuit system capable of efficiently implementing negative weights, the present invention is characterized in that the shift weight is 1/2 of the maximum width of the weight indicated by the synapse.

In addition, the present invention is a neuromorphic circuit system capable of efficiently implementing negative weights, wherein the shift weighting element and the synapse are implemented by the same element.

As described above, according to the neuromorphic circuit system capable of efficiently implementing negative weights according to the present invention, by implementing negative weights using a weight shift in a neuromorphic system, the conventional 2 Unlike the method using two arrays, using only one array is not only efficient in terms of an arrangement space or power consumption, but also has an effect of easily and efficiently manufacturing hardware.

Accordingly, the present invention can be used more efficiently in consideration of a neuromorphic system requiring operation characteristics such as high integration and low power. For example, the present invention can be applied to systems such as facial recognition and speech recognition based on a neuromorphic system, or to systems such as data processing or transformation based on a neuromorphic system.

1 is a block diagram for implementing a negative weight according to the prior art.
FIG. 2 is a hardware-based detailed circuit configuration diagram of the negative weight implementation system of FIG. 1.
3 is a block diagram of a configuration of a neuromorphic circuit system capable of implementing an efficient negative weight according to an embodiment of the present invention.
4 is a diagram for explaining an operation method of a weight shift of a neuromorphic circuit system capable of implementing an efficient negative weight according to an embodiment of the present invention.
5 is a diagram illustrating an implementation example of a neuromorphic circuit system capable of implementing an efficient negative weight according to an embodiment of the present invention.
6 is a table showing the experimental results of python to which the system according to the present invention is applied and pspice according to the prior art.

Hereinafter, specific details for the implementation of the present invention will be described with reference to the drawings.

In addition, in describing the present invention, the same parts are denoted by the same reference numerals, and repeated explanations thereof are omitted.

First, a configuration of a neuromorphic circuit system capable of implementing an efficient negative weight according to an embodiment of the present invention will be described with reference to FIG. 3.

As shown in FIG. 3, the neuromorphic system 100 according to an embodiment of the present invention includes a plurality of free neurons 10 and a plurality of row lines extending in a row direction from the free neurons 10 ( R), a plurality of post neurons 20, a plurality of column lines (C) extending in the column direction from the plurality of post neurons 20, the intersection of the row lines (R) and the column lines (C) A subtraction circuit for subtracting the shift weighted value from the outputs of the synapses 30 of the plurality of synapses 30 arranged on the fields, the shift weighting element 40 connected to each free neuron 10, and the synapses of each column line C It consists of 50.

First, the free neuron 10 may transmit an electrical signal to the synapse 30 through the row line R in a learning mode or a read mode. The free neuron 10 is composed of N neurons. Also, the row line R is composed of N lines.

Meanwhile, preferably, an input voltage V _i may be input to each free neuron i as an input value (here, i = 1, 2, ..., N). Here, the row line R extending from the free neuron i is referred to as a row line i.

Next, the post neuron 20 may transmit an electrical signal to the synapse 30 or receive an electrical signal from the synapse 30 through the column line C in the learning mode or the read mode. The post neuron 20 is composed of M neurons.

Further, the column line C is a line corresponding to each of the post neurons 20 and is composed of M lines. The column line C corresponding to the post neuron j is referred to as a column line j. j = 1, 2, ..., M.

Each column line j is connected to the synapses (1,j), (2,j), ..., (N,j) provided at the points intersecting the row lines 1 to N. Therefore, the output I _j of each column line j is the sum of the outputs of all synapses of the column line j.

Next, each synapse 30 is an element capable of storing a plurality of levels of conductance, in particular, a positive level condensation. In the case of synaptic devices, various types of memory devices may be used. For example, a phase change memory, a resistive random access memory, a ferroelectric random access memory (FRAM), a flash memory, or the like may be used. In a hardware-based neuromorphic system, the synaptic weight is used as a weight by the conductance of the device.

On the other hand, each synapse (i, j) is provided at the intersection of the row line i and the column line j, the synapses 30 constitute a synapse array (synapse array). Preferably, each synapse (i,j) is connected to a row line i at one end and a column line j at the other end.

In addition, each synapse (i,j) represents a weight w _ij . In particular, synapse (i,j) is an element that stores or stores shifted weights. That is, the synapse (i,j) stores or stores the shifted weight as a conductance value.

That is, the shifted weight w _ij of the synapse (i,j) can be expressed as the following equation.

[Equation 1]

In the above equation, w _ij is the weight of the device and always has a positive value. In addition, w ⁰ _ij is a weight to be actually used and includes a negative weight.

And w _shift is the shift weight. Therefore, the above equation assumes that the weight of the device is shifted by w _shift from the weight to be used originally. That is, because the weight of the device always has a positive value.

[Equation 1-2]

Therefore, in order to create a weight to be used, it can be implemented by subtracting the shift weight from the weight of the device. That is, the weight to be used originally (or the weight before shift) should have positive and negative values, but since the synaptic element physically has only positive conductance values, it must be shifted in the negative direction to use negative weights. .

For example, the weight of a synaptic device having a range of 9 to 11 always has a positive value, but if it is shifted by 10, a weight in the range of -1 to 1 (ie, including a negative weight) can be used. As shown in the example, the shift weight w _shift should be 1/2 of the maximum width of the synaptic device weight. Therefore, the shift weight w _shift is a shift value that makes the positive weight that the device always has to the weight including the negative range.

Next, the shift weight element 40 stores (stores) a shift weight w _shift as an element having a shift weight. The shift weight element 40 is an element connected to each of the free neurons 10 or a row line R, and is composed of N elements. Therefore, each shift weight element 40 weights the input value of a corresponding free neuron with a shift weight. In addition, each of the shift weight elements 40 are all connected to the shift line S, and the outputs of each element are summed.

That is, the shift weight elements 40 constitute a shift circuit that weights and adds shift weights to inputs of a plurality of free neurons, and outputs the summed result. Preferably, the shift circuit includes a shift weight element 40 connected to each free neuron 10 and a shift line S connecting the shift weight elements in a column direction.

Preferably, the shift weighting element 40 may be composed of a resistive element. That is, it can be implemented as a resistance element having a resistance R _shift as shown in the following equation.

[Equation 2]

Here, w _shift is a shift weight.

Meanwhile, the shift weight element 40 is not limited to a resistance element, and any element having a conductance of the shift weight w _shift (reverse number of resistance) may be used. For example, for the efficiency of the neuromorphic circuit manufacturing process, it may be implemented with the same device such as a memristor constituting a synapse.

In addition, a shift line S in the column direction connecting all of the shift weight elements 40 is provided. Accordingly, all output values weighted by the shift weight elements 40 may be summed and output to the shift line S.

The output I _shift of the shift line S is as follows.

[Equation 3]

Here, w _shift is the shift weight, and V _i is the input value of the free neuron i.

In addition, when the shift weighting element 40 is implemented as a resistive element, the output I _shift of the shift line S can be expressed as follows.

[Equation 4]

Here, R _shift is the resistance value of the shift weight element 40.

Next, the subtraction circuit 50 subtracts shift-weighted values from the outputs of the synapses 30 of each column line C. That is, the output value of the shift line S is subtracted from the output value of each column line C.

As described above, the output I _j of each column line j is the sum of the outputs of all synapses of the column line j. Therefore, the output I _j of each column line j can be expressed by the following equation.

[Equation 5]

Here, w _ij represents the weight of the synapse (i,j), and V _i is the input value of the free neuron i.

The final output I ^* _j of the subtraction circuit 50 is a value obtained by subtracting the output I _shift of the shift line S from the output value I _j of the column line C. Therefore, the final output I ^* _j is as follows.

[Equation 6]

In addition, when the shift weight element 40 is implemented as a resistance element, the final output I ^* _j is as follows.

[Equation 7]

Meanwhile, the subtraction circuit 50 may add a load resistance R _Load to calculate the output of each column line C as a voltage.

Next, an example of a configuration of a neuromorphic circuit system according to the present invention will be described with reference to FIGS. 4 and 5.

The neuromorphic circuit system according to the present invention is a simple system for implementing negative weights. It operates by shifting the conductance, that is, the weight value of the synaptic element used in the array, in a negative direction.

4, the shift weight was set to 10. This is because the weight value of the synaptic element is assumed to be 9 to 11. That is, it is assumed that the weights from -1 to 1 to be used are shifted by 10. Also, a value of 1/2 of the weight range of the synaptic element may be calculated as a shift value of 10.

Accordingly, a value obtained by multiplying a weight of 10 and an input signal is subtracted from a value output through a device having a weight of 9 to 11 as shown in FIG. 4. That is, it performs a re-shift process.

In other words, the output value obtained by multiplying the weight having a value of 10 and the input signal is subtracted from the output value that has passed through an array composed of elements having 9 to 11 weights. Through this, a result in which negative weights are used can be obtained. That is, a weight of 10, which is an average value of the minimum (9) and maximum (11) of the used device weight, is used.

As shown in FIG. 5, a device capable of having a value of V as an input voltage and a weight in the range of 9 to 11 as a synapse weight (conductance) was used.

As shown in FIG. 5, when an input, that is, a signal voltage is applied from a pre-neuron into a synapse array, an output current output from the synaptic array in a post neuron direction Is equal to Equation 5. That is, it is expressed as the sum of products of conductance, which is the weight of the elements in each synaptic array, and the applied voltage. The output at this time is the output through an array composed of elements (with a weight of 9 to 11) that does not take into account the negative weight. So it is only positive output.

Therefore, in order to consider the negative weight, the shift circuit shown in Fig. 5 is used. In this case, the shift weight element, that is, the shift resistor in the shift circuit, should use the reciprocal of the weights having 10 values among the weight values of the elements having a weight of 9 to 11.

For example, if a device with a weight value of 9 to 11 and a conductance value of 0.001 to 0.005 is used, the value of R _shift is used as 1/0.003; Since resistance is the reciprocal of conductance, the reciprocal of the average value of 0.001 and 0.005 is used.

Therefore, by subtracting a value output from an array and a value output from a shift circuit, an output considering the negative weight can be obtained. In this example, the voltage output was used by attaching a load resistance, and the voltage output to the first output (or post neuron) is (i ₁ × R _load )-(i _shift × R _load )

Next, the effects of the present invention will be described in more detail through experiments.

In order to experimentally prove the system according to the present invention, an experiment was conducted using 16 input signals and 5 post neurons (output units). First, using a python-based software, training was performed through the operation method shown in Fig. 4 and a synapse weight value to be used in the circuit was obtained (16 x 5 = 80 weight values).

And the obtained synapse weights were compared with software after designing the circuit through the pspice, one of the programs for circuit simulation. The output value of the post neuron on python and the output voltage on the neuromorphic circuit designed with pspice were compared and analyzed.

FIG. 6 is a comparison of five output values in python and pspice. As a result, I got the output on pspice that matched the software output on python.

In the above, the invention made by the present inventor has been described in detail according to the above embodiment, but the invention is not limited to the above embodiment, and it goes without saying that various modifications can be made without departing from the gist of the invention.

10: pre-neuron 20: post neuron
30: synapse 40: shift weight element
50: subtraction circuit

Claims (7)

In a neuromorphic circuit system capable of efficiently implementing negative weights,
A large number of free neurons;
Multiple post neurons;
A plurality of synapses disposed on intersections of a plurality of row lines extending in a row direction from each of the free neurons and a plurality of column lines corresponding to each of the post neurons to form a synaptic array;
A shift circuit that weights and adds shift weights to the inputs of the plurality of free neurons, and outputs the summed result; And,
A subtraction circuit for subtracting the output of the shift circuit from the output of each of the plurality of column lines, and outputting the subtracted output to each of the post neurons,
Each of the plurality of synapses has a weight shifted from an original weight to the shift weight. A neuromorphic circuit system capable of implementing an efficient negative weight.
The method of claim 1,
The shift circuit is a neuromorphic capable of implementing an efficient negative weight, characterized in that it is composed of a plurality of shift weight elements each having the shift weight and connected to a free neuron, and a shift line connecting all of the plurality of shift weight elements. Circuit system.
The method of claim 2,
The shift weight element is a neuromorphic circuit system capable of implementing an efficient negative weight, characterized in that consisting of a resistance element.
The method of claim 3,
A neuromorphic circuit system capable of implementing an efficient negative weight, characterized in that having a load resistance in the subtraction circuit.
The method of claim 3,
A neuromorphic circuit system capable of implementing an efficient negative weight, characterized in that the resistance R _shift of the resistance element is set to a resistance according to Equation 1.
[Equation 1]

However, w _shift is the shift weight.
The method of claim 1,
The shift weight is a neuromorphic circuit system capable of implementing an efficient negative weight, characterized in that the value of 1/2 of the maximum width of the weight represented by the synapse.
The method of claim 2,
A neuromorphic circuit system capable of implementing an efficient negative weight, characterized in that the shift weight element and the synapse are implemented by the same element.

Images