KR20190114212A - Single Column Memristor Crossbar and CMOS Activation Function Circuit for Binarized Neural Networks - Google Patents

Abstract

The present invention relates to a single column memristor crossbar and a CMOS activation function circuit for a binary value based neural network which calculate positive and negative values of a neural network all together using a single column which is capable of reducing power consumption and an area of the crossbar by half. The single column memristor crossbar comprises: a single column memristor array making row lines and column lines form intersecting points, having memristor devices connected between the column lines and row lines at the intersecting points, and programmed in a high resistance state (HRS) or a low resistance state (LRS) to be analyzed with binary snap connections (g_(0,0),g_(1,1),···,g_(n,m)) by applying voltage or current pulses; an input circuit end multiplying input pulses (x_0,x_1,x_2,···,x_n), supplied to the row lines of the single column memristor, by g_b conductance; a current subtracter subtracting the multiplication between the input pulses (x_0,x_1,x_2,···,x_n) and the g_b conductance, obtained by the input circuit end, from each column of the single column memristor; and a current-voltage converter converting output current of each column into output voltage and outputting final outputs (y_0,y_1,···,y_m) using the converted output voltage as an input of an activation function circuit.

Description

Single Column Memristor Crossbar and CMOS Activation Function Circuit for Binarized Neural Networks

The present invention relates to a binary value-based neural network configuration. Specifically, a binary that calculates both the positive and negative values of a neural network together using a single column to reduce the power consumption and area of the crossbar by half. A single column memristor crossbar and a CMOS activation function circuit for value based neural networks.

Since the development of electronics, there have been only three types of circuit components: resistors, inductors, and capacitors.

But in 1972, UC Berkeley researcher Leon Chua theorized that a fourth type of component could exist. It is a memristor that can measure the flow of current.

A memristor is a compound word of memory and a register that can "remember" how much current has passed through the memristor.

In addition, by alternating the amount of current passing through, the memristor can be a single element circuit component having unique characteristics.

The most prominent feature of memristors is the ability to store electronic states even when the current is off, which is why they are attracting attention as the next-generation devices that can replace today's flash memory.

Memristors are theoretically cheaper, much faster than flash memory, and enable higher memory densities. It also replaces RAM chips, so when you turn off your computer and turn it back on, you can remember exactly what you were working on and go back to work immediately.

Such memristor crossbar arrays can be used in a variety of applications, including nonvolatile solid state memory, programmable logic, signal processing, control systems, pattern recognition and other applications.

In addition, the memristor's nonlinear charge flux relationship can be used to describe synapses in neurobiological systems such as the human brain, and in addition to this nonlinear relationship, the memristor crossbar array has a very compact memristor crossbar structure and Since it is the same three-dimensional, it is suitable for realizing a brain-like structure.

1 is a conceptual diagram of a binary neural network circuit (BNN) having binary synaptic weights of +1 and -1, and FIG. 2 is a configuration diagram of a memristor crossbar for implementing a binary neural network circuit (BNN).

Binary value-based neural networks have received a lot of attention recently because they can avoid the complex real computations required by real-based neural networks.

Memristors can store analog values, but they can also store binary values with high and low resistance, which can be applied to neural network implementation.

3 is a diagram illustrating a double column structured memristor crossbar for implementing a conventional binary neural network (BNN).

The double column structured memristor crossbar for implementing the conventional binary neural network (BNN) as described above calculates the positive and negative values separately using two columns. Inefficient in terms of

Therefore, there is a need for development of a technique for a memristor crossbar having a new structure suitable for implementing a binary neural network (BNN).

Korean Patent Registration No. 10-1282884 Korean Laid-Open Patent No. 10-2017-0108627 Korean Laid-Open Patent No. 10-2017-0074234

The present invention is to solve the problem of the prior art binary neural network (BNN) and the memristor crossbar to implement the same, crossbar by calculating both the positive and negative values of the neural network using a single column together The aim is to provide a single-column memristor crossbar and CMOS activation function circuit for binary-based neural networks that can reduce power consumption and area by half.

The present invention provides a single-column memristor crossbar and a CMOS activation function circuit for a binary value-based neural network that enables the entire binary value-based neural network to be implemented as a hybrid system of memristors and CMOS by implementing the activation function circuit as a CMOS circuit. The purpose is to provide.

The present invention can be implemented in a small area by using a MOSFET transistor instead of a resistor without adding a resistor to the memristor array, and includes the entire neural network circuit as a CMOS-memistor mixed circuit including a CMOS circuit capable of implementing an activation function. The objective is to provide a single column memristor crossbar and CMOS active function circuit for a binary-based neural network that can be implemented.

The objects of the present invention are not limited to the above-mentioned objects, and other objects that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, a single column memristor crossbar and a CMOS activation function circuit for a binary value-based neural network according to the present invention form row and column lines with intersection points and intersection points. Has memristor devices connected between the column line and the low line, and is programmed to a high resistance state (HRS) or low resistance state (LRS) by applying a voltage or current pulse to the binary synaptic connection ( a single column memristor array that is interpreted as g _0,0 , g _1,1 , .., g _{n, m} ); input pulses (x ₀ , x supplied to the row lines of the single column memristor array) ₁ , x ₂ , ... x _n ) and the input circuit stage to multiply the g _b conductance; multiply the input pulse (x ₀ , x ₁ , x ₂ , ... x _n ) and g _b conductance obtained from the input circuit stage. Subtracted to each column in a single column memristor array Flow subtractor; each output current of the column _{(I 0, I 1, ..} , I j, I m) for converting the output voltage using the input of the activation function circuit final output (y _0, y _1, .. , y _m ) outputs a current-voltage converter.

Here, the input pulse (x ₀ , x ₁ , x ₂ , ... x _n ) is an input neuron connected to the final output (y ₀ , y ₁ , .., y _m ) which is an output neuron by synaptic connection. It is done.

만큼 생성하는 것을 특징으로 한다.We then generate a current I _b using g _b to compute both +1 and -1 values using a single memristor column, and the g _b column applied by the input pulses in the column produces I _b current.

It is characterized by generating as much.

The generated I _b is copied to each column, and can be subtracted from all crossbar columns at the same time.

The jth neuron yj is combined by combining I _b subtraction and the activation function f, where each column current is converted into a voltage.

으로 나타내는 것을 특징으로 한다.

It is characterized by.

And if g _b is given by (g _LRS + g _HRS ) / 2, then the synaptic linkage of g _{i, j} -g _b is + (g _LRS -g _HRS ) / 2 _or- (g _LRS -g _HRS ) / 2 Where g _{i, j} is g _LRS or g _HRS ,

These two values of + (g _LRS -g _HRS ) / 2 _and- (g _LRS -g _HRS ) / 2 are characterized as being interpreted as binary synapses.

The current subtractor includes an operational amplifier OP ₀ , and is configured between the output terminal of the operational amplifier OP ₀ and each column to provide an NMOSFET M ₁ , which serves as a resistor for subtracting I _b currents from all memristor columns. And a bias voltage V _c for controlling the channel resistance is applied to the gate terminal of the NMOSFET M ₁ .

And the current-to-voltage converter has an activation function circuit of a rectified linear unit (ReLU) or sigmoid, and the output current (I ₀ , I ₁ , .., I _j ) of each column , along a I _m) to the activation function final output _{(y 0, y 1, ..} , y m) and an output, the final output _{(y 0, y 1, ..} , y m) are each cycle in accordance with the input pulse It is characterized by a change every time.

The ReLU circuit is used for hidden neurons, and the sigmoid circuit is used for output neurons.

In addition, an activation function circuit of a rectified linear unit (ReLU) or a sigmoid includes a first operational amplifier OP ₁ for converting I _j current into -I _j * R 1 voltage and an inversion buffer. And a second operational amplifier (OP ₂ ) that inverts -I _j * R1 to + I _j * R1, and acts as a limiter so that if the output y _j voltage is above VDD or below GND, the output voltage is either VDD or And a second operational amplifier OP ₃ limited to GND.

In the sigmoid activation function circuit, V _bias is applied to the inverting terminal (-) through the resistor (R ₂ ) so that the transfer curve of the sigmoid moves the transfer curve of the ReLU by -V _bias / R ₂ . It is characterized by obtained by.

Such a single column memristor crossbar and a CMOS activation function circuit for a binary value based neural network according to the present invention have the following effects.

First, we can reduce both the power consumption and the area of the crossbar by calculating both the positive and negative values of the neural network using a single column.

Second, by implementing the activation function circuit as a CMOS circuit, the entire binary value-based neural network can be implemented as a hybrid system of memristor and CMOS.

Third, a small area can be achieved by using MOSFET transistors instead of resistors without adding resistors to the memristor array.

Fourth, the entire neural network can be implemented as a CMOS-memistor mixed circuit including a CMOS circuit capable of implementing an activation function, thereby providing a new memristor crossbar structure suitable for neural network circuits.

1 is a conceptual diagram of a binary neural network (BNN) with binary synaptic weights of +1 and -1
2 is a diagram illustrating a memristor crossbar for implementing a binary neural network (BNN).
3 is a diagram illustrating a double column structured memristor crossbar for implementing a conventional binary neural network (BNN).
4 is a block diagram of a memristor crossbar of a single column structure for implementing a binary neural network (BNN) according to the present invention.
5A is a detailed block diagram of a single column memristor crossbar and a CMOS activation function circuit for a binary value based neural network according to the present invention.
5B is an operational waveform diagram of a single column memristor crossbar and a CMOS activation function circuit.
6A is a block diagram showing a ReLU activation circuit and input / output characteristics
6B is a schematic diagram showing a sigmoid activation circuit and input / output characteristics

Hereinafter, a preferred embodiment of a single column memristor crossbar and a CMOS activation function circuit for a binary value based neural network according to the present invention will be described in detail.

Features and advantages of a single column memristor crossbar and a CMOS activation function circuit for a binary value based neural network according to the present invention will become apparent from the detailed description of each embodiment below.

Figure 4 is a block diagram of a single column structure for implementing a binary neural network (BNN) according to the present invention.

5A is a detailed configuration diagram of a single column memristor crossbar and a CMOS activation function circuit for a binary value based neural network, and FIG. 5B is an operation waveform diagram of a single column memristor crossbar and a CMOS activation function circuit.

The single column memristor crossbar and CMOS activation function circuit for binary-based neural networks according to the present invention calculates both the positive and negative values of the neural network together using a single column to halve the power consumption and area of the crossbar. To reduce it.

The present invention may include a configuration that enables the entire binary-based neural network to be implemented as a hybrid system of memristor and CMOS by implementing the activation function circuit as a CMOS circuit.

The present invention can be implemented in a small area by using a MOSFET transistor instead of a resistor without adding a resistor to the memristor array, and includes the entire neural network circuit as a CMOS-meister circuit including a CMOS circuit capable of implementing an activation function. It can include a configuration that can be implemented.

The memristor array may be programmed to a high resistance state (HRS) or a low resistance state (LRS) by applying a voltage or current pulse, and may be applied to a neural network implementation.

Referring to Figure 4 will be described the configuration of a single column structure of the memristor crossbar for implementing a binary neural network (BNN) according to the present invention.

4 shows a structure in which the power consumption and the area of the crossbar can be reduced by half by calculating both the positive value and the negative value of the neural network using a single column.

A single column is used to calculate the binary synaptic connections of +1 and -1.

Thus, two columns of (+) and (-) are more efficient in terms of area and power consumption than the structure of FIG. 3 used to calculate positive and negative synaptic connections, respectively.

Here, x ₀ , x ₁ , x _2, and the like are input neurons connected to neurons such as y ₀ , y ₁ , and y ₂ by synaptic connections.

G ₀ , ₀ is the binary synaptic connection between x ₀ and y ₀ , and the current I _b is generated using g _b to calculate both +1 and -1 values using a single memristor column.

만큼 생성한다. _Referring to I _b of FIG. 4, the column g _b applied by the input pulse in the first column indicated by the dotted line shows the I _b current.

Generate as many.

I _b is then copied into each column and allowed to be subtracted from all crossbar columns at the same time.

The f block denotes an activation function circuit in which each column current is converted into a voltage according to the activation function f.

The revolving Lister crossbar structure of the double column structure of Figure 3 I _b generator and a subtracter is used, but, in the revolving Lister crossbar structure of single-column structure of Figure 4 according to the present invention uses an activation function circuit.

In addition, an I _b generator and an I _b subtractor are configured using a CMOS circuit without using passive resistors as shown in FIG. 3.

The detailed structure of the I _b generator, I _b subtractor and activation function circuit according to the present invention is as in FIG. 5A.

By combining I _b subtraction and f activation, we can represent the j th neuron yj in the following way:

Given g _b from (1), given by (g _LRS + g _HRS ) / 2, the synaptic linkage of g _{i, j} -g _b is + (g _LRS -g _HRS ) / 2 _or- (g _LRS -g _HRS ) / 2.

Where g _{i, j} is g _LRS or g _HRS .

These two values of + (g _LRS -g _HRS ) / 2 _and- (g _LRS -g _HRS ) / 2 are interpreted as binary synapses as in FIG.

Such a single column structure of the memristor-based BNN of FIG. 4 according to the present invention has two obvious advantages over the double column structure of FIG. 3.

First, the membrane programming time can be halved because the number of crossbar columns in FIG. 4 is half the number of columns in FIG.

Also, the area of the memristor array can be halved compared to the double column structure for the same reasons as the first advantage.

5A is a detailed block diagram of a single column memristor crossbar and a CMOS activation function circuit for a binary value based neural network according to the present invention.

As in FIG. 5A, the Row lines and the Column lines form intersections, have memristor devices connected between the column line and the row line at the intersection and apply a voltage or current pulse to Supply to the single column memristor array 50 and the low lines of the single column memristor array 50 that are programmed to a high resistance state (HRS) or a low resistance state (LRS). The input circuit stage 60 that multiplies the input pulses (x ₀ , x ₁ , x ₂ , ... x _n ) and g _b conductance, and the input pulses (x ₀ , x ₁ , a current subtractor 70 that subtracts the product of x ₂ , ... x _n ) and g _b conductance into each column of the single column memristor array 50, and converts the output current of each column into an output voltage to activate the function. using the input circuit of the final output _{(y 0, y 1, ..} , y m) the output current-I And a converter (80).

Memristor HRS and LRS can be used to calculate binary synaptic connections between neurons of different layers.

x ₀ , x ₁ , x ₂ , ... x _n are the input neurons that deliver the input pulses to the memristor crossbars, and the binary synaptic weights are stored at the intersections.

y ₀ , y ₁ , .., y _m and the like represent hidden neurons.

And g _b can be simply implemented as a diode-connected NMOSFET, M _b .

By adjusting the NMOSFET size, the channel conductance of M _b in FIG. 5A can be adjusted to be the same as g _b in FIG. 4. x _0, input pulse from the x _1, x _2, etc. is applied to the g _b, which generates a I _b.

I _b current is copied to each column and subtracted from all memristor columns as shown in FIG. 5A.

And g _0,0 represents a binary synaptic connection between x ₀ and y ₀ , and g _1,1 represents a binary connection between x ₁ and y ₁ .

OP ₀ is the op amp of the I _b generator and subtractor, M ₁ serves as a resistor to subtract the I _b current from all memristor columns, and V _c refers to a bias voltage for controlling the channel resistance of M ₁ .

The block denoted by f in FIG. 5A means a current-to-voltage converter, and the column current I _j is converted into the y _j voltage according to the activation function f.

The activation function used here is a rectified linear unit (ReLU) or sigmoid.

5B is an operational waveform diagram of a single column memristor crossbar and CMOS activation function circuit for the binary value based neural network of FIG. 5A.

In the first cycle, x ₀ , x ₁ , x ₂ , and x ₃ deliver input pulses of 1, 1, 0, 1 to the crossbar, in which case LRS, LRS, HRS and LRS are stored in the y ₀ column, respectively. The I ₀ current can be the largest of all columns.

The I ₀ current enters the f block where it is converted to the y ₀ voltage according to the activation function.

Assuming that the activation function is ReLU, the converted y ₀ voltage is as in FIG. 5B.

The voltage y ₀ may change every cycle according to the input pulse as shown in FIG. 5B.

FIG. 6A is a diagram illustrating a ReLU activation circuit and input / output characteristics, and FIG. 6B is a diagram illustrating a sigmoid activation circuit and input / output characteristics.

ReLU is used for hidden neurons and sigmoids for output neurons.

In the ReLU circuit, OP ₁ converts I _j current to -I _j * R1 voltage, OP ₂ is an invert buffer and -I _j * R1 is inverted to + I _j * R1.

OP ₃ acts as a limiter and if the output y _j voltage is above VDD or below GND, the output voltage is limited to VDD or GND, respectively.

The transfer curve of the ReLU circuit is as in FIG. 6A.

The sigmoid circuit is as in FIG. 6B, where R ₂ and V _bias are added to the sigmoid circuit. By doing this, the transfer curve of sigmoid is obtained by shifting the transfer curve of ReLU by -V _bias / R ₂ .

A single column memristor crossbar and a CMOS activation function circuit for a binary value based neural network according to the present invention having such a structure has an array structure for a binary value based neural network or a ternary neural network rather than an analog neural network. Have

The present invention enables the implementation in a small area by using a MOSFET transistor instead of a resistor, not a method of adding a resistor to a memristor array.

In particular, to provide a single-column memristor crossbar and CMOS activation function circuits for binary-based neural networks that allow the entire neural network to be implemented as a CMOS-memristor mixed circuit, including CMOS circuits that can implement activation functions. It is.

It will be understood that the present invention is implemented in a modified form without departing from the essential features of the present invention as described above.

Therefore, the described embodiments should be considered in descriptive sense only and not for purposes of limitation, and the scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent scope are included in the present invention. It should be interpreted.

50. Single Column Memristor Array 60. Input Circuit
70. Current subtractor 80. Current-to-voltage converter

Claims (11)

Row lines and column lines form intersections, have memristor devices connected between column and row lines at the intersection, and a high resistance state by applying a voltage or current pulse Single column memristor array programmed to a low resistance state (LRS) or to be interpreted as a binary synaptic connection (g _0,0 , g ₁ , ₁ , .., g _{n, m} );
An input circuit stage that multiplies the input pulses (x ₀ , x ₁ , x ₂ , ... x _n ) and g _b conductances supplied to the row lines of the single column memristor array;
A current subtractor which subtracts the product of the input pulses (x ₀ , x ₁ , x ₂ ,... X _n ) and g _b conductance obtained at the input circuit stage to each column of the single column memristor array;
The output current (I ₀ , I ₁ , .., I _j , I _m ) of each column is converted to an output voltage and used as the input of the activation function circuit to the final output (y ₀ , y ₁ , .., y _m A single-column memristor crossbar and a CMOS activation function circuit for a binary value-based neural network. 2. The input of claim 1, wherein the input pulses (x ₀ , x ₁ , x ₂ , ... x _n ) are connected by an synaptic connection to a final output (y ₀ , y ₁ , .., y _m ) that is an output neuron. A single column memristor crossbar and CMOS activation function circuit for a binary value based neural network, characterized in that it is a neuron. The method of claim 1, wherein generating a current I _b with a g _b to use a single memristor column to calculate both the +1 and -1 values,
The g _b column applied by the input pulses in the column gives the I _b current

A single column memristor crossbar and CMOS activation function circuit for a binary-based neural network, characterized in that it generates. 4. The single column memristor crossbar and CMOS activation function circuit for binary-based neural networks of claim 3, wherein the generated I _b is copied to each column and subtracted from all crossbar columns at the same time. 4. The j-th neuron yj of claim 3, wherein the combination of I _b subtraction and the activation function f, in which each column current is converted to voltage,

A single column memristor crossbar and a CMOS activation function circuit for a binary value-based neural network. 6. The method of claim 5, g _b is (g + g _{LRS HRS)} / 2 is given by the synaptic connections of g _{i, j} -g _b is _{+ (g LRS - g HRS)} / 2 or - (g _LRS - g _HRS ) / 2
Where g _{i, j} is g _LRS or g _HRS ,
These two values, + (g _LRS -g _HRS ) / 2 _and- (g _LRS -g _HRS ) / 2, are interpreted as binary synapses, enabling single-column memristor crossbars and CMOS activation for binary-based neural networks. Function circuit. The method of claim 1, wherein the current subtractor comprises an operational amplifier OP ₀ ,
An NMOSFET (M ₁ ) configured between the output terminal of the operational amplifier (OP ₀ ) and each column to serve as a resistor for subtracting I _b currents from all memristor columns,
A single-column memristor crossbar and CMOS activation function circuit for binary-based neural networks, characterized in that a bias voltage (V _c ) is applied to the gate of the NMOSFET (M ₁ ) to control channel resistance. The method of claim 1, wherein the current-voltage converter,
Has an activation function circuit of a rectified linear unit (ReLU) or sigmoid,
Output the output current (I ₀ , I ₁ , .., I _j , I _m ) of each column to the final output (y ₀ , y ₁ , .., y _m ) according to the activation function,
Single column memristor crossbar and CMOS activation function circuit for binary-based neural networks, wherein the final output (y ₀ , y ₁ , .., y _m ) changes every cycle in response to an input pulse. 10. The single column memristor crossbar and CMOS activation function circuit of claim 8 wherein the ReLU circuit is used for hidden neurons and the sigmoid circuit is used for output neurons. The circuit of claim 8, wherein an activation function circuit of a rectified linear unit (ReLU) or a sigmoid includes:
A first operational amplifier (OP ₁ ) for converting I _j current into a voltage -I _j * R ₁ ,
A second operational amplifier (OP ₂ ) used as an inversion buffer to invert -I _j * R1 to + I _j * R1,
Acting as a limiter, if the output y _j voltage is above VDD or below GND, the output voltage includes a second operational amplifier OP ₃ that limits to VDD or GND, respectively. Column memristor crossbar and CMOS activation function circuit. The circuit of claim 10, wherein the sigmoid activation function circuit is
Binary value-based neural network, characterized in that the V _bias is applied to the inverting terminal (-) through the resistor (R ₂ ) so that the transfer curve of the sigmoid is obtained by moving the transfer curve of the ReLU by -V _bias / R ₂ . Single Column Memristor Crossbar and CMOS Activation Function Circuit.

Images