KR102567230B1 - Artificial neural network system using phase change material - Google Patents

Abstract

According to an embodiment, an input layer receiving input data; an output layer outputting output data corresponding to the input data; and a plurality of hidden layers positioned between the input layer and the output layer, wherein the hidden layers include a first node outputting a first result value by performing an activation function operation and a first result value output from the first node An artificial neural network system using a phase change element is provided, and an activation function of the first node is determined according to resistance-voltage characteristics of the phase change element.

Description

Artificial neural network system using phase change material}

An embodiment of the present invention relates to an artificial neural network system using a phase change element, and specifically, to an artificial neural network system using a phase change element that can be applied to a 3D Xpoint memory element.

3D cross-point technology is based on fast-acting, non-volatile phase change materials. The short-term goal of this technology is to use it as a new type of memory/storage module that solves the speed bottleneck between DRAM and NAND Flash, and the long-term goal is to use it as a storage class memory that replaces both DRAM and NAND Flash.

The phase change material can adjust the reliability range of the 3D cross point memory/storage module according to its characteristics. This 3D cross-point technology is also receiving a lot of attention for its possibility of being used as a neuromorphic chip because it is a structure suitable for parallel calculation optimized for deep learning.

The 3D cross-point technology has linearity, which indicates the degree to which the resistance rises linearly according to the number of applied pulses, and symmetry, in which the resistance level changes symmetrically according to the alternating application of set-reset pulses ( It is absolutely necessary to develop a phase change material with excellent symmetricity at the same time. Through this, Ohm's law and Kirchhoff's current law can be directly applied to the artificial neural network system and used at 1-PCM/cell. However, until now, the development of phase change materials that satisfy linearity and symmetry enough to demonstrate effective performance in artificial neural network systems is very slow.

A technical problem to be achieved by the present invention is to provide an artificial neural network system using a phase change device optimized for a 3D cross point memory device.

According to an embodiment, an input layer receiving input data; an output layer outputting output data corresponding to the input data; and a plurality of hidden layers positioned between the input layer and the output layer, wherein the hidden layers include a first node outputting a first result value by performing an activation function operation and a first result value output from the first node An artificial neural network system using a phase change element is provided, and an activation function of the first node is determined according to resistance-voltage characteristics of the phase change element.

The phase change element may include an upper metal line and a lower metal line, a phase change material disposed between the upper metal line and the lower metal line, and a selector.

The voltage-resistance characteristics of the phase change element may change according to the form of a sigmoid function.

The second node may add current values flowing through the phase change element.

The output layer may output the output data by assigning a weight to the value summed at the second node.

The phase change material may be made of a Ge-Sb-Te-based material.

The phase change material may be made of an Ag-In-Sb-Te-based material.

The phase change material may be made of a Ti-Sb-Te-based material.

The voltage-resistance characteristics of the phase change element may change according to the shape of the ReLu function.

Voltage-resistance characteristics of the phase change element may change according to a function form in which a plurality of phase change materials are combined.

An artificial neural network system using a phase change device according to the present invention can provide an artificial neural network system optimized for a 3D cross point memory device.

In addition, the calculation cost can be greatly reduced.

In addition, it can greatly contribute to reducing manufacturing cost and improving yield.

1 is a conceptual diagram of an artificial neural network system according to an embodiment.
2 is a conceptual diagram of a phase change element according to an embodiment.
3 and 4 are voltage-resistance graphs of phase change elements according to embodiments.
5 is a graph showing voltage-resistance characteristics of Ge-Sb-Te-based phase change materials.
6 is a graph showing voltage-resistance characteristics of Ag-In-Sb-Te-based phase change materials.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

However, the technical idea of the present invention is not limited to some of the described embodiments, but may be implemented in a variety of different forms, and if it is within the scope of the technical idea of the present invention, one or more of the components among the embodiments can be selectively implemented. can be used by combining and substituting.

In addition, terms (including technical and scientific terms) used in the embodiments of the present invention, unless explicitly specifically defined and described, can be generally understood by those of ordinary skill in the art to which the present invention belongs. It can be interpreted as meaning, and commonly used terms, such as terms defined in a dictionary, can be interpreted in consideration of contextual meanings of related technologies.

Also, terms used in the embodiments of the present invention are for describing the embodiments and are not intended to limit the present invention.

In this specification, the singular form may also include the plural form unless otherwise specified in the phrase, and when described as "at least one (or more than one) of A and (and) B and C", A, B, and C are combined. may include one or more of all possible combinations.

Also, terms such as first, second, A, B, (a), and (b) may be used to describe components of an embodiment of the present invention.

These terms are only used to distinguish the component from other components, and the term is not limited to the nature, order, or order of the corresponding component.

In addition, when a component is described as being 'connected', 'coupled' or 'connected' to another component, the component is not only directly connected to, combined with, or connected to the other component, but also with the component. It may also include the case of being 'connected', 'combined', or 'connected' due to another component between the other components.

In addition, when it is described as being formed or disposed on the "top (above) or bottom (bottom)" of each component, the top (top) or bottom (bottom) is not only a case where two components are in direct contact with each other, but also one A case in which another component above is formed or disposed between two components is also included. In addition, when expressed as "up (up) or down (down)", it may include the meaning of not only an upward direction but also a downward direction based on one component.

Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings, but the same or corresponding components regardless of reference numerals are given the same reference numerals, and overlapping descriptions thereof will be omitted.

1 is a conceptual diagram of an artificial neural network system according to an embodiment. Referring to FIG. 1 , an artificial neural network system using a phase change element according to an embodiment may include an input layer 110 , a hidden layer 120 and an output layer 130 .

In the artificial neural network system 100 using a phase change element, artificial neurons that simplify the functions of biological neurons are used, and the artificial neurons are connected through connection lines having connection weights (W ₁ , W ₂ , ..., W _n ). can be interconnected. The connection weight (W ₁ , W ₂ , ..., W _n ) is a specific value of the connection line and can also be expressed as connection strength. An artificial neuron may be referred to as a node.

The input layer 110, the hidden layer 120, and the output layer 130 may include a plurality of nodes 110-1, ..., 110-n. For example, the input layer 110 may include a plurality of input nodes 110-1, ..., 110-n. Each hidden layer may include a plurality of hidden nodes 121-1, 122-1, ..., 121-n, 122-n. The output layer 130 may include a plurality of output nodes 130-1, ..., 130-n. The number of nodes included in each layer (input layer, hidden layer, output layer) may be different.

The input layer 110 may receive input data. Each of the plurality of input nodes 110-1, 110-n constituting the input layer 110 may receive input values. That is, one input value may be input to one node. According to an embodiment, the artificial neural network system 100 using a phase change element may provide an output corresponding to an input. The input layer 110 may receive an input for performing learning and transfer it to the plurality of hidden layers 120 . For example, the input may include audio data, video data, biometric data, and handwriting data.

The plurality of hidden layers 120 may include a first hidden layer to an n-th hidden layer. The plurality of hidden layers may change learning data transmitted through the input layer into values that are easy to predict. Each hidden layer 120 transfers the input value received in the previous hidden layer to the next hidden layer based on a specific criterion, and may apply different connection weights to each hidden layer before forwarding the input value. In the embodiment, one hidden layer is illustrated for convenience of explanation, but a plurality of hidden layers may be applied to the present invention.

Nodes 121-1, 122-1, ..., 121-n, and 122-n of the hidden layer 120 may receive node inputs. Node inputs can contain input values. Input values may be received from nodes in the previous hidden layer. Each of the input values may be multiplied with a connection weight. The weights may correspond one-to-one with connection lines with nodes of the previous hidden layer. The product of each of the input values and each of the connection weights may be summed by a transfer function. The activation function may output a node output by limiting the range of the output value of the transfer function.

The activation function limits the range of the output value of the transfer function, provides convenience in calculation, and serves to enable the artificial neural network to make complex decisions rather than simple calculations. Accordingly, the learning efficiency of the artificial neural network system 100 using the phase change element may vary depending on which activation function is used.

The hidden layer 120 may include a first node 121-1, ..., 121-n and a second node 122-1, ..., 122-n. The hidden layer 120 includes a first node 121-1, ..., 121-n and a first node 121-1, ..., 121-n that perform an activation function operation and output a first result value. ) may include a second node 122-1, ..., 122-n for summing the first result values output.

In an embodiment, the activation function of the first node 121-1, ..., 121-n may be determined according to resistance-voltage characteristics of the phase change element.

2 is a conceptual diagram of a phase change element according to an embodiment. Referring to FIG. 2, the first node 121-1, ..., 121-n composed of the phase change element 1210 includes a word metal line 1213, column metal lines 1211 and 1212, and a word metal line. A phase change material and selectors 1215 and 1217 disposed between the line 1213 and the column metal lines 1211 and 1212 may be included.

The phase change element 1210 includes memory cells 1214 and 1216 made of a phase change material at the cross points of the word line 1213 and the column lines 1211 and 1212. It can have a two-layered structure in a 3D form.

In the phase change element 1210, the top four metal lines become the second-layer column lines 1211, and the bottom four metal lines of the same shape become the first-layer column lines 1212. In addition, four metal lines orthogonal to the middle column lines 1211 and 1212 are word lines 1213, and memory cells 1214 and 1216 of the first and second layers may share them.

The memory cells 1214 and 1216 may be formed of a phase change material disposed at an intersection of the word line 1213 and the column lines 1211 and 1212, and a selector attached to each memory cell 1214 and 1216. Depending on the voltage applied to 1215 and 1217, whether a read operation or a write operation may be determined.

Referring back to FIG. 1 , in the embodiment, the voltage-resistance characteristics of the phase change element may change according to the form of a sigmoid function. The activation function of the artificial neural network system 100 can perform deeper learning as it has nonlinear characteristics. The sigmoid function is a function suitable for binary classification, and input data can be classified by classifying it as a value of 0 or 1 based on a certain value. Since the sigmoid function is activated only when a specific threshold value is exceeded in the artificial neural network system 100, it can be used as an activation function.

3 and 4 are voltage-resistance graphs of phase change elements according to embodiments. Referring to FIG. 3, when a voltage is gradually applied to the phase change element according to the embodiment while maintaining a constant pulse width (ns), the resistance value rapidly decreases at a constant voltage. section can be found. That is, the phase change element maintains a constant resistance value in a certain voltage range, but takes the form of an S-curve in which the resistance value rapidly decreases and is maintained when a voltage exceeding a threshold voltage value is applied. It can be seen that this S-shaped curve has a curve shape similar to that of the sigmoid function outputting a value of 0 or 1 based on a certain value. In the phase change element according to the embodiment, the resistance value tends to rapidly decrease to 1000 times or less based on a specific threshold voltage value between 1 and 2 [V].

In addition, referring to FIG. 4, when the number of pulse applications is accumulated and increased while maintaining a set pulse width (ns) constant to the phase change element according to the embodiment, the resistance A section where the value rapidly decreases can be found. That is, the phase change element maintains a constant resistance value even when a certain number of pulses are applied, but takes the form of an S-curve in which the resistance value rapidly decreases and is maintained when the threshold pulse application number is exceeded. It can be seen that this S-shaped curve has a curve shape similar to that of the sigmoid function outputting a value of 0 or 1 based on a certain value. The resistance value of the phase change element according to the embodiment tends to rapidly decrease to 100 times or less based on the cumulative number of pulses applied between about 5 and 10.

Referring back to FIG. 1 , the second nodes 122-1, 122-n may sum the current values flowing through the phase change elements. The current flowing through the memory cells of the first nodes 121-1, 121-n varies according to Ohm's law depending on the voltage-resistance characteristics of the phase change element. That is, the current flowing through the memory cells of the first nodes 121-1, 121-n may be proportional to the voltage value of the phase change element and inversely proportional to the resistance value. The second node 122-1, ..., 122-n corresponds to the current flowing in the memory cell of the first node 121-1, ..., 121-n connected according to Kirchhoff's current law. The values may be summed and transmitted to the output layer 123 .

The phase change material may be made of a Ge-Sb-Te-based material. 5 is a graph showing voltage-resistance characteristics of Ge-Sb-Te-based phase change materials. Referring to FIG. 5 , a phase change may occur in a phase change material by adjusting a pulse width while maintaining voltages of a set pulse and a reset pulse at 3 [V] and 5 [V], respectively. In the case of the set pulse, it can be confirmed that a phase change is caused based on a pulse width of about 400 ns to move from a high resistance state to a low resistance state. In the case of the reset pulse, it can be confirmed that a phase change is caused based on a pulse width of about 170 ns to move from a low resistance state to a high resistance state. It can be confirmed that the Ge-Sb-Te-based phase change material caused a phase change based on the above-described threshold pulse width to rapidly change the resistance state. In addition, it can be confirmed that the voltage-resistance characteristic graph according to the rapid change in resistance takes the form of a sigmoid function.

Alternatively, the phase change material may be made of an Ag-In-Sb-Te-based material. Ag-In-Sb-Te-based phase change material has the advantage of being able to operate with a low reset current and having a fast crystallization speed as a material that compensates for the low crystallization temperature and high melting point raised as disadvantages of other phase change materials. . 6 is a graph showing voltage-resistance characteristics of Ag-In-Sb-Te-based phase change materials. Referring to FIG. 6, when a voltage having a set pulse width of 50 ns was applied to the resistance of the Ag-In-Sb-Te-based phase change material, the phase change did not occur even if the magnitude of the voltage was increased. However, it can be confirmed that when a voltage having a pulse width of 100 ns or more is applied, a phase change occurs in a section exceeding the threshold voltage and moves from a high resistance state to a low resistance state. When a set pulse width of 100n and 400nss was applied, it can be seen that the phase change of the Ag-In-Sb-Te-based phase change material occurred at about 1.1 [V] and moved from a high resistance state to a low resistance state. However, it can be confirmed that the phase change is completed in a faster time when a set pulse width of 400 ns is applied than when a set pulse width of 100 ns is applied.

Alternatively, the phase change material may be made of a Ti-Sb-Te-based material.

Therefore, a sigmoid function close to a step function can be implemented by applying a long set pulse width, and a sigmoid function of a gentle S-curve can be implemented by applying a relatively short set pulse width. In this way, a sigmoid function of a desired shape may be implemented by adjusting the set pulse width or the reset pulse width applied to the Ag-In-Sb-Te-based phase change material.

In addition, the phase change material may implement various sigmoid functions according to electrical properties. Therefore, it is possible to implement a sigmoid function having various forms by selecting a phase change material in consideration of electrical properties such as thermal conductivity, resistance, and on-off ratio. It can be applied as an activation function of the system.

The resistance of the phase change material according to the embodiment is changed in the form of a sigmoid function according to the voltage size of the applied pulse or the number of pulses. Therefore, the process of inputting the vector value transferred from the input layer to the activation function and calculating it can be replaced by using the characteristics of the phase change element. Ah, if the resistances of several memory cells are read at once, the current flowing at this time may be the sum of the currents determined by the resistances of each cell. This is the same as the summation process in the feed forward process of the artificial neural network system. In this way, the operation cost can be greatly reduced by replacing the activation function operation and the sum operation with a method of changing the phase by applying a voltage to the phase change element and reading the current flowing in the cells without an additional process.

In addition, the voltage-resistance characteristics of the phase change element according to the embodiment may change according to the shape of a Rectified linear unit (ReLu) function. The RELU function returns 0 when a value less than 0 is input, and returns the value as it is when a value greater than 0 is input.

In addition, voltage-resistance characteristics of the phase change element according to the embodiment may change according to a function form in which a plurality of phase change materials are combined. For example, the voltage-resistance characteristics of the phase change element may be changed in the form of a function in which a sigmoid function and a RELU function are combined. In addition, it can be changed in the form of a combination of various nonlinear functions such as implementation Tanh and Leaky ReLU.

The output layer 130 may output output data by assigning weights to values summed at the second node. The output layer 130 may output output data corresponding to input data. The output layer 130 may generate an output based on the signal received from the hidden layer 120 . Each of the plurality of output nodes constituting the output layer 130 may output output values. That is, one output value can be output from one output node. A connection weight may be applied between one node and the next node. Connection weights can be determined individually from node to node. Connection weights can be updated through learning and error backpropagation.

The artificial neural network system 100 using the phase change device may train the neural network through supervised learning. Supervised learning is a method of inputting training data and output data corresponding thereto to a neural network, and updating connection weights of connection lines so that output data corresponding to the training data is output. For example, the artificial neural network system 100 using a phase change element may update connection weights between artificial neurons through a delta rule and backpropagation learning.

Error back-propagation learning, after estimating the error with forward computation for the given training data, propagates the estimated error by starting from the output layer and proceeding inversely toward the hidden layer and the input layer, and is connected in the direction of reducing the error. How to update weights. For example, error backpropagation learning can be divided into a step of calculating an output by passing an input to each layer and a step of modifying connection weights in a neural network to reduce an error in an output. The artificial neural network system 100 using a phase change element defines an objective function for measuring how close to optimal the currently set connection weights are, continuously changes the connection weights based on the result of the objective function, and learns can be performed repeatedly. For example, the objective function may be an error function for calculating an error between an output value actually output by the neural network based on training data and an expected value desired to be output. The artificial neural network system 100 using the phase change element may update connection weights in a direction of reducing the value of the error function.

For example, as a method of updating connection weights, there is a gradient descent method. Gradient descent refers to updating the connection weights in a direction that reduces the change in error (slope) according to the change in the connection weight of each node. Since an artificial neural network consists of a plurality of layers, the connection weight of each node may mean one of several connection weights multiplied by a signal.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the claims below. You will understand that it can be done.

100: artificial neural network system
110: input layer
120: hidden layer
130: output layer
1210: phase change element
121-1, ..., 121-n: first node
122-1, ..., 122-n: second node
1211, 1212: column metal line
1213: word metal line
1214, 1216 memory cells
1215, 1217: selector

Claims (10)

an input layer receiving input data;
an output layer outputting output data corresponding to the input data; and
Includes a plurality of hidden layers located between the input layer and the output layer,
The hidden layers include a first node outputting a first result value by performing an activation function operation and a second node summing the first result value output from the first node,
The activation function of the first node is determined according to the resistance-voltage characteristics of the phase change element,
The phase change element includes a memory cell made of a phase change material at an intersection of a word line and a column line and a selector attached to the memory cell,
The phase change element includes a lowermost column line on the first layer, a uppermost column line on the second layer, a middle column line between the first and second layers, and a word line orthogonal to the middle column line, sharing the word line. An artificial neural network system using a phase change element in which memory cells of the first and second layers are respectively disposed.
delete According to claim 1,
An artificial neural network system using a phase change element in which the voltage-resistance characteristic of the phase change element changes according to the form of a sigmoid function.
According to claim 3,
The artificial neural network system using a phase change element in which the second node sums current values flowing through the phase change element.
According to claim 1,
The artificial neural network system using a phase change element, wherein the output layer outputs the output data by assigning a weight to the value summed at the second node.
According to claim 1,
The phase change material is an artificial neural network system using a phase change element made of a Ge-Sb-Te-based material.
According to claim 1,
The phase change material is an artificial neural network system using a phase change element made of Ag-In-Sb-Te-based material.
According to claim 1,
The phase change material is an artificial neural network system using a phase change element made of a Ti-Sb-Te-based material.
According to claim 1,
An artificial neural network system using a phase change element in which the voltage-resistance characteristic of the phase change element changes according to the shape of the ReLu function.
According to claim 1,
The artificial neural network system using a phase change element in which the voltage-resistance characteristic of the phase change element changes according to a function form in which a plurality of phase change materials are combined.

Images