KR20210126335A - Neuromorphic device and operating method of the same - Google Patents

Abstract

Provided is a neuromorphic device for performing artificial neural network calculation. The neuromorphic device comprises: an on-chip memory for reading and storing neuromorphic data; and a neuromorphic processor including at least one synaptic array to perform artificial neural network calculation depending on neuromorphic data in the synaptic array. The synaptic array may include: a plurality of input lines connected to a plurality of axon circuits respectively to receive input signals based on neuromorphic data independently; a plurality of bit lines arranged with gaps from the input lines and outputting output current depending on artificial neural network calculation; a plurality of neuron circuits connected to the bit lines respectively to sum the output current and outputting output voltage if the summed output current exceeds a preset threshold value; a plurality of electrode pads stacked and arranged at determined intervals between the input lines and the bit lines; a plurality of memrister penetration structures connected respectively between any one among the input lines and any one among the bit lines through the electrode pads to form the current pass of the output current; a plurality of string selection transistors arranged respectively between the memrister penetration structures and the bit lines; and a decoder connected to the electrode pads to apply a word line selection signal and a string selection signal. Therefore, provided is a highly integrated high-power synaptic array.

Description

Neuromorphic device and its operation method

The present invention relates to a neuromorphic device, and more particularly, to a neuromorphic device including a three-dimensional synaptic array.

Interest in neuromorphic processors resembling the human nervous system is increasing. There have been studies to implement a neuromorphic processor by designing neuronal circuits and synaptic circuits, respectively, corresponding to neurons and synapses existing in the human nervous system. Such a neuromorphic processor may be used to drive various neural networks, such as a Convolutional NeuralNetwork (CNN), a Recurrent Neural Network (RNN), a Feedforward Neural Network (FNN), etc., and perform data classification or image recognition. ) can be used in fields such as

The neuromorphic device may include a synapse circuit that stores connection strength between neurons, and the synaptic circuit may be implemented using a memory device including variable resistance elements that store one or more bits. .

The technical problem to be solved by the present invention is to provide a highly integrated, high-power synaptic array.

Another technical problem to be solved by the present invention is to provide a highly integrated, high-power neuromorphic device.

Another technical problem to be solved by the present invention is to provide a method of operating a neuromorphic device with high integration and high power.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

Neuromorphic device according to some embodiments of the present invention for achieving the above technical object includes at least one synaptic array, the synaptic array is arranged to extend in a first direction, and input signals independently from each other from a plurality of axon circuits connected thereto A plurality of input lines for receiving A plurality of cell strings including a string selection transistor and at least two memristor elements connected in series in a third direction between any one of the bit lines of A plurality of electrode pads that are spaced apart from each other in the direction and are stacked and are connected to the string selection transistor and at least two resistive memristor elements, a decoder that applies a string selection signal or a word line selection signal to each of the plurality of electrode pads, and one cell and a plurality of neuron circuits connected to each of the bit lines connected to the string, summing the output signals, and converting and outputting the output signals when they exceed a preset threshold, wherein the synaptic array includes at least one cell string to which the string selection signal is applied. An artificial neural network operation can be performed on the input signal in the memristor element of the layer activated by the word line selection signal.

Neuromorphic device according to some embodiments for achieving the above technical problem includes an on-chip memory that reads and stores neuromorphic data and at least one synaptic array, It includes a neuromorphic processor that executes a neural network operation, and the synaptic array is connected to a plurality of axon circuits to receive input signals based on neuromorphic data independently of each other. a plurality of bit lines, each of which outputs an output current according to an artificial neural network operation, is connected to each of the plurality of bit lines to sum the output currents, and output as an output voltage when the summed output current exceeds a preset threshold A plurality of neuron circuits, a plurality of electrode pads stacked at a predetermined interval between a plurality of input lines and a plurality of bit lines, each passing through the plurality of electrode pads, between any one of the input lines and any one of the bit lines is connected to a plurality of memristor through structures forming a current path of the output current, each of which is connected to a plurality of string select transistors and a plurality of electrode pads disposed between the plurality of memristor through structures and the plurality of bit lines, , a decoder for applying a word line selection signal and a string selection signal.

The method of operating a neuromorphic device according to some embodiments for achieving the above technical problem includes at least one three-dimensional synaptic array, and when the neuromorphic device receives a plurality of input signals independent of each other, a word line An artificial neural network operation is performed on the input signal in a plurality of resistive memristor elements of a layer corresponding to the selection signal, and a plurality of output currents are outputted to the neuron circuit through the bit line as a result of the operation, and the output current is summed to a preset threshold value. is output as an output voltage, and the 3D synaptic array includes a plurality of layers, so that an artificial neural network operation independent of each other can be performed for each layer activated according to a word line selection signal.

The details of other embodiments are included in the detailed description and drawings.

1 shows a mathematical model for a neuromorphic operation.
2 is a conceptual diagram for explaining a neuromorphic operation.
3 to 5 are diagrams for explaining a configuration of a synaptic array circuit of a neuromorphic device according to some embodiments.
6A to 6D are diagrams illustrating one string of a synaptic array according to some embodiments.
7 is a diagram for describing an operation when any one layer is selected in a synaptic array according to some embodiments.
8 to 9 are conceptual diagrams for explaining the operation of a neuromorphic device according to some embodiments.
10 to 11 are conceptual diagrams for explaining an operation of a neuromorphic device according to some embodiments.
12 to 13 are conceptual diagrams for explaining an operation of each layer in a synaptic array according to some embodiments.
14 is a block diagram illustrating a hardware configuration of a neuromorphic device according to an embodiment.
15 is a block diagram illustrating a configuration of an electronic system according to some embodiments.

Hereinafter, embodiments according to the technical spirit of the present invention will be described with reference to the accompanying drawings.

1 shows a mathematical model for the neuromorphic operation, and FIG. 2 is a conceptual diagram for explaining the neuromorphic operation.

Biological neurons refer to cells present in the human nervous system. Biological neurons are one of the basic biological computational entities. The human brain contains about 100 billion biological neurons and 100 trillion interconnections between biological neurons.

The human brain can learn and memorize vast amounts of information by transmitting and processing various signals through a neural network formed by connecting a large number of neurons. Since the vast number of connections between neurons in the human brain is directly correlated with the massively parallel nature of biological computing, various attempts have been made to efficiently process vast amounts of information by simulating artificial neural networks. . For example, as a computing system designed to implement artificial neural networks at the neuron level, neuromorphic devices are being studied.

The behavior of biological neurons can be simulated with mathematical models. A mathematical model corresponding to a biological neuron is an example of a neuromorphic operation. A multiplication operation for multiplying information from a plurality of neurons by a synaptic weight, an addition operation for values multiplied by the synaptic weight (Σ), and an addition operation It may include an operation that applies the characteristic function (b) and the activation function (f) to the result.

Referring to FIG. 1 , an example of an artificial neural network including an input layer, hidden layers, and an output layer is illustrated. The artificial neural network may perform an operation based on received input data (eg, Vin 1 , Vin 2 , and Vin3 ) and generate output data (eg, Vout1 and Vout2 ) based on the execution result.

The artificial neural network may be a deep neural network (DNN) or n-layers neural networks including two or more hidden layers. For example, as shown in FIG. 1 , the artificial neural network may be a DNN including an input layer (Layer 1), two hidden layers (Layer 2 and Layer 3), and an output layer (Layer 4). DNNs may include, but are not limited to, CNNs, RNNs, FNNs, Deep Belief Networks, Restricted Boltzman Machines, and the like.

Meanwhile, although the artificial neural network of FIG. 1 is illustrated as including four layers, this is only an example and the artificial neural network may include fewer or more layers. Also, the artificial neural network may include layers having various structures different from those shown in FIG. 1 .

Each of the layers included in the artificial neural network includes a plurality of artificial neurons, also known as “neuron”, “processing element (PE)”, “unit” or similar terms. may include For example, as shown in FIG. 1 , Layer 1 may include 3 neurons, and Layer 2 may include 4 neurons. However, this is only an example, and each of the layers included in the artificial neural network may include a variable number of neurons.

1 , neurons included in each of the layers included in the artificial neural network may be connected to each other to exchange data. For example, one neuron may receive data from other neurons of a previous layer and perform an operation, and may output an operation result to other neurons of a next layer.

An output value of each of the neurons may be referred to as activation. Activation may be an output value of one neuron and an input value of neurons included in the next layer. Meanwhile, each of the neurons may determine its own activation value based on activations and weights received from the neurons included in the previous layer. The weight is a parameter used to calculate activation in each neuron, and may be a value assigned to a connection relationship between neurons. Weights can be stored in synapses that connect neurons.

는 (i-1) 번째 레이어에 포함된 k번째 뉴런으로부터 i 번째 레이어에 포함된 j번째 뉴런으로의 웨이트이며,

는 i 번째 레이어에 포함된 j 번째 뉴런의 바이어스(bias)이고,

는 i 번째 레이어의 j 번째 뉴런의 액티베이션이라고 할 때, 액티베이션

는 다음과 같은 수학식 1을 따를 수 있다.Each of the neurons may be a computational unit that receives an input and outputs an activation, and may map input-output. For example, σ is the activation function,

is the weight from the k-th neuron included in the (i-1)-th layer to the j-th neuron included in the i-th layer,

is the bias of the j-th neuron included in the i-th layer,

is the activation of the j-th neuron of the i-th layer,

may follow Equation 1 as follows.

<Equation 1>

As such, the operation of the artificial neural network may include a multiplication operation of multiplying the output value of a neuron of a previous layer and a weight of a synapse, and an addition operation of adding the result values of each multiplication in the receiving neuron.

3 to 5 are diagrams for explaining a configuration of a synaptic array circuit of a neuromorphic device according to some embodiments.

Referring to FIG. 3 , the configuration of the array circuit of the neuromorphic device 200 is illustrated in two dimensions. The neuromorphic device 200 includes axon circuits 210 , synaptic circuits 220 , dendrite circuits 230 , neuronal circuits 240 , and a network 260 . The neuromorphic device 200 has first direction lines (or axon lines) extending in a first direction from the axon circuits 210 and a second direction in a second direction corresponding to the dendrite circuits 230 . lines (or dendrite lines). The first direction lines and the second direction lines may cross each other, and synaptic circuits 220 may be disposed on the intersection points of the first direction lines and the second direction lines. Meanwhile, referring to FIG. 3 , an example in which the first direction is a row direction and the second direction is a column direction is illustrated, but is not limited thereto.

Each of the axon circuits 210 may refer to a circuit that mimics the axon of a biological neuron. The axon of a neuron performs a function of transmitting signals from a neuron to another neuron, and each of the axon circuits 210 simulating the axon of a neuron may receive an activation and transmit it to lines in the first direction. Activation corresponds to a neurotransmitter transmitted through a neuron, and may refer to an electrical signal input to each of the axon circuits 210 . Meanwhile, each of the axon circuits 210 may include a memory or a register for storing input information.

Each of the synaptic circuits 220 may refer to a circuit that mimics a neuron and a synapse between the neurons. The synaptic circuits 220 may store weights corresponding to the strength of the neuron and the connection between the neurons. Each of the synaptic circuits 220 may include a memory element for storing the weight or may be connected to a memory element storing the weight. Here, such a memory device may correspond to a memristor.

The dendrite circuits 230 may refer to circuits simulating dendrites (ie, dendrites) of neurons. The dendrite circuits 230 may perform a function of receiving a signal from another external neuron, and may provide the result of calculation of weight and activation to each of the neuron circuits 240 through the second direction line. Each of the neuron circuits 240 may determine whether to output a spike based on operation results received through the corresponding second direction line. For example, each of the neuron circuits 240 may output a spike when the accumulated value of the received operation results is equal to or greater than a preset threshold value. The spikes output from the neuron circuits 240 may correspond to activations input to the axons of the next stage through the network 260 .

Neuron circuits 240 are located at the rear end with respect to the synaptic circuits 220 , and may be referred to as post-synaptic neuron circuits, and the axon circuits 21 are front end with respect to the synaptic circuits 220 . As it is located in , it may be referred to as pre-synaptic neuron circuits.

Meanwhile, each of the synaptic circuits 220 may be implemented as a memory device such as the memristor device 250 . The memristor element 250 is an element in which weights are stored through a memristor-based design and a multiplication operation (ie, AND operation) can be performed at the intersection. The memristor element 250 of each of the synaptic circuits 220 according to the present embodiment is a PRAM (Phase Change Random Access Memory) using a phase change material or an RRAM using a variable resistance material such as a transition metal oxide (Complex Metal Oxide). (It may be implemented as a resistive element such as a resistive random access memory. Materials constituting the resistive elements have their resistance values variable according to the magnitude and/or direction of the current or voltage, and maintain their resistance value even when the current or voltage is blocked. It has non-volatile properties that remain intact.

Referring to FIG. 4 , a synaptic array according to some embodiments may include a plurality of input lines IL, a plurality of bit lines BL, and a plurality of cell strings disposed between each input line and the bit line. . For convenience of description, a synaptic array including three bit lines and three input lines is described. However, according to various embodiments, a larger number of bit lines and input lines may be included.

The input line IL and the bit line BL of the synaptic array are orthogonal to each other while being spaced apart in the Z direction. For example, the input line IL may extend in the X direction, and the bit line may extend in the Y direction. The input line IL may be a line to which an axon circuit is connected, and the bit line BL may be a line to which a dendrite circuit is connected.

A plurality of cell strings may be connected in parallel to each of the bit lines BL1 to BL3. The plurality of cell strings may be connected to input lines IL1 to IL3 independent of each other. That is, each of the plurality of cell strings may be connected to one input line (eg, IL1) and one bit line (eg, BL1).

Each of the plurality of input lines IL1 - IL3 may extend in the X direction and be spaced apart from each other in the Y direction to be parallel to each other. An input signal may be applied to each of the input lines IL1-IL3 independently of each other. According to some embodiments, the axon circuit 210 may be implemented as input switches SW and 310 and may be connected to input terminals of respective input lines IL1-IL3 to apply input signals Vin1-Vin3. According to some embodiments, the axon circuit 210 may be implemented as a memory or a register to store and output the input signals Vin1-Vin3.

For example, each of the cell strings includes a string select transistor connected to the bit line BL, a plurality of memristor elements 320 connected in series between the string select transistor and the input line IL, and an input line BL. can do. Each of the memristor elements 320 may include a word line transistor and a weight strogae element. One cell string is vertically stacked to reduce overhead by sharing and using one neuron circuit. The cell string will be described in detail with reference to FIGS. 6A to 6D .

In some embodiments, the synaptic array operates in units of layers. The neuromorphic device can learn by performing artificial neural network calculations independently for each layer.

One layer may include memristor elements disposed on the same plane, activated according to one word line WL in the synaptic array. Each layer may include a different number of memristor elements as described with reference to FIG. 1 . The number of memristor elements included in one layer may vary according to the control of the decoder 370 and the peripheral circuit 360 . For example, the first layer may include a 3×3 memristor device, and the second layer may include a 2×3 memristor device.

Each of the bit lines BL1 - BL2 outputs an output signal Iout to the neuron circuits 340 . According to some embodiments, one neuron circuit 340 may be connected to each one bit line BL.

The neuron circuit 340 may convert the output signal Iout from a current value to a voltage value Vout and transmit it to the peripheral circuit 360 .

The decoder 370 may activate at least one word line among the plurality of word lines according to the word line selection signals WL1-WL3 and the string selection signal Select to activate the memristor device of a specific address. In this case, a pass voltage Vpass or an off voltage Voff may be applied to the word line.

Referring to FIG. 5 , when describing the operation in more detail, the peripheral circuit 460 may include at least two transistors 461 and 462 . The first and second transistors 461 and 462 are connected between the peripheral circuit 460 and the bit line BL, the first transistor 461 is on the input side of the synaptic array, and the second transistor 462 is synaptic. It may be connected to the output side of the array.

The peripheral circuit 460 first outputs the input signal Vin to the axon circuit 410 . When the input signal Vin is applied, the training pulse generating circuit 401 generates a training pulse based on a control signal according to whether to perform learning. The first transistor 461 is connected between the training pulse generating circuit 401 and the bit line BL, and outputs a training pulse to the bit line under the control of the peripheral circuit 461 .

When at least one string X is selected according to the string selection signal Select, the off voltage Off is applied to the at least one word line selected through the decoder 470 according to the control signal CON, and is not selected. A pass voltage Vpass may be applied to the remaining word lines that are not.

For example, when the word line WL3 is selected, the off voltage Off is applied to the transistor of the word line WL3, and the pass voltage Vpass is applied to the word lines WL1, WL2, and WL4. The synaptic array performs artificial neural network operation in the layer corresponding to the word line WL3. For example, the memristor element 450 of the word line WL3 may multiply the input signal Vin by a weight and output it as the output signal Iout.

In the bit line BL, the output signals Iout output from each string may be summed. When the second transistor 462 receives the summed output signal Itot, and is turned on under the control of the peripheral circuit 460 , the second transistor 462 outputs the summed output signal Itot to the neuron circuit 440 . When the summed output signal Itot exceeds a preset threshold value, the neuron circuit 440 may convert it into an output voltage Vout and output it to the peripheral circuit 460 .

6A to 6D are diagrams illustrating one string of a synaptic array according to some embodiments.

Referring to FIG. 6A , one cell string X may include a string selection transistor and at least two memristor elements 320 connected in series between the bit line BL and the input line IL. The memristor device includes one word line select transistor and a resistive memory cell. In the cell string X, the memristor element corresponding to each word line may be included in different layers.

Referring to FIG. 6B , electrode pads extending planarly in the X-Y direction may be stacked between the bit lines BL and the input lines IL that are spaced apart along the Z-axis and cross each other orthogonally.

The electrode pads may be stacked to be spaced apart from each other by a predetermined interval in the Z direction, and an inter-electrode insulating layer ILD may be interposed between the electrode pads WL1 - WL4 . The inter-electrode insulating layer may include, for example, silicon oxide, but is not limited thereto. Each of the electrode pads may include word lines WL1-WL3 of memory cells. Also, although not shown, each of the electrode pads may include a gate electrode included in the string selection transistor described with reference to FIGS. 4 to 6A and an output terminal of an axon circuit.

The memristor elements connected in series may be formed as a through-memristor structure. The through-memristor structure has a columnar shape extending along the Z-axis, and is disposed to pass through the stacked electrode pads to connect one bit line BL and one input line IL.

6C and 6D , the memristor element Y includes a control gate electrode 510 , a gate insulating layer 520 , a poly-silicon oxide 530 , a resistive material or a phase-change material 540 , and a pillar oxide. (Pillar Oxide, 550).

The memristor element Y has a pillar shape surrounded by a resistive material 540 , a poly-silicon oxide 530 , a gate insulating layer 520 , and a control gate electrode 510 in the order of the pillar oxide 550 . can

6B to 6D , the control gate electrode 510 may be an electrode pad. The control gate electrode 510 includes platinum (Pt), ruthenium (Ru), iridium (Ir), silver (Ag), aluminum (Al), titanium (Ti), tantalum (Ta), tungsten (W), and silicon (Si). ), copper (Cu), nickel (Ni), cobalt (Co), a conductive nitride thereof, for example, TiN, or a combination thereof, for example, a Ti/TiN film. Preferably, the control gate electrode 510 may be TiN compatible with Si CMOS based processes.

According to some embodiments, the memristor elements Y may include at least one of materials having a characteristic (ie, variable resistance characteristic) whose resistance can be selectively changed by a current passing through the memristor element. . According to some embodiments, the resistive material 540 may be a transition metal oxide film, for example, any one or a combination of TiO2, NiO, HfO2, Al2O3, ZrO2, and ZnO layers. Alternatively, it may be a material capable of variable switching of resistance, such as a chalcogen-based compound film or a perovskite-based compound film. A material having a large ratio of the resistance value in the low-resistance state to the resistance value in the high-resistance state and a small driving voltage can be selected to reduce power consumption.

In the illustrated example, the resistive material 540 and the control gate electrode 510 are illustrated as a single layer, but may have a multilayer structure including a barrier layer suitable for other embodiments.

7 is a diagram for describing an operation when any one layer is selected in a synaptic array according to some embodiments.

Referring to FIG. 7 , a current path is formed in a direction from an input line IL to a bit line BL, and an off voltage OFF is applied to any one of the electrode pads 511 to 514 , , when the pass voltage Vpass is applied to the remaining electrode pads 511 , 512 , and 514 , the current flows to the channel layer, that is, the polysilicon oxide 530 , and in the layer corresponding to the electrode pad 513 , the resistance due to the off voltage It may be diverted to the material 540 and then flow back to the polysilicon oxide 530 at the next electrode pad 514 .

When the current flows through the resistive material 540 for the selected word line, the resistance value changes according to the degree of passing through the resistive material 540, and the output current Iout according to the changed resistance value compared to the input voltage Vin. can be determined. In the cell string, the resistive memory devices are connected in series, and the output current Iout may be summed in the neuron circuit according to the linear characteristic of the current with respect to the resistive component.

8 to 9 are conceptual diagrams for explaining the operation of a neuromorphic device according to some embodiments.

8 and 9 , the neuromorphic device 600 may include at least two synaptic arrays 621 and 622 . The at least two synaptic arrays 621 and 622 may include an axon circuit 611 and a neuron circuit 641 , respectively, and may be serially connected between the synaptic arrays 621 and 622 .

Each layer of the synaptic arrays 621 and 622 may perform an artificial neural network operation independent of each other. According to some embodiments, the first layer corresponding to the word line WL1 of the synaptic arrays 621 and 622 is an artificial neural network operation for face recognition, and the second layer corresponding to the word line WL2 is an artificial neural network operation for cryptographic analysis, a word The third layer corresponding to the line WL3 may perform an artificial neural network operation for trend inference, and the fourth layer corresponding to the word line WL4 may perform an artificial neural network operation for pattern recognition.

Although the first synaptic array 621 and the second synaptic array 622 implemented as a 4x4 array according to some embodiments are described in FIG. 9 , they may be implemented as at least a 2x2 array according to various embodiments.

The decoder 630 is respectively connected to the first synaptic array 621 and the second synaptic array 622, selects any one synaptic array, and applies a string selection signal and a word line voltage of the selected synaptic array. .

When the input signal Vin1, that is, Vin11, Vin12, Vin13 is input, it is input to the first synaptic array 621 through the first axon circuit 611, and the decoder 630 receives any one according to the input signal Vin1. An artificial neural network operation can be performed by selecting a layer of (Array4 in the example shown). As a result of the artificial neural network operation, when the output signals Iout1, Iout2, and Iout3 are output from the first synaptic array 621, the input signals Vin2, that is, Vin21, Vin22 in the first neuron circuit 641, that is, 6411, 6412, 6413. , Vin23). The input signal Vin2 may be output when the output signals Iout1, Iout2, and Iout3 summed from the dendrite, that is, the bit line, exceed a preset threshold.

The input signal Vin2 is input to the second synaptic array 622 through the second axon circuit 612, and the decoder 630 selects any one layer (Array4 in the example shown) according to the input signal Vin2. You can select to perform artificial neural network calculations. As a result of the artificial neural network operation, when the output signals Iout2, that is, Iout21, Iout22, and Iout23 are output from the second synaptic array 622, the output signals Vout21 and Vout22 from the second neuron circuit 642, that is, 6421, 6422, 6423. , Vout23). The output signals Vout21, Vout22, and Vout23 may be output when the dendrite, that is, the output signals Iout21, Iout22, and Iout23 summed from the bit lines, respectively, exceeds a predetermined threshold.

According to some embodiments, the neuromorphic device 600 may perform an artificial neural network operation by activating the same layer of a plurality of synaptic arrays. That is, by activating Array4 of the first synaptic array 621 and Array4 of the second synaptic array 622 through the decoder 630 , an artificial neural network operation may be performed to output an output signal. In this case, Array4 of the first synaptic array 621 , Array4 of the second synaptic array 622 may have the same NxM array (N and M are each a natural number equal to or greater than 2).

Array1 to Array4 of the first and second synaptic arrays 621 and 622 may be arrays having sizes independent of each other for artificial neural network operation. 6A, Array 1 may be a 3x3 array, Array 2 may be a 2x3 array, Array3 may be a 4x4 array, and Array4 may be a 2x2 array, as described with reference to FIG. 6A.

10 to 11 are conceptual diagrams for explaining an operation of a neuromorphic device according to some embodiments.

10 and 11 , according to some embodiments, the neuromorphic device 700 includes an axon circuit 710 , a single synaptic array 720 , a decoder 730 , a neuron circuit 740 and a latch circuit ( 750) may be included.

Unlike the embodiment of FIGS. 8 and 9 , the neuromorphic device 700 may perform an artificial neural network operation using one synaptic array 720 as shown in FIG. 10( b ). In this case, the operation result in the previous layer is stored in the latch circuit 750, and when the next artificial neural network learning is required, the value stored in the latch circuit 750 is input to the synaptic array 720, and the next artificial neural network operation can be performed. .

Each layer of the synaptic array 720 may perform an artificial neural network operation linked to each other. According to some embodiments, the first layer corresponding to the word line WL1 of the synaptic arrays 621 and 622 to the fourth layer corresponding to the word line WL4 of the synaptic arrays 621 and 622 are values stored in the latch circuit 750 for artificial neural network operation for face recognition according to some embodiments. You can learn by receiving the input and performing artificial neural network calculations for face recognition. In other words, read disturb is minimized by performing artificial neural network operations with different layers in one synaptic array, and when there are a plurality of synaptic arrays, the degree of freedom of connection for connecting the arrays can be improved.

10A and 10B, when an input signal Vin1 is input to the neuromorphic device 700, it is input to the synaptic array 720 through the axon circuit 710, and the decoder 730 may select any one layer (eg, a layer corresponding to WL1) according to the input signal Vin1 to perform an artificial neural network operation. As a result of the artificial neural network operation, when the output signal Iout1 is output from the synaptic array 720 , the neuron circuit 740 may change the output signal from the current Iout1 to the voltage signal Vout1 and output it. The converted output signal Vout1 may be stored in the latch circuit 750 , and may be ^{called during the next artificial neural network operation 2 nd} , and may be input as an input signal Vin2 to a layer corresponding to WL2 . Similarly, the third artificial neural network operation (3 ^rd ) and the fourth artificial neural network operation (4 ^th ) also receive the previous operation results (Vin3, Vin4) and store them in the latch circuit 750 (ie, 7501, 7502, 7503) and then sequentially The output signal Vout may be output through the neuron circuit 740 (ie, 7401 , 7402 , 7403 ) by input to another layer (a layer corresponding to WL3 and a layer corresponding to WL4 in the illustrated example).

12 and 13 are conceptual diagrams for explaining an operation of each layer in a synaptic array according to some embodiments. Assume that artificial neural network learning is performed four times according to some embodiments.

12 , when the neuromorphic device 600 includes a plurality of synaptic arrays according to some embodiments, the synaptic arrays may perform an artificial neural network operation independent of each other for each layer corresponding to each word line. Four synaptic arrays are serially connected to each synaptic array for learning 4 times, and sequentially the first artificial neural network operation, the second in the layer for the same word line (Array 4 in the example shown) An artificial neural network operation, a third artificial neural network operation, and a fourth artificial neural network operation may be performed, and an output signal Vout may be output.

In FIG. 13 , when the neuromorphic device 700 includes one synaptic array according to some embodiments, all layers of the synaptic array may perform the same artificial neural network operation. For the fourth artificial neural network learning, the synaptic array is sequentially activated for each layer, and the first artificial neural network operation in the first layer (Array 1), the second artificial neural network operation in the second layer (Array 2), and the third layer (Array) The third artificial neural network operation in 3) and the fourth artificial neural network operation in the fourth layer (Array 4) may be performed, and the output signal Vout may be output.

14 is a block diagram illustrating a hardware configuration of a neuromorphic device according to an embodiment.

Referring to FIG. 14 , the neuromorphic device 100 includes a neuromorphic chip 110 including a neuromorphic processor 112 and an on-chip memory 114 , and an external memory 120 . However, only the components related to the present embodiments are shown in the neuromorphic device 100 shown in FIG. 14 . Accordingly, the neuromorphic device 100 includes other general-purpose components in addition to the components shown in FIG. 14 , for example, a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), and a sensor. It is apparent to those skilled in the art that a module, a communication module, and the like may be further included.

The neuromorphic apparatus 100 may be an apparatus included in various types of electronic devices such as a personal computer (PC), a server device, a mobile device, and an embedded device. The neuromorphic apparatus 100 is, for example, a smart phone, a tablet device, an augmented reality (AR) device, an Internet of Things (IoT) device, and an autonomous device that performs voice recognition, image recognition, and image classification using a neural network. It may correspond to a hardware configuration included in a driving vehicle, robotics, medical device, and the like. That is, the neuromorphic apparatus 100 may correspond to a dedicated hardware accelerator mounted on the electronic device as described above, and the neuromorphic apparatus 100 is a neural processing (NPU) dedicated module for driving a neural network. unit), a Tensor Processing Unit (TPU), or a hardware accelerator such as a Neural Engine, but is not limited thereto.

The neuromorphic chip 110 serves to control overall functions for driving the neural network in the neuromorphic device 100 . For example, the neuromorphic processor 112 of the neuromorphic chip 110 may store neuromorphic data (eg, an axon value, a synaptic value, etc.) stored in the external memory 120 in the neuromorphic device 100 . ) to execute neuromorphic-related programs, thereby controlling the neuromorphic device 100 as a whole. The neuromorphic processor 112 may include a synaptic array according to some embodiments illustrated in FIGS. 1 to 13 . The neuromorphic chip 110 may drive a neural network under the control of a CPU, GPU, or AP provided inside or outside the neuromorphic device 100 .

The external memory 120 is hardware for storing various neuromorphic data processed in the neuromorphic chip 110 , and the external memory 120 includes data processed in the neuromorphic chip 110 and data to be processed. can be saved In addition, the external memory 120 may store applications, drivers, etc. to be driven by the neuromorphic chip 110 . The external memory 120 is a random access memory (RAM), such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD - may include ROM, Blu-ray or other optical disk storage, a hard disk drive (HDD), a solid state drive (SSD), or flash memory.

The on-chip memory 114 of the neuromorphic chip 110 reads and stores (or buffers) neuromorphic data for pre-synaptic neuron circuits, for example, axon values and synaptic values from the external memory 120 . and executes neural network, that is, artificial neural network operation using the stored neuromorphic data. In addition, the on-chip memory 114 may store data for post-synaptic neuron circuits, such as neuron values and spike values generated as a result of execution of the neural network.

15 is a block diagram illustrating a configuration of an electronic system according to some embodiments.

Referring to FIG. 15 , the electronic system 1000 analyzes input data in real time based on a neural network to extract valid information, makes a situation determination based on the extracted information, or an electronic system in which the electronic system 1000 is mounted. It is possible to control the configurations of the device. For example, the electronic system 1000 may include a drone, a robot device such as an Advanced Drivers Assistance System (ADAS), a smart TV, a smartphone, a medical device, a mobile device, an image display device, a measurement device, and an IoT device. and the like, and may be mounted on at least one of various types of electronic devices.

The electronic system 1000 may include a processor 1010 , a RAM 1020 , a neuromorphic device 1030 , a memory 1040 , a sensor module 1050 , and a communication module 1060 . The electronic system 1000 may further include an input/output module, a security module, a power control device, and the like. Some of the hardware components of the electronic system 1000 may be mounted on at least one semiconductor chip.

The processor 1010 controls the overall operation of the electronic system 1200 . The processor 1010 may include one processor core (Single Core) or a plurality of processor cores (Multi-Core). The processor 1010 may process or execute programs and/or data stored in the memory 1040 . In some embodiments, the processor 1010 may control the function of the neuromorphic device 1030 by executing programs stored in the memory 1040 . The processor 1010 may be implemented as a CPU, GPU, AP, or the like.

The RAM 1020 may temporarily store programs, data, or instructions. For example, programs and/or data stored in the memory 1040 may be temporarily stored in the RAM 1020 according to a control or boot code of the processor 1010 . The RAM 1020 may be implemented as a memory such as dynamic RAM (DRAM) or static RAM (SRAM).

The neuromorphic device 1030 may perform an operation of the neural network based on received input data, and may generate an information signal based on the result of the operation. Neural networks may include, but are not limited to, CNNs, RNNs, FNNs, Deep Belief Networks, Restricted Boltzman Machines, and the like. The neuromorphic device 1030 is a hardware accelerator dedicated to a neural network or a device including the same, and may include the neuromorphic processor described above.

The information signal may include one of various types of recognition signals such as a voice recognition signal, an object recognition signal, an image recognition signal, and a biometric information recognition signal. For example, the neural network device 1030 may receive frame data included in a video stream as input data, and generate a recognition signal for an object included in an image indicated by the frame data from the frame data. However, the present invention is not limited thereto, and the neuromorphic device 1030 may receive various types of input data according to the type or function of the electronic device on which the electronic system 1000 is mounted, and may receive a recognition signal according to the input data. can create

The memory 1040 is a storage location for storing data, and may store an operating system (OS), various programs, and various data. In an embodiment, the memory 1040 may store intermediate results generated in the process of performing an operation of the neuromorphic device 1030 .

The memory 1040 may be a DRAM, but is not limited thereto. The memory 1040 may include at least one of a volatile memory and a nonvolatile memory. Nonvolatile memory includes ROM, PROM, EPROM, EEPROM, flash memory, PRAM, MRAM, RRAM, FRAM, and the like. Volatile memory includes DRAM, SRAM, SDRAM, PRAM, MRAM, RRAM, FeRAM, and the like. In an embodiment, the memory 1040 may include at least one of HDD, SSD, CF, SD, Micro-SD, Mini-SD, xD, and Memory Stick.

The sensor module 1050 may collect information around the electronic device on which the electronic system 1000 is mounted. The sensor module 1050 may sense or receive a signal (eg, an image signal, an audio signal, a magnetic signal, a biosignal, a touch signal, etc.) from the outside of the electronic device, and may convert the sensed or received signal into data. To this end, the sensor module 1050 includes various types of sensing devices, such as a microphone, an imaging device, an image sensor, a light detection and ranging (LIDAR) sensor, an ultrasonic sensor, an infrared sensor, a biosensor, and a touch sensor. may include at least one of

The sensor module 1050 may provide the converted data as input data to the neuromorphic device 1030 . For example, the sensor module 1050 may include an image sensor, generate a video stream by photographing an external environment of the electronic device, and input data to the neuromorphic device 1030 with successive data frames of the video stream. can be provided in order. However, the present invention is not limited thereto, and the sensor module 1050 may provide various types of data to the neuromorphic device 1030 .

The communication module 1060 may include various wired or wireless interfaces capable of communicating with an external device. For example, the communication module 1060 may include a wired local area network (LAN), a wireless local area network (WLAN) such as Wi-Fi (Wireless Fidelity), and a wireless personal network such as Bluetooth (Bluetooth). Personal Area Network (WPAN), Wireless USB (Wireless Universal Serial Bus), Zigbee, NFC (Near Field Communication), RFID (Radio-frequency identification), PLC (Power Line communication), or 3G (3rd Generation), 4G (4th Generation), and a communication interface connectable to a mobile cellular network, such as LTE (Long Term Evolution), and the like.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above embodiments, but may be manufactured in various different forms, and those of ordinary skill in the art to which the present invention pertains. It will be understood by those skilled in the art that the present invention may be embodied in other specific forms without changing the technical spirit or essential features of the present invention. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: neuromorphic device 110: neuromorphic chip
112: processor 114: on-chip memory
120: external memory

Claims (10)

an on-chip memory that reads and stores neuromorphic data;
A neuromorphic processor that includes at least one synaptic array and executes an artificial neural network operation according to the neuromorphic data in the synaptic array,
The synaptic array is
a plurality of input lines respectively connected to a plurality of axon circuits to independently receive an input signal based on the neuromorphic data;
a plurality of bit lines spaced apart from the plurality of input lines and each outputting an output current according to the artificial neural network operation;
a plurality of neuron circuits connected to each of the plurality of bit lines, summing the output currents, and outputting the summed output currents as an output voltage when the summed output current exceeds a preset threshold;
a plurality of electrode pads stacked at a predetermined interval between the plurality of input lines and the plurality of bit lines;
a plurality of through-memristor structures each passing through the plurality of electrode pads and connected between any one of the input line and the bit line to form a current path of the output current;
a plurality of string select transistors, each of which is disposed between the plurality of memristor through structures and the plurality of bit lines; and
and a decoder connected to each of the plurality of electrode pads to apply a word line selection signal and a string selection signal. The method of claim 1, wherein each memristor through structure
A neuromorphic device having a columnar shape surrounded by layers of a resistive material layer, a polysilicon oxide layer, and a gate insulating layer with a pillar oxide as the center. 3. The method of claim 2,
The word line selection signal includes a pass voltage and an off voltage,
The memristor penetrating structure is
In a region corresponding to the electrode pad to which the pass voltage is applied, the current path is formed of a polysilicon oxide layer, and in a region corresponding to the electrode pad to which the off voltage is applied, the current path is formed by the resistive material layer. which outputs the output current to a bit line, a neuromorphic device The method of claim 1, wherein the synaptic array is
at least two layers including at least two memristor elements corresponding to respective electrode pads of the plurality of electrode pads among the through-memristor structure;
the layer is
The number of the memristor elements included in the layer is changed according to the string selection signal, the neuromorphic device. According to claim 1,
The first synaptic array and the second synaptic array are connected in series,
The first synaptic array and the second synaptic array are spaced apart in a first direction or a second direction to share the plurality of electrode pads for each layer. 6. The method of claim 5,
A first artificial neural network operation is performed in a first layer of the first synaptic array according to the input signal, and a second artificial neural network is performed in a second layer of the second synaptic array according to an intermediate signal output from the first synaptic array. generating the output signal by performing an operation,
wherein the first layer and the second layer are activated according to the same word line selection signal. The method of claim 1, wherein the synaptic array is
and a plurality of latch circuits each connected between the plurality of neuron circuits and the plurality of input lines to store the output voltage. The method of claim 7, wherein the neuromorphic device is
A first artificial neural network operation is performed in the first layer of the synaptic array according to the input signal, the intermediate signal output from the neuron circuit is stored in the latch circuit, and the second layer of the synaptic array is stored according to the stored intermediate signal. To perform a second artificial neural network operation in the to output the output signal from the neuron circuit, a neuromorphic device. In the neuromorphic device,
The neuromorphic device comprises at least one three-dimensional synaptic array,
The neuromorphic device is
When receiving a plurality of input signals independent of each other,
performing an artificial neural network operation on the input signal in a plurality of resistive memristor elements of a layer corresponding to the word line selection signal;
Outputting a plurality of output currents as a result of the operation to the neuron circuit through the bit line,
When the sum of the output currents exceeds a preset threshold, output as an output voltage,
The three-dimensional synaptic array is
A method of operating a neuromorphic device, including a plurality of layers, and performing an artificial neural network operation independent of each other for each layer activated according to the word line selection signal. 10. The method of claim 9, wherein the neuromorphic device is
Electrically connected in series, including a first synaptic array and a second synaptic array spaced apart from each other,
outputting an intermediate signal obtained by performing a first artificial neural network operation in a first layer of the first synaptic array according to the input signal to the second synaptic array;
generating the output current by performing a second artificial neural network operation in the second layer of the second synaptic array according to the intermediate signal;
The method of claim 1, wherein the first layer and the second layer are activated according to the same word line selection signal.

Images