KR20210144417A - Apparatus for performing in memory processing and computing apparatus having the same - Google Patents

Abstract

An apparatus for performing in-memory processing comprises: a memory cell array providing a current sum when input signals are applied; a capacitor charging a sampling voltage corresponding to the current sum; and a processing circuit performing an operation for comparing between a currently charged voltage in the capacitor and a reference voltage and determining a quantization level corresponding to the sampling voltage by performing time-digital converting at a time when the currently charged voltage reaches the reference voltage.

Description

Apparatus for performing in-memory processing and computing device including the same

It relates to an apparatus for performing in-memory processing and a computing device including the same, and more particularly, to an apparatus for performing in-memory processing based on time-digital conversion and a computing device including the same.

A neural network is a computing system implemented with reference to a computational architecture. With the recent development of neural network technology, research on analyzing input data and extracting valid information using a neural network in various types of electronic systems is being actively conducted. The processing of neural networks requires a large amount of computation on complex input data. As the data of the neural network increases and the connectivity of the architecture constituting the neural network becomes complicated, the amount of computation and the memory access frequency of the processing device are excessively increased, resulting in inefficient performance in miniaturization and commercialization. For example, the processing of the neural network may include a multiply-accumulate (MAC) operation that repeats multiplication and addition. Various attempts have been made on efficient hardware architectures and hardware driving methods to process repetitive MAC operations that occupy a large amount of computation in neural network processing with low power and high speed.

An object of the present invention is to provide an apparatus for performing in-memory processing and a computing device including the same. The technical problem to be achieved by the present embodiment is not limited to the technical problems as described above, and other technical problems may be inferred from the following embodiments.

According to one aspect, an apparatus for performing in-memory processing includes a sum of currents of column currents flowing through each column line when input signals are applied through each row line. a memory cell array comprising a plurality of memory cells providing a current sum; a sampling circuit including a capacitor for charging a sampling voltage corresponding to the sum of the currents of each of the column lines; and a comparison operation between a voltage currently charged in the capacitor and a reference voltage is started by application of a trigger pulse generated at a timing corresponding to a predetermined quantization level among quantization levels divided according to the charging time of the capacitor, and the current and a processing circuit configured to determine a quantization level corresponding to the sampling voltage by performing time-digital conversion when the charged voltage reaches the reference voltage.

According to another aspect, a computing device includes: a host processor; memory device; and an in-memory processing device, wherein the in-memory processing device is a current sum of column currents flowing through each column line when input signals are applied through each row line. ), providing a memory cell array comprising a plurality of memory cells; a sampling circuit including a capacitor for charging a sampling voltage corresponding to the sum of the currents of each of the column lines; and a comparison operation between a voltage currently charged in the capacitor and a reference voltage is started by application of a trigger pulse generated at a timing corresponding to a predetermined quantization level among quantization levels divided according to the charging time of the capacitor, and the current and a processing circuit configured to determine a quantization level corresponding to the sampling voltage by performing time-digital conversion when the charged voltage reaches the reference voltage.

According to another aspect, a method of performing in-memory processing includes: applying an input signal to a plurality of memory cells through each row line of a memory cell array; charging a sampling voltage corresponding to a current sum of column currents flowing through each column line of the memory cell array into a capacitor connected to each column line; performing a comparison operation between a voltage currently charged in the capacitor and a reference voltage by applying a trigger pulse generated at a timing corresponding to a predetermined quantization level among quantization levels divided according to a charging time of the capacitor to comparators; and determining a quantization level corresponding to the sampling voltage by performing time-digital conversion when the currently charged voltage reaches the reference voltage.

1 is a diagram for explaining a biological neuron and its operation.
2 is a diagram for explaining the configuration of a two-dimensional array circuit for performing a neuromorphic operation.
3 is a diagram for explaining a method of processing a neuromorphic operation.
4 is a diagram illustrating an in-memory processing device according to an embodiment.
5 is a diagram for describing a mapping relationship between a combined resistance value of a certain column line and a sum bit value according to an exemplary embodiment.
6 is a diagram for describing an in-memory processing device according to an embodiment.
7 is a diagram for explaining setting time intervals of quantization levels according to a time when a voltage is charged in a capacitor, according to an embodiment.
8 is a diagram for explaining a timing of applying a trigger pulse TRIGGER to a comparator according to an exemplary embodiment.
9 is a diagram for describing time-digital conversion performed in an in-memory processing device according to an embodiment.
10 is a diagram for explaining outputting a counting value for each column line (output line) through time-digital conversion according to an exemplary embodiment.
11 is a flowchart of a method for performing in-memory processing according to an embodiment.
12 is a block diagram illustrating a computing device according to an embodiment.
13 is a diagram for describing an example of a neural network.
14 is a flowchart of a method for performing in-memory processing according to an embodiment.

Terms used in the embodiments are selected as currently widely used general terms as possible, which may vary according to intentions or precedents of those of ordinary skill in the art, emergence of new technologies, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the corresponding description. Therefore, the terms used in the specification should be defined based on the meaning of the term and the contents of the entire specification, not the simple name of the term.

Terms such as “consisting” or “comprising” used in the present embodiments should not be construed as necessarily including all of the various components or various steps described in the specification, and some components or It should be construed that some steps may not be included, or may further include additional components or steps.

Hereinafter, an embodiment will be described in detail with reference to the accompanying drawings. However, the embodiment may be implemented in several different forms and is not limited to the examples described herein.

1 is a diagram for explaining a biological neuron and its operation.

Referring to FIG. 1 , a biological neuron 10 refers to a cell existing in the human nervous system, and is one of the basic biological calculation entities. The human brain contains about 100 billion biological neurons and 100 trillion interconnections between them.

Biological neuron 10 is a single cell and includes a cell body (neuron cell body) including a cell nucleus (nucleus) and various organelles (organelles). Various organelles contain mitochondria, numerous dendrites radiating from the cell body, and axons terminating in many branching extensions.

In general, an axon performs a function of transmitting signals from one neuron to another neuron, and a dendrite performs a function of receiving a signal from another neuron. For example, when different neurons are connected, a signal transmitted through an axon of a neuron may be received by a dendrite of another neuron. At this time, a signal is transmitted between neurons through a specialized connection called a synapse, and several neurons are connected to each other to form a neural network. A neuron that secretes a neurotransmitter based on a synapse is called a pre-synaptic neuron, and a neuron that receives information transmitted through a neurotransmitter is called a post-synaptic neuron. may be referred to.

On the other hand, the human brain can learn and remember a vast amount of information by transmitting and processing various signals through a neural network formed by connecting a large number of neurons. The vast number of connections between neurons in the human brain directly correlates to the massively parallel nature of biological computing. Various attempts have been made to efficiently process For example, as a computing system designed to implement artificial neural networks at the neuron level, neuromorphic devices are being studied.

Meanwhile, the operation of the biological neuron 10 may be simulated by the mathematical model 11 . The mathematical model 11 corresponding to the biological neuron 10 is an example of a neuromorphic operation, a multiplication operation for multiplying information from a plurality of neurons by a synaptic weight, and values multiplied by the synaptic weight (ω ₀ x ₀₎ , ω ₁ x ₁ , ω ₂ x ₂ ), and an operation of applying a characteristic function (b) and an activation function (f) to the result of the addition operation. A neuromorphic operation result may be provided by the neuromorphic operation. Here, values of x ₀ , x ₁ , x ₂ , ... etc. may be referred to as axon values, and values of ω ₀ , ω ₁ , ω ₂ , ... etc. may be referred to as synaptic weights.

2 is a diagram for explaining the configuration of a two-dimensional array circuit for performing a neuromorphic operation.

Referring to FIG. 2 , the configuration 20 of the two-dimensional array circuit consists of N (N is an arbitrary natural number) axon circuits (A ₁ to A _N ) 210 , M (M is an arbitrary natural number) neuron circuits (N ₁ to N _M ) 230 and N x M synaptic arrays (S ₁₁ to S _NM ) 220 .

Each synapse of the synapse array (S ₁₁ to S _NM ) 220 has first direction lines and neuron circuits N ₁ to N _{extending in the first direction from the axon circuits A 1} to A _{N 210 . M} ) 230 may be disposed at intersections of lines in the second direction extending in the second direction. Here, for convenience of description, it is illustrated that the first direction is a row direction and the second direction is a column direction, but the present invention is not limited thereto, and the first direction may be a column direction and the second direction may be a row direction.

Each of the axon circuits A ₁ to A _N 210 may refer to a circuit that mimics the axon of a biological neuron ( 10 in FIG. 1 ). Since the axon of a neuron performs the function of transmitting signals from a neuron to another neuron, each of the axon circuits (A ₁ to A _N ) 210 mimicking the axon of a neuron is activated (eg, axons a ₁ , a ₂ , ..., a _N ) may be received and transmitted to lines in the first direction. Activation corresponds to a neurotransmitter transmitted through a neuron, and may mean an electrical signal input to each of the _{axon circuits A 1} to A _{N 210 .} Meanwhile, each of the axon circuits A ₁ to A _N 210 may include a memory, a register, or a buffer for storing input information. Meanwhile, the activation may be a binary activation having a binary value. For example, binary activation may include 1-bit information corresponding to a logical value 0 or 1 (or a logical value -1 or 1). However, the present invention is not necessarily limited thereto, and the activation may have a ternary value or a multi-bit value.

Each synapse of the synapse array (S ₁₁ to S _NM ) 220 may refer to a circuit that mimics a neuron and a synapse between the neurons. The synaptic arrays S ₁₁ to S _NM 220 may store synaptic weights corresponding to the strength of a neuron-to-neuron connection. 2 , as examples of synaptic weights to be stored in each synapse, w ₁ , w ₂ , ..., w _M are shown in FIG. 2 , but other synaptic weights may be stored in each synapse. Each synapse of the synapse array (S ₁₁ to S _NM ) 220 may include a memory element for storing the synaptic weight or may be connected to another memory element storing the synaptic weight. Here, such a memory device may correspond to, for example, a memristor or a resistive memory cell. Memristor or resistive memory cells include static random-access memory (SRAM), phase change memory (PCM), oxide-based memory (OXRAM), magnetoresistive random-access memory (MRAM), and spin-transfer torque random-access (STT-RAM). Memory), Conductive-Bridge Random-Access Memory (CBRAM), Resistive RAM (RRAM), Ferroelectric RAM (FRAM), Magnetic Tunnel Junction (MTJ) devices, etc., but are not limited thereto.

Each of the synaptic arrays (S ₁₁ to S _NM ) 220 may receive an activation input transmitted from each _{of the axon circuits A 1} to A _N 210 through a corresponding first direction line, and the stored The result of neuromorphic operation between the synaptic weight and the activation input can be output. For example, the neuromorphic operation between the synaptic weight and the activation input may be a multiplication operation (ie, an AND operation), but is not limited thereto. That is, the result of the neuromorphic operation between the synaptic weight and the activation input may be a value obtained by any other suitable operation for simulating the intensity or size of activation adjusted according to the strength of the connection between the neuron and the neuron.

Depending on the neuromorphic operation between the synaptic weight and the activation input, the magnitude or intensity of a signal transmitted _{from the axon circuits (A 1} to A _N ) 210 to the neuron circuits (N ₁ to N _{M ) 230 is adjusted.} can In this way, using the synaptic arrays (S ₁₁ to S _NM ) 220 , an operation in which the size or intensity of a signal transmitted to the next neuron is adjusted according to the connection strength between the neuron and the neuron can be simulated.

Each of the neuron circuits N ₁ to N _M 230 may refer to a circuit simulating a neuron including a dendrite. A dendrite of a neuron performs a function of receiving a signal from another neuron, and each of the neuronal circuits (N ₁ to N _M ) 230 is a neuromorphic between a synaptic weight and an activation input through a corresponding second direction line. The result of the operation can be received. Each of the neuron circuits N ₁ to N _M 230 may determine whether to output a spike based on a result of a neuromorphic operation. For example, each of the neuron circuits N ₁ to N _M may output a spike when an accumulated value of a neuromorphic operation result is equal to or greater than a preset threshold value. The spikes output from the neuron circuits N ₁ to N _M 230 may correspond to activation input to the axon circuits of the next stage.

Meanwhile, the neuron circuits (N ₁ to N _M ) 230 are located at the rear end with respect to the synaptic array (S _{11 to S NM} ) 220 , and may be referred to as post-synaptic neuron circuits, and may be referred to as axon circuits. The ones (A ₁ to A _N ) 210 are located at the front end with respect to the synaptic array (S _{11 to S NM} ) 220 , and may be referred to as pre-synaptic neuron circuits.

3 is a diagram for explaining a method of processing a neuromorphic operation.

A two-dimensional array circuit for processing a neuromorphic operation may use a current summation method for each column line. For example, the two-dimensional array circuit is a column line ( _{S 11} , S ₂₁ , S _(N-1)1 , S _N1 ) through the synapses (S 11 , S 21 , S (N-1)1 , S N1 ) by activation transmitted from the _{axon circuits (A 1} to A _{N ).} 310), and outputs a spike when the magnitude or intensity of the summed current is greater than or equal to a preset threshold value. In this case, in order to obtain the magnitude or strength of the summed current, it is necessary to provide a peripheral circuit such as an analog to digital converter (ADC), a digital to analog converter (DAC), and the like. However, ADC or DAC can be a source of inefficient overhead in terms of power and area for the overall circuit configuration. Therefore, in the following, a circuit design based on a time-to-digital converter (TDC) instead of an ADC or a DAC for a neuromorphic operation such as a MAC operation that repeats addition and multiplication according to the present embodiments. A method for implementing an on-chip system having a high degree of integration will be described.

4 is a diagram illustrating an in-memory processing device according to an embodiment.

Referring to FIG. 4 , the in-memory processing device 100 may be configured as a circuit that outputs a result of performing multiplication and addition for a neuromorphic operation.

The in-memory processing device 100 may include a plurality of memory cells 110 , a capacitor C, and a time-to-digital converter (TDC) 140 . In addition, the in-memory processing device 100 may further include a comparator 130 for generating time information that is passed to the time digital converter 140 . In FIG. 4 , for convenience of description, only column lines 120 and row lines corresponding to a part of a memory cell array included in the in-memory processing device 100 are illustrated. Accordingly, the in-memory processing device 100 may include a memory cell array in which a plurality of memory cells are disposed at positions where a plurality of column lines and row lines intersect.

As described above, the memory cells 110 are implemented as memristors or resistive memory devices, and may be devices having a variable resistance. _{Voltages V 1} , V ₂ , V ₃ , ..., V _m may be applied to the memory cells 110 through each row line in response to an input signal. For example, an input voltage signal of an input signal may be directly applied to the memory cells 110 , or a supply voltage may be applied by the input signal.

One end of each of the memory cells 110 may be configured to receive a voltage through a switch SW, and the other end of each of the memory cells 110 may be connected to the capacitor C and the comparator 130 . That is, the capacitor C and the comparator 130 are connected to each column line including the memory cells 110 .

A current having a current value calculated based on Ohm's law flows in the column line 120 according to the resistance value of each of the memory cells 110 and the voltage value of the input signal applied to each of the memory cells 110 . _{Accordingly, a current sum I o} of column currents flowing through the column line 120 may correspond to a result value of a MAC operation between corresponding memory cells and input signals.

Each of the input signals (ie, the input voltages V ₁ , V ₂ , V ₃ , ..., V _m ) may be applied to each of the memory cells 110 in response to the start signal START. To this end, the in-memory processing device 100 may include a plurality of switches 101 switched by the start signal START. One end of the switches 101 may be connected to one end of the memory cells 110 . For example, the first switch SW ₁ is connected to one end of the first memory cell R ₁ , ..., the m-th switch SW _m is connected to one end of the _m- th memory cell R m . can Here, m is a natural number greater than or equal to 1. The other end of the switches 101 may be connected to an input signal (ie, input voltages V ₁ , V ₂ , V ₃ , ..., V _{m ).} Meanwhile, the input signal is not always applied to all memory cells 110 , and the input signal may not be applied to some memory cells according to the value of the input signal (input voltage value). In this case, the unapplied input signal may mean a case in which the input voltage is 0, but is not limited thereto and may be a specific voltage value.

The input signals may correspond to individual bit values of an input bit sequence consisting of a series of binary values. Specifically, each of the plurality of row lines in the in-memory processing device 100 may correspond to a respective bit position of the input bit sequence. When the bit value of a certain bit position is 1, an input signal having a voltage value corresponding to the bit value 1 may be applied to a row line corresponding to the corresponding bit position. Alternatively, when the bit value of a certain bit position is 0, an input signal of a voltage value (eg, 0V voltage) corresponding to the bit value 0 may be applied to a row line corresponding to the corresponding bit position.

The resistance value of each of the memory cells 110 may have a bit value (eg, a weight or a synaptic weight) multiplied by each bit of the input bit sequence. Since the memory cells 110 may be implemented as resistive memory devices having a variable resistance, a memory cell corresponding to a bit value of 1 among the memory cells 110 has a first resistance value, and a memory cell corresponding to a bit value of 0 is It may have a second resistance value. Alternatively, without being limited thereto, the memory cells 110 may be implemented in a circuit configuration in which a resistor corresponding to a corresponding bit value is selected from among a plurality of resistors having different resistance values using a switching element.

Meanwhile, in the present embodiment, it is assumed that the bit value is 1 or 0, but it is not limited thereto, and the bit value may be 1 or -1, or other binary bit values or ternary bit values.

The capacitor C is connected to the column line 120 to which the memory cells 110 are connected, and may be charged with a voltage corresponding to the _{sum of currents I o of column currents flowing through the column line 120 .}

In the in-memory processing device 100 , a capacitor is connected for each column line, and the capacitor samples a voltage corresponding to the _{sum of currents I o of the corresponding column line.} Accordingly, capacitors connected for each column line in the in-memory processing device 100 constitute a sampling circuit for charging a sampling voltage _{V o} corresponding to the _{sum of currents I o of the corresponding column line.} can

The comparator 130 may be connected to one end of the capacitor C. The comparator 130 may perform a comparison operation of comparing a current charged voltage with a reference voltage V _{ref while the sampling voltage V o is charged in the capacitor C .} The comparator 130 may be implemented as, for example, an operational amplifier (OP amp). _{A sampling voltage V o} of the capacitor C may be input to one input node of the operational amplifier, and a reference voltage V _ref may be input to another input node.

The sampling voltage (V _{o ) is a voltage corresponding to the sum of currents (I o} ) of the column line 120 connected to the capacitor (C), and corresponds to the MAC operation result between the resistance values of the memory cells 110 and the applied input signals. It may be a corresponding value.

The charging voltage of the capacitor C may be changed over a predetermined time based on a time constant (τ=R*C) determined by the combined resistance value and the capacitance of the column line 120 . Here, the combined resistance value of the column line 120 is a value dependent on _{the sampling voltage Vo of the} column line 120 or the sum of currents I _{o of the column line 120 .} For example, if the sum of currents I _o of the column line 120 is relatively large (or if the sampling voltage V _o is relatively small), since the time constant τ of the capacitor C becomes small, Capacitor C will charge faster. Conversely, if the sum of currents I _o of the column line 120 is relatively small (or if the sampling voltage V _o is relatively large), since the time constant τ of the capacitor C becomes large, the capacitor C ) will be charged more slowly.

The reference voltage (V _ref ) is an arbitrary voltage for determining the voltage change time (eg, charging time) that has elapsed while the _{sampling voltage (V 0} ) is being charged in the capacitor (C), and the voltage change time (eg, For example, it may mean a voltage serving as a measurement standard for charging time). The reference voltage V _{ref considers} circuit element characteristics such as the capacitor C and the memory cells 110 , and the time it takes _{for the voltage charged in the capacitor C to reach the reference voltage V ref .} can be set arbitrarily. For example, it is desired that it takes about 40 ns for _{the voltage charged in the capacitor C to reach the reference voltage V ref} , based on circuit element characteristics such as the capacitor C and the memory cells 110 . In this case, the reference voltage V _ref may be set to a value between 0.1V and 0.3V. However, these are only exemplary values for convenience of description, and the reference voltage V _ref is a value that may be set in consideration of the circuit configuration of the in-memory processing device 100 .

The comparator 130 responds to the case where _{the voltage currently charged in the capacitor C exceeds the reference voltage V ref} while the capacitor C is gradually charged by the application of the _{sampling voltage V o , and an end signal} (STOP) can be output. In other words, the comparator 130 may output a termination signal STOP in response to the case in which the voltage currently charged in the capacitor C _{reaches the reference voltage V ref .}

On the other hand, the comparator 130 is activated (or enabled) only while the trigger pulse (TRIGGER) signal is applied to perform a comparison operation, which will be described in detail in the corresponding section below.

When the time-to-digital converter (TDC) 140 receives the end signal STOP from the comparator 130 , the counting pulse COUNT applied in synchronization with the trigger pulse TRIGGER at the time when the end signal STOP is received ) of the counting value (T _out ) is output as a digital value. A digital value output from the time-to-digital converter (TDC) 140 may represent a quantization level corresponding to the _{sampling voltage Vo .}

Meanwhile, the memory cells 110 , the capacitor C, the comparator 130 and the time-to-digital converter 140 shown in FIG. 4 are one output line (ie, one output line) on the memory cell array in the in-memory processing device 100 . , one column line). However, as described above, the memory cell array may include a plurality of output lines (ie, a plurality of column lines).

5 is a diagram for describing a mapping relationship between a combined resistance value of a certain column line and a sum bit value according to an exemplary embodiment.

Referring to FIG. 5 , in the graph 500 , the x-axis represents the summed bit value calculated in a certain column line. In the graph 500 , the y-axis represents a normalized synthesized resistance value in a corresponding column line. Here, the combined resistance value in the corresponding column line is inversely proportional to the sum of currents in the corresponding column line.

Referring to the graph 500 , it can be seen that the combined resistance value in a column line is in inverse proportion to the sum bit value corresponding to the MAC operation result in the corresponding column line. In other words, it can be seen that the sum of currents in a certain column line has a proportional relationship with the sum bit value corresponding to the MAC operation result in the corresponding column line. Accordingly, in view of such a correlation, it can be seen that the MAC operation result can also be estimated from _{the sampling voltage V o applied to the capacitor C.}

6 is a diagram for describing an in-memory processing device according to an embodiment.

Previously, in FIG. 4 , a plurality of input lines (row lines) and one output line (column line 120 ) in the in-memory processing device 100 were described, but in FIG. 6 , a plurality of input lines (row lines) lines) and an in-memory processing device 60 having a memory cell array 690 including a plurality of output lines (column lines).

The memory cell array 690 in the in-memory processing device 60 has a plurality of input lines (row lines) capable of individually receiving an input signal and a plurality of output lines (column lines) for individually outputting an output signal. ) may be included. Each of the input lines (row lines) may intersect the output lines (column lines). Although FIG. 6 shows that the input line (row line) and the output line (column line) cross each other perpendicularly, the present invention is not limited thereto.

Each of the input lines (row lines) includes an input signal b ₁ , b ₂ , b ₃ , b ₄ , ..., b _j , ..., b _m , and an input signal by a start signal START. A switch for switching the application of is connected. The input signals b ₁ , b ₂ , b ₃ , b ₄ , ..., b _j , ..., b _m may correspond to an input voltage (or input current) representing a binary value, but is limited thereto doesn't happen For example, an input signal indicating a bit value of 1 may indicate an arbitrary voltage, and an input signal indicating a bit value of 0 may indicate a floating voltage.

Memory cells 610 are provided at positions where input lines (row lines) and output lines (column lines) cross each other.

Each of the memory cells 610 may be configured to receive an input signal (input voltage) through an input line (row line) in which a corresponding memory cell is disposed among input lines (row lines). For example, the memory cells disposed along the j-th input line 691 _{may be configured to receive the j-} th input signal b j in response to the start signal START.

The processing circuit 600 in the in-memory processing device 60 includes capacitors 620 and comparators 640 connected to one end of each of the output lines (column lines), and a time-to-digital converter (TDC) 650 . ) and an output unit 660 .

The capacitors 620 are individually arranged for each output line (column line), and a capacitor connected to an output line (column line) is a sampling circuit that charges a sampling voltage corresponding to the sum of currents of the corresponding output line (column line). make up

The capacitors 620 may be charged while the amount of charge is gradually increased by the application of the sampling voltage. For example, the capacitor 621 disposed on the i-th output line (the i-th column line) 692 is the input of the input signals and resistance values of the memory cells of the i-th output line (the i-th column line) 692 . A voltage (ie, a sampling voltage) corresponding to the sum of currents based on the voltages may be applied to be charged.

Meanwhile, the capacitors 620 connected to the output lines (column lines) may constitute a sampling circuit and may have the same capacitance. Accordingly, the difference in the time constant between the capacitors 620 may depend on the difference in the combined resistance value of each of the output lines (column lines), that is, the difference in the sum of currents of each of the output lines (column lines).

The comparators 640 may be individually arranged for each output line (column line). Each of the comparators 640 determines whether the voltage currently charged in each of the capacitors 620 of the corresponding output line (column line) _{reaches the reference voltage V ref .} Each of the comparators 640, when compared that the voltage currently charged in each of the capacitors 620 connected to the reference voltage (V _ref ) has reached the time-to-digital converter ( TDC) (650).

The comparators 640 receive a trigger pulse TRIGGER as a start signal of a comparison operation. That is, the comparators 640 are activated (enabled) only when receiving the trigger pulse TRIGGER from the control signal generator 630 , and are deactivated (disabled) when not receiving the trigger pulse TRIGGER. Reception of the trigger pulse TRIGGER means reception of a pulse signal indicating high, but is not limited thereto.

When a start signal START indicating application of an input signal to the memory cell array 690 is applied, the control signal generator 630 may generate a trigger pulse TRIGGER and a counting pulse COUNT. The trigger pulse TRIGGER may be generated at a timing corresponding to a predetermined quantization level among quantization levels divided according to the charging time of the capacitors 620 in the sampling circuit. The quantization levels will be described in detail with reference to FIGS. 7 and 8 . The counting pulse COUNT is a pulse signal generated in synchronization with the trigger pulse TRIGGER when the application of the trigger pulse TRIGGER is started, and is a signal for counting the time when the trigger pulse TRIGGER is applied. The counting pulse COUNT is provided to a time-to-digital converter (TDC) 650 .

The time-to-digital converter (TDC) 650 performs time-digital conversion when _{the voltage currently charged in a certain capacitor reaches the reference voltage (V ref ).}

Specifically, the time-to-digital converter (TDC) 650 may receive a termination signal STOP from each of the comparators 640 connected to each output line (column line). As described above, the termination signal STOP is a signal indicating that _{the voltage currently charged in a certain capacitor has reached the reference voltage V ref .} The time-to-digital converter (TDC) 650 latches the counting value of the counting pulse COUNT at the point in time when the end signal STOP is received.

For example, when the voltage currently charged in the capacitor 621 of the i-th output line (i-th column line) 692 _{reaches the reference voltage (V ref} ), the i-th output line (i-th column line) ( The comparator 641 of 692 _{outputs the end signal STOP i} to the time-to-digital converter (TDC) 650 . Time-latch to-digital converter (TDC) (650) is a stop signal (STOP _i) a, an end signal (STOP) the counting value (T _{out, i)} of counting pulses (COUNT) at the reception point is received.

The output unit 660 may output a counting value for a certain output line (column line) output from the time-digital converter (TDC) 650 as a digital value OUT. Specifically, the output unit 660 may output a quantization level corresponding to a counting value among preset quantization levels. Here, the output quantization level is a value derived from the sampling voltage, and eventually corresponds to the MAC operation result of the corresponding output line (column line).

The time-to-digital converter (TDC) 650 outputs a counting value for each output line, and the output unit 350 outputs a counting value for each output line as a quantization level. Meanwhile, as mentioned above, the quantization levels will be described in detail with reference to FIGS. 7 and 8 .

7 is a diagram for explaining setting time intervals of quantization levels according to a time when a voltage is charged in a capacitor, according to an embodiment.

Referring to FIG. 7 , a graph 700 shows a change in the charging voltage of a certain capacitor constituting a sampling circuit included in the in-memory processing device. In the graph 700 , the x-axis represents the elapsed time after the charging of the sampling voltage in the capacitor is started, and the y-axis represents the change in the charging voltage of the capacitor over time.

The reference voltage (V _ref ) is a voltage input to the comparator for comparison with the currently charged voltage of the capacitor.

It can be assumed that the in-memory processing device is a device capable of providing an operation result of k-bit resolution (k is a natural number). Also, it can be assumed that k-bit resolution has n (n is a natural number) quantization levels.

As described above, the sum of the currents in any column line can be applied to the capacitor as a corresponding sampling voltage. Since the capacitors included in the sampling circuit have all been described as having the same capacitance, the only factor causing the difference in the time constant (τ=R*C) between the capacitors is the resistance value (ie, the combined resistance value) of the column line to which each capacitor is connected. point can be seen. Based on Ohm's law, the combined resistance value of a column line and the intensity (ie, sum of currents) of the column current flowing along the column line are inversely proportional to each other. Accordingly, it can be derived that the time constant difference between the capacitors depends on the sum of the currents of each of the column lines. For example, if the sum of the currents in the column lines is relatively large (or the sampling voltage is relatively small), the capacitor will charge faster to the _{reference voltage (V ref ) because the time constant (τ) of the capacitor will be small.} . Conversely, if the sum of currents in the column line is relatively small (or if the sampling voltage is relatively large), the capacitor will charge more slowly to the _{reference voltage (V ref ) because the time constant (τ) of the capacitor becomes large.}

A correlation exists, in which the sum of currents in a column line corresponds to a MAC operation result in that column line. Accordingly, based on such a correlation, it can be estimated that the time constant difference between the capacitors eventually corresponds to the difference in the MAC operation result.

For k-bit resolution, a total time interval in which the charging voltage is changed until the sampling voltage is charged in the capacitor may be quantized into n sub-time intervals. Each of the n sub-time intervals corresponds to each of the n quantization levels.

Specifically, a case in which m memory cells 710 are connected to the column line CL _{i will be described.} Each of the memory cells 710 may have a variable resistance, and each of the memory cells 710 may have a first resistance value (R _{1_max} , R _{2_max} , ..., R _{m_max} ) or a second resistance value (R _{1_min} , R _{2_min ).} , ..., R _{m_min} ), and it is assumed that the first resistance value is greater than the second resistance value.

When the resistance values of each of the m memory cells 710 in the column line CL _{i all have the first resistance values R 1_max} , R _{2_max} , ..., R _{m_max} , the The combined resistance value is maximum. Conversely, when the resistance values of each of the m memory cells 710 in the _{column line CL i all have the second resistance values R 1_min} , R _{2_min} , ..., R _{m_min} , the m memory cells 710 . ) is the minimum combined resistance value.

Accordingly, the _{combined resistance value for the column line CL i} is the maximum combined resistance value calculated from the case in which the memory cells 710 all have the first resistance values R _{1_max} , R _{2_max} , ..., R _{m_max .} and the second resistance values R _{1_min} , R _{2_min} , ..., R _{m_min} ).

A case in which the column line CL _i has the maximum combined resistance value corresponds to a case in which a column current having the minimum sum of currents flows through the _{column line CL i .} Conversely, a case in which the column line CL _i has a minimum combined resistance value corresponds to a case in which a column current having the maximum sum of currents flows through the _{column line CL i .} Based on this principle, sub-time intervals corresponding to each of the n quantization levels may be set.

Sub-time period 701 may correspond to the case where the column current having the sum of the minimum current to the column lines (CL _i) flowing through the sub-time period 702 having a total maximum current to the column lines (CL _i) It can respond to the case where column current flows. For example, sub-time interval 701 may correspond to a quantization level in which each bit value of k-bit resolution is all 0, and sub-time interval 702 is a quantization level in which each bit value of k-bit resolution is all 1. It can correspond to the level.

The time interval between the sub-time interval 701 and the sub-time interval 702 includes sub-time intervals corresponding to the remaining quantization levels, and may be divided to have equal intervals according to the number of remaining quantization levels. have. However, the present invention is not limited thereto, and intervals of sub-time intervals corresponding to the remaining quantization levels may be divided not to be the same. For example, according to a general simulation result for the MAC operation result, most values of the MAC operation result may be obtained as normalized by a Gaussian distribution. Accordingly, based on the Gaussian distribution, sub-time intervals corresponding to the quantization levels may be set differently.

Referring to FIG. 7 , in the graph 700 , the quantization level may be set to increase by one level from the sub-time section 701 to the sub-time section 702 .

However, the present invention is not limited thereto, and the quantization level may be set to decrease by one level from the sub-time period 701 to the sub-time period 702 . For example, although it has been described in FIG. 7 that the case having the maximum synthesis resistance corresponds to the minimum quantization level and the case having the minimum synthesis resistance corresponds to the maximum quantization level, the opposite may be set. That is, the quantization level indicated by each time period may vary according to which logic values each of the variable resistance values (the first resistance value and the second resistance value) of the memory cells 710 will represent.

8 is a diagram for explaining a timing of applying a trigger pulse TRIGGER to a comparator according to an exemplary embodiment.

Referring to FIG. 8 , a trigger pulse TRIGGER may be generated from a timing (time point) 810 corresponding to a predetermined quantization level. Here, the predetermined quantization level may be a quantization level that is one level higher than the minimum quantization level among the quantization levels. However, the present invention is not limited thereto, and the trigger pulse TRIGGER may be generated from a predetermined elapsed time (eg, after 40 ns) after the capacitor starts charging without considering a time point corresponding to the quantization level.

Since the trigger pulse TRIGGER is applied to the comparators from a timing (time point) 810 corresponding to a predetermined quantization level, power consumption as the comparators are operated at the minimum quantization level such as “0000” may be reduced. The trigger pulse TRIGGER may be generated by being applied to the comparator only up to a time interval corresponding to a quantization level (eg, “1110”) that is one level lower than the maximum quantization level (eg, “1111”), thereby A further reduction in power consumption may be sought.

Meanwhile, the pulse width of the trigger pulse TRIGGER may be set to be less than or equal to the interval of the sub-time interval corresponding to each of the quantization levels.

The comparator in the in-memory processing device may be implemented as a latched comparator type. Accordingly, the comparator may output the comparison result at the falling edge of the trigger pulse TRIGGER. For example, when the charging voltage of the capacitor _{reaches the reference voltage (V ref} ) at the time point 820 , the comparator ends at the falling edge of the trigger pulse TRIGGER corresponding to the time point 820 . STOP) can be output.

A time point at which the end signal STOP is output may be output as a quantization level “1010” by time-digital conversion.

The time constant of the capacitor varies according to the sampling voltage (or the sum of currents of the column line) applied to the capacitor of a certain column line, and accordingly, the timing at which _{the charging voltage of the capacitor reaches the reference voltage V ref also varies.} As shown in FIG. 8 , the sub-time intervals 830 of the 4-bit quantization levels may be set to correspond to points in time at which _{the charging voltage of the capacitor reaches the reference voltage V ref .}

9 is a diagram for describing time-digital conversion performed in an in-memory processing device according to an embodiment.

Referring to FIG. 9 , the control signal generator 910 includes a quantization level determiner 711 , a pulse generator 712 , and a counter 713 . The pulse generator 712 generates a trigger pulse TRIGGER for initiating a comparison operation of the comparators 720 . The counter 713 generates a counting pulse COUNT for identifying when the end signal STOP provided from each of the comparators 720 is received. The application of the trigger pulse (TRIGGER) and the counting pulse (COUNT) is synchronized with each other, and accordingly, the counting pulse (COUNT) is also time-digital converter (TDC) ( 730) may be authorized.

The timing at which the trigger pulse TRIGGER is generated may be determined by the quantization level determiner 711 . The quantization level determiner 711 determines the total number of quantization levels to be set (that is, the number of bits of the digital value), the device characteristics of the capacitor charging the sampling voltage (eg, capacitance, time constant), and each column line. The timing at which the trigger pulse TRIGGER is generated may be determined in consideration of resistance values of the connected memory cells.

For example, the quantization level determiner 711 may determine the timing at which the trigger pulse TRIGGER is generated based on the methods described with reference to FIGS. 7 and 8 . Specifically, the quantization level determiner 711 generates a trigger pulse TRIGGER from a timing (time point) (810 in FIG. 8) corresponding to a predetermined quantization level (ie, a quantization level that is one level greater than the minimum quantization level). , the pulse generator 912 may be controlled. Furthermore, the quantization level determiner 711 may control the pulse generator 912 so that the trigger pulse TRIGGER is generated only up to a time interval corresponding to a quantization level that is one level lower than the maximum quantization level. Meanwhile, the counter 913 may generate the counting pulse COUNT only while the trigger pulse TRIGGER is applied.

The comparators 920 receive a trigger pulse TRIGGER as a start signal of a comparison operation, and compare a voltage currently charged in a capacitor connected to each of the comparators 920 with a reference voltage. When it is detected that the voltage currently charged in the capacitor connected to each of the comparators 920 has reached the reference voltage, each of the comparators 920 transmits a time-to-digital converter (TDC) ( 930) is output.

Each of the comparators 920 may be implemented as a latched comparator type. Accordingly, the comparators 920 may output a comparison result at the falling edge of the trigger pulse TRIGGER. For example, when comparator s detects that the charging voltage of the capacitor has reached the reference voltage at a certain point in time, comparator s generates a termination signal (STOP) at the falling edge of the trigger pulse (TRIGGER) corresponding to that point in time. It can be output as a flip-flop of a time-to-digital converter (TDC) 930 connected to .

On the other hand, among the comparators 920 , a signal (done) indicating that the output of the stop signal (STOP) is completed may be fed back to the comparator outputting the stop signal (STOP), and the comparator to which the signal (done) is fed back is deactivated ( can be disabled).

A time-to-digital converter (TDC) 930 includes flip-flops 935 coupled to each of the comparators 920 . The flip-flops 935 may be activated (enabled) by a counting pulse COUNT received from the counter 913 .

Each of flip-flops 935 are now counted value at the when the counting pulses (COUNT) are applied, and the stop signal (STOP) from each of comparators 920 received while the stop signal (STOP) is received, the time (T _out ) is latched. For example, if the flip-flop connected to the comparator s receives a stop signal STOP from the comparator s while the counting pulse COUNT is being applied, the flip-flop returns the current counting value T _{out_s of the counting pulse COUNT.} print out

In this way, each of the flip-flops 935 provided in the time-to-digital converter (TDC) 930 _{individually outputs a counting value T out} corresponding to the reception time of the end signal STOP, and thus the memory cell array Time-digital conversion may be performed on each of the column lines (output lines) of . As described above, the processing circuit 600 _{maps the digital value of the quantization level corresponding to the counting value T out for} each column line (output line), and the mapped quantization level for each column line (output line) It is possible to output a digital value (eg, k-bit resolution) of .

10 is a diagram for explaining outputting a counting value for each column line (output line) through time-digital conversion according to an exemplary embodiment.

Referring to FIG. 10 , each of the time points t _a , t _b and t _c represents a time point at which the voltage currently charged in each of the capacitors C _x , C _y , C _{z reaches the reference voltage V ref .} indicates. If listed in the order of the earliest time, the time point (t _b ), the time point (t _a ), and the time point (t _c ) are in the order.

A column current having a sum of currents I _{col_x} , I _{col_y} , I _{col_z} flows through the column lines 1001 , 1002 , and 1003 , respectively, and a sampling voltage corresponding to the sum of currents I _{col_x} , I _{col_y} , I _{col_z is a capacitor} (C _x , C _y , C _z ) are applied respectively. A trigger pulse TRIGGER is applied to the comparators 1021 , 1022 , and 1023 from the same time point, and the comparators 1021 , 1022 , 1023 each have capacitors C _x , Comparison is performed between the voltage currently charged in C _y , C _z ) and the reference voltage (V _{ref ).} In this case, the comparators 1021 , 1022 , and 1023 may be latch comparators, and accordingly, each of the comparators 1021 , 1022 , and 1023 may perform a comparison operation for each falling edge of the trigger pulse TRIGGER.

First, at a time point (t _b ), the comparator 1022 determines that the voltage currently charged in the _{capacitor (C y ) has reached the reference voltage (V ref} ), and converts the stop signal (STOP @t _b ) to a time-digital converter (TDC) is output to the flip-flop 1032 of the 1030. Meanwhile, the comparator 1022 in which the output of the stop signal STOP @t _b is completed may be deactivated (disabled).

The time-to-digital converter (TDC) 1030 is synchronized with the application of the trigger pulse TRIGGER to apply the counting pulse COUNT. The flip-flop 1032 latches _{the counting value T out} @t _b of the counting pulse COUNT at the time t _{b when the end signal STOP @t b is received.} Accordingly, time-digital conversion of the column line 1002 among the column lines 1001 , 1002 , and 1003 is first completed. Meanwhile, the counting value T _out @t _b may be output as a digital value of the mapped quantization level.

Next, at _a time point ta a, the comparator 1021 _{outputs a} stop signal STOP @ta a to the flip-flop 1031 . The comparator 1021 in which the output of the stop signal STOP @t _a is completed may be deactivated (disabled). The flip-flop 1031 latches _{the counting value T out} @t _a of the counting pulse COUNT at the time ta _{a when the end signal STOP @t a is received.} Since the counting value (T _out @t _a ) corresponds to a value different from the counting value (T _out @t _b ), the digital value of the quantization level mapped to the _{counting value (T out} @t _{a ) is the counting value (T out} @ t _b ) may be different from the digital value of the mapped quantization level. However, each of the different points in time (t _a, t _b) counting values _{(T out @t a, T out} @t b) of the Figure, different points in time has been output even (t _a, t _b), the same quantization level in In the case of belonging to the same sub-time interval corresponding to , time-digital conversion may be performed to a digital value of the same quantization level.

At a time point t _c , the comparator 1023 _{outputs a stop signal STOP @t c} to the flip-flop 1033 . The comparator 1021 in which the output of the stop signal STOP @t _c is completed may be deactivated (disabled). The flip-flop 1033 latches _{the counting value T out} @t _c of the counting pulse COUNT at the _{time t c} at which the _{end signal STOP @t c is received.}

The processing circuit in the in-memory processing device may perform a MAC operation for each column line (output line) through time-digital conversion as described above. In the in-memory processing device according to the present embodiments, data transfer speed and power consumption may be improved, unlike the Von Neumann structure in which the memory and the operation unit are separated. In addition, since the in-memory processing device does not need to include analog-to-digital converters (ADCs) in individual column lines, power consumption and circuit area may be reduced compared to an architecture including ADCs.

Meanwhile, in the described embodiments, the voltage value of the input signal and the resistance value of the memory cell are equal to a value corresponding to ON (or logic 1) and a value corresponding to OFF (or logic 0). Although it has been described on the assumption that it has a binary value, the present embodiments are not limited thereto. The voltage value of the input signal and the resistance value of the memory cell may have values that are differentiated into multi-states. For example, when a 2-bit value is input to one input line (low line), the input signal (input voltage) is floated for "00", and the first voltage value, " A second voltage value greater than the first voltage value for 10” and a third voltage value greater than the second voltage value for “11” may be allocated as the input signal. In addition, when the memory cell indicates a 2-bit value, the memory cell has a first resistance value for “00”, a second resistance value greater than the first resistance value for “01”, and a third resistance value for “10”. A fourth resistance value may be assigned to “resistance values” and “11.” It is not limited to an input signal received on each input line (row line) and each memory cell indicating a 2-bit multi-state, Multiple states corresponding to the above bits may be indicated, and values according to systems other than the binary system may be assigned.

11 is a flowchart of a method for performing in-memory processing according to an embodiment. Referring to FIG. 11 , since the method of performing in-memory processing is related to the embodiments described in the drawings described above, the contents described in the drawings are also included in the method of FIG. 11 , even if omitted below. can be applied.

In step 1101 , the control signal generator 630 of the processing circuit 600 resets control signals (trigger pulse TRIGGER and counting pulse COUNT).

In operation 1102 , an input signal is applied to the plurality of memory cells 610 through each row line of the memory cell array 690 .

In operation 1103 , each of the capacitors 620 of the processing circuit 600 charges a sampling voltage corresponding to a current sum of column currents flowing through each column line of the memory cell array 690 .

In operation 1104 , the control signal generator 630 of the processing circuit 600 generates a control signal (a trigger pulse TRIGGER and a counting pulse COUNT) based on the set quantization level. A trigger pulse TRIGGER is applied to the comparators 640 , and a counting pulse COUNT is applied to a time-to-digital converter (TDC) 650 .

In operation 1105 , the comparators 640 of the processing circuit 600 compare the voltage Vc currently charged in the capacitors 620 connected thereto and the reference voltage Vref while the trigger pulse TRIGGER is applied. do.

In operation 1106 , each of the comparators 640 determines whether the voltage Vc currently charged in the capacitors 620 has reached the reference voltage Vref. The comparator that determines that the currently charged voltage Vc has reached the reference voltage Vref outputs a stop signal STOP to the time-to-digital converter (TDC) 650 , and step 1107 is performed. Otherwise, step 1106 is performed again.

In step 1107, the comparator outputting the end signal STOP is deactivated (disabled).

In operation 1108 , the time-to-digital converter (TDC) 650 of the processing circuit 600 performs time-digital conversion for outputting the current counting value of the counting pulse COUNT.

In operation 1109 , the output unit 660 of the processing circuit 600 outputs a digital value of a quantization level corresponding to the counting value.

12 is a block diagram illustrating a computing device according to an embodiment.

Referring to FIG. 12 , the computing device 1200 analyzes input data in real time based on a neural network to extract valid information, makes a situation determination based on the extracted information, or an electronic device on which the computing device 1200 is mounted. It is possible to control the configurations of the device. For example, the computing device 1200 may include a drone, a robot device such as an Advanced Drivers Assistance System (ADAS), a smart TV, a smartphone, a medical device, a mobile device, an image display device, a measurement device, and an IoT device. and the like, and may be mounted on at least one of various types of electronic devices.

The computing device 1200 may include a host processor 1210 , a RAM 1220 , an in-memory processing device 1230 , a memory device 1240 , a sensor module 1250 , and a communication module 1260 . The computing device 1200 may further include an input/output module, a security module, a power control device, and the like. Some of the hardware components of the computing device 1200 may be mounted on at least one semiconductor chip. The in-memory processing device 1230 is an apparatus including the in-memory processing device described above with reference to the drawings, and may correspond to a neural network dedicated hardware accelerator itself or a neural network apparatus including the same.

The host processor 1210 controls the overall operation of the computing device 1200 . The host processor 1210 may include one processor core or a plurality of processor cores (multi-core). The host processor 1210 may process or execute programs and/or data stored in the memory device 1240 . The host processor 1210 may control the function of the in-memory processing device 1230 by executing programs stored in the memory device 1240 . The host processor 1210 may be implemented as a CPU, GPU, AP, or the like.

The RAM 1220 may temporarily store programs, data, or instructions. For example, programs and/or data stored in the memory device 1240 may be temporarily stored in the RAM 1220 according to a control or boot code of the host processor 1210 . The RAM 1220 may be implemented as a memory such as dynamic RAM (DRAM) or static RAM (SRAM).

The in-memory processing device 1230 may perform the neuromorphic operation described above with reference to the drawings, for example, the MAC operation, and output the MAC operation result. However, the in-memory processing device 1230 may perform various other in-memory computing.

The memory device 1240 is a storage location for storing data, and may store an operating system (OS), various programs, and various data. In an embodiment, the memory device 1240 includes data (eg, input signal data, weight data, etc.) necessary for the operation of the in-memory processing device 1230 and operation result data (eg, MAC). calculation results (ie, quantization level data), etc.) may be stored.

The memory device 1240 may be a DRAM, but is not limited thereto. Memory device 1240 may include at least one of volatile memory or nonvolatile memory. Nonvolatile memory includes ROM, PROM, EPROM, EEPROM, flash memory, PRAM, MRAM, RRAM, FRAM, and the like. Volatile memory includes DRAM, SRAM, SDRAM, PRAM, MRAM, RRAM, FeRAM, and the like. In an embodiment, the memory device 1240 may include at least one of HDD, SSD, CF, SD, Micro-SD, Mini-SD, xD, and Memory Stick.

The sensor module 1250 may collect information around the electronic device on which the computing device 1200 is mounted. The sensor module 1250 may sense or receive a signal (eg, an image signal, an audio signal, a magnetic signal, a biosignal, a touch signal, etc.) from the outside of the electronic device, and may convert the sensed or received signal into data. To this end, the sensor module 1250 includes various types of sensing devices, such as a microphone, an imaging device, an image sensor, a light detection and ranging (LIDAR) sensor, an ultrasonic sensor, an infrared sensor, a biosensor, and a touch sensor. may include at least one of

The sensor module 1250 may provide the converted data as input data to the in-memory processing device 1230 . For example, the sensor module 1250 may include an image sensor, capture an external environment of the electronic device to generate a video stream, and input successive data frames of the video stream to the in-memory processing device 1230 . It can be provided in order as data. However, the present invention is not limited thereto, and the sensor module 1250 may provide various types of data to the in-memory processing device 1230 .

The communication module 1260 may include various wired or wireless interfaces capable of communicating with an external device. For example, the communication module 1260 may include a wired local area network (LAN), a wireless local area network (WLAN) such as Wi-Fi (Wireless Fidelity), and a wireless personal network such as Bluetooth (Bluetooth). Personal Area Network (WPAN), Wireless USB (Wireless Universal Serial Bus), Zigbee, NFC (Near Field Communication), RFID (Radio-frequency identification), PLC (Power Line communication), or 3G (3rd Generation), 4G (4th Generation), Long Term Evolution (LTE), 5th Generation (5G), and the like, may include a communication interface connectable to a mobile cellular network.

13 is a diagram for describing an example of a neural network.

Referring to FIG. 13 , a neural network 1300 may correspond to an example of a deep neural network (DNN). For convenience of description, although the neural network 1300 is illustrated as including two hidden layers, it may include various numbers of hidden layers. Also, although the neural network 1300 is illustrated as including a separate input layer 1310 for receiving input data in FIG. 13 , the input data may be directly input to the hidden layer.

In the neural network 1300 , artificial nodes of layers other than an output layer may be connected to artificial nodes of a next layer through links for transmitting an output signal. An output of an activation function regarding weighted inputs of artificial nodes included in the previous layer may be input to the artificial node through these links. The weighted input is an input (node value) of an artificial node multiplied by a weight. The input corresponds to axon values, and the weight corresponds to synaptic weights. The weight may be referred to as a parameter of the neural network 1300 . The activation function may include a sigmoid, a hyperbolic tangent (tanh), and a rectified linear unit (ReLU), and nonlinearity may be formed in the neural network 1300 by the activation function. have.

The in-memory processing device described in the drawings may be used prior to in-memory processing or in-memory computing for driving a deep learning algorithm. For example, the calculation of the weighted input transferred between the nodes 1321 of the neural network 1300 may be configured as a MAC operation. The output of any one node 1321 included in the neural network 1300 may be expressed as Equation 1 below.

Equation 1 may represent _{an output value y i} of the i-th node 1321 for m input values in an arbitrary layer. x _j may represent the output value of the j-th node of the previous layer, and w _j,i may represent the output value of the j-th node and a weight applied to the i-th node 1321 of the current layer. f() may represent an activation function. As shown in Equation 1, for the activation function, the multiplication and accumulation result of the _{input value x j} and the weight w _{j,i can be used.} In other words, an operation (MAC operation) of multiplying and adding an _{appropriate input value x j} and a weight w _{j,i at a desired time may be repeated.} In addition to these uses, there are various application fields requiring MAC operation, and for this purpose, a neuromorphic device capable of processing MAC operation in an analog domain may be used.

The plurality of memory cells of the in-memory processing device may have resistance corresponding to a connection weight of a connection line connecting the plurality of nodes in the neural network 1300 including one or more layers including the plurality of nodes. can An input signal provided along input lines (row lines) in which memory cells are disposed may represent a value corresponding to the _{node value x j .} Accordingly, the in-memory processing device may perform at least some of operations required to implement the neural network 1300 .

On the other hand, the application of the in-memory processing device is not necessarily limited to neuromorphic calculations, and in addition, it can be utilized for calculations requiring fast processing of multiple input data using analog circuit characteristics with low power.

14 is a flowchart of a method for performing in-memory processing according to an embodiment. Referring to FIG. 14 , since the method of performing in-memory processing is related to the embodiments described in the drawings described above, the contents described in the drawings are also included in the method of FIG. 14 , even if omitted below. can be applied.

In operation 1401 , an input signal is applied to the plurality of memory cells 610 through each row line of the memory cell array 690 .

In operation 1402 , the processing circuit 600 charges the capacitor 620 connected to each column line with a sampling voltage corresponding to the sum of currents of column currents flowing through each column line of the memory cell array 690 .

In step 1403 , the processing circuit 600 applies a trigger pulse generated at a timing corresponding to a predetermined quantization level among the quantization levels divided according to the charging time of the capacitor 620 to the comparators to provide a current to the capacitor 620 . A comparison operation is performed between the charged voltage and the reference voltage.

In operation 1404 , the processing circuit 600 determines a quantization level corresponding to the sampling voltage by performing time-digital conversion when the currently charged voltage reaches the reference voltage.

Meanwhile, the above-described embodiments can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. In addition, the structure of data used in the above-described embodiments may be recorded in a computer-readable recording medium through various means. The computer-readable recording medium includes a storage medium such as a magnetic storage medium (eg, a ROM, a floppy disk, a hard disk, etc.) and an optically readable medium (eg, a CD-ROM, a DVD, etc.).

Those of ordinary skill in the art related to the present embodiment will understand that the embodiment may be implemented in a modified form without departing from the essential characteristics of the above description. Therefore, the disclosed embodiments are to be considered in an illustrative rather than a restrictive sense. The scope of the rights is indicated in the claims rather than the above description, and all differences within the scope equivalent thereto should be construed as being included in the present embodiment.

Claims (20)

An apparatus for performing in-memory processing, comprising:
a memory cell array including a plurality of memory cells, providing a current sum of column currents flowing through each column line when input signals are applied through each row line;
a sampling circuit including a capacitor for charging a sampling voltage corresponding to the sum of the currents of each of the column lines; and
A comparison operation between a voltage currently charged in the capacitor and a reference voltage is started by application of a trigger pulse generated at a timing corresponding to a predetermined quantization level among quantization levels divided according to a charging time of the capacitor, and the current charging and a processing circuit for determining a quantization level corresponding to the sampling voltage by performing time-to-digital conversion at a point in time when the obtained voltage reaches the reference voltage. The method of claim 1,
The processing circuit is
a control signal generator generating the trigger pulse at the timing corresponding to the predetermined quantization level and generating a counting pulse in synchronization with the generation of the trigger pulse; and
and comparators connected for each column line activated by the application of the trigger pulse. The method of claim 1,
Each of the quantization levels is
corresponding to each of the sub-time intervals separated from the total time interval in which the charging voltage is varied until the sampling voltage is charged to the capacitor. 4. The method of claim 3,
The sub-time intervals are
determined based on at least one of a resistance value of the memory cells and a capacitance of the capacitor. The method of claim 1,
The predetermined quantization level corresponding to the timing at which the trigger pulse is applied is a quantization level that is one level larger than a minimum quantization level among the quantization levels. The method of claim 1,
The processing circuit is
Comparators connected for each column line that receive the trigger pulse as a start signal of the comparison operation and output an end signal when the voltage currently charged in the capacitor reaches the reference voltage when compared to the reference voltage ; and
and a time-to-digital converter (TDC) for outputting, as a digital value, a counting value of a counting pulse applied in synchronization with the trigger pulse when the end signal is received when the end signal is received. 7. The method of claim 6,
The time-to-digital converter (TDC) is
and flip-flops coupled to each of the comparators that latch a current counting value at the time the end signal is received when the end signal is received. 7. The method of claim 6,
and a comparator outputting the end signal among the comparators is deactivated. The method of claim 1,
wherein the comparators are implemented as a latched comparator type. The method of claim 1,
The trigger pulse is
The apparatus is applied up to a time interval corresponding to a quantization level that is one level smaller than a maximum quantization level among the quantization levels. The method of claim 1,
The minimum quantization level among the quantization levels corresponds to a case in which a plurality of memory cells included in one column line have maximum combined resistance,
The maximum quantization level among the quantization levels corresponds to a case in which a plurality of memory cells included in one column line have minimum combined resistance.
Device. In the computing device,
host processor;
memory device; and
an in-memory processing device;
The in-memory processing device is
a memory cell array including a plurality of memory cells, providing a current sum of column currents flowing through each column line when input signals are applied through each row line;
a sampling circuit including a capacitor for charging a sampling voltage corresponding to the sum of the currents of each of the column lines; and
A comparison operation between a voltage currently charged in the capacitor and a reference voltage is started by application of a trigger pulse generated at a timing corresponding to a predetermined quantization level among quantization levels divided according to a charging time of the capacitor, and the current charging A processing circuit for determining a quantization level corresponding to the sampling voltage by performing time-digital conversion at a point in time when the converted voltage reaches the reference voltage,
computing device. 13. The method of claim 12,
The processing circuit is
a control signal generator generating the trigger pulse at the timing corresponding to the predetermined quantization level and generating a counting pulse in synchronization with the generation of the trigger pulse; and
Computing device comprising comparators connected for each column line activated by the application of the trigger pulse. 13. The method of claim 12,
The quantization levels are
The computing device corresponds to each of the sub-time intervals divided from the total time interval in which the charging voltage is changed until the sampling voltage is charged in the capacitor. 13. The method of claim 12,
The predetermined quantization level corresponding to the timing at which the trigger pulse is applied is a quantization level that is one level higher than a minimum quantization level among the quantization levels. 13. The method of claim 12,
The processing circuit is
Comparators connected for each column line that receive the trigger pulse as a start signal of the comparison operation and output an end signal when the voltage currently charged in the capacitor reaches the reference voltage when compared to the reference voltage ; and
and a time-to-digital converter (TDC) for outputting, as a digital value, a counting value of a counting pulse applied in synchronization with the trigger pulse when the end signal is received when the end signal is received. 17. The method of claim 16,
The time-to-digital converter (TDC) is
and flip-flops coupled to each of the comparators that latch a current counting value at the time the end signal is received when the end signal is received. 17. The method of claim 16,
The comparator to which the end signal is output among the comparators is deactivated. A method for performing in-memory processing, comprising:
applying an input signal to the plurality of memory cells through each row line of the memory cell array;
charging a sampling voltage corresponding to a current sum of column currents flowing through each column line of the memory cell array into a capacitor connected to each column line;
performing a comparison operation between a voltage currently charged in the capacitor and a reference voltage by applying a trigger pulse generated at a timing corresponding to a predetermined quantization level among quantization levels divided according to a charging time of the capacitor to comparators; and
and determining a quantization level corresponding to the sampling voltage by performing time-digital conversion at a time point when the currently charged voltage reaches the reference voltage. 20. The method of claim 19,
generating the trigger pulse at the timing corresponding to the predetermined quantization level; and
generating a counting pulse synchronized to the generation of the trigger pulse;
The step of determining the quantization level is
When the time-to-digital converter (TDC) receives an end signal indicating that the voltage currently charged in the capacitor has reached the reference voltage from the comparator, the counting value of the counting pulse at the time the end signal is received is a digital value determining the quantization level by outputting

Images