KR20210093126A - Processing-In-Memory(PIM) system and operating method of the PIM system - Google Patents

Abstract

A PIM system includes a PIM device and a PIM controller. The PIM device includes: a first storage area; a second storage area; and an MAC operator which receives first data and second data from the first storage area and the second storage area, respectively, and performs a multiply-accumulate addition (MAC) operation. The PIM controller is configured to control a memory mode operation and a MAC mode operation of the PIM device. The PIM controller is configured to generate and transmit a memory command signal to the PIM device in the memory mode, and transmit first to fifth MAC command signals to the PIM device in the MAC mode. The present invention provides a PIM system configured to perform the MAC operation in a deterministic manner and an operating method thereof.

Description

Processing-In-Memory (PIM) system and operating method of the PIM system

Various embodiments of the present disclosure relate to a Processing-In-Memory (PIM) system, and more particularly, to a PIM system including a PIM device and a PIM controller, and an operating method thereof.

Recently, interest in artificial intelligence is rapidly increasing not only in the information technology (IT) industry, but also in the financial and medical industries. Accordingly, the introduction of artificial intelligence, more precisely, deep learning, is being considered and prototyped in various fields. In general, deep learning is used for pattern recognition or inference by effectively learning deep neural networks (DNNs) or deep networks that have increased the number of layers of existing neural networks. referred to as the technology used.

One of the backgrounds and causes of such widespread interest in deep learning may be an improvement in the performance of a processor that performs computations. In order to improve the performance of artificial intelligence, learning is done by stacking up to hundreds of layers of neural networks. This trend has continued in recent years, and as a result, the amount of computation required for the hardware that actually performs the computation has increased exponentially. Moreover, in the case of the existing hardware system in which the memory and the processor are separated, the improvement of the artificial intelligence hardware performance is hindered by the limitation of the amount of data communication between the memory and the processor. In order to solve this problem, a PIM device in which a processor and a memory are integrated in a semiconductor chip itself has recently been used as a neural network computing device. Since the PIM device directly performs a calculation operation inside the PIM device, the data processing speed in the neural network is improved.

The problem to be solved by the present application is that the multiplying-accumulating (hereinafter MAC) operation of the PIM device is configured to be performed by the PIM controller, and furthermore, the MAC operation can be performed in a deterministic manner. To provide a PIM system and an operating method thereof.

A PIM system according to an example of the present disclosure includes a PIM device and a PIM controller. The PIM device receives first data and second data from a first storage area, a second storage area, and the first and second storage areas, respectively, and performs a multiply-accumulate addition (MAC) operation MAC operator that does The PIM controller controls a memory mode operation and a MAC mode operation of the PIM device, and generates and transmits a memory command signal to the PIM device in the memory mode, and receives first to fifth MAC command signals in the MAC mode and transmit to the PIM device.

A PIM system according to an example of the present disclosure includes a PIM device and a PIM controller. The PIM device includes a storage area, a global buffer, and a MAC operator that receives first data and second data from the storage area and the global buffer, respectively, and performs a MAC operation. The PIM controller controls a memory mode operation and a MAC mode operation of the PIM device, and generates and transmits a memory command signal to the PIM device in the memory mode, and receives first to fourth MAC command signals in the MAC mode and transmit to the PIM device.

A method of operating a PIM system according to an example of the present disclosure is a method of operating a PIM system including a PIM device for performing a MAC operation and a PIM controller for controlling a MAC operation operation of the PIM device. A first MAC command signal is transmitted to the PIM device, and from the first storage device of the PIM device, an R row (R is “1”) of a weight matrix of MXN (M, N is a natural number greater than 1) for MAC operation. initially set) elements are input to a MAC operator of the PIM device, and transmitting a second MAC command signal from the PIM controller to the PIM device, and NX1 for MAC operation from a second storage device of the PIM device allowing the elements of the first column of the vector matrix to be input to the MAC operator of the PIM device, and sending a third MAC command signal from the PIM controller to the PIM device, so that the R row of the weight matrix in the MAC operator performing input latching on elements, and transmitting a fourth MAC command signal from the PIM controller to the PIM device, so that the input latching of the first column elements of the vector matrix is performed in the MAC operator performing a matrix multiplication operation of elements in the R-th row of the weight matrix and elements in the first column of the vector matrix in the MAC operator; and sending a fifth MAC command signal from the PIM controller to the PIM. transmitting to the device so that output latching is performed on the result data, which is the result of the matrix multiplication operation, in the MAC operator; and transmitting a sixth MAC command signal from the PIM controller to the PIM device, in the MAC operator allowing the output latched result data to be output from the MAC operator, and when R is less than or equal to the last row after increasing R by +1, the step It includes the step of repeatedly performing the steps.

A method of operating a PIM system according to an example of the present disclosure is a method of operating a PIM system including a PIM device for performing a MAC operation and a PIM controller for controlling a MAC operation operation of the PIM device. A first MAC command signal is transmitted to the PIM device, and the R row (R is “1” initially set to “1”) of a weight matrix of MXN (M, N are natural numbers greater than 1) for MAC operation from the storage device of the PIM device. allowing elements to be input to the MAC operator of the PIM device, and allowing the first column elements of the vector matrix of NX1 for MAC operation to be input from the global buffer of the PIM device to the MAC operator of the PIM device; , by transmitting a second MAC command signal from the PIM controller to the PIM device, so that an input latch for the elements of the Rth row of the weight matrix and the elements of the first column of the vector matrix is performed in the MAC operator performing a matrix multiplication operation of elements in the R-th row of the weight matrix and elements in the first column of the vector matrix in the MAC operator; and sending a third MAC command signal from the PIM controller to the PIM device. sending an output latch on the result data that is the result of the matrix multiplication operation in the MAC operator; and transmitting a fourth MAC command signal from the PIM controller to the PIM device, so that the MAC operator allowing output latched result data to be output from the MAC operator, and repeating the above steps when R is equal to or smaller than the last row after increasing R by +1.

According to various embodiments, it is possible to provide a PIM system and a method of operating the same, wherein the MAC operation of the PIM device is performed by the PIM controller, and in particular, the MAC operation is configured to be performed in a deterministic manner.

1 is a block diagram illustrating a PIM system according to an example of the present disclosure.
2 is a diagram illustrating a MAC command output from a MAC command generator of a PIM controller in a PIM system according to an example of the present disclosure.
3 is a block diagram illustrating an example of a configuration of a MAC operator of a PIM device in a PIM system according to an example of the present disclosure.
4 is a diagram illustrating an example of a MAC operation performed in a PIM system according to an example of the present disclosure.
5 is a flowchart illustrating a process of performing the MAC operation of FIG. 4 in the PIM system according to an example of the present disclosure.
6 to 12 are diagrams illustrating a process of performing the MAC operation of FIG. 4 in the PIM system according to an example of the present disclosure.
13 is a diagram illustrating another example of a MAC operation performed in a PIM system according to an example of the present disclosure.
14 is a flowchart illustrating a process of performing the MAC operation of FIG. 13 in the PIM system according to an example of the present disclosure.
15 is a diagram illustrating an example of a MAC operator configuration for performing the MAC operation of FIG. 14 in a PIM system according to an example of the present disclosure.
16 is a diagram illustrating another example of a MAC operation performed in a PIM system according to an example of the present disclosure.
17 is a flowchart illustrating a process of performing the MAC operation of FIG. 16 in the PIM system according to an example of the present disclosure.
18 is a diagram illustrating an example of a MAC operator configuration for performing the MAC operation of FIG. 16 in a PIM system according to an example of the present disclosure.
19 is a diagram illustrating a PIM system according to another example of the present disclosure.
20 is a diagram illustrating a MAC command output from a MAC command generator of a PIM controller in a PIM system according to another example of the present disclosure.
21 is a flowchart illustrating a process of performing the MAC operation of FIG. 4 in a PIM system according to another example of the present disclosure.
22 to 25 are diagrams illustrating a process of performing the MAC operation of FIG. 4 in a PIM system according to another example of the present disclosure.
26 is a flowchart illustrating a process of performing the MAC operation of FIG. 13 in a PIM system according to another example of the present disclosure.
27 is a flowchart illustrating a process of performing the MAC operation of FIG. 16 in a PIM system according to another example of the present disclosure.
28 is a block diagram illustrating another PIM system according to another example of the present disclosure.
29 is a block diagram illustrating another PIM system according to another example of the present disclosure.

In the description of examples of the present application, descriptions such as "first" and "second" are for distinguishing elements, and are not used to limit the members themselves or to mean a specific order. A description that one component is “connected” or “connected” to another component may be directly or electrically connected to another component or may be connected to another component, or another separate component in the middle. Components may be interposed to form a connection relationship or a connection relationship. The term "preset" means that when a parameter is used in a process or algorithm, the value of the parameter is predetermined. The value of the parameter may be set when a process or algorithm is started or set during a period during which the process or algorithm is performed, according to an embodiment. “Logic high level” and “logic low level” are used to describe the logic levels of signals. A signal having a “logic high level” is distinguished from a signal having a “logic low level”. For example, when the signal having the first voltage corresponds to the “logic high level”, the signal having the second voltage may correspond to the “logic low level”. According to an embodiment, the “logic high level” may be set to a voltage greater than the “logic low level”. Meanwhile, the logic levels of the signals may be set to different logic levels or opposite logic levels according to embodiments. For example, a signal having a logic high level may be set to have a logic low level according to an embodiment, and a signal having a logic low level may be set to have a logic high level according to an embodiment.

Hereinafter, the present invention will be described in more detail through examples. These examples are only for illustrating the present invention, and the scope of protection of the rights of the present invention is not limited by these examples.

1 schematically shows a block diagram of a PIM system 10 according to a first embodiment of the present disclosure. As shown in FIG. 1 , the PIM system 10 includes a PIM device 100 and a PIM controller 200 . The PIM device 100 includes a first memory bank (BANK0) 111 , a second memory bank (BANK1 ) 112 , a MAC operator (MAC OPERATOR) 120 , and an interface (I/F) 131 . ) and a data input/output pad (DQ) 132 . The first memory bank 111 , the second memory bank BANK1 , and the MAC operator 120 of the PIM device 100 constitute one MAC unit (MAC UNIT). In another example, the PIM device 100 may include a plurality of MAC units. The first memory bank 111 and the second memory bank 112 are areas in which data is stored, and may be, for example, a unit of a memory area constituting a DRAM. The first memory bank 111 and the second memory bank 112 are units that can be independently activated, and have the same data bus width as the data input/output line in the PIM device 100 . can In one example, the first memory bank 111 and the second memory bank 112 may operate in an interleaving manner in which activation operations are performed in parallel while other memory banks are being accessed. The first memory bank 111 and the second memory bank 112 include at least one memory cell array configured by disposing memory unit cells at intersections of a plurality of rows and columns, respectively. may include

Although not shown in the drawing, a core may be disposed in an area adjacent to the first memory bank 111 and the second memory bank 112 . The core may include X-decoders XDEC and Y-decoders/input/output circuits YDEC/IO. The X-decoder (XDEC) is also called a word line decoder or a row decoder. The X-decoder XDEC receives the row address ADD_R transmitted from the PIM controller 200 and decodes the received row address ADD_R to enable the selected row of the selected memory bank. Each of the Y-decoders/input/output circuits YDEC/IO may include a Y-decoder YDEC and an input/output circuit IO. The Y-decoder (YDEC) is also called a bitline decoder or a column decoder. The Y-decoder YDEC receives the column address ADDR_C transmitted from the PIM controller 200 , decodes the received column address ADDR_C, and reads selected columns among columns of a selected row in the selected memory bank enable The input/output circuit IO may include an input/output sense amplifier for sensing and amplifying read data in a read operation for the first memory bank 111 and the second memory bank 112 . Also, the input/output circuit IO may include a write driver for driving write data in a write operation for the first memory bank 111 and the second memory bank 112 .

The interface 131 of the PIM device 100 receives a memory command M_CMD, MAC commands MAC_CMDs, a bank selection signal BS, and row/column addresses ADDR_R/ADDR_C from the PIM controller 200 . receive The interface 131 transmits the received memory command M_CMD to the first memory bank 111 or the second memory bank 112 together with the bank select signal BS and the row/column addresses ADDR_R/ADDR_C. . The interface 131 transmits the received MAC commands MAC_CMDs to the first memory bank 111 , the second memory bank 112 , and the MAC operator 120 . In this case, the interface 131 may transmit the bank selection signal BS and the row/column addresses ADDR_R/ADDR_C to the first memory bank 111 and the second memory bank 112 together. The data input/output pad 132 of the PIM device 100 communicates data between the first and second memory banks 111 and 112 in the PIM device 100 and the MAC operator 120 and the outside of the PIM device 100 . It functions as a terminal. Here, the outside of the PIM device 100 may be the PIM controller 200 constituting the PIM system 10 or a host outside the PIM system 10 . Accordingly, data transmitted from the host or the PIM controller 200 may be input into the PIM device 100 through the data input/output pad 132 . Also, data in the PIM device 100 may be transmitted to the outside of the PIM device 100 through the data input/output pad 132 .

The PIM controller 200 controls the operation of the PIM device 100 . In an example, the PIM controller 200 may control the memory mode operation of the PIM device 100 or may control the MAC mode operation. When the PIM controller 200 controls the PIM device 100 to operate in the memory mode, the PIM device 100 reads data and writes data to the first and second memory banks 111 and 112 . An action may be performed. When the PIM controller 200 controls the PIM device 100 to operate in the MAC mode, a MAC operation operation by the MAC operator 120 may be performed in the PIM device 100 . When the PIM controller 200 controls the PIM device 100 to operate in the MAC mode, the PIM device 100 writes data to the first memory bank 111 and the second memory bank 112 for MAC operation. and a lead operation may be performed.

The PIM controller 200 may include a command queue 210 , a scheduler 220 , a memory command generator 230 , a MAC command generator 240 , and an address generator 250 . The command queue 210 receives and stores a command CMD transmitted from the outside of the PIM system 10 , for example, a host. The command queue 210 may transmit information about the storage state of the command CMD to the scheduler 220 whenever the command CMD is stored. The commands CMD stored in the command queue 210 are transmitted to the memory command generator 230 or the MAC command generator 240 in an order determined by the scheduler 220 . When the command CMD output from the command queue 210 is a command for requesting the memory mode operation of the PIM device 100 , the command queue 210 transmits the command CMD to the memory command generator 230 . On the other hand, when the command CMD output from the command queue 210 is a command for requesting the MAC mode operation of the PIM device 100 , the command queue 210 transmits the command CMD to the MAC command generator 240 . do. Information on which mode operation the command CMD requests may be provided from the scheduler 220 .

The scheduler 220 adjusts the timing at which the command CMD stored in the command queue 210 is output from the command queue 210 . To this end, the scheduler 220 analyzes information about the storage state of the command CMD transmitted from the command queue 210 , and readjusts the command CMD processing order so that the commands CMD are processed in an appropriate order. The scheduler 220 transmits information about whether the command CMD output from the command queue 210 requests the memory mode operation or the MAC mode operation of the PIM device 100 to the command queue 210 . can be transmitted To this end, the scheduler 220 may include a mode selector 221 . The mode selector 221 is a mode selection signal regarding whether the command CMD stored in the command queue 210 is a command for requesting a memory mode operation of the PIM device 100 or a command for requesting a MAC mode operation. may be transmitted to the command queue 210 .

The memory command generator 230 receives a command CMD for requesting the memory mode operation of the PIM device 100 from the command queue 210 . The memory command generator 230 decodes the transmitted command CMD to generate and output the memory command M_CMD. The memory command M_CMD output from the memory command generator 230 is transmitted to the PIM device 100 . In an example, the memory command M_CMD may include a memory read command and a memory write command. When a memory read command is output from the memory command generator 230 , the PIM device 100 performs a data read operation on the first memory bank 111 or the second memory bank 112 . Data read from the PIM device 100 is transmitted to the outside of the PIM device 100 through the data input/output line 132 . The read data output from the PIM device 100 may be transmitted to the host through the PIM controller 200 . When a memory write command is output from the memory command generator 230 , the PIM device 100 performs a data write operation on the first memory bank 111 or the second memory bank 112 . In this case, data to be written to the PIM device 100 is transmitted from the host to the PIM device 100 through the PIM controller 200 . The write data transmitted to the PIM device 100 is transmitted to the first memory bank 111 or the second memory bank 112 through the data input/output line 132 .

The MAC command generator 240 receives a command CMD for requesting the MAC mode operation of the PIM device 100 from the command queue 210 . The MAC command generator 240 decodes the transmitted command CMD to generate and output MAC commands MAC_CMDs. MAC commands MAC_CMDs output from the MAC command generator 240 are transmitted to the PIM device 100 . The data read operation for the first memory bank 111 and the second memory bank 112 of the PIM device 100 and the MAC operator 120 by the MAC commands MAC_CMDs transmitted from the MAC command generator 240 ) of the MAC operation is controlled. The MAC commands (MAC_CMDs) and the MAC mode operation of the PIM device 100 will be described in more detail with reference to FIG. The address generator 250 generates a bank selection signal BS for selecting one of the first memory bank 111 and the second memory bank 112 and transmits the generated bank selection signal BS to the PIM device 100 . In addition, the address generator 250 generates a row address ADDR_R and a column address ADDR/C designating regions to be accessed of the first memory bank 111 and the second memory bank 112 to generate the PIM device 100 . ) is sent to

FIG. 2 shows MAC commands MAC_CMDs generated from the MAC command generator 240 constituting the PIM controller 200 in the PIM system 10 according to the first embodiment of the present disclosure. As shown in FIG. 2 , the MAC commands MAC_CMDs may include first to sixth MAC command signals. In an example, the first MAC command signal may be a first MAC read signal MAC_RD_BK0. The second MAC command signal may be a second MAC read signal MAC_RD_BK1. The third MAC command signal may be the first MAC input latch signal MAC_L1. The fourth MAC command signal may be a second MAC input latch signal MAC_L2. The fifth MAC command signal may be a MAC output latch signal MAC_L3. In addition, the sixth MAC command signal may be a MAC latch reset signal MAC_L_RST.

The first MAC read signal MAC_RD_BK0 controls an operation of reading first data, for example, weight data, from the first memory bank 111 and transmitting the read data to the MAC operator 120 . The second MAC read signal MAC_RD_BK1 controls an operation of reading second data, for example, vector data, from the second memory bank 112 and transmitting the read signal to the MAC operator 120 . The first MAC input latch signal MAC_L1 controls an input latch operation for weight data transmitted from the first memory bank 111 to the MAC operator 120 . The second MAC input latch signal MAC_L2 controls an input latch operation for vector data transmitted from the second memory bank 112 to the MAC operator 120 . When an input latch operation is performed on the weight data and the vector data, the MAC operator 120 performs a MAC operation and generates MAC result data as a result of the operation. The MAC output latch signal MAC_L3 controls an output latch operation for MAC result data generated in the MAC operator 120 . In addition, the MAC latch reset signal MAC_L_RST controls an output operation of the MAC result data generated in the MAC operator 120 and a reset operation of the output latch in the MAC operator 120 .

The PIM system 10 according to the present example is configured to perform a deterministic MAC operation operation. The term "deterministic MAC operation operation" used in the present disclosure may be defined as that the MAC operation operation performed in the PIM system 10 is predetermined and performed during a fixed time. Accordingly, each of the MAC commands MAC_CMDs transmitted from the PIM controller 200 to the PIM device 100 is sequentially generated at predetermined time intervals. Therefore, the PIM controller 200 transmits a separate operation completion signal from the PIM device 100 to the PIM controller 200 to generate each of the MAC commands (MAC_CMDs) for controlling the MAC operation by the PIM device 100 . do not ask to send In an example, for the deterministic MAC operation, a latency for each of the MAC operation operations performed by each of the MAC commands MAC_CMDs in the PIM device 100 may be fixedly set. In addition, each of the MAC commands MAC_CMDs output from the PIM controller 200 may be sequentially generated at a set latency interval.

3 shows an example of the MAC operator 120 configuring the PIM device 100 in the PIM system 10 according to the first embodiment of the present disclosure. Referring to FIG. 3 , the MAC operator 120 may include a data input unit 121 , a MAC circuit 122 , and a data output unit 123 . The data input unit 121 may include a first input latch 121-1 and a second input latch 121-2. The MAC circuit 122 may include a multiplication logic 122-1 and an addition logic 122-2. The data output unit 123 may include an output latch 123-1, a transfer gate 123-2, a delay circuit 123-3, and an inverter 123-4. In an example, the first input latch 121-1, the second input latch 121-2, and the output latch 123-1 may be configured as flip-flops.

The data input unit 121 of the MAC operator 120 synchronizes the first data DA1 transmitted from the first memory bank 111 through the data transmission line with the first MAC input latch signal MAC_L1 to the MAC circuit ( 122) is entered. In addition, the data input unit 121 synchronizes the second data DA2 transmitted from the second memory bank 112 through the data transmission line to the second MAC input latch signal MAC_L1 to be input to the MAC circuit 122 . . The MAC operator 120 of the PIM device 100 sequentially receives the first MAC input latch signal MAC_L1 and the second MAC input latch signal MAC_L2 from the MAC command generator 240 of the PIM controller 200 at regular time intervals. ), the first data DA1 is first input to the MAC circuit 122 in the MAC operator 120 , and then the second data DA2 is input to the MAC circuit 122 .

The MAC circuit 122 performs a MAC operation on the first data DA1 and the second data DA2 input through the data input unit 121 . The multiplication logic 122-1 of the MAC circuit 122 may include a plurality of multipliers 122-11. Each of the multipliers 122-11 performs multiplication operations on the first data DA1 and the second data DA2 inputted through the first input latch 121-1 and the second input latch 121-2, respectively. , and output the resulting data. Bit values constituting the first data DA1 are divided and input to each of the multipliers 122 - 11 . Similarly, bit values constituting the second data DA2 are also divided and input to each of the multipliers 122 - 11 . For example, when each of the first data DA1 and the second data DA2 is configured as an N-bit binary stream, each of the M multipliers 122-11 includes (N/M)-bit first data DA1. and second data DA2 may be input.

The addition logic 122 - 2 of the MAC circuit 122 may include a plurality of adders 122 - 21 . Although not shown in the drawing, the plurality of adders 122-21 may be arranged in a tree structure having a plurality of stages. Each of the adders 122-21 of the first stage receives multiplication result data from two multipliers 122-11 among the multipliers 122-11 of the multiplication logic 122-1, and performs an addition operation. After that, the result data is output. Each of the adders 122-21 of the second stage receives addition result data from two adders 122-21 among the adders 122-21 of the first stage, performs an addition operation, and then performs the result data outputs The adder 122-21 of the last stage receives addition result data from the two adders 122-21 of the previous stage, performs an addition operation, and outputs the result data. Although not shown in the figure, the addition logic 122-2 is applied to the MAC result data DA_MAC output from the last stage adder 122-21 and the output latch 123-1 of the data output unit 123. It may further include an adder for cumulatively adding the previously stored MAC result data DA_MAC.

The data output unit 123 outputs the MAC result data DA_MAC output from the MAC circuit 122 to the data transmission line. Specifically, the output latch 123 - 1 of the data output unit 123 latches and outputs the MAC result data DA_MAC output from the MAC circuit 122 in synchronization with the MAC output latch signal MAC_L3 . The MAC result data DA_MAC output from the output latch 123 - 1 may be fed back to the MAC circuit 122 for a cumulative addition operation. Also, the MAC result data DA_MAC may be input to the transfer gate 123 - 2 for output to the data transmission line. The output latch 123-1 is initialized when the latch reset signal LATCH_RST is input, and thus all data latched in the output latch 123-1 is removed. In an example, the latch reset signal LATCH_RST may be activated by the generation of the MAC latch reset signal MAC_L_RST to be input to the output latch 123 - 1 .

The MAC latch reset signal MAC_L_RST transmitted from the MAC command generator 240 of the PIM controller 200 to the data output unit 123 of the MAC operator 120 is a transfer gate 123 - 2 of the data output unit 123 . ), the delay circuit 123-3, and the inverter 123-4. The inverter 123-4 inverts the logic level of the MAC latch reset signal MAC_L_RST to input the inverted buffer to the transfer gate 123-2. The transfer gate 123-2 outputs the MAC result data DA_MAC transmitted from the output latch 123-1 to the data transmission line in response to the MAC latch reset signal MAC_L_RST. The delay circuit 123-3 delays the input MAC latch reset signal MAC_L_RST by a predetermined time point and then outputs it as the latch control signal PINSTB.

4 shows an example of a MAC operation performed in the PIM system 10 according to the first embodiment of the present disclosure. As shown in FIG. 4 , the MAC operation performed by the PIM system 10 is performed through a matrix operation. Specifically, the PIM device 100, under the control of the PIM controller 200, M rows and N columns (MXN) (M and N are natural numbers), for example, a weight matrix of 8 rows and 8 columns (8X8) (WEIGHT MATRIX) ) and a vector matrix (VECTOR MATRIX) of N rows and 1 column (NX1), for example, 8 rows and 1 column (8X1). The elements W0.1, ..., and W7.7 constituting the weight matrix correspond to the first data DA1 input from the first memory bank 111 to the MAC operator 120 . The elements X0.0, ..., X7.0 constituting the vector matrix correspond to the second data DA2 input from the second memory bank 112 to the MAC operator 120 . Each of the elements (W0.1, ..., W7.7) constituting the weight matrix and each of the elements (X0.0, ..., X7.0) constituting the vector matrix are binary streams having a plurality of bit values. (binary stream) can be composed. The number of bits of each of the elements (W0.1, ..., W7.7) constituting the weight matrix is the same as the number of bits of each of the elements (X0.0, ..., X7.0) constituting the vector matrix.

Such a multiplication operation of a weight matrix and a vector matrix may conform to the structure of a multi-layer perceptron (MLP) neural network. In general, an MLP neural network for performing deep learning includes an input layer, a plurality of, for example, at least three or more hidden layers, and an output layer. The multiplication operation of the weight matrix and the vector matrix shown in FIG. 4 , that is, the MAC operation corresponds to an operation in one of the hidden layers. For the first hidden layer, MAC operation is performed using input vector data. However, MAC operation in the hidden layer from the second hidden layer to the last hidden layer is performed using the operation result of the previous hidden layer as vector data.

FIG. 5 is a flowchart illustrating a process of performing the MAC operation process described with reference to FIG. 4 in the PIM system 10 according to the first embodiment of the present disclosure. 6 to 12 are block diagrams for explaining processes of performing the MAC operation of FIG. 4 in the PIM system 10 according to the first embodiment of the present disclosure. Referring to FIG. 5 together with FIGS. 6 to 12 , before performing the MAC operation, first data, that is, weight data, is written to the first memory bank 111 in step 301 . Accordingly, weight data is stored in the first memory bank 111 of the PIM device 100 . In this example, it is assumed that the weight data are elements (W0.0, ..., W7.7) constituting the weight matrix of FIG. 4 .

In step 302, it is determined whether there is an inference request. The inference request may be transmitted from the outside of the PIM system 10 to the PIM controller 200 . In one example, if there is no inference request, the PIM system 10 may maintain a standby state until the inference request is transmitted. In another example, if there is no inference request, the PIM system 10 may perform an operation other than the MAC operation operation, for example, a memory mode operation such as data read/write, until the inference request is transmitted. . In this example, it is assumed that the second data, ie, vector data, is transmitted together with the inference request. Also, in this example, it is assumed that the vector data are elements (X0.0, ..., X7.0) constituting the vector matrix of FIG. 4 . When the inference request is transmitted in step 302 , in step 303 , the PIM controller 200 writes the vector data transmitted along with the inference request to the second memory bank 112 . Accordingly, vector data is stored in the second memory bank 112 of the PIM device 100 .

In step 304 , as shown in FIG. 6 , the MAC command generator 240 of the PIM controller 200 generates a first MAC read signal MAC_RD_BK0 and transmits it to the PIM device 100 . At this time, the address generator 250 of the PIM controller 200 generates a bank selection signal BS and row/column addresses ADDR_R/ADDR_C and transmits them to the PIM device 100 . The bank selection signal BS designates the first memory bank 111 among the first memory bank 111 and the second memory bank 112 . Accordingly, the first MAC read signal MAC_RD_BK0 controls a data read operation for the first memory bank 111 of the PIM device 100 . The first memory bank 111, in response to the first MAC read signal MAC_RD_BK0, is an element in the first row of the weight matrix among the weight data stored in the area designated by the row/column addresses ADDR_R/ADDR_C The values W0.0, ..., W0.7 are transmitted to the MAC operator 120 . In one example, data transmission from the first memory bank 111 to the MAC operator 120 is performed through a global input/output (GIO) line that provides a data transmission path within the PIM device 100 . can In another example, data transmission from the first memory bank 111 to the MAC operator 120 includes a first bank input/output (Bank Input/Output) for only data transmission between the first memory bank 111 and the MAC operator 120 ; Hereinafter, BIO) line may be used.

In step 305 , as shown in FIG. 7 , the MAC command generator 240 of the PIM controller 200 generates a second MAC read signal MAC_RD_BK1 and transmits it to the PIM device 100 . At this time, the address generator 250 of the PIM controller 200 generates a bank selection signal BS designating the second memory bank 112 and row/column addresses ADDR_R/ADDR_C to the PIM device 100 . send. The second MAC read signal MAC_RD_BK1 controls a data read operation for the second memory bank 112 of the PIM device 100 . In response to the second MAC read signal MAC_RD_BK10, the second memory bank 112 stores vector data stored in the area designated by the row/column addresses ADDR_R/ADDR_C, that is, the element in the first column of the vector matrix. (X0.0, ..., X7.0) is transmitted to the MAC operator (120). In one example, data transmission from the second memory bank 112 to the MAC operator 120 may be performed through a GIO line in the PIM device 100 . In another example, data transfer from the second memory bank 112 to the MAC operator 120 may be performed through a second BIO line for only data transfer between the second memory bank 112 and the MAC operator 120 .

In step 306 , as shown in FIG. 8 , the MAC command generator 240 of the PIM controller 200 generates a first MAC input latch signal MAC_L1 and transmits it to the PIM device 100 . The first MAC input latch signal MAC_L1 controls an input latch operation of the first data to the MAC operator 120 of the PIM device 100 . By this input latch operation, the elements W0.0, ..., W0.7 in the first row of the weight matrix are input to the MAC circuit 122 of the MAC operator 120 as shown in FIG. 10 . . The MAC circuit 122 includes a number equal to the number of columns of the weight matrix, ie, eight multipliers 122-11. Each of the elements W0.0, ..., W0.7 in the first row of the weight matrix is input to each of the multipliers 122-11.

In step 307 , as shown in FIG. 9 , the MAC command generator 240 of the PIM controller 200 generates a second MAC input latch signal MAC_L2 and transmits it to the PIM device 100 . The second MAC input latch signal MAC_L2 controls an input latch operation of the second data to the MAC operator 120 of the PIM device 100 . By this input latch operation, the elements X0.0, ..., X7.0 in the first column of the vector matrix are input to the MAC circuit 122 of the MAC operator 120 as shown in FIG. 10 . Each of the elements X0.0, ..., X7.0 in the first column of the vector matrix is input to each of the multipliers 122-11.

In step 308, the MAC circuit 122 of the MAC operator 120 performs a MAC operation on the R-th row of the input weight matrix and the first column of the vector matrix. R=1 is set as an initial value, and MAC operation is performed on the first row of the weight matrix and the first column of the vector matrix. Specifically, each of the multipliers 122-11 of the multiplication logic 122-1 performs a multiplication operation on input data, and inputs the result data to the addition logic 122-2. The addition logic 122-2 includes four adders 122-21A disposed in the first stage, two adders 122-21B disposed in the second stage, and the third stage. Adders 122-21C are included.

Each of the adders 122-21A of the first stage receives output data from two multipliers 122-11 among the multipliers 122-11, adds them, and outputs the received data. Each of the adders 122-21B of the second stage receives, adds, and outputs the output data of the two adders 122-21A among the adders 122-21A of the first stage. The adder 122-21C of the third stage receives, adds, and outputs the output data of the adders 122-21B of the second stage. The output data output from the addition logic 122-2 is MAC operation result data of the first row of the weight matrix and the vector matrix. Accordingly, the output data is the element (MAC0) in the first row and first column of the MAC result matrix of 8 rows and 1 column in which the elements MAC0.0, ..., MAC7.0 are composed of the MAC result data shown in FIG. .0) is configured. The output data MAC0.0 output from the addition logic 122-2 is an output latch 123-1 disposed in the data output unit 123 of the MAC operator 120, as described with reference to FIG. 3 . is entered as

In step 309 , as shown in FIG. 11 , the MAC command generator 240 of the PIM controller 200 generates a MAC output latch signal MAC_L3 and transmits it to the PIM device 100 . The MAC output latch signal MAC_L3 controls an output latch operation of the MAC result data MAC0.0 in the MAC operator 120 of the PIM device 100 . By this output latch operation, as described with reference to FIG. 3 , the MAC result data MAC0.0 input from the MAC circuit 122 of the MAC operator 120 is synchronized with the MAC output latch signal MAC_L3. output from the output latch 123-1. The MAC result data MAC0.0 output from the output latch 123-1 is input to the transfer gate 123-2 of the data output unit 123.

In step 310 , as shown in FIG. 12 , the MAC command generator 240 of the PIM controller 200 generates a MAC latch reset signal MAC_L_RST and transmits it to the PIM device 100 . The MAC latch reset signal MAC_L_RST controls the output operation of the MAC result data MAC0.0 and the reset operation of the output latch by the MAC operator 120 of the PIM device 100 . As described with reference to FIG. 3 , the transfer gate 123-2 receiving the MAC result data MAC0.0 from the output latch 123-1 of the data output unit 123 of the MAC operator 120, The MAC result data MAC0.0 is output in synchronization with the MAC latch reset signal MAC_L_RST. In one example, the MAC result data MAC0.0 output from the MAC operator 120 is the first memory bank 111 or the second memory through the first BIO line or the second BIO line in the PIM device 100 . It may be stored in the bank 112 .

In step 311, "R" which is the number of rows of the weight matrix on which the MAC operation is performed is increased by one. As the MAC operation is performed on the first row among the first to eighth rows of the weight matrix so far, the number of rows of the weight matrix is changed from the first row to the second row in step 311 . In step 312, it is determined whether the number of rows of the weight matrix changed by step 311 is greater than the last row, that is, the eighth row of the weight matrix. Since the number of rows of the weight matrix changed by step 311 is the second row, the determination in step 312 determines that the number of rows of the weight matrix changed by step 311 is smaller than the eighth row, and the process of step 304 is performed again.

When fed back from step 312, the same processes of steps 304 to 310 as described above are performed except that the number of rows of the weight matrix on which the MAC operation is performed is different. That is, as the number of rows of the weight matrix is increased to the second row in step 311, there is a difference only in that the MAC operation of the second row and the vector matrix is performed instead of the first row of the weight matrix. The process of performing steps 304 to 311 after being fed back in step 312 is performed until the number of all rows of the weight matrix, that is, the MAC operation of the first to eighth rows and the vector matrix is finished. When the MAC operation on the 8th row of the weight matrix is completed and the number of rows of the weight matrix is changed from the 8th row to the 9th row in step 311, the ninth row, which is the number of rows of the weight matrix, is the last row in the judgment of step 312. Since it is greater than 8 lines, the MAC operation is terminated.

13 shows another example of an operation performed in the PIM system 10 according to the first embodiment of the present disclosure. As shown in FIG. 13 , the operation performed by the PIM system 10 further includes an addition operation of the MAC calculated MAC result matrix (MAC RESULT MATRIX) and the bias matrix (BIAS MATRIX). Specifically, as described with reference to FIG. 4 , the PIM device 100, under the control of the PIM controller 200 , includes, for example, a weight matrix of 8 rows and 8 columns (8X8) and, for example, 8 rows and A matrix multiplication operation is performed on the vector matrix (VECTOR MATRIX) of one column (8X1). As a result of the matrix multiplication operation of the weight matrix and the vector matrix, a MAC result matrix (MAC RESULT MATRIX) of 8 rows and 1 column (8X1) consisting of 8 elements (MAC0.0, ..., MAC7.0) is generated. The MAC result matrix of 8 rows and 1 column (8X1) is subjected to matrix addition operation with the bias matrix (BIAS MATRIX) of 8 rows and 1 column (8X1). The bias matrix has elements B0.1, ..., B7.7 composed of bias data. This bias data can be arbitrarily set to reduce the error of the MAC result matrix. As a result of matrix addition of MAC result matrix and bias matrix, a biased result matrix (BIASED RESULT MATRIX) of 8 rows and 1 column (8X1) consisting of 8 elements (Y0.0, ..., Y7.0) is generated. do.

FIG. 14 is a flowchart illustrating a process of performing the calculation process described with reference to FIG. 13 in the PIM system 10 according to the first embodiment of the present disclosure. And FIG. 15 shows an example of the configuration of the MAC operator 120-1 for implementing the operation of FIG. 13 in the PIM system 10 according to the first embodiment of the present disclosure. In FIG. 15 , the same reference numerals as those of FIG. 3 denote the same components, and repeated descriptions will be omitted below. Referring to FIG. 14 , in step 321 , in order to perform an operation in the PIM device 100 , first data, that is, weight data, is written to the first memory bank 111 . Accordingly, weight data is stored in the first memory bank 111 of the PIM device 100 . In this example, it is assumed that the weight data are elements (W0.0, ..., W7.7) constituting the weight matrix of FIG. 13 .

In step 322, it is determined whether there is an inference request. The inference request may be transmitted from the outside of the PIM system 10 to the PIM controller 200 . In one example, if there is no inference request, the PIM system 10 may maintain a standby state until the inference request is transmitted. In another example, if there is no inference request, the PIM system 10 may perform an operation other than the MAC operation operation, for example, a memory mode operation such as data read/write, until the inference request is transmitted. . In this example, it is assumed that the second data, ie, vector data, is transmitted together with the inference request. Also, in this example, it is assumed that the vector data are elements (X0.0, ..., X7.0) constituting the vector matrix of FIG. 13 . When the inference request is transmitted in step 322 , in step 323 , the PIM controller 200 writes the vector data transmitted along with the inference request to the second memory bank 112 . Accordingly, vector data is stored in the second memory bank 112 of the PIM device 100 .

In step 324, the output latch of the MAC operator is initially set as bias data, and the initially set bias data is fed back to the cumulative adder of the MAC operator. This process is a process for the matrix addition operation of the bias matrix to the MAC result matrix in the operation described with reference to FIG. 13 . That is, as shown in FIG. 15 , the output latch 123-1 in the data output unit 123-A of the MAC operator 120-1 is initially set to bias data. Since the matrix multiplication operation is performed on the first row of the weight matrix, the first row and first column elements B0.0 of the bias matrix are initially set in the output latch 123-1 as bias data. The output latch 123-1 outputs the bias data B0.0, and the bias data B0.0 output from the output latch 123-1 is the accumulation adder 122 of the addition logic 122-2. -21D) is entered.

In one example, in order to output the bias data B0,0 from the output latch 123-1 and feed it back to the accumulation adder 122-21D, the MAC command generator 240 of the PIM controller 200 generates a MAC output latch signal. (MAC_L3) may be transmitted to the MAC operator 120 - 1 of the PIM device 100 . The accumulation adder 122-21D of the MAC operator 120-1 performs the MAC result output from the adder 122-21C disposed in the last stage of the addition logic 122-2 when the MAC operation is subsequently performed. The data MAC0.0 and the fed back bias data B0.0 are added, and the resulting biased data Y0.0 is output and input to the output latch 123-1. The biased data Y0.0 may be output from the output latch 123 - 1 in synchronization with the subsequently transmitted MAC output latch signal MAC_L3 .

In step 325 , the MAC command generator 240 of the PIM controller 200 generates a first MAC read signal MAC_RD_BK0 and transmits it to the PIM device 100 . Also, the address generator 250 of the PIM controller 200 generates a bank selection signal BS and row/column addresses ADDR_R/ADDR_C and transmits them to the PIM device 100 . The process of step 325 is the same as the process described with reference to FIG. 6 . In operation 326 , the MAC command generator 240 of the PIM controller 200 generates a second MAC read signal MAC_RD_BK1 and transmits it to the PIM device 100 . In addition, the address generator 250 of the PIM controller 200 generates a bank selection signal BS designating the second memory bank 112 and row/column addresses ADDR_R/ADDR_C to the PIM device 100 . send. The process of step 326 is the same as the process described with reference to FIG. 7 .

In step 327 , the MAC command generator 240 of the PIM controller 200 generates a first MAC input latch signal MAC_L1 and transmits it to the PIM device 100 . The process of step 327 is the same as the process described with reference to FIG. 8 . The first MAC input latch signal MAC_L1 controls an input latch operation of the first data to the MAC operator 120 of the PIM device 100 . The operation of latching the input of the first data is the same as the process described with reference to FIG. 10 . In step 328 , the MAC command generator 240 of the PIM controller 200 generates a second MAC input latch signal MAC_L2 and transmits it to the PIM device 100 . The process of step 328 is the same as the process described with reference to FIG. 9 . The second MAC input latch signal MAC_L2 controls an input latch operation of the second data to the MAC operator 120 of the PIM device 100 . The second data input latch operation is the same as described with reference to FIG. 10 .

In step 329, the MAC circuit 122 of the MAC operator 120 performs a MAC operation on the R-th row of the input weight matrix and the first column of the vector matrix. R=1 is set as an initial value, and MAC operation is performed on the first row of the weight matrix and the first column of the vector matrix. Specifically, each of the multipliers 122-11 of the multiplication logic 122-1 performs a multiplication operation on input data, and inputs the result data to the addition logic 122-2. The addition logic 122-2, as shown in FIG. 15, includes four adders 122-21A disposed in the first stage and two adders 122-21B disposed in the second stage; It includes an adder 122-21C disposed in the three-stage stage, and an accumulation adder 122-21D. The accumulation adder 122-21D adds and outputs the output data of the adder 122-21C and the feedback data fed back from the output latch 123-1. Output data output from the adder 122-21C of the addition logic 122-2 is matrix multiplication result data MAC0.0 of the first row of the weight matrix and the first column of the vector matrix. The accumulation adder 122-21D adds and outputs the matrix multiplication result data MAC0.0 output from the adder 122-21C and the bias data B0.0 fed back from the output latch 123-1. . The output data Y0.0 output from the accumulation adder 122-21D is input to the output latch 123-1 disposed in the data output unit 123-A of the MAC operator 120.

In step 330 , the MAC command generator 240 of the PIM controller 200 generates a MAC output latch signal MAC_L3 and transmits it to the PIM device 100 . The process of step 330 is the same as the process described with reference to FIG. 11 . The MAC output latch signal MAC_L3 controls an output latch operation of the MAC result data MAC0.0 in the MAC operator 120 of the PIM device 100 . By this output latch operation, the biased result data Y0.0 input from the MAC circuit 122 of the MAC operator 120 is synchronized with the MAC output latch signal MAC_L3 from the output latch 123-1. is output The biased result data Y0.0 output from the output latch 123-1 is input to the transfer gate 123-2.

In step 331 , the MAC command generator 240 of the PIM controller 200 generates a MAC latch reset signal MAC_L_RST and transmits it to the PIM device 100 . The process of step 331 is the same as the process described with reference to FIG. 12 . The MAC latch reset signal MAC_L_RST controls an output operation of the biased result data Y0.0 by the MAC operator 120 of the PIM device 100 and a reset operation of the output latch 123 - 1 . The transfer gate 123-2 receiving the bias result data Y0.0 from the output latch 123-1 of the data output unit 123-A of the MAC operator 120 receives the MAC latch reset signal MAC_L_RST ) and output the biased result data (Y0.0). In one example, the biased result data Y0.0 output from the MAC operator 120 is the first memory bank 111 or the second through the first BIO line or the second BIO line in the PIM device 100 . It may be stored in the memory bank 112 .

In step 332, "R" which is the number of rows of the weight matrix on which the MAC operation is performed is increased by one. As the MAC operation has been performed on the first row among the first to eighth rows of the weight matrix so far, the number of rows of the weight matrix is changed from the first row to the second row in step 332 . In step 333, it is determined whether the number of rows of the weight matrix changed by step 332 is greater than the last row, that is, the eighth row of the weight matrix. Since the number of rows of the weight matrix changed by step 332 is the second row, the determination in step 333 determines that the number of rows of the weight matrix changed by step 332 is smaller than the eighth row, and the process of step 324 is performed again.

When fed back from step 333, the same processes of steps 324 to 331 as described above are performed except that the number of rows of the weight matrix on which the MAC operation is performed is different. That is, as the number of rows of the weight matrix is increased to the second row in step 332, there is a difference in that a matrix multiplication operation is performed between the second row and the vector matrix instead of the first row of the weight matrix. There is a difference only in that the bias data initially set in the output latch 123-1 in step 324 is changed to the element B1.0 in the second row and first column of the bias matrix. The process of performing steps 324 to 332 after being fed back in step 333 is performed until the number of all rows of the weight matrix, that is, the MAC operation of the first to eighth rows and the vector matrix is finished. When the MAC operation on the 8th row of the weight matrix is finished and the number of rows of the weight matrix is changed from the 8th row to the 9th row in step 332, the ninth row, which is the number of rows of the weight matrix, is the last row in the judgment of step 333 Since it is greater than 8 lines, the MAC operation is terminated.

FIG. 16 shows another example of an operation performed in the PIM system 10 according to the first embodiment of the present disclosure. As shown in FIG. 16 , the operation performed by the PIM system 10 further includes a process of applying an activation function (ACTIVE FUNCTION) to a biased result matrix (BIASED RESULT MATRIX). Specifically, as described with reference to FIG. 13 , the PIM device 100, under the control of the PIM controller 200 , includes, for example, a weight matrix of 8 rows and 8 columns (8X8) and, for example, 8 rows and A matrix multiplication operation is performed on the vector matrix (VECTOR MATRIX) of one column (8X1). Then, a matrix addition operation of the resulting MAC result matrix (MAC RESULT MATRIX) and the bias matrix (BIAS MATRIX) is performed to generate a biased result matrix (BIASED RESULT MATRIX).

The biased result matrix is output after the ACTIVE FUNCTION is applied. The activation function refers to a function used to calculate a unique output value by comparing the MAC calculated value in the MLP neural network with a threshold value. In an example, the activation function may be a unipolar activation function capable of outputting only a positive value or a bipolar activation function capable of outputting a negative value as well. In one example, the activation function may include a Sigmoid function, a Tanh function, a rectified linear (ReLU) function, a Leaky ReLU function, an identity function, and a Max Out function. there is.

FIG. 17 is a flowchart illustrating a process of performing the calculation process described with reference to FIG. 16 in the PIM system 10 according to the first embodiment of the present disclosure. In addition, FIG. 18 shows an example of the configuration of the MAC operator 120-2 for implementing the operation of FIG. 16 in the PIM system 10 according to the first embodiment of the present disclosure. In FIG. 18, the same reference numerals as those of FIG. 3 denote the same components, and repeated descriptions will be omitted below. Referring to FIG. 17 , in step 341 , first data, that is, weight data, is written to the first memory bank 111 in order to perform a MAC operation. Accordingly, weight data is stored in the first memory bank 111 of the PIM device 100 . In this example, it is assumed that the weight data are elements (W0.0, ..., W7.7) constituting the weight matrix of FIG. 16 .

In step 342, it is determined whether there is an inference request. The inference request may be transmitted from the outside of the PIM system 10 to the PIM controller 200 . In one example, if there is no inference request, the PIM system 10 may maintain a standby state until the inference request is transmitted. In another example, if there is no inference request, the PIM system 10 may perform an operation other than the MAC operation operation, for example, a memory mode operation such as data read/write, until the inference request is transmitted. . In this example, it is assumed that the second data, ie, vector data, is transmitted together with the inference request. Also, in this example, it is assumed that the vector data are elements (X0.0, ..., X7.0) constituting the vector matrix of FIG. 16 . When the inference request is transmitted in step 342 , in step 343 , the PIM controller 200 writes the vector data transmitted along with the inference request to the second memory bank 112 . Accordingly, vector data is stored in the second memory bank 112 of the PIM device 100 .

In step 344, the output latch of the MAC operator is initially set as bias data, and the initially set bias data is fed back to the cumulative adder of the MAC operator. This process is a process for the matrix addition operation of the bias matrix to the MAC result matrix in the operation described with reference to FIG. 16 . That is, as shown in FIG. 18 , the output latch 123-1 in the data output unit 123-B of the MAC operator 120-2 is initially set to bias data. Since the matrix multiplication operation is performed on the first row of the weight matrix, the first row and first column elements B0.0 of the bias matrix are initially set in the output latch 123-1 as bias data. The output latch 123-1 outputs the bias data B0.0, and the bias data B0.0 output from the output latch 123-1 is the accumulation adder 122 of the addition logic 122-2. -21D) is entered.

In one example, in order to output the bias data B0,0 from the output latch 123-1 and feed it back to the accumulation adder 122-21D, the MAC command generator 240 of the PIM controller 200 generates a MAC output latch signal. (MAC_L3) may be transmitted to the MAC operator 120 - 2 of the PIM device 100 . The accumulation adder 122-21D of the MAC operator 120-2 is the MAC result output from the adder 122-21C disposed in the last stage of the addition logic 122-2 when the MAC operation is subsequently performed. The data MAC0.0 and the feedback bias data B0.0 are added, and the resultant biased data Y0.0 is output and input to the output latch 123-1. The biased data Y0.0 is placed in the data output unit 123-B of the MAC operator 120-2 from the output latch 123-1 in synchronization with the subsequently transmitted MAC output latch signal MAC_L3. may be inputted to the activation function logic 123 - 5 .

In step 345 , the MAC command generator 240 of the PIM controller 200 generates a first MAC read signal MAC_RD_BK0 and transmits it to the PIM device 100 . Also, the address generator 250 of the PIM controller 200 generates a bank selection signal BS and row/column addresses ADDR_R/ADDR_C and transmits them to the PIM device 100 . The process of step 345 is the same as the process described with reference to FIG. 6 . In step 346 , the MAC command generator 240 of the PIM controller 200 generates a second MAC read signal MAC_RD_BK1 and transmits it to the PIM device 100 . In addition, the address generator 250 of the PIM controller 200 generates a bank selection signal BS designating the second memory bank 112 and row/column addresses ADDR_R/ADDR_C to the PIM device 100 . send. The process of step 346 is the same as the process described with reference to FIG. 7 .

In step 347 , the MAC command generator 240 of the PIM controller 200 generates a first MAC input latch signal MAC_L1 and transmits it to the PIM device 100 . The process of step 347 is the same as the process described with reference to FIG. 8 . The first MAC input latch signal MAC_L1 controls an input latch operation of the first data to the MAC operator 120 of the PIM device 100 . The operation of latching the input of the first data is the same as the process described with reference to FIG. 10 . In step 348 , the MAC command generator 240 of the PIM controller 200 generates a second MAC input latch signal MAC_L2 and transmits it to the PIM device 100 . The process of step 348 is the same as the process described with reference to FIG. 9 . The second MAC input latch signal MAC_L2 controls an input latch operation of the second data to the MAC operator 120 of the PIM device 100 . The second data input latch operation is the same as described with reference to FIG. 10 .

In step 349, the MAC circuit 122 of the MAC operator 120 performs a MAC operation on the R-th row of the input weight matrix and the first column of the vector matrix. R=1 is set as an initial value, and MAC operation is performed on the first row of the weight matrix and the first column of the vector matrix. Specifically, each of the multipliers 122-11 of the multiplication logic 122-1 performs a multiplication operation on input data, and inputs the result data to the addition logic 122-2. The addition logic 122-2, as shown in FIG. 18, includes four adders 122-21A disposed in the first stage and two adders 122-21B disposed in the second stage; It includes an adder 122-21C disposed in the three-stage stage, and an accumulation adder 122-21D. The accumulation adder 122-21D adds and outputs the output data of the adder 122-21C and the feedback data fed back from the output latch 123-1. Output data output from the adder 122-21C of the addition logic 122-2 is matrix multiplication result data MAC0.0 of the first row of the weight matrix and the first column of the vector matrix. The accumulation adder 122-21D adds and outputs the matrix multiplication result data MAC0.0 output from the adder 122-21C and the bias data B0.0 fed back from the output latch 123-1. . The output data Y0.0 output from the accumulation adder 122-21D is input to the output latch 123-1 disposed in the data output unit 123-A of the MAC operator 120.

In step 350 , the MAC command generator 240 of the PIM controller 200 generates a MAC output latch signal MAC_L3 and transmits it to the PIM device 100 . The process of step 350 is the same as the process described with reference to FIG. 11 . The MAC output latch signal MAC_L3 controls an output latch operation of the output latch 123 - 1 in the MAC operator 120 of the PIM device 100 . By this output latch operation, the biased result data Y0.0 input from the MAC circuit 122 of the MAC operator 120 is synchronized with the MAC output latch signal MAC_L3 from the output latch 123-1. is output The biased result data Y0.0 output from the output latch 123-1 is input to the activation function logic 123-5. And in step 351, the activation function logic 123-5 applies the activation function to the input biased result data Y0.0 to generate a final output value and input it to the transfer gate (123-2 in FIG. 3). .

In step 352 , the MAC command generator 240 of the PIM controller 200 generates a MAC latch reset signal MAC_L_RST and transmits it to the PIM device 100 . The process of step 352 is the same as the process described with reference to FIG. 12 . The MAC latch reset signal MAC_L_RST controls the output operation of the final output value from the MAC operator 120 of the PIM device 100 and the reset operation of the output latch 123 - 1 . The transfer gate 123-2 receiving the final output value from the activation function logic 123-5 of the data output unit 123-B of the MAC operator 120 is synchronized with the MAC latch reset signal MAC_L_RST to obtain the final output value. outputs In one example, the final output value output from the MAC operator 120 is stored in the first memory bank 111 or the second memory bank 112 through the first BIO line or the second BIO line in the PIM device 100 . can be

In step 353, "R" which is the number of rows of the weight matrix on which the MAC operation is performed is increased by one. As the MAC operation is performed on the first row among the first to eighth rows of the weight matrix so far, the number of rows of the weight matrix is changed from the first row to the second row in step 353 . In step 354, it is determined whether the number of rows of the weight matrix changed by step 353 is greater than the last row, that is, the eighth row of the weight matrix. Since the number of rows of the weight matrix changed by step 353 is the second row, the determination in step 354 determines that the number of rows of the weight matrix changed by step 353 is smaller than the eighth row, and the process of step 344 is performed again.

Upon feedback from step 354, the same process as described above in steps 344 to 354 is performed except that the number of rows of the weight matrix on which the MAC operation is performed is different. That is, as the number of rows of the weight matrix is increased to the second row in step 353, there is a difference in that a matrix multiplication operation is performed between the second row and the vector matrix instead of the first row of the weight matrix. There is a difference only in that the bias data initially set in the output latch 123-1 in step 344 is changed to the element B1.0 in the second row and first column of the bias matrix. The process of performing steps 344 to 354 after being fed back in step 354 is performed until the number of all rows of the weight matrix, that is, the MAC operation of the first to eighth rows and the vector matrix is finished. When the MAC operation on the 8th row of the weight matrix is completed and the number of rows of the weight matrix is changed from the 8th row to the 9th row in step 354, the ninth row, which is the number of rows of the weight matrix, is the last row in the judgment of step 354. Since it is greater than 8 lines, the MAC operation is terminated.

19 schematically shows a block diagram of a PIM system 30 according to a second embodiment of the present disclosure. In FIG. 19, the same reference numerals as in FIG. 1 denote the same configuration. As shown in FIG. 19 , the PIM system 30 includes a PIM device 400 and a PIM controller 500 . The PIM device 400 includes a memory bank 411 , a global buffer 412 , a MAC OPERATOR 420 , an interface (I/F) 431 , and a data input/output pad (DQ) 432 . The memory bank 411 and the MAC operator 420 of the PIM device 400 constitute one MAC unit (MAC UNIT). In another example, the PIM device 400 may include a plurality of MAC units. The memory bank 411 is an area in which data is stored, and may be, for example, a unit of a memory area constituting a DRAM. The global buffer 412 is also an area in which data is stored, and may be formed of, for example, DRAM or SRAM. The memory bank 411 is a unit that can be independently activated, and may have the same data bus width as the data input/output line in the PIM device 400 . In one example, the memory bank 411 may operate in an interleaving manner in which activation operations are performed in parallel while another memory bank is being accessed. The memory bank 411 may include at least one memory cell array in which a memory unit cell is disposed at each of intersections of a plurality of rows and columns.

Although not shown in the drawing, a core may be disposed in an area adjacent to the memory bank 411 . The core may include X-decoders XDEC and Y-decoders/input/output circuits YDEC/IO. The X-decoder (XDEC) is also called a word line decoder or a row decoder. The X-decoder XDEC receives the row address ADD_R transmitted from the PIM controller 500 and decodes the received row address ADD_R to enable the selected row of the selected memory bank. Each of the Y-decoders/input/output circuits YDEC/IO may include a Y-decoder YDEC and an input/output circuit IO. The Y-decoder (YDEC) is also called a bitline decoder or a column decoder. The Y-decoder YDEC receives the column address ADDR_C transmitted from the PIM controller 500, decodes the received column address ADDR_C, and reads selected columns among columns of a selected row in the selected memory bank. enable The input/output circuit IO may include an input/output sense amplifier for sensing and amplifying read data in a read operation on the memory bank 411 . Also, the input/output circuit IO may include a write driver for driving write data in a write operation on the memory bank 411 .

The MAC operator 420 of the PIM device 400 is mostly the same as the MAC operator 120 described with reference to FIG. 3 . That is, as described with reference to FIG. 3 , the MAC operator 420 according to this example may include a data input unit 121 , a MAC circuit 122 , and a data output unit 123 . The data input unit 121 may include a first input latch 121-1 and a second input latch 121-2. The MAC circuit 122 may include a multiplication logic 122-1 and an addition logic 122-2. The data output unit 123 may include an output latch 123-1, a transfer gate 123-2, a delay circuit 123-3, and an inverter 123-4. In an example, the first input latch 121-1, the second input latch 121-2, and the output latch 123-1 may be configured as flip-flops.

3 in that the MAC operator 420 in this example simultaneously inputs the MAC input latch signal MAC_L1 to the clock terminals of the first input latch 121-1 and the second input latch 121-2, respectively. It is different from the MAC operator 120 of As will be described in detail below, weight data and vector data transmission to the MAC operator 420 within the PIM device 400 of the PIM system 30 according to the present example may be simultaneously performed. That is, the first data DA1 (weight data) and the second data are respectively provided to the first input latch 121-1 and the second input latch 121-2 constituting the data input unit 121 of the MAC operator 420 . (DA2) (vector data) is input simultaneously. Therefore, there is no need to apply a separate control signal to the clock terminal of the first input latch 121-1 and the clock terminal of the second input latch 121-2. Accordingly, in the MAC operator 420 of this example, the first The MAC input latch MAC_L1 is commonly input to the clock terminal of the input latch 121-1 and the clock terminal of the second input latch 121-2.

In another example, the MAC operator 420 of the PIM device 400 may be configured in the same manner as the MAC operator 120-1 described with reference to FIG. 15 in order to perform the operation shown in FIG. 13 . Also in this case, it should be noted that the MAC input latch MAC_L1 is commonly input to the clock terminal of the first input latch 121-1 and the clock terminal of the second input latch 121-2 constituting the data input unit 121. Except that, it has the same configuration as described with reference to FIG. 15 . In another example, the MAC operator 420 of the PIM device 400 may be configured the same as the MAC operator 120 - 2 described with reference to FIG. 18 to perform the operation shown in FIG. 16 . Also in this case, it should be noted that the MAC input latch MAC_L1 is commonly input to the clock terminal of the first input latch 121-1 and the clock terminal of the second input latch 121-2 constituting the data input unit 121. Except that, it has the same configuration as described with reference to FIG. 18 .

The interface 431 of the PIM device 400 receives a memory command M_CMD, MAC commands MAC_CMDs, a bank selection signal BS, and row/column addresses ADDR_R/ADDR_C from the PIM controller 500 . receive The interface 431 transmits the memory command M_CMD received from the PIM controller 500 to the memory bank 411 together with the bank selection signal BS and the row/column addresses ADDR_R/ADDR_C. The interface 431 transmits the MAC commands MAC_CMDs received from the PIM controller 500 to the memory bank 411 and the MAC operator 420 . In this case, the interface 431 may transmit the bank selection signal BS and the row/column addresses ADDR_R/ADDR_C together to the memory bank 411 . The data input/output pad 432 of the PIM device 400 includes a memory bank 411 in the PIM device 400 , a global buffer 412 , and data communication between the MAC operator 420 and the outside of the PIM device 400 . It functions as a terminal. Here, the outside of the PIM device 400 may be the PIM controller 500 constituting the PIM system 30 or a host outside the PIM system 30 . Accordingly, data transmitted from the host or the PIM controller 500 may be input into the PIM device 400 through the data input/output pad 432 . Also, data in the PIM device 400 may be transmitted to the outside of the PIM device 400 through the data input/output pad 432 .

The PIM controller 500 controls the operation of the PIM device 400 . In an example, the PIM controller 500 may control the memory mode operation of the PIM device 400 or may control the MAC mode operation. When the PIM controller 500 controls the PIM device 400 in the memory mode, the PIM device 500 may perform a data read operation and a data write operation on the memory bank 411 . When the PIM controller 500 controls the PIM device 400 in the MAC mode, a MAC operation operation by the MAC operator 420 may be performed in the PIM device 400 . When the PIM controller 500 controls the PIM device 400 to operate in the MAC mode, the PIM device 400 writes and reads data to the memory bank 411 and the global buffer 412 for MAC operation. This may be done.

The PIM controller 500 may include a command queue 210 , a scheduler 220 , a memory command generator 230 , a MAC command generator 540 , and an address generator 550 . The scheduler 220 may include a mode selector 221 . The command queue 210 receives and stores the command CMD transmitted from the outside of the PIM system 30 . The commands CMD stored in the command queue 210 are transmitted to the memory command generator 230 or the MAC command generator 540 in an order determined by the scheduler 220 . The scheduler 220 adjusts the timing at which the command CMD stored in the command queue 210 is output from the command queue 210 . The scheduler 210 receives a mode selection signal regarding whether the command CMD stored in the command queue 210 is a command for requesting a memory mode operation of the PIM device 400 or a command for requesting a MAC mode operation. and a mode selector 221 to generate. The memory command generator 230 receives a command CMD for requesting the memory mode operation of the PIM device 400 from the command queue 210 , and generates and outputs the memory command M_CMD. The command queue 210 , the scheduler 220 , the mode selector 221 , and the memory command generator 230 are the same as described with reference to FIG. 1 .

The MAC command generator 540 receives a command CMD for requesting the MAC mode operation of the PIM device 400 from the command queue 210 . The MAC command generator 540 decodes the transmitted command CMD to generate and output MAC commands MAC_CMDs. MAC commands MAC_CMDs output from the MAC command generator 540 are transmitted to the PIM device 400 . The data read operation for the memory bank 411 of the PIM device 400 and the MAC operation operation of the MAC operator 420 are controlled by the MAC commands MAC_CMDs transmitted from the MAC command generator 540 . The MAC commands (MAC_CMDs) and the MAC mode operation of the PIM device 400 will be described in more detail below with reference to FIG. 20 . The address generator 550 generates a bank selection signal BS for selecting the memory bank 411 and transmits it to the PIM device 400 . In addition, the address generator 550 generates a row address ADDR_R and a column address ADDR/C designating a region to be accessed of the memory bank 411 and transmits the generated row address ADDR/C to the PIM device 400 .

FIG. 20 shows MAC commands MAC_CMDs generated from the MAC command generator 540 constituting the PIM controller 500 in the PIM system 30 according to the second embodiment of the present disclosure. As shown in FIG. 20 , the MAC commands MAC_CMDs may include first to fourth MAC command signals. In an example, the first MAC command signal may be a MAC read signal MAC_RD_BK. The second MAC command signal may be a MAC input latch signal MAC_L1. The third MAC command signal may be a MAC output latch signal MAC_L3. In addition, the fourth MAC command signal may be a MAC latch reset signal MAC_L_RST.

The MAC read signal MAC_RD_BK controls an operation of reading first data, for example, weight data, from the memory bank 411 and transmitting the read signal to the MAC operator 420 . The MAC input latch signal MAC_L1 controls an input latch operation for weight data transmitted from the memory bank 411 to the MAC operator 420 . The MAC output latch signal MAC_L3 controls an output latch operation for the MAC result data generated in the MAC operator 420 . In addition, the MAC latch reset signal MAC_L_RST controls an output operation of the MAC result data generated in the MAC operator 420 and a reset operation of the output latch in the MAC operator 420 .

The PIM system 30 according to the present example is also configured to perform a deterministic MAC operation operation. Accordingly, each of the MAC commands MAC_CMDs transmitted from the PIM controller 500 to the PIM device 400 is sequentially generated at predetermined time intervals. Accordingly, the PIM controller 500 transmits a separate operation completion signal from the PIM device 400 to the PIM controller 500 to generate each of the MAC commands (MAC_CMDs) for controlling the MAC operation by the PIM device 400 . do not ask to send In one example, for the deterministic MAC operation, a latency for each of the MAC operation operations performed by each of the MAC commands MAC_CMDs in the PIM device 400 may be fixedly set. In addition, each of the MAC commands MAC_CMDs output from the PIM controller 500 may be sequentially generated at a set latency interval.

21 is a flowchart illustrating a process of performing the MAC operation process described with reference to FIG. 4 in the PIM system 30 according to the second embodiment of the present disclosure. 22 to 25 are block diagrams for explaining processes of performing the MAC operation of FIG. 4 in the PIM system 30 according to the second embodiment of the present disclosure. Referring to FIG. 21 together with FIGS. 22 to 25 , in step 361 , first data, ie, weight data, is written to the memory bank 411 in order to perform a MAC operation. Accordingly, the weight data is stored in the memory bank 411 of the PIM device 400 . In this example, it is assumed that the weight data are elements (W0.0, ..., W7.7) constituting the weight matrix of FIG. 4 .

In step 362, it is determined whether there is an inference request. The inference request may be transmitted from the outside of the PIM system 30 to the PIM controller 500 . In an example, if there is no inference request, the PIM system 30 may maintain a standby state until the inference request is transmitted. In another example, if there is no inference request, the PIM system 30 may perform an operation other than the MAC operation operation, for example, a memory mode operation such as data read/write, until the inference request is transmitted. . In this example, it is assumed that the second data, ie, vector data, is transmitted together with the inference request. Also, in this example, it is assumed that the vector data are elements (X0.0, ..., X7.0) constituting the vector matrix of FIG. 4 . When the inference request is transmitted in step 362 , the PIM controller 500 writes the vector data transmitted together with the inference request to the global buffer 412 in step 363 . Accordingly, vector data is stored in the global buffer 412 of the PIM device 400 .

In step 364 , as shown in FIG. 22 , the MAC command generator 540 of the PIM controller 500 generates a MAC read signal MAC_RD_BK and transmits it to the PIM device 400 . In this case, the address generator 550 of the PIM controller 500 generates row/column addresses ADDR_R/ADDR_C and transmits the generated row/column addresses ADDR_R/ADDR_C to the PIM device 400 . Although not shown in the drawing, when a plurality of memory banks are disposed in the PIM device 400 , the address generator 550 selects a bank designating the memory bank 411 among the plurality of memory banks in the PIM device 400 . signals can be transmitted together. The MAC read signal MAC_RD_BK transmitted to the PIM device 400 controls a data read operation for the memory bank 411 of the PIM device 400 . The memory bank 411 is, in response to the MAC read signal MAC_RD_BK, in the first row of the weight matrix among the weight data stored in the area designated by the row/column addresses ADDR_R/ADDR_C in the memory bank 411 The present elements (W0.0, ..., W0.7) are transmitted to the MAC operator 420 . In one example, data transfer from the memory bank 411 to the MAC operator 420 may be performed through a BIO line for data transfer only between the memory bank 411 and the MAC operator 420 .

On the other hand, the vector data (X0.0, ..., X7.0) stored in the global buffer 412 in synchronization with the clock to which the weight data is transmitted from the memory bank 411 to the MAC operator 420 also 420). To this end, a control signal for controlling a read operation on the global buffer 412 may be generated in synchronization with the transmission of the MAC read signal MAC_RD_BK from the MAC command generator 540 of the PIM controller 500 . Data transmission between the global buffer 412 and the MAC operator 420 may be performed through a GIO line. Accordingly, vector data transmission from the global buffer 412 to the MAC operator 420 may be performed independently of the weight data transmission from the memory bank 411 to the MAC operator 420 through the BIO line. In an example, transmission of weight data and transmission of vector data to the MAC operator 420 may be performed simultaneously through a BIO line and a GIO line, respectively.

In step 365 , as shown in FIG. 23 , the MAC command generator 540 of the PIM controller 500 generates a MAC input latch signal MAC_L1 and transmits it to the PIM device 400 . The MAC input latch signal MAC_L1 controls an input latch operation of weight data and an input latch operation of vector data to the MAC operator 420 of the PIM device 400 . By this input latch operation, the elements in the first row (W0.0, ..., W0.7) of the weight matrix and the elements in the first column of the vector matrix are sent to the MAC circuit 122 of the MAC operator 420 . (X0.0, …, X7.0) is input. The MAC circuit 122 may include a number equal to the number of columns of the weight matrix and the number of rows of the vector matrix, ie, eight multipliers 122-11. Each of the elements (W0.0, ..., W0.7) in the first row of the weight matrix and each of the elements (X0.0, ..., X7.0) in the first column of the vector matrix are multipliers 122 -11) is entered in each.

In step 366, the MAC circuit 122 of the MAC operator 420 performs a MAC operation on the R-th row of the input weight matrix and the first column of the vector matrix. R=1 is set as an initial value, and MAC operation is performed on the first row of the weight matrix and the first column of the vector matrix. As specifically described with reference to FIG. 3 , each of the multipliers 122-11 of the multiplication logic 122-1 performs a multiplication operation on input data, and adds the result to the addition logic 122-2. ) is entered. The addition logic 122-2 receives output data from the multipliers 122-11, adds them, and outputs them. The output data output from the addition logic 122-2 is MAC operation result data of the first row of the weight matrix and the vector matrix. Accordingly, the output data is the element (MAC0) in the first row and first column of the MAC result matrix of 8 rows and 1 column in which the elements MAC0.0, ..., MAC7.0 are composed of the MAC result data shown in FIG. .0) is configured. The output data MAC0.0 output from the addition logic 122-2 is an output latch 123-1 disposed in the data output unit 123 of the MAC operator 420, as described with reference to FIG. 3 . is entered as

In step 367 , as shown in FIG. 24 , the MAC command generator 540 of the PIM controller 500 generates a MAC output latch signal MAC_L3 and transmits it to the PIM device 400 . The MAC output latch signal MAC_L3 controls an output latch operation of the MAC result data MAC0.0 in the MAC operator 420 of the PIM device 400 . By this output latch operation, as described with reference to FIG. 3 , the MAC result data MAC0.0 input from the MAC circuit 122 of the MAC operator 420 is synchronized with the MAC output latch signal MAC_L3. output from the output latch 123-1. The MAC result data MAC0.0 output from the output latch 123-1 is input to the transfer gate 123-2 of the data output unit 123.

In step 368 , as shown in FIG. 25 , the MAC command generator 540 of the PIM controller 500 generates a MAC latch reset signal MAC_L_RST and transmits it to the PIM device 400 . The MAC latch reset signal MAC_L_RST controls the output operation of the MAC result data MAC0.0 and the reset operation of the output latch by the MAC operator 420 of the PIM device 400 . As described with reference to FIG. 3 , the transfer gate 123-2 receiving the MAC result data MAC0.0 from the output latch 123-1 of the data output unit 123 of the MAC operator 420, The MAC result data MAC0.0 is output in synchronization with the MAC latch reset signal MAC_L_RST. In an example, the MAC result data MAC0.0 output from the MAC operator 120 may be stored in the memory bank 411 through the BIO line in the PIM device 400 .

In step 369, "R" which is the number of rows of the weight matrix on which the MAC operation is performed is increased by one. As the MAC operation is performed on the first row among the first to eighth rows of the weight matrix so far, the number of rows of the weight matrix is changed from the first row to the second row in step 369 . In step 370, it is determined whether the number of rows of the weight matrix changed by step 369 is greater than the last row, that is, the eighth row of the weight matrix. Since the number of rows of the weight matrix changed by step 369 is the second row, the determination in step 370 determines that the number of rows of the weight matrix changed by step 369 is smaller than the eighth row, and the process of step 364 is performed again.

Upon feedback from step 370, the same processes as described above in steps 364 to 370 are performed except that the number of rows of the weight matrix on which the MAC operation is performed is different. That is, as the number of rows of the weight matrix is increased to the second row in step 369, there is a difference only in that the MAC operation of the second row and the vector matrix is performed instead of the first row of the weight matrix. The process of performing steps 364 to 370 after being fed back in step 370 is performed until the number of all rows of the weight matrix, that is, the MAC operation of the first to eighth rows and the vector matrix is finished. When the MAC operation on the 8th row of the weight matrix is completed and the number of rows of the weight matrix is changed from the 8th row to the 9th row in step 369, the ninth row, which is the number of rows of the weight matrix, is the last row in the judgment of step 370. Since it is greater than 8 lines, the MAC operation is terminated.

FIG. 26 is a flowchart illustrating a process of performing the calculation process described with reference to FIG. 13 in the PIM system 30 according to the second embodiment of the present disclosure. In order to perform the calculation process according to the present example, the MAC operator 420 of the PIM device 400 has the same configuration as the MAC operator 120-1 shown in FIG. 15 . Referring to FIG. 26 together with FIG. 19 , in step 381 , first data, that is, weight data, is written to the memory bank 411 in order to perform a MAC operation. Accordingly, the weight data is stored in the memory bank 411 of the PIM device 400 . In this example, it is assumed that the weight data are elements (W0.0, ..., W7.7) constituting the weight matrix of FIG. 13 .

In step 382, it is determined whether there is an inference request. The inference request may be transmitted from the outside of the PIM system 30 to the PIM controller 500 . In an example, if there is no inference request, the PIM system 30 may maintain a standby state until the inference request is transmitted. In another example, if there is no inference request, the PIM system 30 may perform an operation other than the MAC operation operation, for example, a memory mode operation such as data read/write, until the inference request is transmitted. . In this example, it is assumed that the second data, ie, vector data, is transmitted together with the inference request. Also, in this example, it is assumed that the vector data are elements (X0.0, ..., X7.0) constituting the vector matrix of FIG. 13 . When the inference request is transmitted in step 382 , in step 383 , the PIM controller 500 writes vector data transmitted along with the inference request to the global buffer 412 . Accordingly, vector data is stored in the global buffer 412 of the PIM device 400 .

In step 384 , the output latch of the MAC operator 420 is initially set to bias data, and the initially set bias data is fed back to the cumulative adder of the MAC operator 420 . This process is a process for the matrix addition operation of the bias matrix to the MAC result matrix in the operation described with reference to FIG. 13 . That is, as described with reference to FIG. 15 , the output latch 123-1 in the data output unit 123-A of the MAC operator 420 is initially set to bias data. Since the matrix multiplication operation is performed on the first row of the weight matrix, the first row and first column elements B0.0 of the bias matrix are initially set in the output latch 123-1 as bias data. The output latch 123-1 outputs the bias data B0.0, and the bias data B0.0 output from the output latch 123-1 is the accumulation adder 122 of the addition logic 122-2. -21D) is entered.

In one example, in order to output the bias data B0,0 from the output latch 123-1 and feed it back to the accumulation adder 122-21D, the MAC command generator 540 of the PIM controller 500 generates a MAC output latch signal. (MAC_L3) may be transmitted to the MAC operator 420 of the PIM device 400 . The accumulation adders 122-21D of the MAC operator 420, when the MAC operation is subsequently performed, the MAC result data outputted from the adders 122-21C disposed in the last stage of the addition logic 122-2 ( MAC0.0) and the fed back bias data B0.0 are added, and the resulting biased data Y0.0 is output and input to the output latch 123-1. The biased data Y0.0 may be output from the output latch 123 - 1 in synchronization with the subsequently transmitted MAC output latch signal MAC_L3 .

In step 385 , as described with reference to FIG. 22 , the MAC command generator 540 of the PIM controller 500 generates a MAC read signal MAC_RD_BK and transmits it to the PIM device 400 . In this case, the address generator 550 of the PIM controller 500 generates row/column addresses ADDR_R/ADDR_C and transmits the generated row/column addresses ADDR_R/ADDR_C to the PIM device 400 . The MAC read signal MAC_RD_BK transmitted to the PIM device 400 controls a data read operation for the memory bank 411 of the PIM device 400 . The memory bank 411 is, in response to the MAC read signal MAC_RD_BK, in the first row of the weight matrix among the weight data stored in the area designated by the row/column addresses ADDR_R/ADDR_C in the memory bank 411 The present elements (W0.0, ..., W0.7) are transmitted to the MAC operator 420 . In one example, data transfer from the memory bank 411 to the MAC operator 420 may be performed through a BIO line for data transfer only between the memory bank 411 and the MAC operator 420 .

On the other hand, the vector data (X0.0, ..., X7.0) stored in the global buffer 412 in synchronization with the clock to which the weight data is transmitted from the memory bank 411 to the MAC operator 420 also 420). To this end, a control signal for controlling a read operation on the global buffer 412 may be generated in synchronization with the transmission of the MAC read signal MAC_RD_BK from the MAC command generator 540 of the PIM controller 500 . Data transmission between the global buffer 412 and the MAC operator 420 may be performed through a GIO line. Accordingly, vector data transmission from the global buffer 412 to the MAC operator 420 may be performed independently of the weight data transmission from the memory bank 411 to the MAC operator 420 through the BIO line. In an example, transmission of weight data and transmission of vector data to the MAC operator 420 may be performed simultaneously through a BIO line and a GIO line, respectively.

In step 386 , as described with reference to FIG. 23 , the MAC command generator 540 of the PIM controller 500 generates a MAC input latch signal MAC_L1 and transmits it to the PIM device 400 . The MAC input latch signal MAC_L1 controls an input latch operation of weight data and an input latch operation of vector data to the MAC operator 420 of the PIM device 400 . By this input latch operation, the elements in the first row (W0.0, ..., W0.7) of the weight matrix and the elements in the first column of the vector matrix are sent to the MAC circuit 122 of the MAC operator 420 . (X0.0, …, X7.0) is input. The MAC circuit 122 may include a number equal to the number of columns of the weight matrix and the number of rows of the vector matrix, ie, eight multipliers 122-11. Each of the elements (W0.0, ..., W0.7) in the first row of the weight matrix and each of the elements (X0.0, ..., X7.0) in the first column of the vector matrix are multipliers 122 -11) is entered in each.

In step 387, the MAC circuit 122 of the MAC operator 420 performs a MAC operation on the R-th row of the input weight matrix and the first column of the vector matrix. R=1 is set as an initial value, and MAC operation is performed on the first row of the weight matrix and the first column of the vector matrix. Specifically, each of the multipliers 122-11 of the multiplication logic 122-1 performs a multiplication operation on input data, and inputs the result data to the addition logic 122-2. The addition logic 122-2 receives output data from the multipliers 122-11, performs an addition operation, and outputs the result to the accumulation adder 122-21D. The output data output from the adder 122-21C of the addition logic 122-2 is MAC operation result data MAC0.0 of the first row of the weight matrix and the vector matrix. The accumulation adder 122-21D adds and outputs the matrix multiplication result data MAC0.0 output from the adder 122-21C and the bias data B0.0 fed back from the output latch 123-1. . The output data output from the accumulation adder 122-21D, that is, the biased result data Y0.0, is transferred to the output latch 123-1 disposed in the data output unit 123-A of the MAC operator 120. is input

In step 388 , as described with reference to FIG. 24 , the MAC command generator 540 of the PIM controller 500 generates a MAC output latch signal MAC_L3 and transmits it to the PIM device 400 . The MAC output latch signal MAC_L3 controls an output latch operation of the output latch 123 - 1 in the MAC operator 420 of the PIM device 400 . By this output latch operation, the output latch 123-1 of the MAC operator 420 outputs the biased result data Y0.0 in synchronization with the MAC output latch signal MAC_L3. The biased result data Y0.0 output from the output latch 123-1 is input to the transfer gate 123-2 of the data output unit 123-A.

In step 389 , as described with reference to FIG. 25 , the MAC command generator 540 of the PIM controller 500 generates a MAC latch reset signal MAC_L_RST and transmits it to the PIM device 400 . The MAC latch reset signal MAC_L_RST controls the output operation of the biased result data Y0.0 and the reset operation of the output latch by the MAC operator 420 of the PIM device 400 . The transfer gate 123-2 receiving the bias result data Y0.0 from the output latch 123-1 of the data output unit 123 of the MAC operator 420 receives the MAC latch reset signal MAC_L_RST. Synchronously biased result data (Y0.0) is output. In one example, the biased result data Y0.0 output from the MAC operator 120 may be stored in the memory bank 411 through the BIO line in the PIM device 400 .

In step 390, "R" which is the number of rows of the weight matrix on which the MAC operation is performed is increased by one. As the MAC operation is performed on the first row among the first to eighth rows of the weight matrix so far, the number of rows of the weight matrix is changed from the first row to the second row in step 390 . In step 391, it is determined whether the number of rows of the weight matrix changed by step 390 is greater than the last row, that is, the eighth row of the weight matrix. Since the number of rows of the weight matrix changed by step 390 is the second row, it is determined in step 391 that the number of rows of the weight matrix changed by step 390 is smaller than the eighth row, and the process of step 384 is performed again.

When fed back from step 391, the same processes of steps 384 to 391 as described above are performed except that the number of rows of the weight matrix on which the MAC operation is performed is different. That is, as the number of rows of the weight matrix is increased to the second row in step 389, there is a difference only in that the MAC operation of the second row and the vector matrix is performed instead of the first row of the weight matrix. The process of performing steps 384 to 390 after being fed back in step 390 is performed until the number of all rows of the weight matrix, that is, the MAC operation of the first to eighth rows and the vector matrix is finished. When the MAC operation on the eighth row of the weight matrix is finished and the number of rows of the weight matrix is changed from the eighth row to the ninth row in step 389, the ninth row, which is the number of rows of the weight matrix, is the last row in the judgment of step 390. Since it is greater than 8 lines, the MAC operation is terminated.

FIG. 27 is a flowchart illustrating a process of performing the calculation process described with reference to FIG. 16 in the PIM system 30 according to the second embodiment of the present disclosure. In order to perform the calculation process according to the present example, the MAC operator 420 of the PIM device 400 has the same configuration as the MAC operator 120 - 2 shown in FIG. 18 . Referring to FIG. 27 together with FIG. 18 , in step 601 , first data, that is, weight data, is written to the memory bank 411 in order to perform a MAC operation. Accordingly, the weight data is stored in the memory bank 411 of the PIM device 400 . In this example, it is assumed that the weight data are elements (W0.0, ..., W7.7) constituting the weight matrix of FIG. 16 .

In step 602, it is determined whether there is an inference request. The inference request may be transmitted from the outside of the PIM system 30 to the PIM controller 500 . In an example, if there is no inference request, the PIM system 30 may maintain a standby state until the inference request is transmitted. In another example, if there is no inference request, the PIM system 30 may perform an operation other than the MAC operation operation, for example, a memory mode operation such as data read/write, until the inference request is transmitted. . In this example, it is assumed that the second data, ie, vector data, is transmitted together with the inference request. Also, in this example, it is assumed that the vector data are elements (X0.0, ..., X7.0) constituting the vector matrix of FIG. 16 . When the inference request is transmitted in step 602 , in step 603 , the PIM controller 500 writes the vector data transmitted along with the inference request to the global buffer 412 . Accordingly, vector data is stored in the global buffer 412 of the PIM device 400 .

In step 604 , the output latch of the MAC operator 420 is initially set to bias data, and the initially set bias data is fed back to the cumulative adder of the MAC operator 420 . This process is a process for the matrix addition operation of the bias matrix to the MAC result matrix in the operation described with reference to FIG. 16 . That is, as described with reference to FIG. 18 , the output latch 123-1 in the data output unit 123-B of the MAC operator 420 is initially set to bias data. Since the matrix multiplication operation is performed on the first row of the weight matrix, the first row and first column elements B0.0 of the bias matrix are initially set in the output latch 123-1 as bias data. The output latch 123-1 outputs the bias data B0.0, and the bias data B0.0 output from the output latch 123-1 is the accumulation adder 122 of the addition logic 122-2. -21D) is fed back.

In one example, in order to output the bias data B0,0 from the output latch 123-1 and feed it back to the accumulation adder 122-21D, the MAC command generator 540 of the PIM controller 500 generates a MAC output latch signal. (MAC_L3) may be transmitted to the MAC operator 420 of the PIM device 400 . The accumulation adders 122-21D of the MAC operator 420, when the MAC operation is subsequently performed, the MAC result data outputted from the adders 122-21C disposed in the last stage of the addition logic 122-2 ( MAC0.0) and the fed back bias data B0.0 are added, and the resulting biased data Y0.0 is output and input to the output latch 123-1. The biased data Y0.0 may be output from the output latch 123 - 1 in synchronization with the subsequently transmitted MAC output latch signal MAC_L3 .

In step 605 , as described with reference to FIG. 22 , the MAC command generator 540 of the PIM controller 500 generates a MAC read signal MAC_RD_BK and transmits the generated MAC read signal MAC_RD_BK to the PIM device 400 . In this case, the address generator 550 of the PIM controller 500 generates row/column addresses ADDR_R/ADDR_C and transmits the generated row/column addresses ADDR_R/ADDR_C to the PIM device 400 . The MAC read signal MAC_RD_BK transmitted to the PIM device 400 controls a data read operation for the memory bank 411 of the PIM device 400 . The memory bank 411 is, in response to the MAC read signal MAC_RD_BK, in the first row of the weight matrix among the weight data stored in the area designated by the row/column addresses ADDR_R/ADDR_C in the memory bank 411 The present elements (W0.0, ..., W0.7) are transmitted to the MAC operator 420 . In one example, data transfer from the memory bank 411 to the MAC operator 420 may be performed through a BIO line for data transfer only between the memory bank 411 and the MAC operator 420 .

On the other hand, the vector data (X0.0, ..., X7.0) stored in the global buffer 412 in synchronization with the clock to which the weight data is transmitted from the memory bank 411 to the MAC operator 420 also 420). To this end, a control signal for controlling a read operation on the global buffer 412 may be generated in synchronization with the transmission of the MAC read signal MAC_RD_BK from the MAC command generator 540 of the PIM controller 500 . Data transmission between the global buffer 412 and the MAC operator 420 may be performed through a GIO line. Accordingly, vector data transmission from the global buffer 412 to the MAC operator 420 may be performed independently of the weight data transmission from the memory bank 411 to the MAC operator 420 through the BIO line. In an example, transmission of weight data and transmission of vector data to the MAC operator 420 may be performed simultaneously through a BIO line and a GIO line, respectively.

In step 606 , as described with reference to FIG. 23 , the MAC command generator 540 of the PIM controller 500 generates a MAC input latch signal MAC_L1 and transmits it to the PIM device 400 . The MAC input latch signal MAC_L1 controls an input latch operation of weight data and an input latch operation of vector data to the MAC operator 420 of the PIM device 400 . By this input latch operation, the elements in the first row (W0.0, ..., W0.7) of the weight matrix and the elements in the first column of the vector matrix are sent to the MAC circuit 122 of the MAC operator 420 . (X0.0, …, X7.0) is input. The MAC circuit 122 may include a number equal to the number of columns of the weight matrix and the number of rows of the vector matrix, ie, eight multipliers 122-11. Each of the elements (W0.0, ..., W0.7) in the first row of the weight matrix and each of the elements (X0.0, ..., X7.0) in the first column of the vector matrix are multipliers 122 -11) is entered in each.

In step 607, the MAC circuit 122 of the MAC operator 420 performs a MAC operation on the R-th row of the input weight matrix and the first column of the vector matrix. R=1 is set as an initial value, and MAC operation is performed on the first row of the weight matrix and the first column of the vector matrix. Specifically, each of the multipliers 122-11 of the multiplication logic 122-1 performs a multiplication operation on input data, and inputs the result data to the addition logic 122-2. The addition logic 122-2 receives output data from the multipliers 122-11, performs an addition operation, and outputs the result to the accumulation adder 122-21D. The output data output from the adder 122-21C of the addition logic 122-2 is MAC operation result data MAC0.0 of the first row of the weight matrix and the vector matrix. The accumulation adder 122-21D adds and outputs the matrix multiplication result data MAC0.0 output from the adder 122-21C and the bias data B0.0 fed back from the output latch 123-1. . The biased result data Y0.0 output from the accumulation adder 122-21D is input to the output latch 123-1 disposed in the data output unit 123-A of the MAC operator 120.

In step 608 , as described with reference to FIG. 24 , the MAC command generator 540 of the PIM controller 500 generates a MAC output latch signal MAC_L3 and transmits it to the PIM device 400 . The MAC output latch signal MAC_L3 controls an output latch operation of the output latch 123 - 1 in the MAC operator 420 of the PIM device 400 . By this output latch operation, the output latch 123-1 of the MAC operator 420 outputs the biased result data Y0.0 in synchronization with the MAC output latch signal MAC_L3. The biased result data Y0.0 output from the output latch 123-1 is input to the activation function logic 123-5. And in step 610, the activation function logic 123-5 applies the activation function to the input biased result data Y0.0 to generate a final output value and input it to the transfer gate (123-2 in FIG. 3). .

In step 610 , as described with reference to FIG. 25 , the MAC command generator 540 of the PIM controller 500 generates a MAC latch reset signal MAC_L_RST and transmits it to the PIM device 400 . The MAC latch reset signal MAC_L_RST controls the output operation of the final output value from the MAC operator 420 of the PIM device 400 and the reset operation of the output latch. The transfer gate 123-2 receiving the final output value from the activation function logic 123-5 of the data output unit 123-B of the MAC operator 420 is synchronized with the MAC latch reset signal MAC_L_RST to obtain the final output value. outputs In an example, the final output value output from the MAC operator 120 may be stored in the memory bank 411 through the BIO line in the PIM device 400 .

In step 611, "R" which is the number of rows of the weight matrix on which the MAC operation is performed is increased by one. As the MAC operation is performed on the first row among the first to eighth rows of the weight matrix so far, the number of rows of the weight matrix is changed from the first row to the second row in step 611 . In step 612, it is determined whether the number of rows of the weight matrix changed in step 611 is greater than the last row, that is, the eighth row of the weight matrix. Since the number of rows of the weight matrix changed by step 611 is the second row, the determination in step 612 determines that the number of rows of the weight matrix changed in step 611 is smaller than the eighth row, and the process of step 604 is performed again.

Upon feedback from step 612, the same steps 604 to 612 as described above are performed except that the number of rows of the weight matrix, the number of columns of the vector matrix, and the number of columns of the bias matrix are different from each other. That is, as the number of rows of the weight matrix is increased to the second row in step 611, the target of the MAC operation is the second row and the vector matrix of the weight matrix, and the target of the bias operation is the first row and the second row of the MAC result matrix. It differs only in that it is a column element (MAC1.0) and a first row and second column element (B1.0) of the bias matrix. The process of performing steps 604 to 611 after being fed back in step 612 is performed until the number of all rows of the weight matrix, that is, the MAC operation of the first to eighth rows and the vector matrix is finished. When the MAC operation on the 8th row of the weight matrix is finished and the number of rows of the weight matrix is changed from the 8th row to the 9th row in step 611, the ninth row, which is the number of rows of the weight matrix, is the last row in the judgment of step 612. Since it is greater than 8 lines, the MAC operation is terminated.

28 is a block diagram of a PIM system 20 according to a third embodiment of the present disclosure. As shown in FIG. 28 , the PIM system 20 according to this example is the PIM system described with reference to FIG. 1 , except that the PIM controller 200 further includes a mode register 260 . Same as (10). Accordingly, the same description as described with reference to FIG. 1 will be omitted. The mode register 260 in the PIM controller 200 may receive a mode register setting (MRS) signal indicating a setting necessary for the MAC mode operation of the PIM system 20 . In one example, the MRS signal may be provided from the mode selector 221 in the scheduler 220, but this is only an example, and logic for providing a separate MRS signal may be disposed. The mode register 260 receiving the MRS signal transmits the MRS signal to the MAC command generator 240 .

In one example, the MRS signal may include a timing at which each of the MAC commands MAC_CMDs are generated. In this case, the deterministic operation of the PIM system 20 may be performed by MRS data provided by the mode register 260 . In another example, the MRS signal may include information related to the timing between the MAC mode operation and the MAC mode operation or the mode conversion operation between the MAC mode operation and the memory mode operation. In an example, the MRS setting in the mode register 260 may be performed before an inference is transmitted from the outside and vector data is stored in the second memory bank 112 of the PIM device 100 . In another example, the MRS setting in the mode register 260 may be performed after an inference is transmitted from the outside and vector data is stored in the second memory bank 112 of the PIM device 100 .

29 is a block diagram of a PIM system 40 according to a fourth embodiment of the present disclosure. As shown in FIG. 29 , the PIM system 40 according to this example includes the PIM system 30 described with reference to FIG. 19 , except that the PIM controller 500 further includes a mode register 260 . same. Accordingly, the same description as that described with reference to FIG. 19 will be omitted. The mode register 260 in the PIM controller 500 may receive an MRS signal indicating a setting necessary for the MAC mode operation of the PIM system 40 . In one example, the MRS signal may be provided from the mode selector 221 in the scheduler 220, but this is only an example, and logic for providing a separate MRS signal may be disposed. The mode register 260 receiving the MRS signal transmits the MRS signal to the MAC command generator 540 .

In one example, the MRS signal may include a timing at which each of the MAC commands MAC_CMDs are generated. In this case, the deterministic operation of the PIM system 40 may be performed by MRS data provided by the mode register 260 . In another example, the MRS signal may include information related to the timing between the MAC mode operation and the MAC mode operation or the mode conversion operation between the MAC mode operation and the memory mode operation. In an example, the MRS setting in the mode register 260 may be performed before an inference is transmitted from the outside and vector data is stored in the global buffer 412 of the PIM device 400 . In another example, the MRS setting in the mode register 260 may be performed after an inference is transmitted from the outside and vector data is stored in the global buffer 412 of the PIM device 400 .

So far, with respect to the technology of the present application, it has been mainly looked at the embodiments. Those of ordinary skill in the art to which the technology of the present application belongs will understand that the technology of the present application may be implemented in a modified form without departing from the essential characteristics of the technology of the present application. Therefore, the disclosed embodiments are to be considered in an illustrative rather than a restrictive sense. The scope of the technology of the present application is indicated in the claims rather than the above description, and all differences within the scope equivalent thereto should be construed as being included in the technology of the present application.

10...PIM system 100...PIM device
111...First memory bank 112...Second memory bank
120...MAC operator 131...Interface
132...Data input/output terminal (DQ) 200...PIM controller
210...command queue 220...scheduler
221...mode selector 230...memory command generator
240...MAC command generator 250...address generator

Claims (52)

a first storage area, a second storage area, and a MAC operator receiving first data and second data from the first storage area and the second storage area, respectively, and performing a multiply-accumulate addition (MAC) operation; a PIM device; and
A memory mode operation and a MAC mode operation of the PIM device are controlled, wherein a memory command signal is generated and transmitted to the PIM device in the memory mode, and first to fifth MAC command signals are transmitted to the PIM device in the MAC mode A PIM system comprising a PIM controller configured to: According to claim 1,
The transfer of the first data from the first storage area to the MAC operator and the transfer of the second data from the second storage area to the MAC operator are performed via a global input/output (GIO) line in the PIM device. PIM system made. According to claim 1,
The MAC operation is a PIM system configured to operate in a deterministic MAC operation method performed for a fixed fixed time. According to claim 1, wherein the PIM controller,
a memory command generator for generating and outputting the memory command signal in the memory mode; and
and a MAC command generator for generating and outputting the first to fifth MAC command signals in the MAC mode. 5. The method of claim 4,
The MAC command generator is configured to sequentially output the first to fifth MAC command signals at predetermined fixed time intervals. 5. The method of claim 4,
The first MAC command signal is a control signal for reading the first data from the first storage area;
The second MAC command signal is a control signal for reading the second data from the second storage area;
The third MAC command signal is a control signal for latching the first data in the MAC operator,
The fourth MAC command signal is a control signal for latching the second data in the MAC operator, and
The fifth MAC command signal is a control signal for latching MAC result data of the MAC operator. 7. The method of claim 6, wherein the MAC command generator comprises:
outputting the first MAC command signal at a first time,
outputting the second MAC command signal at a second time point when a first latency has elapsed from the first time point;
outputting the third MAC command signal at a third time point when a second latency has elapsed from the second time point;
outputting the fourth MAC command signal at a fourth time point when a third latency has elapsed from the third time point, and
and output the fifth MAC command signal at a fifth time point when a fourth latency has elapsed from the fourth time point. 8. The method of claim 7,
The first latency is set as a time required to read the first data from the first storage area and input it to the MAC operator by the first MAC command signal;
The second latency is set to a time required to read the second data from the second storage area and input it to the MAC operator by the second MAC command signal;
The third latency is set to a time required for the first data input to the MAC operator by the third MAC command signal to be latched;
The fourth latency is set as a time required for the second data input to the MAC operator by the fourth MAC command signal to be latched and to perform MAC operations on the latched first data and second data. being a PIM system. The method of claim 6, wherein the MAC operator,
The first data and the second data are respectively received from the first storage area and the second storage area, and the first data and the second data are respectively synchronized with the third MAC command signal and the fourth MAC command signal. a data input unit for latching;
a MAC circuit for receiving first data and second data latched from the data input unit, performing the MAC operation, and outputting MAC result data; and
and a data output unit receiving the MAC result data from the MAC circuit and including an output latch for latching the MAC result data in synchronization with the fifth MAC command signal. 10. The method of claim 9,
The data output unit may further include a transfer gate configured to output the MAC result data to the GIO line. 7. The method of claim 6,
The MAC command generator is configured to output a sixth MAC command signal after a predetermined fixed time elapses after outputting the fifth MAC command signal. 12. The method of claim 11,
The sixth MAC command signal is a control signal for outputting the MAC result data from the MAC operator and resetting an output latch in the MAC operator. 13. The method of claim 12, wherein the MAC command generator,
outputting the first MAC command signal at a first time,
outputting the second MAC command signal at a second time point when a first latency has elapsed from the first time point;
outputting the third MAC command signal at a third time point when a second latency has elapsed from the second time point;
outputting the fourth MAC command signal at a fourth time point when a third latency has elapsed from the third time point;
outputting the fifth MAC command signal at a fifth time point when a fourth latency has elapsed from the fourth time point, and
and output the sixth MAC command signal at a sixth time point when a fifth latency has elapsed from the fifth time point. 14. The method of claim 13,
The first latency is set as a time required to read the first data from the first storage area and input it to the MAC operator by the first MAC command signal;
The second latency is set to a time required to read the second data from the second storage area and input it to the MAC operator by the second MAC command signal;
The third latency is set to a time required for the first data input to the MAC operator by the third MAC command signal to be latched;
The fourth latency is set as a time required for the second data input to the MAC operator by the fourth MAC command signal to be latched and to perform MAC operations on the latched first data and second data. becomes, and
The fifth latency is set as a time required to perform an output latch on the MAC result data generated by the MAC operation. The method of claim 6, wherein the PIM controller,
a command queue for storing commands transmitted from the outside;
The PIM system further comprising a scheduler that adjusts a timing at which a command stored in the command queue is output from the command queue. 16. The method of claim 15,
and the scheduler includes a mode selector configured to generate a mode selection signal indicating whether a command output from the command queue requests the memory mode operation or the MAC mode operation. 16. The method of claim 15,
The command queue, in response to a control signal from the scheduler, is transmitted to the memory command generator when the command requests a memory mode operation, and is transmitted to the MAC command generator when the command requests a MAC mode operation. PIM system. 7. The method of claim 6,
The PIM controller further includes a mode register configured to set a mode register necessary for the MAC mode operation. 19. The method of claim 18,
The mode register setting includes a timing at which the first to fifth MAC command signals are generated. 19. The method of claim 18,
The mode register setting includes information related to a timing between a MAC mode operation and a MAC mode operation, or a mode conversion operation between a MAC mode operation and a memory mode operation. a PIM device including a storage area, a global buffer, and a MAC operator receiving first data and second data from the storage area and the global buffer, respectively, and performing a MAC operation; and
A memory mode operation and a MAC mode operation of the PIM device are controlled, wherein a memory command signal is generated and transmitted to the PIM device in the memory mode, and first to fourth MAC command signals are transmitted to the PIM device in the MAC mode A PIM system comprising a PIM controller configured to: 22. The method of claim 21,
The MAC operation is a PIM system configured to operate in a deterministic MAC operation method performed for a fixed fixed time. 22. The method of claim 21,
The PIM device further includes a GIO line providing a data transmission path inside the PIM device, and a bank input/output (BIO) line providing a data transmission path between the storage device and the MAC operator,
The transmission of the first data from the storage area to the MAC operator is performed through the BIO line, and the transmission of the second data from the global buffer to the MAC operator is performed through the GIO line. The method of claim 21, wherein the PIM controller,
a memory command generator for generating and outputting the memory command signal in the memory mode; and
and a MAC command generator for generating and outputting the first to third MAC command signals in the MAC mode. 25. The method of claim 24,
The MAC command generator is configured to sequentially output the first to third MAC command signals at predetermined fixed time intervals. 25. The method of claim 24,
The first MAC command signal is a control signal for reading the first data from the storage area;
The second MAC command signal is a control signal for latching the first data and the second data in the MAC operator;
The third MAC command signal is a control signal for latching MAC result data of the MAC operator. 27. The method of claim 26, wherein the MAC command generator comprises:
outputting the first MAC command signal at a first time,
outputting the second MAC command signal at a second time point when a first latency has elapsed from the first time point, and
and output the third MAC command signal at a third time point when a second latency has elapsed from the second time point. 28. The method of claim 27,
The first latency is set to a time required to read the first data from the storage area and input it to the MAC operator by the first MAC command signal, and
In the second latency, the first data and second data input to the MAC operator by the second MAC command signal are latched, and a MAC operation is performed on the latched first data and second data. A PIM system that is set to time. 29. The method of claim 28, wherein the MAC operator,
a data input unit receiving the first data and the second data from the storage area and the global buffer, respectively, and latching the first data and the second data in synchronization with the second MAC command signal;
a MAC circuit for receiving first data and second data latched from the data input unit, performing the MAC operation, and outputting MAC result data; and
and a data output unit receiving the MAC result data from the MAC circuit and including an output latch for latching the MAC result data in synchronization with the third MAC command signal. 30. The method of claim 29,
The data output unit may further include a transfer gate configured to output the MAC result data to the GIO line. 25. The method of claim 24,
The MAC command generator is configured to output a fourth MAC command signal after a predetermined fixed time has elapsed after outputting the third MAC command signal. 32. The method of claim 31,
The fourth MAC command signal is a control signal for outputting the MAC result data from the MAC operator and resetting an output latch in the MAC operator. 33. The method of claim 32, wherein the MAC command generator comprises:
outputting the first MAC command signal at a first time,
outputting the second MAC command signal at a second time point when a first latency has elapsed from the first time point;
outputting the third MAC command signal at a third time point when a second latency has elapsed from the second time point, and
and output the fourth MAC command signal at a fourth time point when a third latency has elapsed from the third time point. 34. The method of claim 33,
The first latency is set to a time required to read the first data from the storage area and input it to the MAC operator by the first MAC command signal;
In the second latency, the first data and second data input to the MAC operator by the second MAC command signal are latched, and a MAC operation is performed on the latched first data and second data. is set to the time to be, and
The third latency is set to a time required to perform an output latch on the MAC result data generated by the MAC operation. The method of claim 24, wherein the PIM controller comprises:
a command queue for storing commands transmitted from the outside;
The PIM system further comprising a scheduler that adjusts a timing at which a command stored in the command queue is output from the command queue. 36. The method of claim 35,
and the scheduler includes a mode selector configured to generate a mode selection signal indicating whether a command output from the command queue requests the memory mode operation or the MAC mode operation. 36. The method of claim 35,
The command queue, in response to a control signal from the scheduler, is transmitted to the memory command generator when the command requests a memory mode operation, and is transmitted to the MAC command generator when the command requests a MAC mode operation. PIM system. 25. The method of claim 24,
The PIM controller further includes a mode register configured to set a mode register necessary for the MAC mode operation. 39. The method of claim 38,
The mode register setting includes a timing at which the first to fifth MAC command signals are generated. 39. The method of claim 38,
The mode register setting includes information related to a timing between a MAC mode operation and a MAC mode operation, or a mode conversion operation between a MAC mode operation and a memory mode operation. A method of operating a PIM system comprising: a PIM device for performing a MAC operation; and a PIM controller for controlling a MAC operation operation of the PIM device, the method comprising:
R row (R is a natural number greater than 1) of a weight matrix of MXN (M, N is a natural number greater than 1) for MAC operation from the first storage device of the PIM device by transmitting a first MAC command signal from the PIM controller to the PIM device initially set to "1") allowing elements to be input to a MAC operator of the PIM device;
transmitting a second MAC command signal from the PIM controller to the PIM device so that the first column elements of the vector matrix of NX1 for MAC operation are input from the second storage device of the PIM device to the MAC operator of the PIM device step;
transmitting a third MAC command signal from the PIM controller to the PIM device, so that an input latch for R row elements of the weight matrix is performed in the MAC operator;
transmitting a fourth MAC command signal from the PIM controller to the PIM device, so that an input latch on the first column elements of the vector matrix is performed in the MAC operator;
performing a matrix multiplication operation on the elements of the R-th row of the weight matrix and the elements of the first column of the vector matrix in the MAC operator;
transmitting a fifth MAC command signal from the PIM controller to the PIM device so that an output latch is performed on the result data that is the result of the matrix multiplication operation in the MAC operator;
transmitting a sixth MAC command signal from the PIM controller to the PIM device so that the output latched result data in the MAC operator is output from the MAC operator; and
and repeating the above steps if R is less than or equal to the last row after increasing R by +1. 42. The method of claim 41,
The transmission of the first to sixth MAC command signals is a method of operating a PIM system in which a deterministic MAC operation method is performed for a predetermined fixed time. 42. The method of claim 41, wherein before transmitting a first MAC command signal from the PIM controller to the PIM device,
writing the weight matrix of the MXN to the first storage device;
and writing the vector matrix of NX1 to the second storage device when there is an inference request from outside the PIM system. 42. The method of claim 41,
and, before sending a first MAC command signal from the PIM controller to the PIM device, initially setting an output latch of the MAC operator to an R-th row element of a bias matrix of NX1. 45. The method of claim 44,
In the step of performing a matrix multiplication operation of the elements of the R-th row of the weight matrix and the elements of the first column of the vector matrix in the MAC operator, the initially set bias matrix is added to the MAC result data, which is the result of the matrix multiplication operation. The method of operation of the PIM system further comprising the step of adding. 46. The method of claim 45,
Before transmitting the sixth MAC command signal from the PIM controller to the PIM device, the result data, which is the result of the matrix multiplication operation output latched in the MAC operator, is obtained by adding the initially set bias matrix in the MAC operator. replaced by the resulting data,
applying an activation function to the result data to which the output latched initially set bias matrix is added; and
The output latched result data output from the MAC operator is replaced with data to which the activation function is applied. A method of operating a PIM system comprising: a PIM device for performing a MAC operation; and a PIM controller for controlling a MAC operation operation of the PIM device, the method comprising:
A first MAC command signal is transmitted from the PIM controller to the PIM device, and the R row (R is “1”) of a weight matrix of MXN (M, N is a natural number greater than 1) for MAC operation from the storage device of the PIM device. (initially set to "), causing the elements to be input to the MAC operator of the PIM device;
allowing first column elements of a vector matrix of NX1 for MAC operation to be input from the global buffer of the PIM device to a MAC operator of the PIM device;
sending a second MAC command signal from the PIM controller to the PIM device so that an input latch is performed on the elements in the R-th row of the weight matrix and the elements in the first column of the vector matrix in the MAC operator; ;
performing a matrix multiplication operation on the elements of the R-th row of the weight matrix and the elements of the first column of the vector matrix in the MAC operator;
transmitting a third MAC command signal from the PIM controller to the PIM device so that an output latch is performed on the result data that is the result of the matrix multiplication operation in the MAC operator;
transmitting a fourth MAC command signal from the PIM controller to the PIM device so that the output latched result data in the MAC operator is output from the MAC operator; and
and repeating the above steps if R is less than or equal to the last row after increasing R by +1. 48. The method of claim 47,
The transmission of the first to fourth MAC command signals is a method of operating a PIM system in which the deterministic MAC operation is performed for a fixed time. 48. The method of claim 47, wherein before transmitting a first MAC command signal from the PIM controller to the PIM device,
writing the weight matrix of the MXN to the storage device;
and storing the vector matrix of the NX1 in the global buffer when there is an inference request from outside the PIM system. 48. The method of claim 47,
and, before sending a first MAC command signal from the PIM controller to the PIM device, initially setting an output latch of the MAC operator to an R-th row element of a bias matrix of NX1. 51. The method of claim 50,
In the step of performing a matrix multiplication operation of the elements of the R-th row of the weight matrix and the elements of the first column of the vector matrix in the MAC operator, the initially set bias matrix is added to the MAC result data, which is the result of the matrix multiplication operation. The method of operation of the PIM system further comprising the step of adding. 52. The method of claim 51,
Before transmitting the fourth MAC command signal from the PIM controller to the PIM device, the result data, which is the result of the matrix multiplication operation output latched in the MAC operator, is obtained by adding the initially set bias matrix in the MAC operator. replaced by the resulting data,
applying an activation function to the result data to which the output latched initially set bias matrix is added; and
The output latched result data output from the MAC operator is replaced with data to which the activation function is applied.

Images