KR20200108768A - Memory Device performing calculation process and Operation Method thereof - Google Patents

Abstract

Disclosed are a memory device for performing operation processing to prevent data collisions by providing an optimized data transfer path therein, and an operation method thereof. According to one aspect of the technical idea of the present disclosure, the memory device comprises: a memory bank including at least one bank group, wherein each bank group includes a plurality of banks; a PIM circuit including a first processing element disposed to correspond to the bank group; a processing element input/output (PEIO) gating circuit for controlling electrical connection between a bank local input/output line (IO) disposed for each of the plurality of banks and a bank group input/output line (IO) disposed for the bank group; and a control logic for performing a control operation to perform a memory operation on the memory bank or perform operation processing by the PIM circuit, wherein the PEIO gating circuit cuts off the electrical connection between the bank local IO and the bank group IO when the operation processing is performed by the first processing element.

Description

TECHNICAL FIELD [0001] A memory device performing calculation process and an operation method of the memory device TECHNICAL FIELD

The technical idea of the present disclosure relates to a memory device, and more particularly, to a memory device that performs arithmetic processing and an operating method thereof.

Semiconductor memory devices widely used in high-performance electronic systems are increasing in capacity and speed. As an example of a memory device, a DRAM (Dynamic Random Access Memory) is a volatile memory and is a memory that determines data based on a charge stored in a capacitor.

A memory device including a DRAM may be used for various purposes, and as an example, may be used to store data used for various types of operation processing, such as a neural network operation, or to store operation results. In this case, in order to efficiently perform a vast amount of operations, a scheme in which at least some of the operation operations are performed in a PIM (Processor in Memory) circuit in a memory device has been proposed.

The problem to be solved by the technical idea of the present invention is to provide a memory device that performs calculation processing capable of preventing data collisions by providing an optimized data transfer path in a memory device that performs calculation processing, and an operating method thereof. have.

In order to achieve the above object, a memory device according to an aspect of the inventive concept includes at least one bank group, each bank group including a memory bank including a plurality of banks, and the bank group And a first processing element disposed in correspondence with the first processing element, wherein the first processing element includes a PIM circuit that performs an operation process using at least one of data provided from a host and data read from the bank group, and the bank group A processing element input/output (PEIO) gating circuit for controlling electrical connection between a bank local input/output line (IO) disposed for each of the plurality of banks and a bank group input/output line (IO) disposed for the bank group, and the A control logic for performing a control operation such that a memory operation for the memory bank is performed or an operation processing by the PIM circuit is performed based on a result of decoding a command/address received from the host, and the first processing When arithmetic processing is performed by an element, the PEIO gating circuit is characterized in that it blocks an electrical connection between the bank local IO and the bank group IO.

Meanwhile, a memory device according to another aspect of the technical idea of the present disclosure includes a bank group, wherein the bank group is a memory bank including first and second banks, and is disposed corresponding to the bank group, and A bank group input/output line (IO) including a data transfer path between the host of the host and the bank group, and data provided to the first bank and read from the first bank, arranged corresponding to the first bank. At least one of a first bank local input/output line (IO) including a data transfer path, and connected to the bank group IO and the first bank local IO, and data provided from the host and data read from the first bank When the first processing element and the first processing element to perform an operation process are performed, data is transferred so that data from the host is provided to the first processing element without being provided to the memory bank. And a first processing element input/output (PEIO) gating circuit for controlling the path.

According to a memory device for performing arithmetic processing according to the technical idea of the present invention and an operation method thereof, since the host can control the arithmetic processing in the memory device, the memory operation is reduced by reducing the collision between the memory operation and the arithmetic processing operation. It can prevent the problem of excessive delay.

In addition, according to the memory device for performing computational processing according to the technical idea of the present invention and the operating method thereof, it is possible to optimize a transmission path of data used for computational processing, and thus data used for computational processing can be prevented from colliding with each other There is an effect.

1 is a block diagram illustrating a memory system including a memory device according to an exemplary embodiment of the present invention.
FIG. 2 is a block diagram illustrating an example implementation of the memory device of FIG. 1.
3 is a block diagram showing a data processing system including a memory device according to an exemplary embodiment of the present invention.
4 is a conceptual diagram illustrating an example of an operation of a memory system according to an exemplary embodiment of the present invention.
5 is a block diagram illustrating an example of various components included in a memory device.
6 and 7 are block diagrams illustrating examples of configurations included in a memory device in relation to operation processing.
8 is a block diagram illustrating an example in which a memory device according to an exemplary embodiment of the present invention includes an HBM.
9 is a flowchart showing a method of operating a memory device according to an exemplary embodiment of the present invention.
10A, B, and C are diagrams illustrating examples of operation commands defined in a memory device according to an embodiment of the present invention.
11 is a waveform diagram illustrating an example of operation processing sequentially performed in a memory device according to embodiments of the present invention.
12A,B through 14A,B are diagrams illustrating implementation examples of a memory device according to various embodiments of the present disclosure.
15 is a diagram illustrating an example implementation of a memory device according to a deformable embodiment of the present invention.
16 is a block diagram showing a server system including a data processing system according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a memory system including a memory device according to an exemplary embodiment of the present invention.

Referring to FIG. 1, the memory system 10 may include a memory controller 100 and a memory device 200. The memory controller 100 may include a memory interface 110, and may control memory operations such as writing and reading by providing various signals to the memory device 200 through the memory interface 110. For example, the memory controller 100 may provide a command CMD and an address ADD to the memory device 200 to access the data DATA of the memory device 200.

The memory controller 100 may access the memory device 200 according to a request from the host HOST, and the memory interface 110 may provide an interface with the memory device 200. The memory controller 100 may communicate with a host using various protocols, for example, the memory controller 100 may be configured with PCI-E (Peripheral Component Interconnect-Express), ATA (Advanced Technology Attachment), SATA (Serial ATA), and PATA. It is possible to communicate with the host using an interface protocol such as (Parallel ATA) or serial attached SCSI (SAS). In addition, various other interface protocols such as USB (Universal Serial Bus), MMC (Multi-Media Card), ESDI (Enhanced Small Disk Interface), or IDE (Integrated Drive Electronics) are applied to the protocol between the host and the memory controller 100. I can.

According to exemplary embodiments, the memory controller 100 may correspond to a host, or the memory controller 100 may correspond to a configuration provided inside the host HOST. The host and the memory device 200 may configure a data processing system, and accordingly, the memory system 10 may correspond to a data processing system or be defined as a configuration included in the data processing system.

The memory device 200 includes a memory bank 210, a processor in memory (PIM) circuit 220, a processing element input/output gating circuit (hereinafter referred to as a PEIO gating circuit 230), and a control logic 240. In addition, the control logic 240 may include a command/address decoder 241. In addition, the memory bank 210 may include a plurality of banks BANK 0 to BANK N-1, and each of the banks BANK 0 to BANK N-1 comprises a cell array including a plurality of memory cells. Can be equipped. The bank may be defined in various ways, and as an example, may be defined as a configuration including memory cells, or may be defined as a configuration including one or more peripheral circuits together with the memory cells.

Meanwhile, two or more banks may be classified into one bank group, and the banks included in the same bank group are provided with the memory controller 100 and data through a common bank group input/output line (hereinafter referred to as bank group IO). (DATA) can be transmitted and received. In addition, as a local path for accessing the cell array, a bank local input/output line (hereinafter referred to as a bank local IO) may be disposed corresponding to each of the banks BANK 0 to BANK N-1.

A bank in which data access is to be performed may be selected by an address ADD from the memory controller 100, and memory cells in the bank may be selected. In addition, the command/address decoder 241 may perform a decoding operation on the command/address (CMD/ADD) from the memory controller 100, and the control logic 240 performs a memory operation according to the decoding result. An internal control operation for the memory device 200 may be performed.

Meanwhile, the memory device 200 may include DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access Memory), LPDDR (Low Power Double Data Rate) SDRAM, GDDR (Graphics Double Data Rate) SDRAM, RDRAM (Rambus Dynamic Random Access Memory), etc. It may be a dynamic random access memory (DRAM). However, embodiments of the present invention need not be limited thereto, and as an example, the memory device 200A includes flash memory, magnetic RAM (MRAM), ferroelectric RAM (FeRAM), phase change RAM (PRAM), and ReRAM. It can be implemented with nonvolatile memory such as (Resistive RAM).

Also, the memory device 200 may correspond to one semiconductor chip, or may have a configuration corresponding to one channel in a memory device including a plurality of channels having independent interfaces. Alternatively, the memory device 200 may have a configuration corresponding to a memory module, or the memory module includes a plurality of memory chips, and the memory device 200 of FIG. 1 corresponds to one memory chip mounted on a module board. You may.

Hereinafter, an example in which the PIM circuit 220 of the memory device 200 performs arithmetic processing according to exemplary embodiments of the present invention will be described. Various types of arithmetic processing operations may be performed in the memory device 200, and as an example, at least some of neural network operations related to artificial intelligence may be performed in the memory device 200. For example, the host HOST may control the memory device 200 through the memory controller 100 so that at least some of the neural network operations may be performed by the memory device 200. Further, in the following embodiments, the memory controller 100 will be described as controlling the memory device 200, but embodiments of the present invention need not be limited thereto. For example, the memory controller 100 corresponds to a configuration included in the host HOST, and it may be described that the host HOST controls the memory device 200.

The memory controller 100 may transmit one or more instructions Inst to the memory device 200 to perform various operation processing. As an example, the memory device 200 may receive and store a plurality of instructions Inst in the instruction loading mode. In FIG. 1, transmission of commands Inst and data DATA is shown separately, but in an embodiment of the present invention, commands Inst and data DATA can be transmitted and received in various forms. As an example, the commands Inst may be transmitted through a separate line irrelevant to the data DATA, or may be transmitted through a line (data line) transmitting data DATA.

As an implementation example, the PIM circuit 220 may include one or more processing elements 221 and an instruction memory (not shown) for storing instructions (Inst), and then processing the operation from the memory controller 100 When the command/address CMD/ADD indicative of is received, an operation corresponding to the instruction Inst read from the instruction memory may be performed. Since the memory controller 100 provides a command/address (CMD/ADD) so that operation processing can be performed in the memory device 200, the memory controller 100 does not cause a collision between the normal memory operation and the operation processing operation. It is possible to control the memory device 200 to prevent it.

Meanwhile, a command/address CMD/ADD for instructing an operation process may be variously defined. As an example, an operation command CMD_PE may be defined separately from a command related to a memory operation such as data writing/reading. The memory device 200 may selectively perform a memory operation or an operation processing operation based on a decoding operation for a command/address (CMD/ADD), and also, based on the address ADD, A location or a location in which an operation result is to be stored may be determined.

Meanwhile, the processing elements 221 may include a variety of processing elements. As an example, each processing element may be disposed corresponding to one bank, or each processing element may be disposed corresponding to two or more banks. Each of the processing elements 221 is at least one of data (DATA) provided from the memory controller 100, data (DATA) read from the memory bank 210, and information stored in a register (not shown) provided in the processing element. You can perform arithmetic processing using.

The PEIO gating circuit 230 may perform a function of controlling a transmission path of information such as data. As an example, the PEIO gating circuit 230 may be disposed corresponding to each of the banks BANK 0 to BANK N-1, and when a plurality of data lines are disposed corresponding to each of the banks, the PEIO gating circuit ( 230) includes a plurality of switches (not shown), and a data transmission path may be controlled according to switching states of the switches. During the memory operation of the memory device 200, data DATA may be transmitted and received between the memory controller 100 and the banks BANK 0 to BANK N-1 by the PEIO gating circuit 230. On the other hand, during operation processing of the memory device 200, data DATA may be transmitted and received between the memory controller 100 and the processing elements 221 by the PEIO gating circuit 230, and banks ( Data DATA may be transmitted/received between BANK 0 to BANK N-1) and the processing elements 221.

As an implementation example, the PEIO gating circuit 230 may be disposed between the bank local IO corresponding to each of the banks BANK 0 to BANK N-1 and the bank group IO corresponding to the bank group. Based on the switching operation of the PEIO gating circuit 230, as the bank local IO and the bank group IO are electrically connected, memory operations such as data writing and data reading for the banks (BANK 0 to BANK N-1) are performed. Can be done.

Meanwhile, the operation of the processing elements 221 will be described with reference to any one processing element (eg, a first processing element corresponding to the first bank BANK0).

The first processing element may perform operation processing using data DATA from the memory controller 100, and in this case, the data DATA transmitted through the bank group IO is a bank local connected to the first bank BANK0. It doesn't have to be provided as an IO. Also, the first processing element may perform an operation process using the data DATA read from the first bank BANK0, and in this case, the data DATA need not be provided to the bank group IO. In addition, the first processing element may perform an operation process using data DATA from the memory controller 100 and data DATA read from the first bank BANK0 together. In this case, the bank local IO and the bank Group IO needs to be separated electrically. Accordingly, in the operation processing operation, the operation processing may be performed in a state in which the electrical connection between the bank group IO and the bank local IO is cut off by the switching operation of the PEIO gating circuit 230.

According to the embodiment of the present invention as described above, in the memory device 200 having the bank group IO and the bank local IO structure, a transfer path of data DATA for memory operation and operation processing may be optimally set. In addition, when the PIM circuit 220 performs arithmetic processing using various data DATA, collisions between the various data DATA may be prevented. In addition, according to an embodiment of the present invention, since the memory device 200 does not perform an operation process on its own, but performs an operation process in response to a request from the memory controller 100, the memory controller 100 or a host including the same (HOST) may control the operation processing timing of the memory device 200, and performance degradation due to collision between the memory operation and the operation processing operation may be prevented.

Meanwhile, although not shown in FIG. 1, the PIM circuit 220 according to embodiments of the present invention may be variously defined, and accordingly, the PIM circuit 220 may include various components. As an example, the PIM circuit 220 may be defined as including the above-described processing elements 221 and various other components related to operation processing. For example, various components such as a controller that controls the overall operation of arithmetic processing, an instruction memory (or instruction queue) that stores instructions Inst, and instruction decoding may be provided in the PIM circuit 220.

Meanwhile, operands transmitted to the processing elements 221 may include various types of information such as data, weight information, and constants, and the terms may be used interchangeably in the following embodiments. In this case, the embodiments of the present invention need not be limited to the above terms.

FIG. 2 is a block diagram illustrating an example implementation of the memory device 200 of FIG. 1. In describing the configuration and operation of the memory device 200 of FIG. 2, the first bank BANK 0 and the second bank BANK 1 will be described as examples. Meanwhile, in FIG. 2, the first to fourth banks BANK 0 to BANK 3 are defined as one bank group BG, and accordingly, the first to fourth banks BANK 0 to BANK 3 are a bank group. An example of sharing IO (BGIO) is shown. The bank group IO (BGIO) can transfer data in both directions between the host and the bank group BG.

Referring to FIGS. 1 and 2, the memory device 200 may further include various configurations related to a memory operation. As an example, a sense amplifier 251 and a write driver corresponding to the first bank (BANK 0) 252) and a bank group input/output gating circuit (hereinafter referred to as a BGIO gating circuit 253). In addition, the sense amplifier 251 and the write driver 252 may be disposed on the bank local IO (BLIO), and the BGIO gating circuit 253 may be disposed to select the data transfer direction of the bank local IO (BLIO). I can. In FIG. 2, an example in which the BGIO gating circuit 253 is disposed on a path through which data DATA is transmitted through the write driver 252 is illustrated, but the embodiment of the present invention need not be limited thereto. As an example, the memory device 200 may be implemented such that the BGIO gating circuit 253 is disposed in a path through which the data DATA is output through the sense amplifier 251.

Meanwhile, the PEIO gating circuit 230 includes a first PEIO gating circuit 231 disposed corresponding to the first bank (BANK 0) and a second PEIO gating circuit 232 disposed corresponding to the second bank (BANK 1). It may include. The first and second PEIO gating circuits 231 and 232 may be disposed to control a moving path of the data DATA in an operation processing operation.

In the memory operation, the first PEIO gating circuit 231 electrically connects the bank local IO (BLIO) and the bank group IO (BGIO), and accordingly, the data DATA from the first bank BANK 0 is a bank group. The IO (BGIO) may be provided, or data DATA from the memory controller 100 may be provided to the first bank BANK 0 through the BGIO gating circuit 253. On the other hand, in the operation processing operation, the first PEIO gating circuit 231 electrically cuts off the bank local IO (BLIO) and the bank group IO (BGIO), and accordingly, the data DATA from the memory controller 100 is It may be provided as the first processing element PE0 without being provided as a local IO (BLIO). In addition, data DATA from the first bank BANK 0 may be provided to the first processing element PE0 without being provided to the bank group IO (BGIO).

Meanwhile, as shown in FIG. 2, the first processing element PE0 may be disposed corresponding to at least two banks, and as an example, the first processing element PE0 is connected to the second bank BANK 1. I can. When the first processing element PE0 performs arithmetic processing using the data DATA from the first bank BANK 0 and the second bank BANK 1, it is arranged corresponding to the second bank BANK 1 The second PEIO gating circuit 232 may cut off electrical connection between the bank group IO (BGIO) and the bank local IO (BLIO) corresponding to the second bank (BANK 1).

In addition, in example embodiments, when the first processing element PE0 is shared among a plurality of banks, the result of operation processing using data DATA of any one bank may be stored in another bank. As an example, data DATA read from the first bank BANK 0 is provided as an operand to the first processing element PE0, and the operation result from the first processing element PE0 is the second bank BANK 1 ).

3 is a block diagram showing a data processing system including a memory device according to an exemplary embodiment of the present invention. As shown in FIG. 3, the data processing system 300 may include an application processor 310 and a memory device 320, and the application processor 310 is a memory communicating with the memory device 320. It may include a control module 311. As an example, the memory control module 311 and the memory device 320 may constitute a memory system. The application processor 310 may perform a host function in FIG. 1.

The application processor 310 may be implemented as a System on Chip (SoC) including a system bus (not shown). As a standard standard for a system bus, the AMBA (Advanced Microcontroller Bus Architecture) protocol of ARM (Advanced RISC Machine) can be applied. The bus type of the AMBA protocol may include Advanced High-Performance Bus (AHB), Advanced Peripheral Bus (APB), Advanced eXtensible Interface (AXI), AXI4, AXI Coherency Extensions (ACE), and the like. In addition, other types of protocols such as uNetwork of Sonics Inc., CoreConnect of IBM, and Open Core Protocol of OCP-IP may be applied.

The memory control module 311 may perform the function of a memory controller in the above-described embodiment, and control a memory operation by transmitting a command/address (CMD/ADD) to the memory device 320 or control a memory operation. ), you can control the operation processing within. According to the above-described embodiments, the memory device 300 may include a memory bank 321, a PIM circuit 322, a PEIO gating circuit 323, and a control logic 324, and the PIM circuit 322 The processing elements 322_1 are included, and the control logic 324 may include a command/address decoder (not shown). In addition, the application processor 310 may provide a plurality of instructions (Inst) to the memory device 320 for processing an operation inside the memory device 320, and an instruction memory (not shown) inside the memory device 320 ) May be stored in the plurality of instructions (Inst).

In addition, according to the above-described embodiment, the application processor 310 may provide a command CMD_PE instructing an operation process to the memory device 320. The control logic 324 may determine an operation processing mode based on a result of decoding the received command/address CMD/ADD, and based on this, the switching state of the PEIO gating circuit 323 may be controlled. For example, the electrical connection between the bank local IO and the bank group IO may be cut off according to the switching state of the PEIO gating circuit 323 in the operation processing operation, and thus data (DATA) used for operation processing as an operand Collision of people can be prevented.

4 is a conceptual diagram illustrating an example of an operation of a memory system according to an exemplary embodiment of the present invention. In FIG. 4, an example in which a memory device communicates with a host HOST is shown, and a memory controller (or a memory control module) performing communication with the memory device may be provided in the host.

Referring to FIG. 4, a host (HOST) can control various operation modes of a memory device, and as the memory device operates in an instruction loading mode, the host (HOST) transmits a number of commands necessary for operation processing to the memory device. The transmitted command may be loaded into the command memory in the memory device. As an example, various types of operation processing, such as addition, subtraction, and multiplication operations, may be performed in the memory device, and the PIM circuit may determine the type of operation corresponding to the instruction by reading the instruction and decoding it in the operation processing mode. .

The host HOST may perform a control operation so that the memory device operates in the normal mode, and as an example, the host may transmit a command/address CMD/ADD for instructing a memory operation to the memory device. . The memory device may decode the received command/address CMD/ADD and perform a corresponding memory operation when the command CMD instructs a memory operation such as a write command CMD_WR and a read command CMD_RD. . In addition, as the processing of the memory operation is completed, a result of the memory operation may be output to the host.

Meanwhile, the host HOST may transmit a command/address CMD/ADD for instructing operation processing to the memory device. As the calculation command CMD_PE is provided to the memory device, the memory device may perform calculation processing based on a command/address (CMD/ADD) decoding operation. As an example, the memory device controls the PEIO gating circuit disposed in response to the bank selected in the operation processing mode, thereby performing operation processing in a state in which the electrical connection between the bank local IO (BLIO) and the bank group IO (BGIO) is cut off. I can. As an example, the processing element in the PIM circuit can perform operation processing using at least one of data from the host, data read from the selected bank, and information stored in the processing element, and the result of the operation is transmitted to the host. Can be. Alternatively, according to an exemplary embodiment, the operation processing result may be stored in a register in the processing element or in a cell array of a bank, and the host transmits a command/address for reading the operation processing result to the memory device. It could also be transmitted.

5 is a block diagram illustrating an example of various components included in a memory device. In FIG. 5, an example in which a memory device includes a plurality of channels (eg, first and second channels CH1 and CH2) is shown, and each channel is a separate input/output line (IO) communicating with an external host. ) Can be included.

Taking the first channel CH1 as an example, the first channel CH1 may include a plurality of bank groups (eg, first and second bank groups BG0 and BG1), and the first and second Each of the bank groups BG0 and BG1 may include a plurality of banks. As an example, the first bank group BG0 may include first to fourth banks BANK 0 to BANK 3, and as one processing element is shared for two banks, the first and The second processing elements PE0 and PE1 may be disposed corresponding to the first bank group BG0. In addition, although not shown in FIG. 5, the bank group IO may be disposed corresponding to the first bank group BG1, and each of the first to fourth banks BANK 0 to BANK 3 is assigned to the corresponding bank local IO. Can be connected.

The data bus may include a path for transmitting data accessed in the first channel CH1, and the bank controller may perform a bank control operation for accessing data of banks included in the first channel CH1. have. As an implementation example, one bank controller may be disposed corresponding to one bank. The bank controller may control an active operation and a precharge operation for rows included in a corresponding bank, and may also control a column selection operation for data access.

Meanwhile, a processing element controller (hereinafter referred to as a PE controller) may control the processing elements PE0 to PE3. The PE controller may be disposed in various ways, and as an example, a PE controller may be disposed corresponding to a bank group or a PE controller may be disposed corresponding to a channel. In addition, the command/address (CMD/ADD) decoder may perform a decoding operation on a command/address from a host, and control a memory operation or an operation processing operation based on a decoding result. As an example, the PE controller may perform various control operations related to arithmetic processing based on a decoding operation of a command/address (CMD/ADD) decoder.

The PE controller may control the enable of the processing element or control the operation of the processing element and control the transmission path of data according to the above-described embodiments. As an example, one or more PEIO gating circuits may be disposed in the memory device, and the PE controller may generate control signals for controlling the switching operation of the PEIO gating circuit.

Meanwhile, in the embodiment illustrated in FIG. 5, bank groups may be variously defined. For example, in the above-described embodiment, the bank group has been described as including a cell array of a plurality of banks, but the bank group may be defined as further including elements related to memory operation and operation processing operation together with the above-described cell array. There will be. As an example, one or more processing elements (PE), PEIO gating circuit may be defined as being included in the bank group.

6 and 7 are block diagrams illustrating examples of configurations included in a memory device in relation to operation processing.

5 to 7, the memory device 400 may include a plurality of processing elements, and the processing element (eg, first PE(PE0)) shown in FIG. 6 is an input selector 410 and an operator. 420, a register file 430, and an output selector 440. In FIG. 6, a SIMD (single instruction multi data) operator is illustrated as an example of the operator 420, but embodiments of the present invention need not be limited thereto, and various types of operators may be applied to the first PE (PE0). will be. In addition, an example in which various pieces of information used for calculation include N bits is illustrated, but the number of bits may be variously changed.

The register file 430 may store various types of information related to operation processing. As an example, the register file 430 may store data including an operand used for an operation, an operation result, various preset information (eg, a constant value for scaling, etc.). File 430 may contain one or more registers.

The input selector 410 may provide data to be used for calculation to the calculator 420 in response to the input control signal INPUT_CTRL. Taking the first PE (PE0) as an example, the input selector 410 may receive data from the first bank (BANK 0) and data from the second bank (BANK 1) through a bank local IO (BLIO), and , Data from the host can be received through the bank group IO (BGIO), and data stored in the register file 430 (eg, register output (reg_out)) can be received, and the input control signal (INPUT_CTRL) In response, at least one piece of data may be provided to the operator 420.

The operator 420 may perform an operation in response to the operation control signal CAL_CTRL, and may provide an operation result as an input reg_in of the register file 430. The operation result may be stored in the register file 430, and the register file 430 transmits stored information (e.g., operation result) to the output selector 440 in response to the register control signal REG_CTRL. Can provide. In addition, the output selector 440 may output an operation result through various paths in response to the output control signal OUTPUT_CTRL, and as an example, the first bank (BANK 0) through the bank local IO (BLIO). Alternatively, it may be provided as a second bank (BANK 1), may be provided to a host through a bank group IO (BGIO), or may be provided as an input of the input selector 410 for an operation operation.

Meanwhile, FIG. 7 shows an example of an implementation of the PE controller. As shown in FIG. 7, the memory device 400 includes a command queue 460 corresponding to a command memory and a command decoder 470 along with the PE controller 450. ) And a command/address decoder 480 may be further included. The various components shown in FIGS. 6 and 7 may be defined as being provided in the PIM circuit described in the above-described embodiment, and as an example, the command queue 460 and the command decoder 470 are included in the PIM circuit. It may be a provided configuration.

The PE controller 450 may perform various control operations related to operation processing based on a command/address (CMD/ADD) or a decoding result thereof. The address ADD may include a bank address BA, a row address RA, and a column address CA. In addition, the PE controller 450 may generate the above-described various control signals (INPUT_CTRL, CAL_CTRL, REG_CTRL, OUTPUT_CTRL) based on the decoded command/address (CMD/ADD), and also according to the above-described embodiments. By controlling the switching state of the PEIO gating circuit, a PEIO control signal PEIO_CTRL for controlling the electrical connection between the bank local IO (BLIO) and the bank group IO (BGIO) can be output.

Meanwhile, the operation processing may include various types of operations such as addition, subtraction, and multiplication operations, and the type of operation may be determined through a decoding operation for an instruction. The command queue 460 may receive a plurality of commands Inst from a host, and may store, for example, commands Inst received through a data bus. The PE controller 450 provides a command control signal INST_CTRL to the command queue 460 to control an operation in which the command Inst is stored in the command queue 460 or the command Inst is read. In addition, the instruction Inst read from the instruction queue 460 may be provided to the instruction decoder 470, and the instruction decoder 470 may perform decoding processing thereon.

8 is a block diagram illustrating an example in which a memory device according to an exemplary embodiment of the present invention includes a high bandwidth memory (HBM).

The HBM 500 may have an increased bandwidth by including a plurality of channels having independent interfaces from each other. Referring to FIG. 5, the HBM 500 may include a plurality of dies, and as an example, may include a buffer die (or logic die 510) and one or more core dies stacked thereon. In the example of FIG. 5, an example in which four core dies are provided in the HBM 500 is illustrated, but the number of core dies may be variously changed. The configuration of FIG. 8 will be described with reference to the first core die 520 among the core dies as follows.

The first core die 520 may include one or more channels, and in the example of FIG. 8, a case where the first core die 520 includes two channels CH1 and CH2 is illustrated. The buffer die 510 may include an interface circuit (not shown) that communicates with a host (or a memory controller), and may receive commands/addresses and data from the host through the buffer die 510. Further, according to an exemplary embodiment of the present invention, each of the channels CH1 and CH2 of the first core die 520 may include a command/address decoder 521 and a PIM circuit 522, and a PIM circuit Reference numeral 522 may control an internal operation process based on a result of decoding a command/address. In addition, although not shown in FIG. 8, various components related to the operation process described in the above-described embodiments may be provided in each of the channels CH1 and CH2. As an example, each of the channels CH1 and CH2 may include a bank group including a plurality of banks, a bank local IO (BLIO) and a bank group IO (BGIO), and a bank local IO (BLIO) and a bank group It may include a PEIO gating circuit for controlling the electrical connection between IO (BGIO).

Meanwhile, in FIG. 8, an implementation example in which each channel of the HBM 500 includes at least two pseudo channels is disclosed. As an example, the first channel CH1 of the first core die 520 may include two pseudo channels PC0 and PC1, and a data bus corresponding to the two pseudo channels PC0 and PC1 While is implemented separately from each other, the two pseudo-channels PC0 and PC1 may share the command/address decoder 521. In addition, according to various embodiments, at least one of various components included in the PIM circuit 522 may be shared by two pseudo channels PC0 and PC1. That is, while the pseudo-channels PC0 and PC1 may interface with the host through separate data buses, they may interface with the host through a common command/address bus.

9 is a flowchart showing a method of operating a memory device according to an exemplary embodiment of the present invention.

Referring to FIG. 9, the memory device may receive commands provided from a host and store them, and may perform operation processing by itself according to a request from the host. The memory device may receive a command/address from the host and may perform decoding processing on the received command/address (S11). In addition, according to the decoding result, the memory device may operate in an operation processing mode. As an example, a separate command for operation processing is defined according to the above-described embodiments, or a plurality of bits included in the command/address The arithmetic processing mode may be selected based on the value.

Based on the decoding result, it is determined whether it corresponds to the operation processing mode (S12), and if it does not correspond to the operation processing mode, the bank local IO (BLIO) and the bank group IO (BGIO) of the bank selected for normal memory operation are electrically As the memory operation is performed (S13), data may be provided to the selected bank or data may be read from the selected bank (S14).

On the other hand, in the case of the operation processing mode, the bank local IO (BLIO) and the bank group IO (BGIO) are electrically cut off for the operation processing operation (S15), and data and/or information for the operation processing is transferred to the processing element ( PE). As an example, data from the host may be provided to the processing element PE through the bank group IO (BGIO) (S16), and data read out from the selected bank may be provided through the bank local IO (BLIO) through the processing element ( PE) may be provided (S17). The processing element PE may perform arithmetic processing using the received data (S18).

Meanwhile, the operation processing performed by the processing element PE according to the above-described embodiments may include neural network operations. As an example, there are various types of neural networks such as deep learning algorithms, and neural network operations are various types of hardware (or processors) such as general hardware such as CPU or GPU, or dedicated hardware optimized for specific software. Can be implemented by As an example, the dedicated hardware may include various types of hardware such as an ASIC, a neural processing unit (NPU), a tensor processing unit (TPU), and a neural engine.

According to an exemplary embodiment of the present invention, at least some of the neural network operations may be performed in a memory device according to embodiments of the present invention. Neural network operations may include operations using data and weights, and data and weights used in the operations may be provided from a host or stored in a memory device. As an example, when data is provided from a host and a weight is stored in a bank of a memory device, data is transferred to the processing element PE through the bank group IO (BGIO). The weight may be provided, and the weight may be read from the selected bank and provided to the processing element PE through the bank local IO BLIO.

10A, B, and C are diagrams illustrating examples of operation commands defined in a memory device according to an embodiment of the present invention.

According to embodiments of the present invention, various operation commands (PERW, PERD, PEWR) may be defined in relation to operation processing as shown in FIGS. 10A, B, and C, and the memory device includes operation commands PERW , PERD, PEWR) is received, it is possible to perform operation processing by decoding the instruction previously stored in the instruction memory. In addition, the transmission path of data used for calculation may be different depending on the type of calculation commands (PERW, PERD, PEWR), and as an implementation example, based on the decoding result of calculation commands (PERW, PERD, PEWR). Paths for transmitting data used for operation processing may be controlled differently.

As an example, referring to FIG. 10A, a first operation command PERW may be provided from a host to a memory device, and an operation using data from the host and data in the memory device in response to the first operation command PERW This can be done. Although not shown in FIGS. 10A, B, and C, one or more banks are selected so that a row in the bank may have an active state, and data is read from the bank according to the column address information COL included in the address ADD, so that the bank is local. It may be provided to the processing element PE through IO (BLIO). In addition, after a predetermined latency, data (DATA) from the host may be transmitted through the bank group IO (BGIO) and provided to the processing element PE, and the processing element PE may perform operation processing using the received data. I can.

Meanwhile, referring to FIG. 10B, a second operation command PERD may be provided from a host to a memory device, and an operation using data in the memory device may be performed in response to the second operation command PERD. As an example, data is read from the bank according to the column address information (COL) and provided to the processing element PE through the bank local IO (BLIO), and the processing element PE processes an operation using data from the bank. You can do it. As an example of operation, data read from a bank corresponds to a weight of a neural network operation, and information (eg, a predetermined constant value) for scaling a value of the weight is included in the processing element PE. When stored in a register, the processing element PE may perform an operation of calculating the weight and the constant value.

Meanwhile, referring to FIG. 10C, a third operation command PEWR may be provided from a host to a memory device, and information stored in a register in the processing element PE in response to the third operation command PEWR (eg, operation The operation in which the result) is written to the bank can be performed. As an example, information on a register in the processing element PE is read based on a result of decoding a command/address, and the read information is a position corresponding to the column address information COL of the bank through the bank local IO (BLIO). Can be provided with.

Although only some of the operations that can be performed in embodiments of the present invention are mentioned in FIGS. 10A, B, and C described above, the memory device of the present invention may also perform various other types of operations. As an example, various types of operations may be performed according to a command read in response to operation commands (PERW, PERD, PEWR), and the types of the operation commands (PERW, PERD, PEWR) are data transfer. It could be used to control the path. For example, the host may further provide data while providing the third operation command PEWR, and the processing element PE may perform operation processing using data provided from the host in response to the third operation command PEWR. Maybe.

Meanwhile, the host may sequentially provide instructions for processing a series of operations to the memory device, and accordingly, determine the type of operation to be performed at each operation timing. The host may select an appropriate command and provide the above-described operation commands PERW, PERD, and PEWR to the memory device based on the type of operation processing performed at each operation timing.

11 is a waveform diagram illustrating an example of operation processing sequentially performed in a memory device according to embodiments of the present invention. In FIG. 11, an example in which rows of a plurality of banks are simultaneously selected and arithmetic processing is performed in parallel by a plurality of processing elements is shown.

Referring to FIG. 11, an active command PEACT for activating a plurality of rows included in a plurality of banks together may be defined, and as a host provides an active command PEACT to a memory device, a plurality of banks Each row (eg, a first row (row0)) may be active.

The host may provide one or more operation commands to the memory device so that operation processing may be performed in series in the memory device. As an example, the first to third operation commands PERW and PERD, PEWR) can be provided as a memory device. As an example, the transmission path of data and/or information of the memory device may be controlled based on the decoding result of the first to third operation commands (PERW, PERD, PEWR), and a command read from the command memory (eg, The type of operation processing may be determined based on the decoding operation of the first instruction (Inst 0).

As an example, the memory device receives an address indicating a first column (col 0) together with a third operation command (PEWR), and according to the decoding result (mov) of the first command (Inst 0), the processing element PE Data and/or information can be written to a register (not shown). As an example, as rows of a plurality of banks are active, a plurality of processing elements PE may perform an operation in parallel. As an example, data from a data bus is registered in response to a third operation command PEWR. As an input reg_in, it may be provided to multiple processing elements PE in parallel.

In addition, the memory device may receive an address indicating the first operation command PERW and the second column col 1 from the host, and in response thereto, read and decode the second command Inst 1, and the second command According to the decoding result (mac) of (Inst 1), each of the plurality of processing elements (PE) is read from the information stored in the register (e.g., information provided from the host and stored in the register) and the second column (col 1) of the bank. Calculation processing (eg, multiplication and accumulation (MAC) processing) using the generated data may be performed.

After the parallel operation processing by the plurality of processing elements PE is completed, a precharge command PEPRE may be provided from the host to the memory device to precharge the rows of the banks. In addition, according to an operation example, operation results stored in a plurality of processing elements PE may be written to the banks. To this end, a host may write a row (eg, a second row) together with an active command PEACT. )) may be provided to the memory device, and the memory device may receive a third operation command PEWR and an address (eg, a first column (col 0)). The memory device may read and decode the third instruction Inst 2 in response to the third operation command PEWR, and the operation processing results stored in the plurality of processing elements PE are each stored according to the decoding result (store). It can be stored in the corresponding bank. Thereafter, rows of banks of the memory device may be precharged according to a precharge command PEPRE from the host.

12A,B through 14A,B are diagrams illustrating implementation examples of a memory device according to various embodiments of the present disclosure.

Referring to FIGS. 12A and 12B, the memory device 600A may include one or more bank groups, and as an example, the first bank group BG0 may include a plurality of banks BANK 0 to BANK 3. In addition, processing elements may be disposed corresponding to each of the banks BANK 0 to BANK 3. As an example, FIG. 12A shows an example in which the first and second processing elements 611A and 612A are arranged corresponding to the first and second banks BANK 0 and BANK 1, and the first PEIO gating circuit ( 621A) may be disposed to control the connection between the bank group IO (BGIO) and the bank local IO (BLIO) of the first bank (BANK 0), and the second PEIO gating circuit 622A is the bank group IO (BGIO). And the bank local IO (BLIO) of the second bank (BANK 1).

Meanwhile, referring to FIG. 12B, the memory device 600A may include a plurality of channels (eg, first and second channels CH1 and CH2), and each channel includes one or more bank groups. can do. As an example, the components illustrated in FIG. 12A may be components included in the first channel CH1 of the memory device 600A. In addition, as in the above-described embodiment, various components for controlling a memory operation and an operation processing operation for each channel may be provided in the memory device 600A. As an example, a data bus and a bank A controller, a PE controller, and a command/address decoder, etc. are illustrated.

According to the embodiment shown in FIGS. 12A and 12B, parallel operation processing may be performed using data stored in each of the banks BANK 0 to BANK 3. Taking a neural network operation using weight information stored in the banks BANK 0 to BANK 3 as an example, data from the host is commonly provided to the processing elements PE0 to PE3 shown in FIG. 12A, and the processing elements ( PE0 to PE3) may perform operations using weights having different values in parallel using the same data.

13A and 13B illustrate an example in which the memory device 600B includes one or more bank groups, and one processing element 610B is disposed in correspondence with one bank group (eg, first bank group BG0). . In addition, PEIO gating circuits 621B to 624B may be disposed corresponding to the banks BANK 0 to BANK 3 of the first bank group BG0.

Data from at least one bank among the banks (BANK 0 to BANK 3) may be provided to the processing element 610B according to the type of operation processing (or operands used in the operation processing), and used for operation processing. Connection states of the PEIO gating circuits 621B to 624B may be controlled according to data and/or information. In addition, in FIG. 13B, when the memory device 600B includes the components as illustrated in FIG. 13A, a plurality of channels (eg, first and second channels CH1) provided in the memory device 600B are , CH2)) is shown.

Meanwhile, in FIG. 14A, an example of a configuration in which one processing element is disposed corresponding to one bank group, and the processing element is disposed on the bank group IO (BGIO) is shown.

Referring to FIGS. 14A and 14B, the memory device 700 includes a first bank group BG0 including a plurality of banks BANK 0 to BANK 3, and is disposed corresponding to the first bank group BG0. The processing element 710 may receive data from a plurality of banks BANK 0 to BANK 3 through a bank group IO (BGIO), and may also receive data from an external host through a data bus. In addition, the memory device 700 further includes a PEIO gating circuit 720 disposed in correspondence with the first bank group BG0, and the PEIO gating circuit 720 is formed between the bank group IO (BGIO) and the data bus. Electrical connection can be controlled.

As an example of operation, during an operation processing operation, the PEIO gating circuit 720 may cut off the electrical connection between the bank group IO (BGIO) and the data bus, and the data from the host provided to the processing element 710 and the bank group IO Collision of data through (BGIO) can be prevented. Meanwhile, in FIG. 14B, when the memory device 700 includes components as illustrated in FIG. 14A, a plurality of channels (eg, first and second channels CH1) provided in the memory device 700 , CH2)) is shown. As an example, unlike the above-described embodiments, the processing elements 710 and PE0 may have a structure connected between the bank group IO (BGIO) of the first bank group BG0 and the data bus.

15 is a diagram illustrating an example implementation of a memory device according to a deformable embodiment of the present invention. In FIG. 15, an example of information stored in a cell array of any one bank among a plurality of banks included in a bank group of a memory device is illustrated.

According to example embodiments, the memory device may include a PIM circuit therein, and may perform operation processing in response to a normal command from a memory controller (or host). As an example, the normal command may include a write command and a read command for instructing a memory operation, and a portion of the cell array stores control information for controlling the PIM circuit along with a command for performing an operation process. It can be used as a PIM area. 15 illustrates an example of storing the above command and control information in the first row (Row 0) of the cell array, embodiments of the present invention need not be limited thereto, and the PIM area may be variously set.

According to an exemplary embodiment, the host may perform a control operation so that the memory device performs operation processing by instructing to read the PIM region of the cell array. The memory device may read information corresponding to a command in the above-described embodiment and various control information for operation processing, and perform operation processing using the information. As an example, the memory device may determine an operation type (e.g., data movement, MAC operation, etc.) based on a read command, and control or process an active/precharge operation based on the read control information. Whether the device is enabled, and control operations of various paths in the above-described embodiments may be performed.

16 is a block diagram showing a server system including a data processing system according to an embodiment of the present invention.

Referring to FIG. 16, the server system 800 may include a manager 810 and a plurality of servers 820_1 to 820_K. Each of the plurality of servers 820_1 to 820_K may correspond to the data processing system described in the above-described embodiments. The plurality of servers 820_1 to 820_K are connected to each other through a bus supporting a predetermined protocol (eg, PCI, PCIe, etc.), and as an example, the plurality of servers 820_1 to 820_K are controlled by the manager 810 Can communicate with each other through a P2P connection structure.

Referring to any one server (eg, the first server 820_1), the first server 820_1 includes a host and one or more memory devices MEM according to the above-described embodiments, and It can perform various types of operation processing and store the processing result. According to an embodiment, each of the memory devices MEM may include a plurality of banks and processing elements arranged corresponding thereto, and the host provides an operation command to control the operation processing timing of the memory device MEM can do. In addition, according to the above-described embodiments, the memory device MEM may have a feature in that the bank local IO (BLIO) and the bank group IO (BGIO) are arranged in relation to a plurality of banks. Components for blocking electrical connection between the IO (BLIO) and the bank group IO (BGIO) may be included. In various embodiments, the server system 800 corresponds to a neural network server system, and the first server 820_1 performs at least some of the operations by the memory device MEM in performing a vast amount of neural network operations. It will be possible to perform a control operation on the memory device MEM so that it is possible.

As described above, exemplary embodiments have been disclosed in the drawings and specifications. In the present specification, embodiments have been described using specific terms, but these are only used for the purpose of describing the technical idea of the present disclosure, and are not used to limit the meaning or the scope of the present disclosure described in the claims. . Therefore, those of ordinary skill in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical scope of the present disclosure should be determined by the technical spirit of the appended claims.

Claims (10)

In the memory device,
A memory bank including at least one bank group, each bank group including a plurality of banks;
A PIM (Processor in Memory) that performs an operation process using at least one of data provided from a host and data read from the bank group, wherein the first processing element is disposed corresponding to the bank group. ) Circuit;
Processing element input/output (PEIO) controlling electrical connection between a bank local input/output line (IO) disposed for each of the plurality of banks of the bank group and a bank group input/output line (IO) disposed for the bank group Gating circuit; And
A control logic for performing a control operation to perform a memory operation for the memory bank or an operation process by the PIM circuit based on a result of decoding the command/address received from the host,
When an operation process is performed by the first processing element, the PEIO gating circuit cuts off an electrical connection between the bank local IO and the bank group IO. The method of claim 1,
Wherein the first processing element performs operation processing using at least one of data from the host, data read from the bank group, and data stored in an internal register. The method of claim 1,
The bank group includes a first bank and a second bank,
The PEIO gating circuit includes a first PEIO gating circuit disposed in a first bank local IO corresponding to the first bank, and a second PEIO gating circuit disposed in a second bank local IO corresponding to the second bank. And
Wherein the first processing element receives data from at least one of the first bank and the second bank to perform the operation processing. The method of claim 3,
When an operation process is performed by the first processing element, an electrical connection between the first and second bank local IOs and the bank group IO is cut off. The method of claim 1,
The memory device further includes a command memory for storing at least one command provided from the host,
And when a command/address instructing the arithmetic process is received from the host, the PIM circuit performs arithmetic processing corresponding to the command read out from the command memory. The method of claim 5, wherein the PIM circuit,
And a processing element (PE) controller for controlling storage and reading of the instruction and generating control signals for controlling the first processing element in connection with the operation processing. The method of claim 6, wherein the first processing element,
An input selector for receiving data transmitted through the bank local IO and data transmitted through the bank group IO, and outputting data to be used for operation;
An operator that performs an operation on one or more data provided from the input selector; And
And a register file storing the result of the operation by the operator. In the memory device,
A memory bank including a bank group, the bank group including first and second banks;
A bank group input/output line (IO) disposed corresponding to the bank group and including a data transmission path between an external host and the bank group;
A first bank local input/output line (IO) disposed corresponding to the first bank and including a transmission path of data provided to the first bank and data read from the first bank;
A first processing element connected to the bank group IO and the first bank local IO and performing an operation process using at least one of data provided from the host and data read from the first bank; And
First processing element input/output (PEIO) gating for controlling a data transfer path so that data from the host is provided to the first processing element without being provided to the memory bank when an operation process is performed by the first processing element A memory device comprising a circuit. The method of claim 8,
The first PEIO gating circuit is disposed between the bank group IO and the first bank local IO,
When the operation processing by the first processing element is performed, the first PEIO gating circuit cuts off a connection between the bank group IO and the first bank local IO. The method of claim 8,
The first PEIO gating circuit is disposed between the bank group IO and a data bus transferring data from the host,
And the first PEIO gating circuit cuts off a connection between the bank group IO and the data bus when operation processing is performed by the first processing element.

Images