KR102656104B1 - Non-volatile memory controller device and non-volatile memory device - Google Patents

Abstract

A non-volatile memory control device connects a non-volatile memory module with a host including a processor and host memory. The first doorbell area is exposed to the configuration space of the host interface for connection to the host, and is updated when the host issues an I/O request command to the host memory. The fetch management module retrieves commands from host memory in response to an event signal generated when the first doorbell area is updated. The data transfer module determines the location of the host memory based on the request information included in the command and transfers target data according to the I/O request between the host memory and the non-volatile memory module. When the completion handling module completes processing of an I/O request in the data transfer module, the completion handling module records the completion request in the host memory and processes the interrupt. The second doorbell area is exposed in the configuration space and is updated when the host terminates the I/O service.

Description

Non-volatile memory control device and non-volatile memory device {NON-VOLATILE MEMORY CONTROLLER DEVICE AND NON-VOLATILE MEMORY DEVICE}

The technology described below relates to non-volatile memory control devices and non-volatile memory devices.

Recently, as new non-volatile memories such as solid-state drives (SSD) as well as phase-change random-access memory (PRAM) or magnetoresistive random-access memory (MRAM) have been used, communication between the host and non-volatile memory has become more common. A protocol is needed. NVMe (non-volatile memory express) is used as a representative protocol.

Currently, many SSD vendors are implementing controllers according to the NVMe specification as firmware. Even if a firmware-based controller is sufficient for flash, firmware execution can be a significant performance bottleneck that exposes the performance superiority offered by fast non-volatile memories such as PRAM and MRAM. The latency of fast non-volatile memory is much shorter than flash. CPU cycles executing firmware cannot be hidden or overlap behind the input/output (I/O) burst times of the backend non-volatile memory.

Specifically, the latency of an NVMe SSD can be decomposed into two parts: NVMe firmware execution time (frontend) and backend memory access latency (backend). Access latency in flash-based backends accounts for the majority (approximately 90% or more) of overall service time, allowing front-end firmware execution to be hidden behind flash I/O bursts. However, in PRAM or MRAM, firmware execution time accounts for most of the total service time, so firmware execution cannot be interleaved with the I/O burst time of PRAM or MRAM.

Therefore, in new non-volatile memories, there is a need to reduce or eliminate firmware execution time, which accounts for the majority of charter service time.

Certain embodiments of the present invention may provide a hardware-automated non-volatile memory control device and a non-volatile memory device.

According to one embodiment of the present invention, a non-volatile memory control device is provided that connects a host including a processor and host memory and a non-volatile memory module. The non-volatile memory control device includes a first doorbell area, a second doorbell area, a fetch management module, a data transmission module, and a completion handling module. The first doorbell area is exposed to the configuration space of the host interface for connection to the host, and is updated when the host issues an I/O (input/output) request command to the host memory. The fetch management module retrieves the command from the host memory in response to an event signal generated when the first doorbell area is updated. The data transfer module confirms the location of the host memory based on the request information included in the command and performs transfer of target data according to the I/O request between the host memory and the non-volatile memory module. . When the data transfer module completes processing of the I/O request, the completion handling module records the completion request in the host memory and processes an interrupt. The second doorbell area is exposed in the setting space and is updated when the host terminates the I/O service.

In some embodiments, the first doorbell area, the fetch management module, the data transmission module, the completion handling module, and the second doorbell area may be implemented as hardware.

In some embodiments, the first doorbell area, the fetch management module, the data transfer module, the completion handling module, and the second doorbell area may be implemented in hardware at a register transfer level (RTL). there is.

In some embodiments, the first doorbell area, the fetch management module, the data transmission module, the completion handling module, and the second doorbell area may be connected by an internal memory bus of the non-volatile memory control device.

In some embodiments, the internal memory bus may include an advanced extensible interface (AXI) interface.

In some embodiments, the first doorbell area and the second doorbell area may be mapped to the address space of the internal memory bus.

In some embodiments, the host interface may include a peripheral component interconnect express (PCIe) interface, and the configuration space may include base address registers (BAR).

In some embodiments, the fetch management module may parse the request information from the command.

In some embodiments, the request information may include a logical address and a host memory address indicating the location of the host memory. The data transmission module includes a first engine that sets a source address and a destination address for the data transmission module based on the logical address and the host memory address, and transmission of the target data based on the source address and the destination address. It may include a second engine that performs.

In some embodiments, the second engine may include a direct memory access (DMA) engine.

In some embodiments, the host memory address may include a physical region page (PRP).

In some embodiments, the first engine retrieves a PRP list from the host memory based on the PRP, the PRP list includes at least one PRP entry referencing a location in the host memory, and the first engine Can detect the location of the host memory referenced by the PRP entry.

In some embodiments, the non-volatile memory control device is connected to the non-volatile memory module through a plurality of channels, and the second engine may include a plurality of DMA cores respectively corresponding to the plurality of channels.

In some embodiments, the second engine may divide the target data of the I/O request into a plurality of data chunks and distribute the plurality of data chunks to the plurality of DMA cores.

In some embodiments, a plurality of queue pairs exist in the host memory, and each queue pair may include a pair of a submission queue (SQ) and a competition queue (CQ). The fetch management module may retrieve the command from the SQ of the queue pair corresponding to the updated entry in the first doorbell area among the plurality of queue pairs.

In some embodiments, the completion handling module may detect a CQ corresponding to the SQ that retrieved the command based on information transmitted from the fetch management module.

According to another embodiment of the present invention, a non-volatile memory control device is provided that connects a host including a processor and host memory and a non-volatile memory module. The non-volatile memory control device includes a first module, a second module, a third module, a first engine, and a second engine. The first module is exposed to the BAR of the PCIe interface for connection to the host, and a first doorbell area where an entry corresponding to the target SQ is updated when a command is written to the target SQ of the host memory and the host It includes a second doorbell area in which an entry corresponding to the target CQ is updated when the I/O service is terminated. The second module retrieves the command from the target SQ in response to an event signal generated when the first doorbell area is updated. The first engine detects the location of the host memory based on the PRP included in the command. The second engine transmits target data according to the I/O request between the location of the host memory detected by the first engine and the non-volatile memory module. When the second engine completes processing of the I/O request, the third module records a completion request in the target CQ and processes an interrupt.

In some embodiments, the first engine retrieves a PRP list from the host memory based on the PRP, the PRP list includes at least one PRP entry referencing a location in the host memory, and the first engine Can detect the location of the host memory referenced by the PRP entry.

In some embodiments, the first module, the second module, the first engine, the second engine, and the third module may be implemented as hardware.

According to another embodiment of the present invention, a non-volatile memory device coupled to a host including a processor and host memory is provided. The non-volatile memory device includes a non-volatile memory module, a first doorbell area, a second doorbell area, a fetch management module, a data transmission module, and a completion handling module. The first doorbell area is exposed to the configuration space of the host interface for connection to the host, and is updated when the host issues an I/O request command to the host memory. The fetch management module retrieves the command from the host memory in response to an event signal generated when the first doorbell area is updated. The data transfer module confirms the location of the host memory based on the request information included in the command and performs transfer of target data according to the I/O request between the host memory and the non-volatile memory module. . When the data transfer module completes processing of the I/O request, the completion handling module records the completion request in the host memory and processes an interrupt. The second doorbell area is exposed in the setting space and is updated when the host terminates the I/O service.

1 is an example block diagram of a computing device according to one embodiment of the present invention.
Figure 2 is a block diagram of a typical storage device.
Figure 3 is an example block diagram of an NVMe device according to one embodiment of the present invention.
Figure 4 is a diagram illustrating an NVMe control device according to an embodiment of the present invention.
Figure 5 is a diagram illustrating the operation of an NVMe control device according to an embodiment of the present invention.
Figure 6 is a diagram illustrating the connection relationship of an NVMe control device according to an embodiment of the present invention.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

In the description below, expressions described as singular may be interpreted as singular or plural, unless explicit expressions such as “one” or “single” are used.

In the flowchart described with reference to the drawings, the order of operations may be changed, several operations may be merged, certain operations may be divided, and certain operations may not be performed.

1 is an example block diagram of a computing device according to one embodiment of the present invention.

Referring to FIG. 1 , the computing device 100 includes a processor 110, a memory 120, a non-volatile memory control device 130, and a memory module 140. 1 is an example of a possible computing device, and the computing device may be implemented in a variety of other structures.

In some embodiments, the computing device may be any of various types of computing devices. Various types of computing devices include mobile phones such as smartphones, tablet computers, laptop computers, desktop computers, multimedia players, and game consoles. It may include game consoles, televisions, various types of Internet of things (IoT) devices, etc.

The processor 110 executes instructions to perform various operations (eg, operations, logic, control, input/output, etc.). The processor may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, or an application processor (AP), but is not limited thereto. Below, the processor 110 is described as CPU 110.

The memory 120 is system memory accessed and used by the CPU 110, and may be, for example, dynamic random-access memory (DRAM). In some embodiments, CPU 110 and memory 120 may be connected via a system bus. A system including the processor 110 and memory 120 may be referred to as a host. The memory 120 may be referred to as host memory.

The memory module 140 is a non-volatile memory-based memory module. In some embodiments, memory module 140 may be a flash memory-based memory module. The flash memory-based memory module 140 may be, for example, a solid state drive (SSD), a secure digital (SD) card, or a universal serial bus (USB) flash drive. In some embodiments, the memory module 140 may be a memory module based on resistance switching memory. In one embodiment, the resistance change memory is a phase-change memory (PCM) that utilizes the resistivity of a storage medium (phase-change material), such as phase-change random-access memory (PRAM). ) may include. As another example, the resistance change memory may include resistive memory or magnetoresistive memory that uses the resistance of a memory element, for example, magnetoresistive random-access memory (MRAM). Next, PRAM, the memory used in the memory module 140, will be described.

The non-volatile memory control device 130 connects the host including the CPU 110 and the memory 120 and the memory module 140. In some embodiments, the non-volatile memory control device 130 may use a non-volatile memory express (NVMe) protocol as a protocol to access the non-volatile memory-based memory module 140. Below, the protocol is described as the NVMe protocol and the non-volatile memory control device 130 is described as the NVMe control device, but the embodiment is not limited to this and other protocols may be used.

In some embodiments, NVMe control device 130 may be connected to a host through a host interface. In some embodiments, the host interface may include a peripheral component interconnect express (PCIe) interface. In the future, the host interface will be described as a PCIe interface, but the embodiment is not limited to this and other host interfaces may be used.

In some embodiments, the computing device 100 may further include an interface device 150 for connecting a host including the CPU 110 and the memory 120 and the non-volatile memory control device 130. In some embodiments, interface device 150 may include a root complex 150 that connects a host and non-volatile memory control device 130 in a PCIe system.

First, a typical storage device will be described with reference to FIG. 2.

Figure 2 is a block diagram of a typical storage device. For convenience of explanation, Figure 2 illustrates an SSD storage device to which NAND flash memory is connected.

Referring to FIG. 2 , the storage device 200 includes a built-in processor 210, an internal memory 220, and a flash medium 230. The storage device 200 is connected to the host through a PCIe interface 240. Storage card 200 may include multiple flash media 230 across multiple channels to improve parallelism and increase back-end storage density. Internal memory 220, such as internal DRAM, is used for data buffering between the host and flash media 230. DRAM 220 may also be used to maintain metadata of firmware running on processor 210.

At the top of the firmware stack of the processor 210, a host interface layer (HIL) 211 exists to provide a block storage compatible interface. HIL 211 manages NVMe queues and can retrieve queue entries associated with NVMe requests and parse commands from the queue entries. If the request is a write, HIL 211 delivers the data in the write request to DRAM 220, and if the request is a read, HIL 211 scans DRAM 220 to directly provide data from DRAM 220. do.

Although data may be buffered in DRAM 220, requests are ultimately served from flash media 230 in a background or foreground manner. Accordingly, the address translation layer (ATL) 212 below the HIL 211 translates the logical block address (LBA) of the request into an address in the backend memory, i.e., the flash medium 230. ATL 212 may issue requests across multiple flash media 230 for parallel processing. The hardware abstraction layer (HAL) 213 at the bottom of the firmware stack manages memory protocol transactions for requests issued by the ATL 212.

Therefore, efficient firmware design plays a key role in storage cards, but with access latency of new non-volatile memory on the order of a few seconds, firmware execution becomes a significant performance bottleneck. For example, when using PRAM as the backend memory of a storage card, firmware latency in the input/output (I/O) path can account for approximately 98% of the I/O service time at the device level. To solve this problem, processors with more cores or higher frequency processors can be used. For example, you can reduce the firmware's CPU burst by distributing computation cycles across multiple cores and allow the firmware latency to overlap with the I/O burst latency of the backend memory. This may reduce overall latency, but many cores and/or high-frequency processors may consume more power, which may not be suitable for new energy-efficient memory-based storage designs. Below, an embodiment to solve this problem will be described.

Figure 3 is an example block diagram of an NVMe device according to one embodiment of the present invention.

Referring to FIG. 3, the NVMe device 300 includes an NVMe control device and a memory module 360. The NVMe control device includes a queue dispatching module 310, a data transmission module 320, and a completion handling module 330.

The queue dispatch module 310 may perform a series of procedures including doorbell monitoring 311, command fetch 312, and command parsing 313. The queue dispatch module 310 is located at the beginning of the data path of the NVMe device 300 and sends a submission queue (SQ) according to monitoring of the register (doorbell area) 311 mapped to the internal memory bus 340. ) from the I/O request (i.e. NVMe commands) and parse it. The doorbell area 311 of the queue dispatch module 310 may be referred to as an SQ tail doorbell area.

The data transmission module 320 confirms the data location in the host memory (e.g., 120 in FIG. 1) through the host memory address and performs data transmission between the host memory 120 and the backend memory module 350. . In some embodiments, the host memory address may be a physical region page (PRP), and the host memory address is described below as a PRP. The PRP indicates the target location of the host memory 120, and in the case of reading, it indicates the location where the target data should be delivered, and in the case of writing, it can indicate the location where the target data should be retrieved. In some embodiments, data transfer may be performed with direct memory access (DMA). For this purpose, the data transmission module 320 may use the DMA engine 321 and the PRP engine 322.

The completion handling module 330 may perform a series of procedures including doorbell monitoring 331, interrupt handling 332, and I/O completion 333. When target data is written to the backend memory module 350 (write) or when target data is transferred from the backend memory module 350 to host memory 120 (read), registers mapped to the internal memory bus 340 (i.e. You can manage interrupts and completion queues (CQ) through the doorbell area. The doorbell area of the completion handling module 330 may be referred to as the CQ head doorbell area.

In some embodiments, queue dispatch module 310, data transfer module 320, and completion handling module 330 may be formed as hardware modules. In some embodiments, queue dispatch module 310, data transfer module 320, and completion handling module 330 may be implemented in an integrated circuit, such as a field-programmable gate array (FPGA). For example, the queue dispatch module 310, data transfer module 320, and completion handling module 330 can be implemented on an FPGA board from Xilinx using an UltraScale ^™ chip and a PCIe Gen3 interface. In some embodiments, queue dispatch module 310, data transfer module 320, and completion handling module 330 may be implemented as hardware modules at register transfer level (RTL).

In some embodiments, the NVMe control device may further include a memory controller 340 that performs I/O services in the backend memory module 350. In some embodiments, the memory controller may be implemented as part of the overall hardware design of the NVMe control device.

In some embodiments, queue dispatch module 310, data transfer module 320, and completion handling module 330 may be coupled with a PCIe interface and memory controller 350 via internal memory bus 340. In some embodiments, the internal memory bus 340 may include an internal system on chip (SoC) memory bus. In some embodiments, internal memory bus 340 may include an advanced extensible interface (AXI) interface. In some embodiments, internal memory bus 340 may include an AXI crossbar.

In some embodiments, FPGA logic modules can be categorized into two types: frontend automation and backend automation. Front-end automation may include most of the logical modules for the queue dispatch module 310 and completion handling module 330. Additionally, front-end automation can be exposed to the host by mapping the configuration space of the host interface to the host address space. Additionally, front-end automation can map the configuration space to the internal memory bus 340. In some embodiments, the configuration space may include PCIe's base address registers (BARs). The queue dispatch module 310 retrieves and parses NVMe commands issued by the host, while the completion handling module 330 may manage I/O request completion, including interrupt handling. The completion handling module 330 can work with the queue dispatch module 310 to automatically pair different SQs and CQs to maintain all contexts. Backend automation may include data transfer module 320. The data transfer module 320 may search all PRP entries and migrate target data between the host memory 120 and the backend memory module 360 through the memory controller and DMA engine.

FIG. 4 is a diagram illustrating an NVMe control device according to an embodiment of the present invention, and FIG. 5 is a diagram illustrating the operation of the NVMe control device according to an embodiment of the present invention.

Referring to FIG. 4, the NVMe control device 400 includes a context module 410, a fetch management module 420, a PRP engine 430, a data transfer engine 440, and a completion handling module 450. In some embodiments, the data transfer engine 440 may be a direct memory access (DMA) engine, and the data transfer engine 440 will now be described as a DMA engine. In some embodiments, NVMe control device 400 may be connected to a host through a PCIe interface. In some embodiments, the NVMe control device 400 may further include a PCIe frontend 460 to connect to the host via a PCIe interface. Additionally, the NVMe control device 400 is connected to the backend memory module 470. In Figure 4, a plurality of memory modules are shown connected to the backend memory module 470, and each memory module 470 may be, for example, a dual in-line memory module (DIMM). In some embodiments, the NVMe control device 400 may further include a memory controller (not shown) for controlling the memory module 470.

In some embodiments, context module 410, fetch management module 420, PRP engine 430, data transfer engine 440, completion handling module 450, and PCIe frontend 460 are connected to internal memory bus 480. ) can be connected to each other. In some embodiments, memory bus 480 may include an AXI interface.

In some embodiments, the context module 410, fetch management module 420, PRP engine 430, data transfer engine 440, completion handling module 450, and PCIe frontend 460 may be formed as hardware modules. You can. In some embodiments, context module 410, fetch management module 420, PRP engine 430, data transfer engine 440, completion handling module 450, and PCIe frontend 460 may be implemented in an FPGA. there is. In some embodiments, the context module 410, the fetch management module 420, the PRP engine 430, the data transfer engine 440, the completion handling module 450, and the PCIe frontend 460 can be used to configure RTL to hardware modules. It can be implemented.

In the host, a pair of a submission queue (SQ) 401 and a completion queue (CQ) 402 is formed to manage input/output (I/O) requests. In some embodiments, when a CPU (eg, 110 in FIG. 1) includes a plurality of cores, a pair of SQ 401 and CQ 402 may be formed for each core. In some embodiments, pairs of SQ 401 and CQ 402 may be formed in host memory. In some embodiments, host memory may include system memory (e.g., 120 in FIG. 1). In Figure 3, for convenience of explanation, a pair (SQ1, CQ1) of SQ (401) and CQ (402) of one core is shown.

Referring to Figures 4 and 5, in order for the host to issue a command (e.g., NVMe command) generated according to an I/O request, the host pushes the NVMe command, that is, an SQ entry, to the SQ 401 (i.e. , written) (S510). In some embodiments, new SQ entries are written to the tail of SQ 401, thereby allowing the host to increment the tail pointer of SQ 401. In this way, when the host issues an NVMe command by creating an SQ entry in the SQ 401, the host creates (i.e. rings) an SQ tail doorbell (DB) (S420). To this end, the context module 410 includes a doorbell area containing a set of doorbell entries 411 tracking the tail pointer for each SQ and a set 412 of doorbell entries tracking the head pointer for each CQ. can do. Accordingly, doorbell entries (SQ0 DB, SQ1 DB, CQ0 DB, CQ1 DB) may be provided for each queue. In some embodiments, the SQ tail doorbell area 411 and the CQ head doorbell area 412 may be mapped to the internal memory bus address space of the NVMe control device 400 and exposed to the BAR 490 of the PCIe interface. . Accordingly, when the host writes an SQ entry in the SQ 401, the host can create a tail pointer in the corresponding doorbell entry in the SQ tail doorbell area 411 through the BAR 490. In some embodiments, since writing a doorbell on a host is the process of writing a PCIe packet (i.e., a PCIe inbound process), the NVMe control device 400 may You can monitor doorbell events.

The context module 410 generates an event signal in response to a doorbell event in the SQ tail doorbell area 411 and transmits the event signal to the fetch management module 420 (S530). In some embodiments, whenever a tail pointer is written to a doorbell entry in the SQ tail doorbell area 411, the context module 410 processes the event through the memory bus address space mapped to the SQ tail doorbell area 411. A signal can be transmitted (S530).

The fetch management module 420 checks the information transmitted through the event signal received from the context module 410 and retrieves the target SQ entry (i.e., NVMe command) from the host memory 120 (S540). In some embodiments, because doorbell events may occur simultaneously, the fetch management module 420 may mediate between different NVMe queues. For example, the fetch management module 420 can mediate between different NVMe queues according to NVMe specifications. After fetching the NVMe command, the fetch management module 420 parses the request information of the I/O request from the NVMe command and transmits it to the PRP engine 430 (S550). In some embodiments, the request information may include an operation code (opcode), logical address, size, and PRP. In some embodiments, the opcode may direct read, write, etc. In some embodiments, the logical address may be a logical block address (LBA), and the logical address is described below as an LBA. The LBA may indicate the address of a logical block to be performed (e.g., read or written) by the operation indicated by the operation code. For example, an LBA may indicate a logical block to be performed (e.g., read or written) with an operation indicated by the operation code. In some embodiments, the PRP may indicate a target location in the host memory 120, and may indicate a location from which the target data should be delivered in the case of reading, and a location from which the target data should be retrieved in the case of writing.

The PRP engine 430 sets the source address and destination address of the DMA engine 440 based on the request information received from the context module 410 (S560). The DMA engine 440 transfers target data between the host memory 120 and the back-end memory module 470 based on the source number and destination address (S560).

In some embodiments, PRP engine 430 accesses locations in host memory 120 via PRP to process page-aligned data, and uses the source address and destination address of DMA engine 440 to process page-aligned data. You can set it. In some embodiments, when the operation code (opcode) of the request information indicates reading, the source address may be the address of the backend memory module 470, and the destination address may be the address of the host memory 120. When the operation code (opcode) of the request information indicates writing, the source address may be the address of the host memory 120, and the destination address may be the address of the backend memory module 470. In some embodiments, the address of host memory 120 may be set based on a PRP, and the address of backend memory module 470 may be set based on a logical address.

In some embodiments, when the operation code of the request information indicates write, the DMA engine 440 reads the target data of the I/O request from the host memory 120 based on the request information and writes the target data based on the request information. Data can be written to the backend memory module 470 (S560). When the operation code of the request information indicates read, the DMA engine 440 reads the target data of the I/O request from the backend memory module 470 based on the request information and stores the target data in the host memory based on the request information. It can be written in (120) (S560).

In some embodiments, in the case of writing, the PRP engine 430 may copy data in the host memory 120 to the DMA engine 440 every 512B chunk. In some embodiments, DMA engine 440 may use multiple DMA cores to parallelize the data transfer process. In some embodiments, the number of DMA cores may be set based on the number of channels of the backend memory module 470. For example, the number of DMA cores may be the same as the number of channels of the backend memory module 470. In some embodiments, DMA engine 440 may split the target data of a single request into multiple small data chunks and distribute the multiple data chunks to multiple DMA cores. Then, the memory I/O delay time can be shortened.

In some embodiments, if the payload size of a PCIe packet is 4 KB, the target data may reside in two different memory pages of host memory 120 if the offset of the request is not aligned by a 4 KB boundary. In this case, as shown in FIG. 5, the PRP engine 430 may retrieve target data from two different host memory 120 locations, respectively referred to as PRP1 and PRP2. In some embodiments, when the size of the request is equal to or smaller than a page, the DMA engine 440 may retrieve or transfer target data through PRP1, which directly indicates the location of the host memory 120. If the size of the request is larger than the page, the PRP engine 430 can retrieve the PRP list from the host memory 120 through PRP2. The PRP list is a set of pointers and may include at least one PRP entry indicating 4KB data. Accordingly, the PRP engine 430 can parse each PRP entry in the PRP list. In some embodiments, PRP engine 430 may parse PRP entries within backend memory module 470. The PRP engine 430 may search all entries in the PRP list and detect the location of the host memory 120 referenced by each entry. When data transmission is completed in the DMA engine 440, the PRP engine 430 generates a completion request (i.e., CQ entry) and a completion handling module 450 that manages message-signaled interrupt (MSI) packets. Send a signal (S570).

As described above, in the case of writing, target data may be transmitted from the host memory 120 to the NVMe control device 400 via a PCIe inbound link. Additionally, in case of reading, target data may be transmitted to the host memory 120 through a PCIe outbound link.

The completion handling module 450 processes I/O completion and interrupts. When the PRP engine 430 and the DMA engine 440 complete processing of the I/O request, the completion handling module 450 automatically detects the target CQ 402 for writing a CQ entry. In some embodiments, the completion handling module 430 may detect the target CQ 402 by checking request information transmitted from the fetch management module 420. When the completion handling module 450 finds the target CQ 402, it posts the CQ entry by creating a CQ entry in the location of the host memory 120 corresponding to the target CQ 402 (S570). In some embodiments, a CQ entry is written to the head of CQ 402, such that completion handling module 430 may increment the head pointer of CQ 402. Next, the completion handling module 450 interrupts the host and notifies the host of I/O service completion (S580). In some embodiments, completion handling module 450 may interrupt the host by pushing (writing) a PCIe packet (MSI packet) to an MSI area managed by the host's driver. In some embodiments, the host driver may invoke its own interrupt service routine to notify the user who requested the I/O service of completion. The host terminates the I/O service and creates a CQ head doorbell (S590). In some embodiments, the host may create a head pointer to a corresponding doorbell entry in the CQ head doorbell area 412 via BAR 490. In this way, by updating the CQ head doorbell area 412, the state of the CQ 402 can be synchronized with the NVMe control device 400. In other words, queue synchronization means that the completion handling module 450 releases the target entries of SQ 401 and CQ 402, and in this way the internal CQ and SQ states of the NVMe control device 400 are synchronized with the host's SQ and May correspond to CQ status.

In some embodiments, to improve the responsiveness of the front end (i.e., the SQ tail doorbell area 411 and the fetch management module 420), the fetch management module 420 connects the DMA module 440 in a pipelined manner. And the remaining requests may continue to be processed during the operation of the completion handling module 450 (S550).

In some embodiments, the SQ tail doorbell area 411 and the CQ head doorbell area 412 can be placed side by side to easily manage NVMe queues. However, since the inflow rate of NVMe submissions and the inflow rate of NVMe completions are asymmetric, the physical integration of the SQ tail doorbell area 411 and the CQ head doorbell area 412 may not be promising in terms of performance and implementation. Accordingly, in some embodiments, the SQ tail doorbell area 411 and the CQ head doorbell area 412 may be completely separated. In this case, the CQ and SQ states can be paired, as described previously.

In some embodiments, the process of recording a doorbell in the SQ tail doorbell area 411, retrieving an SQ entry, retrieving a PRP, and recording a doorbell in the CQ head doorbell area 412 (S520, S540) , S560, S590) may be a PCIe inbound process that records PCIe packets. Meanwhile, in the case of a read request, the process that writes data to the host memory 120 may be a PCIe outbound process. Therefore, in read requests, data and NVMe payload (doorbell, SQ entry, PRP) can be processed in parallel, thereby reducing latency.

In some embodiments, the SQ tail doorbell area 411 and the fetch management module 420 of the context module 410 operate as a queue dispatch module (e.g., 210 in FIG. 2), and the PRP engine 430 and The DMA engine 440 operates as a data transmission module (e.g., 220 in FIG. 2), and the completion handling module 450 and the CQ head doorbell area 412 of the context module 410 operate as a completion handling module (e.g. For example, it can be operated as 230 in FIG. 2.

As described above, an NVMe control device can be implemented by fully automating the NVMe control logic in hardware. Accordingly, NVMe payload and data can be processed without software intervention. Accordingly, latency and heat generation issues that may occur due to firmware execution on the embedded processor of the NVMe control device can be eliminated or reduced.

Figure 6 is a diagram illustrating the connection relationship of an NVMe control device according to an embodiment of the present invention.

Referring to FIG. 6, the NVMe control device 600 includes a context module 610, a fetch management module 620, a PRP engine 630, a DMA engine 640, and a completion handling module 650. In some embodiments, NVMe control device 600 may further include a PCIe frontend 660. These modules 610-660 may be connected through an internal memory bus. In some embodiments, the internal memory bus may include a SoC memory bus. In some embodiments, the internal memory bus may be an advanced extensible interface (AXI) bus.

The context module 610 is connected to the front end 660 through a memory bus and can receive an SQ tail pointer from the host (672). The context module 610 is connected to the fetch management module 620 through a memory bus and can transmit an event signal to the fetch management module 620 (673). The fetch management module 620 is connected to the front end 660 through a memory bus and can retrieve a target SQ entry from the SQ of the host memory (674).

The PRP engine 630 is connected to the fetch management module 620 through a memory bus and can receive request information from the fetch management module 620 (675). The PRP engine 630 is connected to the front end 660 and the DMA engine 640 through a memory bus to retrieve target data from the host memory and transfer it to the DMA engine 640 or target data transmitted through the DMA engine 640. can be transferred to host memory (676).

The completion handling module 650 is connected to the PRP engine 630 through a memory bus to receive a completion request from the PRP engine 630, and is connected to the front end 660 through a memory bus to send a CQ entry to the CQ in the host memory. It can be written in (677). Additionally, the completion handling module 650 is connected to the front end 660 through a memory bus and can interrupt the host to notify the host of I/O service completion (678). The context module 610 is connected to the front end 660 through a memory bus and can receive a CQ head pointer from the host (679).

In some embodiments, to allow the NVMe control device to operate at high frequencies, the inputs and outputs of each module can be connected in only one direction. In some embodiments, if there is needed information that was not conveyed by the source module, the target module may retrieve the needed information directly from the memory address space of a memory bus (e.g., AXI crossbar). In some embodiments, each module processes multiple portions of an I/O request in a pipelined manner, thereby improving I/O processing bandwidth.

In some embodiments, the NVMe control unit separates the memory controller from the logic module (e.g., 310-330 in Figure 3, 410-450 in Figure 4) and then controls the memory controller at the boundary of the super logic region (SLR). Logic modules can be grouped by taking into account. In some embodiments, all logic modules are located around the PCIe and AXI crossbars, but modules associated with the data transfer and DMA engines, including the memory controller, may occupy only two SLRs.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims are also possible. It falls within the scope of rights.

Claims (20)

A non-volatile memory control device connecting a host including a processor and host memory and a non-volatile memory module,
An internal memory bus exposed to BAR (base address registers) of the host interface for connection to the host,
A first doorbell that is mapped to the address space of the internal memory bus and exposed to the BAR through the internal memory bus, and is updated when the host issues an I/O (input/output) request command to the host memory. area,
a fetch management module that retrieves the command from the host memory in response to an event signal generated when the first doorbell area is updated;
a data transfer module that confirms the location of the host memory based on the request information included in the command and performs transfer of target data according to the I/O request between the host memory and the non-volatile memory module;
When the data transfer module completes processing of the I/O request, a completion handling module that records the completion request in the host memory and processes an interrupt, and
A second doorbell area that is mapped to the address space of the internal memory bus and exposed to the BAR through the internal memory bus, and is updated when the host terminates the I/O service.
A non-volatile memory control device comprising a. In paragraph 1:
The first doorbell area, the fetch management module, the data transmission module, the completion handling module, and the second doorbell area are implemented in hardware. In paragraph 2,
The first doorbell area, the fetch management module, the data transfer module, the completion handling module, and the second doorbell area are implemented in hardware at a register transfer level (RTL), non-volatile memory control device. . In paragraph 2,
The first doorbell area, the fetch management module, the data transmission module, the completion handling module, and the second doorbell area are connected by the internal memory bus. In paragraph 1:
The internal memory bus includes an advanced extensible interface (AXI) interface. delete In paragraph 1:
The host interface includes a peripheral component interconnect express (PCIe) interface.
Non-volatile memory control unit. In paragraph 1:
The fetch management module parses the request information from the command. In paragraph 1:
The request information includes a logical address and a host memory address indicating the location of the host memory,
The data transmission module is,
A first engine that sets a source address and a destination address for the data transmission module based on the logical address and the host memory address, and
Comprising a second engine that performs transmission of the target data based on the source address and destination address.
Non-volatile memory control unit. In paragraph 9:
The non-volatile memory control device, wherein the second engine includes a direct memory access (DMA) engine. In paragraph 9:
The host memory address includes a physical region page (PRP). In paragraph 11:
The first engine retrieves a PRP list from the host memory based on the PRP,
The PRP list includes at least one PRP entry referencing a location in the host memory,
The first engine detects the location of the host memory referenced by the PRP entry.
Non-volatile memory control unit. In paragraph 9:
The non-volatile memory control device is connected to the non-volatile memory module through a plurality of channels,
The second engine includes a plurality of DMA cores each corresponding to the plurality of channels.
Non-volatile memory control unit. In paragraph 13:
The second engine divides the target data of the I/O request into a plurality of data chunks, and distributes the plurality of data chunks to the plurality of DMA cores. In paragraph 1:
A plurality of queue pairs exist in the host memory,
Each queue pair includes a pair of submission queues (SQ) and competition queues (CQ).
The fetch management module retrieves the command from the SQ of the queue pair corresponding to the updated entry in the first doorbell area among the plurality of queue pairs.
Non-volatile memory control unit. In paragraph 15:
The completion handling module detects a CQ corresponding to the SQ that brought the command based on information transmitted from the fetch management module. A non-volatile memory control device connecting a host including a processor and host memory and a non-volatile memory module,
An internal memory bus exposed to base address registers (BAR) of a peripheral component interconnect express (PCIe) interface for connection to the host,
When a command is recorded in the target SQ (submission queue) of the host memory, an entry corresponding to the target SQ is updated, and in the target CQ (completion queue) when the I/O service is terminated in the host. A first module including a second doorbell area in which a corresponding entry is updated, and mapped to the address space of the internal memory bus and exposed to the BAR through the internal memory bus,
a second module that retrieves the command from the target SQ in response to an event signal generated when the first doorbell area is updated,
A first engine that detects the location of the host memory based on a physical region page (PRP) included in the command,
a second engine that performs transfer of target data according to the I/O request between the location of the host memory detected by the first engine and the non-volatile memory module, and
When the second engine completes processing of the I/O request, a third module records a completion request in the target CQ and processes an interrupt.
A non-volatile memory control device comprising a. In paragraph 17:
The first engine retrieves a PRP list from the host memory based on the PRP,
The PRP list includes at least one PRP entry referencing a location in the host memory,
The first engine detects the location of the host memory referenced by the PRP entry.
Non-volatile memory control unit. In paragraph 17:
The first module, the second module, the first engine, the second engine, and the third module are implemented in hardware. A non-volatile memory device coupled to a host including a processor and host memory,
non-volatile memory module,
An internal memory bus exposed to BAR (base address registers) of the host interface for connection to the host,
A first doorbell that is mapped to the address space of the internal memory bus and exposed to the BAR through the internal memory bus, and is updated when the host issues an I/O (input/output) request command to the host memory. area,
a fetch management module that retrieves the command from the host memory in response to an event signal generated when the first doorbell area is updated;
a data transfer module that confirms the location of the host memory based on the request information included in the command and performs transfer of target data according to the I/O request between the host memory and the non-volatile memory module;
When the data transfer module completes processing of the I/O request, a completion handling module that records the completion request in the host memory and processes an interrupt, and
A second doorbell area that is mapped to the address space of the internal memory bus and exposed to the BAR through the internal memory bus, and is updated when the host terminates the I/O service.
A non-volatile memory device including a.

Images