CN107526695B - NVMe configuration space implementation method and device - Google Patents
NVMe configuration space implementation method and device Download PDFInfo
- Publication number
- CN107526695B CN107526695B CN201610443715.3A CN201610443715A CN107526695B CN 107526695 B CN107526695 B CN 107526695B CN 201610443715 A CN201610443715 A CN 201610443715A CN 107526695 B CN107526695 B CN 107526695B
- Authority
- CN
- China
- Prior art keywords
- register
- access
- controller
- nvme
- address
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/16—Handling requests for interconnection or transfer for access to memory bus
- G06F13/1668—Details of memory controller
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Bus Control (AREA)
- Storage Device Security (AREA)
Abstract
An NVMe configuration space realization method and device are provided. A method of accessing an NMVe controller register is provided, comprising: identifying an access request to access a memory space of a PCIe device; determining that the access request accesses NVMe controller registers according to the address of the access request, wherein the NVMe controller registers comprise first type registers, doorbell registers and/or device vendor special registers, and the first type registers occupy a predetermined range of memory register space; and if the access request accesses the first type of register, generating a register access message and sending the register access message to the CPU, wherein the register access message comprises a controller register address.
Description
Technical Field
The present invention relates to NVMe controllers, and in particular, to controller registers (ControllerRegisters) in the NVMe protocol.
Background
A controller register (see also "nvmexpress change 1.2" (hereinafter, NVMe protocol) chapter 3, 11/3/2014) is defined in the NVMe protocol. When the NVMe controller is implemented based on the PCIe protocol, NVMe controller registers are mapped to the memory space (MemorySpace) of the PCIe device. In the NVMe protocol, PCIe registers MLBAR and MUBAR (registers BAR0 and BAR1 in the PCIe protocol) define a memory space (referred to herein as a memory register space) for providing NVMe controller registers.
In addition to the PCIe protocol, the NVMe protocol may be implemented based on other bus protocols. In the bus protocol, the device provides a memory space (or memory register space) that can be accessed by other bus entities and maps the NVMe controller registers to the memory space. Thus, to access NVMe controller registers, this is accomplished by accessing memory space in the bus protocol.
The Host (Host) or other bus entity has access to the controller registers in accordance with the NVMe protocol, but should also follow several constraints. For example, Host cannot access two or more controller registers of a device simultaneously; the reserved register or reserved bits in the register are read-only and return a 0 when read.
The controller registers include registers for different purposes. Referring to fig. 1, a layout of control registers in memory space is shown. Under the NVMe protocol, taking the PCIe protocol as an example of a bus protocol, registers (referred to as attribute/control registers in this disclosure) provided in the first 4 kbytes (address range 0-0x0FFF) of the memory space of the PCIe device are used to describe basic attributes of the controller, such as controller capabilities, version, command queue base address, or to provide a control port to the controller. The Doorbell register (Doorbell register) that stores the command queue starts at the 0x1000 address of the memory space, and the memory space after the Doorbell register stores device vendor specific (vendorspecic) information.
The controller registers have multiple types (Type), for example NVMe protocol version 1.2 defines read-only Type (RO), read-write Type (RW), RW1S Type and RW1C Type. In other versions of the NVMe protocol, other controller register types may also be defined. The RW1S type means that the register is readable and writable, when written to, writing a "1" to a bit means that the mask of the interrupt vector is set bit by bit, while writing a "0" has no effect; when read out, the mask value of the current interrupt vector (rather than the register value) is read out. The RW1C type means that the register is readable and writable, writing a "1" to a bit means that the mask of the interrupt vector is cleared bit by bit, while writing a "0" has no effect; when read out, the mask value of the current interrupt vector (rather than the register value) is read out.
Disclosure of Invention
However, the NVMe protocol does not define how controller registers are implemented in the controller. The controller register access scheme is not limited to register read/write operations, and there are special access schemes for the registers that are reserved, special types of registers. The NVMe protocol is rapidly developing, and the current controller register implementation mode needs to meet the requirement of protocol evolution.
According to a first aspect of the invention there is provided a first method of accessing NVMe controller registers according to the first aspect of the invention, comprising: identifying an access request to access a memory space of a PCIe device; determining that the access request accesses NVMe controller registers according to the address of the access request, wherein the NVMe controller registers comprise first type registers, doorbell registers and/or device vendor special registers, and the first type registers occupy a predetermined range of memory register space; and if the access request accesses the first type of register, generating a register access message and sending the register access message to the CPU, wherein the register access message comprises a controller register address.
A first method of accessing NVMe controller registers according to the first aspect of the present invention provides a second method of accessing NVMe controller registers according to the first aspect of the present invention, further comprising: and if the access request accesses the doorbell register, accessing the memory of the PCIe equipment according to the address of the access request.
A first or second method of accessing NVMe controller registers according to the first aspect of the present invention provides a third method of accessing NVMe controller registers according to the first aspect of the present invention, further comprising: and the CPU responds to the received register access message, identifies the type of the accessed first type of register according to the register address of the controller of the register access message, and processes the register access message according to the access mode of the first type of register of the type defined by the NVMe protocol.
According to the first to third methods for accessing NVMe controller registers of the first aspect of the present invention, there is provided a fourth method for accessing NVMe controller registers of the first aspect of the present invention, further comprising, if the access request accesses the device vendor specific register, accessing the memory of the PCIe device according to the address of the access request.
A fifth method of accessing NVMe controller registers according to the first aspect of the invention is provided according to the first aspect of the invention, wherein the specified address range of the memory space of the PCIe device is mapped to the address space of the memory of the PCIe device.
According to the first to fifth methods of accessing NVMe controller registers of the first aspect of the present invention, there is provided a sixth method of accessing NVMe controller registers according to the first aspect of the present invention, wherein the CPU accesses the same address of the memory of the PCIe device according to a plurality of register access messages having different memory space addresses.
According to the first to sixth methods of accessing NVMe controller registers of the first aspect of the present invention, there is provided a sixth method of accessing NVMe controller registers according to the seventh aspect of the present invention, further comprising: and the CPU sends the processing result of the register access message to the sender of the access request.
According to a first method of accessing NVMe controller registers of the first aspect of the present invention, there is provided an eighth method of accessing NVMe controller registers according to the first aspect of the present invention, wherein the register access message further comprises an access type indicating whether to read or write the controller registers.
According to a second aspect of the present invention, there is provided a first method of accessing NVMe controller registers according to the second aspect of the present invention, comprising: identifying an access request to access a memory register (memorry registers); determining that the access request accesses an NVMe controller register according to the address of the access request; generating an NVMe controller register access message and sending the NVMe controller register access message to the CPU, wherein the NVMe controller register access message comprises a controller register space address.
According to a second aspect of the present invention, there is provided a second method for accessing NVMe controller registers, wherein in response to receiving an NVMe controller register access message, the CPU identifies the type of the accessed NVMe controller registers according to the controller register space address of the NVMe controller register access message, and processes the NVMe controller register access message according to the access mode of the type of NVMe controller registers defined by the NVMe protocol.
A method of accessing NVMe controller registers according to the first or second aspect of the present invention provides a method of accessing NVMe controller registers according to the third aspect of the present invention, further comprising: if the access request accesses the doorbell register, accessing an area in a memory corresponding to the controller register space address of the access request according to the access request, and not sending an NVMe controller register access message to the CPU.
A method of accessing NVMe controller registers according to the first or second aspect of the present invention provides a fourth method of accessing NVMe controller registers according to the second aspect of the present invention, further comprising: and the CPU responds to the received NVMe controller register access message, identifies a doorbell register to be accessed according to the controller register space address of the NVMe controller register access message, and stores the doorbell register according to the controller register space address.
According to a third aspect of the invention there is provided a first method of accessing NVMe controller registers according to the third aspect of the invention, comprising: identifying an access request to access a memory register (memorry registers); determining that the access request accesses an NVMe controller register according to the address of the access request, wherein the NVMe controller register comprises a first type register, a doorbell register and/or an equipment manufacturer special register, and the first type register occupies a specified address range of a memory register space; and if the access request accesses the first type of register, generating a register access message and sending the register access message to the CPU, wherein the register access message comprises a controller register address.
According to a fourth aspect of the present invention, there is provided a first NVMe controller according to the fourth aspect of the present invention, comprising: the system comprises a system bus access unit, a controller register access unit, a CPU and a memory; a system bus access unit coupled to the external bus and generating a controller register access request in response to a memory space access request from the system bus, the controller register access request including a first memory address and an access type; the controller register access component responds to a received controller register access request, determines that the controller register access request accesses a first type of register according to the first memory address, generates a register access message and sends the register access message to the CPU; and the CPU responds to the received register access message, identifies the accessed NVMe controller register according to the second memory address of the register access message, and processes the register access message according to the access mode of the NVMe controller register defined by the NVMe protocol.
The first NVMe controller according to a fourth aspect of the present invention provides the second NVMe controller according to the fourth aspect of the present invention, wherein the controller register access section determines that the controller register access request accesses the doorbell register in accordance with the first memory address, accesses the area in the memory corresponding to the first memory address, without sending a register access message to the CPU.
The first or second NVMe controller according to a fourth aspect of the present invention provides the third NVMe controller according to the fourth aspect of the present invention, wherein the CPU sends a processing result of the register access message to an issuer of the memory space access request.
According to the first to third NVMe controllers of the fourth aspect of the present invention, there is provided the fourth NVMe controller of the fourth aspect of the present invention, wherein the controller register access section determines, according to the first memory address, that the controller register access request accesses the first type of registers of the NVMe controller registers, generates a register access message, and sends the register access message to the CPU; wherein the first type of register occupies an address range that is in the first 4KB of the controller register space.
The first to third NVMe controllers according to the fourth aspect of the present invention provide the fifth NVMe controller according to the fourth aspect of the present invention, wherein the CPU accesses the same address of the memory according to the plurality of second memory addresses of the plurality of register access messages.
The first to fifth NVMe controllers according to the fourth aspect of the present invention provide the sixth NVMe controller according to the fourth aspect of the present invention, wherein the system bus access component is a PCIe controller.
According to a fifth aspect of the present invention, there is provided a first NVMe controller according to the fifth aspect of the present invention, comprising: a system bus access unit, a CPU and a memory; the system bus access component is coupled with the external bus, responds to a memory space access request from the system bus to access the NVMe controller register, generates an NVMe controller register access message and sends the NVMe controller register access message to the CPU; the CPU responds to the received NVMe controller register access message, identifies the accessed NVMe controller register according to a controller memory address included in the NVMe controller register access message, and processes the NVMe controller register access message according to an access mode of the NVMe controller register defined by an NVMe protocol.
The first NVMe controller according to a fifth aspect of the present invention provides the second NVMe controller according to the sixth aspect of the present invention, wherein the system bus access means accesses the doorbell register in response to a memory space access request, the memory space access request accessing the memory without generating the NVMe controller register access message.
Drawings
The invention, as well as a preferred mode of use and further objectives and advantages thereof, will best be understood by reference to the following detailed description of an illustrative embodiment when read in conjunction with the accompanying drawings, wherein:
FIG. 1 shows the layout of control registers in memory space;
fig. 2 shows a block diagram of circuitry of an NVMe controller according to an embodiment of the invention;
fig. 3 shows a block diagram of circuitry of an NVMe controller according to yet another embodiment of the invention; and
fig. 4 shows a circuit system block diagram of an NVMe controller according to still another embodiment of the present invention.
Detailed Description
Fig. 2 shows a block diagram of circuitry of an NVMe controller according to an embodiment of the invention. The NVMe controller includes a pci ephy module 210, a system bus access module 220, a controller register access module 230, a memory 240, and a CPU subsystem 260. Both the controller register access module 230 and the CPU subsystem 260 may access the memory 240. The memory stores a Doorbell (Doorbell) register, a vendor specific (vendorspecfic) register, and a message queue for exchanging messages between the controller register access module 230 and the CPU subsystem 260.
The host (host) is coupled to the device through the pci ephy module 210. Included in the NVMe controller is a pci ephy module 210 and a system bus access module 220 coupled to the pci ephy module 210. The PCIe phy module 210 is configured to handle PCIe underlying protocols (e.g., physical layer). The system bus access module 220 is used to handle PCIe upper layer protocols (e.g., data link layer and/or transaction layer). The system bus access module 220 also translates requests sent by the host to access a particular region of the device's memory space (memryspace) into controller register access requests including access type, memory space address, and/or access data (excluding access data for memory space read requests; including access data for memory space write requests), and forwards to the controller register access module 230. Optionally, the system bus access module 220 may use an address format other than memory space addresses (e.g., shorten the address length) in the controller register access request to more efficiently indicate the controller register being accessed.
The controller register access module 230 determines whether the controller register access request is to access the attribute/control register, the doorbell register, or the vendor specific register based on the memory space address. Different handling modes are provided for different types of controller register access requests.
In NVMe1.2 version of the protocol, PCIe registers MLBAR and MUBAR define a memory space for providing NVMe controller registers, while the lower 4KB of this address space is used for attribute/control registers (see fig. 1).
When the doorbell register or the vendor specific register is to be accessed, the controller register access module 230 writes an access request to or reads data from a storage location of the memory 240 corresponding to the memory space address according to the memory space address and returns the data to the host through the system bus access module 220 (and the pci ephy 210).
When a controller register access request is to access an attribute/control register, the controller register access module 230 generates a register access message and inserts the register access message into a message queue in memory. When one or more CPUs in the CPU subsystem 260 find messages to be processed in the message queue of the memory 240, retrieve the messages, determine the attribute/control register to be accessed and the access type, process the register access message, and return the access result to the host through the system bus access module 220. In one example, the CPU subsystem 260 inserts the access result into a message queue in the memory 240, and the controller register access module 230 retrieves the access result from the message queue and returns it to the host through the system bus access module 220.
In this way, a variety of controller registers in the NVMe protocol are implemented, and efficient processing is achieved for different types of controller registers, access constraints, and the like. And after the NVMe protocol is updated, the modification of the access to the controller register introduced in the new version protocol can be effectively adapted by modifying the program running in the CPU subsystem.
Embodiments according to the invention are described in detail below with examples of access to several controller registers.
(1) Doorbell register for tail pointer of host read/write submission queue
By way of example, the commit queue tail pointer doorbell register has 32 bits, with 31:16 bits reserved and of the read-only type and 15:00 bits of the read-write type.
When the host requests to read the full 32 bits of the commit queue doorbell register with PCIe memory space address 0x1000h, the system bus module 220 generates a controller register access request including the address 0x1000h and the read access type and sends to the controller register access module 230.
The controller register access module 230 recognizes that it is the commit queue tail pointer doorbell register that is accessed based on the address 0x1000h, reads the commit queue tail pointer value corresponding to the address 0x1000h from the doorbell register storage area of the memory 240 and transmits it to the host through the system bus access module 220. In yet another example, instead of storing 31:16 bits of the commit queue tail pointer corresponding to address 0x1000h in memory 240, only 15:00 bits of the commit queue tail pointer are stored to save memory space, and 16 bits 0 are padded high when the commit queue tail pointer is read out by controller register access module 230 to comply with NVMe protocol specifications.
When the host requests to write to the full 32 bits of the commit queue doorbell register with PCIe memory space address 0x1000h, the system bus access module 220 generates a controller register access request including address 0x1000h, data to be written, and write access type, and sends it to the controller register access module 230.
The controller register access module 230 identifies that the committed queue tail pointer doorbell register is accessed based on the address 0x1000h, writing the committed queue tail pointer value corresponding to the address 0x1000h to the doorbell register storage area of the memory 240. In yet another example, the 31:16 bits of the commit queue tail pointer corresponding to address 0x1000h need not be stored in memory 240, but only the 15:00 bits of the commit queue tail pointer are written to memory 240 to save storage space.
(2) Host read/write CAP register
The address range of 0x00h-0x07h in the PCIe memory space is CAP register for indicating the capability of NVMe controller, and 64 bits of the CAP register are all read-only type.
When the host requests to read the CAP register with PCIe memory space address 0x00h, the system bus access module 220 generates a controller register access request including address 0x00h, length 8 bytes, and read access type, and sends the controller register access request to the controller register access module 230.
The controller register access module 230 identifies that it is an attribute/control register that is accessed based on address 0x00h, generates a register access message, and inserts the register access message into a message queue in memory 240. Information indicating address 0x00h, length 8 bytes, and read access type is included in the register access message.
It is noted that the address 0x00h indicated in the register access message may be the same or different in form as the address in the controller register access request (0x00 h). For example, the address in the controller register access request may be a PCIe memory space address. The address in the register access message may also be a PCIe memory space address. The controller registers are mapped to a portion of the PCIe memory space. In another example, the address in the controller register access request may be a PCIe memory space address, while the address indicated in the register access message is an address dedicated to describing a controller register or an attribute/control register. The address indicated in the register access message may thus be smaller in size than the address in the controller register access request (e.g., the address indicated in the register access message is from the lower order bits of the PCIe memory space address).
When the CPU processes the message queue, it determines the CAP register to be accessed according to the address 0x00h, and sends 64-bit information indicating the NVMe controller capability corresponding to the CAP register to the host through the system bus access module 220. As another example, the CPU writes 64-bit information indicating NVMe controller capability corresponding to the CAP register to a message queue in memory 240. The controller register access module 230 detects that the CPU updates the message queue, and transmits 64-bit information indicating the NVMe controller capability corresponding to the CAP register to be updated to the host through the system bus access module 220.
When a host requests to write to a CAP register with PCIe memory space address 0x00h, the system bus access module 220 generates a controller register access request including address 0x00h, the 8 bytes of data to be written, and the write access type, and sends it to the controller register access module 230.
The controller register access module 230 identifies that it is an attribute/control register that is accessed based on address 0x00h, generates a register access message, and inserts the register access message into a message queue in memory 240. Information indicating the address 0x00h, the 8 bytes of data to be written, and the write access type is included in the register access message.
The CPU260 determines that it is the CAP register to be accessed from the address 0x00h and the access type is write while processing the message queue. The CPU260 finds that the CAP register is of the read-only type and ignores the message. Alternatively, the CPU260 generates an exception message and sends it to the host through the system bus access module 220 to indicate that the host has performed an illegal operation on the controller register.
(3) Host read/write Reserved (Reserved) register
The 0x18h-0x1Bh address range of PCIe memory space is reserved in NVMe protocol version 1.2.
When the host requests to read a register having a PCIe memory space address of 0x18h, the system bus access module 220 generates a controller register access request including address 0x18h and read access type, and sends the controller register access request to the controller register access module 230.
The controller register access module 230 identifies that it is an attribute/control register that is accessed based on address 0x18h, generates a register access message, and inserts the register access message into a message queue in memory 240. Information indicating the address 0x18h and the read access type is included in the register access message.
The CPU260 determines that the reserved register is to be accessed according to the address 0x18h while processing the message queue, and transmits the access result of all 0 s to the host through the system bus access module 220. As another example, the CPU writes the results of all 0's access to the message queue in memory 240. The controller register access module 230 detects that the CPU updates the message queue, and transmits updated access results of all 0 s to the host through the system bus access module 220.
When the host requests to write to a register having a PCIe memory space address of 0x18h, the system bus access module 220 generates a controller register access request including address 0x18h, the data to be written, and the write access type, and sends to the controller register access module 230.
The controller register access module 230 identifies that it is an attribute/control register that is accessed based on address 0x18h, generates a register access message, and inserts the register access message into a message queue in memory 240. Information indicating address 0x18h, the data to be written, and the write access type is included in the register access message.
The CPU260 determines from the address 0x18h that the reserved register is to be accessed and the access type is write when processing the message queue, and thus ignores the message. Alternatively, the CPU260 generates an exception message and sends it to the host through the system bus access module 220 to indicate that the host has performed an illegal operation on the controller register.
(4) INTMS register read/write by host
The 0x0Ch-0x0Fh address range of PCIe memory space is INTMS registers. The INTMS register is used to set the interrupt mask and has a special access type (RW 1S). When the INTMS register is written, if a bit of the INTMS register is written to "1", the interrupt corresponding to the bit is masked. When the INTMS register is read, the value read indicates the current interrupt mask.
When the host requests to read a register having a PCIe memory space address of 0x0Ch, the system bus access module 220 generates a controller register access request including the address 0x0Ch and the read access type, and sends the controller register access request to the controller register access module 230.
Controller register access module 230 identifies that it is an attribute/control register that is accessed based on address 0x0Ch, generates a register access message, and inserts the register access message into a message queue in memory 240. Information indicating address 0x0Ch and the read access type is included in the register access message.
When the CPU260 processes the message queue, it determines that the current interrupt vector is the INTMS register according to the address 0x0Ch, the CPU260 obtains the interrupt mask of the current interrupt vector, the corresponding bit "1" of the masked interrupt in the INTMS register, the corresponding bit "0" of the unmasked interrupt in the INTMS register, and sends the result to the host through the system bus access module 220. As another example, the CPU writes the result of the access of the INTMS register to a message queue in the memory 240. The controller register access module 230 detects that the CPU updates the message queue, and sends the access result of the updated INTMS register to the host through the system bus access module 220.
When the host requests to write 0x0001 into a register with PCIe memory space address 0x0Ch, the system bus access module generates a controller register access request including address 0x0Ch, data to be written 0x0001, and a write access type, and sends to the controller register access module 230.
Controller register access module 230 identifies that it is an attribute/control register that is accessed based on address 0x0Ch, generates a register access message, and inserts the register access message into a message queue in memory 240. Information indicating the address 0x0Ch, the data to be written 0x0001, and the write access type is included in the register access message.
The CPU260 determines from the address 0x0Ch that it is the INTMS register to access and the access type is write when processing the message queue. The CPU260 masks the interrupt corresponding to the lowest bit of the INTMS according to the value 0x0001h to be written.
Since the attribute/control registers in the NVMe controller registers have multiple meanings, formats, access control rules (read only/write only/reserved, etc.) and control roles, in an embodiment of the present invention, the controller registers 230 recognize accesses to the attribute/control registers and issue register access messages to the CPU260, with accesses to the attribute/control registers being handled by the CPU 260. In a further example, where multiple reserved registers having different memory addresses are read, CPU260 provides the same value (0) in response, and thus may provide a single storage location in memory 240 to store the response value of the reserved register and use the response value to respond to accesses to the reserved register having different memory addresses. In another example, a plurality of NVMe controllers are provided in the NVMe device, each having a respective controller register, some of the controller registers of the respective NVMe controllers may have the same value (e.g., CAP registers). The CPU260 may access a single location of the memory 240 in response to host access to the controller registers of multiple NVMe controllers.
In another embodiment according to the present invention, the system bus access module 220 and the controller register access module 230 in FIG. 2 are integrated together (also referred to as a system bus access module). The system bus access module integrated with the controller register access module identifies the PCIe memory space address of a memory access request sent by a host, generates a register access message for the memory access request for accessing the attribute/control register, and sends the register access message to the CPU, and the CPU responds to the attribute/control register access request of the host by processing the register access message. And when the system bus access module identifies that a memory access request sent by the host accesses the doorbell register and/or the device-manufacturer-specific register, accessing a corresponding doorbell register storage area and/or a device-manufacturer-specific register storage area of the memory.
Fig. 3 shows a block diagram of circuitry of an NVMe controller according to yet another embodiment of the present invention.
The host (host) is coupled to the device through a pci ephy 310. The NVMe controller includes a pci ephy310 and a system bus access module 320 coupled to the pci ephy 310. System bus access module 320 translates requests sent by the host to access the memory space of the device into controller register access requests that include an access type, memory space address, and/or access data (excluding access data for memory space read requests; including access data for memory space write requests) and forwards to controller register access module 330.
The controller register access module 330 determines whether the controller register access request is to access the attribute/control register, the doorbell register, or the vendor specific register based on the memory space address.
When the doorbell register or the vendor specific register is to be accessed, the controller register access module 330 writes an access request to a storage location of the memory 340 corresponding to the memory space address based on the memory space address or reads data from a storage location of the memory 340 corresponding to the memory space address and returns the data to the host via the system bus access module 320.
When attribute/control registers are to be accessed, controller register access module 330 generates a register access message and sends the register access message to CPU360 via message queue 365. The message queue 365 is a message distribution circuit. Controller register access module 330 transmits register access messages to CPU360 through message queue 365. When a message is present in message queue 365 for delivery to CPU360, an interrupt signal may be generated to the CPU instructing CPU360 to process the message in message 365. As another example, CPU360 polls message queue 365 to discover and retrieve messages in queue 365 that are passed to CPU 360.
One or more CPUs in the CPU subsystem 360 fetches messages from the message queue 365, determines the attribute/control registers to be accessed and the type of access, and returns the results of the access to the host through the system bus access module 320.
While for other types of controller register accesses than attribute/control registers, controller register access module 330 accesses memory 340 directly.
In this way, access to various controller registers in the NVMe protocol is achieved according to embodiments of the present invention, and efficient processing is achieved for different types of controller registers, access constraints, and the like. And after the NVMe protocol is updated, the modifications introduced in the new version of the protocol to access the controller registers are efficiently adapted by modifying the program running in the CPU subsystem 360.
In addition to implementing NVMe controllers on the PCIe protocol, NVMe controllers may also be implemented on other protocols. NVMe controllers based on communication protocols such as Ethernet (Ethernet), FC (fibre channel), ib (infiniband) and the like have been developed, as well as NVMe controllers based on transport layer protocols such as RDMA. Embodiments of the present invention are implemented in these NVMe controllers. By identifying an access address to the memory space, it is identified that a controller register mapped to the memory space is to be accessed. While the access to the attribute/control register is sent by the controller register access module (fig. 2, controller register access module 230, or fig. 3, controller register access module 330) to the CPU through the message queue, the access to the attribute/control register is handled by the CPU, and for the doorbell register and/or the device-vendor specific register, a one-to-one mapping of the doorbell register and/or the device-vendor specific register is provided in memory, and the memory is accessed by the controller register access module in response to the access to the doorbell register and/or the device-vendor specific register. In another example, accesses to the doorbell registers and/or the vendor specific registers are also sent to the CPU via a message queue, and the CPU handles the accesses to these controller registers.
Fig. 4 shows a circuit system block diagram of an NVMe controller according to still another embodiment of the present invention. In the example of fig. 4, the nodes are connected together by a network. The node may be a host, a server, an NVMe device, etc. The external node is coupled to the NVMe device according to an embodiment of the present invention through a physical layer module (e.g., ethernet phy 410). The protocol processing module 420 processes the underlying protocol (e.g., IP layer) and sends an access request to the NVMe controller registers access module 430. The access request to the NVMe controller registers includes an access type, a memory space address, and/or access data (for read requests, access data is not included; for write requests, access data is included).
The controller register access module 430 determines whether the controller register access request is to access the attribute/control register, the doorbell register, or the vendor specific register based on the memory space address.
When the doorbell register or vendor specific register is to be accessed, the controller register access module 430 writes an access request to the memory 440 or reads data from the memory 440 and returns to the requesting node through the protocol processing module 420.
When an attribute/control register is to be accessed, controller register access module 430 generates an attribute/control register access message and sends the attribute/control register access message to message queue 465. One or more CPUs in the CPU subsystem fetches messages from the message queue 465, determines the attribute/control registers to be accessed and the access type, and returns the access results to the requesting node through the protocol processing module 420. In another example, the message queue is stored in memory 440 and the CPU subsystem 460 retrieves the message queue entry from memory 440.
Although the present invention has been described with reference to examples, which are intended to be illustrative only and not to be limiting of the invention, changes, additions and/or deletions may be made to the embodiments without departing from the scope of the invention.
Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these embodiments pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.
Claims (10)
1. A method of accessing NVMe controller registers, comprising:
identifying an access request to access a memory space of a PCIe device;
determining that the access request accesses NVMe controller registers according to the address of the access request, wherein the NVMe controller registers comprise first type registers, doorbell registers and/or device vendor special registers, and the first type registers occupy a predetermined range of memory register space;
if the access request accesses the first type of register, generating a register access message and sending the register access message to a CPU (central processing unit), wherein the register access message comprises a controller register address;
determining that the access request accesses the NVMe controller register according to the address of the access request, specifically, converting the access request into a controller register access request including an access type, a memory space address and/or access data, and determining the type of the NVMe controller register to be accessed according to the memory space address.
2. The method of claim 1, further comprising:
and if the access request accesses the doorbell register, accessing the memory of the PCIe equipment according to the address of the access request.
3. The method according to one of claims 1-2, further comprising:
and the CPU responds to the received register access message, identifies the type of the accessed first type of register according to the register address of the controller of the register access message, and processes the register access message according to the access mode of the first type of register defined by the NVMe protocol.
4. A method of accessing NVMe controller registers, comprising:
identifying an access request to access a memory register (memorry registers);
determining that the access request accesses an NVMe controller register according to the address of the access request;
generating an NVMe controller register access message and sending the NVMe controller register access message to a CPU, wherein the NVMe controller register access message comprises a controller register space address;
determining that the access request accesses the NVMe controller register according to the address of the access request, specifically, converting the access request into a controller register access request including an access type, a memory space address and/or access data, and determining the type of the NVMe controller register to be accessed according to the memory space address.
5. The method of claim 4, further comprising:
the CPU responds to the received NVMe controller register access message, identifies the type of the accessed NVMe controller register according to the controller register space address of the NVMe controller register access message, and processes the NVMe controller register access message according to the access mode of the NVMe controller register defined by the NVMe protocol.
6. The method of one of claims 4-5, further comprising:
if the access request accesses the doorbell register, accessing an area in a memory corresponding to the controller register space address of the access request according to the access request, and not sending an NVMe controller register access message to the CPU.
7. An NVMe controller comprising: the system comprises a system bus access unit, a controller register access unit, a CPU and a memory;
a system bus access unit coupled to the external bus and generating a controller register access request in response to a memory space access request from the system bus, the controller register access request including a first memory address and an access type; the controller register access component responds to a received controller register access request, determines that the controller register access request accesses a first type of register according to the first memory address, generates a register access message and sends the register access message to the CPU; the first type register is an attribute/control register;
the CPU responds to the received register access message, identifies the accessed NVMe controller register according to a second memory address of the register access message, and processes the register access message according to an access mode of the NVMe controller register defined by an NVMe protocol;
the method comprises the steps of responding to a memory space access request from a system bus to generate a controller register access request, specifically converting the access request into a controller register access request comprising an access type, a memory space address and/or access data, and determining the type of an NVMe controller register to be accessed according to the memory space address.
8. The NVMe controller of claim 7, wherein
The controller register access unit determines that the controller register access request accesses a doorbell register according to the first memory address, accesses an area in the memory corresponding to the first memory address, and does not send a register access message to the CPU.
9. The NVMe controller of any of claims 7-8, wherein
The controller register access component determines a first type of register of the NVMe controller register to be accessed by the controller register access request according to the first memory address, generates a register access message and sends the register access message to the CPU; wherein the first type of register occupies an address range within a predetermined range of controller register space.
10. An NVMe controller comprising: a system bus access unit, a CPU and a memory;
the system bus access component is coupled with the external bus, responds to a memory space access request from the system bus to access the NVMe controller register, generates an NVMe controller register access message and sends the NVMe controller register access message to the CPU;
the CPU responds to the received NVMe controller register access message, identifies an accessed first NVMe controller register according to a controller register address included in the NVMe controller register access message, and processes the NVMe controller register access message according to an access mode of the first NVMe controller register defined by an NVMe protocol;
the method comprises the steps of responding to a memory space access request from a system bus to access an NVMe controller register, generating an NVMe controller register access message, specifically converting the access request into a controller register access request comprising an access type, a memory space address and/or access data, and determining the type of the NVMe controller register to be accessed according to the memory space address.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610443715.3A CN107526695B (en) | 2016-06-20 | 2016-06-20 | NVMe configuration space implementation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610443715.3A CN107526695B (en) | 2016-06-20 | 2016-06-20 | NVMe configuration space implementation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107526695A CN107526695A (en) | 2017-12-29 |
| CN107526695B true CN107526695B (en) | 2021-03-02 |
Family
ID=60734661
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610443715.3A Active CN107526695B (en) | 2016-06-20 | 2016-06-20 | NVMe configuration space implementation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107526695B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20220118004A (en) * | 2021-02-18 | 2022-08-25 | 에스케이하이닉스 주식회사 | Memory system and operating method of memory system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2426607A2 (en) * | 2009-02-11 | 2012-03-07 | SanDisk IL Ltd. | System and method of host request mapping |
| CN103885908A (en) * | 2014-03-04 | 2014-06-25 | 中国科学院计算技术研究所 | Data transmission system and method based on external device and accessible registers |
| CN104011689A (en) * | 2011-11-04 | 2014-08-27 | 英特尔公司 | Nonvolatile memory wear management |
| CN104536701A (en) * | 2014-12-23 | 2015-04-22 | 记忆科技(深圳)有限公司 | Realizing method and system for NVME protocol multi-command queues |
| CN104583977A (en) * | 2012-08-23 | 2015-04-29 | 苹果公司 | Host-assisted compaction of memory blocks |
| CN104704569A (en) * | 2012-12-19 | 2015-06-10 | 惠普发展公司，有限责任合伙企业 | NVRAM path selection |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7441070B2 (en) * | 2006-07-06 | 2008-10-21 | Qimonda North America Corp. | Method for accessing a non-volatile memory via a volatile memory interface |
| US9141571B2 (en) * | 2010-12-28 | 2015-09-22 | Avago Technologies General Ip (Singapore) Pte. Ltd. | PCI express switch with logical device capability |
| US8996781B2 (en) * | 2012-11-06 | 2015-03-31 | OCZ Storage Solutions Inc. | Integrated storage/processing devices, systems and methods for performing big data analytics |
| US9122515B2 (en) * | 2012-12-19 | 2015-09-01 | Dell Products L.P. | Completion notification for a storage device |
-
2016
- 2016-06-20 CN CN201610443715.3A patent/CN107526695B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2426607A2 (en) * | 2009-02-11 | 2012-03-07 | SanDisk IL Ltd. | System and method of host request mapping |
| CN104011689A (en) * | 2011-11-04 | 2014-08-27 | 英特尔公司 | Nonvolatile memory wear management |
| CN104583977A (en) * | 2012-08-23 | 2015-04-29 | 苹果公司 | Host-assisted compaction of memory blocks |
| CN104704569A (en) * | 2012-12-19 | 2015-06-10 | 惠普发展公司，有限责任合伙企业 | NVRAM path selection |
| CN103885908A (en) * | 2014-03-04 | 2014-06-25 | 中国科学院计算技术研究所 | Data transmission system and method based on external device and accessible registers |
| CN104536701A (en) * | 2014-12-23 | 2015-04-22 | 记忆科技(深圳)有限公司 | Realizing method and system for NVME protocol multi-command queues |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107526695A (en) | 2017-12-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10922259B2 (en) | System and method for extended peripheral component interconnect express fabrics | |
| US11169938B2 (en) | Non-volatile memory (NVM) express (NVMe) data processing method and system | |
| US9025495B1 (en) | Flexible routing engine for a PCI express switch and method of use | |
| US6611883B1 (en) | Method and apparatus for implementing PCI DMA speculative prefetching in a message passing queue oriented bus system | |
| US7822908B2 (en) | Discovery of a bridge device in a SAS communication system | |
| US20200117518A1 (en) | System and method for computational storage device intercommunication | |
| US10061707B2 (en) | Speculative enumeration of bus-device-function address space | |
| JP2006505867A (en) | Method and apparatus for configurable address mapping and protection architecture and hardware for on-chip systems | |
| US8401000B2 (en) | Method of processing data packets | |
| EP3608790B1 (en) | Modifying nvme physical region page list pointers and data pointers to facilitate routing of pcie memory requests | |
| US10929060B2 (en) | Data access request specifying enable vector | |
| US11201838B2 (en) | System, apparatus and method for increasing efficiency of link communications | |
| US7827343B2 (en) | Method and apparatus for providing accelerator support in a bus protocol | |
| US10911261B2 (en) | Method, apparatus and system for hierarchical network on chip routing | |
| CN111078597B (en) | Interrupt message generation device and method and end equipment | |
| KR101865261B1 (en) | Input output data alignment | |
| CN107526696B (en) | Software defined SR-IOV module | |
| CN107526695B (en) | NVMe configuration space implementation method and device | |
| KR20210092222A (en) | Chaining memory requests on the bus | |
| US20120324078A1 (en) | Apparatus and method for sharing i/o device | |
| US20230266917A1 (en) | Storage system | |
| US20060277326A1 (en) | Data transfer system and method | |
| US9535850B1 (en) | System and method for efficient DMA transfers | |
| JP6053921B2 (en) | Bus relay device | |
| KR100643815B1 (en) | I/o configuration messaging within a link-based computing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |