TWI774255B - Bridge circuit and computer system - Google Patents

Abstract

A bridge circuit includes an NVMe device controller, a network subsystem, and a data transfer circuit. The NVMe device controller is used to communicate with a host via a PCIe bus. The network subsystem is used to communicate with an NVMe-TCP device via a network. The data transfer circuit is coupled between the NVMe device controller and the network subsystem, and is used to deal with data transfer associated with the NVMe-TCP device without intervention of the host.

Description

Bridging circuits and computer systems

The present invention relates to data processing, especially to a bridge circuit that can provide conversion between PCIe-NVMe protocol and NVMe-TCP protocol and a computer system using the bridge circuit.

NVMe (Non-Volatile Memory Express) is a host control interface specification for non-volatile memory. It is a communication protocol specially designed for flash memory devices using PCIe (Peripheral Component Interconnect Express) bus. The PCIe-NVMe protocol is used to write data into a solid-state drive (SSD) with a PCIe interface, and the data stored in a solid-state drive with a PCIe interface can be read through the PCIe-NVMe protocol. For SSDs using other I/O interfaces, SSDs with PCIe interface can have faster read and write speeds through the PCIe-NVMe protocol. However, since the computer host needs to access the SSD through the PCIe bus, the standard PCIe-NVMe protocol cannot be used for data access of remote storage devices, so NVMe-oF (NVMe-over-Fabrics) ) solution to support network storage in data centers, in which the NVMe-TCP (NVMe-over-TCP) protocol can operate on existing network infrastructure without special switches, routers and network cards. The TCP protocol can easily implement the deployment of remote storage devices.

However, for the host side, the complexity of the NVMe-TCP protocol itself will greatly increase the computing load of the CPU. In addition, in order to access NVMe-TCP devices (such as solid-state drives connected to the local area network), the host side In addition, a new driver needs to be installed, because different drivers are required to separate Dealing with the PCIe-NVMe protocol and the NVMe-TCP protocol will inevitably increase the system complexity.

Therefore, one of the objectives of the present invention is to provide a bridge circuit that can provide conversion between the PCIe-NVMe protocol and the NVMe-TCP protocol and a computer system using the bridge circuit.

In one embodiment of the present invention, a bridge circuit is disclosed. The bridge circuit includes an NVMe device controller, a network subsystem, and a data transfer circuit. The NVMe device controller is used to communicate with a computer host through a PCIe bus. The network subsystem is used to communicate with an NVMe-TCP device through a network. The data transfer circuit is coupled between the NVMe device controller and the network subsystem, and is used for processing data transfer related to the NVMe-TCP device without the intervention of the computer host.

In another embodiment of the present invention, a computer system is disclosed. The computer system includes a computer host and a bridge circuit. The bridge circuit includes an NVMe device controller, a network subsystem, and a data transfer circuit. The NVMe device controller is used to communicate with the computer host through a PCIe bus. The network subsystem is used to communicate with an NVMe-TCP device through a network. The data transfer circuit is coupled between the NVMe device controller and the network subsystem, and is used for processing data transfer related to the NVMe-TCP device. The bridge circuit allows the NVMe-TCP device to be accessed by the computer host as a PCIe-NVMe device.

In another embodiment of the present invention, a computer system is disclosed. The computer system includes a computer host and a bridge circuit. The bridge circuit includes an NVMe device controller, a network subsystem, and a data transfer circuit. The NVMe device controller is used to communicate with the communicate with the host computer. The network subsystem is used to communicate with an NVMe-TCP device through a network. The data transfer circuit is coupled between the NVMe device controller and the network subsystem, and is used for processing data transfer related to the NVMe-TCP device. The computer host loads and executes the PCIe-NVMe driver to control the communication between the computer host and the NVMe device controller, and does not need to have the NVMe-TCP driver.

The bridge circuit disclosed in the present invention can offload the processing of NVMe-TCP from the host-side processor, thereby greatly reducing the computing load of the host-side processor. In addition, the host-side processor only needs to execute the PCIe-NVMe driver. The bridge circuit disclosed in the present invention completes the data access of the NVMe-TCP device without installing and executing the NVMe-TCP driver, so the system complexity can be reduced.

100: Computer System

102: Computer host

103: PCIe bus

104: PCIe interface card

105: Internet

106: NVMe-TCP device

112: CPU

114: System Storage Device

116:Submit Queue

118: Complete the queue

122,200: Bridge circuit

124: local storage device

132: non-volatile memory

201: Wafer

202,300: NVMe Device Controller

204,400: Data transfer circuit

206,500: Network Subsystem

302: PCIe Controller

304: Host Controller

306: Submission queue processing circuit

308: Complete the queue processing circuit

310: storage address processing circuit

312, 314, 316, 318: Functional Blocks

402: Data access circuit

404, 406, 408: Storage elements

410: Lookup Table

412, 414: Linked Sequence

502: Unload engine

504: NVMe-TCP Controller

FIG. 1 is a schematic diagram of a computer system using a bridge circuit according to the present invention.

FIG. 2 is a schematic diagram of a bridge circuit according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of an NVMe device controller according to an embodiment of the present invention.

FIG. 4 is a schematic diagram of a data transfer circuit according to an embodiment of the present invention.

FIG. 5 is a schematic diagram of a network subsystem according to an embodiment of the present invention.

FIG. 1 is a schematic diagram of a computer system using a bridge circuit according to the present invention. The computer system 100 includes a computer host 102, a PCIe interface card 104 and at least one NVMe-TCP device 106. For the sake of brevity, only one NVMe-TCP device 106 is shown in FIG. 1. However, in practical applications, the computer system 100 A plurality of NVMe-TCP devices 106 may be provided according to application requirements. The computer host 102 includes a central processing unit 112 As with system storage 114, for example, system storage 114 may be memory. In addition, the system storage device 114 is provided with a submission queue (SQ) 116 and a completion queue (CQ) 118 , wherein the submission queue 116 is used for storing commands issued by the computer host 102 to complete the queues. Column 118 is paired with commit queue 116 to store completion information for the command. A bridge circuit 122 and a local storage device 124 that can be used by the bridge circuit 122 are disposed on the PCIe interface card 104 . For example, the local storage device 124 may be a memory. The PCIe interface card 104 can be installed in a PCIe slot on the computer host 102 , so the bridge circuit 122 can communicate with the computer host 102 through the PCIe bus bar 103 . In this embodiment, the bridge circuit 122 and the NVMe-TCP device 106 are both connected to the network 105, for example, the network 105 is a local area network. The NVMe-TCP device 106 includes a non-volatile memory 132, for example, the NVMe-TCP device 106 is a solid state drive using the NVMe-TCP protocol. In this embodiment, the bridge circuit 122 can provide conversion between the PCIe-NVMe protocol and the NVMe-TCP protocol. In other words, the bridge circuit 122 communicates with the computer host 102 through the PCIe-NVMe protocol, and communicates with the NVMe through the NVMe-TCP protocol -Communicate with the TCP device 106, therefore, the computer host 102 itself does not need to process the NVMe-TCP protocol stack, so the computer host 102 only needs to load and execute the PCIe-NVMe driver DRV_NVMe to control the communication between the computer host 102 and the bridge circuit 122 In fact, there is no need for an NVMe-TCP driver to communicate with each other. Further, the bridge circuit 104 allows the NVMe-TCP device 106 to be accessed by the computer host 102 as a PCIe-NVMe device.

FIG. 2 is a schematic diagram of a bridge circuit according to an embodiment of the present invention. The bridge circuit 122 shown in FIG. 1 can be implemented by the bridge circuit 200 shown in FIG. 2 . As shown in FIG. 2 , the bridge circuit 200 includes an NVMe device controller 202 , a data transfer circuit 204 and a network subsystem 206 . In this embodiment, the NVMe device controller 202 , the data transfer circuit 204 and the network subsystem 206 are all disposed in the same chip 201 . The NVMe device controller 202 is used to communicate with the computer host 102 through the PCIe bus 103 . The network subsystem 206 is used to communicate with the NVMe-TCP device 106 through the network (eg, a local area network) 105 . The data transfer circuit 204 is capable of direct memory access (DMA) It is coupled between the NVMe device controller 202 and the network subsystem 206 for processing data transfers related to the NVMe-TCP device 106 (including through the NVMe device controller 202 and the network subsystem 206) without the intervention of the computer host 102. The network subsystem 206 transfers data from the computer host 102 to the non-volatile memory 132 in the NVMe-TCP device 106 , and through the NVMe device controller 202 and the network subsystem 206 to transfer the non-volatile memory of the NVMe-TCP device 106 The data stored in the memory 132 is forwarded to the computer host 102).

FIG. 3 is a schematic diagram of an NVMe device controller according to an embodiment of the present invention. The NVMe device controller 202 shown in FIG. 2 may be implemented by the NVMe device controller 300 shown in FIG. 3 . As shown in FIG. 3 , the NVMe device controller 300 includes a PCIe controller 302 , a host controller 304 , a commit queue processing circuit 306 , a completion queue processing circuit 308 , and a storage address processing circuit 310 . The PCIe controller 302 is used to access the PCIe bus 103 . The host controller 304 includes a plurality of functional blocks, wherein the functional block (marked as DNTRFC) 312 is used to control the downstream transaction of the PCIe bus 103, and the functional block (marked as UPTRFC) 314 is used to control the PCIe For the upstream transaction of the bus 103, the functional block (labeled as PCIe_cfg) 316 includes registers to store PCIe configuration settings, and the functional block (labeled as NVMe_cfg) 318 includes registers to store NVMe configuration settings.

The submission queue processing circuit 306 is used to read and process commands (eg, write commands or read commands) in the submission queue 116 of the host computer 102. After the command) is written to the commit queue 116, the computer host 102 will notify the NVMe device controller 300, and the commit queue processing circuit 306 will subsequently send the command (such as a write command or a read command) from the commit queue 116 Extract it. The completion queue processing circuit 308 is used to write the completion information of the command (such as a write command or a read command) into the completion queue 118 of the computer host 102, for example, when the data transfer circuit 204 has indicated the write command host-side data transfer to non-volatile memory in NVMe-TCP device 106 body 132, the completion queue processing circuit 308 will write the completion information of the write command into the completion queue 118; similarly, when the data transfer circuit 204 has transmitted the device-side data indicated by the read command to the host In the system storage device 114 in the device 102 , the completion queue processing circuit 308 writes the completion information of the read command into the completion queue 118 .

The storage address processing circuit 310 is used for retrieving the storage address information attached to the command (such as a write command or a read command), and can set the data transfer circuit 204 according to the storage address information, such as the storage address The information specifies the storage address in the system storage device 114 to be written to the host-side data of the non-volatile memory 132 in the NVMe-TCP device 106 when the command to be processed in the submission queue 116 is a write command will be located at the storage address specified by the storage address information. In addition, when the command to be processed in the submission queue 116 is a read command, it will be read from the non-volatile memory 132 of the NVMe-TCP device 106 Device-side data will be written to the storage address specified by the storage address information. In this embodiment, the storage address information can be recorded by using the data structure of the Scatter Gather List (SGL). However, the present invention is not limited to this. In other embodiments, the storage address information The data structure of Physical Region Page (PRP) can also be used to record. Since the storage address information attached to the NVMe command (eg, write command or read command) specifies the storage address in the system storage device 114 , the storage address processing circuit 310 can configure the data transfer circuit 204 accordingly. , so that the data transfer circuit 204 can correctly process the data transfer operation related to the NVMe-TCP device 106 .

FIG. 4 is a schematic diagram of a data transfer circuit according to an embodiment of the present invention. The data transfer circuit 204 shown in FIG. 2 can be realized by the data transfer circuit 400 shown in FIG. 4 . As shown in FIG. 4 , the data transfer circuit 400 includes a data access circuit 402 and a plurality of storage elements (eg, memories) 404 , 406 and 408 . The storage element 406 is used for storing a linked list 412 , wherein each node in the linked list 412 records a storage address in the system storage device 114 , for example, a plurality of nodes in the linked list 412 . The plurality of storage addresses ADDR_A1 , ADDR_A2 , ADDR_A3 , and ADDR_A4 in the system storage device 114 are respectively recorded. The storage element 408 is used to store another link series 414, wherein each node in the link series 414 records a storage address in the local storage device 124, for example, a plurality of nodes in the link series 414 respectively record the local storage device A plurality of storage addresses ADDR_B1, ADDR_B2, ADDR_B3, ADDR_B4 in 124. The storage element 404 is used for storing a look-up table 410, wherein the look-up table 410 records the storage address PTR_1 and The first node in the chain 414 (in this embodiment, the node recording the storage address ADDR_B1 ) is connected to the storage address PTR_2 in the storage element 408 . The data access circuit 402 is used for reading the link string 412 and the link string 414 according to the lookup table 410 to process the data transfer operation between the computer host 102 and the NVMe-TCP device 106 .

When the computer host 102 writes an NVMe command (such as a write command or a read command) to the submission queue 116, the NVMe command will have a corresponding NVMe command identifier HID. In addition, through the network 105, the bridge circuit 122 will Communicates with the NVMe-TCP device 106 through the TCP session ID (TCP session ID) SID, and generates and transmits NVMe-TCP commands (such as write commands or read commands) based on the PCIe-NVMe commands to the NVMe-TCP device 106, and the NVMe-TCP command will also have a corresponding command identifier CID. In this embodiment, the NVMe command identifier HID will be bound to a corresponding pair of TCP connection identifiers SID and NVMe- The TCP command identifier CID is used as an index of the lookup table 410. As shown in FIG. 4, the lookup table 410 maps a set of NVMe command identifiers HID, TCP connection identifiers SID and NVMe-TCP command identifiers CID to links The storage address PTR_1 of the first node in the string 412 in the storage element 406 and the storage address of the first node in the link string 414 in the storage element 408, therefore, the data access circuit 402 can follow the set of NVMe commands The identification code HID, the TCP connection identification code SID and the NVMe-TCP command identification code CID come from the lookup table 410 to read out the index about the starting point of the link sequence (including the first node in the link sequence 412 in the storage element 406 ) Storage address PTR_1 and link string 414 The first node in the storage element 408 is at the storage address PTR_2).

According to the NVMe command (such as a write command or a read command) issued by the computer host 102, the data access circuit 402 can refer to the namespace to be accessed to determine which NVMe-TCP device on the network 105 is to be stored Take, if the NVMe-TCP device 106 is to be accessed, the data access circuit 402 will bind the TCP connection between the bridge circuit 122 and the NVMe-TCP device 106 to the namespace of the NVMe-TCP device 106, When the bridge circuit 122 transmits the NVMe-TCP command to the NVMe-TCP device 106, the data access circuit 402 will create the look-up table 410 and process the information provided by the circuit 310 according to the storage address (that is, the information attached to the NVMe command). storage address information) to configure link string 412 in storage element 406 and link string 414 in storage element 408.

If the NVMe command issued by the computer host 102 is a write command, the data access circuit 402 starts to read the data recorded in the link string 412 according to the storage address PTR_1 of the first node in the link string 412 in the storage element 406 . The storage addresses ADDR_A1~ADDR_A4 are used to read a piece of data from the system storage device 114 (read from the storage addresses ADDR_A1~ADDR_A4), and according to the storage address PTR_2 of the first node in the link string 414 in the storage element 408 Start to read the storage addresses ADDR_B1 ˜ADDR_B4 recorded in the link string 414 to write the data to the local storage device 124 (write to the storage addresses ADDR_B1 ˜ADDR_B4 ). After that, the data access circuit 402 starts to read the storage addresses ADDR_B1-ADDR_B4 recorded in the link string 414 according to the storage address PTR_2 of the first node in the link string 414 in the storage element 408, so as to obtain the data from the local storage device. 124 reads the data (read from storage addresses ADDR_B1 to ADDR_B4 ), and transmits the data to the NVMe-TCP device 106 through the network subsystem 206 for writing to the non-volatile memory 132 .

If the NVMe command issued by the computer host 102 is a read command, the network subsystem 206 A piece of data is received from the NVMe-TCP device 106. In addition, the data access circuit 402 starts to read the storage recorded in the link string 414 according to the storage address PTR_2 of the first node in the link string 414 in the storage element 408 Addresses ADDR_B1 to ADDR_B4 are used to write the data to the local storage device 124 (write to storage addresses ADDR_B1 to ADDR_B4). Afterwards, the data access circuit 402 will start to read the storage addresses ADDRB1-ADDR_B4 recorded in the link string 414 according to the storage address PTR_2 of the first node in the link string 414 in the storage element 408 to store the data locally The device 124 reads the data (read from the storage addresses ADDR_B1 to ADDR_B4), and starts to read the record in the link string 412 according to the storage address PTR_1 of the first node in the link string 412 in the storage element 406 to write the data to the system storage device 114 (write to the storage addresses ADDR_A1 to ADDR_A4).

As previously mentioned, the network subsystem 206 communicates with the NVMe-TCP device 106 through the network (eg, a local area network) 105 , eg, the network subsystem 206 passes commands to the NVMe-TCP device 106 through the network 105 , pass write data to NVMe-TCP device 106 , and receive read data from NVMe-TCP device 106 . FIG. 5 is a schematic diagram of a network subsystem according to an embodiment of the present invention. The network subsystem 206 shown in FIG. 2 may be implemented by the network subsystem 500 shown in FIG. 5 . As shown in FIG. 5 , the network subsystem 500 includes an offload engine 502 and an NVMe-TCP controller 504 . The offload engine 502 is dedicated hardware for handling the Transmission Control Protocol/Internet Protocol stack (TCP/IP stack) between the network subsystem 500 and the NVMe-TCP device 106, so the bridge circuit 122/200 does not require a processor Involves the processing of the Transmission Control Protocol/Internet Protocol stack. The NVMe-TCP controller 504 is used to trigger the data transfer circuit 204/400 to read data from the computer host 102 for transmission to the NVMe-TCP device 106 through the offload engine 502, and to trigger the data transfer circuit 204/400 to transfer the offload engine 502 from the NVMe - The data received by the TCP device 106 is transmitted to the computer host 102 . In addition, the NVMe-TCP controller 504 can also encode the TCP connection identifier SID and the NVMe-TCP command identifier CID into the network packet according to the NVMe command identifier HID. , and also encode the NVMe-TCP data (including write commands or read commands) and add the corresponding Cyclic Redundancy Check (Cyclic Redundancy Check) data to the network packet; and the NVMe-TCP controller 504 can also Decode the network packets to obtain NVMe-TCP data (including the completion information of the write command or the completion information of the read command), and perform data correctness checking and error correction according to the corresponding CRC data.

To sum up, the bridge circuit disclosed in the present invention can offload the processing of NVMe-TCP from the host-side processor, thereby greatly reducing the computing load of the host-side processor. In addition, the host-side processor only needs to execute PCIe-NVMe The driver can complete the data access of the NVMe-TCP device through the bridge circuit disclosed in the present invention, and there is no need to install and execute the NVMe-TCP driver, so the system complexity can be reduced.

The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of the present invention.

100: Computer Systems

102: Computer host

103: PCIe bus

104: PCIe interface card

105: Internet

106: NVMe-TCP device

112: CPU

114: System Storage Device

116:Submit Queue

118: Complete the queue

122: Bridge circuit

124: local storage device

132: non-volatile memory

Claims (12)

A bridge circuit includes: an NVMe device controller for communicating with a computer host through a PCIe bus; a network subsystem for communicating with an NVMe-TCP device through a network; and A data transfer circuit, coupled between the NVMe device controller and the network subsystem, is used to process data transfer about the NVMe-TCP device without the intervention of the computer host; wherein the NVMe device controller , the network subsystem and the data transfer circuit are all arranged on the same chip. The bridge circuit of claim 1, wherein the NVMe device controller comprises: a PCIe controller for accessing the PCIe bus; a host controller coupled to the PCIe controller for To control the upstream data processing and downstream data processing of the PCIe bus, NVMe configuration settings and PCIe configuration settings; a submission queue processing circuit, coupled to the host controller, for reading and processing the computer host. a command in the submission queue; a completion queue processing circuit, coupled to the host controller, for writing the completion information of the command into a completion queue of the computer host; and a storage address processing circuit , which is coupled to the host controller and used for retrieving the storage address information attached to the command and setting the data transfer circuit according to the storage address information. The bridge circuit of claim 1, wherein the network subsystem comprises: an offload engine for processing TCP/IP stacking between the network subsystem and the NVMe-TCP device ;as well as an NVMe-TCP controller for triggering the data transfer circuit to read data from the computer host to transmit to the NVMe-TCP device through the offload engine, and triggering the data transfer circuit to read data from the NVMe-TCP by the offload engine The data received by the device is transmitted to the computer host. The bridge circuit as described in claim 1, wherein the data transfer circuit comprises: a first storage element for storing a first link series, and each node in the first link series records the a storage address in a system storage device of the computer host; a second storage element for storing a second link string, each node record in the second link string can be used by the bridge circuit a storage address in the local storage device; a third storage element for storing a look-up table, the look-up table records the storage address of the first node in the first link string in the first storage element and the The storage address of the first node in the second link string in the second storage element; and a data access circuit for reading the first link string and the second link string according to the look-up table , to handle data transfer between the computer host and the NVMe-TCP device. The bridge circuit as described in claim 4, wherein the lookup table further records a set of NVMe command identifiers, TCP connection identifiers and NVMe-TCP command identifiers, which are mapped to the first one in the first link sequence the storage address of the node in the first storage element and the storage address of the first node in the second link string in the second storage element; and the data access circuit is further used for identifying according to the set of NVMe commands code, TCP connection ID and NVMe-TCP command ID from the look-up table to read the storage address of the first node in the first link string in the first storage element and the second link string The storage address of the first node in the second storage element. The bridge circuit as described in claim 4, wherein the data access circuit starts to read the first link string according to the storage address of the first node in the first link string in the first storage element row, to read a piece of data from the system storage device, and start reading the second link string according to the storage address of the first node in the second link string in the second storage element, so as to Pen data is written to the local storage device. The bridge circuit as described in claim 6, wherein the data access circuit further starts to read the second link according to the storage address of the first node in the second link string in the second storage element serial to read the data from the local storage device and transmit the data to the NVMe-TCP device through the network subsystem. The bridge circuit of claim 4, wherein the network subsystem receives a piece of data from the NVMe-TCP device; and the data access circuit stores the data in the second storage according to the first node in the second link string The storage address in the device starts to read the second link string to write the data to the local storage device. The bridge circuit as described in claim 8, wherein the data access circuit further starts to read the second link according to the storage address of the first node in the second link string in the second storage element a series, to read the data from the local storage device, and start reading the first link series according to the storage address of the first node in the first link series in the first storage element, so as to Write the data to the system storage device. The bridge circuit as described in claim 1, wherein the bridge circuit is disposed on a PCIe interface card. A computer system includes: a computer host; and a bridge circuit, including: an NVMe device controller for communicating with the computer host through a PCIe bus; a network subsystem for connecting through a network communication with an NVMe-TCP device; and a data transfer circuit, coupled between the NVMe device controller and the network subsystem, for processing data transfer related to the NVMe-TCP device; wherein the NVMe device The controller, the network subsystem and the data transfer circuit are all disposed on the same chip, and the bridge circuit allows the NVMe-TCP device to be accessed by the computer host as a PCIe-NVMe device. A computer system includes: a computer host; and a bridge circuit, including: an NVMe device controller for communicating with the computer host through a PCIe bus; a network subsystem for connecting through a network communication with an NVMe-TCP device; and a data transfer circuit, coupled between the NVMe device controller and the network subsystem, for processing data transfer related to the NVMe-TCP device; wherein the NVMe device The controller, the network subsystem and the data transfer circuit are all set on the same chip, and the computer host loads and executes the PCIe-NVMe driver to control the communication between the computer host and the NVMe device controller, and does not need to have NVMe-TCP driver.

Images