WO2021063154A1

WO2021063154A1 - Queue communication method and system, and related devices

Info

Publication number: WO2021063154A1
Application number: PCT/CN2020/113497
Authority: WO
Inventors: 李涛; 江锦毅; 李生; 周宇; 杨荣玉
Original assignee: 华为技术有限公司
Priority date: 2019-09-30
Filing date: 2020-09-04
Publication date: 2021-04-08
Also published as: CN110750367B; CN110750367A

Abstract

The present application provides a queue communication method. The method comprises: a forwarding device receives a first doorbell message from a computing device; the forwarding device obtains information about a first host queue and information about a second host queue from the first doorbell message, obtains, from the first host queue, first data to be sent, and obtains, from the second host queue, second data to be sent; the forwarding device respectively stores, according to the queue correspondence between forwarding queues and the host queues, said first data and said second data into the forwarding queue corresponding to the first host queue and the second host queue, the first host queue and the second host queue corresponding to the same forwarding queue in the at least one forwarding queue.

Description

Method, system and related equipment for queue communication

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office, the application number is 201910944642.X, and the invention title is "a method, system and related equipment for queue communication" on September 30, 2019. The entire content of the application is approved. The reference is incorporated in this application.

Technical field

This application relates to the field of computers, and in particular to a method, system and related equipment for queue communication.

Background technique

The rise of technologies such as multi-core processors, virtualization technology, and distributed storage has prompted the emergence of a new type of computing model, namely cloud computing. Cloud computing is based on a configurable shared computing resource pool to provide users with convenient, on-demand access services. In general, cloud computing service providers will provide a variety of virtual machine specifications, and flexibly distribute virtual machines of different specifications according to user needs. For example, when users have greater demands on the network, users can be divided into a large-format virtual machine with a 64-bit Virtual Central Processing Unit (vCPU) and 128GB of storage space.

The virtual machine runs on the computing device, and configuring the virtual machine also includes allocating multiple host queues on the computing device to the virtual machine. When a virtual machine communicates with a device other than the computing device where it is located, it needs to send data to the network card. For a virtual machine with a configured network card queue, when you need to send data to the network card, you can directly send the data to be sent to the host queue assigned to the virtual machine, and then the host queue assigned to the virtual machine and the network card queue on the network card pass The queue communication mechanism copies the to-be-sent data in the host queue allocated to the virtual machine to the network card queue of the network card to complete a data communication between the virtual machine and the network card.

In the queue communication mechanism of related technologies, the host queue on the computing device and the network card queue of the network card are one-to-one mapping. In other words, the data to be sent from the virtual machine stored in a host queue of the computing device can only be sent to the network card A network card queue on the. Normally, the performance of queue communication is restricted by the forwarding ability of the host queue on the computing device and the forwarding ability of the network card queue of the network card. When the virtual machine sends data to the network card, it is subject to the forwarding ability of the virtual machine vCPU. The forwarding ability of the queue assigned to the virtual machine on the host is usually weaker than the forwarding ability of the queue on the network card, which causes the forwarding performance of the queue on the network card to not be fully utilized. , Affect the efficiency of communication.

Summary of the invention

The present application provides a queue communication method, system and related equipment, which are used to solve the problem that the queue forwarding performance on the network card cannot be fully utilized, which causes the communication efficiency to be affected.

In a first aspect, a method for queue communication is provided, the method is applied to a forwarding device, the forwarding device is connected to a computing device, the forwarding device includes at least one forwarding queue, and the computing device includes multiple host queues, A plurality of data to be sent are stored in the plurality of host queues, and the method includes:

The forwarding device receives a first doorbell message from the computing device, where the first doorbell message carries information about a first host queue and information about a second host queue, and the first host queue and the The second host queues all belong to the multiple host queues;

The forwarding device obtains the information of the first host queue and the information of the second host queue from the first doorbell message, obtains the first data to be sent from the first host queue, and obtains the first data to be sent from the first host queue. 2. Obtain the second data to be sent from the host queue;

The forwarding device stores the first to-be-sent data and the second to-be-sent data into the first host queue and the second host respectively according to the queue correspondence between the forwarding queue and the host queue A forwarding queue corresponding to the queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.

Implement the method described in the first aspect, by generating a queue correspondence between multiple host queues and the same forwarding queue, so that multiple host queues with low forwarding performance can share a high forwarding queue for queue communication, thereby solving the problem. Since the host queue forwarding performance is lower than the network card queue, the queue forwarding performance on the network card cannot be fully utilized, thereby improving the communication efficiency.

In an embodiment, before the forwarding device receives the first doorbell message from the computing device, the method further includes: the forwarding device receives a device query message from the computing device; responding to the device query Message, the forwarding device reports the number of extended forwarding queues to the computing device, where the number of extended forwarding queues is determined by the number of the at least one forwarding queue and the expansion coefficient, and the extended The subsequent number of forwarding queues is used to establish the queue correspondence.

In an embodiment, the forwarding device includes an attribute register, and the number of the extended forwarding queues is stored in the attribute register; the forwarding device reporting the number of extended forwarding queues to the computing device includes : The forwarding device queries the attribute register, obtains the number of the extended forwarding queues from the attribute register, and reports the number of the extended forwarding queues to the computing device.

In the foregoing embodiment, the forwarding device sends the expanded number of forwarding queues to the computing device, so that the computing device can allocate multiple host queues to one forwarding queue to generate a queue correspondence relationship, without any upgrade of the computing device. , It can solve the problem that the queue forwarding performance on the network card cannot be fully exerted because the host queue forwarding performance is lower than the network card queue, and the communication efficiency is improved.

In an embodiment, the information of the first host queue includes a first head pointer, the first head pointer indicates the first to-be-sent data, and the information about the second host queue includes a second head pointer, so The second head pointer indicates the second data to be sent, and multiple sets of head pointers and multiple sets of tail pointers are stored in the forwarding device, and each set of head pointers and each set of tail pointers respectively correspond to multiple hosts of the computing device The head pointer and the tail pointer of a host queue in the queue, the forwarding device also stores at least one set of head pointers and at least one set of tail pointers of the at least one forwarding queue; Obtaining the information of the first host queue and the information of the second host queue in the message includes: the forwarding device obtains the first head pointer and the second head pointer from the first doorbell message; The forwarding queue acquiring the first to-be-sent data from the first host queue includes: the forwarding device refreshes the head pointer corresponding to the first host queue in the plurality of sets of head pointers according to the first head pointer The forwarding device obtains the first host queue from the first host queue according to the head pointer corresponding to the first host queue after the refresh and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers A data to be sent; the forwarding queue obtains the second data to be sent from the second host queue, including: the forwarding device refreshes the second host in the multiple sets of head pointers according to the second head pointer The head pointer corresponding to the queue; the forwarding device, according to the head pointer corresponding to the second host queue after being refreshed, and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers, in the second host queue The second data to be sent is obtained in the forwarding device; after the forwarding device stores the first data to be sent and the second data to be sent into the forwarding queues corresponding to the first host queue and the second host queue, respectively The method further includes: refreshing the tail pointer of the first host queue and the tail pointer of the second host queue; refreshing the head pointer of the corresponding forwarding queue; and refreshing the tail pointer of the first host queue after refreshing And sending the refreshed tail pointer of the second host queue to the computing device.

In an embodiment, the forwarding device includes a queue expansion device and a network card, the at least one forwarding queue includes at least one queue expansion device queue and at least one network card queue, and the at least one queue expansion device queue is located in the queue expansion device. The at least one network card queue is located in the network card, the at least one set of head pointers and the at least one set of tail pointers are stored in the queue expansion device, and the queue correspondence relationship includes a first queue correspondence relationship and a second queue Correspondence, the first queue correspondence is the correspondence between the multiple host queues and the at least one queue expansion device queue, and the second queue correspondence is the at least one network card queue and the at least one queue The corresponding relationship between the expansion device queues, the at least one network card queue corresponds to the at least one queue expansion device queue, and the queue expansion device stores multiple sets of head pointers and multiple sets of tail pointers, each set of head pointers and each set of tail pointers The group tail pointers respectively correspond to the head pointer and the tail pointer of a host queue in the plurality of host queues. The queue expansion device also stores at least one set of queue expansion device head pointers and at least one set of queue expansion device tails. Pointers, the network card stores at least one set of network card head pointers and at least one set of network card tail pointers, the at least one set of queue expansion device head pointers corresponds to the at least one set of network card head pointers, and the at least one The tail pointer of the group queue expansion device corresponds to the tail pointer of the at least one group of network cards in a one-to-one correspondence; the forwarding device obtains the information of the first host queue and the information of the second host queue from the first doorbell message , Including: the queue expansion device in the forwarding device obtains the first head pointer and the second head pointer from the first doorbell message; the forwarding device obtains the first head pointer from the first host queue The data to be sent includes: the queue expansion device in the forwarding device refreshes the head pointer corresponding to the first host queue among the multiple sets of head pointers according to the first head pointer and the first queue correspondence relationship The queue expansion device in the forwarding device is based on the head pointer corresponding to the first host queue after refreshing, and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers, in the first host queue Acquiring the first data to be sent in the forwarding device; acquiring the second data to be sent from the second host queue by the forwarding device includes: a queue expansion device in the forwarding device according to the second head pointer, and The first queue correspondence relationship, refreshing the head pointer corresponding to the second host queue in the multiple sets of head pointers; the queue expansion device in the forwarding device according to the refreshed head pointer corresponding to the second host queue, and The tail pointer corresponding to the second host queue in the multiple sets of tail pointers acquires the second data to be sent in the second host queue; the forwarding device combines the first data to be sent with the Respectively storing the second data to be sent in the first host queue and the forwarding queue corresponding to the second host queue includes: the queue expansion device in the forwarding device transfers the first data to be sent according to the corresponding relationship of the first queue Data and the second pending Send data to the first host queue and the queue expansion device queue corresponding to the second host queue, wherein the corresponding queue expansion device queue belongs to the queue expansion device queue, and the first host queue and the second host queue The two host queues correspond to the same queue expansion device queue in the at least one queue expansion device queue; the queue expansion device in the forwarding device refreshes the head pointer of the corresponding queue expansion device queue, and the first host queue The tail pointer and the tail pointer of the second host queue; the network card in the forwarding device receives the second doorbell message from the queue expansion device, wherein the second doorbell message carries the corresponding queue expansion device queue Head pointer; the network card in the forwarding device obtains the head pointer of the corresponding queue expansion device queue from the second doorbell message; the network card in the forwarding device expands the head pointer of the device queue according to the corresponding queue , And the second queue correspondence relationship, refreshing the network card head pointer corresponding to the head pointer of the corresponding queue expansion device queue in the at least one set of network card head pointers; the network card in the forwarding device is based on the corresponding The head pointer of the network card, and the tail pointer corresponding to the corresponding queue expansion device queue in the at least one set of network card tail pointers, and the first data to be sent and the second data to be sent are acquired in the corresponding queue expansion device queue Send data; the network card in the forwarding device stores the first to-be-sent data and the second to-be-sent data in the network card queue corresponding to the corresponding queue expansion device queue according to the second queue correspondence, wherein , The corresponding network card queue belongs to the network card queue.

In the foregoing embodiment, a plurality of host queues in the computing device and a queue expansion device queue in the queue expansion device generate a first queue correspondence, and a queue expansion device queue in the queue expansion device and a network card queue in a network card generate a first queue correspondence relationship. The corresponding relationship between the two queues. In this way, without the need to upgrade both the computing device and the network card, the first data to be sent and the second data to be sent from the computing device are forwarded to the network card through the queue expansion device, which can solve the problem The host queue forwarding performance is lower than the network card queue, which leads to the problem that the queue forwarding performance on the network card cannot be fully utilized, and the communication efficiency is improved.

In an embodiment, the first data to be sent is a first direct memory access DMA read descriptor, the second data to be sent is a second DMA read descriptor, and the first data to be sent is And after the second data to be sent is stored in the network card queue corresponding to the corresponding queue expansion device queue, the method further includes: the network card in the forwarding device reads the descriptor according to the first DMA and the second DMA read descriptor to generate a first DMA read command and a second DMA read command; the network card in the forwarding device sends the first DMA read command and the second DMA read command to the queue expansion device in the forwarding device; The queue expansion device in the forwarding device obtains first data to be read from the computing device according to the first DMA read command, and obtains second data to be read from the computing device according to the second DMA read command The queue expansion device in the forwarding device forwards the first data to be read and the second data to be read to the network card in the forwarding device.

In an embodiment, the first data to be sent is a first DMA write descriptor, and the second data to be sent is a second DMA write descriptor, wherein the first DMA write descriptor is used to transfer the first DMA write descriptor. A data to be written is written from the queue expansion device to the computing device, the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device, and the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device. The storing of the first data to be sent and the second data to be sent in the first host queue and the queue expansion device queue corresponding to the second host queue includes: the queue expansion device in the forwarding device transfers the first The DMA write descriptor is converted into a first network card DMA write descriptor, and the second DMA write descriptor is converted into a second network card DMA write descriptor, where the first DMA write descriptor is used to transfer the first to be written Incoming data is written from the network card to the queue expansion device, and the second DMA descriptor is used to write the second data to be written from the network card to the queue expansion device; the queue expansion in the forwarding device The device stores the first network card DMA write descriptor and the second network card DMA write descriptor in the corresponding queue expansion device queue; the storing the first data to be sent and the second data to be sent in After the network card queue corresponding to the corresponding queue expansion device queue, the method further includes: the network card in the forwarding device obtains the data stored in the network card according to the DMA write descriptor of the first network card. According to the first fragment data corresponding to the DMA write descriptor of the first network card, according to the DMA write descriptor of the second network card, obtain the second piece of data stored in the network card and corresponding to the DMA write descriptor of the second network card. Fragmented data, wherein the first to-be-written data includes n first fragmented data, and the second to-be-written data includes m second fragmented data; the network card in the forwarding device is based on the The first network card DMA write descriptor and the first fragmented data generate a first DMA write command, and the second network card DMA write descriptor and the second fragmented data generate a second DMA write command; the forwarding The network card in the device sends the first DMA write command and the second DMA write command to the queue expansion device; the queue expansion device in the forwarding device according to the first DMA write command and the second DMA Write command to write the first fragmented data and the second fragmented data into the memory of the queue expansion device; in the memory of the queue expansion device in the forwarding device, the amount of the first fragmented data is In the case of n, a third DMA write command is generated according to the first data to be written, and in the case that the number of the second fragmented data is m, according to the second data to be written, a third DMA write command is generated The fourth DMA write command, wherein the third DMA write command is used for the computing device to execute and write the first data to be written into the computing device, and the fourth DMA write command is used for The computing device executes and writes the first data to be written into the computing device; the queue expansion device in the forwarding device The third DMA write command and the fourth DMA write command are sent to the computing device, so that the computing device will transfer the first standby command according to the third DMA write command and the fourth DMA write command. The write data and the second data to be written are written into the computing device.

In the above embodiment, the queue expansion device stores the first piece of data sent by the network card in the memory, and when all n pieces of data of the first data to be written are collected, the first data to be written is sent For the computing device, the data received by the network card is usually fragmented data, thereby avoiding the problem that the computing device receives out-of-sequence of the first fragmented data and causing the computing device to deal with the problem of heavy processing pressure, thereby improving the efficiency of queue communication.

In an embodiment, after storing the first data to be sent and the second data to be sent in the network card queue corresponding to the corresponding queue expansion device queue, the method further includes: in the forwarding device The network card sends a completion message and an interrupt message to the queue expansion device; the queue expansion device in the forwarding device sends the completion message and interrupt message to the computing device, so that the computing device refreshes the first host according to the completion message The tail pointer of the queue and the second host queue.

In a second aspect, a queue communication system is provided, the system includes a forwarding device and a computing device, the forwarding device is connected to the computing device, the forwarding device includes at least one forwarding queue, and the computing device includes multiple host queues , Multiple data to be sent are stored in the multiple host queues, where:

The computing device is configured to send a first command message to the forwarding device, where the first doorbell message carries information about the first host queue and the second host queue, and the first host queue and the all The second host queues all belong to the multiple host queues;

The forwarding device is used to receive the first doorbell message from the computing device;

The forwarding device is configured to obtain the information of the first host queue and the information of the second host queue from the first doorbell message, obtain the first data to be sent from the first host queue, and obtain the first data to be sent from the first host queue. Acquiring the second to-be-sent data from the second host queue;

The forwarding device is configured to store the first data to be sent and the second data to be sent into the first host queue and the first host queue according to the queue correspondence between the forwarding queue and the host queue. Two forwarding queues corresponding to the host queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.

In an embodiment, the computing device is configured to send a device query message to the forwarding device before sending the first doorbell message to the forwarding device; the forwarding device is configured to receive a device query from the computing device Message; the forwarding device is used to report the number of extended forwarding queues to the computing device in response to the device query message, wherein the number of extended forwarding queues is determined by the number of the at least one forwarding queue The number and expansion coefficient are determined, and the expanded number of forwarding queues is used to establish the queue correspondence relationship.

In one embodiment, the forwarding device includes an attribute register, and the number of the expanded forwarding queues is stored in the attribute register; the forwarding device is used to query the attribute register and obtain it from the attribute register The number of the extended forwarding queues is reported to the computing device.

In an embodiment, the information of the first host queue includes a first head pointer, the first head pointer indicates the first to-be-sent data, and the information about the second host queue includes a second head pointer, so The second head pointer indicates the second data to be sent, and multiple sets of head pointers and multiple sets of tail pointers are stored in the forwarding device, and each set of head pointers and each set of tail pointers respectively correspond to multiple hosts of the computing device The head pointer and the tail pointer of a host queue in the queue, the forwarding device also stores at least one set of head pointers and at least one set of tail pointers of the at least one forwarding queue; Acquire the first head pointer and the second head pointer in a doorbell message; the forwarding device is configured to refresh the head pointers corresponding to the first host queue in the multiple sets of head pointers according to the first head pointers The forwarding device is used to obtain the first host queue from the first host queue according to the head pointer corresponding to the first host queue after being refreshed, and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers The first data to be sent; the forwarding device is used to refresh the head pointer corresponding to the second host queue in the multiple sets of head pointers according to the second head pointer; the forwarding device is used to refresh the head pointer according to the refreshed The head pointer corresponding to the second host queue, and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers, acquire the second data to be sent in the second host queue; the forwarding device uses After the first data to be sent and the second data to be sent are respectively stored in the first host queue and the corresponding forwarding queue of the second host queue, the tail pointer of the first host queue and the The tail pointer of the second host queue; the forwarding device is used to refresh the head pointer of the corresponding forwarding queue; the forwarding device is used to combine the refreshed tail pointer of the first host queue and the refreshed The tail pointer of the second host queue is sent to the computing device.

In an embodiment, the forwarding device includes a queue expansion device and a network card, the at least one forwarding queue includes at least one queue expansion device queue and at least one network card queue, and the at least one queue expansion device queue is located in the queue expansion device. The at least one network card queue is located in the network card, the at least one set of head pointers and the at least one set of tail pointers are stored in the queue expansion device, and the queue correspondence relationship includes a first queue correspondence relationship and a second queue Correspondence, the first queue correspondence is the correspondence between the multiple host queues and the at least one queue expansion device queue, and the second queue correspondence is the at least one network card queue and the at least one queue The corresponding relationship between the expansion device queues, the at least one network card queue corresponds to the at least one queue expansion device queue, and the queue expansion device stores multiple sets of head pointers and multiple sets of tail pointers, each set of head pointers and each set of tail pointers The group tail pointers respectively correspond to the head pointer and the tail pointer of a host queue in the plurality of host queues. The queue expansion device also stores at least one set of queue expansion device head pointers and at least one set of queue expansion device tails. Pointers, the network card stores at least one set of network card head pointers and at least one set of network card tail pointers, the at least one set of queue expansion device head pointers corresponds to the at least one set of network card head pointers, and the at least one The tail pointer of the group queue expansion device corresponds to the at least one group of network card tail pointers one-to-one; the queue expansion device is configured to obtain the first head pointer and the second head pointer from the first doorbell message; The queue expansion device is configured to refresh the head pointer corresponding to the first host queue in the multiple sets of head pointers according to the first head pointer and the first queue correspondence; the queue expansion device is configured to Acquiring the first data to be sent from the first host queue according to the head pointer corresponding to the first host queue after the refresh and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers; The queue expansion device is configured to refresh the head pointer corresponding to the second host queue in the plurality of sets of head pointers according to the second head pointer and the corresponding relationship of the first queue; the queue expansion device is configured to Acquiring the second data to be sent from the second host queue according to the head pointer corresponding to the second host queue after the refresh and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers; The queue expansion device is configured to store the first data to be sent and the second data to be sent in the queue expansion device corresponding to the first host queue and the second host queue according to the corresponding relationship of the first queue Queue, wherein the corresponding queue expansion device queue belongs to the queue expansion device queue, and the first host queue and the second host queue correspond to the same queue expansion device queue in the at least one queue expansion device queue The queue expansion device is used to refresh the head pointer of the corresponding queue expansion device queue, the tail pointer of the first host queue, and the tail pointer of the second host queue; the network card is used to receive from the queue extension equipment The second doorbell message, wherein the second doorbell message carries the head pointer of the corresponding queue expansion device queue; the network card is used to obtain the corresponding queue expansion device queue information from the second doorbell message Head pointer; the network card in the forwarding device refreshes the corresponding queue expansion device queue in the at least one set of network card head pointers according to the head pointer of the corresponding queue expansion device queue and the second queue correspondence The head pointer of the network card corresponds to the head pointer of the network card; the network card is used for refreshing the corresponding head pointer of the network card, and the tail pointer corresponding to the corresponding queue expansion device queue in the at least one set of network card tail pointers, in all The first data to be sent and the second data to be sent are acquired from the corresponding queue expansion device queue; the network card is used to combine the first data to be sent and the second data to be sent according to the correspondence between the second queues The data is stored in the network card queue corresponding to the corresponding queue expansion device queue.

In one embodiment, the first data to be sent is a first direct memory access DMA read descriptor, the second data to be sent is a second DMA read descriptor, and the network card is also used to After the first data to be sent and the second data to be sent are stored in the network card queue corresponding to the corresponding queue expansion device queue, they are generated according to the first DMA read descriptor and the second DMA read descriptor The first DMA read command and the second DMA read command; the network card is used to send the first DMA read command and the second DMA read command to the queue expansion device in the forwarding device; the queue expansion device is used for The first DMA read command obtains first data to be read from the computing device, and obtains second data to be read from the computing device according to the second DMA read command; the queue expansion device is used to The first data to be read and the second data to be read are forwarded to the network card in the forwarding device.

In an embodiment, the first data to be sent is a first DMA write descriptor, and the second data to be sent is a second DMA write descriptor, wherein the first DMA write descriptor is used to transfer the first DMA write descriptor. A data to be written is written from the queue expansion device to the computing device, the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device, the queue The expansion device is used to convert the first DMA write descriptor to a first network card DMA write descriptor, and convert the second DMA write descriptor to a second network card DMA write descriptor, wherein the first DMA write descriptor The descriptor is used to write the first data to be written from the network card into the queue expansion device, and the second DMA descriptor is used to write the second data to be written from the network card into the queue expansion device The queue expansion device is used to store the first network card DMA write descriptor and the second network card DMA write descriptor into the corresponding queue expansion device queue; the network card is also used in the After the first data to be sent and the second data to be sent are stored in the network card queue corresponding to the corresponding queue expansion device queue, the data stored in the network card is obtained according to the DMA write descriptor of the first network card, and According to the first fragment data corresponding to the DMA write descriptor of the first network card, according to the DMA write descriptor of the second network card, obtain the first fragment data stored in the network card and corresponding to the DMA write descriptor of the second network card. Two pieces of data, wherein the first data to be written includes n pieces of first piece data, and the second data to be written includes m pieces of second piece data; the network card is configured to A network card DMA write descriptor and the first fragment data generate a first DMA write command, and a second DMA write command is generated according to the second network card DMA write descriptor and the second fragment data; the network card uses Sending the first DMA write command and the second DMA write command to the queue expansion device; the queue expansion device is configured to transfer all data according to the first DMA write command and the second DMA write command The first fragmented data and the second fragmented data are written into the memory of the queue expansion device; the queue expansion device is used to store the first fragmented data in the memory of the queue expansion device in the forwarding device When the number is n, a third DMA write command is generated according to the first data to be written, and when the number of the second fragmented data is m, according to the second data to be written , Generate a fourth DMA write command, where the third DMA write command is used for the computing device to execute and write the first data to be written into the computing device, and the fourth DMA write command is used for For the computing device to execute and write the first data to be written into the computing device; the queue expansion device is used to send the third DMA write command and the fourth DMA to the computing device Write command; the computing device is used to write the first data to be written and the second to be written according to the third DMA write command and the fourth DMA write command Write data into the computing device.

In one embodiment, the network card is further configured to send the first data to be sent and the second data to be sent to the network card queue corresponding to the corresponding queue expansion device queue after storing the first data to be sent and the second data to be sent to the queue The expansion device sends a completion message and an interrupt message; the queue expansion device is also used to send the completion message and interrupt to the computing device, so that the computing device refreshes the first host queue and the second host according to the completion message The tail pointer of the queue.

In a third aspect, a forwarding device is provided, the forwarding device is connected to a computing device, the forwarding device includes at least one forwarding queue, the computing device includes multiple host queues, and how many host queues are stored in the multiple host queues. Pieces of data to be sent, wherein the forwarding device includes:

A receiving unit, the receiving unit is configured to receive a first doorbell message from the computing device, wherein the first doorbell message carries information about a first host queue and a second host queue, and the first Both the host queue and the second host queue belong to the multiple host queues;

An acquiring unit configured to acquire information about the first host queue and information about the second host queue from the first doorbell message, and acquire first data to be sent from the first host queue , Acquiring second data to be sent from the second host queue;

A storage unit, configured to store the first data to be sent and the second data to be sent in the first host respectively according to the queue correspondence between the forwarding queue and the host queue The queue and the forwarding queue corresponding to the second host queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.

In an embodiment, the forwarding device further includes a response unit, and the receiving unit is further configured to receive a device query message from the computing device before the receiving unit receives the first doorbell message from the computing device The response unit is used to respond to the device query message, and the forwarding device reports the number of the extended forwarding queues to the computing device, wherein the number of the extended forwarding queues is determined by the at least one The number of forwarding queues and the expansion coefficient are determined, and the expanded number of forwarding queues is used to establish the queue correspondence.

In one embodiment, the forwarding device includes an attribute register, the number of the expanded forwarding queues is stored in the attribute register, and the response unit is used to query the attribute register and obtain it from the attribute register. The number of the extended forwarding queues is reported to the computing device.

In an embodiment, the information of the first host queue includes a first head pointer, the first head pointer indicates the first to-be-sent data, and the information about the second host queue includes a second head pointer, so The second head pointer indicates the second data to be sent, and multiple sets of head pointers and multiple sets of tail pointers are stored in the forwarding device, and each set of head pointers and each set of tail pointers respectively correspond to multiple queues of the computing device The forwarding device also stores at least one set of head pointers and at least one set of tail pointers of the at least one queue; the forwarding device further includes a refreshing unit, and the acquiring unit Is used to obtain the first head pointer and the second head pointer from the first doorbell message; the refresh unit is used to refresh the first head pointer in the plurality of sets of head pointers according to the first head pointer The head pointer corresponding to the host queue; the acquiring unit is used to obtain the head pointer corresponding to the first host queue after being refreshed, and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers. The first data to be sent is acquired from a host queue; the refresh unit is configured to refresh the head pointer corresponding to the second host queue in the plurality of sets of head pointers according to the second head pointer; the acquisition unit uses According to the head pointer corresponding to the second host queue after being refreshed, and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers, the second data to be sent is acquired from the second host queue The refresh unit is used to refresh the first host queue after storing the first data to be sent and the second data to be sent in the corresponding forwarding queues of the first host queue and the second host queue respectively The tail pointer and the tail pointer of the second host queue; the refresh unit is used to refresh the head pointer of the corresponding forwarding queue; the refresh unit is used to refresh the tail pointer of the first host queue and The refreshed tail pointer of the second host queue is sent to the computing device.

In a fourth aspect, a queue expansion device is provided. The queue expansion device is connected to a computing device and connected to a network card, the queue expansion device is connected to a computing device, the network card is connected to the computing device, and the queue expands The device includes at least one queue expansion device queue, the network card includes at least one network card queue, the computing device includes a plurality of host queues, a plurality of data to be sent are stored in the plurality of host queues, and the queue expansion device includes:

A storage unit, which is configured to store the first data to be sent and the second data to be sent in the storage unit according to the first queue correspondence between the queue expansion device queue and the host queue. The first host queue and the queue expansion device queue corresponding to the second host queue;

A sending unit, the sending unit is configured to send a second doorbell message to the network card, the second doorbell message carrying the corresponding queue expansion device queue information, so that the network card according to the second doorbell message , Acquiring the first data to be sent and the second data to be sent from the corresponding queue expansion device queue, and according to the second queue correspondence between the network card queue and the queue expansion device queue, The first data to be sent and the second data to be sent are stored in the network card queue corresponding to the corresponding queue expansion device queue, where the corresponding queue expansion device queue belongs to the queue expansion device queue, and the The first host queue and the second host queue correspond to the same queue expansion device queue in the at least one queue expansion device queue, and the corresponding network card queue belongs to the network card queue.

In an embodiment, the device further includes a response unit, and the receiving unit is further configured to receive a first device query message from the computing device before receiving the first doorbell message from the computing device; The sending unit is used to send a second device query message to the network card; the response unit is used to receive the number of network card queues sent by the network card, wherein the number of network card queues is used to establish the second queue correspondence The response unit is configured to report the number of expanded queue expansion device queues to the computing device in response to the first device query message, wherein the number of expanded queue expansion device queues is determined by the The number of network card queues and the expansion coefficient are determined, and the expanded number of queue expansion device queues is used to establish the first queue correspondence relationship.

In an embodiment, the device includes an attribute register, and the number of the expanded forwarding queues is stored in the attribute register; the response unit is also used to query the attribute register and obtain it from the attribute register The number of queues of the expanded queue expansion device is reported to the computing device.

In an embodiment, the information of the first host queue includes a first head pointer, the first head pointer indicates the first to-be-sent data, and the information about the second host queue includes a second head pointer, so The second head pointer indicates the second data to be sent, and the queue expansion device stores multiple sets of head pointers and multiple sets of tail pointers, and each set of head pointers and each set of tail pointers respectively correspond to the multiple host queues The head pointer and the tail pointer of a host queue in, the queue expansion device also stores at least one set of queue expansion device head pointers and at least one set of queue expansion device tail pointers, and at least one set of network cards is stored in the network card A head pointer and at least one set of network card tail pointers, the at least one set of queue extension device head pointers correspond to the at least one set of network card head pointers, and the at least one set of queue extension device tail pointers corresponds to the at least one There is a one-to-one correspondence between the tail pointers of the network card, and the queue expansion device further includes a refreshing unit, and the acquiring unit is configured to acquire the first head pointer and the second head pointer from the first doorbell message; the refreshing The unit is used to refresh the head pointer corresponding to the first host queue in the multiple sets of head pointers according to the first head pointer and the corresponding relationship of the first queue; the acquisition unit is used to refresh the head pointer corresponding to the first host queue according to the refresh The head pointer corresponding to the first host queue, and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers, acquire the first data to be sent in the first host queue; the refresh unit uses According to the second head pointer and the corresponding relationship of the first queue, the head pointer corresponding to the second host queue among the multiple sets of head pointers is refreshed; the acquiring unit is configured to refresh the second head pointer according to the refreshed second head pointer. The head pointer corresponding to the host queue, and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers, acquire the second to-be-sent data in the second host queue; the storage unit is used to obtain the second data to be sent according to In the first queue correspondence, the first data to be sent and the second data to be sent are stored in the first host queue and the queue expansion device queue corresponding to the second host queue, wherein the corresponding The queue expansion device queue belongs to the queue expansion device queue, and the first host queue and the second host queue correspond to the same queue expansion device queue in the at least one queue expansion device queue; the refresh unit is used for refreshing The head pointer of the corresponding queue expansion device queue, the tail pointer of the first host queue, and the tail pointer of the second host queue; the sending unit is further configured to send the second doorbell message to the network card, wherein The second doorbell message carries the head pointer of the corresponding queue expansion device queue, so that the network card refreshes the at least one queue according to the head pointer of the corresponding queue expansion device queue and the second queue correspondence relationship. The network card head pointer corresponding to the head pointer of the corresponding queue expansion device queue in the network card head pointer, and the corresponding queue expansion device queue in the corresponding network card head pointer and the at least one set of network card tail pointers after refreshing Corresponding tail pointer, obtain the first to-be-sent in the corresponding queue expansion device queue The data and the second data to be sent are stored in the network card queue corresponding to the corresponding queue expansion device queue according to the corresponding relationship of the second queue.

In an embodiment, the first data to be sent is a first direct memory access DMA read descriptor, the second data to be sent is a second DMA read descriptor, and the receiving unit is further configured to receive the The first DMA read command and the second DMA read command sent by the network card, wherein the first DMA read command is generated by the network card according to the first DMA descriptor, and the second DMA read command is The network card is generated according to the second DMA descriptor; the obtaining unit is further configured to obtain the first data to be read from the computing device according to the first DMA read command, and from the computing device according to the second DMA read command Obtain second data to be read; the sending unit is further configured to forward the first data to be read and the second data to be read to the network card.

In an embodiment, the first data to be sent is a first DMA write descriptor, and the second data to be sent is a second DMA write descriptor, wherein the first DMA write descriptor is used to transfer the first DMA write descriptor. A data to be written is written from the queue expansion device to the computing device, and the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device, and the device It also includes a conversion unit configured to convert the first DMA write descriptor to a first network card DMA write descriptor, and convert the second DMA write descriptor to a second network card DMA write descriptor, wherein , The first DMA write descriptor is used to write the first data to be written from the network card to the queue expansion device, and the second DMA descriptor is used to write the second data to be written from the network card Write to the queue expansion device; the storage unit is used to store the first network card DMA write descriptor and the second network card DMA write descriptor into the corresponding queue expansion device queue; the receiving unit uses When receiving the first DMA write command and the second DMA write command, the first DMA write command is the network card obtained by the network card according to the first network card DMA write descriptor and stored in the network card, and The first fragmented data corresponding to the first network card DMA write descriptor, the first DMA write command generated according to the first network card DMA write descriptor and the first fragmented data, the second DMA write command After the network card obtains the second fragmented data corresponding to the second network card DMA write descriptor stored in the network card according to the second network card DMA write descriptor, and then writes according to the second network card DMA write descriptor Descriptor and the second DMA write command generated by the second fragmented data, wherein the first to-be-written data includes n first fragmented data, and the second to-be-written data includes m second fragments. Fragmented data; the storage unit is used to write the first fragmented data and the second fragmented data into the memory of the queue expansion device according to the first DMA write command and the second DMA write command The conversion unit is used to generate a third DMA write command based on the first data to be written when the number of first fragmented data is n in the memory of the queue expansion device In the case where the number of the second fragmented data is m, a fourth DMA write command is generated according to the second data to be written, where the third DMA write command is used for the computing device to execute and Writing the first data to be written into the computing device, and the fourth DMA write command is used for execution by the computing device and writing the first data to be written into the computing device; The sending unit is configured to send the third DMA write command and the fourth DMA write command to the computing device, so that the computing device transfers all data according to the third DMA write command and the fourth DMA write command. The first data to be written and the second data to be written are written into the computing device.

In an embodiment, the receiving unit is further configured to receive a completion message and an interrupt sent by the network card, wherein, the completion message and the interrupt indicate that the network card is sending the first data to be sent and the second The data to be sent is generated after being stored in the network card queue corresponding to the corresponding queue expansion device queue; the sending unit is also used to send a completion message and an interrupt to the computing device, so that the computing device according to the completion message Refresh the tail pointers of the first host queue and the second host queue.

In a fifth aspect, a forwarding device is provided, the device includes a processor and a memory, and the processor executes the code in the memory to implement the method described in the first aspect.

In a sixth aspect, a computer program product is provided. When the computer program product is read and executed by a computing device, the method described in the first aspect is implemented.

In a seventh aspect, a computer non-transitory storage medium is provided, including instructions, which when run on a computing device, cause the computing device to implement the method described in the first aspect.

In an eighth aspect, a forwarding device is provided, the device includes a logic circuit, and the forwarding device implements the method described in the first aspect through the logic circuit.

In a ninth aspect, a queue expansion device is provided, the device includes a processor and a memory, and the processor executes the code in the memory to implement the queue expansion device described in the fourth aspect.

In a tenth aspect, a computer program product is provided, which is characterized in that, when the computer program product is read and executed by a computing device, the queue expansion device described in the fourth aspect is implemented.

In an eleventh aspect, a computer non-transitory storage medium is provided, which is characterized in that it includes an instruction that, when the instruction runs on a computing device, causes the computing device to implement the queue expansion device described in the fourth aspect.

In a twelfth aspect, a transfer queue expansion device is provided, the device includes a logic circuit, and the forwarding device implements the queue expansion device described in the fourth aspect through the logic circuit.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art.

Figure 1 is a schematic diagram of the logical structure of a network system to which the solution provided in this application is applicable;

FIG. 2 is a schematic diagram of the logical structure of a forwarding device for queue communication in a network system provided by this application;

3 is a schematic diagram of the logical structure of a queue expansion device for queue communication in a network system provided by the present application;

Figure 4 is a flow chart of the steps of a queue communication method provided by the present application;

5A-5C are schematic diagrams of a queue structure;

Figure 6 is a flow chart of the steps of another queue communication method provided by the present application;

FIG. 7 is a flowchart of the steps of obtaining the first data to be sent and the second data to be sent according to the doorbell message in a queue communication method provided by the present application;

FIG. 8 is a flow chart of the steps of a queue communication method provided by the present application in an application scenario;

FIG. 9 is a flow chart of the steps of a queue communication method provided by the present application in another application scenario;

FIG. 10 is a schematic structural diagram of a forwarding device provided by this application;

FIG. 11 is a schematic structural diagram of a queue expansion device provided by the present application;

FIG. 12 is a schematic diagram of the hardware structure of a forwarding device provided by the present application;

FIG. 13 is a schematic diagram of the hardware structure of a queue expansion device provided by the present application.

Detailed ways

The terminology used in the implementation mode part of this application is only used to explain the specific embodiments of this application, and is not intended to limit this application.

The solution provided in this application is applicable to a network system that can implement data communication between computing devices. As shown in FIG. 1, FIG. 1 is a schematic diagram of the logical structure of a network system applicable to this application. The network system includes: a computing device 110 and a computing device 120, a forwarding device 210 and a forwarding device 220, and a network 300, wherein, The computing device 110 and the forwarding device 210 are connected and communicated, the forwarding device 210 is connected and communicated with the network 300, the computing device 120 is connected and communicated with the forwarding device 220, and the forwarding device 220 is connected and communicated with the network 300. It should be noted that for ease of understanding, FIG. 1 only shows two computing devices and two forwarding devices, and each computing device is connected to only one forwarding device. In actual situations, the network system may include one or more computing devices and The number of one or more forwarding devices, computing devices, and forwarding devices is not a limitation to the embodiments of this application. Each computing device can also be connected to multiple forwarding devices. The system can make adjustments according to user needs as described above. This application does not Make specific restrictions. Moreover, Figure 1 is only for the convenience of description and only shows part of the resources. In specific implementation, the computing device shown in Figure 1 may also include hardware resources such as memory and hard disk, as well as software resources such as operating systems and applications that need to be run. Go into details again.

Wherein, when the computing device 110 needs to transmit data to be sent to the computing device 120, the data to be sent on the computing device 110 side can be sent to the forwarding device 210, and the network card 210 sends to the network card 220 through the network 300 connected to it. For the data to be sent, the forwarding device 220 may receive the data to be sent through the network 300 and send the data to be sent to the computing device 120, so that the computing device 110 sends the data to be sent to the computing device 120. This application mainly focuses on the communication process between the computing device and the forwarding device, so the process of data transmission by the forwarding device through the network will not be repeated. The process of implementing the communication between the computing device and the forwarding device in this application will be explained in detail below.

In the embodiment of the present application, the forwarding device may be a network card. In a specific implementation, the communication system shown in FIG. 1 may be as shown in FIG. 2, where the computing device 110 includes a plurality of host queues 111 (shown in the figure 4 host queues Q00-Q03), the forwarding device 210 includes at least one forwarding queue 211 (only two forwarding queues Q10 and Q11 are drawn in the figure), and the at least one forwarding queue 211 and the multiple hosts There is a queue correspondence between the queues 111. Among them, one forwarding queue 211 may have a queue correspondence with multiple host queues 211. For example, there is a queue between the host queues Q00-Q01 and the forwarding queue Q10 shown in FIG. 2 Correspondence, there is a queue correspondence between the host queues Q02-Q03 and the forwarding queue Q11. It should be noted that since the queue communication process between the forwarding device 220 and the computing device 120 is similar to the queue communication process between the computing device 110 and the forwarding device 210, FIG. 1 only shows the computing device 110 and the forwarding device for ease of understanding. 210, I won't repeat it here.

Then, when the computing device 110 needs to transmit the data to be sent to the forwarding device 210, it can first put the data to be sent into the host queue, such as in the host queues Q00 and Q01 in FIG. 2, and then according to the queue correspondence, The forwarding queue Q10 corresponding to the host queues Q00 and Q01 is determined, and the data to be sent is transferred from the host queues Q00 and Q01 to the forwarding queue Q10, so as to transmit the data to be sent from the computing device 110 to the forwarding device 210. It is understandable that, in the forwarding queue in the forwarding device provided in this application, each forwarding queue can generate a queue correspondence with multiple host queues, thereby avoiding the problem that the forwarding ability of the host queue is weaker than the forwarding ability of the network card queue. The communication efficiency is low, so that the forwarding performance of the network card queue can be brought into full play.

In the embodiment of the present application, the forwarding device may also include a queue expansion device and a network card. In a specific implementation, the communication system shown in FIG. 1 may be as shown in FIG. 3, where the computing device 110 includes multiple host queues 111 ( 4 host queues Q00-Q03 are drawn in the figure), the forwarding device 210 includes a queue expansion device 2101 and a network card 2102, wherein the queue expansion device 2010 includes at least one queue expansion device Queue 2111 (only 2 queues are drawn in the figure Expanded device queues Q10 and Q11), the network card 2102 includes at least one network card queue 2112 (only two network card queues Q20 and Q21 are drawn in the figure), and the at least one queue expands the device queue 2111 and the multiple host queues 111 There is a first queue correspondence between them, and there is a second queue correspondence between the at least one network card queue 2112 and the at least one queue expansion device queue 2111. Wherein, one queue expansion device queue 2111 may have a first queue correspondence relationship with multiple host queues 211, for example, there is a queue correspondence relationship between the host queues Q00-Q01 and the queue expansion device queue Q10 shown in FIG. 3; One queue expansion device queue 2111 may have a second queue correspondence relationship with one network card queue 2112. For example, there is a queue correspondence relationship between the queue expansion device queue Q10 and the network card queue Q20 shown in FIG. 3.

Then, when the computing device 110 needs to transmit the data to be sent to the network card 2112, it can first put the data to be sent into the host queue, for example, into the host queues Q00 and Q01 in FIG. 3, and then correspond to the first queue Relationship, determine the queue expansion device queue Q10 corresponding to the host queues Q00 and Q01, so as to transfer the data to be sent from the host queues Q00 and Q01 to the queue expansion device queue Q00; then determine the queue expansion according to the second queue correspondence relationship The network card queue Q20 corresponding to the device queue Q10 transfers the data to be sent from the queue expansion device queue Q10 to the network card queue Q20, thereby transmitting the data to be sent from the computing device 110 to the network card 2112. It is understandable that in the queue expansion device provided in this application, each queue expansion device queue can generate a queue correspondence with multiple host queues, and the expansion device queue and the network card queue have a one-to-one correspondence, so that Without any upgrade of the network card and computing device, the inefficiency of communication caused by the forwarding capacity of the host queue being weaker than the forwarding capacity of the network card queue is avoided, so that the forwarding performance of the network card queue can be fully utilized.

In the embodiment of the present application, the network system shown in FIG. 1 may also be a data center virtualization scenario. In other words, the computing device 110 and the computing device 120 may be physical machines including multiple virtual machines, or may be located in The two virtual machines in the same physical machine may also be two virtual machines located in different physical machines, which are not specifically limited in this application.

The following describes the method for queue communication based on the forwarding device provided by the present application based on the network architecture shown in FIG. 1 to FIG. 2.

As shown in Figure 4, the present application provides a method for queue communication. The forwarding device is connected to a computing device, the forwarding device includes at least one forwarding queue, the computing device 400 includes multiple host queues, and multiple host queues of the computing device store multiple data to be sent, In simple terms, the data to be sent in the multiple host queues of the computing device will be sent to at least one forwarding queue of the forwarding device. The computing device 400 in FIG. 4 may be the computing device 110 and the computing device 120 in FIGS. 1-2, and the method of queue communication includes the following steps:

S101: The forwarding device receives a device query message from the computing device.

In an embodiment, the forwarding device may be a PCIe device. After the forwarding device is connected to the computing device, the computing device needs to identify the forwarding device connected to it through PCIe device enumeration, and then can communicate with the forwarding device.

S102: In response to the device query message, the forwarding device reports the expanded number of forwarding queues to the computing device, thereby establishing a queue correspondence between the forwarding queue and the host queue. Wherein, the number of the extended forwarding queues is determined by the number of the at least one forwarding queue and the expansion coefficient, and the number of the extended forwarding queues is used to establish a queue correspondence relationship.

In a specific implementation, the forwarding device includes an attribute register, and the number of the extended forwarding queues is stored in the attribute register; the forwarding device reporting the number of extended forwarding queues to the computing device includes: The forwarding device queries the attribute register, obtains the expanded number of forwarding queues from the attribute register, and reports the expanded number of forwarding queues to the computing device. Wherein, the expansion factor n may be determined according to the queue processing capability of the forwarding device and the single thread processing capability of the processor of the computing device.

S103: The forwarding device receives a first doorbell message from the computing device, where the first doorbell message carries information about a first host queue and information about a second host queue, and the first host queue and The second host queues all belong to the multiple queues.

S104: The forwarding device obtains the information of the first host queue and the information of the second host queue from the first doorbell message, obtains the first to-be-sent data from the first host queue, and obtains the The second data to be sent is acquired from the second host queue.

S105: The forwarding device stores the first data to be sent and the second data to be sent into the first host queue and the first host queue, respectively, according to the queue correspondence between the forwarding queue and the host queue. Two forwarding queues corresponding to the host queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.

In specific implementation, the host queue and the forwarding queue can be understood as a section of memory. This section of memory will be divided into several memory blocks of equal length, corresponding to a unit of work. Each unit of work is used to store a fixed-length message, such as computing. The data request (read request or write request) initiated by the device to the forwarding device. Therefore, similar to other storage mechanisms, each queue maintained by the computing device and the forwarding device also includes a head pointer (Head) and a tail pointer (Tail). By comparing the positions of the two pointers, the storage status of the queue can be obtained. Specifically, the refresh rule of the head pointer is: point to the next empty position in the queue, and the refresh rule of the tail pointer: point to the location of the next command to be executed. As new commands are added to the queue, the tail pointer keeps moving forward. As the commands are continuously taken out for consumption, the head pointer also keeps moving forward. Therefore, the queue can be logically understood as shown in Figure 5A The circular circular queue shown. However, it should be understood that the actual state of the memory in the queue is a long strip of storage space, as shown in Figure 5B. When the head and tail pointers point to the same position, it means that the queue is currently empty, as shown in Figure 5C. When the next pointer of the tail pointer is the head pointer, that is, when head=Tail+1, the queue is full and no new messages can be received. It should be understood that the above examples are only for illustration and cannot constitute a specific limitation. It can be understood that each host queue in the computing device 400 includes a head and tail pointer. Therefore, the information of the first host queue includes a first head pointer, and the information of the second host queue includes a second head pointer. Wherein, the first head pointer indicates the first data to be sent, and the second head pointer indicates the second data to be sent. For example, as shown in Figure 6, the first data to be sent is located in 4 units A0-A3 in the first host queue, then the first head pointer can point to A3, and the second data to be sent is located in 3 in the second host queue. In each unit B0-B2, the second head pointer can point to B2. Therefore, the first doorbell message sent by the computing device 400 to the forwarding device 300 in step S103 will include the first head pointer A3 and the second head pointer B2.

Correspondingly, the forwarding device 300 also stores multiple sets of head pointers and multiple sets of tail pointers, and each set of head pointers and each set of tail pointers respectively correspond to the head pointers of one of the multiple host queues of the computing device. And tail pointers, the forwarding device also stores at least one set of head pointers and at least one set of tail pointers of the at least one forwarding queue. In specific implementation, as shown in FIG. 6, the forwarding device 300 further includes a doorbell register for storing the head and tail pointer values of the first host queue and the second host queue in the computing device 400, and the value of the forwarding queue in the forwarding device. The value of the head pointer and tail pointer.

Therefore, the specific steps of step S104 may be as follows:

S1041: The forwarding device obtains the first head pointer and the second head pointer from the first doorbell message.

S1042: The forwarding device refreshes the head pointer corresponding to the first host queue in the multiple sets of head pointers according to the first head pointer.

S1043: The forwarding device refreshes the head pointer corresponding to the second host queue in the multiple sets of head pointers according to the second head pointer.

S1044: The forwarding device obtains the first host queue from the first host queue according to the head pointer corresponding to the first host queue after refreshing and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers. The first data to be sent.

S1045: The forwarding device obtains the second host queue from the second host queue according to the head pointer corresponding to the second host queue after the refresh and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers. The second data to be sent.

For example, as shown in FIG. 6, after the forwarding device 300 receives the first doorbell message sent by the computing device 400, first, step S1041 may obtain the first head pointer A3 and the second head pointer A3 from the first doorbell message. Head pointer B2. In step S1042, the value of the first head pointer stored in the doorbell register can also be refreshed to A3 according to the first head pointer A3 acquired in the first doorbell message. Similarly, in step S1043, the value of the first head pointer stored in the doorbell register can be updated to A3. The second head pointer B2 obtained in the message also refreshes the value of the second head pointer stored in the memory in the doorbell register to B2. Finally, the forwarding device obtains the first data to be sent A0-A3 from the first host queue by comparing the refreshed first head pointer A3 and the first tail pointer A0 in the doorbell register, and according to the second head pointer B2 and The value of the second tail pointer B0 obtains the second to-be-sent data B0-B2 from the second host queue.

It is understandable that after the forwarding device obtains the first data to be sent and the second data to be sent, in step S105, it may first obtain the forwarding corresponding to the first host queue and the second host queue according to the queue correspondence established in step S102. queue. Then, the first data to be sent and the second data to be sent are stored in the forwarding queue to complete a data communication.

In an embodiment, after the forwarding device stores the first to-be-sent data and the second to-be-sent data into the forwarding queues corresponding to the first host queue and the second host queue, respectively, the method further includes : Refresh the tail pointer of the first host queue and the tail pointer of the second host queue; refresh the head pointer of the corresponding forwarding queue; refresh the tail pointer of the first host queue and the refreshed tail pointer The tail pointer of the second host queue is sent to the computing device. For example, as shown in FIG. 6, after the forwarding device stores the first data to be sent and the second data to be sent in the forwarding queue C corresponding to the first host queue and the second host queue, the forwarding device may forward the doorbell register The tail pointer of the queue C is refreshed to C7. Since the data in the first host queue and the second host queue have been moved into the forwarding queue C, the first tail pointer of the first host queue is refreshed to A3, and the second host queue is the second The tail pointer is refreshed to B3, and the values of the first tail pointer and the second tail pointer are sent to the computing device, so as to inform the computing device that the first data to be sent and the second data to be sent have been removed by the forwarding device.

In summary, using the queue communication method provided in this application, the forwarding device sends the expanded network card queue number to the computing device, so that the performance of the network card hardware can be fully exploited, and the thread message forwarding rate of the computing device no longer becomes the network card Dealing with the bottleneck of message forwarding, further avoiding resource waste.

The following describes the queue communication method of the queue expansion device provided by the present application based on the system architecture shown in FIG. 3. It can be understood that the forwarding device 300 at this time includes the queue expansion device 500 and the network card 600, and at least one of the forwarding devices 300 The forwarding queue includes at least one queue expansion device queue and at least one network card queue, the at least one queue expansion device queue is located in the queue expansion device 500, and the at least one network card queue is located in the network card 600. The queue correspondence relationship includes a first queue correspondence relationship and a second queue correspondence relationship. The first queue correspondence relationship is a correspondence relationship between the plurality of host queues and the at least one queue extension device queue, and the second queue The correspondence relationship is the correspondence relationship between the at least one network card queue and the at least one queue expansion device queue, and the at least one network card queue corresponds to the at least one queue expansion device queue in a one-to-one correspondence. As shown in Figure 7, the queue communication method includes the following steps:

S201: The computing device 400 sends a device query message to the queue expansion device 500.

S202: The queue expansion device 500 sends a device query message to the network card 600.

S203: The network card 600 reports the number of network card queues to the queue expansion device 500, thereby establishing a second queue correspondence relationship between the queue expansion device and the network card queue. Wherein, the network card queue and the queue expansion device queue have a one-to-one correspondence. In a specific implementation, the queue expansion device 500 includes an attribute register, and the queue expansion device 500 stores the number of network card queues in the attribute register after receiving the number of network card queues of the network card 600.

S204: The queue expansion device 500 reports the number of expanded queues to the computing device 400, so as to establish a first queue correspondence between the queue expansion device queue and the host queue. Wherein, the number of queues after expansion is determined by the number of queues of the network card and the expansion coefficient.

In specific implementation, the number of the expanded queues is stored in the attribute register. Wherein, the queue expansion device 500 may query the attribute register to obtain the number of network card queues from the attribute register, calculate the number of expanded queues according to the expansion coefficient k, and calculate the number of expanded queues. The number is reported to the computing device. The expansion coefficient k may be determined according to the message forwarding rate a of the network card and the message forwarding rate b of a single thread of the processor of the computing device. specifically,

For example, suppose that the message forwarding rate of the network card is a=2Mpps, and the message forwarding rate of a single thread of the processor of the computing device is b=0.5Mpps, then the expansion ratio k≤4 at this time.

S205: The queue expansion device 500 receives a first doorbell message from the computing device 400, where the first doorbell message carries information about the first host queue and information about the second host queue, and the first Both the host queue and the second host queue belong to the multiple host queues.

S206: The queue expansion device 500 obtains the information of the first host queue and the information of the second host queue from the first doorbell message, and obtains the first data to be sent from the first host queue. Obtain the second to-be-sent data from the second host queue.

S207: The queue expansion device 500 stores the first to-be-sent data and the second to-be-sent data into the queue of the queue expansion device respectively according to the correspondence between the first queue and the first host queue and the second queue. The queue expansion device queue corresponding to the second host queue, so that the data to be sent in the first host queue and the second host queue in the computing device are stored in the queue expansion device queue in the queue expansion device 500.

S208: The network card 600 receives a second doorbell message from the queue expansion device 500, where the second doorbell message carries information about a corresponding queue expansion device queue, and the corresponding queue expansion device queue belongs to the description. The queue expands the device queue.

S209: The network card 600 obtains the information of the corresponding queue expansion device queue from the second doorbell message, and obtains the first data to be sent and the second data to be sent from the corresponding queue expansion device queue .

S210: The network card 600 stores the first to-be-sent data and the second to-be-sent data into the network card corresponding to the corresponding queue expansion device queue in the network card queue according to the second queue correspondence Queue, so as to store the data to be sent in the host queue in the computing device into the network card queue in the network card 600.

With reference to the content of FIGS. 5A-5C, it can be seen that each host queue in the computing device 400 includes a head and tail pointer, and each network card queue in the network card 600 includes a head and tail pointer. Therefore, correspondingly, the at least one set of The head pointer and the at least one set of tail pointers are stored in the queue expansion device 500, the network card stores at least one set of network card head pointers and at least one set of network card tail pointers, and the at least one set of queue expansion device head pointers 500 corresponds to the at least one set of network card head pointers, and the at least one set of queue expansion device tail pointers corresponds to the at least one set of network card tail pointers. In a specific implementation, as shown in FIG. 8, the queue expansion device 500 includes a first doorbell register for storing the head and tail pointer values of the first host queue and the second host queue in the computing device 400, and the queue expansion device The value of the head pointer and tail pointer of the forwarding queue in 500. The network card 600 includes a second doorbell register for storing the head pointer and tail pointer values of the corresponding queue expansion device queue in the queue expansion device 500, and the head pointer and tail pointer values of the network card queue in the network card 600.

Therefore, referring to the specific process of obtaining the first data to be sent and the second data to be sent from the first doorbell information in the embodiment of FIG. 4, in step S206, the queue expansion device obtains the first data to be sent from the first doorbell information and The specific steps of the second data to be sent may be:

S2061: The queue expansion device obtains the first head pointer and the second head pointer from the first doorbell message;

S2062: The queue expansion device refreshes the head pointer of the queue expansion queue corresponding to the first host queue among the sets of head pointers according to the first head pointer and the corresponding relationship of the first queue;

S2063: The queue expansion device obtains all data from the first host queue according to the head pointer corresponding to the first host queue after refreshing and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers. The first data to be sent;

S2064: The queue expansion device refreshes the head pointer corresponding to the second host queue in the multiple sets of head pointers according to the second head pointer and the corresponding relationship of the first queue;

S2065: The queue expansion device obtains all data from the second host queue according to the head pointer corresponding to the second host queue after refreshing and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers. The second data to be sent.

For the specific details of the above steps S2061-S2065, reference may be made to the process in which the forwarding device obtains the first data to be sent and the second data to be sent from the first doorbell information in the embodiment of FIG. 6, which will not be repeated here.

It is understandable that after the queue expansion device obtains the first data to be sent and the second data to be sent in step S206, it will determine the queue expansion device queues corresponding to the first host queue and the second host queue according to the corresponding relationship of the first queue. , That is, the corresponding queue expansion device queue, and then the first data to be sent and the second data to be sent are stored in the corresponding queue expansion device queue, and the head pointer of the corresponding queue expansion device queue is refreshed and sent to the network card 600 The second doorbell information including the third head pointer causes the network card to repeat the above steps. According to the third head pointer in the second doorbell information, obtain the first data to be sent and the second data to be sent from the corresponding queue expansion device queue According to the corresponding relationship of the second queue, the network card queue corresponding to the corresponding queue expansion device queue is determined, that is, the corresponding network card queue, and the first data to be sent and the second data to be sent are stored in the corresponding network card queue. That is to say, that the forwarding device acquires the information of the first host queue and the information of the second host queue from the first doorbell message includes: the queue expansion device in the forwarding device obtains information from the first host queue. Obtaining the first head pointer and the second head pointer from a doorbell message; the forwarding device obtaining the first to-be-sent data from the first host queue includes: the queue expansion device in the forwarding device according to The first head pointer and the first queue correspondence relationship, refresh the head pointer corresponding to the first host queue in the multiple sets of head pointers; the queue expansion device in the forwarding device is based on the refreshed first A head pointer corresponding to a host queue, and a tail pointer corresponding to the first host queue in the multiple sets of tail pointers, acquire the first data to be sent in the first host queue; the forwarding device obtains the first data to be sent from the first host queue; Obtaining the second data to be sent from the second host queue includes: the queue expansion device in the forwarding device refreshes the data in the multiple sets of head pointers according to the second head pointer and the corresponding relationship of the first queue. The head pointer corresponding to the second host queue; the queue expansion device in the forwarding device is based on the head pointer corresponding to the second host queue after being refreshed, and the tail corresponding to the second host queue in the multiple sets of tail pointers Pointer to obtain the second data to be sent in the second host queue; the forwarding device stores the first data to be sent and the second data to be sent into the first host queue and the first host queue respectively The forwarding queue corresponding to the second host queue includes: the queue expansion device in the forwarding device stores the first to-be-sent data and the second to-be-sent data in the first host according to the first queue correspondence relationship The queue and the queue expansion device queue corresponding to the second host queue, wherein the corresponding queue expansion device queue belongs to the queue expansion device queue, and the first host queue and the second host queue correspond to the at least one queue The same queue in the expansion device queue expands the device queue; the queue expansion device in the forwarding device refreshes the head pointer of the corresponding queue expansion device queue, the tail pointer of the first host queue, and the tail of the second host queue Pointer; the network card in the forwarding device receives a second doorbell message from the queue expansion device, wherein the second doorbell message carries the head pointer of the corresponding queue expansion device queue; in the forwarding device The network card obtains the head pointer of the corresponding queue expansion device queue from the second doorbell message; the network card in the forwarding device expands the device queue head pointer according to the corresponding queue and the second queue correspondence relationship , Refresh the network card head pointer corresponding to the head pointer of the corresponding queue expansion device queue in the at least one set of network card head pointers; the network card in the forwarding device is refreshed according to the corresponding network card head pointer, and the at least The tail pointer corresponding to the corresponding queue expansion device queue in a set of network card tail pointers, and the first data to be sent and the second data to be sent are acquired in the corresponding queue expansion device queue; in the forwarding device According to the second Queue correspondence relationship, storing the first to-be-sent data and the second to-be-sent data in a network card queue corresponding to the corresponding queue expansion device queue, wherein the corresponding network card queue belongs to the network card queue. It should be understood that, for the specific details of the foregoing process, reference may be made to the foregoing embodiments in FIG. 4 and FIG. 6, and details are not described herein again.

In an embodiment, the first data to be sent may be a first Direct Memory Access (DMA) read descriptor, which is used to send the first data to be read at the computing device to the Network card, the second data to be sent may be a second DMA read descriptor, which is used to send the second data to be read from the computing device to the network card. With reference to the foregoing content, we can see that the actual form of a queue is actually a segment divided into several equal-length memory blocks, corresponding to a unit of work, and each unit of work is used to store a fixed-length message, that is, the size of each unit of work The storage capacity is very small. If the computing device currently needs to send the first data to be read to the network card, the DMA read descriptor of the first data to be read can be used as the data to be sent and placed in the host queue. Among them, the DMA read descriptor may include the read address of the first data to be read, the length of the data packet, and description information, etc., and after steps S201-S210, the network card may receive the DMA descriptor sent by the computing device. Generate a DMA read command that directly reads the first data to be read from the computing device.

In a specific implementation, the first data to be sent is a first direct memory access DMA read descriptor, the second data to be sent is a second DMA read descriptor, and the first data to be sent is combined with the first data to be sent. 2. After the data to be sent is stored in the network card queue corresponding to the corresponding queue expansion device queue, the method further includes: the network card in the forwarding device reads the descriptor according to the first DMA read descriptor and the second DMA read Descriptor, generating a first DMA read command and a second DMA read command; the network card in the forwarding device sends the first DMA read command and the second DMA read command to the queue expansion device in the forwarding device; The queue expansion device in the forwarding device obtains the first data to be read from the computing device according to the first DMA read command, and obtains the second data to be read from the computing device according to the second DMA read command; The queue expansion device in the forwarding device forwards the first data to be read and the second data to be read to the network card in the forwarding device.

Second data to be read second data to be read second data to be read In one embodiment, the first data to be sent is a first DMA write descriptor, and the second data to be sent is a second DMA Write descriptor, wherein the first DMA write descriptor is used to write the first data to be written from the queue expansion device to the computing device, and the second DMA descriptor is used to write the second data to be written to the computing device. Incoming data is written from the queue expansion device to the computing device. The difference between the second data to be written and the above-mentioned DMA read descriptor is that the data received by the network card is usually fragmented data. For example, the first data to be written can be transferred to n first fragmented data. For the network card, if the network card sends the first fragmented data to the computing device every time it receives the first fragmented data, the computing device will receive a lot of out-of-order first fragmented data, which increases the processing pressure of the computing device. Therefore, the queue expansion device also includes memory, such as Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM). In specific implementation, the queue expansion device can store the first fragment data received from the network card in the DDR, and when all n fragments of the first data to be written are collected, then the first data to be written Incoming data is sent to the computing device. Therefore, at step S207, the queue expansion device needs to convert the received DMA write descriptor used for writing to the computing device into a network card DMA write descriptor written to the queue expansion device.

In a specific implementation, the first data to be sent is a first DMA write descriptor, the second data to be sent is a second DMA write descriptor, and the first DMA write descriptor is used to transfer the first DMA write descriptor. The write data is written from the queue expansion device to the computing device, the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device, and the The storage of the first data to be sent and the second data to be sent in the first host queue and the queue expansion device queue corresponding to the second host queue includes: the queue expansion device in the forwarding device writes the first DMA The descriptor is converted into a first network card DMA write descriptor, and the second DMA write descriptor is converted into a second network card DMA write descriptor, where the first DMA write descriptor is used to transfer the first data to be written The queue expansion device is written from the network card, and the second DMA descriptor is used to write the second data to be written from the network card into the queue expansion device; the queue expansion device in the forwarding device will The first network card DMA write descriptor and the second network card DMA write descriptor are stored in the corresponding queue expansion device queue; the first data to be sent and the second data to be sent are stored in the After the corresponding queue expands the network card queue corresponding to the device queue, the method further includes: the network card in the forwarding device obtains, according to the first network card DMA write descriptor, the data stored in the network card and the first network card According to the first segment data corresponding to the DMA write descriptor of a network card, obtain the second segment stored in the network card and corresponding to the second network card DMA write descriptor according to the DMA write descriptor of the second network card Data, wherein the first data to be written includes n first fragments of data, and the second data to be written includes m second fragments of data; the network card in the forwarding device is based on the first The network card DMA write descriptor and the first fragmented data generate a first DMA write command, and the second network card DMA write descriptor and the second fragmented data generate a second DMA write command; in the forwarding device The network card in the forwarding device sends the first DMA write command and the second DMA write command to the queue expansion device; the queue expansion device in the forwarding device according to the first DMA write command and the second DMA write command , Write the first fragment data and the second fragment data into the memory of the queue expansion device; in the memory of the queue expansion device in the forwarding device, the number of the first fragment data is n In this case, a third DMA write command is generated according to the first data to be written, and a fourth DMA write command is generated according to the second data to be written when the number of the second fragmented data is m. DMA write command, wherein the third DMA write command is used for the computing device to execute and write the first data to be written into the computing device, and the fourth DMA write command is used for the computing device The computing device executes and writes the first data to be written into the computing device; the queue expansion device in the forwarding device sends the The computing device sends the third DMA write command and the fourth DMA write command, so that the computing device writes the first to-be-written command according to the third DMA write command and the fourth DMA write command The data and the second data to be written are written into the computing device. The second data to be written, the second data to be written, the second data to be written, the second data to be written, and the second data to be written. After the sending data is stored in the network card queue corresponding to the corresponding queue expansion device queue, the method further includes: the network card in the forwarding device sends a completion message and an interrupt message to the queue expansion device; the queue in the forwarding device The extension device sends the completion message and interrupt to the computing device, so that the computing device refreshes the tail pointers of the first host queue and the second host queue according to the completion message. In specific implementation, after the forwarding device or the network card has processed the read command or the write command, it will also write the command completion information in the network card queue, and send it to the queue expansion device together with the interrupt message, and the queue expansion device will then add the interrupt message to the queue expansion device. The command completion information is sent to the computing device, thereby completely completing a data communication process.

In summary, using the queue communication method provided in this application, after obtaining the number of network card queues in the network card, the queue expansion device sends the expanded network card queue number to the computing device, so that the performance of the network card hardware can be fully explored. The thread message forwarding rate of the virtual machine no longer becomes a bottleneck for the network card to process message forwarding, further avoiding resource waste.

As shown in Figure 8, the following takes the application scenario where the first data to be sent and the second data to be sent are DMA read descriptors as an example (the computing device sends the first data to be read and the second data to be read to the network card) , Give an example of the queue communication method described in FIG. 7.

As shown in Figure 8, assuming that the queue expansion coefficient n=2, the queue expansion device has generated the second queue correspondence with the network card, and the first queue correspondence with the computing device, and stored the two queue correspondences in the queue. Mapping table. There is a first queue correspondence between the first host queue A and the second host queue B and the corresponding queue expansion device queue C, and there is a second queue correspondence between the corresponding queue expansion device queue C and the corresponding network card queue D relationship. Currently, the computing device needs to send the first data to be read and the second data to be read to the network card 600, so the specific communication process may be as follows:

1. The computing device 400 puts the DMA read descriptor of the first data to be read, that is, the first data to be sent, into the 4 queue units of the first host queue A, and reads the DMA of the second data to be read The descriptor, that is, the second data to be sent, is put into the 3 queue units of the second host queue B.

2. The computing device 400 refreshes the value of the first head pointer of the first host queue to A3, and refreshes the value of the second head pointer of the second host queue to B3.

3. The computing device 400 sends the first doorbell message to the queue expansion device 500.

4. The queue expansion device 500 obtains the first head pointer A3 and the second head pointer B3 from the first doorbell message.

5. The queue expansion device 500 refreshes the first head pointer in the first doorbell register as A3, and the second head pointer as B2.

6. The queue expansion device 500 obtains the data of the four queue units A0-A3 from the computing device according to the refreshed first doorbell register, the head and tail pointers of the first host queue are A3 and A0, that is, the first to be sent Data, similarly, obtain the second to-be-sent data from the computing device.

7. The queue expansion device 500 determines the queue expansion device queue C corresponding to the first host queue A and the second host queue B according to the first mapping relationship, and combines the first to-be-sent data (4 queue units) and the second to-be-sent data The sending data (3 queue units) is put into the corresponding queue expansion device queue C.

8. The queue expansion device 500 refreshes the value of the third head pointer of the corresponding queue expansion device queue C to C7.

9. The queue expansion device 500 sends the second doorbell message to the network card 600.

10. The network card 600 obtains the value C7 of the third head pointer from the second doorbell message.

11. The network card 600 refreshes the value C7 of the third head pointer in the second doorbell register.

12. The network card 600 obtains the data of the 7 queue units C0-C7 from the queue expansion device 500 according to the refreshed second doorbell register, the head and tail pointers of the corresponding queue expansion device queue are C7 and C0, that is, the first The data to be sent and the second data to be sent.

13. The network card 600 determines the network card queue D corresponding to the corresponding queue expansion device queue C according to the second mapping relationship, and puts a total of 7 queue units of the first data to be sent and the second data to be sent into the corresponding network card Queue D.

14. The network card 600 refreshes the value of the fourth head pointer of the corresponding network card queue D to D7.

15. The network card 600 generates a first DMA read command and a second DMA read command according to the DMA descriptors in the 7 queue units, and sends them to the queue expansion device 500.

16. The queue expansion device 500 transparently transmits the first DMA read command and the second DMA read command to the computing device.

17. The computing device 400 obtains the first to-be-read data stored in the computing device 400 corresponding to the first DMA read command and the second DMA read command according to the first DMA read command and the second DMA read command And the second data to be read.

18.The computing device 400 sends the first data to be read and the second data to be read to the queue expansion device 500.

19. The queue expansion device 500 sends the first data to be read and the second data to be read to the network card 600.

20. The network card 600 updates the tail pointer of the corresponding queue expansion device queue C to C7.

21. The network card 600 sends the completion queue unit and interrupt to the queue expansion device, and carries the value of the third tail pointer C7 to the queue expansion device 500, so that the queue expansion device 500 refreshes the value of the third tail pointer in the first doorbell register .

22. The queue expansion device 500 parses and completes the queue unit and interrupt, refreshes the value of the first tail pointer of the first host queue in the first doorbell register as A3, and the value of the second tail pointer of the second host queue is B2, and generates a new The completion queue unit and interrupt are sent to the computing device, so that the computing device can refresh the tail pointer of the first host queue and the tail pointer of the second host queue maintained by itself, and complete the first data to be read and the second data to be read. send.

It is understandable that using the queue communication method provided in this application, after obtaining the number of network card queues in the network card, the queue expansion device sends the expanded network card queue number to the computing device, so that the performance of the network card hardware can be fully exploited , The thread message forwarding rate of the virtual machine no longer becomes the bottleneck of the network card processing message forwarding, further avoiding the waste of resources.

As shown in FIG. 9, the following takes the application scenario where the first data to be sent and the second data to be sent are DMA write descriptors as an example (the network card sends the first data to be written and the second data to be written to the computing device), The queue communication method described in FIG. 7 is illustrated by an example.

As shown in Figure 9, assuming that the queue expansion coefficient n=2, the queue expansion device has generated the second queue correspondence with the network card, and the first queue correspondence with the computing device, and stored the two queue correspondences in the queue. Mapping table. There is a first queue correspondence between the first host queue A and the second host queue B and the corresponding queue expansion device queue C, and there is a second queue correspondence between the corresponding queue expansion device queue C and the corresponding network card queue D relationship. Currently, the network card 600 needs to send the first data to be written and the second data to be written to the computing device 500, so the specific communication process may be as follows:

1. The computing device 400 puts the first DMA write descriptor of the first data to be written, that is, the first data to be sent into the 4 queue units of the first host queue A, and transfers the second data to be written The second DMA write descriptor, that is, the second data to be sent, is put into the 3 queue units of the second host queue B. The first DMA write descriptor is used to write first data to be written into the computing device, and the second DMA descriptor is used to write second data to be written into the computing device.

2. The computing device 400 refreshes the first host queue with a first head pointer value of A3, and refreshes the second host queue with a second head pointer value of B2.

4. The queue expansion device 500 obtains the first head pointer A3 and the second head pointer B2 from the first doorbell message.

6. The queue expansion device 500 obtains the data of the four queue units A0-A3 from the computing device according to the refreshed first doorbell register, the head and tail pointers of the first host queue are A3 and A0, that is, the first DMA write The descriptor, in the same way, obtains the second DMA write descriptor from the computing device.

7. The queue expansion device 500 converts the first DMA write descriptor into a first network card DMA write descriptor, and converts the second DMA write descriptor into a second network card DMA write descriptor, where the first The DMA write descriptor is used to write the first data to be written into the queue expansion device, and the first DMA write descriptor is used to write the second data to be written into the queue expansion device.

8. The queue expansion device 500 determines the queue expansion device queue C corresponding to the first host queue A and the second host queue B according to the first mapping relationship, and writes the converted first DMA descriptor (4 queue units) And the second DMA write descriptor (3 queue units) is put into the corresponding queue expansion device queue C.

9. The queue expansion device 500 refreshes the value of the third head pointer of the corresponding queue expansion device queue C to C7.

10. The queue expansion device 500 sends the second doorbell message to the network card 600.

11. The network card 600 obtains the value C7 of the third head pointer from the second doorbell message.

12. The network card 600 refreshes the value C7 of the third head pointer in the second doorbell register.

13. According to the refreshed second doorbell register, the network card 600 obtains the data of the 7 queue units C0-C7 from the queue expansion device 500 according to the head and tail pointers of the corresponding queue expansion device queue as C7 and C0, that is, after conversion The first DMA write descriptor and the second DMA write descriptor.

14. The network card 600 determines the network card queue D corresponding to the corresponding queue expansion device queue C according to the second mapping relationship, and puts a total of 7 queue units of the first data to be sent and the second data to be sent into the corresponding network card Queue D.

15. The network card 600 refreshes the value of the fourth head pointer of the corresponding network card queue D to D7.

16. The network card 600 obtains, according to the first network card DMA write descriptor, the first fragmented data stored in the network card and corresponds to the first network card DMA write descriptor, and writes according to the second network card DMA write descriptor. Descriptor, acquiring the second fragmented data stored in the network card and corresponding to the DMA write descriptor of the second network card, wherein the first data to be written includes n pieces of first fragmented data, so The second data to be written includes m pieces of second data.

17. The network card 600 generates a first DMA write command according to the first network card DMA write descriptor and the first fragment data, and generates a second DMA write command according to the second network card DMA write descriptor and the second fragment data. DMA write command.

18. The network card 600 updates the tail pointer of the corresponding queue expansion device queue C to C7, and sends the first DMA write command and the second DMA write command to the queue expansion device, as well as the sending completion queue unit and interrupt, The value of the third tail pointer C7 is carried to the queue expansion device 500, so that the queue expansion device 500 refreshes the value of the third tail pointer in the first doorbell register.

19. The queue expansion device writes the first fragmented data and the second fragmented data into the memory of the queue expansion device according to the first DMA write command and the second DMA write command, and refreshes all Describe the value of the third tail pointer in the first doorbell register.

20. The queue expansion device 500 parses the completion queue unit and the interrupt, and according to the address information of the queue completion unit, determines the write addresses of the first data to be written and the second data to be written to the computing device, so that the first When the number of fragmented data is n, a third DMA write command is generated according to the above write address and the first data to be written, and when the number of second fragmented data is m, according to the above write The address and the second data to be written generate the fourth DMA write command, the value of the first tail pointer of the first host queue in the first doorbell register is refreshed is A3, and the value of the second tail pointer of the second host queue is B2. Generate a new completion queue unit and interrupt.

21. In the memory of the queue expansion device 500, when the number of the first fragmented data is n, generate the third DMA write command, and when the number of the second fragmented data is m, generate the fourth DMA write command, wherein the third DMA write command is used for the computing device to execute and write the first data to be written into the computing device, and the fourth DMA write command is used for the computing device The computing device executes and writes the first data to be written into the computing device.

22. The queue expansion device sends the third DMA write command, the fourth DMA write command, the new completion queue unit and the interrupt to the computing device, so that the computing device can refresh the tail pointer of the first host queue maintained by itself and The second host queue tail pointer completes the sending of the first data to be read and the second data to be read this time.

23. The computing device writes the first data to be written and the second data to be written into the computing device according to the third DMA write command and the fourth DMA write command. Complete the process of this queue communication.

With reference to the related embodiments shown in FIGS. 1 to 9 above, the related devices involved in the embodiments of the present application are described below. Please refer to FIG. 10, which is a schematic structural diagram of a forwarding device provided by the present application. The forwarding device 300 is connected to a computing device, the forwarding device includes at least one forwarding queue, the computing device includes a plurality of host queues, and a plurality of data to be sent are stored in the plurality of host queues, wherein the forwarding device The device includes a receiving unit 310, an acquiring unit 320, a storage unit 330, a response unit 340, and a refresh unit 350.

The receiving unit 310 is configured to receive a first doorbell message from the computing device, where the first doorbell message carries information about a first host queue and information about a second host queue, and the first host queue And the second host queue both belong to the multiple host queues;

The acquiring unit 320 is configured to acquire the information of the first host queue and the information of the second host queue from the first doorbell message, acquire the first to-be-sent data from the first host queue, and from Acquiring second data to be sent from the second host queue;

The storage unit 330 is configured to store the first data to be sent and the second data to be sent into the first host queue respectively according to the queue correspondence between the forwarding queue and the host queue. A forwarding queue corresponding to the second host queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.

In an embodiment, the forwarding device further includes a response unit 340, and the receiving unit 310 is further configured to receive the device from the computing device before the receiving unit receives the first doorbell message from the computing device Query message; the response unit 340 is configured to report the number of extended forwarding queues to the computing device in response to the device query message, wherein the number of extended forwarding queues is forwarded by the at least one The number of queues and the expansion coefficient are determined, and the expanded number of forwarding queues is used to establish the queue correspondence.

In an embodiment, the forwarding device includes an attribute register, the number of the expanded forwarding queues is stored in the attribute register, and the response unit 340 is configured to query the attribute register and obtain the information from the attribute register. The number of the extended forwarding queues is acquired, and the number of the extended forwarding queues is reported to the computing device.

In an embodiment, the information of the first host queue includes a first head pointer, the first head pointer indicates the first to-be-sent data, and the information about the second host queue includes a second head pointer, so The second head pointer indicates the second data to be sent, and multiple sets of head pointers and multiple sets of tail pointers are stored in the forwarding device, and each set of head pointers and each set of tail pointers respectively correspond to multiple queues of the computing device The forwarding device also stores at least one set of head pointers and at least one set of tail pointers of the at least one queue; the forwarding device further includes a refresh unit 350, and the acquiring The unit 320 is configured to obtain the first head pointer and the second head pointer from the first doorbell message; the refresh unit 350 is configured to refresh all the groups of head pointers according to the first head pointer. The head pointer corresponding to the first host queue; the obtaining unit 320 is configured to use the head pointer corresponding to the first host queue after refreshing and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers, Acquiring the first data to be sent in the first host queue; the refresh unit 350 is configured to refresh the head pointer corresponding to the second host queue in the plurality of sets of head pointers according to the second head pointer; The acquiring unit 320 is used to acquire all the data from the second host queue according to the head pointer corresponding to the second host queue after refreshing and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers. The second data to be sent; the refresh unit 350 is configured to store the first data to be sent and the second data to be sent into the forwarding queues corresponding to the first host queue and the second host queue respectively, and then refresh The tail pointer of the first host queue and the tail pointer of the second host queue; the refresh unit 350 is used to refresh the head pointer of the corresponding forwarding queue; the refresh unit 350 is used to refresh all the The tail pointer of the first host queue and the refreshed tail pointer of the second host queue are sent to the computing device.

It should be noted that FIG. 10 is only an implementation of the embodiment of the present application. For the multiple modules or units mentioned in the embodiment of the present application, those skilled in the art should know that the functions of the foregoing multiple modules or units can be split into More sub-modules or sub-units can also be combined into fewer modules or units to achieve the same technical effect, so they should all fall into the protection scope of the embodiments of this application

Fig. 11 is a schematic structural diagram of a queue expansion device provided by the present application. The queue expansion device is connected to a computing device, the network card is connected to the computing device, the queue expansion device includes at least one queue expansion device queue, the network card includes at least one network card queue, and the computing device includes multiple hosts A queue, a plurality of data to be sent are stored in the plurality of host queues. As shown in FIG. 11, the queue expansion device 500 includes a receiving unit 510, an acquiring unit 520, a storage unit 530, a sending unit 540, a response unit 550, and a refresh unit 560.

The receiving unit 510 is configured to receive a first doorbell message from the computing device, where the first doorbell message carries information about a first host queue and information about a second host queue, and the first host queue And the second host queue both belong to the multiple host queues;

The obtaining unit 520 is configured to obtain the information of the first host queue and the information of the second host queue from the first doorbell message, obtain the first to-be-sent data from the first host queue, and from Acquiring second data to be sent from the second host queue;

The storage unit 530 is configured to store the first data to be sent and the second data to be sent into the first queue respectively according to the first queue correspondence between the queue expansion device queue and the host queue. A host queue and a queue expansion device queue corresponding to the second host queue;

The sending unit 540 is configured to send a second doorbell message to the network card, and the second doorbell message carries the information of the corresponding queue expansion device queue, so that the network card receives information from the network card according to the second doorbell message. The first data to be sent and the second data to be sent are acquired from the corresponding queue expansion device queue, and according to the second queue correspondence between the network card queue and the queue expansion device queue, the second queue A data to be sent and the second data to be sent are stored in the network card queue corresponding to the corresponding queue expansion device queue, wherein the corresponding queue expansion device queue belongs to the queue expansion device queue, and the first The host queue and the second host queue correspond to the same queue expansion device queue in the at least one queue expansion device queue, and the corresponding network card queue belongs to the network card queue.

In an embodiment, the device further includes a response unit 550, and the receiving unit 510 is further configured to receive a first device query message from the computing device before receiving the first doorbell message from the computing device; The sending unit 540 is configured to send a second device query message to the network card; the response unit 550 is configured to receive the number of network card queues sent by the network card, wherein the number of network card queues is used to establish the first Correspondence between two queues; the response unit 550 is configured to report the number of expanded queue expansion device queues to the computing device in response to the first device query message, wherein the expanded queue expansion device queue The number is determined by the number of network card queues and an expansion coefficient, and the expanded number of queue expansion device queues is used to establish the first queue correspondence relationship.

In an embodiment, the device includes an attribute register, and the number of the expanded forwarding queues is stored in the attribute register; the response unit 550 is further configured to query the attribute register, and obtain information from the attribute register. The number of queues of the expanded queue expansion device is acquired, and the number of queues of the expanded queue expansion device is reported to the computing device.

In an embodiment, the information of the first host queue includes a first head pointer, the first head pointer indicates the first to-be-sent data, and the information about the second host queue includes a second head pointer, so The second head pointer indicates the second data to be sent, and the queue expansion device stores multiple sets of head pointers and multiple sets of tail pointers, and each set of head pointers and each set of tail pointers respectively correspond to the multiple host queues The head pointer and the tail pointer of a host queue in, the queue expansion device also stores at least one set of queue expansion device head pointers and at least one set of queue expansion device tail pointers, and at least one set of network cards is stored in the network card A head pointer and at least one set of network card tail pointers, the at least one set of queue extension device head pointers correspond to the at least one set of network card head pointers, and the at least one set of queue extension device tail pointers corresponds to the at least one set of network card head pointers. There is a one-to-one correspondence between the tail pointers of the network card, the queue expansion device further includes a refresh unit 560, and the acquisition unit 520 is configured to acquire the first head pointer and the second head pointer from the first doorbell message; The refreshing unit 560 is configured to refresh the head pointers corresponding to the first host queue in the multiple sets of head pointers according to the first head pointer and the first queue correspondence; the acquiring unit 520 is configured to refresh according to After refreshing, the head pointer corresponding to the first host queue, and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers, acquire the first data to be sent in the first host queue; The refresh unit 560 is configured to refresh the head pointer corresponding to the second host queue in the multiple sets of head pointers according to the second head pointer and the corresponding relationship of the first queue; the acquiring unit 520 is configured to refresh according to After refreshing, the head pointer corresponding to the second host queue, and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers, acquire the second data to be sent in the second host queue; The storage unit 530 is configured to store the first data to be sent and the second data to be sent in the queue expansion device queues corresponding to the first host queue and the second host queue according to the corresponding relationship of the first queue , Wherein the corresponding queue expansion device queue belongs to the queue expansion device queue, and the first host queue and the second host queue correspond to the same queue expansion device queue in the at least one queue expansion device queue; The refresh unit 560 is configured to refresh the head pointer of the corresponding queue expansion device queue, the tail pointer of the first host queue, and the tail pointer of the second host queue; the sending unit 540 is also configured to send a message to the network card Sending the second doorbell message, wherein the second doorbell message carries the head pointer of the corresponding queue expansion device queue, so that the network card expands the head pointer of the device queue according to the corresponding queue and the first pointer Correspondence of the two queues: refresh the head pointer of the corresponding queue expansion device in the at least one set of network card head pointers, and refresh the corresponding network card head pointer and the at least one set of network card tail pointers The tail pointer corresponding to the corresponding queue expansion device queue described in , Acquiring the first data to be sent and the second data to be sent in the corresponding queue expansion device queue, so as to combine the first data to be sent and the second data to be sent according to the corresponding relationship of the second queue Stored in the network card queue corresponding to the corresponding queue expansion device queue.

In an embodiment, the first data to be sent is a first direct memory access DMA read descriptor, the second data to be sent is a second DMA read descriptor, and the receiving unit 510 is further configured to receive The first DMA read command and the second DMA read command sent by the network card, wherein, the first DMA read command is generated by the network card according to the first DMA descriptor, and the second DMA read command is The network card is generated according to the second DMA descriptor; the obtaining unit 520 is further configured to obtain the first data to be read from the computing device according to the first DMA read command, and obtain the first data to be read from the computing device according to the second DMA read command. The computing device obtains the second data to be read; the sending unit 540 is further configured to forward the first data to be read and the second data to be read to the network card.

In an embodiment, the first data to be sent is a first DMA write descriptor, and the second data to be sent is a second DMA write descriptor, wherein the first DMA write descriptor is used to transfer the first DMA write descriptor. A data to be written is written from the queue expansion device to the computing device, and the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device, and the device It also includes a conversion unit 570, which is configured to convert the first DMA write descriptor to a first network card DMA write descriptor, and convert the second DMA write descriptor to a second network card DMA write descriptor , Wherein the first DMA write descriptor is used to write the first data to be written from the network card to the queue expansion device, and the second DMA descriptor is used to write the second data to be written from the network card to the queue expansion device. The network card is written into the queue expansion device; the storage unit 530 is configured to store the first network card DMA write descriptor and the second network card DMA write descriptor into the corresponding queue expansion device queue; The receiving unit 510 is configured to receive a first DMA write command and the second DMA write command, where the first DMA write command is obtained by the network card according to the first network card DMA write descriptor and stored in the network card In the first segment data corresponding to the first network card DMA write descriptor, the first DMA write command generated according to the first network card DMA write descriptor and the first segment data, the first The second DMA write command is that after the network card obtains the second fragment data stored in the network card corresponding to the second network card DMA write descriptor according to the second network card DMA write descriptor, according to the first The second network card DMA write descriptor and the second DMA write command generated by the second fragmented data, wherein the first to-be-written data includes n pieces of first fragmented data, and the second to-be-written data includes m second fragmented data; the storage unit 530 is configured to write the first fragmented data and the second fragmented data into the first fragmented data according to the first DMA write command and the second DMA write command In the memory of the queue expansion device; the conversion unit 570 is configured to generate a third piece of data according to the first data to be written when the number of first fragmented data in the memory of the queue expansion device is n. DMA write command, when the number of the second fragmented data is m, a fourth DMA write command is generated according to the second data to be written, wherein the third DMA write command is used for The computing device executes and writes the first data to be written into the computing device, and the fourth DMA write command is used for the computing device to execute and write the first data to be written into the computing device. The computing device; the sending unit 540 is configured to send the third DMA write command and the fourth DMA write command to the computing device, so that the computing device according to the third DMA write command and the first Four DMA write commands to write the first data to be written and the second data to be written into the computing device.

In an embodiment, the receiving unit 510 is further configured to receive a completion message and an interrupt sent by the network card, where the completion message and the interrupt are when the network card is sending the first data to be sent and the first data to be sent. 2. The data to be sent is generated after being stored in the network card queue corresponding to the corresponding queue expansion device queue; the sending unit 540 is also configured to send a completion message and an interrupt to the computing device, so that the computing device according to the The completion message refreshes the tail pointers of the first host queue and the second host queue.

It should be noted that FIG. 11 is only an implementation of the embodiment of the present application. For the multiple modules or units mentioned in the embodiment of the present application, those skilled in the art should know that the functions of the foregoing multiple modules or units can be split into More sub-modules or sub-units can also be combined into fewer modules or units to achieve the same technical effect, and therefore all should fall within the protection scope of the embodiments of the present application.

FIG. 12 is a schematic structural diagram of a forwarding device provided by an embodiment of the present application. As shown in FIG. 12, the forwarding device 700 includes one or more processors 701, a communication interface 702, and a memory 703. Wherein, the processor 801, the communication interface 702, and the memory 703 may be connected through a bus 704. The bus 704 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 704 can be divided into an address bus, a data bus, a control bus, and so on. For ease of representation, only one thick line is used in FIG. 12 to represent it, but it does not mean that there is only one bus or one type of bus.

In a specific implementation, the processor 701 includes one or more general-purpose processors, where the general-purpose processor may be any type of device capable of processing electronic instructions, including a CPU, a microprocessor, a microcontroller, a main processor, and a controller. And ASIC and so on. The processor 701 executes various types of digital storage instructions, such as software or firmware programs stored in the memory 703, which enables the computing device to provide a wide variety of services. In the embodiment of the present application, the processor 701 executes the program instructions in the memory 703 to implement the aforementioned methods in FIG. 4 and FIG. 6. For example, the processor 701 may include a queue expansion module, a mapping module, a queue forwarding module, etc., where the queue expansion module may implement processing functions by calling program codes in the memory 703, such as calculating the number of expanded queues; The mapping module can realize the processing function by calling the program code in the memory 304, and generate the queue correspondence between the host queue and the forwarding queue; the queue forwarding module can realize the processing function by calling the program code in the memory 703, and realize the connection with the computing device. Queue communication. The processor 701 may also include other modules for executing other steps described in the embodiment of FIG. 4 and FIG. 6, which will not be repeated here.

The communication interface 702 may be a wired interface (for example, an Ethernet interface) for communicating with other devices or modules.

The memory 703 may include a volatile memory (Volatile Memory), such as a random access memory (Random Access Memory, RAM); the memory may also include a non-volatile memory (Non-Volatile Memory), such as a read-only memory (Read-Only Memory). Memory, ROM, Flash Memory, Hard Disk Drive (HDD), or Solid-State Drive (SSD) memory may also include a combination of the foregoing types of memories. The memory 703 may include a doorbell register, an attribute register, and a forwarding queue. The doorbell register is used to store the head and tail pointers of at least one set of queues, the attribute register is used to store the number of expanded queues, and the forwarding queue is used to store 400 The data to be sent sent. It should be understood that the above examples are only for illustration, and the memory 703 may also include a mapping relationship table, which is used to store queue correspondences, and may also be used to store other program codes or program data for the processor to implement FIG. 4 And the other steps described in the embodiment of FIG. 6 to FIG. 9, which will not be repeated here.

FIG. 13 is a schematic structural diagram of a queue expansion device 800 provided by the present application. As shown in FIG. 13, the queue expansion device 800 includes a processor 802, a memory 504, an endpoint port 501, and a root port 503. Among them, the port endpoint 501, the root port 505, and the memory 504 are all connected to the processor 502 through a bus 505.

The queue expansion device 800 includes one or more processors 801, a communication interface 802, and a memory 803. Wherein, the processor 801, the communication interface 802, and the memory 803 may be connected through a bus 804. The bus 804 may be a PCI bus, an EISA bus, or the like. The bus 704 can be divided into an address bus, a data bus, a control bus, and so on. For ease of representation, only one thick line is used in FIG. 13, but it does not mean that there is only one bus or one type of bus.

In a specific implementation, the processor 802 includes one or more general-purpose processors, where the general-purpose processor may be any type of device capable of processing electronic instructions, including a CPU, a microprocessor, a microcontroller, a main processor, and a controller. And ASIC and so on. The processor 802 executes various types of digital storage instructions, such as software or firmware programs stored in the memory 804, which enables the computing device to provide a wide variety of services. In the embodiment of the present application, the processor 801 executes the program instructions in the memory 803 to implement the aforementioned methods in FIG. 7, FIG. 8, and FIG. 9. For example, the processor 802 may include a queue expansion module, a mapping module, a queue forwarding module, etc., where the queue expansion module may implement processing functions by calling program codes in the memory 804, such as calculating the number of expanded queues; The mapping module can realize the processing function by calling the program code in the memory 804, generate the first queue correspondence between the host queue and the queue expansion queue, or generate the second queue correspondence between the queue expansion queue and the network card queue, etc. The queue forwarding module can realize the processing function by calling the program code in the memory 804, and complete the forwarding between the extended queue device and the network card for the received data to be sent from the computing device. The processor may also include other modules for executing other steps described in the embodiments in FIG. 5 to FIG. 9, and details are not described herein again.

The communication interface 802 may be a wired interface (for example, an Ethernet interface) for communicating with other devices or modules.

The memory 804 may include volatile memory, such as RAM; the memory may also include non-volatile memory, such as ROM, flash memory, HDD, or SSD. The memory may also include a combination of the above-mentioned types of memories. The memory 804 may include a first doorbell register, an attribute register, and a queue expansion device queue. The first doorbell register is used to store the head and tail pointers of at least one set of queues, and the attribute register is used to store the number of expanded queues. The device queue is used to store the to-be-sent data sent by the computing device. It should be understood that the above example is only for illustration, and the memory 804 may also include a mapping relationship table for storing the correspondence between the first queue and the second queue. Corresponding relationship, the memory 804 can also be used to store the first segment data and the second segment data, etc., and can also be used to store other program codes or program data for the processor to implement the description of the embodiments in FIGS. 7-9 The other steps are not repeated here.

It should be noted that FIG. 12 and FIG. 13 are only a possible implementation manner of the embodiment of the present application. In actual applications, the forwarding device 700 and the queue expansion device 800 may also include more or fewer components, which are not limited here. Regarding the content that is not shown or described in the embodiments of the present application, please refer to the relevant descriptions in the embodiments described in FIG. 1 to FIG. 9, which will not be repeated here.

The queue expansion device 500 shown in FIG. 11 can be implemented by a logic circuit, specifically, it can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (Programmable Logic Device, PLD). The above-mentioned PLD can be a Complex Programmable Logical Device (CPLD), a Field-Programmable Gate Array (FPGA), a Generic Array Logic (GAL), or any combination thereof. This application does not Specific restrictions.

The embodiment of the present invention also provides a computer non-transitory storage medium, the computer non-transitory storage medium stores instructions, when it runs on the processor, any of the descriptions in FIGS. 4, 6 and 9 A method flow is realized.

The embodiment of the present invention also provides a computer program product. When the computer program product runs on a processor, the process of any method described in FIGS. 4 and 6 to 9 can be realized.

The steps of the method or algorithm described in combination with the disclosure of the embodiment of the present invention may be implemented in a hardware manner, or may be implemented in a manner in which a processor executes software instructions. Software instructions can be composed of corresponding software modules, which can be stored in RAM, flash memory, ROM, erasable programmable read-only memory (Erasable Programmable ROM, EPROM), and electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), registers, hard disk, mobile hard disk, CD-ROM or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and the storage medium may be located in the ASIC. In addition, the ASIC may be located in the computing device. Of course, the processor and the storage medium may also exist as discrete components in the computing device.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through a computer program. The program can be stored in a computer readable storage medium, and the program can be stored in a computer readable storage medium. When executed, it may include the procedures of the above-mentioned method embodiments. The aforementioned storage media include: ROM, RAM, magnetic disks or optical disks and other media that can store program codes.

Claims

A method for queue communication, wherein the method is applied to a forwarding device, the forwarding device is connected to a computing device, the forwarding device includes at least one forwarding queue, the computing device includes multiple host queues, and the A plurality of data to be sent is stored in a plurality of host queues, and the method includes:

The forwarding device receives a first doorbell message from the computing device, where the first doorbell message carries information about a first host queue and information about a second host queue, and the first host queue and the The second host queues all belong to the multiple host queues;

The forwarding device obtains the information of the first host queue and the information of the second host queue from the first doorbell message, obtains the first data to be sent from the first host queue, and obtains the first data to be sent from the first host queue. 2. Obtain the second data to be sent from the host queue;

The forwarding device stores the first to-be-sent data and the second to-be-sent data into the first host queue and the second host respectively according to the queue correspondence between the forwarding queue and the host queue A forwarding queue corresponding to the queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.
The method according to claim 1, wherein before the forwarding device receives the first doorbell message from the computing device, the method further comprises:

The forwarding device receives a device query message from the computing device;

In response to the device query message, the forwarding device reports the expanded number of forwarding queues to the computing device, where the expanded number of forwarding queues is determined by the number of the at least one forwarding queue and the expansion The coefficient is determined, and the expanded number of forwarding queues is used to establish the queue correspondence.
The method according to claim 2, wherein the forwarding device comprises an attribute register, and the number of the extended forwarding queues is stored in the attribute register;

The reporting by the forwarding device of the expanded number of forwarding queues to the computing device includes:

The forwarding device queries the attribute register, obtains the expanded number of forwarding queues from the attribute register, and reports the expanded number of forwarding queues to the computing device.
The method according to any one of claims 1-3, characterized in that,

The information of the first host queue includes a first head pointer, the first head pointer indicates the first data to be sent, and the information of the second host queue includes a second head pointer, the second head pointer indicates For the second data to be sent, multiple sets of head pointers and multiple sets of tail pointers are stored in the forwarding device, and each set of head pointers and each set of tail pointers respectively correspond to one of the multiple host queues of the computing device At least one set of head pointers and at least one set of tail pointers of the at least one forwarding queue are also stored in the forwarding device;

The acquiring, by the forwarding device, the information of the first host queue and the information of the second host queue from the first doorbell message includes:

Acquiring, by the forwarding device, the first head pointer and the second head pointer from the first doorbell message;

The forwarding queue acquiring the first to-be-sent data from the first host queue includes:

Refreshing, by the forwarding device, the head pointer corresponding to the first host queue in the multiple sets of head pointers according to the first head pointer;

The forwarding device obtains the first host queue from the first host queue according to the head pointer corresponding to the first host queue after the refresh and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers. Data to be sent;

The acquiring, by the forwarding queue, of the second to-be-sent data from the second host queue, includes:

Refreshing, by the forwarding device, the head pointer corresponding to the second host queue in the plurality of sets of head pointers according to the second head pointer;

The forwarding device obtains the second host queue from the second host queue according to the head pointer corresponding to the second host queue after the refresh and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers. Data to be sent;

After the forwarding device stores the first data to be sent and the second data to be sent into the corresponding forwarding queues of the first host queue and the second host queue, respectively, the method further includes:

Refreshing the tail pointer of the first host queue and the tail pointer of the second host queue;

Refreshing the head pointer of the corresponding forwarding queue;

Send the refreshed tail pointer of the first host queue and the refreshed tail pointer of the second host queue to the computing device.
The method according to any one of claims 1 to 4, wherein the forwarding device includes a queue expansion device and a network card, and the at least one forwarding queue includes at least one queue expansion device queue and at least one network card queue, so The at least one queue expansion device queue is located in the queue expansion device, the at least one network card queue is located in the network card, the at least one set of head pointers and the at least one set of tail pointers are stored in the queue expansion device, and the The queue correspondence relationship includes a first queue correspondence relationship and a second queue correspondence relationship. The first queue correspondence relationship is the correspondence relationship between the plurality of host queues and the at least one queue expansion device queue, and the second queue correspondence relationship Is the correspondence between the at least one network card queue and the at least one queue expansion device queue, the at least one network card queue corresponds to the at least one queue expansion device queue one-to-one, and the queue expansion device stores multiple groups of headers Pointers and multiple sets of tail pointers, each set of head pointers and each set of tail pointers respectively correspond to the head pointer and the tail pointer of a host queue in the plurality of host queues, and at least one set of queues is also stored in the queue expansion device Extension device head pointer and at least one set of queue extension device tail pointers, said network card stores at least one set of network card head pointers and at least one set of network card tail pointers, said at least one set of queue extension device head pointers and said at least One set of network card head pointers corresponds to one to one, and the at least one set of queue expansion device tail pointers corresponds to the at least one set of network card tail pointers;

The acquiring, by the forwarding device, the information of the first host queue and the information of the second host queue from the first doorbell message includes:

Acquiring, by the queue expansion device in the forwarding device, the first head pointer and the second head pointer from the first doorbell message;

The obtaining, by the forwarding device, the first to-be-sent data from the first host queue includes:

The queue expansion device in the forwarding device refreshes the head pointer corresponding to the first host queue among the sets of head pointers according to the first head pointer and the corresponding relationship between the first queue;

According to the queue expansion device in the forwarding device, the head pointer corresponding to the first host queue after being refreshed, and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers, in the first host queue Acquiring the first to-be-sent data;

The obtaining, by the forwarding device, the second to-be-sent data from the second host queue includes:

The queue expansion device in the forwarding device refreshes the head pointer corresponding to the second host queue in the multiple sets of head pointers according to the second head pointer and the corresponding relationship of the first queue;

According to the head pointer corresponding to the second host queue after being refreshed, the queue expansion device in the forwarding device and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers are in the second host queue. Acquiring the second to-be-sent data;

The storing, by the forwarding device, the first data to be sent and the second data to be sent into the first host queue and the forwarding queue corresponding to the second host queue respectively includes:

The queue expansion device in the forwarding device stores the first data to be sent and the second data to be sent in the queues corresponding to the first host queue and the second host queue according to the corresponding relationship of the first queue An expansion device queue, wherein the corresponding queue expansion device queue belongs to the queue expansion device queue, and the first host queue and the second host queue correspond to the same queue expansion in the at least one queue expansion device queue Equipment queue

Refreshing, by the queue expansion device in the forwarding device, the head pointer of the corresponding queue expansion device queue, the tail pointer of the first host queue, and the tail pointer of the second host queue;

The network card in the forwarding device receives the second doorbell message from the queue expansion device, where the second doorbell message carries the queue head pointer of the corresponding queue expansion device;

Acquiring, by the network card in the forwarding device, the head pointer of the corresponding queue expansion device queue from the second doorbell message;

The network card in the forwarding device refreshes the head pointer of the corresponding queue expansion device queue in the at least one set of network card head pointers according to the head pointer of the corresponding queue expansion device queue and the second queue correspondence relationship Corresponding network card head pointer;

The network card in the forwarding device expands the device queue in the corresponding queue according to the corresponding head pointer of the network card after refreshing, and the tail pointer corresponding to the corresponding queue expansion device queue in the at least one set of network card tail pointers Acquiring the first data to be sent and the second data to be sent in;

The network card in the forwarding device stores the first to-be-sent data and the second to-be-sent data in a network card queue corresponding to the corresponding queue expansion device queue according to the second queue correspondence.
The method according to claim 5, wherein the first data to be sent is a first direct memory access DMA read descriptor, the second data to be sent is a second DMA read descriptor, and the After the first data to be sent and the second data to be sent are stored in the network card queue corresponding to the corresponding queue expansion device queue, the method further includes:

The network card in the forwarding device generates a first DMA read command and a second DMA read command according to the first DMA read descriptor and the second DMA read descriptor;

Sending, by the network card in the forwarding device, the first DMA read command and the second DMA read command to the queue expansion device in the forwarding device;

The queue expansion device in the forwarding device obtains first data to be read from the computing device according to the first DMA read command, and obtains second data to be read from the computing device according to the second DMA read command ；

The queue expansion device in the forwarding device forwards the first data to be read and the second data to be read to the network card in the forwarding device.
The method according to claim 5, wherein the first data to be sent is a first DMA write descriptor, and the second data to be sent is a second DMA write descriptor, wherein the first DMA The write descriptor is used to write the first data to be written from the queue expansion device to the computing device, and the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device. In the computing device, storing the first data to be sent and the second data to be sent in the first host queue and the queue expansion device queue corresponding to the second host queue includes:

The queue expansion device in the forwarding device converts the first DMA write descriptor to a first network card DMA write descriptor, and converts the second DMA write descriptor to a second network card DMA write descriptor, where all The first DMA write descriptor is used to write the first data to be written from the network card into the queue expansion device, and the second DMA descriptor is used to write the second data to be written from the network card into the queue expansion device. The queue expansion device;

Storing, by the queue expansion device, the first network card DMA write descriptor and the second network card DMA write descriptor in the corresponding queue expansion device queue;

After storing the first data to be sent and the second data to be sent in the network card queue corresponding to the corresponding queue expansion device queue, the method further includes:

According to the first network card DMA write descriptor, the network card in the forwarding device obtains the first fragment data stored in the network card and corresponds to the first network card DMA write descriptor, and according to the second The network card DMA write descriptor is used to obtain the second segment data stored in the network card and corresponding to the second network card DMA write descriptor, wherein the first data to be written includes n first segments Data, the second to-be-written data includes m second fragmented data;

The network card in the forwarding device generates a first DMA write command according to the first network card DMA write descriptor and the first fragmented data, and generates a first DMA write command according to the second network card DMA write descriptor and the second fragmented data Generate the second DMA write command;

Sending, by the network card in the forwarding device, the first DMA write command and the second DMA write command to the queue expansion device;

The queue expansion device in the forwarding device writes the first fragmented data and the second fragmented data into the memory of the queue expansion device according to the first DMA write command and the second DMA write command ；

In the memory of the queue expansion device in the forwarding device, when the number of first fragmented data is n, a third DMA write command is generated according to the first data to be written, and the third DMA write command is generated in the second fragment. When the number of pieces of data is m, a fourth DMA write command is generated according to the second data to be written, where the third DMA write command is used for the computing device to execute and the first DMA write command Write data to be written into the computing device, and the fourth DMA write command is used for the computing device to execute and write the first data to be written into the computing device;

The queue expansion device in the forwarding device sends the third DMA write command and the fourth DMA write command to the computing device, so that the computing device is based on the third DMA write command and the first DMA write command. Four DMA write commands to write the first data to be written and the second data to be written into the computing device.
The method according to any one of claims 5-7, wherein the storing the first data to be sent and the second data to be sent in a network card queue corresponding to the corresponding queue expansion device queue After that, the method further includes:

The network card in the forwarding device sends a completion message and an interrupt message to the queue expansion device;

The queue expansion device in the forwarding device sends the completion message and interrupt to the computing device, so that the computing device refreshes the tail pointers of the first host queue and the second host queue according to the completion message.
A queue communication system, characterized in that the system includes a forwarding device and a computing device, the forwarding device is connected to the computing device, the forwarding device includes at least one forwarding queue, the computing device includes multiple host queues, and the There are multiple data to be sent stored in the multiple host queues, among which,

The computing device is configured to send a first command message to the forwarding device, where the first doorbell message carries information about the first host queue and the second host queue, and the first host queue and the all The second host queues all belong to the multiple host queues;

The forwarding device is used to receive the first doorbell message from the computing device;

The forwarding device is configured to obtain the information of the first host queue and the information of the second host queue from the first doorbell message, obtain the first data to be sent from the first host queue, and obtain the first data to be sent from the first host queue. Acquiring the second to-be-sent data from the second host queue;

The forwarding device is configured to store the first data to be sent and the second data to be sent into the first host queue and the first host queue according to the queue correspondence between the forwarding queue and the host queue. Two forwarding queues corresponding to the host queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.
The system according to claim 9, wherein the computing device is configured to send a device query message to the forwarding device before sending the first doorbell message to the forwarding device;

The forwarding device is used to receive a device query message from the computing device;

The forwarding device is configured to report the number of extended forwarding queues to the computing device in response to the device query message, wherein the number of the extended forwarding queues is determined by the number of the at least one forwarding queue And the expansion coefficient are determined, and the expanded number of forwarding queues is used to establish the queue correspondence.
The system according to claim 9, wherein the forwarding device comprises an attribute register, and the number of the extended forwarding queues is stored in the attribute register;

The forwarding device is configured to query the attribute register, obtain the number of the extended forwarding queues from the attribute register, and report the number of the extended forwarding queues to the computing device.
The system according to any one of claims 9-11, wherein:

The information of the first host queue includes a first head pointer, the first head pointer indicates the first data to be sent, and the information of the second host queue includes a second head pointer, the second head pointer indicates For the second data to be sent, multiple sets of head pointers and multiple sets of tail pointers are stored in the forwarding device, and each set of head pointers and each set of tail pointers respectively correspond to one of the multiple host queues of the computing device At least one set of head pointers and at least one set of tail pointers of the at least one forwarding queue are also stored in the forwarding device;

The forwarding device is configured to obtain the first head pointer and the second head pointer from the first doorbell message;

The forwarding device is configured to refresh the head pointer corresponding to the first host queue in the multiple sets of head pointers according to the first head pointer;

The forwarding device is configured to obtain the first host queue from the first host queue according to the head pointer corresponding to the first host queue after being refreshed, and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers. The first data to be sent;

The forwarding device is configured to refresh the head pointer corresponding to the second host queue in the multiple sets of head pointers according to the second head pointer;

The forwarding device is configured to obtain the second host queue according to the head pointer corresponding to the second host queue after refreshing and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers. The second data to be sent;

The forwarding device is configured to refresh the first host queue after storing the first to-be-sent data and the second to-be-sent data in the first host queue and the corresponding forwarding queue of the second host queue, respectively A tail pointer and the tail pointer of the second host queue;

The forwarding device is used to refresh the head pointer of the corresponding forwarding queue;

The forwarding device is configured to send the refreshed tail pointer of the first host queue and the refreshed tail pointer of the second host queue to the computing device.
The system according to any one of claims 9-12, wherein the forwarding device includes a queue expansion device and a network card, and the at least one forwarding queue includes at least one queue expansion device queue and at least one network card queue, so The at least one queue expansion device queue is located in the queue expansion device, the at least one network card queue is located in the network card, the at least one set of head pointers and the at least one set of tail pointers are stored in the queue expansion device, and the The queue correspondence relationship includes a first queue correspondence relationship and a second queue correspondence relationship. The first queue correspondence relationship is the correspondence relationship between the plurality of host queues and the at least one queue expansion device queue, and the second queue correspondence relationship Is the correspondence between the at least one network card queue and the at least one queue expansion device queue, the at least one network card queue corresponds to the at least one queue expansion device queue one-to-one, and the queue expansion device stores multiple groups of headers Pointers and multiple sets of tail pointers, each set of head pointers and each set of tail pointers respectively correspond to the head pointer and the tail pointer of a host queue in the plurality of host queues, and at least one set of queues is also stored in the queue expansion device Extension device head pointer and at least one set of queue extension device tail pointers, said network card stores at least one set of network card head pointers and at least one set of network card tail pointers, said at least one set of queue extension device head pointers and said at least One set of network card head pointers corresponds one to one, and the at least one set of queue expansion device tail pointers corresponds to the at least one set of network card tail pointers;

The queue expansion device is configured to obtain the first head pointer and the second head pointer from the first doorbell message;

The queue expansion device is configured to refresh the head pointer corresponding to the first host queue in the multiple sets of head pointers according to the first head pointer and the corresponding relationship of the first queue;

The queue expansion device is configured to obtain all data from the first host queue according to the head pointer corresponding to the first host queue after being refreshed, and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers. The first data to be sent;

The queue expansion device is configured to refresh the head pointer corresponding to the second host queue in the multiple sets of head pointers according to the second head pointer and the corresponding relationship of the first queue;

The queue expansion device is configured to obtain all data from the second host queue according to the head pointer corresponding to the second host queue after refreshing and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers. The second data to be sent;

The queue expansion device is configured to store the first data to be sent and the second data to be sent in the queue expansion device corresponding to the first host queue and the second host queue according to the corresponding relationship of the first queue Queue, wherein the corresponding queue expansion device queue belongs to the queue expansion device queue, and the first host queue and the second host queue correspond to the same queue expansion device queue in the at least one queue expansion device queue ；

The queue expansion device is used to refresh the head pointer of the corresponding queue expansion device queue, the tail pointer of the first host queue, and the tail pointer of the second host queue;

The network card is configured to receive a second doorbell message from the queue expansion device, where the second doorbell message carries the queue head pointer of the corresponding queue expansion device;

The network card is configured to obtain the head pointer of the corresponding queue expansion device queue from the second doorbell message;

The network card in the forwarding device refreshes the head pointer of the corresponding queue expansion device queue in the at least one set of network card head pointers according to the head pointer of the corresponding queue expansion device queue and the second queue correspondence relationship Corresponding network card head pointer;

The network card is configured to obtain from the corresponding queue expansion device queue according to the corresponding head pointer of the network card after refreshing and the tail pointer corresponding to the corresponding queue expansion device queue in the at least one set of network card tail pointers The first data to be sent and the second data to be sent;

The network card is configured to store the first data to be sent and the second data to be sent in a network card queue corresponding to the corresponding queue expansion device queue according to the corresponding relationship of the second queue.
The system according to claim 13, wherein the first data to be sent is a first direct memory access DMA read descriptor, the second data to be sent is a second DMA read descriptor, and the network card It is also used for storing the first data to be sent and the second data to be sent in the network card queue corresponding to the corresponding queue expansion device queue, according to the first DMA read descriptor and the first Two DMA read descriptors, generating the first DMA read command and the second DMA read command;

The network card is configured to send the first DMA read command and the second DMA read command to a queue expansion device in the forwarding device;

The queue expansion device is configured to obtain first data to be read from the computing device according to the first DMA read command, and obtain second data to be read from the computing device according to the second DMA read command;

The queue expansion device is used to forward the first data to be read and the second data to be read to the network card in the forwarding device.
The system according to claim 13, wherein the first data to be sent is a first DMA write descriptor, and the second data to be sent is a second DMA write descriptor, wherein the first DMA The write descriptor is used to write the first data to be written from the queue expansion device to the computing device, and the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device. Said computing equipment,

The queue expansion device is used to convert the first DMA write descriptor to a first network card DMA write descriptor, and convert the second DMA write descriptor to a second network card DMA write descriptor, wherein the first DMA write descriptor is A DMA write descriptor is used to write the first data to be written from the network card into the queue expansion device, and the second DMA descriptor is used to write the second data to be written from the network card into the queue expansion device. Queue expansion device;

The queue expansion device is configured to store the first network card DMA write descriptor and the second network card DMA write descriptor into the corresponding queue expansion device queue;

The network card is further configured to write the descriptor according to the first network card DMA after storing the first data to be sent and the second data to be sent in the network card queue corresponding to the corresponding queue expansion device queue , Obtain the first fragmented data stored in the network card and corresponding to the DMA write descriptor of the first network card, and obtain the data stored in the network card and the data stored in the network card according to the DMA write descriptor of the second network card. The second fragmented data corresponding to the DMA write descriptor of the second network card, wherein the first data to be written includes n first fragments of data, and the second data to be written includes m second fragments data;

The network card is configured to generate a first DMA write command according to the first network card DMA write descriptor and the first fragment data, and generate a first DMA write command according to the second network card DMA write descriptor and the second fragment data. Two DMA write commands;

The network card is configured to send the first DMA write command and the second DMA write command to the queue expansion device;

The queue expansion device is configured to write the first fragmented data and the second fragmented data into the memory of the queue expansion device according to the first DMA write command and the second DMA write command;

The queue expansion device is configured to generate a third DMA write command according to the first data to be written in the case that the number of first fragmented data is n in the memory of the queue expansion device in the forwarding device , In the case that the number of the second fragmented data is m, a fourth DMA write command is generated according to the second data to be written, wherein the third DMA write command is used for the calculation The device executes and writes the first data to be written into the computing device, and the fourth DMA write command is used for execution by the computing device and writes the first data to be written into the computing device ；

The queue expansion device is configured to send the third DMA write command and the fourth DMA write command to the computing device;

The computing device is configured to write the first data to be written and the second data to be written into the computing device according to the third DMA write command and the fourth DMA write command.
The system according to any one of claims 13-15, wherein the network card is further used for storing the first data to be sent and the second data to be sent in the corresponding queue After expanding the network card queue corresponding to the device queue, sending a completion message and an interrupt message to the queue expansion device;

The queue expansion device is further configured to send the completion message and interrupt to the computing device, so that the computing device refreshes the tail pointers of the first host queue and the second host queue according to the completion message.
A forwarding device, characterized in that the forwarding device is connected to a computing device, the computing device includes a plurality of host queues, and a plurality of data to be sent are stored in the plurality of host queues, wherein the forwarding device includes A receiving unit, an acquiring unit, and a storage unit, the storage unit includes at least one forwarding queue:

The receiving unit is configured to receive a first doorbell message from the computing device, wherein the first doorbell message carries information about a first host queue and information about a second host queue, and the first host queue And the second host queue both belong to the multiple host queues;

The acquiring unit is configured to acquire the information of the first host queue and the information of the second host queue from the first doorbell message, acquire the first to-be-sent data from the first host queue, and Acquiring second data to be sent from the second host queue;

The storage unit is configured to store the first data to be sent and the second data to be sent into the first host queue respectively according to the queue correspondence between the forwarding queue and the host queue, and A forwarding queue corresponding to the second host queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.
The device according to claim 16, wherein the forwarding device further comprises a response unit,

The receiving unit is further configured to receive a device query message from the computing device before the receiving unit receives the first doorbell message from the computing device;

The response unit is configured to respond to the device query message, and the forwarding device reports the number of extended forwarding queues to the computing device, wherein the number of extended forwarding queues is forwarded by the at least one The number of queues and the expansion coefficient are determined, and the expanded number of forwarding queues is used to establish the queue correspondence.
The device according to claim 17, wherein the response unit comprises an attribute register, and the number of the expanded forwarding queues is stored in the attribute register,

The response unit is used to query the attribute register, obtain the expanded number of forwarding queues from the attribute register, and report the expanded number of forwarding queues to the computing device.
The device according to any one of claims 16-19, wherein the information of the first host queue includes a first head pointer, the first head pointer indicates the first data to be sent, and the The information of the second host queue includes a second head pointer, the second head pointer indicates the second data to be sent, the forwarding device stores multiple sets of head pointers and multiple sets of tail pointers, each set of head pointers and each set of tail pointers The group tail pointers respectively correspond to the head pointer and the tail pointer of one of the multiple queues of the computing device, and the forwarding device also stores at least one set of head pointers and at least one set of tail pointers of the at least one queue; The forwarding device further includes a refresh unit,

The acquiring unit is configured to acquire the first head pointer and the second head pointer from the first doorbell message;

The refresh unit is configured to refresh the head pointer corresponding to the first host queue in the multiple sets of head pointers according to the first head pointer;

The acquiring unit is configured to acquire the first host queue from the first host queue according to the head pointer corresponding to the first host queue after the refresh and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers. The first data to be sent;

The refresh unit is configured to refresh the head pointer corresponding to the second host queue in the plurality of groups of head pointers according to the second head pointer;

The acquiring unit is configured to acquire the second host queue from the head pointer corresponding to the second host queue after refreshing and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers. The second data to be sent;

The refresh unit is configured to refresh the tail of the first host queue after storing the first data to be sent and the second data to be sent into the forwarding queues corresponding to the first host queue and the second host queue, respectively Pointer and the tail pointer of the second host queue;

The refresh unit is used to refresh the head pointer of the corresponding forwarding queue;

The refresh unit is configured to send the refreshed tail pointer of the first host queue and the refreshed tail pointer of the second host queue to the computing device.
A queue expansion device, wherein the queue expansion device queue is connected to a computing device and a network card, the queue expansion device includes at least one queue expansion device queue, the network card includes at least one network card queue, and the computing device includes A plurality of host queues, a plurality of data to be sent are stored in the plurality of host queues, and the queue expansion device includes:

A receiving unit, the receiving unit is configured to receive a first doorbell message from the computing device, wherein the first doorbell message carries information about a first host queue and a second host queue, and the first Both the host queue and the second host queue belong to the multiple host queues;

An acquiring unit configured to acquire information about the first host queue and information about the second host queue from the first doorbell message, and acquire first data to be sent from the first host queue , Acquiring second data to be sent from the second host queue;

A storage unit, which is configured to store the first data to be sent and the second data to be sent in the storage unit according to the first queue correspondence between the queue expansion device queue and the host queue. The first host queue and the queue expansion device queue corresponding to the second host queue;

A sending unit, the sending unit is configured to send a second doorbell message to the network card, the second doorbell message carrying the corresponding queue expansion device queue information, so that the network card according to the second doorbell message , Acquiring the first data to be sent and the second data to be sent from the corresponding queue expansion device queue, and according to the second queue correspondence between the network card queue and the queue expansion device queue, The first data to be sent and the second data to be sent are stored in the network card queue corresponding to the corresponding queue expansion device queue, where the corresponding queue expansion device queue belongs to the queue expansion device queue, and the The first host queue and the second host queue correspond to the same queue expansion device queue in the at least one queue expansion device queue, and the corresponding network card queue belongs to the network card queue.
The device according to claim 21, wherein the device further comprises a response unit,

The receiving unit is further configured to receive a first device query message from the computing device before receiving the first doorbell message from the computing device;

The sending unit is configured to send a second device query message to the network card;

The response unit is configured to receive the number of network card queues sent by the network card, wherein the number of network card queues is used to establish the second queue correspondence;

The response unit is configured to report the number of expanded queue expansion device queues to the computing device in response to the first device query message, where the number of expanded queue expansion device queues is determined by the network card The number of queues and the expansion coefficient are determined, and the expanded number of queue expansion device queues is used to establish the first queue correspondence.
The device according to claim 22, wherein the device comprises an attribute register, and the number of the extended forwarding queues is stored in the attribute register;

The response unit is also used to query the attribute register, obtain the number of the expanded queue expansion device queues from the attribute register, and report the number of the expanded queue expansion device queues to the computing device .
The device according to any one of claims 21-23, wherein the information of the first host queue includes a first head pointer, the first head pointer indicating the first data to be sent, and the The information of the second host queue includes a second head pointer, the second head pointer indicates the second to-be-sent data, the queue expansion device stores multiple sets of head pointers and multiple sets of tail pointers, and each set of head pointers and Each set of tail pointers respectively corresponds to the head pointer and the tail pointer of a host queue in the plurality of host queues, and the queue expansion device also stores at least one set of queue expansion device head pointers and at least one set of queue expansion device Tail pointer, the network card stores at least one set of network card head pointers and at least one set of network card tail pointers, the at least one set of queue expansion device head pointers corresponds to the at least one set of network card head pointers, and the at least The tail pointers of a group of queue expansion devices correspond one-to-one with the tail pointers of the at least one group of network cards, and the queue expansion device further includes a refresh unit,

The acquiring unit is configured to acquire the first head pointer and the second head pointer from the first doorbell message;

The refresh unit is configured to refresh the head pointer corresponding to the first host queue in the multiple sets of head pointers according to the first head pointer and the corresponding relationship between the first queue;

The acquiring unit is configured to acquire the first host queue from the first host queue according to the head pointer corresponding to the first host queue after the refresh and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers. The first data to be sent;

The refreshing unit is configured to refresh the head pointer corresponding to the second host queue in the plurality of sets of head pointers according to the second head pointer and the corresponding relationship of the first queue;

The acquiring unit is configured to acquire the second host queue from the head pointer corresponding to the second host queue after refreshing and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers. The second data to be sent;

The storage unit is configured to store the first data to be sent and the second data to be sent in the queue expansion device queues corresponding to the first host queue and the second host queue according to the corresponding relationship of the first queue , Wherein the corresponding queue expansion device queue belongs to the queue expansion device queue, and the first host queue and the second host queue correspond to the same queue expansion device queue in the at least one queue expansion device queue;

The refresh unit is configured to refresh the head pointer of the corresponding queue expansion device queue, the tail pointer of the first host queue, and the tail pointer of the second host queue;

The sending unit is further configured to send the second doorbell message to the network card, wherein the second doorbell message carries the head pointer of the corresponding queue expansion device queue, so that the network card is configured according to the corresponding The head pointer of the queue expansion device queue and the corresponding relationship between the second queue and the corresponding network card head pointer corresponding to the head pointer of the corresponding queue expansion device queue in the at least one set of network card head pointers are refreshed, and the corresponding network card is refreshed A head pointer and a tail pointer corresponding to the corresponding queue expansion device queue in the at least one set of network card tail pointers, acquiring the first data to be sent and the second data to be sent in the corresponding queue expansion device queue, Therefore, according to the second queue correspondence, the first to-be-sent data and the second to-be-sent data are stored in the network card queue corresponding to the corresponding queue expansion device queue.
The device according to claim 24, wherein the first data to be sent is a first direct memory access DMA read descriptor, and the second data to be sent is a second DMA read descriptor,

The receiving unit is further configured to receive a first DMA read command and a second DMA read command sent by the network card, where the first DMA read command is generated by the network card according to the first DMA descriptor , The second DMA read command is generated by the network card according to the second DMA descriptor;

The acquiring unit is further configured to acquire first data to be read from the computing device according to a first DMA read command, and acquire second data to be read from the computing device according to a second DMA read command;

The sending unit is further configured to forward the first data to be read and the second data to be read to the network card.
The device according to claim 24, wherein the first data to be sent is a first DMA write descriptor, and the second data to be sent is a second DMA write descriptor, wherein the first DMA The write descriptor is used to write the first data to be written from the queue expansion device to the computing device, and the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device. The computing device, the device further includes a conversion unit,

The conversion unit is configured to convert the first DMA write descriptor to a first network card DMA write descriptor, and convert the second DMA write descriptor to a second network card DMA write descriptor, wherein the first The DMA write descriptor is used to write the first data to be written from the network card into the queue expansion device, and the second DMA descriptor is used to write the second data to be written from the network card into the queue Expansion equipment;

The storage unit is configured to store the first network card DMA write descriptor and the second network card DMA write descriptor into the corresponding queue expansion device queue;

The receiving unit is configured to receive a first DMA write command and the second DMA write command, wherein the first DMA write command is obtained by the network card according to the first network card DMA write descriptor and stored in the In the network card, the first fragment data corresponding to the first network card DMA write descriptor is generated according to the first network card DMA write descriptor and the first DMA write command generated by the first fragment data, the The second DMA write command is after the network card obtains the second fragment data stored in the network card corresponding to the second network card DMA write descriptor according to the second network card DMA write descriptor, and then according to the The second network card DMA write descriptor and the second DMA write command generated by the second fragmented data, wherein the first to-be-written data includes n pieces of first fragmented data, and the second to-be-written data Including m second fragment data;

The storage unit is configured to write the first fragmented data and the second fragmented data into the memory of the queue expansion device according to the first DMA write command and the second DMA write command;

The conversion unit is configured to generate a third DMA write command according to the first data to be written in the case where the number of first fragmented data in the memory of the queue expansion device is n. When the number of two-slice data is m, a fourth DMA write command is generated according to the second data to be written, where the third DMA write command is used for the computing device to execute and The first data to be written is written into the computing device, and the fourth DMA write command is used for the computing device to execute and write the first data to be written into the computing device;

The sending unit is configured to send the third DMA write command and the fourth DMA write command to the computing device, so that the computing device according to the third DMA write command and the fourth DMA write command, The first data to be written and the second data to be written are written into the computing device.
The device according to any one of claims 24-26, wherein:

The receiving unit is further configured to receive a completion message and an interrupt sent by the network card, wherein, the completion message and the interrupt are when the network card is storing the first data to be sent and the second data to be sent in and Generated after the corresponding network card queue corresponding to the corresponding queue expansion device queue;

The sending unit is further configured to send a completion message and an interrupt to the computing device, so that the computing device refreshes the tail pointers of the first host queue and the second host queue according to the completion message.
A forwarding device, the device includes a processor and a memory, including a processor and a memory, and the processor executes the code in the memory to implement the method according to any one of claims 1 to 8.
A computer program product, characterized in that, when the computer program product is read and executed by a computing device, the method according to any one of claims 1 to 8 will be implemented.
A computer non-transitory storage medium, characterized by comprising instructions, when the instructions are run on a computing device, the computing device is made to implement the method according to any one of claims 1 to 8.
A forwarding device, the device comprising a logic circuit, and the forwarding device executes the method according to any one of claims 1 to 8 through the logic circuit.