WO2021063154A1 - Procédé et système de communication de file d'attente, et dispositifs s'y rapportant - Google Patents

Procédé et système de communication de file d'attente, et dispositifs s'y rapportant Download PDF

Info

Publication number
WO2021063154A1
WO2021063154A1 PCT/CN2020/113497 CN2020113497W WO2021063154A1 WO 2021063154 A1 WO2021063154 A1 WO 2021063154A1 CN 2020113497 W CN2020113497 W CN 2020113497W WO 2021063154 A1 WO2021063154 A1 WO 2021063154A1
Authority
WO
WIPO (PCT)
Prior art keywords
queue
data
host
network card
forwarding
Prior art date
Application number
PCT/CN2020/113497
Other languages
English (en)
Chinese (zh)
Inventor
李涛
江锦毅
李生
周宇
杨荣玉
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2021063154A1 publication Critical patent/WO2021063154A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/546—Message passing systems or structures, e.g. queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Definitions

  • This application relates to the field of computers, and in particular to a method, system and related equipment for queue communication.
  • Cloud computing is based on a configurable shared computing resource pool to provide users with convenient, on-demand access services.
  • cloud computing service providers will provide a variety of virtual machine specifications, and flexibly distribute virtual machines of different specifications according to user needs. For example, when users have greater demands on the network, users can be divided into a large-format virtual machine with a 64-bit Virtual Central Processing Unit (vCPU) and 128GB of storage space.
  • vCPU Virtual Central Processing Unit
  • the virtual machine runs on the computing device, and configuring the virtual machine also includes allocating multiple host queues on the computing device to the virtual machine.
  • a virtual machine communicates with a device other than the computing device where it is located, it needs to send data to the network card.
  • For a virtual machine with a configured network card queue when you need to send data to the network card, you can directly send the data to be sent to the host queue assigned to the virtual machine, and then the host queue assigned to the virtual machine and the network card queue on the network card pass The queue communication mechanism copies the to-be-sent data in the host queue allocated to the virtual machine to the network card queue of the network card to complete a data communication between the virtual machine and the network card.
  • the host queue on the computing device and the network card queue of the network card are one-to-one mapping.
  • the data to be sent from the virtual machine stored in a host queue of the computing device can only be sent to the network card A network card queue on the.
  • the performance of queue communication is restricted by the forwarding ability of the host queue on the computing device and the forwarding ability of the network card queue of the network card.
  • the virtual machine sends data to the network card, it is subject to the forwarding ability of the virtual machine vCPU.
  • the forwarding ability of the queue assigned to the virtual machine on the host is usually weaker than the forwarding ability of the queue on the network card, which causes the forwarding performance of the queue on the network card to not be fully utilized. , Affect the efficiency of communication.
  • the present application provides a queue communication method, system and related equipment, which are used to solve the problem that the queue forwarding performance on the network card cannot be fully utilized, which causes the communication efficiency to be affected.
  • a method for queue communication is provided, the method is applied to a forwarding device, the forwarding device is connected to a computing device, the forwarding device includes at least one forwarding queue, and the computing device includes multiple host queues, A plurality of data to be sent are stored in the plurality of host queues, and the method includes:
  • the forwarding device receives a first doorbell message from the computing device, where the first doorbell message carries information about a first host queue and information about a second host queue, and the first host queue and the The second host queues all belong to the multiple host queues;
  • the forwarding device obtains the information of the first host queue and the information of the second host queue from the first doorbell message, obtains the first data to be sent from the first host queue, and obtains the first data to be sent from the first host queue. 2. Obtain the second data to be sent from the host queue;
  • the forwarding device stores the first to-be-sent data and the second to-be-sent data into the first host queue and the second host respectively according to the queue correspondence between the forwarding queue and the host queue A forwarding queue corresponding to the queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.
  • the method before the forwarding device receives the first doorbell message from the computing device, the method further includes: the forwarding device receives a device query message from the computing device; responding to the device query Message, the forwarding device reports the number of extended forwarding queues to the computing device, where the number of extended forwarding queues is determined by the number of the at least one forwarding queue and the expansion coefficient, and the extended The subsequent number of forwarding queues is used to establish the queue correspondence.
  • the forwarding device includes an attribute register, and the number of the extended forwarding queues is stored in the attribute register; the forwarding device reporting the number of extended forwarding queues to the computing device includes : The forwarding device queries the attribute register, obtains the number of the extended forwarding queues from the attribute register, and reports the number of the extended forwarding queues to the computing device.
  • the forwarding device sends the expanded number of forwarding queues to the computing device, so that the computing device can allocate multiple host queues to one forwarding queue to generate a queue correspondence relationship, without any upgrade of the computing device. , It can solve the problem that the queue forwarding performance on the network card cannot be fully exerted because the host queue forwarding performance is lower than the network card queue, and the communication efficiency is improved.
  • the information of the first host queue includes a first head pointer, the first head pointer indicates the first to-be-sent data, and the information about the second host queue includes a second head pointer, so
  • the second head pointer indicates the second data to be sent, and multiple sets of head pointers and multiple sets of tail pointers are stored in the forwarding device, and each set of head pointers and each set of tail pointers respectively correspond to multiple hosts of the computing device
  • the head pointer and the tail pointer of a host queue in the queue, the forwarding device also stores at least one set of head pointers and at least one set of tail pointers of the at least one forwarding queue;
  • Obtaining the information of the first host queue and the information of the second host queue in the message includes: the forwarding device obtains the first head pointer and the second head pointer from the first doorbell message;
  • the forwarding queue acquiring the first to-be-sent data from the first host queue includes: the forwarding device refreshes the head pointer corresponding to the first host queue in the plurality of
  • the forwarding device includes a queue expansion device and a network card
  • the at least one forwarding queue includes at least one queue expansion device queue and at least one network card queue
  • the at least one queue expansion device queue is located in the queue expansion device.
  • the at least one network card queue is located in the network card
  • the at least one set of head pointers and the at least one set of tail pointers are stored in the queue expansion device
  • the queue correspondence relationship includes a first queue correspondence relationship and a second queue
  • the first queue correspondence is the correspondence between the multiple host queues and the at least one queue expansion device queue
  • the second queue correspondence is the at least one network card queue and the at least one queue
  • the corresponding relationship between the expansion device queues, the at least one network card queue corresponds to the at least one queue expansion device queue
  • the queue expansion device stores multiple sets of head pointers and multiple sets of tail pointers, each set of head pointers and each set of tail pointers
  • the group tail pointers respectively correspond to the head pointer and the tail pointer of a host queue in the pluralit
  • the queue expansion device also stores at least one set of queue expansion device head pointers and at least one set of queue expansion device tails.
  • Pointers the network card stores at least one set of network card head pointers and at least one set of network card tail pointers, the at least one set of queue expansion device head pointers corresponds to the at least one set of network card head pointers, and the at least one The tail pointer of the group queue expansion device corresponds to the tail pointer of the at least one group of network cards in a one-to-one correspondence;
  • the forwarding device obtains the information of the first host queue and the information of the second host queue from the first doorbell message , Including: the queue expansion device in the forwarding device obtains the first head pointer and the second head pointer from the first doorbell message; the forwarding device obtains the first head pointer from the first host queue
  • the data to be sent includes: the queue expansion device in the forwarding device refreshes the head pointer corresponding to the first host queue among the multiple sets of head pointers according to the first head pointer and
  • a plurality of host queues in the computing device and a queue expansion device queue in the queue expansion device generate a first queue correspondence
  • a queue expansion device queue in the queue expansion device and a network card queue in a network card generate a first queue correspondence relationship.
  • the first data to be sent is a first direct memory access DMA read descriptor
  • the second data to be sent is a second DMA read descriptor
  • the first data to be sent is And after the second data to be sent is stored in the network card queue corresponding to the corresponding queue expansion device queue, the method further includes: the network card in the forwarding device reads the descriptor according to the first DMA and the second DMA read descriptor to generate a first DMA read command and a second DMA read command; the network card in the forwarding device sends the first DMA read command and the second DMA read command to the queue expansion device in the forwarding device; The queue expansion device in the forwarding device obtains first data to be read from the computing device according to the first DMA read command, and obtains second data to be read from the computing device according to the second DMA read command The queue expansion device in the forwarding device forwards the first data to be read and the second data to be read to the network card in the forwarding device.
  • the first data to be sent is a first DMA write descriptor
  • the second data to be sent is a second DMA write descriptor, wherein the first DMA write descriptor is used to transfer the first DMA write descriptor.
  • a data to be written is written from the queue expansion device to the computing device
  • the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device
  • the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device.
  • the storing of the first data to be sent and the second data to be sent in the first host queue and the queue expansion device queue corresponding to the second host queue includes: the queue expansion device in the forwarding device transfers the first The DMA write descriptor is converted into a first network card DMA write descriptor, and the second DMA write descriptor is converted into a second network card DMA write descriptor, where the first DMA write descriptor is used to transfer the first to be written Incoming data is written from the network card to the queue expansion device, and the second DMA descriptor is used to write the second data to be written from the network card to the queue expansion device; the queue expansion in the forwarding device
  • the device stores the first network card DMA write descriptor and the second network card DMA write descriptor in the corresponding queue expansion device queue; the storing the first data to be sent and the second data to be sent in After the network card queue corresponding to the corresponding queue expansion device queue, the method further includes: the network card in the forwarding device obtains the data stored in the network card
  • the DMA write descriptor of the second network card obtain the second piece of data stored in the network card and corresponding to the DMA write descriptor of the second network card.
  • Fragmented data wherein the first to-be-written data includes n first fragmented data, and the second to-be-written data includes m second fragmented data;
  • the network card in the forwarding device is based on the The first network card DMA write descriptor and the first fragmented data generate a first DMA write command, and the second network card DMA write descriptor and the second fragmented data generate a second DMA write command;
  • the forwarding The network card in the device sends the first DMA write command and the second DMA write command to the queue expansion device;
  • n a third DMA write command is generated according to the first data to be written, and in the case that the number of the second fragmented data is m,
  • the queue expansion device stores the first piece of data sent by the network card in the memory, and when all n pieces of data of the first data to be written are collected, the first data to be written is sent
  • the data received by the network card is usually fragmented data, thereby avoiding the problem that the computing device receives out-of-sequence of the first fragmented data and causing the computing device to deal with the problem of heavy processing pressure, thereby improving the efficiency of queue communication.
  • the method further includes: in the forwarding device The network card sends a completion message and an interrupt message to the queue expansion device; the queue expansion device in the forwarding device sends the completion message and interrupt message to the computing device, so that the computing device refreshes the first host according to the completion message The tail pointer of the queue and the second host queue.
  • a queue communication system in a second aspect, includes a forwarding device and a computing device, the forwarding device is connected to the computing device, the forwarding device includes at least one forwarding queue, and the computing device includes multiple host queues , Multiple data to be sent are stored in the multiple host queues, where:
  • the computing device is configured to send a first command message to the forwarding device, where the first doorbell message carries information about the first host queue and the second host queue, and the first host queue and the all The second host queues all belong to the multiple host queues;
  • the forwarding device is used to receive the first doorbell message from the computing device
  • the forwarding device is configured to obtain the information of the first host queue and the information of the second host queue from the first doorbell message, obtain the first data to be sent from the first host queue, and obtain the first data to be sent from the first host queue. Acquiring the second to-be-sent data from the second host queue;
  • the forwarding device is configured to store the first data to be sent and the second data to be sent into the first host queue and the first host queue according to the queue correspondence between the forwarding queue and the host queue. Two forwarding queues corresponding to the host queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.
  • the computing device is configured to send a device query message to the forwarding device before sending the first doorbell message to the forwarding device; the forwarding device is configured to receive a device query from the computing device Message; the forwarding device is used to report the number of extended forwarding queues to the computing device in response to the device query message, wherein the number of extended forwarding queues is determined by the number of the at least one forwarding queue The number and expansion coefficient are determined, and the expanded number of forwarding queues is used to establish the queue correspondence relationship.
  • the forwarding device includes an attribute register, and the number of the expanded forwarding queues is stored in the attribute register; the forwarding device is used to query the attribute register and obtain it from the attribute register The number of the extended forwarding queues is reported to the computing device.
  • the information of the first host queue includes a first head pointer, the first head pointer indicates the first to-be-sent data, and the information about the second host queue includes a second head pointer, so
  • the second head pointer indicates the second data to be sent
  • multiple sets of head pointers and multiple sets of tail pointers are stored in the forwarding device, and each set of head pointers and each set of tail pointers respectively correspond to multiple hosts of the computing device
  • the head pointer and the tail pointer of a host queue in the queue, the forwarding device also stores at least one set of head pointers and at least one set of tail pointers of the at least one forwarding queue; Acquire the first head pointer and the second head pointer in a doorbell message;
  • the forwarding device is configured to refresh the head pointers corresponding to the first host queue in the multiple sets of head pointers according to the first head pointers
  • the forwarding device is used to obtain the first host queue from the first host queue according to the head pointer corresponding to the first host queue after being refreshed, and the
  • the forwarding device includes a queue expansion device and a network card
  • the at least one forwarding queue includes at least one queue expansion device queue and at least one network card queue
  • the at least one queue expansion device queue is located in the queue expansion device.
  • the at least one network card queue is located in the network card
  • the at least one set of head pointers and the at least one set of tail pointers are stored in the queue expansion device
  • the queue correspondence relationship includes a first queue correspondence relationship and a second queue
  • the first queue correspondence is the correspondence between the multiple host queues and the at least one queue expansion device queue
  • the second queue correspondence is the at least one network card queue and the at least one queue
  • the corresponding relationship between the expansion device queues, the at least one network card queue corresponds to the at least one queue expansion device queue
  • the queue expansion device stores multiple sets of head pointers and multiple sets of tail pointers, each set of head pointers and each set of tail pointers
  • the group tail pointers respectively correspond to the head pointer and the tail pointer of a host queue in the pluralit
  • the queue expansion device also stores at least one set of queue expansion device head pointers and at least one set of queue expansion device tails.
  • Pointers the network card stores at least one set of network card head pointers and at least one set of network card tail pointers, the at least one set of queue expansion device head pointers corresponds to the at least one set of network card head pointers, and the at least one The tail pointer of the group queue expansion device corresponds to the at least one group of network card tail pointers one-to-one;
  • the queue expansion device is configured to obtain the first head pointer and the second head pointer from the first doorbell message;
  • the queue expansion device is configured to refresh the head pointer corresponding to the first host queue in the multiple sets of head pointers according to the first head pointer and the first queue correspondence;
  • the queue expansion device is configured to Acquiring the first data to be sent from the first host queue according to the head pointer corresponding to the first host queue after the refresh and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers;
  • the first data to be sent is a first direct memory access DMA read descriptor
  • the second data to be sent is a second DMA read descriptor
  • the network card is also used to After the first data to be sent and the second data to be sent are stored in the network card queue corresponding to the corresponding queue expansion device queue, they are generated according to the first DMA read descriptor and the second DMA read descriptor
  • the first DMA read command and the second DMA read command the network card is used to send the first DMA read command and the second DMA read command to the queue expansion device in the forwarding device
  • the queue expansion device is used for The first DMA read command obtains first data to be read from the computing device, and obtains second data to be read from the computing device according to the second DMA read command
  • the queue expansion device is used to The first data to be read and the second data to be read are forwarded to the network card in the forwarding device.
  • the first data to be sent is a first DMA write descriptor
  • the second data to be sent is a second DMA write descriptor, wherein the first DMA write descriptor is used to transfer the first DMA write descriptor.
  • a data to be written is written from the queue expansion device to the computing device
  • the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device
  • the queue The expansion device is used to convert the first DMA write descriptor to a first network card DMA write descriptor, and convert the second DMA write descriptor to a second network card DMA write descriptor, wherein the first DMA write descriptor
  • the descriptor is used to write the first data to be written from the network card into the queue expansion device
  • the second DMA descriptor is used to write the second data to be written from the network card into the queue expansion device
  • the queue expansion device is used to store the first network card DMA write descriptor and the second network card DMA write descriptor into
  • the network card is configured to A network card DMA write descriptor and the first fragment data generate a first DMA write command, and a second DMA write command is generated according to the second network card DMA write descriptor and the second fragment data;
  • the network card uses Sending the first DMA write command and the second DMA write command to the queue expansion device;
  • the queue expansion device is configured to transfer all data according to the first DMA write command and the second DMA write command
  • the first fragmented data and the second fragmented data are written into the memory of the queue expansion device; the queue expansion device is used to store the first fragmented data in the memory of the queue expansion device in the forwarding device
  • a third DMA write command is generated according to the first data to be written, and when the number of the second fragmented data is m, according to the second data to be written , Generate a fourth DMA write command
  • the network card is further configured to send the first data to be sent and the second data to be sent to the network card queue corresponding to the corresponding queue expansion device queue after storing the first data to be sent and the second data to be sent to the queue
  • the expansion device sends a completion message and an interrupt message; the queue expansion device is also used to send the completion message and interrupt to the computing device, so that the computing device refreshes the first host queue and the second host according to the completion message The tail pointer of the queue.
  • a forwarding device is provided, the forwarding device is connected to a computing device, the forwarding device includes at least one forwarding queue, the computing device includes multiple host queues, and how many host queues are stored in the multiple host queues. Pieces of data to be sent, wherein the forwarding device includes:
  • a receiving unit configured to receive a first doorbell message from the computing device, wherein the first doorbell message carries information about a first host queue and a second host queue, and the first Both the host queue and the second host queue belong to the multiple host queues;
  • An acquiring unit configured to acquire information about the first host queue and information about the second host queue from the first doorbell message, and acquire first data to be sent from the first host queue , Acquiring second data to be sent from the second host queue;
  • a storage unit configured to store the first data to be sent and the second data to be sent in the first host respectively according to the queue correspondence between the forwarding queue and the host queue
  • the queue and the forwarding queue corresponding to the second host queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.
  • the forwarding device further includes a response unit, and the receiving unit is further configured to receive a device query message from the computing device before the receiving unit receives the first doorbell message from the computing device
  • the response unit is used to respond to the device query message, and the forwarding device reports the number of the extended forwarding queues to the computing device, wherein the number of the extended forwarding queues is determined by the at least one The number of forwarding queues and the expansion coefficient are determined, and the expanded number of forwarding queues is used to establish the queue correspondence.
  • the forwarding device includes an attribute register, the number of the expanded forwarding queues is stored in the attribute register, and the response unit is used to query the attribute register and obtain it from the attribute register. The number of the extended forwarding queues is reported to the computing device.
  • the information of the first host queue includes a first head pointer, the first head pointer indicates the first to-be-sent data, and the information about the second host queue includes a second head pointer, so
  • the second head pointer indicates the second data to be sent, and multiple sets of head pointers and multiple sets of tail pointers are stored in the forwarding device, and each set of head pointers and each set of tail pointers respectively correspond to multiple queues of the computing device
  • the forwarding device also stores at least one set of head pointers and at least one set of tail pointers of the at least one queue;
  • the forwarding device further includes a refreshing unit, and the acquiring unit Is used to obtain the first head pointer and the second head pointer from the first doorbell message;
  • the refresh unit is used to refresh the first head pointer in the plurality of sets of head pointers according to the first head pointer
  • the acquiring unit is used to obtain the head pointer corresponding to the first host queue after being refreshed, and the tail pointer
  • the first data to be sent is acquired from a host queue; the refresh unit is configured to refresh the head pointer corresponding to the second host queue in the plurality of sets of head pointers according to the second head pointer; the acquisition unit uses According to the head pointer corresponding to the second host queue after being refreshed, and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers, the second data to be sent is acquired from the second host queue
  • the refresh unit is used to refresh the first host queue after storing the first data to be sent and the second data to be sent in the corresponding forwarding queues of the first host queue and the second host queue respectively The tail pointer and the tail pointer of the second host queue; the refresh unit is used to refresh the head pointer of the corresponding forwarding queue; the refresh unit is used to refresh the tail pointer of the first host queue and The refreshed tail pointer of the second host queue is sent to the computing device.
  • a queue expansion device is provided.
  • the queue expansion device is connected to a computing device and connected to a network card, the queue expansion device is connected to a computing device, the network card is connected to the computing device, and the queue expands
  • the device includes at least one queue expansion device queue, the network card includes at least one network card queue, the computing device includes a plurality of host queues, a plurality of data to be sent are stored in the plurality of host queues, and the queue expansion device includes:
  • a receiving unit configured to receive a first doorbell message from the computing device, wherein the first doorbell message carries information about a first host queue and a second host queue, and the first Both the host queue and the second host queue belong to the multiple host queues;
  • An acquiring unit configured to acquire information about the first host queue and information about the second host queue from the first doorbell message, and acquire first data to be sent from the first host queue , Acquiring second data to be sent from the second host queue;
  • a storage unit which is configured to store the first data to be sent and the second data to be sent in the storage unit according to the first queue correspondence between the queue expansion device queue and the host queue.
  • a sending unit the sending unit is configured to send a second doorbell message to the network card, the second doorbell message carrying the corresponding queue expansion device queue information, so that the network card according to the second doorbell message , Acquiring the first data to be sent and the second data to be sent from the corresponding queue expansion device queue, and according to the second queue correspondence between the network card queue and the queue expansion device queue,
  • the first data to be sent and the second data to be sent are stored in the network card queue corresponding to the corresponding queue expansion device queue, where the corresponding queue expansion device queue belongs to the queue expansion device queue, and the The first host queue and the second host queue correspond to the same queue expansion device queue in the at least one queue expansion device queue, and the corresponding network card queue belongs to the network card queue.
  • the device further includes a response unit, and the receiving unit is further configured to receive a first device query message from the computing device before receiving the first doorbell message from the computing device;
  • the sending unit is used to send a second device query message to the network card;
  • the response unit is used to receive the number of network card queues sent by the network card, wherein the number of network card queues is used to establish the second queue correspondence
  • the response unit is configured to report the number of expanded queue expansion device queues to the computing device in response to the first device query message, wherein the number of expanded queue expansion device queues is determined by the The number of network card queues and the expansion coefficient are determined, and the expanded number of queue expansion device queues is used to establish the first queue correspondence relationship.
  • the device includes an attribute register, and the number of the expanded forwarding queues is stored in the attribute register; the response unit is also used to query the attribute register and obtain it from the attribute register The number of queues of the expanded queue expansion device is reported to the computing device.
  • the information of the first host queue includes a first head pointer
  • the first head pointer indicates the first to-be-sent data
  • the information about the second host queue includes a second head pointer
  • the queue expansion device stores multiple sets of head pointers and multiple sets of tail pointers, and each set of head pointers and each set of tail pointers respectively correspond to the multiple host queues
  • the head pointer and the tail pointer of a host queue in, the queue expansion device also stores at least one set of queue expansion device head pointers and at least one set of queue expansion device tail pointers, and at least one set of network cards is stored in the network card
  • the at least one set of queue extension device head pointers correspond to the at least one set of network card head pointers
  • the at least one set of queue extension device tail pointers corresponds to the at least one There is a one-to-one correspondence between the tail pointers of the network card
  • the queue expansion device further includes
  • the storage unit is used to obtain the second data to be sent according to In the first queue correspondence, the first data to be sent and the second data to be sent are stored in the first host queue and the queue expansion device queue corresponding to the second host queue, wherein the corresponding The queue expansion device queue belongs to the queue expansion device queue, and the first host queue and the second host queue correspond to the same queue expansion device queue in the at least one queue expansion device queue;
  • the refresh unit is used for refreshing The head pointer of the corresponding queue expansion device queue, the tail pointer of the first host queue, and the tail pointer of the second host queue;
  • the sending unit is further configured to send the second doorbell message to the network card, wherein The second doorbell message carries the head pointer of the corresponding queue expansion device queue, so that the network card refreshes the at least one queue according to the head pointer of
  • the data and the second data to be sent are stored in the network card queue corresponding to the corresponding queue expansion device queue according to the corresponding relationship of the second queue.
  • the first data to be sent is a first direct memory access DMA read descriptor
  • the second data to be sent is a second DMA read descriptor
  • the receiving unit is further configured to receive the The first DMA read command and the second DMA read command sent by the network card, wherein the first DMA read command is generated by the network card according to the first DMA descriptor, and the second DMA read command is The network card is generated according to the second DMA descriptor
  • the obtaining unit is further configured to obtain the first data to be read from the computing device according to the first DMA read command, and from the computing device according to the second DMA read command Obtain second data to be read
  • the sending unit is further configured to forward the first data to be read and the second data to be read to the network card.
  • the first data to be sent is a first DMA write descriptor
  • the second data to be sent is a second DMA write descriptor
  • the first DMA write descriptor is used to transfer the first DMA write descriptor.
  • a data to be written is written from the queue expansion device to the computing device, and the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device, and the device It also includes a conversion unit configured to convert the first DMA write descriptor to a first network card DMA write descriptor, and convert the second DMA write descriptor to a second network card DMA write descriptor, wherein ,
  • the first DMA write descriptor is used to write the first data to be written from the network card to the queue expansion device, and the second DMA descriptor is used to write the second data to be written from the network card Write to the queue expansion device;
  • the storage unit is used to store the first network card DMA write descriptor and the second network card DMA write descriptor
  • the storage unit is used to write the first fragmented data and the second fragmented data into the memory of the queue expansion device according to the first DMA write command and the second DMA write command
  • the conversion unit is used to generate a third DMA write command based on the first data to be written when the number of first fragmented data is n in the memory of the queue expansion device In the case where the number of the second fragmented data is m, a fourth DMA write command is generated according to the second data to be written, where the third DMA write command is used for the computing device to execute and Writing the first data to be written into the computing device, and the fourth DMA write command is used for execution by the computing device and writing the first data to be written into the computing device;
  • the sending unit is configured to send the third DMA write command and the fourth DMA write command to the computing device, so that the computing device transfers all data according to the third DMA write command and the fourth DMA write command.
  • the first data to be written and the second data to be written are written into the computing device.
  • the receiving unit is further configured to receive a completion message and an interrupt sent by the network card, wherein, the completion message and the interrupt indicate that the network card is sending the first data to be sent and the second The data to be sent is generated after being stored in the network card queue corresponding to the corresponding queue expansion device queue; the sending unit is also used to send a completion message and an interrupt to the computing device, so that the computing device according to the completion message Refresh the tail pointers of the first host queue and the second host queue.
  • a forwarding device in a fifth aspect, includes a processor and a memory, and the processor executes the code in the memory to implement the method described in the first aspect.
  • a computer program product is provided.
  • the computer program product is read and executed by a computing device, the method described in the first aspect is implemented.
  • a computer non-transitory storage medium including instructions, which when run on a computing device, cause the computing device to implement the method described in the first aspect.
  • a forwarding device in an eighth aspect, the device includes a logic circuit, and the forwarding device implements the method described in the first aspect through the logic circuit.
  • a queue expansion device in a ninth aspect, includes a processor and a memory, and the processor executes the code in the memory to implement the queue expansion device described in the fourth aspect.
  • a computer program product is provided, which is characterized in that, when the computer program product is read and executed by a computing device, the queue expansion device described in the fourth aspect is implemented.
  • a computer non-transitory storage medium which is characterized in that it includes an instruction that, when the instruction runs on a computing device, causes the computing device to implement the queue expansion device described in the fourth aspect.
  • a transfer queue expansion device in a twelfth aspect, the device includes a logic circuit, and the forwarding device implements the queue expansion device described in the fourth aspect through the logic circuit.
  • Figure 1 is a schematic diagram of the logical structure of a network system to which the solution provided in this application is applicable;
  • FIG. 2 is a schematic diagram of the logical structure of a forwarding device for queue communication in a network system provided by this application;
  • FIG. 3 is a schematic diagram of the logical structure of a queue expansion device for queue communication in a network system provided by the present application;
  • Figure 4 is a flow chart of the steps of a queue communication method provided by the present application.
  • 5A-5C are schematic diagrams of a queue structure
  • Figure 6 is a flow chart of the steps of another queue communication method provided by the present application.
  • FIG. 7 is a flowchart of the steps of obtaining the first data to be sent and the second data to be sent according to the doorbell message in a queue communication method provided by the present application;
  • FIG. 8 is a flow chart of the steps of a queue communication method provided by the present application in an application scenario
  • FIG. 9 is a flow chart of the steps of a queue communication method provided by the present application in another application scenario.
  • FIG. 10 is a schematic structural diagram of a forwarding device provided by this application.
  • FIG. 11 is a schematic structural diagram of a queue expansion device provided by the present application.
  • FIG. 12 is a schematic diagram of the hardware structure of a forwarding device provided by the present application.
  • FIG. 13 is a schematic diagram of the hardware structure of a queue expansion device provided by the present application.
  • FIG. 1 is a schematic diagram of the logical structure of a network system applicable to this application.
  • the network system includes: a computing device 110 and a computing device 120, a forwarding device 210 and a forwarding device 220, and a network 300, wherein, The computing device 110 and the forwarding device 210 are connected and communicated, the forwarding device 210 is connected and communicated with the network 300, the computing device 120 is connected and communicated with the forwarding device 220, and the forwarding device 220 is connected and communicated with the network 300.
  • FIG. 1 is a schematic diagram of the logical structure of a network system applicable to this application.
  • the network system includes: a computing device 110 and a computing device 120, a forwarding device 210 and a forwarding device 220, and a network 300, wherein, The computing device 110 and the forwarding device 210 are connected and communicated, the forwarding device 210 is connected and communicated with the network 300, the computing device 120 is connected and communicated with the forwarding device 220, and
  • Figure 1 only shows two computing devices and two forwarding devices, and each computing device is connected to only one forwarding device.
  • the network system may include one or more computing devices and The number of one or more forwarding devices, computing devices, and forwarding devices is not a limitation to the embodiments of this application.
  • Each computing device can also be connected to multiple forwarding devices.
  • the system can make adjustments according to user needs as described above. This application does not Make specific restrictions.
  • Figure 1 is only for the convenience of description and only shows part of the resources.
  • the computing device shown in Figure 1 may also include hardware resources such as memory and hard disk, as well as software resources such as operating systems and applications that need to be run. Go into details again.
  • the data to be sent on the computing device 110 side can be sent to the forwarding device 210, and the network card 210 sends to the network card 220 through the network 300 connected to it.
  • the forwarding device 220 may receive the data to be sent through the network 300 and send the data to be sent to the computing device 120, so that the computing device 110 sends the data to be sent to the computing device 120.
  • This application mainly focuses on the communication process between the computing device and the forwarding device, so the process of data transmission by the forwarding device through the network will not be repeated. The process of implementing the communication between the computing device and the forwarding device in this application will be explained in detail below.
  • the forwarding device may be a network card.
  • the communication system shown in FIG. 1 may be as shown in FIG. 2, where the computing device 110 includes a plurality of host queues 111 (shown in the figure 4 host queues Q00-Q03), the forwarding device 210 includes at least one forwarding queue 211 (only two forwarding queues Q10 and Q11 are drawn in the figure), and the at least one forwarding queue 211 and the multiple hosts There is a queue correspondence between the queues 111.
  • one forwarding queue 211 may have a queue correspondence with multiple host queues 211. For example, there is a queue between the host queues Q00-Q01 and the forwarding queue Q10 shown in FIG.
  • FIG. 1 only shows the computing device 110 and the forwarding device for ease of understanding. 210, I won't repeat it here.
  • the computing device 110 when it needs to transmit the data to be sent to the forwarding device 210, it can first put the data to be sent into the host queue, such as in the host queues Q00 and Q01 in FIG. 2, and then according to the queue correspondence, The forwarding queue Q10 corresponding to the host queues Q00 and Q01 is determined, and the data to be sent is transferred from the host queues Q00 and Q01 to the forwarding queue Q10, so as to transmit the data to be sent from the computing device 110 to the forwarding device 210.
  • the host queue such as in the host queues Q00 and Q01 in FIG. 2
  • the forwarding queue Q10 corresponding to the host queues Q00 and Q01 is determined, and the data to be sent is transferred from the host queues Q00 and Q01 to the forwarding queue Q10, so as to transmit the data to be sent from the computing device 110 to the forwarding device 210.
  • each forwarding queue can generate a queue correspondence with multiple host queues, thereby avoiding the problem that the forwarding ability of the host queue is weaker than the forwarding ability of the network card queue.
  • the communication efficiency is low, so that the forwarding performance of the network card queue can be brought into full play.
  • the forwarding device may also include a queue expansion device and a network card.
  • the communication system shown in FIG. 1 may be as shown in FIG. 3, where the computing device 110 includes multiple host queues 111 ( 4 host queues Q00-Q03 are drawn in the figure), the forwarding device 210 includes a queue expansion device 2101 and a network card 2102, wherein the queue expansion device 2010 includes at least one queue expansion device Queue 2111 (only 2 queues are drawn in the figure Expanded device queues Q10 and Q11), the network card 2102 includes at least one network card queue 2112 (only two network card queues Q20 and Q21 are drawn in the figure), and the at least one queue expands the device queue 2111 and the multiple host queues 111 There is a first queue correspondence between them, and there is a second queue correspondence between the at least one network card queue 2112 and the at least one queue expansion device queue 2111.
  • one queue expansion device queue 2111 may have a first queue correspondence relationship with multiple host queues 211, for example, there is a queue correspondence relationship between the host queues Q00-Q01 and the queue expansion device queue Q10 shown in FIG. 3;
  • One queue expansion device queue 2111 may have a second queue correspondence relationship with one network card queue 2112. For example, there is a queue correspondence relationship between the queue expansion device queue Q10 and the network card queue Q20 shown in FIG. 3.
  • the computing device 110 when it needs to transmit the data to be sent to the network card 2112, it can first put the data to be sent into the host queue, for example, into the host queues Q00 and Q01 in FIG. 3, and then correspond to the first queue Relationship, determine the queue expansion device queue Q10 corresponding to the host queues Q00 and Q01, so as to transfer the data to be sent from the host queues Q00 and Q01 to the queue expansion device queue Q00; then determine the queue expansion according to the second queue correspondence relationship
  • the network card queue Q20 corresponding to the device queue Q10 transfers the data to be sent from the queue expansion device queue Q10 to the network card queue Q20, thereby transmitting the data to be sent from the computing device 110 to the network card 2112.
  • each queue expansion device queue can generate a queue correspondence with multiple host queues, and the expansion device queue and the network card queue have a one-to-one correspondence, so that Without any upgrade of the network card and computing device, the inefficiency of communication caused by the forwarding capacity of the host queue being weaker than the forwarding capacity of the network card queue is avoided, so that the forwarding performance of the network card queue can be fully utilized.
  • the network system shown in FIG. 1 may also be a data center virtualization scenario.
  • the computing device 110 and the computing device 120 may be physical machines including multiple virtual machines, or may be located in The two virtual machines in the same physical machine may also be two virtual machines located in different physical machines, which are not specifically limited in this application.
  • the following describes the method for queue communication based on the forwarding device provided by the present application based on the network architecture shown in FIG. 1 to FIG. 2.
  • the present application provides a method for queue communication.
  • the forwarding device is connected to a computing device, the forwarding device includes at least one forwarding queue, the computing device 400 includes multiple host queues, and multiple host queues of the computing device store multiple data to be sent, In simple terms, the data to be sent in the multiple host queues of the computing device will be sent to at least one forwarding queue of the forwarding device.
  • the computing device 400 in FIG. 4 may be the computing device 110 and the computing device 120 in FIGS. 1-2, and the method of queue communication includes the following steps:
  • the forwarding device receives a device query message from the computing device.
  • the forwarding device may be a PCIe device. After the forwarding device is connected to the computing device, the computing device needs to identify the forwarding device connected to it through PCIe device enumeration, and then can communicate with the forwarding device.
  • the forwarding device In response to the device query message, the forwarding device reports the expanded number of forwarding queues to the computing device, thereby establishing a queue correspondence between the forwarding queue and the host queue.
  • the number of the extended forwarding queues is determined by the number of the at least one forwarding queue and the expansion coefficient, and the number of the extended forwarding queues is used to establish a queue correspondence relationship.
  • the forwarding device includes an attribute register, and the number of the extended forwarding queues is stored in the attribute register; the forwarding device reporting the number of extended forwarding queues to the computing device includes: The forwarding device queries the attribute register, obtains the expanded number of forwarding queues from the attribute register, and reports the expanded number of forwarding queues to the computing device.
  • the expansion factor n may be determined according to the queue processing capability of the forwarding device and the single thread processing capability of the processor of the computing device.
  • the forwarding device receives a first doorbell message from the computing device, where the first doorbell message carries information about a first host queue and information about a second host queue, and the first host queue and The second host queues all belong to the multiple queues.
  • the forwarding device obtains the information of the first host queue and the information of the second host queue from the first doorbell message, obtains the first to-be-sent data from the first host queue, and obtains the The second data to be sent is acquired from the second host queue.
  • the forwarding device stores the first data to be sent and the second data to be sent into the first host queue and the first host queue, respectively, according to the queue correspondence between the forwarding queue and the host queue. Two forwarding queues corresponding to the host queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.
  • the host queue and the forwarding queue can be understood as a section of memory.
  • This section of memory will be divided into several memory blocks of equal length, corresponding to a unit of work.
  • Each unit of work is used to store a fixed-length message, such as computing.
  • the data request (read request or write request) initiated by the device to the forwarding device. Therefore, similar to other storage mechanisms, each queue maintained by the computing device and the forwarding device also includes a head pointer (Head) and a tail pointer (Tail). By comparing the positions of the two pointers, the storage status of the queue can be obtained.
  • Head head pointer
  • Tiil tail pointer
  • the refresh rule of the head pointer is: point to the next empty position in the queue
  • the refresh rule of the tail pointer point to the location of the next command to be executed.
  • the queue can be logically understood as shown in Figure 5A
  • the actual state of the memory in the queue is a long strip of storage space, as shown in Figure 5B.
  • the head and tail pointers point to the same position, it means that the queue is currently empty, as shown in Figure 5C.
  • each host queue in the computing device 400 includes a head and tail pointer. Therefore, the information of the first host queue includes a first head pointer, and the information of the second host queue includes a second head pointer. Wherein, the first head pointer indicates the first data to be sent, and the second head pointer indicates the second data to be sent.
  • the first data to be sent is located in 4 units A0-A3 in the first host queue, then the first head pointer can point to A3, and the second data to be sent is located in 3 in the second host queue.
  • the second head pointer can point to B2. Therefore, the first doorbell message sent by the computing device 400 to the forwarding device 300 in step S103 will include the first head pointer A3 and the second head pointer B2.
  • the forwarding device 300 also stores multiple sets of head pointers and multiple sets of tail pointers, and each set of head pointers and each set of tail pointers respectively correspond to the head pointers of one of the multiple host queues of the computing device. And tail pointers, the forwarding device also stores at least one set of head pointers and at least one set of tail pointers of the at least one forwarding queue.
  • the forwarding device 300 further includes a doorbell register for storing the head and tail pointer values of the first host queue and the second host queue in the computing device 400, and the value of the forwarding queue in the forwarding device. The value of the head pointer and tail pointer.
  • step S104 may be as follows:
  • the forwarding device obtains the first head pointer and the second head pointer from the first doorbell message.
  • the forwarding device refreshes the head pointer corresponding to the first host queue in the multiple sets of head pointers according to the first head pointer.
  • the forwarding device refreshes the head pointer corresponding to the second host queue in the multiple sets of head pointers according to the second head pointer.
  • the forwarding device obtains the first host queue from the first host queue according to the head pointer corresponding to the first host queue after refreshing and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers. The first data to be sent.
  • the forwarding device obtains the second host queue from the second host queue according to the head pointer corresponding to the second host queue after the refresh and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers.
  • the second data to be sent The second data to be sent.
  • step S1041 may obtain the first head pointer A3 and the second head pointer A3 from the first doorbell message. Head pointer B2.
  • the value of the first head pointer stored in the doorbell register can also be refreshed to A3 according to the first head pointer A3 acquired in the first doorbell message.
  • the value of the first head pointer stored in the doorbell register can be updated to A3.
  • the second head pointer B2 obtained in the message also refreshes the value of the second head pointer stored in the memory in the doorbell register to B2.
  • the forwarding device obtains the first data to be sent A0-A3 from the first host queue by comparing the refreshed first head pointer A3 and the first tail pointer A0 in the doorbell register, and according to the second head pointer B2 and The value of the second tail pointer B0 obtains the second to-be-sent data B0-B2 from the second host queue.
  • the forwarding device may first obtain the forwarding corresponding to the first host queue and the second host queue according to the queue correspondence established in step S102. queue. Then, the first data to be sent and the second data to be sent are stored in the forwarding queue to complete a data communication.
  • the method further includes : Refresh the tail pointer of the first host queue and the tail pointer of the second host queue; refresh the head pointer of the corresponding forwarding queue; refresh the tail pointer of the first host queue and the refreshed tail pointer
  • the tail pointer of the second host queue is sent to the computing device. For example, as shown in FIG. 6, after the forwarding device stores the first data to be sent and the second data to be sent in the forwarding queue C corresponding to the first host queue and the second host queue, the forwarding device may forward the doorbell register The tail pointer of the queue C is refreshed to C7.
  • the first tail pointer of the first host queue is refreshed to A3
  • the second host queue is the second
  • the values of the first tail pointer and the second tail pointer are sent to the computing device, so as to inform the computing device that the first data to be sent and the second data to be sent have been removed by the forwarding device.
  • the forwarding device sends the expanded network card queue number to the computing device, so that the performance of the network card hardware can be fully exploited, and the thread message forwarding rate of the computing device no longer becomes the network card Dealing with the bottleneck of message forwarding, further avoiding resource waste.
  • the forwarding device 300 at this time includes the queue expansion device 500 and the network card 600, and at least one of the forwarding devices 300
  • the forwarding queue includes at least one queue expansion device queue and at least one network card queue, the at least one queue expansion device queue is located in the queue expansion device 500, and the at least one network card queue is located in the network card 600.
  • the queue correspondence relationship includes a first queue correspondence relationship and a second queue correspondence relationship.
  • the first queue correspondence relationship is a correspondence relationship between the plurality of host queues and the at least one queue extension device queue
  • the second queue The correspondence relationship is the correspondence relationship between the at least one network card queue and the at least one queue expansion device queue, and the at least one network card queue corresponds to the at least one queue expansion device queue in a one-to-one correspondence.
  • the queue communication method includes the following steps:
  • the computing device 400 sends a device query message to the queue expansion device 500.
  • the queue expansion device 500 sends a device query message to the network card 600.
  • the network card 600 reports the number of network card queues to the queue expansion device 500, thereby establishing a second queue correspondence relationship between the queue expansion device and the network card queue.
  • the network card queue and the queue expansion device queue have a one-to-one correspondence.
  • the queue expansion device 500 includes an attribute register, and the queue expansion device 500 stores the number of network card queues in the attribute register after receiving the number of network card queues of the network card 600.
  • the queue expansion device 500 reports the number of expanded queues to the computing device 400, so as to establish a first queue correspondence between the queue expansion device queue and the host queue.
  • the number of queues after expansion is determined by the number of queues of the network card and the expansion coefficient.
  • the number of the expanded queues is stored in the attribute register.
  • the queue expansion device 500 may query the attribute register to obtain the number of network card queues from the attribute register, calculate the number of expanded queues according to the expansion coefficient k, and calculate the number of expanded queues. The number is reported to the computing device.
  • the expansion coefficient k may be determined according to the message forwarding rate a of the network card and the message forwarding rate b of a single thread of the processor of the computing device. specifically,
  • the queue expansion device 500 receives a first doorbell message from the computing device 400, where the first doorbell message carries information about the first host queue and information about the second host queue, and the first Both the host queue and the second host queue belong to the multiple host queues.
  • the queue expansion device 500 obtains the information of the first host queue and the information of the second host queue from the first doorbell message, and obtains the first data to be sent from the first host queue. Obtain the second to-be-sent data from the second host queue.
  • the queue expansion device 500 stores the first to-be-sent data and the second to-be-sent data into the queue of the queue expansion device respectively according to the correspondence between the first queue and the first host queue and the second queue.
  • the network card 600 receives a second doorbell message from the queue expansion device 500, where the second doorbell message carries information about a corresponding queue expansion device queue, and the corresponding queue expansion device queue belongs to the description.
  • the queue expands the device queue.
  • the network card 600 obtains the information of the corresponding queue expansion device queue from the second doorbell message, and obtains the first data to be sent and the second data to be sent from the corresponding queue expansion device queue .
  • the network card 600 stores the first to-be-sent data and the second to-be-sent data into the network card corresponding to the corresponding queue expansion device queue in the network card queue according to the second queue correspondence Queue, so as to store the data to be sent in the host queue in the computing device into the network card queue in the network card 600.
  • each host queue in the computing device 400 includes a head and tail pointer
  • each network card queue in the network card 600 includes a head and tail pointer. Therefore, correspondingly, the at least one set of The head pointer and the at least one set of tail pointers are stored in the queue expansion device 500, the network card stores at least one set of network card head pointers and at least one set of network card tail pointers, and the at least one set of queue expansion device head pointers 500 corresponds to the at least one set of network card head pointers, and the at least one set of queue expansion device tail pointers corresponds to the at least one set of network card tail pointers.
  • the queue expansion device 500 includes a first doorbell register for storing the head and tail pointer values of the first host queue and the second host queue in the computing device 400, and the queue expansion device The value of the head pointer and tail pointer of the forwarding queue in 500.
  • the network card 600 includes a second doorbell register for storing the head pointer and tail pointer values of the corresponding queue expansion device queue in the queue expansion device 500, and the head pointer and tail pointer values of the network card queue in the network card 600.
  • the queue expansion device obtains the first data to be sent from the first doorbell information and
  • the specific steps of the second data to be sent may be:
  • the queue expansion device obtains the first head pointer and the second head pointer from the first doorbell message
  • the queue expansion device refreshes the head pointer of the queue expansion queue corresponding to the first host queue among the sets of head pointers according to the first head pointer and the corresponding relationship of the first queue;
  • the queue expansion device obtains all data from the first host queue according to the head pointer corresponding to the first host queue after refreshing and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers.
  • the queue expansion device refreshes the head pointer corresponding to the second host queue in the multiple sets of head pointers according to the second head pointer and the corresponding relationship of the first queue;
  • the queue expansion device obtains all data from the second host queue according to the head pointer corresponding to the second host queue after refreshing and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers. The second data to be sent.
  • the queue expansion device After the queue expansion device obtains the first data to be sent and the second data to be sent in step S206, it will determine the queue expansion device queues corresponding to the first host queue and the second host queue according to the corresponding relationship of the first queue. , That is, the corresponding queue expansion device queue, and then the first data to be sent and the second data to be sent are stored in the corresponding queue expansion device queue, and the head pointer of the corresponding queue expansion device queue is refreshed and sent to the network card 600
  • the second doorbell information including the third head pointer causes the network card to repeat the above steps.
  • the network card queue corresponding to the corresponding queue expansion device queue is determined, that is, the corresponding network card queue, and the first data to be sent and the second data to be sent are stored in the corresponding network card queue. That is to say, that the forwarding device acquires the information of the first host queue and the information of the second host queue from the first doorbell message includes: the queue expansion device in the forwarding device obtains information from the first host queue.
  • the forwarding device obtaining the first to-be-sent data from the first host queue includes: the queue expansion device in the forwarding device according to The first head pointer and the first queue correspondence relationship, refresh the head pointer corresponding to the first host queue in the multiple sets of head pointers; the queue expansion device in the forwarding device is based on the refreshed first A head pointer corresponding to a host queue, and a tail pointer corresponding to the first host queue in the multiple sets of tail pointers, acquire the first data to be sent in the first host queue; the forwarding device obtains the first data to be sent from the first host queue; Obtaining the second data to be sent from the second host queue includes: the queue expansion device in the forwarding device refreshes the data in the multiple sets of head pointers according to the second head pointer and the corresponding relationship of the first queue.
  • the head pointer corresponding to the second host queue; the queue expansion device in the forwarding device is based on the head pointer corresponding to the second host queue after being refreshed, and the tail corresponding to the second host queue in the multiple sets of tail pointers Pointer to obtain the second data to be sent in the second host queue; the forwarding device stores the first data to be sent and the second data to be sent into the first host queue and the first host queue respectively
  • the forwarding queue corresponding to the second host queue includes: the queue expansion device in the forwarding device stores the first to-be-sent data and the second to-be-sent data in the first host according to the first queue correspondence relationship
  • the queue and the queue expansion device queue corresponding to the second host queue wherein the corresponding queue expansion device queue belongs to the queue expansion device queue, and the first host queue and the second host queue correspond to the at least one queue
  • the same queue in the expansion device queue expands the device queue; the queue expansion device in the forwarding device refreshes the head pointer of the corresponding queue expansion device queue, the tail pointer of the
  • the first data to be sent may be a first Direct Memory Access (DMA) read descriptor, which is used to send the first data to be read at the computing device to the Network card
  • the second data to be sent may be a second DMA read descriptor, which is used to send the second data to be read from the computing device to the network card.
  • DMA Direct Memory Access
  • the DMA read descriptor of the first data to be read can be used as the data to be sent and placed in the host queue.
  • the DMA read descriptor may include the read address of the first data to be read, the length of the data packet, and description information, etc., and after steps S201-S210, the network card may receive the DMA descriptor sent by the computing device. Generate a DMA read command that directly reads the first data to be read from the computing device.
  • the first data to be sent is a first direct memory access DMA read descriptor
  • the second data to be sent is a second DMA read descriptor
  • the first data to be sent is combined with the first data to be sent.
  • the method further includes: the network card in the forwarding device reads the descriptor according to the first DMA read descriptor and the second DMA read Descriptor, generating a first DMA read command and a second DMA read command; the network card in the forwarding device sends the first DMA read command and the second DMA read command to the queue expansion device in the forwarding device;
  • the queue expansion device in the forwarding device obtains the first data to be read from the computing device according to the first DMA read command, and obtains the second data to be read from the computing device according to the second DMA read command;
  • the queue expansion device in the forwarding device forwards the first data to be read and the second data to be read to the network
  • the first data to be sent is a first DMA write descriptor
  • the second data to be sent is a second DMA Write descriptor, wherein the first DMA write descriptor is used to write the first data to be written from the queue expansion device to the computing device, and the second DMA descriptor is used to write the second data to be written to the computing device.
  • Incoming data is written from the queue expansion device to the computing device.
  • the difference between the second data to be written and the above-mentioned DMA read descriptor is that the data received by the network card is usually fragmented data.
  • the first data to be written can be transferred to n first fragmented data.
  • the queue expansion device also includes memory, such as Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM).
  • DDR SDRAM Double Data Rate Synchronous Dynamic Random Access Memory
  • the queue expansion device can store the first fragment data received from the network card in the DDR, and when all n fragments of the first data to be written are collected, then the first data to be written Incoming data is sent to the computing device. Therefore, at step S207, the queue expansion device needs to convert the received DMA write descriptor used for writing to the computing device into a network card DMA write descriptor written to the queue expansion device.
  • the first data to be sent is a first DMA write descriptor
  • the second data to be sent is a second DMA write descriptor
  • the first DMA write descriptor is used to transfer the first DMA write descriptor.
  • the write data is written from the queue expansion device to the computing device
  • the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device
  • the The storage of the first data to be sent and the second data to be sent in the first host queue and the queue expansion device queue corresponding to the second host queue includes: the queue expansion device in the forwarding device writes the first DMA
  • the descriptor is converted into a first network card DMA write descriptor
  • the second DMA write descriptor is converted into a second network card DMA write descriptor, where the first DMA write descriptor is used to transfer the first data to be written
  • the queue expansion device is written from the network card
  • the second DMA descriptor is used to write the second data to be written from the network card into
  • DMA write command wherein the third DMA write command is used for the computing device to execute and write the first data to be written into the computing device, and the fourth DMA write command is used for the computing device
  • the computing device executes and writes the first data to be written into the computing device; the queue expansion device in the forwarding device sends the The computing device sends the third DMA write command and the fourth DMA write command, so that the computing device writes the first to-be-written command according to the third DMA write command and the fourth DMA write command
  • the data and the second data to be written are written into the computing device.
  • the method further includes: the network card in the forwarding device sends a completion message and an interrupt message to the queue expansion device; the queue in the forwarding device The extension device sends the completion message and interrupt to the computing device, so that the computing device refreshes the tail pointers of the first host queue and the second host queue according to the completion message.
  • the forwarding device or the network card after the forwarding device or the network card has processed the read command or the write command, it will also write the command completion information in the network card queue, and send it to the queue expansion device together with the interrupt message, and the queue expansion device will then add the interrupt message to the queue expansion device.
  • the command completion information is sent to the computing device, thereby completely completing a data communication process.
  • the queue expansion device sends the expanded network card queue number to the computing device, so that the performance of the network card hardware can be fully explored.
  • the thread message forwarding rate of the virtual machine no longer becomes a bottleneck for the network card to process message forwarding, further avoiding resource waste.
  • the following takes the application scenario where the first data to be sent and the second data to be sent are DMA read descriptors as an example (the computing device sends the first data to be read and the second data to be read to the network card) , Give an example of the queue communication method described in FIG. 7.
  • the queue expansion device has generated the second queue correspondence with the network card, and the first queue correspondence with the computing device, and stored the two queue correspondences in the queue. Mapping table. There is a first queue correspondence between the first host queue A and the second host queue B and the corresponding queue expansion device queue C, and there is a second queue correspondence between the corresponding queue expansion device queue C and the corresponding network card queue D relationship.
  • the computing device needs to send the first data to be read and the second data to be read to the network card 600, so the specific communication process may be as follows:
  • the computing device 400 puts the DMA read descriptor of the first data to be read, that is, the first data to be sent, into the 4 queue units of the first host queue A, and reads the DMA of the second data to be read
  • the descriptor that is, the second data to be sent, is put into the 3 queue units of the second host queue B.
  • the computing device 400 refreshes the value of the first head pointer of the first host queue to A3, and refreshes the value of the second head pointer of the second host queue to B3.
  • the computing device 400 sends the first doorbell message to the queue expansion device 500.
  • the queue expansion device 500 obtains the first head pointer A3 and the second head pointer B3 from the first doorbell message.
  • the queue expansion device 500 refreshes the first head pointer in the first doorbell register as A3, and the second head pointer as B2.
  • the queue expansion device 500 obtains the data of the four queue units A0-A3 from the computing device according to the refreshed first doorbell register, the head and tail pointers of the first host queue are A3 and A0, that is, the first to be sent Data, similarly, obtain the second to-be-sent data from the computing device.
  • the queue expansion device 500 determines the queue expansion device queue C corresponding to the first host queue A and the second host queue B according to the first mapping relationship, and combines the first to-be-sent data (4 queue units) and the second to-be-sent data The sending data (3 queue units) is put into the corresponding queue expansion device queue C.
  • the queue expansion device 500 refreshes the value of the third head pointer of the corresponding queue expansion device queue C to C7.
  • the queue expansion device 500 sends the second doorbell message to the network card 600.
  • the network card 600 obtains the value C7 of the third head pointer from the second doorbell message.
  • the network card 600 refreshes the value C7 of the third head pointer in the second doorbell register.
  • the network card 600 obtains the data of the 7 queue units C0-C7 from the queue expansion device 500 according to the refreshed second doorbell register, the head and tail pointers of the corresponding queue expansion device queue are C7 and C0, that is, the first The data to be sent and the second data to be sent.
  • the network card 600 determines the network card queue D corresponding to the corresponding queue expansion device queue C according to the second mapping relationship, and puts a total of 7 queue units of the first data to be sent and the second data to be sent into the corresponding network card Queue D.
  • the network card 600 refreshes the value of the fourth head pointer of the corresponding network card queue D to D7.
  • the network card 600 generates a first DMA read command and a second DMA read command according to the DMA descriptors in the 7 queue units, and sends them to the queue expansion device 500.
  • the queue expansion device 500 transparently transmits the first DMA read command and the second DMA read command to the computing device.
  • the computing device 400 obtains the first to-be-read data stored in the computing device 400 corresponding to the first DMA read command and the second DMA read command according to the first DMA read command and the second DMA read command And the second data to be read.
  • the computing device 400 sends the first data to be read and the second data to be read to the queue expansion device 500.
  • the queue expansion device 500 sends the first data to be read and the second data to be read to the network card 600.
  • the network card 600 updates the tail pointer of the corresponding queue expansion device queue C to C7.
  • the network card 600 sends the completion queue unit and interrupt to the queue expansion device, and carries the value of the third tail pointer C7 to the queue expansion device 500, so that the queue expansion device 500 refreshes the value of the third tail pointer in the first doorbell register .
  • the queue expansion device 500 parses and completes the queue unit and interrupt, refreshes the value of the first tail pointer of the first host queue in the first doorbell register as A3, and the value of the second tail pointer of the second host queue is B2, and generates a new
  • the completion queue unit and interrupt are sent to the computing device, so that the computing device can refresh the tail pointer of the first host queue and the tail pointer of the second host queue maintained by itself, and complete the first data to be read and the second data to be read. send.
  • the queue expansion device after obtaining the number of network card queues in the network card, the queue expansion device sends the expanded network card queue number to the computing device, so that the performance of the network card hardware can be fully exploited , The thread message forwarding rate of the virtual machine no longer becomes the bottleneck of the network card processing message forwarding, further avoiding the waste of resources.
  • the following takes the application scenario where the first data to be sent and the second data to be sent are DMA write descriptors as an example (the network card sends the first data to be written and the second data to be written to the computing device),
  • the queue communication method described in FIG. 7 is illustrated by an example.
  • the queue expansion device has generated the second queue correspondence with the network card, and the first queue correspondence with the computing device, and stored the two queue correspondences in the queue. Mapping table. There is a first queue correspondence between the first host queue A and the second host queue B and the corresponding queue expansion device queue C, and there is a second queue correspondence between the corresponding queue expansion device queue C and the corresponding network card queue D relationship.
  • the network card 600 needs to send the first data to be written and the second data to be written to the computing device 500, so the specific communication process may be as follows:
  • the computing device 400 puts the first DMA write descriptor of the first data to be written, that is, the first data to be sent into the 4 queue units of the first host queue A, and transfers the second data to be written
  • the second DMA write descriptor that is, the second data to be sent, is put into the 3 queue units of the second host queue B.
  • the first DMA write descriptor is used to write first data to be written into the computing device
  • the second DMA descriptor is used to write second data to be written into the computing device.
  • the computing device 400 refreshes the first host queue with a first head pointer value of A3, and refreshes the second host queue with a second head pointer value of B2.
  • the computing device 400 sends the first doorbell message to the queue expansion device 500.
  • the queue expansion device 500 obtains the first head pointer A3 and the second head pointer B2 from the first doorbell message.
  • the queue expansion device 500 refreshes the first head pointer in the first doorbell register as A3, and the second head pointer as B2.
  • the queue expansion device 500 obtains the data of the four queue units A0-A3 from the computing device according to the refreshed first doorbell register, the head and tail pointers of the first host queue are A3 and A0, that is, the first DMA write
  • the descriptor in the same way, obtains the second DMA write descriptor from the computing device.
  • the queue expansion device 500 converts the first DMA write descriptor into a first network card DMA write descriptor, and converts the second DMA write descriptor into a second network card DMA write descriptor, where the first The DMA write descriptor is used to write the first data to be written into the queue expansion device, and the first DMA write descriptor is used to write the second data to be written into the queue expansion device.
  • the queue expansion device 500 determines the queue expansion device queue C corresponding to the first host queue A and the second host queue B according to the first mapping relationship, and writes the converted first DMA descriptor (4 queue units) And the second DMA write descriptor (3 queue units) is put into the corresponding queue expansion device queue C.
  • the queue expansion device 500 refreshes the value of the third head pointer of the corresponding queue expansion device queue C to C7.
  • the queue expansion device 500 sends the second doorbell message to the network card 600.
  • the network card 600 obtains the value C7 of the third head pointer from the second doorbell message.
  • the network card 600 refreshes the value C7 of the third head pointer in the second doorbell register.
  • the network card 600 obtains the data of the 7 queue units C0-C7 from the queue expansion device 500 according to the head and tail pointers of the corresponding queue expansion device queue as C7 and C0, that is, after conversion The first DMA write descriptor and the second DMA write descriptor.
  • the network card 600 determines the network card queue D corresponding to the corresponding queue expansion device queue C according to the second mapping relationship, and puts a total of 7 queue units of the first data to be sent and the second data to be sent into the corresponding network card Queue D.
  • the network card 600 refreshes the value of the fourth head pointer of the corresponding network card queue D to D7.
  • the network card 600 obtains, according to the first network card DMA write descriptor, the first fragmented data stored in the network card and corresponds to the first network card DMA write descriptor, and writes according to the second network card DMA write descriptor.
  • Descriptor acquiring the second fragmented data stored in the network card and corresponding to the DMA write descriptor of the second network card, wherein the first data to be written includes n pieces of first fragmented data, so The second data to be written includes m pieces of second data.
  • the network card 600 generates a first DMA write command according to the first network card DMA write descriptor and the first fragment data, and generates a second DMA write command according to the second network card DMA write descriptor and the second fragment data. DMA write command.
  • the network card 600 updates the tail pointer of the corresponding queue expansion device queue C to C7, and sends the first DMA write command and the second DMA write command to the queue expansion device, as well as the sending completion queue unit and interrupt,
  • the value of the third tail pointer C7 is carried to the queue expansion device 500, so that the queue expansion device 500 refreshes the value of the third tail pointer in the first doorbell register.
  • the queue expansion device writes the first fragmented data and the second fragmented data into the memory of the queue expansion device according to the first DMA write command and the second DMA write command, and refreshes all Describe the value of the third tail pointer in the first doorbell register.
  • the queue expansion device 500 parses the completion queue unit and the interrupt, and according to the address information of the queue completion unit, determines the write addresses of the first data to be written and the second data to be written to the computing device, so that the first When the number of fragmented data is n, a third DMA write command is generated according to the above write address and the first data to be written, and when the number of second fragmented data is m, according to the above write
  • the address and the second data to be written generate the fourth DMA write command, the value of the first tail pointer of the first host queue in the first doorbell register is refreshed is A3, and the value of the second tail pointer of the second host queue is B2.
  • the queue expansion device sends the third DMA write command, the fourth DMA write command, the new completion queue unit and the interrupt to the computing device, so that the computing device can refresh the tail pointer of the first host queue maintained by itself and The second host queue tail pointer completes the sending of the first data to be read and the second data to be read this time.
  • the computing device writes the first data to be written and the second data to be written into the computing device according to the third DMA write command and the fourth DMA write command. Complete the process of this queue communication.
  • the queue expansion device after obtaining the number of network card queues in the network card, the queue expansion device sends the expanded network card queue number to the computing device, so that the performance of the network card hardware can be fully exploited , The thread message forwarding rate of the virtual machine no longer becomes the bottleneck of the network card processing message forwarding, further avoiding the waste of resources.
  • FIG. 10 is a schematic structural diagram of a forwarding device provided by the present application.
  • the forwarding device 300 is connected to a computing device, the forwarding device includes at least one forwarding queue, the computing device includes a plurality of host queues, and a plurality of data to be sent are stored in the plurality of host queues, wherein the forwarding device
  • the device includes a receiving unit 310, an acquiring unit 320, a storage unit 330, a response unit 340, and a refresh unit 350.
  • the receiving unit 310 is configured to receive a first doorbell message from the computing device, where the first doorbell message carries information about a first host queue and information about a second host queue, and the first host queue And the second host queue both belong to the multiple host queues;
  • the acquiring unit 320 is configured to acquire the information of the first host queue and the information of the second host queue from the first doorbell message, acquire the first to-be-sent data from the first host queue, and from Acquiring second data to be sent from the second host queue;
  • the storage unit 330 is configured to store the first data to be sent and the second data to be sent into the first host queue respectively according to the queue correspondence between the forwarding queue and the host queue.
  • a forwarding queue corresponding to the second host queue, and the first host queue and the second host queue correspond to the same forwarding queue in the at least one forwarding queue.
  • the forwarding device further includes a response unit 340
  • the receiving unit 310 is further configured to receive the device from the computing device before the receiving unit receives the first doorbell message from the computing device Query message;
  • the response unit 340 is configured to report the number of extended forwarding queues to the computing device in response to the device query message, wherein the number of extended forwarding queues is forwarded by the at least one The number of queues and the expansion coefficient are determined, and the expanded number of forwarding queues is used to establish the queue correspondence.
  • the forwarding device includes an attribute register
  • the number of the expanded forwarding queues is stored in the attribute register
  • the response unit 340 is configured to query the attribute register and obtain the information from the attribute register.
  • the number of the extended forwarding queues is acquired, and the number of the extended forwarding queues is reported to the computing device.
  • the information of the first host queue includes a first head pointer, the first head pointer indicates the first to-be-sent data, and the information about the second host queue includes a second head pointer, so
  • the second head pointer indicates the second data to be sent, and multiple sets of head pointers and multiple sets of tail pointers are stored in the forwarding device, and each set of head pointers and each set of tail pointers respectively correspond to multiple queues of the computing device
  • the forwarding device also stores at least one set of head pointers and at least one set of tail pointers of the at least one queue; the forwarding device further includes a refresh unit 350, and the acquiring
  • the unit 320 is configured to obtain the first head pointer and the second head pointer from the first doorbell message;
  • the refresh unit 350 is configured to refresh all the groups of head pointers according to the first head pointer.
  • the head pointer corresponding to the first host queue; the obtaining unit 320 is configured to use the head pointer corresponding to the first host queue after refreshing and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers, Acquiring the first data to be sent in the first host queue; the refresh unit 350 is configured to refresh the head pointer corresponding to the second host queue in the plurality of sets of head pointers according to the second head pointer; The acquiring unit 320 is used to acquire all the data from the second host queue according to the head pointer corresponding to the second host queue after refreshing and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers.
  • the second data to be sent; the refresh unit 350 is configured to store the first data to be sent and the second data to be sent into the forwarding queues corresponding to the first host queue and the second host queue respectively, and then refresh The tail pointer of the first host queue and the tail pointer of the second host queue; the refresh unit 350 is used to refresh the head pointer of the corresponding forwarding queue; the refresh unit 350 is used to refresh all the The tail pointer of the first host queue and the refreshed tail pointer of the second host queue are sent to the computing device.
  • FIG. 10 is only an implementation of the embodiment of the present application.
  • the functions of the foregoing multiple modules or units can be split into More sub-modules or sub-units can also be combined into fewer modules or units to achieve the same technical effect, so they should all fall into the protection scope of the embodiments of this application
  • Fig. 11 is a schematic structural diagram of a queue expansion device provided by the present application.
  • the queue expansion device is connected to a computing device, the network card is connected to the computing device, the queue expansion device includes at least one queue expansion device queue, the network card includes at least one network card queue, and the computing device includes multiple hosts A queue, a plurality of data to be sent are stored in the plurality of host queues.
  • the queue expansion device 500 includes a receiving unit 510, an acquiring unit 520, a storage unit 530, a sending unit 540, a response unit 550, and a refresh unit 560.
  • the receiving unit 510 is configured to receive a first doorbell message from the computing device, where the first doorbell message carries information about a first host queue and information about a second host queue, and the first host queue And the second host queue both belong to the multiple host queues;
  • the obtaining unit 520 is configured to obtain the information of the first host queue and the information of the second host queue from the first doorbell message, obtain the first to-be-sent data from the first host queue, and from Acquiring second data to be sent from the second host queue;
  • the storage unit 530 is configured to store the first data to be sent and the second data to be sent into the first queue respectively according to the first queue correspondence between the queue expansion device queue and the host queue.
  • the sending unit 540 is configured to send a second doorbell message to the network card, and the second doorbell message carries the information of the corresponding queue expansion device queue, so that the network card receives information from the network card according to the second doorbell message.
  • the first data to be sent and the second data to be sent are acquired from the corresponding queue expansion device queue, and according to the second queue correspondence between the network card queue and the queue expansion device queue, the second queue A data to be sent and the second data to be sent are stored in the network card queue corresponding to the corresponding queue expansion device queue, wherein the corresponding queue expansion device queue belongs to the queue expansion device queue, and the first The host queue and the second host queue correspond to the same queue expansion device queue in the at least one queue expansion device queue, and the corresponding network card queue belongs to the network card queue.
  • the device further includes a response unit 550, and the receiving unit 510 is further configured to receive a first device query message from the computing device before receiving the first doorbell message from the computing device;
  • the sending unit 540 is configured to send a second device query message to the network card;
  • the response unit 550 is configured to receive the number of network card queues sent by the network card, wherein the number of network card queues is used to establish the first Correspondence between two queues;
  • the response unit 550 is configured to report the number of expanded queue expansion device queues to the computing device in response to the first device query message, wherein the expanded queue expansion device queue The number is determined by the number of network card queues and an expansion coefficient, and the expanded number of queue expansion device queues is used to establish the first queue correspondence relationship.
  • the device includes an attribute register, and the number of the expanded forwarding queues is stored in the attribute register; the response unit 550 is further configured to query the attribute register, and obtain information from the attribute register. The number of queues of the expanded queue expansion device is acquired, and the number of queues of the expanded queue expansion device is reported to the computing device.
  • the information of the first host queue includes a first head pointer
  • the first head pointer indicates the first to-be-sent data
  • the information about the second host queue includes a second head pointer
  • the queue expansion device stores multiple sets of head pointers and multiple sets of tail pointers, and each set of head pointers and each set of tail pointers respectively correspond to the multiple host queues
  • the head pointer and the tail pointer of a host queue in, the queue expansion device also stores at least one set of queue expansion device head pointers and at least one set of queue expansion device tail pointers, and at least one set of network cards is stored in the network card A head pointer and at least one set of network card tail pointers, the at least one set of queue extension device head pointers correspond to the at least one set of network card head pointers, and the at least one set of queue extension device tail pointers corresponds to the at least one set of network card head pointers.
  • the queue expansion device further includes a refresh unit 560, and the acquisition unit 520 is configured to acquire the first head pointer and the second head pointer from the first doorbell message;
  • the refreshing unit 560 is configured to refresh the head pointers corresponding to the first host queue in the multiple sets of head pointers according to the first head pointer and the first queue correspondence;
  • the acquiring unit 520 is configured to refresh according to After refreshing, the head pointer corresponding to the first host queue, and the tail pointer corresponding to the first host queue in the multiple sets of tail pointers, acquire the first data to be sent in the first host queue;
  • the refresh unit 560 is configured to refresh the head pointer corresponding to the second host queue in the multiple sets of head pointers according to the second head pointer and the corresponding relationship of the first queue;
  • the acquiring unit 520 is configured to refresh according to After refreshing, the head pointer corresponding to the second host queue, and the tail pointer corresponding to the second host queue in the multiple sets of tail pointers
  • the first data to be sent is a first direct memory access DMA read descriptor
  • the second data to be sent is a second DMA read descriptor
  • the receiving unit 510 is further configured to receive The first DMA read command and the second DMA read command sent by the network card, wherein, the first DMA read command is generated by the network card according to the first DMA descriptor, and the second DMA read command is The network card is generated according to the second DMA descriptor
  • the obtaining unit 520 is further configured to obtain the first data to be read from the computing device according to the first DMA read command, and obtain the first data to be read from the computing device according to the second DMA read command.
  • the computing device obtains the second data to be read
  • the sending unit 540 is further configured to forward the first data to be read and the second data to be read to the network card.
  • the first data to be sent is a first DMA write descriptor
  • the second data to be sent is a second DMA write descriptor
  • the first DMA write descriptor is used to transfer the first DMA write descriptor.
  • a data to be written is written from the queue expansion device to the computing device
  • the second DMA descriptor is used to write the second data to be written from the queue expansion device to the computing device
  • the device It also includes a conversion unit 570, which is configured to convert the first DMA write descriptor to a first network card DMA write descriptor, and convert the second DMA write descriptor to a second network card DMA write descriptor , Wherein the first DMA write descriptor is used to write the first data to be written from the network card to the queue expansion device, and the second DMA descriptor is used to write the second data to be written from the network card to the queue expansion device.
  • the network card is written into the queue expansion device;
  • the storage unit 530 is configured to store the first network card DMA write descriptor and the second network card DMA write descriptor into the corresponding queue expansion device queue;
  • the receiving unit 510 is configured to receive a first DMA write command and the second DMA write command, where the first DMA write command is obtained by the network card according to the first network card DMA write descriptor and stored in the network card In the first segment data corresponding to the first network card DMA write descriptor, the first DMA write command generated according to the first network card DMA write descriptor and the first segment data, the first The second DMA write command is that after the network card obtains the second fragment data stored in the network card corresponding to the second network card DMA write descriptor according to the second network card DMA write descriptor, according to the first The second network card DMA write descriptor and the second DMA write command generated by the second fragmented data, wherein the first to-be-written data includes n pieces of first fragmented data
  • a fourth DMA write command is generated according to the second data to be written, wherein the third DMA write command is used for
  • the computing device executes and writes the first data to be written into the computing device, and the fourth DMA write command is used for the computing device to execute and write the first data to be written into the computing device.
  • the computing device; the sending unit 540 is configured to send the third DMA write command and the fourth DMA write command to the computing device, so that the computing device according to the third DMA write command and the first Four DMA write commands to write the first data to be written and the second data to be written into the computing device.
  • the receiving unit 510 is further configured to receive a completion message and an interrupt sent by the network card, where the completion message and the interrupt are when the network card is sending the first data to be sent and the first data to be sent. 2.
  • the data to be sent is generated after being stored in the network card queue corresponding to the corresponding queue expansion device queue; the sending unit 540 is also configured to send a completion message and an interrupt to the computing device, so that the computing device according to the The completion message refreshes the tail pointers of the first host queue and the second host queue.
  • FIG. 11 is only an implementation of the embodiment of the present application.
  • the functions of the foregoing multiple modules or units can be split into More sub-modules or sub-units can also be combined into fewer modules or units to achieve the same technical effect, and therefore all should fall within the protection scope of the embodiments of the present application.
  • FIG. 12 is a schematic structural diagram of a forwarding device provided by an embodiment of the present application.
  • the forwarding device 700 includes one or more processors 701, a communication interface 702, and a memory 703.
  • the processor 801, the communication interface 702, and the memory 703 may be connected through a bus 704.
  • the bus 704 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc.
  • the bus 704 can be divided into an address bus, a data bus, a control bus, and so on. For ease of representation, only one thick line is used in FIG. 12 to represent it, but it does not mean that there is only one bus or one type of bus.
  • the processor 701 includes one or more general-purpose processors, where the general-purpose processor may be any type of device capable of processing electronic instructions, including a CPU, a microprocessor, a microcontroller, a main processor, and a controller. And ASIC and so on.
  • the processor 701 executes various types of digital storage instructions, such as software or firmware programs stored in the memory 703, which enables the computing device to provide a wide variety of services.
  • the processor 701 executes the program instructions in the memory 703 to implement the aforementioned methods in FIG. 4 and FIG. 6.
  • the processor 701 may include a queue expansion module, a mapping module, a queue forwarding module, etc., where the queue expansion module may implement processing functions by calling program codes in the memory 703, such as calculating the number of expanded queues;
  • the mapping module can realize the processing function by calling the program code in the memory 304, and generate the queue correspondence between the host queue and the forwarding queue;
  • the queue forwarding module can realize the processing function by calling the program code in the memory 703, and realize the connection with the computing device. Queue communication.
  • the processor 701 may also include other modules for executing other steps described in the embodiment of FIG. 4 and FIG. 6, which will not be repeated here.
  • the communication interface 702 may be a wired interface (for example, an Ethernet interface) for communicating with other devices or modules.
  • the memory 703 may include a volatile memory (Volatile Memory), such as a random access memory (Random Access Memory, RAM); the memory may also include a non-volatile memory (Non-Volatile Memory), such as a read-only memory (Read-Only Memory). Memory, ROM, Flash Memory, Hard Disk Drive (HDD), or Solid-State Drive (SSD) memory may also include a combination of the foregoing types of memories.
  • the memory 703 may include a doorbell register, an attribute register, and a forwarding queue.
  • the doorbell register is used to store the head and tail pointers of at least one set of queues
  • the attribute register is used to store the number of expanded queues
  • the forwarding queue is used to store 400 The data to be sent sent.
  • the memory 703 may also include a mapping relationship table, which is used to store queue correspondences, and may also be used to store other program codes or program data for the processor to implement FIG. 4 And the other steps described in the embodiment of FIG. 6 to FIG. 9, which will not be repeated here.
  • a mapping relationship table which is used to store queue correspondences, and may also be used to store other program codes or program data for the processor to implement FIG. 4 And the other steps described in the embodiment of FIG. 6 to FIG. 9, which will not be repeated here.
  • FIG. 13 is a schematic structural diagram of a queue expansion device 800 provided by the present application.
  • the queue expansion device 800 includes a processor 802, a memory 504, an endpoint port 501, and a root port 503. Among them, the port endpoint 501, the root port 505, and the memory 504 are all connected to the processor 502 through a bus 505.
  • the queue expansion device 800 includes one or more processors 801, a communication interface 802, and a memory 803. Wherein, the processor 801, the communication interface 802, and the memory 803 may be connected through a bus 804.
  • the bus 804 may be a PCI bus, an EISA bus, or the like.
  • the bus 704 can be divided into an address bus, a data bus, a control bus, and so on. For ease of representation, only one thick line is used in FIG. 13, but it does not mean that there is only one bus or one type of bus.
  • the processor 802 includes one or more general-purpose processors, where the general-purpose processor may be any type of device capable of processing electronic instructions, including a CPU, a microprocessor, a microcontroller, a main processor, and a controller. And ASIC and so on.
  • the processor 802 executes various types of digital storage instructions, such as software or firmware programs stored in the memory 804, which enables the computing device to provide a wide variety of services.
  • the processor 801 executes the program instructions in the memory 803 to implement the aforementioned methods in FIG. 7, FIG. 8, and FIG. 9.
  • the processor 802 may include a queue expansion module, a mapping module, a queue forwarding module, etc., where the queue expansion module may implement processing functions by calling program codes in the memory 804, such as calculating the number of expanded queues;
  • the mapping module can realize the processing function by calling the program code in the memory 804, generate the first queue correspondence between the host queue and the queue expansion queue, or generate the second queue correspondence between the queue expansion queue and the network card queue, etc.
  • the queue forwarding module can realize the processing function by calling the program code in the memory 804, and complete the forwarding between the extended queue device and the network card for the received data to be sent from the computing device.
  • the processor may also include other modules for executing other steps described in the embodiments in FIG. 5 to FIG. 9, and details are not described herein again.
  • the communication interface 802 may be a wired interface (for example, an Ethernet interface) for communicating with other devices or modules.
  • the memory 804 may include volatile memory, such as RAM; the memory may also include non-volatile memory, such as ROM, flash memory, HDD, or SSD. The memory may also include a combination of the above-mentioned types of memories.
  • the memory 804 may include a first doorbell register, an attribute register, and a queue expansion device queue.
  • the first doorbell register is used to store the head and tail pointers of at least one set of queues, and the attribute register is used to store the number of expanded queues.
  • the device queue is used to store the to-be-sent data sent by the computing device. It should be understood that the above example is only for illustration, and the memory 804 may also include a mapping relationship table for storing the correspondence between the first queue and the second queue.
  • the memory 804 can also be used to store the first segment data and the second segment data, etc., and can also be used to store other program codes or program data for the processor to implement the description of the embodiments in FIGS. 7-9 The other steps are not repeated here.
  • FIG. 12 and FIG. 13 are only a possible implementation manner of the embodiment of the present application.
  • the forwarding device 700 and the queue expansion device 800 may also include more or fewer components, which are not limited here.
  • the content that is not shown or described in the embodiments of the present application please refer to the relevant descriptions in the embodiments described in FIG. 1 to FIG. 9, which will not be repeated here.
  • the queue expansion device 500 shown in FIG. 11 can be implemented by a logic circuit, specifically, it can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (Programmable Logic Device, PLD).
  • ASIC application-specific integrated circuit
  • PLD programmable logic device
  • the above-mentioned PLD can be a Complex Programmable Logical Device (CPLD), a Field-Programmable Gate Array (FPGA), a Generic Array Logic (GAL), or any combination thereof. This application does not Specific restrictions.
  • the embodiment of the present invention also provides a computer non-transitory storage medium, the computer non-transitory storage medium stores instructions, when it runs on the processor, any of the descriptions in FIGS. 4, 6 and 9 A method flow is realized.
  • the embodiment of the present invention also provides a computer program product.
  • the computer program product runs on a processor, the process of any method described in FIGS. 4 and 6 to 9 can be realized.
  • the steps of the method or algorithm described in combination with the disclosure of the embodiment of the present invention may be implemented in a hardware manner, or may be implemented in a manner in which a processor executes software instructions.
  • Software instructions can be composed of corresponding software modules, which can be stored in RAM, flash memory, ROM, erasable programmable read-only memory (Erasable Programmable ROM, EPROM), and electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), registers, hard disk, mobile hard disk, CD-ROM or any other form of storage medium known in the art.
  • An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium.
  • the storage medium may also be an integral part of the processor.
  • the processor and the storage medium may be located in the ASIC.
  • the ASIC may be located in the computing device.
  • the processor and the storage medium may also exist as discrete components in the computing device.
  • the program can be stored in a computer readable storage medium, and the program can be stored in a computer readable storage medium. When executed, it may include the procedures of the above-mentioned method embodiments.
  • the aforementioned storage media include: ROM, RAM, magnetic disks or optical disks and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

La présente invention concerne un procédé de communication de file d'attente. Le procédé comprend les étapes suivantes : un dispositif de transfert reçoit un premier message de sonnette d'un dispositif informatique ; le dispositif de transfert obtient des informations concernant une première file d'attente hôte et des informations concernant une seconde file d'attente hôte à partir du premier message de sonnette, obtient, à partir de la première file d'attente hôte, des premières données à envoyer, et obtient, à partir de la seconde file d'attente hôte, des secondes données à envoyer ; le dispositif de transfert stocke respectivement, en fonction de la correspondance de file d'attente entre les files d'attente de transfert et les files d'attente hôtes, lesdites premières données et lesdites secondes données dans la file d'attente de transfert correspondant à la première file d'attente hôte et à la seconde file d'attente hôte, la première file d'attente hôte et la seconde file d'attente hôte correspondant à la même file d'attente de transfert dans la ou les files d'attente de transfert.
PCT/CN2020/113497 2019-09-30 2020-09-04 Procédé et système de communication de file d'attente, et dispositifs s'y rapportant WO2021063154A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910944642.X 2019-09-30
CN201910944642.XA CN110750367B (zh) 2019-09-30 2019-09-30 一种队列通信的方法、系统及相关设备

Publications (1)

Publication Number Publication Date
WO2021063154A1 true WO2021063154A1 (fr) 2021-04-08

Family

ID=69277683

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/113497 WO2021063154A1 (fr) 2019-09-30 2020-09-04 Procédé et système de communication de file d'attente, et dispositifs s'y rapportant

Country Status (2)

Country Link
CN (1) CN110750367B (fr)
WO (1) WO2021063154A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110750367B (zh) * 2019-09-30 2023-03-17 超聚变数字技术有限公司 一种队列通信的方法、系统及相关设备
CN112162875B (zh) * 2020-10-12 2024-08-02 上交所技术有限责任公司 一种交易系统内部高可靠消息传输方法
CN114785714B (zh) * 2022-03-01 2023-08-22 阿里巴巴（中国）有限公司 一种报文传输时延检测方法、存储介质及设备
CN116795605B (zh) * 2023-08-23 2023-12-12 珠海星云智联科技有限公司 一种外围器件互联扩展设备异常自动恢复系统以及方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1968186A (zh) * 2006-06-19 2007-05-23 华为技术有限公司 一种调度报文发送的装置及方法
CN106789152A (zh) * 2016-11-17 2017-05-31 东软集团股份有限公司 基于多队列网卡的处理器扩展方法及装置
CN107005495A (zh) * 2017-01-20 2017-08-01 华为技术有限公司 用于转发数据包的方法、网卡、主机设备和计算机系统
US20190065206A1 (en) * 2017-08-22 2019-02-28 Bank Of America Corporation Predictive Queue Control and Allocation
CN109729024A (zh) * 2018-12-29 2019-05-07 中盈优创资讯科技有限公司 数据包处理系统及方法
CN110750367A (zh) * 2019-09-30 2020-02-04 华为技术有限公司 一种队列通信的方法、系统及相关设备

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5843397B2 (ja) * 2012-07-20 2016-01-13 富士通テレコムネットワークス株式会社 通信装置
CN106713185B (zh) * 2016-12-06 2019-09-13 瑞斯康达科技发展股份有限公司 一种多核cpu的负载均衡方法及装置
KR101716715B1 (ko) * 2016-12-27 2017-03-15 주식회사 티맥스클라우드 가상 머신 환경의 네트워크 입출력 장치 가상화 방법 및 장치
CN109218226A (zh) * 2017-07-03 2019-01-15 迈普通信技术股份有限公司 报文处理方法及网络设备
CN109962859A (zh) * 2017-12-26 2019-07-02 北京华为数字技术有限公司 一种报文调度方法及设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1968186A (zh) * 2006-06-19 2007-05-23 华为技术有限公司 一种调度报文发送的装置及方法
CN106789152A (zh) * 2016-11-17 2017-05-31 东软集团股份有限公司 基于多队列网卡的处理器扩展方法及装置
CN107005495A (zh) * 2017-01-20 2017-08-01 华为技术有限公司 用于转发数据包的方法、网卡、主机设备和计算机系统
US20190065206A1 (en) * 2017-08-22 2019-02-28 Bank Of America Corporation Predictive Queue Control and Allocation
CN109729024A (zh) * 2018-12-29 2019-05-07 中盈优创资讯科技有限公司 数据包处理系统及方法
CN110750367A (zh) * 2019-09-30 2020-02-04 华为技术有限公司 一种队列通信的方法、系统及相关设备

Also Published As

Publication number Publication date
CN110750367B (zh) 2023-03-17
CN110750367A (zh) 2020-02-04

Similar Documents

Publication Publication Date Title
WO2021063154A1 (fr) Procédé et système de communication de file d'attente, et dispositifs s'y rapportant
US11500810B2 (en) Techniques for command validation for access to a storage device by a remote client
US10810135B2 (en) Data transmission method, apparatus, device, and system
WO2023185035A1 (fr) Architecture, système et procédé d'accès direct à la mémoire, et dispositif électronique et support
EP3033681B1 (fr) Procédé et appareil permettant de délivrer des interruptions msi-x par l'intermédiaire de ponts non transparents à des ressources informatiques dans des grappes pci-express
WO2018076793A1 (fr) Dispositif nvme et procédés de lecture et d'écriture de données nvme
CN114265800B (zh) 中断消息处理方法、装置、电子设备及可读存储介质
US6631430B1 (en) Optimizations to receive packet status from fifo bus
US20200201578A1 (en) Method and Apparatus for Transmitting Data Processing Request
US11397697B2 (en) Core-to-core communication
US6684281B1 (en) Fast delivery of interrupt message over network
US20210075745A1 (en) Methods and apparatus for improved polling efficiency in network interface fabrics
WO2019153702A1 (fr) Procédé de traitement d'interruptions, appareil et serveur
US11741039B2 (en) Peripheral component interconnect express device and method of operating the same
WO2017101080A1 (fr) Procédé de traitement de demande d'écriture, processeur et ordinateur
US11966585B2 (en) Storage device and storage system
US20160241666A1 (en) Network interface
WO2020252763A1 (fr) Sélection adaptative de pipeline pour accélérer des opérations de copie en mémoire
US9288163B2 (en) Low-latency packet receive method for networking devices
US10628342B1 (en) System and method for accelerating performance of non-volatile memory RAID stacks
US8041902B2 (en) Direct memory move of multiple buffers between logical partitions
CN113986134B (zh) 存储数据的方法、读取数据的方法和装置
EP4254207A1 (fr) Appareil et procédé de traitement des données, et dispositif associé
CN112732176B (zh) 基于fpga的ssd访问方法及装置、存储系统及存储介质
WO2022170452A1 (fr) Système et procédé pour accéder à une ressource distante

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20872992

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20872992

Country of ref document: EP

Kind code of ref document: A1