WO2023122995A1 - Procédé et dispositif de transmission de paquets - Google Patents

Procédé et dispositif de transmission de paquets Download PDF

Info

Publication number
WO2023122995A1
WO2023122995A1 PCT/CN2021/142211 CN2021142211W WO2023122995A1 WO 2023122995 A1 WO2023122995 A1 WO 2023122995A1 CN 2021142211 W CN2021142211 W CN 2021142211W WO 2023122995 A1 WO2023122995 A1 WO 2023122995A1
Authority
WO
WIPO (PCT)
Prior art keywords
receiving
message
interface
sending
receiving interface
Prior art date
Application number
PCT/CN2021/142211
Other languages
English (en)
Chinese (zh)
Inventor
程传宁
张兆亮
程中武
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN202180099252.2A priority Critical patent/CN117461289A/zh
Priority to PCT/CN2021/142211 priority patent/WO2023122995A1/fr
Publication of WO2023122995A1 publication Critical patent/WO2023122995A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks

Definitions

  • the present application relates to the field of chip technology, and in particular to a message transmission method and device.
  • a common method is to provide multiple sets of physical interfaces and cooperate with multipath technology to increase communication bandwidth.
  • TCP Transmission Control Protocol
  • IP Internet Protocol
  • MP Multiple Path
  • each TCP subflow both the sender and the receiver need to maintain a set of states for each TCP subflow. For example, the sender and the receiver need to perform 3 handshakes and 4 handshakes for the TCP subflow. , the software and hardware resource overhead of the sending end and the receiving end is relatively large. Moreover, each TCP subflow needs to be bound to a physical interface for communication, and the bandwidth of the single path where each physical interface is located is still limited, and the transmission efficiency is low.
  • Embodiments of the present application provide a method and device for message transmission, which can increase communication bandwidth requirements between the sending end and the receiving end, and improve transmission efficiency through the multi-interface cooperative work of the sending end and the multi-interface cooperative work of the receiving end.
  • a message transmission method including: when the sending end determines to send the first message, determine the first sending interface for sending the first message according to the loads of multiple sending interfaces at the sending end, and determine the first sending interface for sending the first message according to the loads of multiple sending interfaces at the receiving end
  • the congestion information of the first receiving interface determines the first receiving interface for receiving the first message; the sending end sends the first message through the first sending interface, and the first message includes the identifier of the first sending interface and the identifier of the first receiving interface.
  • the sending interface and receiving interface in this application can be understood as the network port of the device.
  • the loads of multiple sending interfaces may be, for example, the resource occupancy rate of the sending interface, or the packet loss rate of packets, and the like.
  • the congestion information of multiple receiving interfaces may be, for example, the resource occupancy rate of the receiving interfaces, or the packet loss rate of packets, and the like.
  • the resource occupancy rate of each sending interface is, for example, the length of a queue for packet processing of the sending interface.
  • the message processing queue length of the sending interface can be understood as the number of messages currently processed in the sending queue of the sending interface, which is the queue of messages to be sent;
  • the resource occupancy rate of each receiving interface is, for example, the queue length for packet processing of the receiving interface.
  • the message processing queue length of the receiving interface can be understood as the number of currently processed messages in the receiving queue of the receiving interface, and the receiving queue is a queue of received messages to be processed.
  • the sending end does not need to bind the same flow or the same connection to the interface, including the need to bind the same flow or the same connection to a sending interface of the sending end, and does not need to bind the same A stream or the same connection is bound to a receiving interface on the receiving end.
  • the sending interface of the sending end may be different each time the message is sent, and at the same time, the receiving interface of the receiving end may be different each time the message is received.
  • through which sending interface a packet in a flow or a single connection is sent and through which receiving interface is received depends on the load status of multiple sending interfaces and the congestion information of multiple receiving interfaces.
  • the message rate at which the sending end sends packets can be understood as the sum of the message rates of multiple sending interfaces
  • the message rate at which the receiving end receives packets can be understood as the sum of the message rates of multiple receiving interfaces.
  • this application can give the sending end the ability to flexibly select the sending interface and the receiving interface of the message, and can make full use of the multi-interface coordination between the sending end and the receiving end to increase the bandwidth.
  • the method further includes: the sending end receives congestion information from multiple receiving interfaces sent by the receiving end, where the multiple receiving interfaces include the first receiving interface. That is, the sending end can determine the receiving interface through which the receiving end receives the message when each message is sent to the receiving end. That is, the receiving interface is flexibly determined by the sending end according to the congestion information of multiple receiving interfaces, which can make the multiple interfaces of the receiving end cooperate and improve the bandwidth of the receiving end.
  • the method further includes: when the sending end determines that sending the first message fails, determining the second receiving interface on which the receiving end receives the first retransmitted message according to the congestion information of multiple receiving interfaces, the first The retransmission message is a repeated message of the first message; the sending end sends the first retransmission message through the first sending interface, and the first retransmission message includes the identifier of the first sending interface, the identifier of the second receiving interface, and Marker for duplicate packets.
  • the receiving interface for receiving the original message and the receiving interface for receiving the retransmitted message may also be different, depending on the congestion information of the receiving interface Sure.
  • the sending interface of the original message and the sending interface of the retransmitted message of the same message at the sending end may also be different.
  • the method when the first retransmission message arrives at the receiving end later than the first message, the method further includes: the sending end receives the first response message of the first message through the second sending interface, and the first A response message includes the identification of the first sending interface; when the second sending interface is different from the first sending interface, the sending end controls the second sending interface to send the first response message to the first sending interface; the sending end controls the first sending interface Sending a first notification message to the first receiving interface, where the first notification message is used to instruct the first receiving interface to release the context information of the first message.
  • the sending interface for sending the first message if the sending interface for sending the first message is different from the sending interface for receiving the first response message, the sending interface for receiving the first response message also needs to forward the first response message to the sending interface for sending the first response message.
  • the sending interface of a message is the first sending interface, so that the first sending interface instructs the first receiving interface to release the context information of the first message.
  • the method when the first message arrives at the receiving end later than the first retransmission message, the method further includes: the sending end receives the first retransmission response of the first retransmission message through the third sending interface message, the first retransmission response message includes the identifier of the first sending interface; when the third sending interface is different from the first sending interface, the sending end controls the third sending interface to send the first retransmission response message to the first sending interface ; The sending end controls the first sending interface to send a second notification message to the second receiving interface, and the second notification message is used to instruct the second receiving interface to release the context information of the first retransmission message.
  • the sending interface that sends the first retransmission message is different from the sending interface that receives the first retransmission response message, the sending interface that receives the first retransmission response message also needs to send the first retransmission response message
  • the response message is forwarded to the sending interface that sent the first retransmission message, that is, the first sending interface, so that the first sending interface instructs the second receiving interface to release the context information of the first retransmission message.
  • a message transmission method includes: the receiving end sends congestion information of multiple receiving interfaces to the sending end, and the congestion information of the multiple receiving interfaces is used by the sending end to determine the time at which the receiving end receives the first message
  • the first receiving interface the receiving end receives the first message through the first receiving interface, and the first message includes the identifier of the first sending interface through which the sending end sends the first message and the identifier of the first receiving interface.
  • the receiving interface of each received message may be different, and the sending end can determine which receiving interface to receive the message according to the congestion information of multiple receiving interfaces. arts.
  • multiple receiving interfaces at the receiving end can work together, and the message rate of receiving packets at the receiving end can be understood as the sum of the message rates of multiple receiving interfaces, which improves the bandwidth efficiency of receiving packets at the receiving end.
  • the method further includes: when the receiving end determines that the first packet is received from the first receiving interface, determining whether other receiving interfaces of the plurality of receiving interfaces except the first receiving interface have received the first When the receiving end determines that no other receiving interface has received the repeated message of the first message, the receiving end controls the first receiving interface to process the first message; the receiving end determines that the other receiving interfaces in the When at least one receiving interface has received a duplicate packet of the first packet, the receiving end controls the first receiving interface to discard the first packet.
  • multiple receiving interfaces on the receiving end may be the receiving interfaces of the original message and the retransmitted message of the same message.
  • multiple different receiving interfaces receive repeated messages of the same message, in order to avoid The same message is processed repeatedly on different receiving interfaces.
  • a first receiving interface receives the first message, it is necessary to determine whether other receiving interfaces have received duplicate messages of the first message. If not received, the first receiving interface can process the first packet. But, as long as there is a duplicate message of the first message received by a receiving interface in other receiving interfaces, that is, the first message has been registered, the first receiving interface cannot process the first message, so as to avoid The text is processed repeatedly.
  • Determining whether the multiple receiving interfaces other than the first receiving interface have received the repeated message of the first message includes: the receiving end determines whether the first receiving interface has received the first registration request of other receiving interfaces, the first The registration request is used to request to process the first retransmission message.
  • the first retransmission message is a retransmission message of the first message; the receiving end determines that no other receiving interface has received the duplicate message of the first message Including: the receiving end determines that the first receiving interface has not received the first registration request sent by other receiving interfaces; the receiving end determines that at least one receiving interface in the other receiving interfaces has received the first message.
  • the repeated message includes: the receiving end determines The first receiving interface has received the first registration request sent by at least one receiving interface among the other receiving interfaces.
  • a first receiving interface wants to process the first message as the original message, the first receiving interface needs to determine whether to receive the first registration request for the first retransmission message . If a first registration request has been received, the first message arrives later than the first retransmission message; if the first registration request has not been received, the first message is considered to be the first to arrive, and the first A receiving interface may process the first message.
  • the method further includes: when the receiving end determines that the first receiving interface receives the first registration request sent by the second receiving interface among the other receiving interfaces after receiving the first message, the receiving end The first receiving interface is controlled to send a first registration response to the second receiving interface, where the first registration response is used to indicate that the second receiving interface fails to register. That is to say, if the first receiving interface is processing the first message and receives the first registration request for the first retransmission message from the second receiving interface, in order to prevent the message from being processed repeatedly, the first receiving interface needs to indicate The registration of the second receiving interface fails, so as to avoid repeated processing of the same packet.
  • determining whether multiple receiving interfaces other than the first receiving interface have received the repeated message of the first message includes: the receiving end Controlling the first receiving interface to send a first registration request to each of the other receiving interfaces, where the first registration request is used to request processing of the first message; the receiving end determines that none of the other receiving interfaces has received the first message
  • the repeated message includes: the receiving end determines that the first receiving interface has received multiple registration responses sent by other receiving interfaces, and the multiple registration responses indicate that the first receiving interface has successfully registered; the receiving end determines that at least one of the other receiving interfaces has received
  • the repeated message that the interface has received the first message includes: the receiving end determines that the first receiving interface has received a registration response sent by one of the other receiving interfaces indicating that the registration of the first receiving interface failed.
  • the first receiving interface is not sure whether the first packet is a duplicate packet that arrives later than the original packet, and the first receiving interface needs to report to other receiving interfaces Request permission to process the first packet, and if each registration response indicates that the first receiving interface can process the first packet, the first receiving interface processes the first packet. If there is a returned registration response indicating that the first receiving interface fails to register, it means that the first message received by the first receiving interface arrives at the receiving end later than the original message, and the first receiving interface cannot process the first message.
  • the method further includes: after the receiving end determines that the second receiving interface among the other receiving interfaces has received a first registration request, and then receives the original message of the first message, the receiving end controls The second receiving interface discards the original packet of the first packet. That is to say, if the second receiving interface has received a first registration request requesting to process the same message as the first message, it means that the receiving interface has already received the same message before the second receiving interface. When the second receiving interface receives the original message of the first message again, the second receiving interface has no authority to process the original message of the first message, so as to avoid repeated processing of the same message on different receiving interfaces.
  • the receiving end determines that after the first receiving interface receives the first message, the first receiving interface also receives the first message sent by the second receiving interface. Registration request, the first registration request is used to request to process the repeated message of the first message, if the number of the first receiving interface is less than the number of the second receiving interface, the receiving end controls the first receiving interface to reject the first registration ask.
  • This possible design needs to be executed simultaneously with the next possible design, specifically refer to the description in one possible design below.
  • the receiving end determines that the first receiving interface receives the first message, if the number of the first receiving interface is not the number of all receiving interfaces of the receiving end
  • the receiving end controls the first receiving interface to collect multiple registration responses about the first message from all receiving interfaces other than the first receiving interface;
  • the receiving interface sends a first registration response among the multiple registration responses, where the first registration response is used to indicate that the second receiving interface successfully registers for the repeated message of the first message.
  • the receiving interface with a smaller number You can directly reject the registration request of the receiving interface with a large number for the same retransmission message; at the same time, if a receiving interface has already received a retransmission message, the receiving interface with a large number needs to determine all other receiving interfaces about the retransmission message registration status. Moreover, the receiving interface with a large number needs to receive valid registration results, for example, one receiving interface registered successfully, some receiving interfaces registered successfully but did not receive the duplicate message, and some receiving interfaces failed to register, etc. , the receiving interface with a smaller number can determine the receiving interface that processes the duplicate message, so as to avoid the situation that multiple receiving interfaces repeatedly process when receiving multiple duplicate messages of the same message.
  • a communication device including: a processor, configured to determine the first sending interface for sending the first message according to the loads of multiple sending interfaces of the sending end when sending the first message, and according to the load of the receiving end
  • the congestion information of multiple receiving interfaces determines the first receiving interface receiving the first message
  • the transceiver includes a first sending interface, the first sending interface is used to send the first message, and the first message includes the first sending interface ID and ID of the first receiving interface.
  • the transceiver is further configured to: receive congestion information from multiple receiving interfaces sent by the receiving end, where the multiple receiving interfaces include the first receiving interface.
  • the processor is further configured to: when determining that sending the first message fails, determine the second receiving interface on which the receiving end receives the first retransmission message according to the congestion information of multiple receiving interfaces, and the first retransmission
  • the transmission message is a repeated message of the first message; the first sending interface is also used to send the first retransmission message, and the first retransmission message includes the identifier of the first sending interface, the identifier of the second receiving interface, and Marker for duplicate packets.
  • the transceiver when the first retransmission message arrives at the receiving end later than the first message, the transceiver further includes a second sending interface, and the second sending interface is also used to receive the first response of the first message message, the first response message includes the identification of the first sending interface; when the second sending interface is different from the first sending interface, the second sending interface is also used to send the first response message to the first sending interface;
  • the sending interface is further configured to send a first notification message to the first receiving interface, where the first notification message is used to instruct the first receiving interface to release the context information of the first packet.
  • the transceiver when the first message arrives at the receiving end later than the first retransmission message, the transceiver further includes a third sending interface, and the third sending interface is used to receive the first retransmission message of the first retransmission message.
  • a retransmission response message, the first retransmission response message includes the identifier of the first sending interface; when the third sending interface is different from the first sending interface, the third sending interface is also used to send the first retransmission to the first sending interface Response message; the first sending interface is further configured to send a second notification message to the second receiving interface, and the second notification message is used to instruct the second receiving interface to release the context information of the first retransmission message.
  • a communication device including: a transceiver, configured to send congestion information of multiple receiving interfaces to the sending end, and the congestion information of the multiple receiving interfaces is used by the sending end to determine the first packet received by the receiving end.
  • a receiving interface; the transceiver includes a first receiving interface, the first receiving interface is used to receive the first message, and the first message includes the identification of the first sending interface and the identification of the first receiving interface that the sending end sends the first message .
  • it also includes a processor, configured to: determine whether other receiving interfaces except the first receiving interface of the plurality of receiving interfaces have received the first packet when receiving the first message from the first receiving interface. A repeated message of a message; when it is determined that other receiving interfaces have not received the repeated message of the first message, control the first receiving interface to process the first message; determine that at least one receiving interface in other receiving interfaces has When a duplicate packet of the first packet is received, the first receiving interface is controlled to discard the first packet.
  • the first receiving interface is used to determine whether to receive the first registration request from other receiving interfaces, and the first registration request is used to request the first retransmission
  • the message is processed, and the first retransmission message is a retransmission message of the first message; it is determined that the first receiving interface has not received the first registration request sent by other receiving interfaces, and the first receiving interface is used to register the first registration request. or, determine that the first receiving interface has received the first registration request sent by at least one receiving interface among the other receiving interfaces, and the first receiving interface is used to discard the first message.
  • the first receiving interface after receiving the first message, when receiving the first registration request sent by the second receiving interface among the other receiving interfaces, the first receiving interface is also used to send the registration request to the second receiving interface A first registration response, where the first registration response is used to indicate that the registration of the second receiving interface fails.
  • the first receiving interface is configured to send a first registration request to each of the other receiving interfaces, and the first registration request is used to request The first message is processed; the multiple registration responses sent by other receiving interfaces received by the first receiving interface are determined, and the multiple registration responses all indicate that the registration of the first receiving interface is successful, and the first message is processed; The interface has received a registration response sent by one of the other receiving interfaces indicating that the registration of the first receiving interface failed, and controls the first receiving interface to discard the first packet.
  • the second receiving interface among the other receiving interfaces After the second receiving interface among the other receiving interfaces has received a first registration request, it receives the original message of the first message, and the second receiving interface is used to discard the first registration request.
  • the original message of the text After the second receiving interface among the other receiving interfaces has received a first registration request, it receives the original message of the first message, and the second receiving interface is used to discard the first registration request.
  • the original message of the text After the second receiving interface among the other receiving interfaces has received a first registration request, it receives the original message of the first message, and the second receiving interface is used to discard the first registration request.
  • the original message of the text After the second receiving interface among the other receiving interfaces has received a first registration request, it receives the original message of the first message, and the second receiving interface is used to discard the first registration request.
  • the original message of the text After the second receiving interface among the other receiving interfaces has received a first registration request, it receives the original message of the first message, and the second receiving interface is
  • the first receiving interface when the first message is a retransmission message: after the first receiving interface receives the first message, the first receiving interface also receives the first registration request sent by the second receiving interface, When the first registration request is used to request to process the repeated message of the first message, if the number of the first receiving interface is smaller than the number of the second receiving interface, the first receiving interface is used to reject the first registration request.
  • the first receiving interface when the first message is a retransmission message: after the first receiving interface receives the first message, if the number of the first receiving interface is not the smallest number of all receiving interfaces at the receiving end Number, the first receiving interface, used to collect multiple registration responses about the first message from all receiving interfaces other than the first receiving interface; Sending a first registration response among the multiple registration responses, where the first registration response is used to indicate that the second receiving interface successfully registers for the repeated message of the first message.
  • a communication device including at least one processor, multiple sending interfaces, and multiple receiving interfaces, at least one processor is connected to the memory, multiple sending interfaces, and multiple receiving interfaces through a system bus, and at least one processing
  • the device is used to read and execute the program stored in the memory, and the program can act on multiple sending interfaces and multiple receiving interfaces, so that the device executes the method according to the first aspect or any one of the first aspect.
  • a chip is provided, the chip is coupled with a memory, the chip includes a plurality of pins coupled to a sending interface and a plurality of receiving interfaces, and the chip is used to read and execute program instructions stored in the memory to achieve the above-mentioned The first aspect or the method of any one of the first aspects.
  • an electronic device including multiple sending interfaces, multiple receiving interfaces, one or more processors, and one or more memories.
  • the one or more memories are coupled with one or more processors, the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device performs A message transmission method in any one of the above aspects and any possible implementation manner.
  • the embodiment of the present application provides a computer-readable storage medium, including computer instructions, which, when the computer instructions are run on the electronic device, cause the electronic device to perform any of the above-mentioned aspects and any of the possible implementations.
  • Message transmission method
  • the embodiment of the present application provides a computer program product.
  • the computer program product runs on a computer or a processor
  • the computer or processor executes the report in any one of the above-mentioned aspects and any possible implementation. file transfer method.
  • the embodiment of the present application provides a system, and the system may include the sending end and the receiving end in any possible implementation manner of any of the above aspects.
  • the sending end and the receiving end may implement the message transmission method in any one of the foregoing aspects and any possible implementation manner.
  • any of the sending end, receiving end, chip system, computer-readable storage medium, or computer program product provided above can be applied to the corresponding method provided above. Therefore, the achieved For the beneficial effect, reference may be made to the beneficial effect in the corresponding method, which will not be repeated here.
  • FIG. 1 is a schematic diagram of a TCP subflow processing architecture in an MPTCP technology provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of a bandwidth aggregation architecture in RDMA bonding provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of a multi-path transmission technology architecture provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a multi-interface coordination architecture in a multi-path transmission technology provided by an embodiment of the present application
  • FIG. 5 is a schematic diagram of a network hardware interface structure in a network device provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of an architecture of a sender performing packet splitting and interface scheduling provided by an embodiment of the present application
  • FIG. 7 is a schematic diagram of an architecture of a receiving end performing message splitting and interface scheduling provided by an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of a message transmission method provided by an embodiment of the present application.
  • FIG. 9 is a schematic flowchart of a message transmission method provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of signal interaction in a message transmission method provided in an embodiment of the present application.
  • FIG. 11 is a schematic diagram of signal interaction in a message transmission method provided in an embodiment of the present application.
  • FIG. 12 is a schematic diagram of signal interaction in a message transmission method provided by an embodiment of the present application.
  • FIG. 13 is a schematic diagram of signal interaction in a message transmission method provided in an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 15 is a schematic structural diagram of a communication device provided in an embodiment of the present application.
  • FIG. 16 is a schematic structural diagram of a communication device provided by an embodiment of the present application.
  • MPTCP When establishing a TCP connection, only one IP address has always been bound, but with more and more hosts with multiple network cards, data can often be reached by multiple links from one host to another. is to allow TCP connections to use multiple paths to maximize channel resource usage.
  • RDMA Remote Direct Memory Access
  • QP Queue Pair
  • Host Channel Adapter It can be used in enterprise data centers, high-performance computing, and embedded environments, and provides high-bandwidth, low-latency solutions for server/storage cluster applications.
  • applications such as clustered databases, parallel applications, processing services, and high-performance embedded input/output (Input/Out, I/O)
  • the HCA card can maximize the performance of the application, shorten the operating cycle, and reduce the resource loss.
  • Infiniband InfiniBand, IB: A computer network communication standard for high-performance computing, which has extremely high throughput and extremely low latency, and is used for data interconnection between computers. InfiniBand is also used as a direct or switched interconnect between servers and storage systems, and as an interconnect between storage systems.
  • first and second are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, a feature defined as “first” and “second” may explicitly or implicitly include one or more of these features. In the description of this embodiment, unless otherwise specified, “plurality” means two or more.
  • a common method is to provide multiple sets of physical interfaces and cooperate with multi-path transmission technology to increase communication bandwidth.
  • the original TCP protocol includes an Internet Protocol (Internet Protocol, IP) layer, a TCP layer and an application layer (application).
  • IP Internet Protocol
  • application application layer
  • the TCP protocol can be shown as (b) in FIG. 1 , and the TCP flow of a single socket in the user layer can be split into multiple TCP subflows (subflow) at the MPTCP layer.
  • TCP subflow here is transmitted through a separate path, and each TCP subflow corresponds to a TCP connection.
  • multiple received TCP sub-flows need to be reassembled before processing.
  • each TCP subflow needs to be transmitted after three handshakes and four handshakes between the sending end and the receiving end.
  • the resource overhead of software and hardware is relatively large.
  • TCP is located at the transport layer, and its role is to provide reliable byte stream services.
  • the TCP protocol adopts a three-way handshake strategy.
  • the three-way handshake strategy includes: 1) The sending end first sends a data packet with a synchronization (synchronize, SYN) flag to the receiving end. 2) After receiving, the receiving end returns a data packet with a SYN/acknowledgment (Acknowledgment, ACK) flag to the sending end, so as to convey the confirmation information of the received data packet to the sending end. 3) The sending end returns a data packet with an ACK flag to the receiving end to indicate that the sending end has received the confirmation information, and the three-way handshake ends.
  • TCP connections are full-duplex, each direction must be closed separately.
  • the principle is that when one party completes its data sending task, it can send a FIN (FinNet) to terminate the connection in this direction. But receiving a FIN only means that there is no data flow in this direction, and a TCP connection can still send data after receiving a FIN.
  • the party that does the shutdown first (the active party) will perform the active shutdown, while the other party (the passive party) will perform the passive shutdown.
  • the passive party receives the FIN message notification from the active party, it only means that the active party has no data to send to the passive party.
  • This process can be called four hand waving, and specifically includes: 1) The active party sends a FIN to the passive party, which is used to indicate that the data transmission from the active party to the passive party is to be closed. 2) The passive party receives FIN, and the passive party sends ACK to the active party. 3) The passive release closes the connection with the active party, and sends a FIN to the active party. 4) After the active party receives the FIN in 3), the active party sends back an ACK to the passive party, indicating that the FIN in 3) has been received
  • each TCP subflow maintains a connection state, including the state of three-way handshake and four-way handshake, the resource overhead is relatively large.
  • the sending end needs to bind a physical interface for a TCP sub-flow, and the receiving end also needs to bind a physical interface for the TCP sub-flow. Traffic congestion is prone to occur in this interface binding, and the processing bandwidth of the device will be limited.
  • RDMA bonding another multipath technology, the RDMA in a sender or receiver can be aggregated into a set of logical resources to realize bandwidth aggregation.
  • RDMA HAL hardware abstraction layer, hardware abstraction layer
  • services can provide the API for the Application to call HCA.
  • RDMA bonding can shield the hardware differences of the HCA by binding one of the two HCAs to achieve multiple With RDMA resource pooling, users do not perceive RDMA device differences.
  • RDMA HAL and services can assign the QP to be processed by the Application to a specific HCA for processing according to some rules.
  • the Bonding HAC driver can drive the HAC allocated by RDMA HAL and services to the QP to process the QP.
  • HCA can be selected based on QP to establish a communication connection, but at the same time, there is also the software and hardware overhead of increasing scheduling and hardware shielding, which is similar to the technology of bonding interfaces in MPTCP. Moreover, there is also the problem that a certain HCA is prone to traffic congestion, and the processing bandwidth of the device is limited.
  • connection 1 transmits data through physical interface 0
  • the sender can only send data on physical interface 1.
  • data on connection 2 can only be sent on physical interface 1.
  • Multiple physical interfaces on the sending side are not working together.
  • it is also bound to a fixed physical port to receive data of a single connection.
  • the present application provides a message transmission method, and the message can be understood as the above data.
  • the current constraint that a connection needs to be bound to a physical interface can be broken, and multiple physical interfaces on the device can transmit data for a single connection at the same time, improving message processing efficiency.
  • the physical interface of the sending end is not bound to send the message, and the receiving mechanism of the same communication flow is not bound.
  • the physical interface at the receiving end receives packets.
  • each of the multiple packets to be sent in the connection may be sent through a different physical interface, for example, packet 1 in the connection is sent through physical interface 0 , packet 2 is sent through physical interface 1.
  • packet 1 in the connection is sent through physical interface 0
  • packet 2 is sent through physical interface 1.
  • the sending end has 3 physical interfaces
  • a single connection can send data through these 3 physical interfaces, and the maximum message rate can reach the sum of all interfaces.
  • the interface for receiving each message may also be different.
  • the multi-interface coordinated multi-path transmission of this application can be applied to various scenarios, for example, it can be applied to multiple nodes in a data center.
  • the sending end of this application can be a computing node
  • the receiving end can be a storage node. node, or the sending end is a storage node, and the receiving end is a computing node, etc.
  • it can also be applied to multiple computing nodes of a high-performance computing (High Performance Computing, HPC) cluster.
  • HPC High Performance Computing
  • the present application can be applied to network equipment, and the network equipment can be a sending end or a receiving end, and the sending end and the receiving end can be applied in the above scenarios.
  • the data transmission between the sending end and the receiving end may be multi-path transmission.
  • the network device can be used as both the sending end and the receiving end. Therefore, as shown in Figure 4, the network device includes multiple sending interfaces and multiple receiving interfaces, which means that the sending end includes multiple sending interfaces and multiple receiving The receiving end also includes multiple receiving interfaces and multiple sending interfaces.
  • the sending interface of the sending end can be used to send the message, multiple receiving interfaces can be used to receive the response of the message, the receiving interface of the receiving end can be used to receive the message, and the receiving end’s
  • the sending interface can be used to send the response of the message.
  • the sending interface and receiving interface in this application can be understood as network hardware interfaces of network devices.
  • the network device may include an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) and a memory.
  • ASIC Application Specific Integrated Circuit
  • Memory can be used to store data.
  • An ASIC may include a processor, other components, and a network hardware interface. All components within the ASIC can communicate data through the system bus.
  • the processor can be used to run an operating system and software protocol stack, etc. Other components can be used to cooperate with the processor to complete various functions. This application can be improved for the network hardware interface in ASIC.
  • the network hardware interface includes a transport layer controller, multiple sending interfaces (such as sending interface 1, sending interface 2 and sending interface 3 shown in Figure 5) and multiple receiving interfaces (such as Receiving interface 1, receiving interface 2 and receiving interface 3 are shown).
  • the sending interface can send messages to other external devices through the switch, and the receiving interface can receive messages from other external devices through the switch.
  • the transport layer controller includes a congestion control module, a reliable transmission module, and a task scheduling module.
  • the reliable transmission module is used to determine whether the sending end needs to retransmit the message according to the message response received by the receiving interface when the network device acts as the sending end.
  • the reliable transmission module is also used to cooperate with multiple receiving interfaces to perform packet deduplication processing to ensure that the same message can only execute different More than once, the details will be introduced later.
  • the congestion control module is used to cooperate with the task scheduling module to realize the flexible selection of the sending interface and the receiving interface in multi-path transmission;
  • the task scheduling module is used to edit the message to be sent and select the sending interface and receiving interface according to the sending interface determined by the congestion control module;
  • the sending interface is used to send messages according to the instructions of the task scheduling module.
  • the receiving interface is used to receive messages.
  • the multi-interface message registration mechanism can be used to realize the coordinated processing of multiple interfaces to ensure that the same message will not be processed repeatedly. That is, the message registration mechanism here is used to avoid repeated processing of messages.
  • the task scheduling module can communicate with all sending interfaces, and the sending interfaces are all decoupled, and each sending interface can maintain its own sending side context.
  • the sending side context may include the sequence number and connection status of the message, and so on.
  • the sending interface can also feed back the congestion information of its own interface, the received congestion information of the receiving end, the load status (packet receiving situation) of the sending interface, and the fault information of the sending interface to the task scheduling module.
  • the task scheduling module is used to perform packet splitting scheduling, and determine the sending interface for sending the message according to the load status of multiple sending interfaces, so as to send the message.
  • the transport layer at the sending end can split message 1 into sub-message 1, sub-message 2 and sub-message 3, and message 2 is not split, when When the task scheduling module receives the three sub-messages and message 2 from the transport layer, assuming that the multi-sending end has three sending interfaces, the task scheduling module can schedule sub-message 1 to sending interface 1 according to the load status of the three sending interfaces , dispatch sub-message 2 and sub-message 3 to sending interface 2, and dispatch message 2 to sending interface 3.
  • sending interface 1 can store the context of sub-message 1
  • sending interface 2 can store the contexts of sub-message 2 and sub-message 3
  • sending interface 3 can store the context of message 2.
  • a message in this application can be understood as a sub-message or an undivided message.
  • the sending interface also needs to add a retransmission mark to the packets so that the receiving end can identify the retransmitted packets.
  • packets in the multi-interface coordination mechanism of the receiving end, taking into account packet discarding and retransmission caused by network congestion, in a reliable transmission scenario, packets can enter from any receiving interface at the receiving end. When retransmitting for the first time, it is necessary to ensure that the same message is processed at most once at the receiving end.
  • a registration mechanism for retransmitted messages is introduced, and receiving interfaces can access each other to avoid repeated processing of messages on different receiving interfaces.
  • a receiving interface when a receiving interface receives a retransmission message, it needs to query and register the processing authority of the retransmission message from other receiving interfaces in the receiving end except the receiving interface. As long as one receiving interface has processed the original message or the retransmitted message of the retransmitted message, it can be determined that the retransmitted message received by the receiving interface is a repeated message.
  • the receiving interface that initiates the query can wait for all other receiving interfaces to complete the confirmation before processing the message, such as discarding the message.
  • the receiving interfaces are processed independently, and the transaction execution sequence can be controlled by the transaction execution sequence management module.
  • the transaction here can be understood as a message, and the above-mentioned transport layer controller can include a transaction execution sequence management module.
  • the above-mentioned transport layer controller can include a transaction execution sequence management module.
  • receiving interface 1 in the receiving end receives sub-message 1 of message 1
  • receiving interface 2 receives sub-message 2 and sub-message 3 of message 1
  • receiving interface 3 receives message 2
  • the receiving interface 1 receives sub-message 1 and sub-message 3 of message 2
  • the receiving interface 3 can establish the context of message 2.
  • the transaction execution sequence management module receives multiple sub-messages transmitted by each receiving interface, it can further process the multiple sub-messages according to the message sequence numbers of the sub-messages.
  • the sending interface of the sending end can not only be used to send messages, but also receive response messages from the peer end.
  • the sending interface of the sending end receives different resources for receiving the response message and the receiving interface of the sending end receives the resource of the message directly sent from the opposite end.
  • the network cable through which the sending interface of the sender receives the response message is physically the same as the network cable through which the receiving interface of the sender receives the message directly sent by the peer, the virtual channel and the virtual channel through which the sending interface receives the response message are physically the same.
  • the virtual channels used by the receiving interface of the sending end to receive the packets directly sent by the peer end are independent of each other and do not affect each other.
  • the receiving interface of the receiving end can not only receive messages, but also send response messages to the peer end.
  • the resources of the response message sent by the receiving interface of the receiving end are different from the resources of the message directly sent by the sending interface of the receiving end.
  • the network cable through which the receiving interface of the receiving end sends the response message is physically the same as the network cable through which the sending interface of the receiving end directly sends the message
  • the virtual channel through which the receiving interface of the receiving end sends the response message is the same as the sending interface of the receiving end.
  • the virtual channels that directly send packets to the peer end are independent of each other and do not affect each other.
  • the present application provides a message transmission method, as shown in FIG. 8 , the method includes the following processing flow.
  • the sending end determines to send the first message, determine the first sending interface for sending the first message according to the loads of the multiple sending interfaces at the sending end, and determine to receive the first message according to the congestion information of the multiple receiving interfaces at the receiving end The first sending interface of .
  • the first message may be the above message or sub-message.
  • the sending end receives congestion information from multiple receiving interfaces sent by the receiving end, and the multiple receiving interfaces include the first receiving interface.
  • the congestion information may be indication information of a packet loss rate of a receiving interface, or indication information of a packet queue length, etc., which are not limited in this application.
  • the sending end can not only determine the first sending interface for sending the first packet according to the load status of the multiple sending interfaces of the sending end, but also determine the first receiving interface for receiving the first packet according to the congestion information of the multiple receiving interfaces.
  • the congestion control module in the sending end may determine the first receiving interface from multiple receiving interfaces according to a preset algorithm. This application does not limit the implementation of the algorithm.
  • Each sending interface in the sending end can send the load status of the sending interface itself to the task scheduling module, and the task scheduling module can determine the first sending interface from the multiple sending interfaces according to the load status.
  • the method of determining the first sending interface according to the load state is not limited.
  • the load state may be indication information of the current packet queue length of each sending interface, or indication information of packet loss rate, etc., which are not limited in this application.
  • the task scheduling module then designates the first sending interface to send the first message.
  • the sending end sends a first packet through the first sending interface, where the first packet includes an identifier of the first sending interface and an identifier of the first receiving interface.
  • a field segment may be added to the first message to record the identifiers of the first sending interface and the first receiving interface that send the first message.
  • this application can endow the sending end with the ability to flexibly select the sending interface and receiving interface of the message, and can make full use of the multi-interface coordination between the sending end and the receiving end to increase bandwidth.
  • the present application provides a message transmission method, as shown in FIG. 9 , the method includes the following processing flow.
  • the receiving end sends congestion information of multiple receiving interfaces to the sending end, and the congestion information of the multiple receiving interfaces is used by the sending end to determine a first receiving interface on which the receiving end receives the first packet.
  • the congestion information here can refer to the description in step 801 .
  • the sending end After the sending end has determined the first receiving interface for sending the first message, in reliable transmission, the sending end can send the indication information of the first receiving interface for receiving the first message to the receiving end through a message on the control plane, so as to receive The end may receive the first packet on the first receiving interface.
  • the receiving end receives the first packet through the first receiving interface, and the first packet includes the identifier of the first sending interface through which the sending end sends the first packet and the identifier of the first receiving interface.
  • the first message can be received through the first receiving interface.
  • the sending end determines that the receiving interface for receiving the second message is the second receiving interface, the receiving end can receive the second message through the second receiving interface.
  • the flexible selection of the receiving interface for receiving the message can also be realized, and the multi-interface coordination of the receiving end can be fully utilized to increase the bandwidth.
  • the sending end in this application can flexibly select the sending interface and the receiving interface each time a message is sent, and considering that there is also a failure to send the message, when the sending end sends the retransmission message of the original message, it can be sent
  • the sending interface of the original message continues to send the retransmission message, and may also re-determine the sending interface for sending the retransmission message according to the current load status of multiple sending interfaces.
  • the sender can also determine the receiving interface for receiving the retransmission message.
  • the receiving interface for receiving the retransmission message may be the same as or different from the receiving interface for receiving the original message.
  • the sending end if the sending end does not receive the ACK message of the message within the specified time period, the sending end considers that the sending of the message fails, and continues to send the retransmission message of the message. However, the original message of the message may not be lost, but the sending end did not receive the ACK message of the message within the specified time period. In this case, for the receiving end, the receiving end may be in a The receiving interface receives the original message of the message first, and it is also possible that a retransmission message of the same message is received first on a receiving interface.
  • the receiving end Based on the principle that a message can only be processed once at the receiving end, if the original message arrives at the receiving end before the retransmitted message, the receiving end only processes the original message; if a retransmitted message arrives at the receiving end before the original message After arriving at the receiving end, the receiving end only processes this retransmission message to avoid repeated processing of the same message on multiple receiving interfaces.
  • the present application provides a message transmission method, assuming that the sender sends the original message and the retransmission message of the same message through the same sending interface, as shown in FIG. 10 , the method includes the following process.
  • the sending end sends the first packet on the first sending interface T1.
  • the first message includes the identifier of the first sending interface T1 of the sending end and the identifier of the first receiving interface R1 of the receiving end.
  • the sending end determines that sending the first message fails, it determines the second receiving interface R2 on which the receiving end receives the first retransmission message according to the congestion information of multiple receiving interfaces, and the first retransmission message is the first message of the first message. Duplicate message.
  • the sending end determines that sending the first message fails. Before sending the duplicate message of the first message, that is, the first retransmission message, the sending end determines the second receiving interface R2 for receiving the first retransmission message again according to the congestion information of multiple receiving interfaces.
  • the second receiving interface R2 may be the same as or different from the first receiving interface R1. In this embodiment, R2 and R1 are different.
  • the sending end sends a first retransmission message through the first sending interface T1, where the first retransmission message includes an identifier of the first sending interface T1, an identifier of the second receiving interface R2, and a marker of the repeated message.
  • the mark of the repeated packet is used to indicate that the first retransmitted packet is a retransmitted packet.
  • the first retransmission message will also carry the original message, that is, the sequence number M of the first message, and M is an integer greater than or equal to 0, so that the receiving end can determine that the first retransmission message is the one with the sequence number M.
  • a retransmission packet of the first packet is used to indicate that the first retransmitted packet is a retransmitted packet.
  • the first retransmission message will also carry the original message, that is, the sequence number M of the first message, and M is an integer greater than or equal to 0, so that the receiving end can determine that the first retransmission message is the one with the sequence number M.
  • step 104 when the first retransmitted message arrives at the receiving end later than the first message, step 104 is performed, or when the first message arrives at the receiving end later than the first retransmitted message, step 107 is performed.
  • the receiving end feeds back a first response packet of the first packet, where the first response packet includes the identifier of the first sending interface T1.
  • the sending end receives the first response message of the first message through the second sending interface T2, and the second sending interface T2 may be the same as or different from the first sending interface T1.
  • T2 and T1 are different.
  • the receiving end Before sending the first response message, the receiving end may send the identifier of the second sending interface T2 to the sending end through control plane signaling, so that the sending end receives the first response message through the second sending interface T2.
  • the sending end controls the second sending interface T2 to send the first response packet to the first sending interface T1.
  • the sending end needs to transmit the first sending interface T2 through the second sending interface T2.
  • the response message is sent to the first sending interface T1.
  • the sending end controls the first sending interface T1 to send a first notification message to the first receiving interface R1, where the first notification message is used to instruct the first receiving interface R1 to release the context information of the first packet.
  • the sending end Since the first receiving interface R1 stores the context information of the first message when receiving the first message, if the first sending interface T1 receives the first response message, the sending end determines that the first message is sent successfully, The sending end can then control the first sending interface T1 to notify the first receiving interface R1 to release the context information of the first packet.
  • the receiving end feeds back a first retransmission response message of the first retransmission message, where the first retransmission response message includes the identifier of the first sending interface T1.
  • the sending end receives the first retransmission response message through the third sending interface T0, and the third sending interface T0 may be the same as or different from the first sending interface T1.
  • T0 and T1 are different.
  • the receiving end Before sending the first retransmission response message, the receiving end may send the identifier of the third sending interface T0 to the sending end through control plane signaling, so that the sending end receives the first retransmission response message through the third sending interface T0.
  • the sending end controls the third sending interface T0 to send the first retransmission response packet to the first sending interface T1.
  • the sending end needs to transmit the first sending interface T0 through the third sending interface T0.
  • the retransmission response message is sent to the first sending interface T1.
  • the sending end controls the first sending interface T1 to send a second notification message to the second receiving interface R2, where the second notification message is used to instruct the second receiving interface R2 to release the context information of the first retransmission packet.
  • the sender can control the first sending interface T1 to notify the second receiving interface R2 to release the context information of the first retransmission message.
  • a multi-interface coordination mechanism can be adopted when sending and receiving messages, so that all sending interfaces of the sending end and all receiving interfaces of the receiving end can serve a single message at the same time, so as to achieve full utilization The capacity of the device bandwidth.
  • the retransmitted message may come in from any receiving interface of the receiving end.
  • the receiving end determines that the first message is received from the first receiving interface, it determines that the Whether other receiving interfaces other than the receiving interface have received duplicate packets of the first packet.
  • the first packet here may be an original packet or a retransmission packet.
  • the receiving end determines that none of the other receiving interfaces has received the repeated message of the first message, the receiving end controls the first receiving interface to process the first message; if the receiving end determines that at least one receiving interface in the other receiving interfaces has received When receiving a duplicate packet of the first packet, the receiving end controls the first receiving interface to discard the first packet. That is to say, each time the receiving interface of the receiving end receives a message, it needs to judge whether the receiving interface can process the message, so as to avoid repeated processing of the message.
  • a registration mechanism is provided at the receiving end.
  • This registration mechanism is aimed at retransmission messages, that is, when a receiving interface of the receiving end receives a retransmission message, it needs to request registration from other receiving interfaces. If a certain If the packet with the sequence number has been registered, the receiving interface cannot process the retransmitted packet. Or, when a receiving interface receives the original message, if the receiving interface determines that a retransmission message with the same content as the original message has been registered, the original message will also be processed as a repeated message, that is, it will not be processed the original message.
  • the sending end sends the first packet through the first sending interface T1, and the first sending interface T1 does not receive the response packet of the first packet within a specified time, and determines that the first packet needs to be retransmitted.
  • the first packet in case 1 is an original packet.
  • the first sending interface T1 does not receive the ACK message of the first message within a specified time, it is determined that the sending of the first message fails.
  • the first receiving interface R1 of the receiving end receives the first message, and the receiving end determines whether the first receiving interface R1 receives the first registration request from other receiving interfaces, and the first registration request is used to request retransmission of the first message to process. If it is determined that the first receiving interface R1 has not received the first registration request from other receiving interfaces, the receiving end processes the first packet at the first receiving interface R1.
  • the first retransmission message is a retransmission message of the first message.
  • step 111 the sending end determines that the sending of the first message has failed, it is possible that the first message just did not arrive at the receiving end on time, and arrived at the receiving end late. If the sending end retransmits the first message, the receiving end may experience repeated processing of the message on different receiving interfaces. Therefore, when the first receiving interface R1 receives an original message, that is, the first message here, it needs to determine whether there is a receiving interface that has received the first retransmission message among other receiving interfaces. Here, by determining the first It is determined whether the receiving interface R1 has received the first registration request from other receiving interfaces.
  • the first receiving interface R1 has received the first registration request sent by at least one of the other receiving interfaces, it means that the retransmission message of the first message arrives at the receiving end later than the first message, specifically will be introduced in case two.
  • the second receiving interface R2 of the receiving end receives the first retransmission message, and controls the second receiving interface R2 to send the first registration request to other receiving interfaces (R1 and R0), so as to determine whether the second receiving interface R2 can handle the first registration request.
  • a retransmission message A retransmission message.
  • the second receiving interface R2 receives the first retransmission message
  • the second receiving interface R2 needs to send the first registration request to the first receiving interface R1 and the third receiving interface R0 to determine whether the second receiving interface R2 can process the first retransmission message.
  • a retransmission message A retransmission message.
  • the third receiving interface Since the third receiving interface does not receive the first packet, the third receiving interface feeds back a registration success response to the second receiving interface R2.
  • the registration status recorded by the third receiving interface R0 is equivalent to that the third receiving interface gives the second receiving interface R2 the right to process the first retransmission message.
  • the receiving end determines that the first receiving interface R1 is processing the first packet, and controls the first receiving interface R0 to feed back a registration failure response to the second receiving interface R2.
  • the receiving end determines that the first receiving interface R1 receives the first registration request sent by the second receiving interface R2 among the other receiving interfaces after receiving the first message
  • the receiving end controls the first receiving interface R1 to
  • the second receiving interface R2 sends a first registration response, where the first registration response is used to indicate that the second receiving interface R2 fails to register.
  • the first receiving interface R1 does not give the second receiving interface R2 the right to process the first retransmission message.
  • the second receiving interface R2 of the receiving end determines that the second receiving interface R2 does not need to process the first retransmission message, and controls the second receiving interface R2 to send 3.
  • the receiving interface R0 sends a registration clear request.
  • the second receiving interface R2 can recognize that the first retransmitted message does not need to be processed by this receiving interface.
  • the third receiving interface R0 Since the third receiving interface R0 also fed back a successful registration response to the second receiving interface R2 before, when the second receiving interface R2 determines that it does not need to process the first retransmission message, the second receiving interface R2 also needs to report to the third receiving interface The interface R0 sends a registration clear request, so that the third receiving interface R0 deletes the registered registration state.
  • the first receiving interface R1 of the receiving end After successfully receiving the first packet, the first receiving interface R1 of the receiving end sends a second packet to the third sending interface T0 of the sending end, where the second packet is used to indicate that the first packet has been received.
  • the first receiving interface R1 is finally the only interface for processing the first message.
  • the first receiving interface R1 After the first receiving interface R1 successfully receives the first packet, it is determined that the result of the successful reception of the first packet needs to be returned, that is, the second packet here.
  • the receiving end similar to the sending end, may also determine the receiving interface for receiving the second message according to the load status of multiple receiving interfaces of the sending end. Assuming that the interface determined to receive the second packet is the third sending interface T0, the second packet may carry the identifier of the first sending interface that the sending end sent the first packet at that time.
  • the third sending interface T0 of the sending end sends an indication that the first packet is successfully received to the first sending interface T1, and the sending end controls the first sending interface T1 to send a message for releasing processing resources to the first receiving interface R1.
  • the first receiving interface R1 stores context information for processing the first packet, and when the first receiving interface R1 receives a message for releasing processing resources, the first receiving interface R1 may clear the context information of the first packet.
  • the registration mechanism provided by this application can effectively identify the retransmitted message as a repeated message, and the repeated message will not be received Port processing to avoid repeated processing of the same packet on different receiving interfaces.
  • the sending end sends a first packet through the first sending interface T1, and the first packet carries the identifier of the first sending interface T1 and the identifier of the second receiving interface R2 of the receiving end.
  • the sending end may send an event that the second receiving interface R2 is used to receive the first message to the receiving end through control plane signaling.
  • the sending end determines that the ACK message of the first message has not been received within the specified time, and the sending end sends the first retransmission message through the first sending port T1.
  • the first retransmission message is a retransmission message of the first message. Send message.
  • the first retransmission message includes the message sequence number of the first message, and also includes the identifier of the first sending interface T1 and the identifier of the first receiving interface R1.
  • the sending end determines through the congestion information that the receiving interface for receiving the first message is the second receiving interface R2, and that the receiving interface for receiving the first retransmitted message is the first receiving interface R1, proceed to step 123.
  • the receiving end receives the first retransmission packet on the first receiving interface R1.
  • the first retransmitted packet arrives at the receiving end earlier than the first packet.
  • the receiving end controls the first receiving interface R1 to send a first registration request to the second receiving interface R2 and the third receiving interface R0 respectively, where the first registration request is used to request processing of the first retransmission packet.
  • the receiving end controls the first receiving interface R1 to send a message to each receiving interface ( R0 and R3) send a first registration request, where the first registration request is used to request processing of the first retransmission message.
  • the receiving end determines that the second receiving interface R2 has not processed the duplicate message of the first retransmission message, and controls the second receiving interface R2 to feed back a successful registration response to the first receiving interface R1; the control end determines that the third receiving interface R0 also The repeated message of the first retransmitted message is not processed, and the third receiving interface R0 is controlled to feed back a registration success response to the first receiving interface R1.
  • the first receiving interface R1 has received the registration success responses of all other receiving interfaces of the receiving end, and the first receiving interface R1 has obtained the authority to process the first retransmission message.
  • the authority to process the first retransmission message has been registered, and the second receiving interface R2 and the third receiving interface R0 no longer have the right to process the first retransmission message and
  • the second receiving interface R2 and the third receiving interface R0 record the first retransmission message as being registered.
  • the receiving end controls the second receiving interface R2 to discard the first packet.
  • step 122 the sending end determines that the sending of the first packet fails, in fact the first packet only arrives at the receiving end later than the first retransmitted packet.
  • the receiving end determines that the second receiving interface R2 of the receiving end has received a first registration request, it means that the second receiving interface R2 has given the first receiving interface R1 the right to process the first retransmission message.
  • the receiving end controls the second receiving interface R2 to discard the first packet.
  • the receiving end determines that the first receiving interface R1 feeds back the second message to the first sending interface T1 of the sending end when it finishes processing the first retransmission message, and the second message indicates that the first receiving interface R1 has received the first retransmission message. Retransmit the message.
  • the sending end controls the first sending interface T1 to send an indication of releasing processing resources to the first receiving interface R1.
  • the first receiving interface R1 When the first receiving interface R1 processes the first retransmission message, it will save the context of the first retransmission message. When the first receiving interface R1 receives the instruction to release processing resources, it may delete the context of the first retransmission packet saved by the first receiving interface R1.
  • the receiving end controls the first receiving interface R1 to send a registration clear message to the second receiving interface R2 and the third receiving interface R0 to instruct the second receiving interface R2 and the third receiving interface R0 to clear the registration status of the first retransmission message .
  • the second receiving interface R2 and the third receiving interface R0 delete the registration status recorded in step 125 .
  • the receiving end when the retransmitted message of the same message arrives at the receiving end before the original message, the receiving end can only process one retransmitted message, and the original message that arrives later is regarded as a repeated message and cannot be processed to avoid repeated processing of the same message.
  • this application provides a method to avoid the same message A mechanism for handling conflicts between multiple retransmitted packets, which meets all of the following conditions:
  • the receiving interface with a small number can directly reject the receiving interface with a large number to register the retransmission message;
  • the receiving interface with a large number must collect the registration responses of all other receiving interfaces before sending a registration response to the receiving interface with a small number;
  • the sending end determines that the sending of the first message fails, and sends the first retransmission message to the receiving end through the first sending interface R1, and the first retransmission message includes the identifier of the first sending interface R1 and the ID of the third receiving interface R0 logo.
  • the first retransmission message is a retransmission message of the first message.
  • This embodiment is described on the basis that the original message, that is, the first message fails to be sent and does not reach the receiving end.
  • the sending end can determine that the receiving interface for receiving the first retransmission message is the third receiving interface R0 through the congestion information of the receiving interface of the receiving end, and carry the first retransmission message in the first retransmission message.
  • the sending end determines that the sending of the first retransmission message fails, and sends the second retransmission message to the receiving end through the first sending interface R1, and the second retransmission message includes the identifier of the first sending interface R1 and the first receiving interface Identification of R1.
  • the sending end decides to continue sending the second retransmission message on the first sending interface R1
  • the second retransmission message can be understood as the second retransmission message of the first message.
  • the sending end determines through the congestion information of the receiving interface of the receiving end that the receiving interface for receiving the second retransmission message is the first receiving interface R1
  • the second retransmission message carries the identifier of the first sending interface R1 and the first receiving interface R1. ID of interface R1.
  • the receiving end controls the third receiving interface R0 to request the first receiving interface R1 and the second receiving interface R2 to register the first retransmission packet.
  • the third receiving interface R0 when the third receiving interface R0 receives the first retransmission message, in order to avoid the same message from being processed repeatedly, the third receiving interface R0 needs to request the first receiving interface R1 and the second receiving interface R2 to register the first retransmission message, that is, request to obtain the permission to process the first retransmission message.
  • the receiving end controls the first receiving interface R1 to request the third receiving interface R0 and the second receiving interface R2 to register the second retransmission packet.
  • step 133 when the first receiving interface R1 receives the second retransmission message, in order to avoid repeated processing of the same message, the first receiving interface R1 needs to send the second receiving interface R2 and the third receiving interface R0 Requesting registration for the second retransmission message means requesting permission to process the second retransmission message.
  • the receiving end determines that the second receiving interface R2 first receives the registration request from the first receiving interface R1, if the second receiving interface R2 does not receive the retransmission message of the first message, control the second receiving interface R2 to The first receiving interface R1 returns a registration success response.
  • the second receiving interface R2 first receives the registration request of the first receiving interface R1
  • the second receiving interface R2 can send the registration request to the first receiving interface R2.
  • a receiving interface returns the authority to process the second retransmission message, and records the registration state that the second retransmission message has been registered by the first receiving interface R1.
  • the receiving end controls the second receiving interface R2 to return a registration failure response to the third receiving interface R0.
  • the second receiving interface R2 Since the second receiving interface R2 has first received the registration request from the first receiving interface R1, and has given the first receiving interface R1 the right to process the second retransmission message, it means that the first receiving interface R1 has completed the registration.
  • the second receiving interface R2 itself does not receive the retransmission message of the first message, and receives the third receiving interface R0 for the same message, that is, when the registration request for the first retransmission message is received, the second receiving interface The interface R2 returns a registration failure response to the third receiving interface R0.
  • the registration failure response indicates that the second receiving interface R2 has not received the duplicate message of the first message, and the first receiving interface R1 has completed the registration of the second retransmission message, and the third receiving interface R0 has not obtained the The processing authority of the first retransmission packet.
  • the receiving end determines that when the third receiving interface R0 receives the registration request from the first receiving interface R1, controls the third receiving interface R0 to return a registration failure response to the first receiving interface R1.
  • the third receiving interface R0 receives the first retransmission message, when the third receiving interface R0 receives the second retransmission message of the first receiving interface R1 for the first message
  • the third receiving interface R0 may be controlled to return a registration failure response to the first receiving interface R1.
  • step 137 can be understood as following the processing mechanism 1) for preventing multiple retransmitted packets from colliding.
  • the third receiving interface R0 has received the first retransmission message, and the third receiving interface R0 has also received the registration request of the first receiving interface R1 whose number is greater than that of the third receiving interface R0 (0 in the number R0 Less than 1) in the number R1, according to the description in processing mechanism 1), the third receiving interface R0 needs to reject the registration request of the first receiving interface R1, so the registration failure response is returned to the first receiving interface R1.
  • the receiving end determines that after the first receiving interface R1 receives the first message, the first receiving interface R1 also receives the first registration request sent by the second receiving interface R2, and the first registration request is used to request registration of the second receiving interface R2.
  • the receiving end controls the first receiving interface R2 Deny the first registration request.
  • the receiving end determines that the first receiving interface R1 needs to discard the second retransmission message after receiving registration responses from all other receiving interfaces of the receiving end, and records the first message on the first receiving interface R1
  • the text has been registered, and the first receiving interface R1 is controlled to return the registration response received by the first receiving interface R1 and the registration response received from the second receiving interface R2 to the third receiving interface R0, that is, the effective registration response in FIG. 13 .
  • the first receiving interface R1 has received responses from all other receiving interfaces R0 and R2 at the receiving end, including the registration failure response returned by R0 and the successful registration that the second receiving interface returned by R2 has not received the retransmission message response.
  • the first receiving interface R1 will determine that the second retransmission message received locally cannot be processed because the same message already exists Already registered on other receiving interfaces.
  • the first receiving interface R1 also needs to locally record the registration state that the second retransmission message has been registered.
  • the first receiving interface R1 follows 2) in the above-mentioned mechanism for avoiding multiple repeated message collisions, that is, the first receiving interface R1 receives the second retransmission message (the retransmission message of the first message) text), if the number of the first receiving interface R1 is not the smallest number among the numbers of all the receiving interfaces of the receiving end, the receiving end controls the first receiving interface R1 to collect all other receiving interfaces (R0 and R2) other than the first receiving interface. Multiple registration responses for one message.
  • the first receiving interface R1 sends the first registration response among multiple registration responses to the receiving interface R0 whose number is smaller than the number of the first receiving interface, and the first registration response is used to indicate that the second receiving interface R2 has not received a re-registration response.
  • the message is transmitted and the duplicate message registration of the first message of the first receiving interface R1 is successful.
  • the receiving end determines that after receiving the registration responses from the first receiving interface R1 and the second receiving interface R2, the third receiving interface R0 processes the received first retransmission message and returns it to the first sending interface R1 process result.
  • the third receiving interface R0 After the third receiving interface R0 with the smallest number collects the registration responses of all other receiving interfaces, the third receiving interface R0 can know that the second receiving interface R2 has not received the retransmitted message and has sent the retransmitted message to The processing right of is given to the first receiving interface R1. However, the third receiving interface R0 has received the registration failure message returned by the first receiving interface R1 from the first receiving interface R1. Therefore, the third receiving interface R0 can know that the remaining receiving interfaces (R1 and R2) have not processed the first registration failure message. Permission to retransmit packets for packets. Furthermore, the third receiving interface R0 obtains the processing authority for the received first retransmission message, and returns the processing result to the first sending interface R1 that sent the first retransmission message.
  • the sending end controls the first sending interface R1 to send a resource release message to the third receiving interface R0.
  • the first sending interface R1 determines that a retransmission message of the first message has been successfully processed, the first sending interface R1 returns a resource release message to the third receiving interface R0, so that the third receiving interface R0 deletes the local A context associated with retransmission packets.
  • the receiving end controls the third receiving interface R0 to instruct the first receiving interface R1 and the second receiving interface R2 to release the registration state for the first packet.
  • the first receiving interface R1 Since the first receiving interface R1 records the registration responses of the second receiving interface R2 and the third receiving interface R0, when the first receiving interface R1 receives a message of releasing the registration status, the first receiving interface R1 can delete the locally stored registration responses.
  • the second receiving interface R2 also records the registration status of the successful registration of the first receiving interface R1, and the registration status of the failure of the third receiving interface R0 registration. When the second receiving interface R2 receives the message of releasing the registration status, the second receiving interface R2 may delete the local registration state.
  • This application can follow the processing mechanism of multiple repeated messages,
  • This mechanism is a message registration mechanism, such as the mechanisms 1), 2) and 3) above.
  • other processing mechanisms can also be used to avoid repeated processing of the same message on multiple receiving interfaces.
  • the electronic device includes hardware and/or software modules corresponding to each function.
  • the electronic device may be the sending end or the receiving end in this application.
  • Combining the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software drives hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions in combination with the embodiments for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.
  • the functional modules of the electronic device may be divided according to the above method example.
  • each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above integrated modules may be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic, and is only a logical function division, and there may be other division methods in actual implementation.
  • FIG. 14 shows a possible composition diagram of an electronic device 140, and the electronic device 140 may be a sending end or a receiving end in this application.
  • the electronic device 140 may include: a processing unit 1401 , a sending unit 1402 and a receiving unit 1403 . It can be understood that the sending end has the function of sending and receiving at the same time, and the receiving end also has the function of sending and receiving at the same time.
  • the processing unit 1401 can be used to support the electronic device 1100 to execute the above steps 801, 102, 504, 108, 112, 113, 114, 115, 116, 124, 125, 126, 128, 133, 134, 135, 136 , 137, 138, 139, 141, etc., and/or other processes for the techniques described herein.
  • the sending unit 1402 may be used to support the electronic device 1100 to execute the above steps 802, 901, 101, 103, 104, 105, 106, 107, 109, 111, 117, 118, 121, 122, 127, 129, 131, 132, 140 etc., and/or other processes for the techniques described herein.
  • the receiving unit 1403 may be used to support the electronic device 1100 to execute the above steps 902, 123, etc., and/or other processes for the technology described herein.
  • sending unit 1402 is equivalent to multiple sending interfaces in this application
  • receiving unit 1403 is equivalent to multiple receiving interfaces in this application.
  • the electronic device 140 provided in this embodiment is used to implement the above message transmission method, so the same effect as the above implementation method can be achieved.
  • the electronic device 140 may include a processing module, a storage module and a communication module.
  • the processing module may be used to control and manage the actions of the electronic device 140 , for example, may be used to support the electronic device 140 to perform the steps performed by the above-mentioned determination unit 1401 .
  • the storage module can be used to support the electronic device 140 to store program codes, data, and the like.
  • the communication module may be used to support communication between the electronic device 140 and other devices, such as communication with a receiving end or a sending end. In this application, the communication module is configured to perform the steps performed by the sending unit 1402 and the receiving unit 1403 above.
  • the processing module may be a processor or a controller. It can implement or execute the various illustrative logical blocks, modules and circuits described in connection with the present disclosure.
  • the processor can also be a combination of computing functions, for example, a combination of one or more microprocessors, a combination of digital signal processing (digital signal processing, DSP) and a microprocessor, and the like.
  • the storage module may be a memory.
  • the communication module may be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, and a Wi-Fi chip.
  • the processing module is a processor
  • the storage module is a memory
  • the communication module is a plurality of sending interfaces and a plurality of receiving interfaces
  • the communication between the sending interfaces and the communication mode between the receiving interfaces may depend on the receiving interfaces and The number of sending interfaces and the layout structure of the chip.
  • the communication between the sending interfaces can be through the system bus to complete the data exchange between the interfaces, and the receiving interface can also be At the same time, the data communication between the interfaces is completed through the system bus.
  • the electronic device involved in this embodiment may be a communication device 150 having the structure shown in FIG. 15 , and the receiving interface includes receiving interface 1, receiving interface 2, and receiving interface 3 in FIG. 15 .
  • the data intercommunication between the sending interface and the receiving interface can be completed through physical direct connection.
  • the electronic device involved in this embodiment may be a communication device 160 having the structure shown in FIG. 16 , and the receiving interface includes receiving interface 1, receiving interface 2, and receiving interface 3 in FIG. 16 .
  • the embodiment of the present application also provides an electronic device, including one or more processors, one or more memories, multiple sending interfaces and multiple receiving interfaces.
  • the one or more memories are coupled with one or more processors, the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device performs The above related method steps implement the message transmission method in the above embodiment.
  • the embodiment of the present application also provides a computer storage medium, the computer storage medium stores computer instructions, and when the computer instructions are run on the electronic device, the electronic device executes the above-mentioned related method steps to realize the message in the above-mentioned embodiment transfer method.
  • the embodiment of the present application also provides a computer program product, which, when running on a computer, causes the computer to execute the above related steps, so as to implement the message transmission method performed by the electronic device in the above embodiment.
  • an embodiment of the present application also provides a device, which may specifically be a chip, a component or a module, and the device may include a connected processor and a memory; wherein the memory is used to store computer-executable instructions, and when the device is running, The processor can execute the computer-executed instructions stored in the memory, so that the chip executes the message transmission method executed by the electronic device in the above method embodiments.
  • the electronic device, computer storage medium, computer program product or chip provided in this embodiment is all used to execute the corresponding method provided above, therefore, the beneficial effects it can achieve can refer to the corresponding method provided above The beneficial effects in the method will not be repeated here.
  • Another embodiment of the present application provides a system, which may include the above-mentioned sending end and receiving end, and may be used to implement the above-mentioned message transmission method.
  • the disclosed devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be Incorporation or may be integrated into another device, or some features may be omitted, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may be one physical unit or multiple physical units, that is, it may be located in one place, or may be distributed to multiple different places . Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a readable storage medium.
  • the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium Among them, several instructions are included to make a device (which may be a single-chip microcomputer, a chip, etc.) or a processor (processor) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: various media that can store program codes such as U disk, mobile hard disk, read only memory (ROM), random access memory (random access memory, RAM), magnetic disk or optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

La présente demande divulgue un procédé et un dispositif de transmission de paquets, qui se rapportent au domaine technique des puces et peuvent améliorer les exigences de bande passante de communication entre un terminal d'envoi et un terminal de réception, ainsi que l'efficacité de transmission, au moyen d'une collaboration multi-interface du terminal d'envoi et d'une collaboration multi-interface du terminal de réception. Le procédé comprend les étapes suivantes : lors de la détermination du fait que le premier paquet doit être envoyé, le terminal d'envoi détermine une première interface d'envoi pour envoyer le premier message selon les charges des multiples interfaces d'envoi du terminal d'envoi, et détermine une première interface de réception pour recevoir le premier paquet selon les informations d'encombrement des multiples interfaces de réception du terminal de réception; et le terminal d'envoi envoie le premier paquet par l'intermédiaire de la première interface d'envoi, le premier paquet comprenant l'identifiant de la première interface d'envoi et l'identifiant de la première interface de réception. Des modes de réalisation de la présente demande sont utilisés pour une collaboration multi-interface entre un côté d'envoi et un côté de réception.
PCT/CN2021/142211 2021-12-28 2021-12-28 Procédé et dispositif de transmission de paquets WO2023122995A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202180099252.2A CN117461289A (zh) 2021-12-28 2021-12-28 一种报文传输方法和装置
PCT/CN2021/142211 WO2023122995A1 (fr) 2021-12-28 2021-12-28 Procédé et dispositif de transmission de paquets

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/142211 WO2023122995A1 (fr) 2021-12-28 2021-12-28 Procédé et dispositif de transmission de paquets

Publications (1)

Publication Number Publication Date
WO2023122995A1 true WO2023122995A1 (fr) 2023-07-06

Family

ID=86996957

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/142211 WO2023122995A1 (fr) 2021-12-28 2021-12-28 Procédé et dispositif de transmission de paquets

Country Status (2)

Country Link
CN (1) CN117461289A (fr)
WO (1) WO2023122995A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103957155A (zh) * 2014-05-06 2014-07-30 华为技术有限公司 报文传输方法、装置及互联接口
CN104506434A (zh) * 2014-12-29 2015-04-08 浪潮(北京)电子信息产业有限公司 一种快速路径应答方法及系统
WO2018108093A1 (fr) * 2016-12-13 2018-06-21 中国移动通信有限公司研究院 Procédé et appareil de contrôle de congestion, et support de stockage informatique
CN109981471A (zh) * 2017-12-27 2019-07-05 华为技术有限公司 一种缓解拥塞的方法、设备和系统
WO2021114793A1 (fr) * 2019-12-09 2021-06-17 华为技术有限公司 Procédé de transfert de données, procédé de mise en mémoire tampon de données, dispositif et appareil associé

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103957155A (zh) * 2014-05-06 2014-07-30 华为技术有限公司 报文传输方法、装置及互联接口
CN104506434A (zh) * 2014-12-29 2015-04-08 浪潮(北京)电子信息产业有限公司 一种快速路径应答方法及系统
WO2018108093A1 (fr) * 2016-12-13 2018-06-21 中国移动通信有限公司研究院 Procédé et appareil de contrôle de congestion, et support de stockage informatique
CN109981471A (zh) * 2017-12-27 2019-07-05 华为技术有限公司 一种缓解拥塞的方法、设备和系统
WO2021114793A1 (fr) * 2019-12-09 2021-06-17 华为技术有限公司 Procédé de transfert de données, procédé de mise en mémoire tampon de données, dispositif et appareil associé

Also Published As

Publication number Publication date
CN117461289A (zh) 2024-01-26

Similar Documents

Publication Publication Date Title
US10868767B2 (en) Data transmission method and apparatus in optoelectronic hybrid network
US20220311544A1 (en) System and method for facilitating efficient packet forwarding in a network interface controller (nic)
US9338019B2 (en) Scalable transport method for multicast replication
US10148581B2 (en) End-to-end enhanced reliable datagram transport
US9479587B2 (en) Scalable object storage using multicast transport
US8190960B1 (en) Guaranteed inter-process communication
US9641432B2 (en) Medical device communication method
US7817634B2 (en) Network with a constrained usage model supporting remote direct memory access
US7782905B2 (en) Apparatus and method for stateless CRC calculation
US5931915A (en) Method for processing early arrival messages within a multinode asynchronous data communications system
US6034962A (en) Communication method with attaching identifiers to receive request and transmit request
US20070208820A1 (en) Apparatus and method for out-of-order placement and in-order completion reporting of remote direct memory access operations
JP5353278B2 (ja) 通信装置
US20120023304A1 (en) Flow control for reliable message passing
JP2013511884A (ja) 動的接続された移送サービス
EP3739784A1 (fr) Procédé d'envoi de paquets de données et dispositif associé
WO2017028399A1 (fr) Procédé et système de transmission de données de communication
WO2021103016A1 (fr) Procédé de transmission de paquets, dispositif de communication et système de communication
WO2012083654A1 (fr) Procédé et système distribué pour traiter des fragments d'un paquet de protocole internet
WO2023122995A1 (fr) Procédé et dispositif de transmission de paquets
WO2023109891A1 (fr) Procédé, appareil et système de transmission de multidiffusion
US8150996B2 (en) Method and apparatus for handling flow control for a data transfer
JPH11328134A (ja) 計算機間データ送受信方法
JPH11249978A (ja) データ転送方法および装置
WO2024103891A1 (fr) Procédé et appareil de traitement de données

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21969373

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202180099252.2

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE