CN117378186A

CN117378186A - Controller for use in a network node and method for use in a network node

Info

Publication number: CN117378186A
Application number: CN202280037422.9A
Authority: CN
Inventors: 塔尔·米兹拉希; 鲁文·科恩; 本-沙哈尔·贝尔彻
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2022-04-19
Filing date: 2022-04-19
Publication date: 2024-01-09
Also published as: WO2023202759A1

Abstract

A controller for use in a network node, the controller for determining delay asymmetry in a network path from the network node to a destination node via zero or more intermediate nodes by generating a probe message, wherein the probe message comprises a first delay field. The controller is further configured to send the probe message to the destination node and cause the zero or more intermediate nodes to adapt the first delay field according to a delay experienced on a route to the destination node. The controller is further configured to receive the probe message from the destination node and cause the zero or more intermediate nodes to adapt the first delay field according to delays experienced on routes from the destination node and determine whether there is delay asymmetry based on the first delay field. Determining delay asymmetry helps to provide proper control of the network path.

Description

Controller for use in a network node and method for use in a network node

Technical Field

The present invention relates generally to the field of data networks, and more particularly to a controller for use in a network node and a method for measuring network delay asymmetry using segmented routing for use in a network node.

Background

In the world today, data can be easily transmitted and received to any communication device via the internet. The internet provides a network that shares data with multiple communication devices at high speed. Data is typically transmitted through a transmission protocol that is responsible for establishing a connection and ensuring that all data arrives securely. In some cases, however, the transport protocol suffers from the problem of asymmetric network delays, i.e., the difference between the forward and reverse delays between the two endpoints. Network delay asymmetry may be caused by various factors including different network paths, different speeds (bandwidths) and different congestion states, which may affect the queuing delay of a router (or switch). The measurement of network delay asymmetry under different congestion conditions is very important as it greatly affects the performance of the transmission protocol. In addition, the measurement of network delay asymmetry facilitates traffic engineering, error detection and correction.

Conventionally, some attempts have been made to measure network delay asymmetry, for example using one-way delay measurements. Unidirectional delay measurement requires two endpoints, endpoint a and endpoint B, to provide a measure of network delay asymmetry from endpoint a to endpoint B and from endpoint B to endpoint a. However, unidirectional delay measurement requires clock synchronization for each endpoint, which is not desirable. In some attempts, in-band telemetry is used to measure network delay asymmetry, where each router in the network path pushes a set of measurement-related data fields into the header of the message. One of the data fields corresponds to the internal delay of the router, so at the end of the network path, the message contains a set of fields specifying the internal delay of each router. However, such attempts provide only an estimate of the one-way delay, which is also undesirable. There is another conventional method to measure the unidirectional transmission delay, for example by adding internal delays of different devices along the path. Conventional approaches use a single field in the header to which each device in the packet transport network path adds its internal delay. However, the conventional method does not explicitly disclose forward and reverse paths, and thus it is difficult to analyze that network delay is caused in the transmission of data or the reception process of data. Furthermore, the conventional method does not provide any explanation about network path control to manage transmission and reception of data according to network delays in the forward path and the reverse path. Therefore, there is a technical problem of how to make a measurement of network delay asymmetry along a physical path with proper control of the network path.

Thus, in light of the above discussion, there is a need to overcome the above-described drawbacks associated with conventional techniques for network delay measurement.

Disclosure of Invention

The invention provides a controller for use in a network node and a method for use in a network node. The invention provides a solution to the existing problem of how to measure the network delay asymmetry of a network path with proper control of the network path. It is an object of the present invention to provide a solution that at least partly overcomes the problems encountered in the prior art and to provide an improved controller and an improved method for measuring network delay asymmetry using segmented routing in a network node.

One or more of the objects of the invention are achieved by the solution provided in the attached independent claims. Advantageous implementations of the invention are further defined in the dependent claims.

In one aspect, the present invention provides a controller for use in a network node. The controller is configured to determine delay asymmetry in a network path from the network node to a destination node via zero or more intermediate nodes by: and generating a detection message, wherein the detection message comprises a first delay field. Further, the probe message is sent to the destination node by the zero or more intermediate nodes, which are caused to adapt the first delay field according to the delay experienced on the route to the destination node. Further, the probe message from the destination node is received by the zero or more intermediate nodes, which are caused to adapt the first delay field according to the delay experienced on the route from the destination node. Further, it is determined whether there is delay asymmetry based on the first delay field.

The controller generates the probe message to determine delay asymmetry in the network path. Further, the controller transmits the probe message from the network node to the destination node, which facilitates storing delays experienced by each of the zero or more intermediate nodes in the first delay field of the probe message. Further, the controller receives the probe message from the destination node to evaluate a delay experienced by each of the zero or more intermediate nodes when the probe message is returned from the destination node to the network node. Further, the controller analyzes the first delay field, which helps determine delay asymmetry during transmission and reception of the probe message. The measurement of delay asymmetry facilitates congestion control in transport protocols such as transport control protocol (transmission control protocol, TCP) or remote direct data access (remote direct memory access, RDMA).

In one implementation, the controller is configured to determine delay asymmetry in the network path by determining that the first delay field indicates a delay that exceeds a threshold.

Considering the delay after the threshold is advantageous because in some networks the delay is negligible.

In another implementation, the controller is configured to determine delay asymmetry in the network path as the value indicated by the first delay field.

Determining delay asymmetry using the value indicated by the first delay field is advantageous because each of the zero or more intermediate nodes stores the value of the delay it experiences in the first delay field during traversal of the network path by the probe message.

In another implementation, the probe message includes a second delay field, and the controller is configured to determine delay asymmetry in the network path by determining a difference between the first delay field and the second delay field.

Determining the delay asymmetry by analyzing the difference between the first delay field and the second delay field is advantageous because the first delay field represents a delay value obtained during transmission of the probe message from the destination node to the network node. Furthermore, the second delay field represents a value obtained during transmission of the probe message from the network node to the destination node.

In another implementation, the controller is configured to determine whether delay asymmetry in the network path indicates that a transmit side delay is longer or a receive side delay is longer based on whether the difference is positive or negative.

The delay asymmetry is measured by subtracting the delay on the transmit side from the delay on the receive side, so this difference helps to analyze whether the delay on the transmit side is longer or the delay on the receive side is longer. Further, whether the delay on the transmitting side is longer or the delay on the receiving side is longer is analyzed to help the transmitting side increase or decrease the transmission rate according to the delay.

In another implementation, the controller is configured to: if the difference is negative, determining that delay asymmetry in the network path indicates that the delay on the transmit side is longer; if the difference is positive, it is determined that delay asymmetry in the network path indicates that the delay on the receiving side is longer.

In such implementations, analyzing which side of the delay is longer helps to control the sending rate of the sender.

In another aspect, the present invention provides a controller for use in a network node. The controller is used for: receiving a detection message, wherein the detection message comprises a first delay field; determining whether the network node is a destination node of the detection message; if not, the following steps are performed: adding one-hop delay to the first delay field; and sending the detection message to the next network node.

If the controller determines that the probe message is on the network node and the network node is not the last node to return the probe message, the controller increases the first delay field by a one-hop delay. An increment of one-hop delay in the first delay field facilitates transmission of the probe message to another node in the network path. Further, the detection message is sent to the next network node, so that the detection message reaches the destination node.

In yet another aspect, the present invention provides a method for use in a network node for determining delay asymmetry in a network path from the network node to a destination node via zero or more intermediate nodes. The method comprises the following steps: generating a detection message, wherein the detection message comprises a first delay field; transmitting, by the zero or more intermediate nodes, the probe message to the destination node, the zero or more intermediate nodes adapting the first delay field according to the delay experienced on the route to the destination node. Further, the probe message from the destination node is received by the zero or more intermediate nodes, which are caused to adapt the first delay field according to the delay experienced on the route from the destination node. Further, it is determined whether there is delay asymmetry based on the first delay field.

The method achieves all the advantages and technical effects of the present invention for a controller for use in a network node.

In yet another aspect, the present invention provides a method for use in a network node. The method comprises the following steps: receiving a detection message, wherein the detection message comprises a first delay field; determining whether the network node is a destination node of the detection message; if not, the following steps are performed: adding one-hop delay to the first delay field; and sending the detection message to the next network node.

In yet another aspect, the present invention provides a method for use in a network for determining delay asymmetry in a network path from an originating network node to a destination node via zero or more intermediate nodes. The method comprises the following steps: the originating network node generates a probe message, wherein the probe message includes a first delay field; and transmitting the detection message to the destination node through the zero or more intermediate nodes. Further, the first delay field is adapted according to the delay experienced by each of the zero or more intermediate nodes on the route to the destination node by adding a one-hop delay to the first delay field. Further, adapting the first delay field without the destination node adding the first delay field; the probe message is sent to the originating node through the zero or more intermediate nodes. Further, by adding a one-hop delay to the first delay field, the first delay field is adapted according to the delay experienced by each of the zero or more intermediate nodes on the route to the originating node. Further, receiving the probe message from the destination node at the originating node through the zero or more intermediate nodes; determining whether there is delay asymmetry based on the first delay field.

The originating network node generates the probe message to determine delay asymmetry in the network path. Advantageously, the originating network node provides means for measuring delay asymmetry by means of the probe message. Further, the originating network node sends the probe message to the destination node, which facilitates measuring asymmetric delays. Further, a one-hop delay is added to the first delay field to arrive at the destination node for the probe packet, and the delay experienced by each of the zero or more intermediate nodes is stored in the first delay field of the probe packet. Further, the first delay field does not increase a one-hop delay when it is determined that the corresponding node is the destination node, so that the probe message is returned to the originating node. Further, the first delay field is received from the destination node, wherein a delay experienced by each of the zero or more intermediate nodes is stored in the first delay field of the probe message. Further, the first delay field is analyzed to measure delay asymmetry in the network path for controlling congestion in a transport protocol, such as transport control protocol (transmission control protocol, TCP) or remote direct data access (remote direct memory access, RDMA).

It should be noted that all devices, elements, circuits, units and means described in this application may be implemented in software or hardware elements or any type of combination thereof. All steps performed by the various entities described in this application, as well as functions to be performed by the various entities described are intended to mean that the respective entities are adapted to perform the respective steps and functions. Although in the following description of specific embodiments, specific functions or steps performed by external entities are not reflected in the description of specific detailed elements of the entity performing the specific steps or functions, it should be clear to a skilled person that these methods and functions may be implemented by corresponding hardware or software elements or any combination thereof. It will be appreciated that features of the invention are susceptible to being combined in various combinations without departing from the scope of the invention as defined by the accompanying claims.

Other aspects, advantages, features and objects of the invention will become apparent from the drawings and from the detailed description of illustrative implementations when read in connection with the following appended claims.

Drawings

The foregoing summary, as well as the following detailed description of illustrative embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the invention, there is shown in the drawings exemplary constructions of the invention. However, the invention is not limited to the specific methods and instrumentalities disclosed herein. Moreover, those skilled in the art will appreciate that the drawings are not drawn to scale. Wherever possible, like elements are designated by like numerals.

Embodiments of the invention will now be described, by way of example only, with reference to the following figures, in which:

fig. 1 is a block diagram of a controller for use in a network node according to an embodiment of the invention;

fig. 2 is a block diagram of an internal structure of a controller according to an embodiment of the present invention;

fig. 3 is a flow chart of a method for use in a network node according to an embodiment of the invention;

fig. 4 is another block diagram of a controller for use in a network node according to an embodiment of the invention;

fig. 5 is another flow chart of a method for use in a network node according to an embodiment of the invention;

FIG. 6 is a diagram depicting a structural representation of a delay asymmetry measurement procedure according to an embodiment of the invention;

FIG. 7A is a diagram depicting the format of a probe message according to an embodiment of the invention;

FIG. 7B is another diagram illustrating the format of a probe message according to an embodiment of the present invention;

fig. 8 is a flow chart of a method for use in a network for determining delay asymmetry in a network path according to an embodiment of the invention.

In the drawings, the underlined numbers are used to denote items where the underlined numbers are located or items adjacent to the underlined numbers. The non-underlined numbers relate to items identified by lines associating the non-underlined numbers with the items. When a number is not underlined and has an associated arrow, the number without the underline is used to identify the general item to which the arrow points.

Detailed Description

The following detailed description illustrates embodiments of the invention and the manner in which the embodiments may be implemented. While some modes for carrying out the invention have been disclosed, those skilled in the art will recognize that other embodiments for carrying out or practicing the invention may also exist.

Fig. 1 is a block diagram of a controller for use in a network node according to an embodiment of the invention. Referring to fig. 1, a block diagram 100 is shown that includes a network node 102, a controller 104, a probe message 106, a first delay field 108, zero or more intermediate nodes 110, such as intermediate node 110A, up to intermediate node 110N. A destination node 112 is also shown.

The network node 102 may comprise suitable logic, circuitry, interfaces, or code that may be an initial node of a network path. The controller 104 initiates data transmission from the network node 102. Examples of network node 102 may include, but are not limited to, a computer, a personal digital assistant, a portable computing device, or an electronic device.

The controller 104 may comprise suitable logic, circuitry, interfaces, hardware, software, or code that may be operable to transmit probe messages 106 from the network node 102 to the destination node 112. Examples of implementations of the controller 104 may include, but are not limited to, a central data processing device, a microprocessor, a microcontroller, a complex instruction set computing (complex instruction set computing, CISC) processor, an application-specific integrated circuit (ASIC) processor, a reduced instruction set (reduced instruction set, RISC) processor, a very long instruction word (very long instruction word, VLIW) processor, a state machine, and other processors or control circuits. In one example, the controller 104 may be a software module in the network node 102. In another example, the controller 104 may be a hardware component.

The probe message 106 is a network path measurement message for collecting data as it traverses the network node 102 to the destination node 112. The first delay field 108 is part of the probe message 106. The first delay field 108 stores data collected by the probe message as it traverses the network node 102 to the destination node 112.

Zero or more intermediate nodes 110 may comprise suitable logic, circuitry, interfaces, or code that may be operable to connect network node 102 and destination node 112 and transmit probe messages 106 without any interruption. Zero or more intermediate nodes 110 further include intermediate node 110A and subsequent intermediate nodes up to intermediate node 110N to provide a path to controller 104 to send probe messages from network node 102 to destination node 112.

The destination node 112 may comprise suitable logic, circuitry, interfaces, or code that may be operable to receive the probe message 106 and also return the probe message 106 to the network node 102.

A controller 104 is provided for use in the network node 102. The controller 104 is configured to determine delay asymmetry in a network path from the network node 102 to the destination node 112 via zero or more intermediate nodes 110. The controller 104 determines delay asymmetry by analyzing the delay experienced by each of the zero or more intermediate nodes 110. Further, the controller 104 initiates a measurement of delay asymmetry from the network node 102, covering all zero or more intermediate nodes 110.

In operation, the controller 104 is configured to generate a probe message 106, the probe message 106 including a first delay field 108. The probe messages 106 are generated by the controller 104 for measuring delay asymmetry in the network path. The probe message 106 collects data while traversing the network path. The first delay field 108 is part of the probe message 106 that stores data collected by the probe message 106. In one implementation, the probe message 106 includes a telemetry header having a first delay field 108 to retain data collected by the probe message 106.

The controller 104 is configured to send probe messages 106 to the destination node 112 through zero or more intermediate nodes 110, such that zero or more intermediate nodes 110 adapt the first delay field 108 according to the delay experienced on the route to the destination node 112. After generating the probe message 106, the controller 104 sends the probe message 106 to the destination node 112 through zero or more intermediate nodes 110. Zero or more intermediate nodes 110 provide paths to the controller 104 for sending probe messages 106 to destination nodes 112. For example, the controller 104 first forwards the probe message 106A and subsequent probe messages to zero or more intermediate nodes 110, and then the probe message 106 and subsequent probe messages arrive at the destination node 112. In one implementation, probe message 106 also includes a telemetry header. Furthermore, zero or more intermediate nodes 110 adapt the first delay field 108 when the probe message 106 traverses zero or more intermediate nodes 110 to reach the destination node 112. In one example, after the telemetry header is detected, the internal delay experienced by each of zero or more intermediate nodes 110 is also added to the first delay field 108 of the probe message 106. For example, the controller 104 sends a probe message 106 to measure asymmetric delays in the network path, and then the probe message 106 reaches the intermediate node 110A first. Further, the intermediate node 110A adapts the first delay field 108 of the probe message 106 and provides the value of the delay experienced after detecting the telemetry header to the first delay field 108. Further, probe message 106 covers all zero or more intermediate nodes 110 and reaches destination node 112.

The controller 104 is configured to receive probe messages 106 from the destination node 112 through zero or more intermediate nodes 110, such that zero or more intermediate nodes 110 adapt the first delay field 108 according to the delay experienced on the route from the destination node 112. After sending the probe message 106, the controller 104 receives the probe message 106 from the destination node 112 through zero or more intermediate nodes 110. For example, probe message 106 returns after reaching destination node 112. When returned from the destination node 112, the probe message 106 traverses zero or more intermediate nodes 110, with zero or more intermediate nodes 110 adapting the first delay field 108 to provide the delay value experienced by each of the zero or more intermediate nodes 110. For example, the probe message 106 is reversed after reaching the destination node 112 to measure the asymmetric delay in the reverse path, and then the probe message 106 reaches the intermediate node 110N first. Further, the intermediate node 110N adapts the first delay field 108 of the probe message 106 and provides the value of the experienced delay to the first delay field 108. Further, probe message 106 covers all zero or more intermediate nodes 110 and reaches network node 102.

The controller 104 is configured to determine whether there is delay asymmetry based on the first delay field 108. After receiving the probe message 106 from the destination node 112, the controller 104 measures the delay asymmetry by analyzing the first delay field 108. For example, probe message 106 traverses zero or more intermediate nodes 110 to reach destination node 112, and probe message 106 traverses zero or more intermediate nodes 110 again to return to network node 102. In this case, zero or more intermediate nodes 110 add their internal delays to the first delay field 108. Further, the controller 104 determines delay asymmetry by evaluating the value of the internal delay stored in the first delay field 108. Further, the values stored in the first delay field 108 are values of the forward path and the reverse path, which helps to control the rate of transmission and reception of any data in the network path based on the delays occurring in the forward path and the reverse path.

According to one embodiment, the controller 104 is further configured to determine delay asymmetry in the network path by determining that the first delay field 108 indicates a delay that exceeds a threshold. In other words, the delay experienced by each of zero or more intermediate nodes 110 is stored in first delay field 108, and controller 104 determines the delay when the analysis delay value has exceeded the threshold. In one implementation, if the delay is below the threshold, the controller 104 disregards the delay because the delay below the threshold is negligible. For example, if the threshold set in the first delay field 108 is 10 microseconds and each of the zero or more intermediate nodes 110 is faced with an internal delay below 10 microseconds, the controller 104 will not determine that the delay in the network path is asymmetric. Further, if each of zero or more intermediate nodes 110 is faced with an internal delay of more than 10 microseconds, controller 104 will account for delay asymmetry in the network path.

According to another embodiment, the controller 104 is configured to determine the delay asymmetry in the network path as the value indicated by the first delay field 108. The value of the internal delay experienced by each of zero or more intermediate nodes 110 is added to first delay field 108, and thus controller 104 analyzes the value indicated by first delay field 108 to determine delay asymmetry in the network path. For example, the controller 104 analyzes the first delay field 108 after receiving a probe message from the destination node 112.

According to one embodiment, probe message 106 includes a second delay field and controller 104 is configured to determine delay asymmetry in the network path by determining a difference between first delay field 108 and the second delay field. The probe message 106 returns upon reaching the destination node 112, and the destination node 112 then passes the value of the internal delay experienced by zero or more intermediate nodes 110 from the first delay field 108 to the second delay field so that the first delay field 108 is reused to store the internal delay of zero or more intermediate nodes 110 upon return from the destination node 114. In other words, the second delay field is used in probe message 106 to store the experienced internal delays of zero or more intermediate nodes in the forward path previously stored in first delay field 108. In addition, probe message 106 traverses zero or more intermediate nodes 110 during the return from destination node 112 and stores the internal delay of zero or more intermediate nodes 110 in first delay field 108. Further, the controller 104 determines delay asymmetry by analyzing the difference between the first delay field 108 and the second delay field. For example, the probe message 106 is sent by the controller 104 to reach the destination node 112, and then the probe message 106 stores the value of the internal delay experienced by zero or more intermediate nodes 110 using the first delay field 108. Further, the probe message arrives at the destination node 112, and the destination node 112 then passes all values of the internal delay from the first delay field 108 to the second delay field. Further, the probe message is returned from the destination node 112, and then the probe message 106 reuses the first delay field 108 to store the value of the internal delay experienced by zero or more intermediate nodes 110. Further, the controller 104 analyzes the difference between the first delay field 108 and the second delay field to determine delay asymmetry, which helps provide proper control of the network path.

According to one embodiment, the controller 104 is configured to determine whether the delay asymmetry in the network path indicates that the transmit side delay is longer or the receive side delay is longer based on whether the difference is positive or negative. The controller 104 analyzes the values stored in the first delay field 108 and the second delay field. In one implementation, the value stored in the first delay field 108Is the value on the transmitting side and the value stored in the second delay field is the value on the receiving side. Further, the controller 104 is configured to determine a difference between the values stored in the first delay field 108 and the second delay field to determine delay asymmetry. For example, the delay on the transmitting side is a forward delay (D _F ) The delay on the receiving side is the reverse delay (D _R ) Thus D _R And D _F Is the delay asymmetry in the network path, the longer delay is D _R Or D is _F Depending on D _R Subtracting D _F The value obtained is negative or positive. Further, the controller 104 is configured to determine whether the transmission side delay is longer or the reception side delay is longer according to a value obtained by evaluating the difference between the first delay field 108 and the second delay field. In such an embodiment, the controller 104 is to: if the difference is negative, determining that the delay asymmetry in the network path indicates that the delay on the transmitting side is longer; if the difference is positive, it is determined that the delay asymmetry in the network path indicates that the delay on the receiving side is longer. The controller 104 determines the delay asymmetry by subtracting the delay on the transmit side from the delay on the receive side. For example, if D _F Equal to 5 microseconds, D _R Equal to 3 microseconds, the difference is D _R –D _F I.e. -2 microseconds. The resulting difference is negative, indicating a longer delay on the transmit side. According to another example, if D _F Equal to 4 microseconds, D _R Equal to 7 microseconds, the difference is D _R –D _F I.e. 3 microseconds. The resulting difference is positive, indicating a longer delay on the receiving side. Advantageously, by determining whether the transmit side delay is longer or the receive side delay is longer, it helps control the rate at which any data is transmitted and received in the network path. For example, if the delay on the transmitting side is longer, the controller 104 decreases the transmission rate, and if the delay on the receiving side is longer, the controller 104 does not decrease the transmission rate.

The controller 104 generates a probe message 106 to determine delay asymmetry in the network path. Advantageously, the controller 104 provides a facility for measuring delay asymmetry by means of the detection messages 106. Further, the controller 104 transmits the probe message 106 from the network node 102 to the destination node 112, which facilitates storing the delay experienced by each of the zero or more intermediate nodes 110 in the first delay field 108 of the probe message 106. Further, the controller 104 receives the probe message 106 from the destination node 112 to evaluate the delay experienced by each of the zero or more intermediate nodes 110 when the probe message 106 is returned from the destination node 112 to the network node 102. Further, the controller 104 analyzes the first delay field 108, which helps determine delay asymmetry during transmission and reception of the probe message 106. The measurement of delay asymmetry facilitates congestion control in transport protocols such as transport control protocol (transmission control protocol, TCP) or remote direct data access (remote direct memory access, RDMA).

Fig. 2 is a block diagram of an internal structure of a controller according to an embodiment of the present invention. Fig. 2 is shown in conjunction with the elements of fig. 1. Referring to fig. 2, a block diagram 200 of the internal structure of the network node 102 is shown. In one implementation, network node 102 also includes a network interface card 202, a memory 204, and a controller 104.

The network interface card 202 may also be referred to as a network interface and may comprise suitable logic, circuitry, and/or interfaces that may be operable to communicate with the network node 102, the controller 104, zero or more intermediate nodes 110, the destination node 112, and the like in fig. 1.

The memory 204 may comprise suitable logic, circuitry, and/or interfaces that may be operable to store information that may be desired to be transmitted by the controller 104. Examples of implementations of Memory 204 may include, but are not limited to, electrically erasable programmable Read-Only Memory (EEPROM), dynamic Random-Access Memory (DRAM), random-Access Memory (Random Access Memory, RAM), read-Only Memory (ROM), hard Disk Drive (HDD), flash Memory, secure Digital (SD) card, solid State Drive (SSD), and/or CPU cache Memory.

Network node 102 is coupled to zero or more intermediate nodes 110 and destination nodes 112 through network interface card 202. Further, the controller 104 generates a probe message 106 that is sent from the network node 102 to the destination node 112. In one implementation, the probe message 106 is stored in the memory 204 after it is received from the destination node 112. The operation of transmitting the probe message 106 is controlled by the controller 104.

Fig. 3 is a flow chart of a method for use in a network node according to an embodiment of the invention. Fig. 3 is described in connection with the elements of fig. 1 and 2. Referring to fig. 3, a flow chart of a method 300 including steps 302 through 308 is shown. The controller 104 is configured to perform the method 300.

A method 300 for use in a network node 102 is provided for determining delay asymmetry in a network path from the network node 102 to a destination node 112 via zero or more intermediate nodes 110. The method 300 provides an improved way of measuring delay asymmetry in a network path. Method 300 discloses determining delay asymmetry by analyzing the delay experienced by each of zero or more intermediate nodes 110. Further, measurement of delay asymmetry is initiated at network node 102 and covers all zero or more intermediate nodes 110. In other words, the method 300 discloses providing delay information for each direction, and the method 300 uses a single field with less overhead.

In step 302, the method 300 includes generating a probe message 106, the probe message 106 including a first delay field 108. The probe message 106 is generated for measuring delay asymmetry in the network path. The probe message 106 collects data while traversing the network path. The first delay field 108 is part of a probe message that stores data collected by the probe message 106. In one implementation, the probe message 106 includes a telemetry header having a first delay field 108 to retain data collected by the probe message 106.

In step 304, the method 300 includes transmitting the probe message 106 to the destination node 112 through zero or more intermediate nodes 110, causing zero or more intermediate nodes 110 to adapt the first delay field 108 according to the delay experienced on the route to the destination node 112. After the probe message 106 is generated, the probe message 106 is sent to the destination node 112 through zero or more intermediate nodes 110. Zero or more intermediate nodes 110 provide paths for sending probe messages 106 to destination node 112. For example, the probe message 106A and subsequent probe messages are first forwarded to zero or more intermediate nodes 110, and then the probe message 106 and subsequent probe messages arrive at the destination node 112. In one implementation, probe message 106 also includes a telemetry header. Furthermore, zero or more intermediate nodes 110 adapt the first delay field 108 when the probe message 106 traverses zero or more intermediate nodes 110 to reach the destination node 112. In one example, after the telemetry header is detected, the internal delay experienced by each of zero or more intermediate nodes 110 is also added to the first delay field 108 of the probe message 106. For example, the probe message 106 is sent to measure asymmetric delays in the network path, and then the probe message 106 reaches the intermediate node 110A first. Further, the intermediate node 110A adapts the first delay field 108 of the probe message 106 and provides the value of the delay experienced after detecting the telemetry header to the first delay field 108. Further, probe message 106 covers all zero or more intermediate nodes 110 and reaches destination node 112.

In step 306, the method 300 includes receiving probe messages 106 from the destination node 112 through zero or more intermediate nodes 110, causing zero or more intermediate nodes 110 to adapt the first delay field 108 according to the delay experienced on the route from the destination node 112. Further, probe messages 106 from destination node 112 are received through zero or more intermediate nodes 110. For example, probe message 106 returns after reaching destination node 112. Further, the probe message 106 traverses zero or more intermediate nodes 110, with zero or more intermediate nodes 110 adapting the first delay field 108 to provide a delay value experienced by each of the zero or more intermediate nodes 110. For example, the probe message 106 is reversed after reaching the destination node 112 to measure the asymmetric delay in the reverse path, and then the probe message 106 reaches the intermediate node 110N first. Further, the intermediate node 110N adapts the first delay field 108 of the probe message 106 and provides the value of the experienced delay to the first delay field 108. Further, probe message 106 covers all zero or more intermediate nodes 110 and reaches network node 102.

In step 308, the method 300 includes: it is determined whether there is delay asymmetry based on the first delay field 108. After receiving the probe message 106 from the destination node 112, the delay asymmetry is measured by analyzing the first delay field 108. For example, probe message 106 traverses zero or more intermediate nodes 110 to reach destination node 112, and probe message 106 traverses zero or more intermediate nodes 110 again to return to network node 102. In this case, zero or more intermediate nodes 110 add their internal delays to the first delay field 108. Further, the delay asymmetry is determined by evaluating the value of the internal delay stored in the first delay field 108. Further, the values stored in the first delay field 108 are values of the forward path and the reverse path, which helps to control the rate of transmission and reception of any data in the network path based on the delays occurring in the forward path and the reverse path.

According to one embodiment, the method 300 further comprises: delay asymmetry in the network path is determined by determining that the first delay field 108 indicates a delay that exceeds a threshold. In other words, the delay experienced by each of zero or more intermediate nodes 110 is stored in first delay field 108 and the delay is determined when the analytical delay value has exceeded a threshold. In one implementation, if the delay is below the threshold, the delay is not considered because the delay below the threshold is negligible. For example, if the threshold set in the first delay field 108 is 10 microseconds and each of the zero or more intermediate nodes 110 is faced with an internal delay below 10 microseconds, then no delay asymmetry in the network path will be determined. Further, if each of zero or more intermediate nodes 110 is faced with an internal delay of more than 10 microseconds, the controller will take into account delay asymmetry in the network path controller.

According to another embodiment, the method 300 further comprises determining delay asymmetry in the network path as the value indicated by the first delay field 108. The value of the internal delay experienced by each of zero or more intermediate nodes 110 is added to first delay field 108, thus analyzing the value indicated by first delay field 108 to determine delay asymmetry in the network path. For example, after receiving the probe message from the destination node 112, the first delay field 108 is analyzed.

According to one embodiment, the method 300 further comprises: determining delay asymmetry in a network path based on whether the difference is positive or negativeIndicating whether the transmission side delay is longer or the reception side delay is longer. The values stored in the first delay field 108 and the second delay field are analyzed. In one implementation, the value stored in the first delay field 108 is the value on the transmitting side and the value stored in the second delay field is the value on the receiving side. Delay asymmetry is determined by the difference between the values stored in the first delay field 108 and the second delay field. For example, the delay on the transmitting side is a forward delay (D _F ) The delay on the receiving side is the reverse delay (D _R ) Thus D _R And D _F Is the delay asymmetry in the network path, the longer delay is D _R Or D is _F Depending on D _R Subtracting D _F The value obtained is negative or positive. Further, the method 300 is used for determining whether the transmission side delay is longer or the reception side delay is longer according to a value obtained by evaluating the difference between the first delay field 108 and the second delay field.

In this embodiment, the method 300 further comprises: if the difference is negative, determining that the delay asymmetry in the network path indicates that the delay on the transmitting side is longer; if the difference is positive, it is determined that the delay asymmetry in the network path indicates that the delay on the receiving side is longer. The delay asymmetry is determined by subtracting the delay on the transmit side from the delay on the receive side. For example, if D _F Equal to 5 microseconds, D _R Equal to 3 microseconds, the difference is D _R –D _F I.e. -2 microseconds. The resulting difference is negative, indicating a longer delay on the transmit side. According to another example, if D _F Equal to 4 microseconds, D _R Equal to 7 microseconds, the difference is DR-DF, i.e., 3 microseconds. The resulting difference is positive, indicating a longer delay on the receiving side.

The method 300 determines delay asymmetry in the network path and provides information on the transmitting side and the receiving side. Further, the method 300 measures delay asymmetry, which is useful for congestion control in transport protocols such as transport control protocol (transmission control protocol, TCP) or remote direct data access (remote direct memory access, RDMA), because TCP and RDMA use delay asymmetry in rate limiting decisions. For example, when the delay on the transmitting side is asymmetrically long, the transmission protocol instructs the transmitting side to decrease the transmission rate; when the delay on the receiving side is asymmetrically long, the transmission protocol instructs the sender to keep the sending rate unchanged. The method 300 discloses a method for generating a probe message 106 to determine delay asymmetry in a network path. Further, the method 300 discloses that in relation to transmitting the probe message 106 from the network node 102 to the destination node 112, this facilitates storing the delay experienced by each of the zero or more intermediate nodes 110 in the first delay field 108 of the probe message 106. Further, the method 300 discloses that with respect to receiving the probe message 106 from the destination node 112, this facilitates storing the delay experienced by each of the zero or more intermediate nodes 110 as the probe message 106 is returned from the destination node 112 to the network node 102. Further, the method 300 analyzes the first delay field 108 to determine delay asymmetry in the network path, which facilitates receiving information about delay asymmetry in the transmitting side and the receiving side.

Steps 302 through 308 are merely illustrative, and other alternatives may be provided, including adding one or more steps, deleting one or more steps, or providing one or more steps in a different order without departing from the scope of the claims herein.

Fig. 4 is a block diagram of a controller for use in a network node according to an embodiment of the invention. Fig. 4 is described in connection with the elements of fig. 1. Referring to fig. 4, a block diagram 400 is shown including a network node 102, a controller 104, a probe message 106, a first delay field 108, and a next network node 402. The controller 104 is for use in a network node.

The next network node 402 is one of the nodes located before the destination node 112. In one example, the next network node 402 corresponds to a node of zero or more intermediate nodes 110.

In operation, the controller 104 is configured to receive a probe message 106, the probe message 106 including a first delay field 108. The probe message 106 and the first delay field 108 are received by the controller 104 through zero or more intermediate nodes 110. In one implementation, the controller 104 is configured to receive a probe message 106 from another controller. In one example, probe message 106 includes a telemetry header that is detected by zero or more intermediate nodes 110. In addition, probe message 106 traverses zero or more intermediate nodes 110, with zero or more intermediate nodes 110 adapting first delay field 108 to add the delay experienced by each of zero or more intermediate nodes 110 to first delay field 108 after the telemetry header is detected.

The controller 104 is also configured to determine whether the network node 102 is the destination node 112 of the probe message 106. In other words, the probe message 106 traverses zero or more intermediate nodes 110 to reach the controller 104. In addition, as probe message 106 traverses the network path, controller 104 determines whether network node 102 is the destination node 112 of probe message 106. The controller 104 is also configured to: if the network node 102 is not the destination node 112 of the probe message 106, the first delay field 108 is increased by one-hop delay. In other words, if the controller 104 determines that the probe message 106 is on the network node 102, and the network node 102 is not the last node to return the probe message 106, the controller 104 adds a one-hop delay to the first delay field 108, for example, adds an internal delay faced by another intermediate node, as the sequence further shown and described in fig. 6.

According to one embodiment, the controller 104 is further configured to: if it is determined that the network node 102 is the destination node 112, the first delay field 108 is adapted without adding the first delay field 108. In other words, if the controller 104 determines that the probe message 106 is on the network node 102 and the network node 102 is the last node to return the probe message 106, the controller 104 adapts the first delay field 108 without increasing the one-hop delay. For example, if probe message 106 is on destination node 112, controller 104 does not add a hop in first delay field 108, as further shown and described in fig. 6.

The controller 104 is further configured to send the probe message 106 to the next network node 402. After adding the first delay field 108 to the one-hop delay, the probe message 106 is sent to the next network node 402. The controller 104 continues to check the location of the probe message 106 and continues to increment the first delay field 108 until the probe message 106 reaches the next network node 402. For example, as probe message 106 traverses the network path, probe message 106 first traverses node a and adds the internal delay faced by node a to first delay field 108. Further, if the controller 104 determines that node a is not the destination node 112, the controller 104 increases the first delay field 108 by one-hop delay and sends the probe message 106 to node B. Further, the controller 104 determines that node B is also not the destination node 112, and the controller again increases the first delay field by one-hop delay and sends the probe message 106 to node C. Further, the controller 104 determines that node C is the destination node 112, and the controller 104 adapts the first delay field 108 without increasing the one-hop delay.

According to one embodiment, the controller 104 is further configured to adapt the first delay field 108 by reversing the sign of the first delay field 108. First, the controller 104 receives the probe message 106 with the first delay field 108 filled with the value of the delay experienced by zero or more intermediate nodes 110. Thereafter, the controller 104 adapts the first delay field 108 by reversing the sign of the value added in the first delay field 108. For example, when the controller 104 receives the probe message 106, the first delay field 108 has a forward delay D _F The controller 104 therefore controls the operation of D by comparing D _F Set as-D _F The first delay field 108 is adapted to also use the first delay field 108 at the receiving side with the addition of zero or more delay values experienced by the intermediate node 110. It should be noted that zero or more intermediate nodes 110 do not change their behavior according to the direction of the probe messages 106.

According to one embodiment, the probe message 106 includes a second delay field, and the controller 104 is further configured to adapt the first delay field 108 by copying the first delay field 108 to the second delay field and resetting the first delay field 108. First, probe message 106 arrives at controller 104 with first delay field 108 filled with zero or more delays experienced by intermediate node 110. Thereafter, the controller 104 copies the delay to the second delay field. Further, upon return of the probe message 106, the controller 104 addsZero or more intermediate nodes 110 experience a delay to reuse the first delay field 108. For example, when the probe message 106 arrives at the controller 104, the first delay field 108 has a forward delay D _F The controller 104 therefore controls the operation of D by comparing D _F Copy to the second delay field to adapt the first delay field 108 to also use the first delay field 108 at the receiving side. Further, after copying the first delay field 108 to the second delay field, the controller 104 is configured to reset the first delay field 108 to fit the first delay field 108 and add zero or more delays experienced by the intermediate node 110 when the probe message 106 returns. Further, when the controller 104 resets the first delay field 108, the added delay value of zero or more intermediate nodes 110 is deleted and the first delay field 108 is reset to add a new value when the probe message 106 traverses the receiving side.

According to one embodiment, the probe message 106 includes a function field, and the controller 104 is configured to adapt the first delay field 108 by performing the function indicated by the function field. The function field is located in the probe message 106, so the controller 104 performs the function provided by the function field to adapt the first delay field 108. The function field has data that detects the address that the message 106 needs to transmit. In one implementation, the function field is located in a segment routing header. In another implementation, the function field has a segment identifier for each network node 102, zero or more intermediate nodes 110, and the next network node 402.

According to one embodiment, the probe message 106 includes a sixth version of an internet protocol based segment routing (Segment Routing Internet Protocol Version, srv 6) header that includes a first delay field 108. In one implementation, the SRv header is used to instruct the destination node 112 to return the probe message 106 without updating its own internal delay in the first delay field 108. According to one method, the SRv header inverts the sign of the first delay so that the first delay field 108 is also used at the receiving side. For example, when the probe message 106 arrives at the destination node 112, the first delay field 108 has a forward delay D _F Thus SRv header command will be forward delayed by D _F Change to-D _F To use the first delay field 108 at the receiving side, also with the addition of zero or more delay values experienced by the intermediate node 110. By inverting the forward delay D _F The SRv header uses one field in both directions, which is advantageous for reducing overall overhead costs. According to another approach, SRv header commands copy the forward delay from the first delay field 108 to the second delay field. Further, the first delay field 108 is reset for reuse in adding the reverse delay. For example, when the probe message 106 arrives at the destination node 112, the first delay field 108 has a forward delay D _F Thus SRv header order node 112 will D _F Copying to the second delay field to also use the first delay field 108 at the receiving side helps to provide more information about the delay asymmetry of the transmitting side and the receiving side. In these embodiments, the SRv header also includes a function field. The function field is part of the SRv header. The function field is used to store a command to perform a delay asymmetry measurement. For example, the probe message 106 is required to traverse the network path, and then the probe message 106 reaches zero or more intermediate nodes 110 mentioned in the function field. In these embodiments, the SRv header further includes a list of segment identifier (segment identifier, SID) entries for the originating network node, the destination network node, and zero or more intermediate network nodes, wherein the list entry for the destination node 112 includes a function field. The list of SID entries of the SRv header provides a path for sending and receiving probe messages 106 by storing the address of the originating network node (e.g., network node 102 of fig. 1), the destination network node (e.g., destination node 112 of fig. 1), and zero or more intermediate network nodes (e.g., zero or more intermediate nodes 110 of fig. 1). For example, the probe message 106 traverses the network path, and thus the list of SID entries provides a list of sequences that the probe message 106 needs to transmit. The SID entry list also includes a function subfield for storing an identification for each node.

The probe message 106 is received by the controller 104 and the probe message 106 has a first delay field 108 that facilitates storing delay values during transmission and reception of the probe message 106. In addition, the first delay field 108 assists the controller 104 in analyzing delay asymmetry. If the controller 104 does not determine that the network node 102 is the destination node 112, the controller 104 adds the first delay field 108 by one hop, which facilitates sending the probe message 106 to another node in the network path. Further, the probe message 106 is sent to the next network node so that the probe message 106 reaches the next network node 402.

Fig. 5 is another flow chart of a method used in a network node according to an embodiment of the invention. Fig. 5 is described in connection with the elements of fig. 1 and 4. Referring to fig. 5, a flow chart of a method 500 including steps 502 through 506 is shown. The controller 104 is configured to perform the method 500.

A method 500 for use in a network node 102 is provided. The method 500 is used to provide delay information for each direction and the method 500 uses a single field with less overhead. Furthermore, the method 500 provides an improved way of measuring delay asymmetry in a network path.

In step 502, the method 500 includes receiving a probe message 106, the probe message 106 including a first delay field 108. The probe message 106 and the first delay field 108 are received by the controller 104 through zero or more intermediate nodes 110. In one implementation, probe message 106 includes telemetry headers detected by zero or more intermediate nodes 110. In addition, probe message 106 traverses zero or more intermediate nodes 110, with zero or more intermediate nodes 110 adapting first delay field 108 to add the delay experienced by each of zero or more intermediate nodes 110 to first delay field 108 after the telemetry header is detected.

In step 504, the method 500 further comprises: it is determined whether the network node 102 is the destination node 112 of the probe message 106. In other words, probe message 106 traverses zero or more intermediate nodes 110 to reach destination node 112. In addition, as probe message 106 traverses the network path, it is determined whether network node 102 is the destination node 112 of probe message 106.

In step 504A, the method 500 includes: if it is determined that the network node 102 is not the destination node 112 of the probe message 106, the first delay field 108 is increased by a one-hop delay. If it is determined that the probe message 106 is on the network node 102 and the network node 102 is not the last node that the probe message 106 needs to return while maintaining a check of the location of the probe message 106, the first delay field 108 is increased by a delay to add the internal delay faced by another intermediate node.

According to one embodiment, the method 500 further comprises: if it is determined that the network node 102 is the destination node 112, the first delay field 108 is adapted without adding the first delay field 108. In other words, if the probe message 106 is on the network node 102 and the network node 102 is the last node to return the probe message 106, the first delay field 108 is adapted without adding one-hop delay. For example, probe message 106 is on destination node 112, then first delay field 108 does not add a one-hop delay, as further shown and described in fig. 6.

In step 506, the method 500 further comprises sending the probe message 106 to the next network node 402. After adding the first delay field 108 to the one-hop delay, the probe message 106 is sent to the next network node 402. The method 500 discloses continuously checking the position of the probe message 106 and continuing to increase the first delay field 108 until the probe message 106 reaches the next network node 402. For example, as probe message 106 traverses the network path, probe message 106 first traverses node a and adds the internal delay faced by node a to first delay field 108. Further, the method 500 includes determining that node a is not the destination node 112, the controller 104 increasing the first delay field 108 by one-hop delay and transmitting the probe message 106 to node B. Further, the method 500 includes determining that node B is not the destination node 112, the controller again increases the first delay field by one-hop delay and sends the probe message 106 to node C. Further, if node C is determined to be the destination node 112, the controller 104 adapts the first delay field 108 without increasing the one-hop delay.

According to one embodiment, the method 500 further comprises adapting the first delay field 108 by reversing the sign of the first delay field 108. First, the controller 104 receives the probe message 106, wherein the first delay field 108 is filled with zerosThe value of the delay experienced by one or more intermediate nodes 110. Thereafter, the first delay field 108 is adapted by reversing the sign of the value added in the first delay field 108. For example, when the controller 104 receives the probe message 106, the first delay field 108 has a forward delay D _F Thus by combining D _F Set as-D _F The first delay field 108 is adapted to also use the first delay field 108 at the receiving side with the addition of zero or more delay values experienced by the intermediate node 110. It should be noted that zero or more intermediate nodes 110 do not change their behavior according to the direction of the probe messages 106.

According to one embodiment, the probe message 106 includes a second delay field, and the controller 104 is further configured to adapt the first delay field 108 by copying the first delay field 108 to the second delay field and resetting the first delay field 108. First, probe message 106 arrives at controller 104 with first delay field 108 filled with zero or more delays experienced by intermediate node 110. Thereafter, the delay is copied to the second delay field. Further, upon return of the probe message 106, zero or more delays experienced by the intermediate node 110 are added to reuse the first delay field 108. For example, when the probe message 106 arrives at the destination node 112, the first delay field 108 has a forward delay D _F Thus by combining D _F Copy to the second delay field to adapt the first delay field 108 to also use the first delay field 108 at the receiving side.

Further, after copying the first delay field 108 to the second delay field, the controller 104 is configured to reset the first delay field 108 to fit the first delay field 108 and add zero or more delays experienced by the intermediate node 110 when the probe message 106 returns. Further, when the controller 104 resets the first delay field 108, the added delay value of zero or more intermediate nodes 110 is deleted and the first delay field 108 is reset to add a new value when the probe message 106 traverses the receiving side.

According to one embodiment, the probe message 106 includes a function field, and the controller 104 is configured to adapt the first delay field 108 by performing the function indicated by the function field. The function field is in the probe message 106, so the controller 104 performs the function provided by the function field to adapt the first delay field 108. The function field has data that detects the address that the message 106 needs to transmit. In one implementation, the function field is located in a segment routing header. In one implementation, the function field has a segment identifier for each network node 102, zero or more intermediate nodes 110, and the next network node 402.

The method 500 measures delay asymmetry in a network path and provides explicit control of the network path. The method 500 discloses increasing the first delay field 108 by one-hop delay upon detecting that the network node 102 is not the destination node 112. Further, the forward and reverse delays stored in the first delay field 108 are analyzed to determine delay asymmetry in the network path, which facilitates receiving information about delay asymmetry in the transmitting side and the receiving side. Further, the method 500 measures delay asymmetry, which is useful for congestion control in transport protocols such as transport control protocol (transmission control protocol, TCP) or remote direct data access (remote direct memory access, RDMA), as TCP and RDMA may benefit by using delay asymmetry in rate limiting decisions. For example, when the delay on the transmitting side is asymmetrically long, the transmission protocol instructs the transmitting side to decrease the transmission rate; when the delay on the receiving side is asymmetrically long, the transmission protocol instructs the sender to keep the sending rate unchanged.

Fig. 6 is a diagram illustrating a structural representation of a delay asymmetry measurement procedure according to another embodiment of the invention. Fig. 6 is explained in conjunction with fig. 1 and 4. Referring to fig. 6, a diagram 600 is shown that includes a source node 602, a first intermediate node 604A, a second intermediate node 604B, a third intermediate node 604C, a fourth intermediate node 604D, a fifth intermediate node 604E, a sixth intermediate node 604F, and a terminal node 606.

The source node 602 may comprise suitable logic, circuitry, interfaces, or code that may be an initial node of a network path. The source node 602 corresponds to the network node 102 (of fig. 1).

Each of the first intermediate node 604A, the second intermediate node 604B, the third intermediate node 604C, the fourth intermediate node 604D, the fifth intermediate node 604E, and the sixth intermediate node 604F may include routers, switches, and the like for interconnecting in a network path. The first intermediate node 604A-the sixth intermediate node 604F provide a path for sending the probe message 106 from the source node 602 to the end node 606. The first through sixth intermediate nodes 604A through 604F correspond to the intermediate nodes 110A through 110N (of fig. 1).

The end node 606 is the last node from which the probe message 106 is returned to the source node 602. The end node 606 corresponds to the destination node 112 (of fig. 1).

The controller 104 generates probe messages 106 sent from the source node 602 to the end node 606 for measuring delay asymmetry in the network path. In one implementation, the probe message 106 includes a segment routing header (segment routing header, SRH) for providing a path for the probe message 106. Further, the SRH includes a list of segment identifiers (segment identifier, SID) for storing addresses of the source node 602, the first through sixth intermediate nodes 604A-604F, and the end node 606. Further, the probe message 106 first arrives at the first intermediate node 604A, and the first intermediate node 604A detects the telemetry header in the probe message 106. The probe message 106 includes a first delay field 108 in which the delay experienced by the first intermediate node 604A is added when the telemetry header is detected. Further, the controller 104 determines that the first intermediate node 604A is not the end node 606, so the controller 104 increases the first delay field by one-hop delay and sends the probe message 106 to the next node, the second intermediate node 604B. Further, the second intermediate node 604B through the sixth intermediate node 604F add their internal delay in the first delay field 108, except for the third intermediate node 604C, which is not traversed by the probe message 106 because the address of the third intermediate node 604C is not provided by the SID. Further, probe message 106 includes a segment routing (Segment Routing Internet Protocol Version, srv 6) header based on internet protocol version six to instruct end node 606. Furthermore, when the probe message 106 arrives at the end node 606, the SRv header instructs the end node to return the probe message 106 without adding its own internal delay in the first delay field 108. According to one method, the SRv header instructs the end node 606 to invert the sign of the forward delay stored in the first delay field 108 to reuse the first delay field 108 to add the reverse delay. According to another approach, the SRv header instructs the end node 606 to copy the forward delay from the first delay field 108 to the second delay field. Further, the first delay field 108 is reset to be reused to add reverse delay during the return of the probe message 106 to the source node 602.

Fig. 7A is a diagram illustrating a format of a probe message according to another embodiment of the present invention. Fig. 7A is explained in conjunction with fig. 1 and 4. Referring to fig. 7A, a diagram 700A depicting the format of a probe message 106 is shown. The format of the probe message 106 includes a sixth version of internet protocol (Internet Protocol Version, ipv 6) header 702, a next header 704, a header extension length 706, a route type 708, a number of remaining segments 710, a last entry 712, a flag 714, a tag 716, segment lists 718A through 718N, a segment list 718I, a type length value (type length value, TLV) 720, a length 722, a reserved 724, a first delay field 108, a payload 726, and padding 728.

An IPv6 header 702 is provided to provide commands to destination node 112. IPv6 corresponds to a segment routing (Segment Routing Internet Protocol Version, srv 6) header based on internet protocol version six. The next header 704 assists zero or more intermediate nodes 110 in detecting probe messages 106. Header extension 706 is an extra bit of the next header 704. The routing type 708 is used to detect the type of zero or more intermediate nodes 110. The remaining segment number 710 is used to store additional information for zero or more intermediate nodes 110. The last entry 712 is used to examine the last delay added in the first delay field 108 by zero or more intermediate nodes 110. The flag 714 is used to flag the delayed entries added in the first delay field 108 by zero or more intermediate nodes 110. The label 716 is used to flag the threshold. The segment lists 718A through 718N include a plurality of segment lists, e.g., segment list 718A, segment list 718I, through segment list 718N. In one implementation, the segment list 718I is used to represent the address of the destination node 112. In addition, segment list 718A is used to represent the address of network node 102. Further, segment list 718N is used to represent the addresses of zero or more intermediate nodes 110. In one example, the segment list 718A, the segment list 718N, and the segment list 718I are 128 bits in size. Further, the TLV 720 stores the delay experienced by each of the zero or more intermediate nodes 110, e.g., the TLV 720 stores the first delay field 108. In addition, length 722 represents the magnitude of the forward delay. The reservation 724 is used to increase the delay of additional nodes in the network path. The payload 726 is used to store information in the probe message 106. Padding 728 is used to ensure that the size of each file in probe message 106 is measured.

The probe messages are generated by the controller 104 and sent to zero or more intermediate nodes 110. The transmission of the probe message 106 originates from the network node 102, and the address of the network node 102 is stored in the segment list 718A. Further, when the addresses of zero or more intermediate nodes 110 are referenced in the segment list 718N, the probe message 106 is sent to zero or more intermediate nodes 110. Further, the delay experienced by each of zero or more intermediate nodes 110 is added to first delay field 108. Thereafter, the probe message 106 arrives at the destination node 112. The address of destination node 112 is referenced in segment list 718I. Further, the IPv6 header 702 instructs the destination node 112 to return the probe message 106 without increasing its own internal delay. In addition, the IPv6 header 702 instructs the destination node 112 to invert the sign of the forward delay stored in the first delay field 108 to reuse the first delay field 108 to add a reverse delay during the return of the probe message 106.

Fig. 7B is another diagram illustrating a format of a probe message according to another embodiment of the present invention. Fig. 7B is explained in conjunction with fig. 1, 4 and 7A. Referring to fig. 7B, a representation 700B depicting the format of the probe message 106 is shown. The format of the probe message 106 includes a sixth version of internet protocol (Internet Protocol Version, ipv 6) header 702, a next header 704, a header extension length 706, a route type 708, a number of remaining segments 710, a last entry 712, a flag 714, a tag 716, segment lists 718A through 718N, a segment list 718I, a type length value (type length value, TLV) 720, a length 722, a reserved 724, a first delay field 108, a payload 726, a padding 728, and a second delay field 730.

The forward delay 732 is an internal delay experienced by zero or more intermediate nodes 110 and is added to the first delay field 108 during the transmission of the probe message 106 to the destination node 112.

The probe messages are generated by the controller 104 and sent to zero or more intermediate nodes 110. The transmission of the probe message 106 originates from the network node 102, and the address of the network node 102 is stored in the segment list 718A. Further, when the addresses of zero or more intermediate nodes 110 are referenced in the segment list 718N, the probe message 106 is sent to zero or more intermediate nodes 110. Further, the delay experienced by each of zero or more intermediate nodes 110 is added to first delay field 108. Further, the probe message 106 reaches the destination node 112. The address of destination node 112 is referenced in segment list 718I. Further, the IPv6 header 702 instructs the destination node 112 to return the probe message 106 without increasing its own internal delay. In addition, the IPv6 header 702 instructs the destination node 112 to copy the forward delay stored in the first delay field 108 to the second delay field 730 and to reset the first delay field 108. Further, the probe message 106 is returned to the network node 102, and a reverse delay is added in the first delay field 108 during the return of the probe message 106.

Fig. 8 is another flow chart of a method for use in a network node for determining delay asymmetry in a network path according to an embodiment of the invention. Fig. 8 is described in connection with the elements of fig. 1 and 4. Referring to fig. 8, a flow chart of a method 800 including steps 802 through 816 is shown.

A method 800 for use in a network is provided for determining delay asymmetry in a network path from an originating network node to a destination node 112 via zero or more intermediate nodes 110. Method 800 provides an improved way of measuring delay asymmetry in a network path. The method 800 discloses probing the packet 106 across the network path to collect data of the delay faced by each node in the network path. In other words, method 800 discloses providing delay information for each direction, and method 800 uses a single field, which is advantageous in reducing overall costs in terms of overhead.

In step 802, the method 800 includes: the originating network node generates a probe message 106, wherein the probe message 106 includes a first delay field 108. The probe message 106 is generated by the originating network node for measuring delay asymmetry in the network path. In one implementation, the originating network node corresponds to network node 102 (of fig. 1). The probe message 106 collects data while traversing the network path. The first delay field 108 is part of the probe message 106 that stores data collected by the probe message 106. For example, the asymmetric delay faced by zero or more intermediate nodes 110 when a probe packet 106 arrives at zero or more intermediate nodes 110 is stored in the first delay field 108 of the probe packet 106. In this way, probe message 106 collects data of delay asymmetry faced by each of zero or more intermediate nodes 110 in the network path.

In step 804, the method 800 includes: the probe message 106 is sent to the destination node 112 through zero or more intermediate nodes 110. Zero or more intermediate nodes 110 provide paths for sending probe messages 106 to destination node 112. For example, probe message 106A and subsequent probe messages are first forwarded to zero or more intermediate nodes 110, and then probe message 106 reaches destination node 112.

In step 806, the method 800 includes: by adding a one-hop delay to the first delay field 108, the first delay field 108 is adapted according to the delay experienced by each of the zero or more intermediate nodes 110 on the route to the destination node 112. In one implementation, probe message 106 also includes a telemetry header. Further, zero or more intermediate nodes 110 adapt the first delay field 108 as the probe message 106 traverses zero or more intermediate nodes 110 to reach the destination node 112, and after the telemetry header is detected, the delay experienced by each of the zero or more intermediate nodes 110 is added to the first delay field 108 of the probe message 106. Furthermore, if the probe message 106 is on the network node 102 and the network node 102 is not the last node to which the probe message 106 needs to be returned, a one-hop delay is added in the first delay field 108 to add the internal delay faced by another intermediate node.

In step 808, the method 800 includes: the first delay field 108 is adapted without the destination node 112 adding the first delay field 108. In other words, as the probe message 106 traverses the network path, the probe message 106 traverses zero or more intermediate nodes 110 to reach the destination node 112. Further, the location of the probe message 106 is checked, and if the probe message 106 is on the network node 102 and the network node 102 is the last node the probe message 106 needs to return, the first delay field 108 of the probe message 106 is adapted without adding any delta.

In step 810, the method 800 includes: the probe message 106 is sent to the originating node through zero or more intermediate nodes 110. When probe message 106 reaches destination node 112, probe message 106 is sent back to the originating node. Further, the probe message 106 traverses zero or more intermediate nodes 110 and stores the reverse delay in the first delay field 108.

In step 812, the method 800 includes: by adding a one-hop delay to the first delay field 108, the first delay field 108 is adapted according to the delay experienced by the node in each of the zero or more intermediate nodes 110 on the route to the originating node. Further, zero or more intermediate nodes 110 adapt the first delay field 108 as the probe message 106 traverses zero or more intermediate nodes 110 to reach the originating node, and after the telemetry header is detected, the delay experienced by each of the zero or more intermediate nodes 110 is added to the first delay field 108 of the probe message 106. Further, checking the location of the probe message 106, if the probe message 106 is not on the originating node, increasing the first delay field 108 by one-hop delay continues to increase by one-hop delay until the probe message 106 reaches the originating node.

In step 814, method 800 includes: probe messages 106 from destination node 112 are received at the originating node through zero or more intermediate nodes 110. Probe messages 106 from destination node 112 are received through zero or more intermediate nodes 110. Upon returning from the destination node 112, the probe message 106 traverses zero or more intermediate nodes 110, updating the first delay field 108 by adding the delay experienced by each of the zero or more intermediate nodes 110.

In step 816, the method 800 includes: it is determined whether there is delay asymmetry based on the first delay field 108. After receiving the probe message 106 from the destination node 112, the controller 104 measures the delay asymmetry by analyzing the first delay field 108. For example, probe message 106 traverses zero or more intermediate nodes 110 to reach destination node 112, and probe message 106 traverses zero or more intermediate nodes 110 again to return to network node 102. In this case, zero or more intermediate nodes 110 add their internal delays to the first delay field 108. Further, the controller 104 determines delay asymmetry by evaluating the value of the internal delay stored in the first delay field 108.

According to one embodiment, the method 800 further comprises: the probe message 106 includes a sixth version of the internet protocol based segment routing (Segment Routing Internet Protocol Version, srv 6) header that includes a first delay field 108. In one implementation, the SRv header is used to instruct the destination node 112 to return the probe message 106 without updating its own internal delay in the first delay field 108. According to one method, the SRv header inverts the sign of the first delay so that the first delay field 108 is also used at the receiving side. For example, when the probe message 106 arrives at the destination node 112, the first delay field 108 has a forward delay D _F Thus SRv header command will be forward delayed by D _F Change to-D _F To use the first delay field 108 at the receiving side, also with the addition of zero or more delay values experienced by the intermediate node 110. By reversing the sign of the forward delay, the SRv6 header uses one field in both directions, which is advantageous in reducing overall overhead costs. According to another approach, SRv header commands copy the forward delay from the first delay field 108 to the second delay field. Further, the first delay field 108 is reset for reuse in adding the reverse delay. For example, when the probe message 106 arrives at the destination node 112, the first delay field 108 has Forward delay D _F Thus SRv header order node 112 will D _F Copying to the second delay field to also use the first delay field 108 at the receiving side helps to provide more information about the delay asymmetry of the transmitting side and the receiving side.

In these embodiments, the SRv header also includes a function field. The function field is part of the SRv header. The function field is used to store a command to perform a delay asymmetry measurement. For example, the probe message 106 is required to traverse the network path, and then the probe message 106 reaches zero or more intermediate nodes 110 mentioned in the function field.

In these embodiments, the SRv header further includes a list of segment identifier (segment identifier, SID) entries for the originating network node, the destination network node, and zero or more intermediate network nodes, wherein the list entry for the destination node 112 includes a function field. The list of SID entries of the SRv header provides a path for sending and receiving probe messages 106 by storing the address of the originating network node (e.g., network node 102 of fig. 1), the destination network node (e.g., destination node 112 of fig. 1), and zero or more intermediate network nodes (e.g., zero or more intermediate nodes 110 of fig. 1). For example, the probe message 106 traverses the network path, and thus the list of SID entries provides a list of sequences that the probe message 106 needs to transmit. The SID entry list also includes a function subfield for storing an identification for each node.

Method 800 measures delay asymmetry in a network path and provides explicit control of the network path through a combination of segment routing and propagation delay measurements. The method 800 discloses increasing the first delay field 108 by one-hop delay upon detecting that the network node 102 is not the destination node 112. Further, the forward and reverse delays stored in the first delay field 108 are analyzed to determine delay asymmetry in the network path, which facilitates receiving information about delay asymmetry in the transmitting side and the receiving side. Further, method 800 measures delay asymmetry, which is useful for congestion control in transport protocols such as transport control protocol (transmission control protocol, TCP) or remote direct data access (remote direct memory access, RDMA), because TCP and RDMA use delay asymmetry in rate limiting decisions. For example, when the delay on the transmitting side is asymmetrically long, the transmission protocol instructs the transmitting side to decrease the transmission rate; when the delay on the receiving side is asymmetrically long, the transmission protocol instructs the sender to keep the sending rate unchanged.

Steps 802 through 816 are merely illustrative, and other alternatives may be provided, including adding one or more steps, deleting one or more steps, or providing one or more steps in a different order without departing from the scope of the claims herein.

Modifications may be made to the embodiments of the invention described above without departing from the scope of the invention, as defined in the accompanying claims. Expressions such as "comprising," "including," "incorporating," "being/being" and the like used to describe and claim the present invention are intended to be interpreted in a non-exclusive manner, i.e. to allow items, components or elements not explicitly described to exist as well. Reference to the singular is also to be construed to relate to the plural. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration. Any embodiment described as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments, or as excluding combinations of features of other embodiments. The word "optionally" as used herein means "provided in some embodiments and not provided in other embodiments. It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable combination or as any suitable other embodiment of the invention.

Claims

1. A controller (104) for use in a network node (102), wherein the controller (104) is configured to determine delay asymmetry in a network path from the network node (102) to a destination node (112) via zero or more intermediate nodes (110) by:

generating a probe message (106), wherein the probe message (106) comprises a first delay field (108);

-transmitting the probe message (106) to the destination node (112) by the zero or more intermediate nodes (110), causing the zero or more intermediate nodes (110) to adapt the first delay field (108) according to the delay experienced on the route to the destination node (112);

-receiving, by the zero or more intermediate nodes (110), the probe message (106) from the destination node (112), causing the zero or more intermediate nodes (110) to adapt the first delay field (108) according to the delay experienced on the route from the destination node (112);

a determination is made based on the first delay field (108) whether there is delay asymmetry.

2. The controller (104) of claim 1, wherein the controller (104) is configured to determine delay asymmetry in the network path by determining that the first delay field (108) indicates a delay that exceeds a threshold.

3. The controller (104) of claim 2, wherein the controller (104) is configured to determine a delay asymmetry in the network path as the value indicated by the first delay field (108).

4. The controller (104) of claim 1, wherein the probe message (106) includes a second delay field, the controller (104) configured to determine delay asymmetry in the network path by determining a difference between the first delay field (108) and the second delay field.

5. The controller (104) of claim 4, wherein the controller (104) is configured to determine whether delay asymmetry in the network path indicates that a transmit side delay is longer or a receive side delay is longer based on whether the difference is positive or negative.

6. The controller (104) according to claim 4 or 5, wherein the controller (104) is configured to: if the difference is negative, determining that delay asymmetry in the network path indicates that the delay on the transmit side is longer; if the difference is positive, it is determined that delay asymmetry in the network path indicates that the delay on the receiving side is longer.

7. A controller (104) for use in a network node (102), wherein the controller (104) is configured to:

receiving a probe message (106), wherein the probe message (106) comprises a first delay field (108);

determining whether the network node (102) is a destination node (112) of the probe message (106); if not, the following steps are performed:

adding a one-hop delay to the first delay field (108);

-sending the probe message (106) to the next network node.

8. The controller (104) of claim 7, wherein the controller (104) is further configured to: if it is determined that the network node (102) is the destination node (112), adapting the first delay field (108) without adding the first delay field (108).

9. The controller (104) of claim 8, wherein the controller (104) is further configured to adapt the first delay field (108) by reversing a sign of the first delay field (104).

10. The controller (104) of claim 8, wherein the probe message (106) includes a second delay field, the controller (104) further configured to adapt the first delay field (108) by:

Copying the first delay field (108) to the second delay field;

the first delay field (108) is reset.

11. The controller (104) according to any one of claims 8 to 10, wherein the probe message (106) comprises a function field, the controller (104) being configured to adapt the first delay field (108) by performing a function indicated by the function field.

12. The controller (104) of any one of the preceding claims, wherein the probe message (106) comprises a sixth version of an internet protocol based segment routing (Segment Routing Internet Protocol Version, srv 6) header comprising the first delay field (108).

13. The controller (104) of claims 11 and 12, wherein the SRv header further comprises the function field.

14. The controller (104) of claim 13, wherein the SRv header further comprises a list of segment identifiers (segment identifier, SID) entries for an originating network node, a destination network node, and zero or more intermediate network nodes, wherein the list entry for the destination node (112) comprises the function field.

15. A method (300) for use in a network node (102) for determining delay asymmetry in a network path from the network node (102) to a destination node (112) via zero or more intermediate nodes (110), wherein the method comprises:

generating a probe message (106), wherein the probe message (106) comprises a first delay field;

16. The method (300) of claim 15, wherein the method (300) further comprises determining delay asymmetry in the network path by determining that the first delay field (108) indicates a delay that exceeds a threshold.

17. The method (300) of claim 16, wherein the method (300) further comprises determining a delay asymmetry in the network path as the value indicated by the first delay field (108).

18. The method (300) of claim 15, wherein the probe message (106) includes a second delay field, and wherein the controller (104) is configured to determine delay asymmetry in the network path by determining a difference between the first delay field (108) and the second delay field.

19. The method (300) of claim 18, wherein the method (300) further comprises determining whether delay asymmetry in the network path indicates that a transmit side delay is longer or a receive side delay is longer based on whether the difference is positive or negative.

20. The method (300) according to claim 18 or 19, wherein the method (300) further comprises: if the difference is negative, determining that delay asymmetry in the network path indicates that the delay on the transmit side is longer; if the difference is positive, it is determined that delay asymmetry in the network path indicates that the delay on the receiving side is longer.

21. A method (500) for use in a network node, the method (500) comprising:

adding a one-hop delay to the first delay field (108);

-sending the probe message (106) to the next network node.

22. The method (500) of claim 21, wherein the method (500) further comprises: if it is determined that the network node (102) is the destination node (112), adapting the first delay field (108) without adding the first delay field (108).

23. The method (500) of claim 22, wherein the method (500) further comprises adapting the first delay field (108) by reversing a sign of the first delay field (108).

24. The method (500) of claim 22, wherein the probe message (106) includes a second delay field, the controller (104) further configured to adapt the first delay field (108) by:

Copying the first delay field (108) to the second delay field;

the first delay field (108) is reset.

25. The method (500) according to any of claims 22 to 24, wherein the probe message (106) comprises a function field, the controller (104) being configured to adapt the first delay field (108) by performing a function indicated by the function field.

26. A method (800) for use in a network for determining delay asymmetry in a network path from an originating network node to a destination node (112) via zero or more intermediate nodes (110), wherein the method comprises:

-the originating network node generating a probe message (106), wherein the probe message (106) comprises a first delay field (108);

-transmitting the probe message (106) to the destination node (112) through the zero or more intermediate nodes (110);

adapting the first delay field (108) according to the delay experienced by each of the zero or more intermediate nodes (110) on the route to the destination node (112) by adding one-hop delay to the first delay field (108);

adapting the first delay field (108) without the destination node (112) adding the first delay field (108);

-sending the probe message (106) to the originating node through the zero or more intermediate nodes (110);

adapting the first delay field (108) according to the delay experienced by each of the zero or more intermediate nodes (110) on the route to the originating node by adding one-hop delay to the first delay field (108);

-receiving, at the originating node (112), the probe message (106) from the destination node (112) through the zero or more intermediate nodes (110);

27. The method (800) of any of claims 15-26, wherein the probe message (106) comprises a sixth version of an internet protocol based segment routing (Segment Routing Internet Protocol Version, srv 6) header comprising the first delay field (108).

28. The method (800) of claims 25 and 27, wherein the SRv header further comprises the function field.

29. The method (800) of claim 28, wherein the SRv header further comprises a list of segment identifiers (segment identifier, SID) entries for an originating network node, a destination network node, and zero or more intermediate network nodes, wherein the list entry for the destination node (800) comprises the function field.