CN115102905A - ECN (engineering-centric networking) waterline adjusting method and device - Google Patents
ECN (engineering-centric networking) waterline adjusting method and device Download PDFInfo
- Publication number
- CN115102905A CN115102905A CN202210749818.8A CN202210749818A CN115102905A CN 115102905 A CN115102905 A CN 115102905A CN 202210749818 A CN202210749818 A CN 202210749818A CN 115102905 A CN115102905 A CN 115102905A
- Authority
- CN
- China
- Prior art keywords
- ecn
- statistical information
- waterline
- queue
- port
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 230000006855 networking Effects 0.000 title description 2
- 230000002776 aggregation Effects 0.000 claims abstract description 133
- 238000004220 aggregation Methods 0.000 claims abstract description 133
- 230000009471 action Effects 0.000 claims abstract description 70
- 238000012545 processing Methods 0.000 claims abstract description 38
- 238000012549 training Methods 0.000 claims description 36
- 239000011435 rock Substances 0.000 claims description 25
- 238000005516 engineering process Methods 0.000 claims description 9
- 230000000875 corresponding effect Effects 0.000 description 61
- 230000008569 process Effects 0.000 description 36
- 239000003795 chemical substances by application Substances 0.000 description 22
- 208000027744 congestion Diseases 0.000 description 15
- 230000002787 reinforcement Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000005457 optimization Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 238000010521 absorption reaction Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/12—Avoiding congestion; Recovering from congestion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
- H04L43/0888—Throughput
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
- H04L47/62—Queue scheduling characterised by scheduling criteria
- H04L47/625—Queue scheduling characterised by scheduling criteria for service slots or service orders
- H04L47/6275—Queue scheduling characterised by scheduling criteria for service slots or service orders based on priority
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/70—Admission control; Resource allocation
- H04L47/80—Actions related to the user profile or the type of traffic
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides an ECN (ECN waterline adjusting method and device, wherein the method comprises the following steps: for a first ROCE queue of a first port of an ECN, acquiring first appointed aggregation statistical information of the first ROCE queue of the first port; inquiring a current Q table according to the first appointed aggregation statistical information to determine a first ECN waterline corresponding to the first appointed aggregation statistical information; the initial value of the Q table is a pre-trained Q table, the Q table takes preset specified aggregation statistical information as a state, and a preset ECN waterline is sent as an action; and carrying out ECN processing on the first ROCE queue of the first port according to the first ECN waterline. The method and the device can improve the rationality of ECN (engineering-centered network) waterline configuration.
Description
Technical Field
The invention relates to the technical field of network communication, in particular to an ECN (engineering-centered network) waterline adjusting method and device.
Background
RDMA (Remote Direct Memory Access) is a technology capable of solving server-side data processing delay in network transmission. RDMA eliminates external memory copy and text exchange operations by quickly moving data from one system to a remote system memory without any impact on the operating system, thus freeing up memory bandwidth and processing by the CPU (central processing Unit).
RoCE (RDMA over Converged Ethernet RDMA technology) technology is one of the mainstream RDMA technologies. In the RoCE network, a lossless Ethernet needs to be constructed to ensure no packet loss during the network transmission process. The key characteristics that need to be supported for constructing the lossless ethernet network include PFC (Priority-based Flow Control), ECN (Explicit Congestion Notification), and the like. The ECN technique is that when Congestion occurs in a device, a receiving end sends a CNP (Congestion Notification Packet) message that reduces a sending rate to a sending end by identifying an ECN field in an IP header of the message, so as to implement end-to-end Congestion management and reduce Congestion diffusion degradation.
The network card realizes a DCQCN (Data Center Quantized Congestion Notification) Congestion control mechanism aiming at RoCEv2, the mechanism only requires a network switch to carry out ECN marking on a message suffering from Congestion, and the network cards at two ends communicating with each other adjust the flow rate according to the ECN marking condition. The existing switches support that ECN marks are marked on messages meeting queue congestion so as to inform a source end of reducing the speed of the flow and relieve the congestion condition of the switches.
The switch performs ECN marking on the message according to the configured ECN waterline. The ECN waterline is set to be higher, the burst absorption capacity of the queue is strong, the throughput is beneficial, but the excessive queue depth and time delay are brought, and the control/protocol flow sensitive to the time delay is not beneficial. On the contrary, the ECN waterline is set to be lower, the depth of the queue is maintained to be lower, the queue delay is lower, and the ECN waterline is beneficial to delay-sensitive service flows, but the burst absorption capability of the queue is weak, so that the ECN waterline is unfavorable to services with high throughput requirements.
How to reasonably set the ECN waterline becomes a technical problem to be solved urgently.
Disclosure of Invention
In view of this, the present invention provides an ECN waterline adjusting method and apparatus to improve the rationality of ECN waterline setting.
In a first aspect, the present invention provides an ECN pipeline adjusting method, applied to a switching device, where the method includes:
for a first converged Ethernet remote direct access technology (ROCE) queue of a first port of an enabling ECN, acquiring first appointed aggregation statistical information of the first ROCE queue of the first port;
inquiring a current Q table according to the first appointed aggregation statistical information to determine a first ECN waterline corresponding to the first appointed aggregation statistical information; the initial value of each element in the Q table is the value of each element in the Q table which is trained in advance, each row of elements of the Q table corresponds to the same state, each column of elements corresponds to the same action, the Q table takes preset specified aggregation statistical information as the state, and takes a preset ECN waterline as the action;
and carrying out ECN processing on the first ROCE queue of the first port according to the first ECN waterline.
In a second aspect, the present invention provides an ECN pipeline adjusting apparatus, applied to a switching device, where the apparatus includes:
an obtaining unit, configured to obtain, for a first converged ethernet remote direct data access technology, ROCE, queue of a first port of an ECN enabled, first specified aggregation statistical information of the first ROCE queue of the first port;
a determining unit, configured to query a current Q table according to the first specified aggregation statistical information to determine a first ECN waterline corresponding to the first specified aggregation statistical information; the initial value of each element in the Q table is the value of each element in the Q table which is trained in advance, each row of elements of the Q table corresponds to the same state, each column of elements corresponds to the same action, the Q table takes preset specified aggregation statistical information as the state, and takes a preset ECN waterline as the action;
and the processing unit is used for carrying out ECN processing on the first ROCE queue of the first port according to the first ECN waterline.
By applying the technical scheme disclosed by the invention, intelligent training is carried out by using reinforcement Learning Q-Learning to obtain the Q table for determining the optimal ECN waterline configuration in each state, further, the optimal ECN waterline corresponding to the obtained specified aggregation statistical information is determined according to the Q table, and ECN processing is carried out according to the optimal ECN waterline, so that the rationality of the ECN waterline configuration is improved, and performance indexes such as network throughput and the like can be effectively improved.
Drawings
Fig. 1 is a schematic flowchart of an ECN pipeline adjustment method according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a Q table training process according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating the correspondence between the probability of marking and the length of a queue;
fig. 4 is a schematic structural diagram of an ECN waterline adjusting apparatus according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of another ECN waterline adjusting apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of another ECN waterline adjustment apparatus according to an embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the embodiments of the present invention, a brief description will be given below of some terms related to the embodiments of the present invention.
1. DCQCN algorithm
The DCQCN algorithm is based on a combination of a data Center TCP (Transmission Control Protocol) (Dtae Center TCP, DCTCP for short) and a quantization notification algorithm. The DCQCN algorithm relies on ECN labeling at the switch end. ECN is a common feature of commercial data center switches. Two bits in the differentiated services field in the packet IP header are used to indicate congestion. These two bits are set to "11" as soon as congestion occurs at the switch.
The flag congestion is a probability function of the queue length as shown in figure 3. Two thresholds for queue length define the tag probability. When the queue length is below the lower threshold, the ECN bits are not marked. When the length of the queue exceeds the upper threshold, all network packets transmitted from the queue are marked by ECN. When the queue length is between two thresholds, the packets are ECN marked with a probability that the length of the queue increases linearly.
The ECN marked packets are propagated to the network card of the receiving party. The network card of the receiving party creates a CNP (Congestion Notification Packet) and sends it to the sender of the data Packet marked by the ECN. The CNP packet includes information of a marked QP (queue pair). When the CNP is received by the sending network card, the sending network card may reduce the transmission rate of the QP based on a specified algorithm (which may be referred to as a DCQCN slowdown algorithm).
The main principle of the DCQCN deceleration algorithm is as follows: if the QP is based on an internal timer and a send byte counter, the algorithm will continuously increase the sending rate; if a CNP packet is received, the designated QP is slowed down. In addition to this it maintains a parameter called α, which reflects the degree of congestion in the network, for the purpose of the slowdown calculation.
2. Reinforced learning
Reinforcement learning is one area of machine learning that emphasizes how to act based on the environment. The agent (agent) will select a different action (action) in the environment (env). These actions are initially not optimal, but through constant trial and error, after constant trial and error, experience is accumulated, learned, and finally the optimal action can be found to achieve the maximum expected benefit.
The environment receives a series of actions performed by the agent, evaluates the series of actions, and converts the series of actions into a quantifiable signal that is fed back to the agent.
The agent may sense the State of the environment (State) and learn to select an appropriate action based on the Reward for feedback (Reward) to maximize the long term total revenue.
Q-Learning is a method of reinforcement Learning that requires a learned strategy to be recorded to inform the agent what actions to take in what circumstances will have the greatest reward value.
The Q-learning algorithm maintains a table (which may be referred to as a Q-table). The first column represents action and the first row represents state. The value of each cell (i.e., each element) represents the maximum future reward expectation (Q (s, a)) for a given state and corresponding action. Based on the Q-table, the expectation of the maximum future reward may be calculated for each action performed on each state. Depending on the desired size, the best action to take for each state may be known. In actual operation, the algorithm will continuously update the value of the corresponding element in the Q table according to the reward of the executed action, thereby updating the policy of the decision action.
In order to make the aforementioned objects, features and advantages of the embodiments of the present invention more comprehensible, embodiments of the present invention are described in detail below with reference to the accompanying drawings.
Referring to fig. 1, a schematic flow diagram of an ECN waterline adjustment method provided in an embodiment of the present invention is shown, where the cloud testing method may be applied to a switching device, such as a switch, and as shown in fig. 1, the ECN waterline adjustment method may include:
it should be noted that, the sequence numbers of the steps in the embodiment of the present invention do not mean the execution sequence, and the execution sequence of each process should be determined by the function and the internal logic of the process, and should not constitute any limitation to the implementation process of the embodiment of the present invention.
In this embodiment of the present invention, the specified aggregation statistical information may include part or all of the aggregation statistical information of the rock queue.
Illustratively, the aggregated statistics of the ROCE queues may include, but are not limited to: the average bandwidth of the rock dequeue, intake (the number of ingress ports corresponding to the rock data stream in the rock queue), the number of data streams of each ingress port (Flow) \\ the number of large bandwidth data streams (BigFlow) \ the average bandwidth \ write)/read/send (send) message type proportion, RTT (Round-Trip Time), NAK (Negative acknowledgement), the average dequeue length, and the number of messages for printing ECN tags, etc.
In the embodiment of the invention, in consideration of the fact that the static ECN configuration cannot be suitable for all flow scenes and the network flow in the actual scene is constantly changed, the optimal ECN waterline can be automatically adjusted according to a real-time perceived network flow model in order to improve the performance indexes such as network throughput and the like, and the network congestion control is realized.
Accordingly, in the embodiment of the present invention, a Q-table for dynamically determining the ECN waterlines adaptive to different aggregation statistical information may be trained in advance by using a Q-learning algorithm, so that in an actual scene, the best ECN waterline for the aggregation statistical information acquired in real time may be determined according to the trained Q-table.
Illustratively, the ECN waterline may include a lower threshold (low-limit), an upper threshold (high-limit), and a discard-probability (discard-probability).
Exemplarily, in an ECN pipeline tuning scenario, the environment is an RDMA lossless network composed of switches and servers (network cards); the agent is an algorithm model; the action is the issued ECN water line value; the state is the environmental data (specified aggregated statistics in the present embodiment) collected by the switch.
In the embodiment of the invention, in the training process of the Q table, the preset specified aggregation statistical information can be used as a state, a preset ECN (equal cost network) waterline is used as an action, and the maximum future reward expectation of any action in any state can be determined through training.
The training process of the Q table may be described below with reference to fig. 2, and is not described herein again in the embodiments of the present invention.
For example, for any rock queue (referred to as a first rock queue) of any ECN-enabled port (referred to as a first port herein), when first specified aggregation statistical information of the first rock queue of the first port is obtained, according to the first specified aggregation statistical information, a current Q table may be queried, a state corresponding to the first specified aggregation statistical information is determined, an action for which a maximum future reward is expected to be maximum in the state is determined, and an ECN waterline (referred to as a first ECN waterline herein) corresponding to the action is determined as an optimal ECN waterline corresponding to the first specified aggregation statistical information.
And 102, carrying out ECN treatment on the first ROCE queue of the first port according to the first ECN water line.
In the embodiment of the present invention, the switching device may perform ECN processing on the first rce queue of the first port according to the first ECN waterline when the first ECN waterline is determined according to the above manner.
For example, assuming that the first ECN pipeline includes a first lower threshold, a first upper threshold, and a first discard probability, when the length of the first rce queue of the first port is lower than the first lower threshold, ECN marking is not performed on the data packet in the first rce queue; when the length of a first ROCE queue of a first port exceeds a first threshold value upper limit, carrying out ECN marking on all data packets in the first ROCE queue; and when the length of the first ROCE queue of the first port is between a first lower threshold value limit and a first upper threshold value limit, marking the data packet in the first ROCE queue according to the first discarding probability.
Therefore, in the method flow shown in fig. 1, intelligent agent training is performed by using reinforcement Learning Q-Learning to obtain a Q table for determining optimal ECN waterline configuration in each state, and then, according to the Q table, an optimal ECN waterline corresponding to the obtained specified aggregation statistical information is determined, and ECN processing is performed according to the optimal ECN waterline, so that rationality of ECN waterline configuration is improved, and performance indexes such as network throughput can be effectively improved.
It should be noted that, in the embodiment of the present invention, in order to improve flexibility of scheme control, for an ROCE queue of an ECN-enabled port, whether an ECN pipeline adjustment scheme (which may be referred to as dynamic ECN pipeline adjustment) provided in the embodiment of the present invention is enabled or not may be selected according to actual needs, and for any ROCE queue of any ECN-enabled port (such as the first ROCE queue of the first port described above), dynamic ECN pipeline adjustment may be performed according to the scheme provided in the embodiment of the present invention under the condition that the ROCE queue of the port enables dynamic ECN pipeline adjustment, that is, an optimal ECN pipeline is dynamically determined according to the manners described in steps 101 to 103, and ECN processing is performed according to the optimal ECN pipeline; when the rock queue of the port does not enable dynamic ECN waterline adjustment, ECN processing may be performed according to a conventional scheme, or ECN processing may be performed by using another policy, which is not limited in the embodiment of the present invention.
In some embodiments, after the performing the ECN processing according to the first ECN pipeline, the method may further include:
acquiring second specified aggregation statistical information of a first ROCE queue of a first port after ECN processing is carried out according to a first ECN waterline;
determining to issue a first reward value corresponding to the first ECN waterline according to the second specified aggregation statistical information;
updating the value of the first element in the current Q table according to the first reward value; the first element is an element in the Q table, which is located in a row corresponding to the first specified aggregation statistical information and is located in a column corresponding to the first ECN waterline.
Illustratively, considering that a difference exists between a training environment of the Q table and an actual application scene, obtaining the Q table in the training environment may not be optimal, and therefore, when applying the Q table obtained by training to the actual application scene for ECN waterline adjustment, the Q table can be iteratively updated according to the actual application scene, so that the agent can quickly adapt to various application scenes and issue corresponding optimal configurations.
Accordingly, after the first ECN waterline is determined according to the method flow shown in fig. 1 and the first rock queue of the first port is ECN-processed according to the first ECN waterline, the specified aggregate statistical information (referred to as the second specified aggregate statistical information herein) of the first rock queue of the first port after ECN-processing according to the first ECN waterline can be obtained.
In the case that the second specified aggregation statistical information is obtained, the award value (referred to as the first award value herein) corresponding to the issuance of the first ECN waterline may be determined according to the obtained second specified aggregation statistical information.
For example, after issuing the first ECN waterline, that is, after performing ECN processing on the first rock queue of the first port according to the first ECN waterline, the better the effect of reducing network congestion is, the higher the reward value corresponding to issuing the first ECN waterline is.
For example, in a case where it is determined that the bonus value corresponding to the first ECN waterline (i.e., the first bonus value) is issued, the current Q table may be updated according to the first bonus value, that is, values of elements in the current Q table corresponding to the first specified aggregation statistical information and the first ECN waterline, that is, elements in the Q table corresponding to the first specified aggregation statistical information (referred to as first elements herein) and in a column corresponding to the first ECN waterline (referred to as rows) are updated.
For example, assuming that the first specified aggregate statistics corresponds to the mth state in the Q table and the first ECN pipeline corresponds to the nth action in the Q table, the value of the element corresponding to the nth action of the mth state in the Q table may be updated according to the first reward value.
It should be noted that, in the embodiment of the present invention, for any rock queue of any ECN-enabled port (for example, the first rock queue of the first port), the switching device may periodically obtain the specified aggregation statistical information (the first specified aggregation statistical information or the second specified aggregation statistical information) of the rock queue of the ECN port, and for example, may obtain the specified aggregation statistical information of the rock queue of the ECN port every 500 ms. When the switching device acquires the specified aggregation statistical information of the ROCE queue of the port each time, the switching device may determine the optimal ECN waterline corresponding to the currently acquired aggregation statistical information according to the manner described in step 102, and adjust the current ECN waterline according to the determined optimal ECN waterline, so as to perform ECN processing on the ROCE queue of the port according to the determined optimal ECN waterline.
For example, when the second specified aggregate statistical information is obtained, the current Q table (the Q table updated according to the first bonus value) may be queried according to the second specified aggregate statistical information, the best ECN waterline corresponding to the second specified aggregate statistical information may be determined, and the ECN waterline may be adjusted according to the best ECN waterline.
Referring to fig. 2, a flow chart of a Q table training method according to an embodiment of the present invention is schematically shown, and as shown in fig. 2, the Q table training method may include the following steps:
And 203, randomly selecting a preset action, and carrying out ECN treatment on a second ROCE queue of a second port according to a second ECN water line corresponding to the preset action.
And 204, acquiring fourth appointed aggregation statistical information of the second ROCE queue of the second port after ECN processing is carried out according to the second ECN waterline.
For example, in the training process, the Q table may be constructed by taking the preset specified aggregation statistical information as a state and taking a preset ECN waterline as an action, and the elements in the Q table may be initialized, for example, the elements in the Q table are initialized to 0.
For example, assuming that the preset specified aggregate statistics information includes 100 value cases (i.e. 100 states in total), and the preset ECN waterline includes 30 value cases (i.e. 30 actions in total), the Q table may be as shown in table 1:
TABLE 1
| Action 1 | Action 2 | … | Action 30 | |
| State 1 | Q(1，1) | Q(1，2) | Q(1，30) | |
| State 2 | Q(2，1) | Q(2，2) | Q(2，30) | |
| … | ||||
| State 100 | Q(100，1) | Q(100，2) | Q(100，30) |
Wherein, except the header (i.e. the first row and the first column), the Q table is a table of 100 × 30, and the element (Q (m, n)) in the mth row and the nth column of the table (except the first row and the first column) is the expected value of the maximum future reward of executing the action n in the state m; here, the initial value of each element may be 0.
For example, in the Q-table training process, the specified aggregate statistical information (referred to as third specified aggregate statistical information herein) of any rock queue (referred to as second rock queue) of any ECN-enabled port (referred to as second port herein) in the training environment may be obtained, and the current state is determined according to the third specified aggregate statistical information.
For example, the Q table may be queried according to the third specified aggregation statistical information, and the state in the Q table matching the third specified aggregation statistical information may be determined as the current state.
In the case that the current state is determined, a preset action may be randomly selected, and ECN processing may be performed with an ECN pipeline (referred to as a second ECN pipeline herein) corresponding to the preset action.
For example, after ECN processing according to the second ECN pipeline, specified aggregate statistics (referred to herein as fourth specified aggregate statistics) of the second rock queue of the second port after ECN processing according to the second ECN pipeline may be obtained.
For example, assuming that the acquisition period of the specified aggregation statistical information is 500ms, the time interval between the time when the fourth specified aggregation statistical information is acquired and the time when the third specified aggregation statistical information is acquired may be 500 ms.
For example, in the case of acquiring the fourth specified aggregation statistical information, on one hand, a reward value (referred to as a second reward value herein) corresponding to the issued second ECN waterline may be calculated according to the fourth specified aggregation statistical information, and a value of an element (referred to as a second element herein) located in a row corresponding to the third specified aggregation statistical information and located in a column corresponding to the second ECN waterline in the Q table may be updated according to the second reward value.
On the other hand, the current state may be updated according to the fourth specified aggregated attack information, and the step of randomly selecting a preset action is skipped, that is, the step 203 is proceeded to, and the update of the Q table is continued until a training end condition is reached, for example, the reinforcement learning algorithm model converges.
In some embodiments, the above-mentioned specified aggregated statistical information may include: link bandwidth, Incast, queue length, egress port traffic, and the number of large bandwidth data streams.
For example, in order to avoid too slow convergence of the reinforcement learning algorithm model during the Q-table training process, the specified aggregate statistical information as a state may not necessarily include all types of aggregate statistical information, but may include some types of aggregate statistical information.
Illustratively, the specified aggregation statistical information may include link bandwidth, Incast, queue length, egress port traffic, and the number of large-bandwidth data streams, that is, the state of the agent is formed by the 5 indexes, and the Q table training efficiency is improved under the condition that the network optimization control effect is ensured.
In one example, the reward value corresponding to the issued ECN waterline is determined by:
and determining a reward value corresponding to the issued ECN waterline according to the queue length and the output port flow in the specified aggregation statistical information acquired after the ECN waterline is issued.
For example, considering that the network congestion condition can be well characterized by the delay and the traffic, the delay and the traffic can be used as an optimal direction for ECN pipelining.
According to laboratory data verification, the high correlation between the queue length and the time delay can be found, and the queue length has the advantage of convenience in measurement, so that the queue length can be selected to measure the optimization reward of the time delay.
Accordingly, for one ECN waterline adjustment, when the specified aggregation statistical information (such as the second specified aggregation statistical information or the fourth specified aggregation statistical information) after the ECN adjustment is obtained, the reward value (such as the first reward value or the second reward value) corresponding to the ECN waterline may be determined to be issued according to the queue length and the egress port flow in the obtained specified aggregation statistical information.
As an example, determining a reward value corresponding to issuing an ECN waterline according to a queue length and an egress port flow in specified aggregation statistical information obtained after issuing the ECN waterline may include:
determining the reward value r corresponding to the ECN waterline by the following formula:
r＝ω 1 ×T(R)-ω 2 ×D(L)
wherein, t (r) is the output port flow in the specified aggregation statistical information obtained after the ECN waterline is issued, d (l) is the queue length, ω, in the specified aggregation statistical information obtained after the ECN waterline is issued 1 +ω 2 1, and ω 1 、ω 2 Are all positive numbers.
For example, it is considered that the larger the egress port traffic in the above specified aggregation statistical information is, the better the network congestion control effect may be indicated, and the larger the queue length in the specified aggregation statistical information is, the worse the network congestion control effect may be indicated.
Accordingly, the reward value corresponding to the ECN waterline is determined by using the formula.
It should be noted that, in practical deployment, different ω may be set for specific application scenarios 1 、ω 2 Giving different weights to queue length D (L) and egress port traffic T (R)And (4) heavy.
For example, for a storage scenario, looking more at heavy traffic optimization, port traffic t (r) may be given a higher weight.
For the AI computation scenario, where latency is more demanding, queue length d (l) may be given a higher weight.
In order to enable those skilled in the art to better understand the technical solution provided by the embodiment of the present invention, the technical solution provided by the embodiment of the present invention is described below with reference to a specific application scenario.
In this embodiment, taking a switch device as an example, the switch device may include an ECN process and an AI application process, and the ECN process may be configured to obtain aggregation statistics for an rce queue of an ECN-enabled port and report the specified aggregation statistics to the AI application process.
For example, the ECN process and the AI application process (hereinafter, simply referred to as an application process) may interact with data in a gRPC (google Remote Procedure Calls) manner, the ECN process may serve as a gRPC client, and the AI application process may serve as a gRPC server.
In this embodiment, the ECN pipeline adjustment scheme is implemented as follows:
1. and the ECN process of the switch counts and manages the ROCE data stream, periodically acquires the specified aggregation statistical information of the specified ROCE queue and the current ECN waterline, and sends the information to the AI application process in a gRPC mode.
Illustratively, the ROCE data stream is generally uniformly mapped onto a designated priority queue by each device on the network, and PFC, ECN and other configurations are enabled on the priority queue to realize lossless forwarding. For example, common RDMA configurations set a cos 5 queue or a cos 4 queue as a ROCE queue.
And the ECN process uploads the ROCE data stream to the FPGA or the CPU in a sampling mirror image mode so as to realize the management and statistics of the 3-tuple of the data stream.
Illustratively, the statistics on the ROCE data stream may include: the number of messages, the total length of the messages, the average length of the messages, the service flow rate, the number of message packets lost, service ingress/egress ports (unicast), VXLAN (Virtual extended Local Area Network) encapsulation information, inactive time, and the like.
In order to assist an application process to adjust an ECN (echo-core) waterline of an ROCE queue, the ECN process of the switch selects an ROCE data stream which takes the ROCE queue as an dequeue by taking the ROCE queue under each port enabling the ECN characteristic as a unit, and performs aggregation statistics.
Illustratively, the aggregated statistics may include: the method comprises the following steps of ROCE dequeue average bandwidth, Incast (the number of ingress ports corresponding to ROCE data streams in the ROCE queue), the number of data streams of each ingress port (Flow) \ the number of large bandwidth data streams (BigFlow) \ average bandwidth \ write/read/send message type proportion, RTT, NAK, average dequeue length, the number of ECN tagged messages and the like.
The ECN process may periodically send these aggregated statistics to the application process in a gRPC manner.
2. AI ECN Qlearning scene definition and parameter selection.
For example, in an AI ECN Qlearning scenario, the environment is an RDMA lossless network composed of switches and servers (network cards); the agent is an algorithm model (i.e. the reinforcement learning algorithm model); the action is the issued ECN waterline (low-limit, high-limit, discard-robustness).
For example, in order to reduce the dimension and increase the convergence speed of the intelligent agent algorithm model, the discard-probability can be set to 0.2, and the low-limit and the high-limit are discretized (10 low-limit and 8 high-limit).
The state includes 5 indexes of link bandwidth, Incast, queue length, output port flow and large bandwidth data flow number.
Illustratively, the states may be discretized into 4480 states (link bandwidths 25G, 100G total 2, incast 2-8 total 7, queue length 10, egress port traffic 8, and large bandwidth data flow number 4).
It should be noted that, when the state is determined according to the obtained aggregation statistical information, precise matching may not be required, for example, taking the output port flow as an example, the above 8 types of output port flows do not need to be a single value, but may be a value range, and when the output port flow in the obtained specified aggregation statistical information falls into a certain value range and other indexes are correspondingly matched, the state corresponding to the currently obtained specified aggregation statistical information may be determined as the state corresponding to the value range.
The design of the reward function r selects two key indexes of queue length D (L) and output port flow T (R), and the optimization directions of the ECN waterline optimization corresponding to time delay and flow.
r＝ω 1 ×T(R)-ω 2 ×D(L)
Wherein, t (r) is the output port flow in the specified aggregation statistical information obtained after the ECN waterline is issued, d (l) is the queue length, ω, in the specified aggregation statistical information obtained after the ECN waterline is issued 1 +ω 2 1, and ω 1 、ω 2 Are all positive numbers.
Illustratively, indexes corresponding to action and state are counted by the ECN process and transferred to the application process by using a gRPC mode.
For example, the ECN process may report the indicator once every 500 ms; and the application process updates the state according to the received indexes, calculates the reward value, updates the Q table and then selects the next action.
Illustratively, queue length is counted every 25ms, and the average over 500ms is counted.
3. The AI ECN agent is trained (i.e., Q-table training described above).
3.1, initializing a Q table, establishing a 4480 x 80 data table, and initializing elements in the data table to 0;
3.2, acquiring appointed aggregation statistical information of an appointed ROCE queue (such as a second ROCE queue of the second port) of an appointed port of the ECN under the training environment, and determining the current state of the intelligent agent according to the appointed aggregation statistical information;
3.3, randomly selecting an action (in the training process, perfecting a Q table in a random action selection mode);
3.4, enabling the action on the switch in a mode of issuing the command line through netconf, namely issuing the ECN waterline corresponding to the action;
3.5, the ECN process transmits the specified aggregation statistical information processed by ECN according to the issued ECN waterline to the intelligent agent in a gRPC mode, and the intelligent agent and the specified aggregation statistical information update the state;
3.6, according to the new state, the intelligent agent calculates the reward value corresponding to the action;
3.7, updating the value of the corresponding element in the Q table according to the reward value (see the relevant description in the above embodiment for the determination mode of the element needing to be updated);
3.8, if the training end condition is not reached, the process goes to step 3.3 (in this case, the state is the updated state in step 3.5), and the training is ended until the training end condition is reached.
Illustratively, over a large number of cycles, the expectation of the maximum future reward in the Q-table may approach the true value. In the subsequent reasoning process, the best action according with the environment can be selected according to the expected value of the maximum future reward.
4. Deployment in an actual scene.
4.1, initializing a Q table: selecting the Q table trained in the step 3) to reduce the training time in the actual environment;
4.2, for any ROCE queue of any ECN-enabled port (such as the first ROCE queue of the first port), acquiring specified aggregation statistical information of the ROCE queue, and determining the state of the agent according to the acquired specified aggregation statistical information;
4.3, selecting action: according to the current state, inquiring the Q table, and selecting the action with the maximum expected value of future rewards;
4.4, enabling the action on the switch in a mode of issuing the command line through netconf, namely issuing the ECN waterline corresponding to the action;
4.5, the ECN process transmits the specified aggregation statistical information processed by ECN according to the issued ECN waterline to the intelligent agent in a gRPC mode, and the intelligent agent and the specified aggregation statistical information update the state;
4.6, according to the new state, the intelligent agent calculates the reward value corresponding to the action
4.7, updating the value of the corresponding element in the Q table according to the reward value (the determination mode of the element needing to be updated is referred to the relevant description in the above embodiment);
4.8, go to step 4.3 (the current state is the updated state in step 4.5).
Referring to fig. 4, a schematic structural diagram of an ECN waterline adjusting apparatus according to an embodiment of the present invention is provided, where the apparatus may be applied to a switching device in the foregoing method embodiment, and as shown in fig. 4, the ECN waterline adjusting apparatus may include:
an obtaining unit 410, configured to obtain, for a first converged ethernet remote direct access technology, rock, queue of a first port of an ECN, first specified aggregation statistical information of the first rock queue of the first port;
a determining unit 420, configured to query a current Q table according to the first specified aggregation statistical information to determine a first ECN waterline corresponding to the first specified aggregation statistical information; the initial value of each element in the Q table is the value of each element in the Q table which is trained in advance, each row of elements of the Q table corresponds to the same state, each column of elements corresponds to the same action, the Q table takes preset specified aggregation statistical information as the state, and takes a preset ECN waterline as the action;
the processing unit 430 is configured to perform ECN processing on the first rock queue of the first port according to the first ECN pipeline.
In some embodiments, the obtaining unit 410 is further configured to obtain second specified aggregation statistical information of the first rock queue of the first port after ECN processing is performed according to the first ECN waterline;
the determining unit 420 is further configured to determine, according to the second specified aggregation statistical information, to issue a first reward value corresponding to the first ECN waterline;
as shown in fig. 5, the apparatus further includes:
an updating unit 440, configured to update a value of a first element in a current Q table according to the first reward value; the first element is an element which is positioned in a row corresponding to the first specified aggregation statistical information and is positioned in a column corresponding to the first ECN waterline in the Q table.
In some embodiments, as shown in fig. 6, the apparatus further comprises:
a training unit 450 for:
initializing a Q table;
for a second ROCE queue of a second port enabling an ECN in a training environment, acquiring third appointed aggregation statistical information of the second ROCE queue of the second port, and determining the current state according to the third appointed aggregation statistical information;
randomly selecting a preset action, and carrying out ECN processing on a second ROCE queue of the second port according to a second ECN waterline corresponding to the preset action;
acquiring fourth appointed aggregation statistical information of a second ROCE queue of the second port after ECN processing is carried out according to the second ECN waterline;
calculating and issuing a second reward value corresponding to the second ECN waterline according to the fourth specified aggregation statistical information, and updating the value of a second element in a Q table according to the second reward value; updating the current state according to the fourth appointed aggregation statistical information, and skipping to the step of randomly selecting a preset action until a training end condition is reached; and the second element is an element which is positioned in a row corresponding to the third specified aggregation statistical information and is positioned in a column corresponding to the second ECN waterline in the Q table.
In some embodiments, the specified aggregated statistics include: link bandwidth, Incast, queue length, egress port traffic, and the number of large bandwidth data streams.
In some embodiments, the determining unit 420 is specifically configured to determine, according to the queue length and the egress port flow in the specified aggregation statistical information acquired after the ECN waterline is issued, a reward value corresponding to the issued ECN waterline.
In some embodiments, the determining unit 420 is specifically configured to determine the reward value r corresponding to the issued ECN waterline by using the following formula:
r＝ω 1 ×T(R)-ω 2 ×D(L)
wherein, t (r) is the output port flow in the specified aggregation statistical information obtained after the ECN waterline is issued, d (l) is the queue length, ω, in the specified aggregation statistical information obtained after the ECN waterline is issued 1 +ω 2 1, and ω 1 、ω 2 Are all positive numbers.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the invention. One of ordinary skill in the art can understand and implement it without inventive effort.
It can be seen from the above embodiments that the Q table for determining the optimal ECN waterline configuration in each state is obtained by performing agent training using reinforcement Learning Q-Learning, and then, the optimal ECN waterline corresponding to the obtained specified aggregation statistical information is determined according to the Q table, and ECN processing is performed according to the optimal ECN waterline, so that the rationality of the ECN waterline configuration is improved, and performance indexes such as network throughput can be effectively improved.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (12)
1. An ECN (event-based network) waterline adjustment method for displaying congestion notification, which is applied to a switching device, and comprises the following steps:
for a first converged Ethernet remote direct access technology (ROCE) queue of a first port of an enabling ECN, acquiring first appointed aggregation statistical information of the first ROCE queue of the first port;
inquiring a current Q table according to the first appointed aggregation statistical information to determine a first ECN waterline corresponding to the first appointed aggregation statistical information; the initial value of each element in the Q table is the value of each element in the Q table which is trained in advance, each row of elements of the Q table corresponds to the same state, each column of elements corresponds to the same action, the Q table takes preset specified aggregation statistical information as the state, and takes a preset ECN waterline as the action;
and carrying out ECN processing on the first ROCE queue of the first port according to the first ECN waterline.
2. The method of claim 1, wherein after said performing ECN processing in accordance with said first ECN pipeline, further comprising:
acquiring second specified aggregation statistical information of a first ROCE queue of the first port after ECN processing is carried out according to the first ECN waterline;
determining to issue a first reward value corresponding to the first ECN waterline according to the second specified aggregation statistical information;
updating the value of the first element in the current Q table according to the first reward value; the first element is an element which is positioned in a row corresponding to the first specified aggregation statistical information and is positioned in a column corresponding to the first ECN waterline in the Q table.
3. The method of claim 1, wherein the training of the Q-table comprises:
initializing a Q table;
for a second ROCE queue of a second port enabling an ECN in a training environment, acquiring third appointed aggregation statistical information of the second ROCE queue of the second port, and determining a current state according to the third appointed aggregation statistical information;
randomly selecting a preset action, and carrying out ECN processing on a second ROCE queue of the second port according to a second ECN waterline corresponding to the preset action;
acquiring fourth appointed aggregation statistical information of a second ROCE queue of the second port after ECN processing is carried out according to the second ECN waterline;
calculating and issuing a second reward value corresponding to the second ECN waterline according to the fourth specified aggregation statistical information, and updating the value of a second element in a Q table according to the second reward value; updating the current state according to the fourth appointed aggregation statistical information, and skipping to the step of randomly selecting a preset action until a training end condition is reached; and the second element is an element which is positioned in a row corresponding to the third specified aggregation statistical information and is positioned in a column corresponding to the second ECN waterline in the Q table.
4. The method of any of claims 1-3, wherein the specifying aggregated statistics comprises: link bandwidth, Incast, queue length, egress port traffic, and the number of large bandwidth data streams.
5. The method of claim 4, wherein the reward value for issuing the ECN waterline is determined by:
and determining a reward value corresponding to the issued ECN waterline according to the queue length and the output port flow in the specified aggregation statistical information acquired after the ECN waterline is issued.
6. The method according to claim 5, wherein determining the reward value corresponding to the issued ECN waterline according to the queue length and the egress port flow in the specified aggregation statistical information acquired after the issued ECN waterline includes:
determining the reward value r corresponding to the issued ECN waterline by the following formula:
r＝ω 1 ×T(R)-ω 2 ×D(L)
wherein, t (r) is the output port flow in the specified aggregation statistical information obtained after the ECN waterline is issued, d (l) is the queue length, ω, in the specified aggregation statistical information obtained after the ECN waterline is issued 1 +ω 2 1, and ω 1 、ω 2 Are all positive numbers.
7. An ECN waterline adjusting device for displaying congestion notification, which is applied to a switching device, and comprises:
an obtaining unit, configured to obtain, for a first converged ethernet remote direct data access technology, ROCE, queue of a first port of an ECN enabled, first specified aggregation statistical information of the first ROCE queue of the first port;
a determining unit, configured to query a current Q table according to the first specified aggregation statistical information to determine a first ECN waterline corresponding to the first specified aggregation statistical information; the initial value of each element in the Q table is the value of each element in the Q table which is trained in advance, each row of elements of the Q table corresponds to the same state, each column of elements corresponds to the same action, the Q table takes preset specified aggregation statistical information as the state, and takes a preset ECN waterline as the action;
and the processing unit is used for carrying out ECN processing on the first ROCE queue of the first port according to the first ECN waterline.
8. The apparatus of claim 7,
the acquiring unit is further configured to acquire second specified aggregation statistical information of the first rock queue of the first port after ECN processing is performed according to the first ECN waterline;
the determining unit is further configured to determine to issue a first reward value corresponding to the first ECN waterline according to the second specified aggregation statistical information;
the device further comprises:
the updating unit is used for updating the value of the first element in the current Q table according to the first reward value; the first element is an element which is positioned in a row corresponding to the first specified aggregation statistical information and is positioned in a column corresponding to the first ECN waterline in the Q table.
9. The apparatus of claim 7, further comprising:
a training unit to:
initializing a Q table;
for a second ROCE queue of a second port enabling an ECN in a training environment, acquiring third appointed aggregation statistical information of the second ROCE queue of the second port, and determining a current state according to the third appointed aggregation statistical information;
randomly selecting a preset action, and carrying out ECN processing on a second ROCE queue of the second port according to a second ECN waterline corresponding to the preset action;
acquiring fourth appointed aggregation statistical information of a second ROCE queue of the second port after ECN processing is carried out according to the second ECN waterline;
calculating and issuing a second reward value corresponding to the second ECN waterline according to the fourth specified aggregation statistical information, and updating the value of a second element in a Q table according to the second reward value; updating the current state according to the fourth appointed aggregation statistical information, and skipping to the step of randomly selecting a preset action until a training end condition is reached; and the second element is an element which is positioned in a row corresponding to the third specified aggregation statistical information and is positioned in a column corresponding to the second ECN waterline in the Q table.
10. The apparatus according to any of claims 7-9, wherein the specified aggregated statistics comprise: link bandwidth, Incast, queue length, egress port traffic, and the number of large bandwidth data streams.
11. The apparatus of claim 10,
the determining unit is specifically configured to determine, according to the queue length and the egress port flow in the specified aggregation statistical information acquired after the ECN waterline is issued, a reward value corresponding to the issued ECN waterline.
12. The apparatus of claim 11,
the determining unit is specifically configured to determine, by the following formula, an award value r corresponding to the issued ECN waterline:
r＝ω 1 ×T(R)-ω 2 ×D(L)
wherein, t (r) is the output port flow in the specified aggregation statistical information obtained after the ECN waterline is issued, d (l) is the queue length, ω, in the specified aggregation statistical information obtained after the ECN waterline is issued 1 +ω 2 1, and ω 1 、ω 2 Are all positive numbers.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210749818.8A CN115102905A (en) | 2022-06-28 | 2022-06-28 | ECN (engineering-centric networking) waterline adjusting method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210749818.8A CN115102905A (en) | 2022-06-28 | 2022-06-28 | ECN (engineering-centric networking) waterline adjusting method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115102905A true CN115102905A (en) | 2022-09-23 |
Family
ID=83293934
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210749818.8A Pending CN115102905A (en) | 2022-06-28 | 2022-06-28 | ECN (engineering-centric networking) waterline adjusting method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115102905A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115801708A (en) * | 2023-02-03 | 2023-03-14 | 中国科学技术大学 | ECN (engineering-simulation-network) waterline self-adaptive configuration method based on reinforcement learning |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013026749A (en) * | 2011-07-19 | 2013-02-04 | Nippon Telegr & Teleph Corp <Ntt> | Communication device, network management method and program |
| CN113328953A (en) * | 2020-02-28 | 2021-08-31 | 华为技术有限公司 | Method, device and storage medium for network congestion adjustment |
| CN114513408A (en) * | 2022-01-05 | 2022-05-17 | 新华三技术有限公司 | ECN (electrically controlled network) threshold configuration method and device |
-
2022
- 2022-06-28 CN CN202210749818.8A patent/CN115102905A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013026749A (en) * | 2011-07-19 | 2013-02-04 | Nippon Telegr & Teleph Corp <Ntt> | Communication device, network management method and program |
| CN113328953A (en) * | 2020-02-28 | 2021-08-31 | 华为技术有限公司 | Method, device and storage medium for network congestion adjustment |
| CN114513408A (en) * | 2022-01-05 | 2022-05-17 | 新华三技术有限公司 | ECN (electrically controlled network) threshold configuration method and device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115801708A (en) * | 2023-02-03 | 2023-03-14 | 中国科学技术大学 | ECN (engineering-simulation-network) waterline self-adaptive configuration method based on reinforcement learning |
| CN115801708B (en) * | 2023-02-03 | 2023-04-21 | 中国科学技术大学 | ECN waterline self-adaptive configuration method based on reinforcement learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112532409B (en) | Network parameter configuration method, device, computer equipment and storage medium | |
| US20160057065A1 (en) | Re-marking of packets for queue control | |
| CN112422443B (en) | Adaptive control method, storage medium, equipment and system of congestion algorithm | |
| Chen et al. | Minimizing age-of-information for fog computing-supported vehicular networks with deep Q-learning | |
| CN113141314B (en) | Congestion control method and equipment | |
| US20220200906A1 (en) | Network Congestion Handling Method, Model Update Method, and Related Apparatus | |
| CN111211988B (en) | Data transmission method and system for distributed machine learning | |
| CN112104562A (en) | Congestion control method and device, communication network and computer storage medium | |
| WO2024012065A1 (en) | Data transmission control method and apparatus, computer-readable storage medium, computer device, and computer program product | |
| Mišic et al. | Architecture for IoT domain with CoAP observe feature | |
| CN115102905A (en) | ECN (engineering-centric networking) waterline adjusting method and device | |
| CN116489104A (en) | Traffic scheduling method and system based on dynamic priority | |
| CN113572655A (en) | Congestion detection method and system for loss-free network | |
| CN113890854A (en) | Data center network transmission method based on deep reinforcement learning | |
| CN111901237B (en) | Source routing method and system, related device and computer readable storage medium | |
| Bhattacharyya et al. | QFlow: A learning approach to high QoE video streaming at the wireless edge | |
| CN115037672B (en) | Multipath congestion control method and device | |
| Ma et al. | An INT-based TCP window modulator for congestion control in data center networks | |
| Liu et al. | DECC: Achieving low latency in data center networks with deep reinforcement learning | |
| Wu et al. | A multipath resource updating approach for distributed controllers in software-defined network | |
| CN114584494B (en) | Method for measuring actual available bandwidth in edge cloud network | |
| Gong et al. | Nuwa-RL: A Reinforcement Learning based Receiver-side Congestion Control Algorithm to Meet Applications Demands over Dynamic Wireless Networks | |
| CN112822230B (en) | Method and system for setting initial rate of sending end based on probability | |
| Deng et al. | Finishing the tiny flows quickly for common data centre services | |
| CN118118421A (en) | Remote direct memory access network congestion identification and control method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |