CN113300971B

CN113300971B - Data processing system and method

Info

Publication number: CN113300971B
Application number: CN202110162707.2A
Authority: CN
Inventors: 高翼枭; 李强; 邱昊楠
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Filing date: 2021-02-05
Publication date: 2024-07-02
Anticipated expiration: 2041-02-05

Abstract

The present specification provides data processing systems and methods, wherein the data processing systems include: the computing module, the determining module and the configuration module; the computing module is configured to acquire a creation request uploaded by a user, and acquire first time information of a transmitting end and second time information of a receiving end according to the creation request; calculating network round trip time according to the first time information and the second time information, and sending the network round trip time to the user; the determining module is configured to receive a congestion control algorithm created by the user based on the network round trip time and send a congestion control signal to the configuring module according to the congestion control algorithm; the configuration module is configured to generate a configuration request based on the congestion control signal and send the configuration request to the sending end; the configuration request is used for configuring the congestion control algorithm at a transmission module of the sending end.

Description

Data processing system and method

Technical Field

The present disclosure relates to the field of computer technology, and in particular, to a data processing system and method.

Background

With the development of internet technology, RDMA (Remote Direct Memory Access, remote direct data access) has been widely used as a new generation high-performance network protocol for a data center among various service parties, while congestion control is used as a control node for network traffic, and is crucial to the transmission performance of the whole network in an unreliable IP network; the current RDMA congestion control is realized based on a packet loss/RTT (Round-Trip Time) mode, and most of adopted control algorithms are DCQCN (DATA CENTER Quantized Congestion Notification) as congestion control algorithms, but the algorithm parameters are more, the principle is complex, deployment and debugging are inconvenient, and the space available for users to select is small, so that an effective scheme is needed to solve the problems.

Disclosure of Invention

In view of this, the present description embodiments provide a data processing system. The present specification relates to two data processing methods, a data processing apparatus, a computing device, and a computer-readable storage medium, to solve the technical drawbacks of the prior art.

According to a first aspect of embodiments of the present specification, there is provided a data processing system comprising:

The computing module, the determining module and the configuration module;

The computing module is configured to acquire a creation request uploaded by a user, and acquire first time information of a transmitting end and second time information of a receiving end according to the creation request; calculating network round trip time according to the first time information and the second time information, and sending the network round trip time to the user;

the determining module is configured to receive a congestion control algorithm created by the user based on the network round trip time and send a congestion control signal to the configuring module according to the congestion control algorithm;

the configuration module is configured to generate a configuration request based on the congestion control signal and send the configuration request to the sending end; the configuration request is used for configuring the congestion control algorithm at a transmission module of the sending end.

Optionally, the computing module is further configured to:

Analyzing the first time information to obtain the sending time and the response time of the sending end; analyzing the second time information to obtain the receiving time and the triggering time of the receiving end; and calculating the network round trip time according to the sending time, the response time, the receiving time and the triggering time.

Optionally, the computing module is further configured to:

Calculating the difference between the response time and the sending time to obtain a sending end time difference; calculating the difference between the trigger time and the receiving time to obtain a receiving end time difference; and calculating the network round trip time according to the sending end time difference and the receiving end time difference.

Optionally, the determining module is further configured to:

Transmitting a congestion control window or a congestion control rate to the configuration module according to the congestion control algorithm;

Accordingly, the configuration module is further configured to:

And generating the configuration request based on the congestion control window or the congestion control rate, and sending the configuration request to the sending end.

Optionally, the method further comprises:

and the receiving information module is configured to receive retransmission adjustment information uploaded by the user, generate a retransmission adjustment signal according to the retransmission adjustment information and send the retransmission adjustment signal to the configuration module.

Optionally, the configuration module is further configured to:

generating a target configuration request according to the congestion control signal and the retransmission adjustment signal; the target configuration request is sent to the sending end; the target configuration request is used for configuring a congestion control algorithm containing the retransmission adjustment information at a transmission module of the sending end.

Optionally, the configuration module is further configured to:

Determining a configuration code corresponding to the congestion control algorithm, and packaging the configuration code to obtain a congestion control packet; and generating the configuration request based on the congestion control packet, and sending the configuration request to the sending end.

According to a second aspect of embodiments of the present specification, there is provided a data processing method comprising:

the method comprises the steps that a calculation module obtains a creation request uploaded by a user, obtains first time information of a sending end and second time information of a receiving end according to the creation request, calculates network round trip time according to the first time information and the second time information, and sends the network round trip time to the user;

A determining module receives a congestion control algorithm created by the user based on the network round trip time and sends a congestion control signal to the configuration module according to the congestion control algorithm;

The configuration module generates a configuration request based on the congestion control signal and sends the configuration request to the sending end; the configuration request is used for configuring the congestion control algorithm at a transmission module of the sending end.

According to a third aspect of embodiments of the present specification, there is provided another data processing method comprising:

Acquiring a creation request uploaded by a user, and acquiring first time information of a transmitting end and second time information of a receiving end according to the creation request;

calculating network round trip time according to the first time information and the second time information, and sending the network round trip time to the user;

receiving a congestion control algorithm created by the user based on the network round trip time, and creating a congestion control signal according to the congestion control algorithm;

and generating a configuration request based on the congestion control signal and sending the configuration request to the sending end, wherein the configuration request is used for configuring the congestion control algorithm at a transmission module of the sending end.

Optionally, the calculating the network round trip time according to the first time information and the second time information includes:

Analyzing the first time information to obtain the sending time and the response time of the sending end, and analyzing the second time information to obtain the receiving time and the triggering time of the receiving end;

and calculating the network round trip time according to the sending time, the response time, the receiving time and the triggering time.

Optionally, the calculating the network round trip time according to the sending time, the response time, the receiving time and the triggering time includes:

Calculating the difference between the response time and the sending time to obtain a sending end time difference, and calculating the difference between the triggering time and the receiving time to obtain a receiving end time difference;

And calculating the network round trip time according to the sending end time difference and the receiving end time difference.

Optionally, the creating a congestion control signal according to the congestion control algorithm includes:

Creating a congestion control window or a congestion control rate according to the congestion control algorithm;

correspondingly, the generating a configuration request based on the congestion control signal and sending the configuration request to the sending end includes:

Optionally, the method further comprises:

and receiving retransmission adjustment information uploaded by the user, and generating a retransmission adjustment signal according to the retransmission adjustment information.

Optionally, the method further comprises:

Generating a target configuration request according to the congestion control signal and the retransmission adjustment signal;

The target configuration request is sent to the sending end; the target configuration request is used for configuring a congestion control algorithm containing the retransmission adjustment information at a transmission module of the sending end.

Optionally, the generating a configuration request based on the congestion control algorithm and sending the configuration request to the sender includes:

Determining a configuration code corresponding to the congestion control algorithm, and packaging the configuration code to obtain a congestion control packet;

And generating the configuration request based on the congestion control packet, and sending the configuration request to the sending end.

According to a fourth aspect of embodiments of the present specification, there is provided a data processing apparatus comprising:

The acquisition unit is configured to acquire a creation request uploaded by a user, and acquire first time information of a transmitting end and second time information of a receiving end according to the creation request;

A calculation unit configured to calculate a network round trip time from the first time information and the second time information, and to transmit the network round trip time to the user;

a receiving unit configured to receive a congestion control algorithm created by the user based on the network round trip time, and to send a congestion control signal to the configuration module according to the congestion control algorithm;

A transmitting unit configured to generate a configuration request based on the congestion control signal and transmit the configuration request to the transmitting end; the configuration request is used for configuring the congestion control algorithm at a transmission module of the sending end.

According to a fifth aspect of embodiments of the present specification, there is provided a computing device comprising:

A memory and a processor;

the memory is for storing computer-executable instructions, and the processor is for executing the computer-executable instructions:

According to a sixth aspect of embodiments of the present description, there is provided a computer readable storage medium storing computer executable instructions which, when executed by a processor, implement the steps of the data processing method.

The data processing system provided by the embodiment comprises a calculation module, a determination module and a configuration module, wherein the calculation module is used for calculating the network round trip time between a current receiving end and a sending end according to a user creation request, then the determination module sends a signal fed back by the congestion control algorithm to the configuration module according to a congestion control algorithm completed by user input, and the configuration module receives the signal fed back by the determination module and configures the congestion control algorithm according to the signal, so that congestion management and control can be performed according to the user requirement when the sending end performs network transmission with the receiving end, the functions of congestion window, network transmission rate and the like can be adjusted according to the user requirement, transmission effectiveness can be ensured, and the user requirement is met.

Drawings

FIG. 1 is a schematic diagram of a data processing system according to one embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a data processing system according to one embodiment of the present disclosure;

FIG. 3 is a schematic diagram of network round trip time measurement in a data processing system according to one embodiment of the present disclosure;

FIG. 4 is a schematic diagram of another data processing system according to one embodiment of the present disclosure;

FIG. 5 is a process flow diagram of a data processing method according to an embodiment of the present disclosure;

FIG. 6 is a process flow diagram of another data processing method according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a data processing apparatus according to an embodiment of the present disclosure;

FIG. 8 is a block diagram of a computing device according to one embodiment of the present disclosure.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many other forms than described herein and similarly generalized by those skilled in the art to whom this disclosure pertains without departing from the spirit of the disclosure and, therefore, this disclosure is not limited by the specific implementations disclosed below.

The terminology used in the one or more embodiments of the specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of the specification. As used in this specification, one or more embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any or all possible combinations of one or more of the associated listed items.

It should be understood that, although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first may also be referred to as a second, and similarly, a second may also be referred to as a first, without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "in response to a determination" depending on the context.

First, terms related to one or more embodiments of the present specification will be explained.

RDMA: (Remote Direct Memory Access, remote direct data access) to account for server-side data processing delays in network transmissions; RDMA transfers data directly into a storage area of a computer through a network, and moves the data from a system to a remote system memory quickly without any impact on an operating system, so that excessive processing functions of the computer do not need to be called; RDMA supports three queues, a Send Queue (SQ) and a Receive Queue (RQ), a Completion Queue (CQ). Wherein the transmit queue and the receive queue are typically created in pairs, referred to as Queue Pairs (QP).

RC (Reliable Connection ): based on the connection-oriented QP transport service, one QP is associated with another QP, and information transport between the two is reliable, embodying correctness and sequence of data.

DCQCN (DATA CENTER Quantized Congestion Notification, congestion control algorithm): the data center quantifies congestion notification, which is a congestion control algorithm, and uses a switch to mark a data packet with ECN to generate a CNP message to notify a sender of network congestion state.

RTT (Round-Trip Time, network Round Trip Time): it is an important performance indicator in computer networks, representing the total time delay that has elapsed from the time the sender sends data to the time the sender receives an acknowledgement from the receiver (the receiver sends an acknowledgement immediately after receiving the data).

In the present specification, a data processing system is provided, and the present specification relates to two data processing methods, a data processing apparatus, a computing device, and a computer-readable storage medium, one by one, in the following embodiments.

In practical application, congestion control is realized through the hardware of DCQCN algorithm, which is a mode widely applied in RDMA, however, because the algorithm has more parameters, the principle is complex, deployment and debugging are inconvenient, and the space available for users to select is small; meanwhile, the algorithm needs to be realized through hardware, so that a user cannot adjust the congestion control algorithm, namely cannot adapt to different application scenes, and the problem is obviously important to solve.

FIG. 1 illustrates a schematic diagram of a data processing system 100 according to one embodiment of the present disclosure, the data processing system 100 including a computing module 110, a determining module 120, and a configuration module 130;

the computing module 110 is configured to obtain a creation request uploaded by a user, and obtain first time information of a sending end and second time information of a receiving end according to the creation request; calculating network round trip time according to the first time information and the second time information, and sending the network round trip time to the user;

the determining module 120 is configured to receive a congestion control algorithm created by the user based on the network round trip time, and send a congestion control signal to the configuring module 130 according to the congestion control algorithm;

the configuration module 130 is configured to generate a configuration request based on the congestion control signal, and send the configuration request to the sender; the configuration request is used for configuring the congestion control algorithm at a transmission module of the sending end.

The sending end is specifically any server capable of carrying out data transmission for the data center, and the receiving end is correspondingly a server with a network transmission relation with the sending end; it should be noted that, the transmitting end and the receiving end are both provided with a transmission module and a processing module, the transmission module is used for transmitting data packets and requests, the processing module is used for storing data packets and processing requests, the transmission module specifically refers to a network card deployed by the transmitting end and the receiving end, the processing module specifically refers to an application of the receiving end and the transmitting end, that is, the transmission module is a hardware module, and the processing module is a software module.

Based on this, the creation request is specifically a request for acquiring network round trip time before a user creates a customized congestion control algorithm, where the congestion control algorithm specifically refers to that when a sender and a receiver generate network congestion in a network transmission process, current network transmission performance can be adjusted according to congestion degree, the adjustment can be to reduce a sending data packet rate of the sender or reduce a maximum data amount allowed to be transmitted by the sender in a certain time in congestion control, and different congestion scenes have different congestion adjustment requirements, so that in order to provide a congestion control algorithm meeting user use requirements for users, the data processing system provided in this embodiment implements the congestion control algorithm meeting user use requirements according to the user creation request configuration.

Further, the network round trip time specifically refers to the round trip time of interaction in the process of network transmission between the sending end and the receiving end; it should be noted that, in order to enable a user to create a congestion control algorithm that meets the use requirement, the network round trip time is provided to the user when the user writes the congestion processing algorithm, so as to implement more accurate generation of the congestion control algorithm.

In addition, in order to improve the efficiency of writing the congestion control algorithm by the user, a large amount of writing logic can be stored in advance, so that the congestion control algorithm can be automatically generated only by inputting writing requirements by the user; the implementation process may be writing congestion processing policy according to the creation request of the user and the network round trip time, generating congestion control information based on the congestion processing policy, and finally writing the congestion control algorithm according to the congestion control information, where the congestion processing policy specifically refers to congestion processing logic written by the user according to the creation requirement, and the congestion control information specifically refers to the creation request of the user and the network round trip time output by the pointer: how to adjust the congestion state under the condition of network transmission congestion, and realize the related information of the high efficiency of the transmission of a sending end and a receiving end.

Referring to fig. 2, in practical application, the data processing system 100 is configured to support a congestion control algorithm framework cooperated by software and hardware of an RDMA network, where the framework is configured to implement a user-customized congestion control algorithm by cooperation of the hardware and the software; the calculation module 110 in the data processing system 100 corresponds to a congestion control algorithm signal input module in the framework, and is configured to calculate a network round trip time between a sending end and a receiving end according to input of a user before creating a congestion control algorithm, where the module provides an accurate measurement scheme, and can calculate the network round trip time according to timestamps of the sending end and the receiving end, and it should be noted that the module is supported by a software module and a hardware module.

The determining module 120 in the data processing system 100 corresponds to a congestion control algorithm logic module in the framework, and is configured to receive a congestion control algorithm input by a user, generate congestion control information according to the congestion control algorithm, and send the congestion control information to the configuration module 130 for configuration processing, where it is to be noted that, in order to facilitate adjustment and use by the user, the portion may mask an irrelevant detail portion, where the congestion control signal specifically refers to a congestion control window or a congestion control rate generated according to a user-defined congestion control algorithm, so that the configuration module is implemented to directly configure the congestion control window or the congestion control rate, and implement management and control of network transmission.

The configuration module 130 in the data processing system 100 corresponds to a congestion control algorithm deployment module in the framework, and is configured to complete configuration processing according to the congestion control signal uploaded by the determination module, and in order to ensure that the congestion control algorithm can be implemented, the configuration module is implemented by software Pacing, and the window implements the function of hardware Pacing; i.e. the module is mainly an application of congestion control algorithm logic in connection with implementation codes.

In a specific implementation, in order to enable the configuration module 130 to configure a congestion control mechanism meeting the user requirement, the determining module 120 may generate a congestion control signal according to the congestion control algorithm and send the congestion control signal to the configuration module 130, where the congestion control signal may be a congestion control window or a congestion control rate, that is, the determining module 120 may generate the congestion control window or the congestion control rate according to the congestion control algorithm that is input in advance by the user and send the congestion control window or the congestion control rate to the configuration module 130, and after receiving the congestion control signal, the configuration module 130 configures the congestion control signal according to the signal, that is, generates the configuration request based on the congestion control window or the congestion control rate and sends the configuration request to the sending end, so as to implement configuring the congestion control window or the congestion control rate meeting the user requirement at the client.

The congestion control return window size based on the window is applied to network transmission, namely user transmission information in RDMA is divided into smaller chunk (block) granularity, and a congestion control algorithm written by a user is deployed based on the chunk granularity; the congestion control algorithm based on the window can block network transmission according to the window returned by the user and the window which is sent but not confirmed in size, so that the congestion control effect is achieved, and when the window is not fully utilized and the transmission chunk size is larger than the window, network transmission is allowed, otherwise, network transmission is stopped; in order to improve the network transmission efficiency, the window of unacknowledged data size is updated each time the data acknowledgement time occurs and the congestion signal is updated. And the rate-based congestion control is to schedule the chunk Pacing by utilizing Corousel technology modules, or limit the speed of RDMA QP (traffic flow) by adopting Pacing functions of hardware, thereby achieving the purpose of congestion control.

It should be noted that congestion control is a mechanism for preventing overload of routers and links caused by too many messages entering a network, and correspondingly, congestion control based on a window means that a sending end determines a window value according to network congestion conditions, and when the sending end really determines a sending window, the sending end can take minimum values of a notification window and a congestion window so as to reduce network congestion degree while ensuring transmission efficiency of the sending end and a receiving end; and the main processes of congestion control are slow start, congestion avoidance, fast retransmission and fast recovery.

Further, in the process of calculating the network round trip time of the receiving end and the sending end, since the network round trip time will affect the accuracy of the congestion control algorithm in adjusting the network congestion state, the accuracy of congestion adjustment policy generation can be improved only when the accuracy of network round trip time measurement is ensured, and in this embodiment, the calculation process of the network round trip time is as follows: the computing module 110 is further configured to: analyzing the first time information to obtain the sending time and the response time of the sending end; analyzing the second time information to obtain the receiving time and the triggering time of the receiving end; and calculating the network round trip time according to the sending time, the response time, the receiving time and the triggering time.

Specifically, the first time information specifically refers to related information of the sending end in a time dimension, and the second time information specifically refers to related information of the receiving end in the time dimension; correspondingly, the sending time specifically refers to the time when the sending end sends the data packet to the receiving end, the response time specifically refers to the time when the receiving end receives the data packet and then feeds back a response request to the sending end, and the time when the sending end receives the request is the response time; the receiving time specifically refers to the time when the receiving end receives the data packet after the sending end sends the data packet to the receiving end, and the triggering time specifically refers to the time when the receiving end feeds back a response request to the sending end.

Based on this, in order to accurately measure the network round trip time RTT between the sending end and the receiving end, when the sending end and the receiving end perform network transmission, the sending end will carry the sending time in a sent data packet, the receiving end will immediately generate the receiving time after receiving the data packet, then the receiving end will generate a response request for the data packet, and send the request to the sending end, and at the same time write a trigger time into the response request, and finally the sending end will receive the response request, and at the moment, the sending end immediately generates the response time; when the network transmission time between the sending end and the receiving end needs to be calculated, the network round trip time can be calculated according to the sending time, the response time, the receiving time and the triggering time.

Still further, the computing module 110 is further configured to: calculating the difference between the response time and the sending time to obtain a sending end time difference; calculating the difference between the trigger time and the receiving time to obtain a receiving end time difference; and calculating the network round trip time according to the sending end time difference and the receiving end time difference.

Specifically, the time difference of the sending end specifically refers to the time difference between the time when the sending end sends the data packet and the time when the receiving end receives the response request, and the time difference of the receiving end specifically refers to the time difference between the receiving end receives the data packet and the time when the receiving end feeds back the response request; based on the above, the sending end time difference of the sending end can be obtained by calculating the difference between the response time and the sending time, meanwhile, the receiving end time difference of the receiving end can be obtained by calculating the difference between the trigger time and the receiving time, and finally, the network round trip time can be obtained by calculating the sending end time difference and the receiving end time difference; in practical application, the specific calculation process of the network round trip time between the sending end and the receiving end can be obtained through calculation according to the following formula (1):

RTT＝(T₄-T₁)-(T₃-T₂)(1)

Wherein RTT refers to network round trip time between a transmitting end and a receiving end, T ₄ refers to the response time, T ₁ refers to the transmission time, T ₃ refers to the trigger time, and T ₂ refers to the receiving time; referring to fig. 3, the sending time, the receiving time, the trigger time and the response time are recorded by transmitting a data packet between the sending end and the receiving end, and finally the network round trip time between the sending end and the receiving end can be calculated by the above formula (1).

For example, the receiving end and the sending end are any servers in the data center having a network transmission relationship, and an RDMA connection is to be established between the receiving end and the sending end, and the connection may be a connection-oriented unreliable connection (UC, unreliable Connected) or a data-oriented unreliable connection (UD, unreliable connection); when the network round-trip time is calculated, the send complex of the sender RDMAUC/UD is utilized to carry the data packet transmission and successfully send out the timestamp T ₁ of the sender network card, the receive completion of the receiver RDMA UC/UD is utilized to carry the timestamp T ₂ of the network card successfully received by the data packet, the receiver can send out the ACK (Acknowledge character) for the data packet at the moment and confirm to receive the data packet, meanwhile, the time T ₃ for sending the ACK is added into the request, the sender can receive the ACK, the time for receiving the ACK is T ₄, finally, the network round-trip time between the sender and the receiver can be calculated through T ₄-T₁-T₃+T₂ = RTT, the subsequent deployment of the congestion control algorithm is convenient,

In sum, by adopting a soft-hard combination mode to calculate the network round trip time between the sending end and the receiving end, the accuracy of calculating the network round trip time can be improved, and the congestion control algorithm of subsequent deployment can be effectively improved to adapt to the demands of users, so that the use demands of the users are further met.

In addition, after the network round trip time between the sending end and the receiving end is accurately calculated by the calculation module 110, if the congestion control algorithm needs to be written automatically according to the creation requirement of the user, the congestion processing policy may be written according to the network round trip time and the creation requirement of the user, and in this process, different users may have different requirements, so as to meet the use requirement of different users, the congestion control information may be generated according to the adjustment information in the creation requirement, specifically means that: determining adjustment information of a user according to the creation request, and compiling congestion processing strategies based on the adjustment information and network round trip time; and generating window information and transmission information according to the congestion processing strategy, and forming the congestion control information based on the window information and the transmission information.

Specifically, the adjustment information specifically refers to requirements of a user for customizing a congestion control algorithm, the congestion processing policy specifically refers to processing logic for compiling the congestion control algorithm meeting the requirements of the user, the window information specifically refers to information related to user adjustment of congestion windows, notification windows, effective windows and the like, and the transmission information specifically refers to information related to user adjustment of transmission rate of a sent data packet.

Wherein, the congestion window (cwnd) is used for limiting the maximum data amount allowed to be transmitted by the sending end in a certain time in congestion control, and is the flow control from the sending end; the notification window (awnd) is used for controlling the flow of the receiving end according to the maximum receiving rate notified by the receiving end to the transmitting end in the connection establishment and transmission process; the effective window (win) is used to represent the actual window size of the sending end packet, and is defined as win=min (cwnd/awnd).

Based on the above, after the network round trip time between the sending end and the receiving end is calculated, the adjustment information of the user-created congestion control algorithm is determined according to the user creation request, the congestion processing strategy is compiled based on the adjustment information and the network round trip time, window information and transmission information are generated according to the congestion processing strategy, and finally the congestion control information can be generated based on the transmission information and the window information, so that the setting of a relevant window and the setting of the transmission rate according to the user use requirement are realized, and the size foundation of the congestion control algorithm meeting the user use requirement is generated for the follow-up.

For example, the user's creation request is to control the congestion window cwnd to slowly increase, i.e. to determine the adjustment information as: adding 2 to the congestion window cwnd of the sending end every round trip time RTT so that the congestion window slowly grows according to a linear rule, and reducing the transmission rate when congestion is met; based on this, at this time, processing logic is written according to the adjustment information and the network round trip time RTT, and the congestion control algorithm logic module outputs congestion window information and transmission rate information, and finally, congestion control information can be formed according to the congestion window information and the transmission rate information, so as to be used for generating a congestion control algorithm according with the user customization requirement according to the information.

In sum, by determining that the module automatically completes the writing of the congestion processing strategy, not only can the time for writing congestion control information be saved, but also irrelevant writing details can be shielded, and the deployment efficiency of deploying the congestion control algorithm for users is effectively improved.

In addition, since the sending end and the receiving end easily generate packet loss due to some factors when performing network transmission, in order to avoid the problem of incomplete packet transmission caused by packet loss, a retransmission mechanism is set, that is, a packet loss event may be retransmitted for a packet loss or all packets corresponding to a transmission request, so as to ensure the integrity of packet transmission between the sending end and the receiving end, and when a congestion control algorithm is deployed, in order to satisfy the situation, a user may also perform configuration of the retransmission mechanism when configuring the congestion control algorithm according to the requirement, in this embodiment, referring to fig. 4, the data transmission system 100 further includes a receiving information module 140, where the receiving information module 140 is configured to receive retransmission adjustment information uploaded by the user, generate a retransmission adjustment signal according to the retransmission adjustment information, and send the retransmission adjustment signal to the configuration module 130.

Specifically, the retransmission adjustment information specifically refers to information how to perform transmission adjustment for the retransmitted data packet under the condition of packet loss; when the user creates the retransmission adjustment information, in order to improve the creation efficiency of the user, a retransmission processing policy may be provided for prompting the user, so as to implement auxiliary acceleration of processing of configuring the congestion control algorithm, where the retransmission processing policy specifically refers to a policy related to how to retransmit the data packet under the condition of packet loss.

Based on this, because the probability of packet loss in the network transmission process is higher, in order to avoid the integrity of the transmitted data packet, retransmission adjustment information uploaded by the user again can be received, and then the retransmission adjustment information number is generated according to the retransmission adjustment information and sent to the configuration module 130, so that when the configuration module configures the congestion control algorithm according to the use requirement of the user, the configuration module can deploy not only the corresponding adjustment algorithm when the network is congested, but also the corresponding adjustment strategy of retransmitting the data packet when the packet loss occurs, thereby improving the experience effect of the user.

Further, in the case of obtaining congestion control information and retransmission adjustment information, the configuration module 130 will configure a target congestion control algorithm that meets the user's use requirement, and in this embodiment, the configuration module 130 is further configured to: generating a target configuration request according to the congestion control signal and the retransmission adjustment signal; the target configuration request is sent to the sending end; the target configuration request is used for configuring a congestion control algorithm containing the retransmission adjustment information at a transmission module of the sending end.

Specifically, the target configuration request specifically refers to a target congestion control algorithm with retransmission and congestion functions, the target congestion control algorithm specifically refers to an algorithm integrating a congestion control algorithm and a retransmission adjustment algorithm (an algorithm corresponding to retransmission adjustment information), the algorithm is adapted to a creation request of a user, and the implementation can be realized by deploying the target congestion control algorithm: when network transmission congestion occurs at a sending end and a receiving end, congestion windows can be adjusted according to the requirements of users when the algorithm is created; when the transmitting end and the receiving end lose packets, the data packet can be retransmitted according to the requirement of a user when the algorithm is created; correspondingly, the target configuration request specifically refers to a request for configuring the target congestion control algorithm by a transmission module at the transmitting end.

Based on this, in the case that the determining module 120 sends the congestion control signal and the retransmission adjustment signal to the configuring module 130, the configuring module 130 generates the target congestion control algorithm corresponding to the creation request according to the congestion control signal and the retransmission adjustment signal (combines the retransmission adjustment signal and the congestion control signal), and generates the target configuration request based on the target congestion control algorithm, so as to implement that the congestion control algorithm including the retransmission adjustment information can be deployed at the sending end, when network transmission congestion occurs between the sending end and the receiving end, the congestion window, the notification window and/or the effective window can be adjusted according to the algorithm, and when packet loss occurs between the sending end and the receiving end, the best-fit retransmission packet mode can be selected according to the algorithm for retransmission, so that the execution logic of the currently deployed target congestion control algorithm is deployed according to the requirement of the user, so as to adapt to the scene meeting the use requirement of the user, and improve the experience of the user.

In practical application, in order to fully utilize network transmission resources in the process of generating the retransmission adjustment information, it may be configured that a data packet with a smaller occupied buffer is directly sent by RDMA send, a data packet with a larger occupied buffer is sent by protocol encapsulation + RDMA WRITE, and the size of the data packet can be flexibly allocated according to the requirement of a user when judging, and the implementation is not limited in any way.

In summary, by combining the retransmission adjustment information when the target congestion control algorithm is deployed, not only can congestion events between a sending end and a receiving end be handled, but also retransmission can be performed when packet loss occurs between the sending end and the receiving end, thereby further meeting the deployment requirement of the congestion control algorithm of a user and improving the use experience of the user.

Further, on the basis that the determining module 120 generates the congestion control information for the user's creation request, the configuring module 130 generates and deploys a congestion control algorithm that meets the user's use requirement according to the congestion control information, where in this embodiment, the configuring module 130 is further configured to: determining a configuration code corresponding to the congestion control algorithm, and packaging the configuration code to obtain a congestion control packet; and generating the configuration request based on the congestion control packet, and sending the configuration request to the sending end.

Specifically, the configuration code is a bottom layer code corresponding to a congestion control algorithm, the congestion control packet is a data packet for deploying the congestion control algorithm, based on this, the configuration module 130 generates the congestion control algorithm according to the congestion control information after receiving the congestion control information, and in order to deploy the congestion control algorithm, determines the configuration code corresponding to the congestion control algorithm, encapsulates the configuration code to obtain the congestion control packet corresponding to the congestion control algorithm, and finally generates a configuration request based on the congestion control packet and sends the configuration request to the sender.

In practical applications, the configuration module 130 may use congestion control based on a window, and the size of the return window may be applied to the throttle, and may also use congestion control based on a rate, and the size of the return rate may be required, which needs to be matched with a paging module in this process, and the paging function may be supported by using a software Carousel Pacing system or using Mellanox CX hardware based on the paging module. In addition, the configuration module 130 may employ HPCC (High-PerformanceComputingCluster ) congestion control for deployment of congestion control algorithms.

Corresponding to the system embodiment, the present disclosure further provides an embodiment of a data processing method, and fig. 5 shows a process flow chart of the data processing method according to an embodiment of the present disclosure. As shown in fig. 5, the method specifically comprises the following steps:

step S502, a calculation module obtains a creation request uploaded by a user, obtains first time information of a sending end and second time information of a receiving end according to the creation request, calculates network round trip time according to the first time information and the second time information, and sends the network round trip time to the user;

Step S504, a determining module receives a congestion control algorithm created by the user based on the network round trip time and sends a congestion control signal to the configuration module according to the congestion control algorithm;

Step S506, a configuration module generates a configuration request based on the congestion control signal and sends the configuration request to the sending end; the configuration request is used for configuring the congestion control algorithm at a transmission module of the sending end.

In an optional embodiment, the computing module analyzes the first time information to obtain a sending time and a response time of the sending end; analyzing the second time information to obtain the receiving time and the triggering time of the receiving end; and calculating the network round trip time according to the sending time, the response time, the receiving time and the triggering time.

In an alternative embodiment, the calculating module calculates a difference between the response time and the transmission time to obtain a time difference of the transmission end; calculating the difference between the trigger time and the receiving time to obtain a receiving end time difference; and calculating the network round trip time according to the sending end time difference and the receiving end time difference.

In an alternative embodiment, the determining module sends a congestion control window or a congestion control rate to the configuring module according to the congestion control algorithm;

correspondingly, the configuration module generates the configuration request based on the congestion control window or the congestion control rate and sends the configuration request to the sending end.

In an alternative embodiment, the data processing method further includes:

And receiving retransmission adjustment information uploaded by the user, generating a retransmission adjustment signal according to the retransmission adjustment information, and sending the retransmission adjustment signal to the configuration module.

In an alternative embodiment, the configuration module generates a target configuration request according to the congestion control signal and the retransmission adjustment signal; the target configuration request is sent to the sending end; the target configuration request is used for configuring a congestion control algorithm containing the retransmission adjustment information at a transmission module of the sending end.

In an optional embodiment, the configuration module determines a configuration code corresponding to the congestion control algorithm, and encapsulates the configuration code to obtain a congestion control packet; and generating the configuration request based on the congestion control packet, and sending the configuration request to the sending end.

In summary, the calculation module is configured to calculate, according to a user's creation request, a network round trip time between a current receiving end and a sending end, and then the determination module sends, according to a congestion control algorithm completed by user input, a signal fed back by the congestion control algorithm to the configuration module, where the configuration module receives the signal fed back by the determination module, and configures the congestion control algorithm according to the signal, so that when the sending end performs network transmission with the receiving end, congestion control can be performed according to a user's requirement, and functions such as a congestion window and a network transmission rate can be adjusted according to a user's requirement, and transmission effectiveness can be guaranteed, so that a user's requirement is met.

The above is an exemplary scheme of a data processing method of the present embodiment. It should be noted that, the technical solution of the data processing method and the technical solution of the data processing system belong to the same conception, and details of the technical solution of the data processing method, which are not described in detail, can be referred to the description of the technical solution of the data processing system.

The present disclosure further provides another embodiment of a data processing method, as shown in fig. 6, and fig. 6 shows a process flow chart of another data processing method provided in an embodiment of the present disclosure, specifically including the following steps:

Step S602, a creation request uploaded by a user is obtained, and first time information of a sending end and second time information of a receiving end are obtained according to the creation request;

step S604, calculating network round trip time according to the first time information and the second time information, and sending the network round trip time to the user;

step S606, a congestion control algorithm created by the user based on the network round trip time is received, and a congestion control signal is created according to the congestion control algorithm;

Step S608, generating a configuration request based on the congestion control signal, and sending the configuration request to the sender, where the configuration request is used for configuring the congestion control algorithm at a transmission module of the sender.

In an alternative embodiment, the calculating the network round trip time according to the first time information and the second time information includes:

In an alternative embodiment, said calculating said network round trip time from said sending time, said response time, said receiving time and said trigger time comprises:

In an alternative embodiment, said creating a congestion control signal according to said congestion control algorithm comprises:

In an alternative embodiment, the method further comprises:

In an optional embodiment, the generating a configuration request based on the congestion control algorithm and sending the configuration request to the sending end includes:

In summary, when the sending end performs network transmission with the receiving end, congestion management and control can be performed according to the demands of users, so that functions such as congestion windows and network transmission rates can be adjusted according to the use demands, transmission effectiveness can be ensured, and the use demands of users are met.

The above is a schematic scheme of another data processing method of the present embodiment. It should be noted that, the technical solution of the data processing method and the technical solution of the data processing system belong to the same conception, and details of the technical solution of the data processing method, which are not described in detail, can be referred to the description of the technical solution of the data processing system.

Corresponding to the method embodiment described above, the present disclosure further provides an embodiment of a data processing apparatus, and fig. 7 shows a schematic structural diagram of a data processing apparatus according to an embodiment of the present disclosure. As shown in fig. 7, the apparatus includes:

An obtaining unit 702, configured to obtain a creation request uploaded by a user, and obtain first time information of a sending end and second time information of a receiving end according to the creation request;

a calculating unit 704 configured to calculate a network round trip time according to the first time information and the second time information, and send the network round trip time to the user;

A receiving unit 706 configured to receive a congestion control algorithm created by the user based on the network round trip time, and to send a congestion control signal to the configuration module according to the congestion control algorithm;

a transmitting unit 708 configured to generate a configuration request based on the congestion control signal and transmit the configuration request to the transmitting end; the configuration request is used for configuring the congestion control algorithm at a transmission module of the sending end.

In an alternative embodiment, the computing unit 704 is further configured to:

Analyzing the first time information to obtain the sending time and the response time of the sending end, and analyzing the second time information to obtain the receiving time and the triggering time of the receiving end; and calculating the network round trip time according to the sending time, the response time, the receiving time and the triggering time.

In an alternative embodiment, the computing unit 704 is further configured to:

Calculating the difference between the response time and the sending time to obtain a sending end time difference, and calculating the difference between the triggering time and the receiving time to obtain a receiving end time difference; and calculating the network round trip time according to the sending end time difference and the receiving end time difference.

In an alternative embodiment, the receiving unit 706 is further configured to:

accordingly, the transmitting unit 708 is further configured to:

In an alternative embodiment, the data processing apparatus further includes:

And the receiving information unit is configured to receive retransmission adjustment information uploaded by the user and generate a retransmission adjustment signal according to the retransmission adjustment information.

In an alternative embodiment, the received information unit is further configured to:

In an alternative embodiment, the transmitting unit 708 is further configured to:

The above is a schematic solution of a data processing apparatus of the present embodiment. It should be noted that, the technical solution of the data processing apparatus and the technical solution of the data processing method belong to the same conception, and details of the technical solution of the data processing apparatus, which are not described in detail, can be referred to the description of the technical solution of the data processing method.

Fig. 8 illustrates a block diagram of a computing device 800 provided in accordance with an embodiment of the present specification. The components of computing device 800 include, but are not limited to, memory 810 and processor 820. Processor 820 is coupled to memory 810 through bus 830 and database 850 is used to hold data.

Computing device 800 also includes access device 840, access device 840 enabling computing device 800 to communicate via one or more networks 860. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. Access device 840 may include one or more of any type of network interface, wired or wireless (e.g., a Network Interface Card (NIC)), such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.

In one embodiment of the present description, the above-described components of computing device 800, as well as other components not shown in FIG. 8, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device illustrated in FIG. 8 is for exemplary purposes only and is not intended to limit the scope of the present description. Those skilled in the art may add or replace other components as desired.

Computing device 800 may be any type of stationary or mobile computing device including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smart phone), wearable computing device (e.g., smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 800 may also be a mobile or stationary server.

Wherein processor 820 is configured to execute computer-executable instructions for:

The foregoing is a schematic illustration of a computing device of this embodiment. It should be noted that, the technical solution of the computing device and the technical solution of the data processing method belong to the same concept, and details of the technical solution of the computing device, which are not described in detail, can be referred to the description of the technical solution of the data processing method.

An embodiment of the present disclosure also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, are configured to:

The above is an exemplary version of a computer-readable storage medium of the present embodiment. It should be noted that, the technical solution of the storage medium and the technical solution of the data processing method belong to the same concept, and details of the technical solution of the storage medium which are not described in detail can be referred to the description of the technical solution of the data processing method.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The computer instructions include computer program code that may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

It should be noted that, for the sake of simplicity of description, the foregoing method embodiments are all expressed as a series of combinations of actions, but it should be understood by those skilled in the art that the present description is not limited by the order of actions described, as some steps may be performed in other order or simultaneously in accordance with the present description. Further, those skilled in the art will appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily all necessary in the specification.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.

The preferred embodiments of the present specification disclosed above are merely used to help clarify the present specification. Alternative embodiments are not intended to be exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the disclosure and the practical application, to thereby enable others skilled in the art to best understand and utilize the disclosure. This specification is to be limited only by the claims and the full scope and equivalents thereof.

Claims

1. A data processing system, comprising:

The computing module, the determining module and the configuration module;

The determining module is configured to receive a congestion control algorithm created by the user based on the network round trip time, and send a congestion control signal to the configuring module according to the congestion control algorithm, wherein the congestion control signal refers to a congestion control window or a congestion control rate generated according to a user-defined congestion control algorithm;

2. The data processing system of claim 1, the computing module further configured to:

3. The data processing system of claim 2, the computing module further configured to:

4. The data processing system of claim 1, the determination module further configured to:

Accordingly, the configuration module is further configured to:

5. The data processing system of claim 1, further comprising:

6. The data processing system of claim 5, the configuration module further configured to:

7. The data processing system of claim 1, the configuration module further configured to:

8. A data processing method, comprising:

the determining module receives a congestion control algorithm created by the user based on the network round trip time and sends a congestion control signal to the configuration module according to the congestion control algorithm, wherein the congestion control signal refers to a congestion control window or a congestion control rate generated according to the user-defined congestion control algorithm;

9. A data processing method, comprising:

receiving a congestion control algorithm created by the user based on the network round trip time, and creating a congestion control signal according to the congestion control algorithm, wherein the congestion control signal refers to a congestion control window or a congestion control rate generated according to a user-defined congestion control algorithm;

10. The data processing method of claim 9, the calculating a network round trip time from the first time information and the second time information, comprising:

11. The data processing method of claim 10, the calculating the network round trip time from the transmit time, the response time, the receive time, and the trigger time, comprising:

12. The data processing method of claim 9, the creating a congestion control signal according to the congestion control algorithm, comprising:

13. The data processing method of claim 9, further comprising:

14. The data processing method of claim 13, further comprising:

15. The data processing method according to claim 9, wherein the generating a configuration request based on the congestion control algorithm and sending the configuration request to the sender includes:

16. A data processing apparatus comprising:

The receiving unit is configured to receive a congestion control algorithm created by the user based on the network round trip time and send a congestion control signal to the configuration module according to the congestion control algorithm, wherein the congestion control signal refers to a congestion control window or a congestion control rate generated according to the user-defined congestion control algorithm;

17. A computing device, comprising:

A memory and a processor;

the memory is configured to store computer-executable instructions and the processor is configured to execute the computer-executable instructions to implement the method of:

18. A computer readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the data processing method of any one of claims 9 to 15.