WO2020048147A1 - 数据传输方法及装置 - Google Patents

数据传输方法及装置 Download PDF

Info

Publication number
WO2020048147A1
WO2020048147A1 PCT/CN2019/086028 CN2019086028W WO2020048147A1 WO 2020048147 A1 WO2020048147 A1 WO 2020048147A1 CN 2019086028 W CN2019086028 W CN 2019086028W WO 2020048147 A1 WO2020048147 A1 WO 2020048147A1
Authority
WO
WIPO (PCT)
Prior art keywords
network
data
sending
rate
tcp
Prior art date
Application number
PCT/CN2019/086028
Other languages
English (en)
French (fr)
Inventor
崔勇
张蕾
王巧灵
Original Assignee
华为技术有限公司
清华大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司, 清华大学 filed Critical 华为技术有限公司
Priority to EP19858201.7A priority Critical patent/EP3780542B1/en
Publication of WO2020048147A1 publication Critical patent/WO2020048147A1/zh
Priority to US17/098,030 priority patent/US11558302B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/19—Flow control; Congestion control at layers above the network layer
    • H04L47/193—Flow control; Congestion control at layers above the network layer at the transport layer, e.g. TCP related
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
    • H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0823—Errors, e.g. transmission errors
    • H04L43/0829—Packet loss
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0852—Delays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0852—Delays
    • H04L43/0864—Round trip delays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876—Network utilisation, e.g. volume of load or congestion level
    • H04L43/0882—Utilisation of link capacity
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876—Network utilisation, e.g. volume of load or congestion level
    • H04L43/0888—Throughput
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876—Network utilisation, e.g. volume of load or congestion level
    • H04L43/0894—Packet rate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/50—Queue scheduling
    • H04L47/54—Loss aware scheduling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/50—Queue scheduling
    • H04L47/62—Queue scheduling characterised by scheduling criteria
    • H04L47/625—Queue scheduling characterised by scheduling criteria for service slots or service orders
    • H04L47/6255—Queue scheduling characterised by scheduling criteria for service slots or service orders queue load conditions, e.g. longest queue first
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • H04L69/163—In-band adaptation of TCP data exchange; In-band control procedures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control

Definitions

  • the embodiments of the present application relate to the field of communication technologies, and in particular, to a data transmission method and device.
  • the University of California, Lauren Berkeley National Laboratory proposed a congestion avoidance mechanism in 1988, which uses a sliding window model for transmission control.
  • the size of the sliding window refers to how much buffer the receiver can use to receive data.
  • the sender can determine the maximum amount of data that can be sent by the size of the sliding window.
  • congestion will occur.
  • the capacity of the network limits the maximum amount of data that the sender can send. Therefore, the sender always maintains two window sizes: the receiver's acknowledgement window and the congestion window, and takes the minimum of the two windows as the number of bytes that can be sent.
  • TCP Transmission Control Protocol
  • the embodiments of the present application provide a data transmission method and device, which can improve data transmission efficiency based on TCP.
  • a data transmission method including: determining a first sending rate according to a network performance target of the first data and a network state of the first transmission control protocol TCP connection; the first TCP connection is used for sending First data; sending the first data according to the first sending rate.
  • the network state of the TCP connection may refer to a current network state corresponding to the TCP connection, or a network state corresponding to the TCP connection when transmitting the first data.
  • the output of decisions depends not only on the network status, but also on the network performance targets, making network congestion control more flexible, sending rate decisions better, and improving TCP-based data transmission efficiency.
  • the network performance target of the first data is determined according to the characteristics of the first data. In this way, different network performance targets are determined according to different characteristics of the first data, and then data transmission control is performed according to the network performance targets, which can be applied to scenarios with different application requirements.
  • the foregoing network performance target includes one or more of an effective throughput target, a delay target, and a packet loss rate target.
  • the network performance target may be a target weight ratio of the effective throughput, the delay, and the packet loss rate.
  • a network performance target can be defined by effective throughput, delay, and packet loss rate, etc., so that the network performance target can meet more requirements.
  • determining the first sending rate according to the network performance target of the first data and the network status of the first TCP connection includes: according to a specified decision period according to the network performance target of the first data and the first TCP connection.
  • the network status determines the first sending rate.
  • the network status of the foregoing first TCP connection is determined according to the acknowledgement character ACK corresponding to the received first TCP connection. In this way, the network status of the TCP connection can be determined with higher accuracy.
  • the foregoing network state includes: a smooth value of queue delay, a rate at which TCP data packets are sent during a decision period, a rate at which TCP data packets are received during a decision period, a smooth value of RTT, and sent TCP
  • the interval of data packets and the interval of sending ACKs; among them, the network status is used to evaluate the delay, the sending interval, the packet loss rate, and the bandwidth.
  • the above parameters are more accurate in evaluating the delay, sending interval, packet loss rate, and bandwidth, and the resulting sending rate is more reasonable, which improves data transmission efficiency.
  • determining the first sending rate according to the network performance target of the first data and the network status of the first TCP connection includes: according to the network performance target of the first data, the network status of the first TCP connection, and The first global measurement value determines a first sending rate; the global measurement value is a measurement value of one or more TCP connections corresponding to the sending end.
  • the one or more TCP connections may include a first TCP connection, and the first global measurement value may be a current global measurement value of the transmitting end, or the transmitting end includes the first data in the transmission through the one or more TCP connections.
  • the corresponding data are global measured values.
  • the embodiment of the present application can further determine the sending rate in combination with the global measurement value sent, so that the determined sending rate is more reasonable, and the entire network resources can be better used.
  • the foregoing global measurement value includes one or more of the following: log (effective throughput of the sending end / maximum effective throughput of the sending end), log (log of one or more TCP connections on the sending end) Minimum RTT / smoothing value of the round-trip time RTT of one or more TCP connections on the sending end), log (packet loss rate on the sending end).
  • log effective throughput of the sending end / maximum effective throughput of the sending end
  • log log of one or more TCP connections on the sending end
  • Minimum RTT / smoothing value of the round-trip time RTT of one or more TCP connections on the sending end log (packet loss rate on the sending end).
  • the method before determining the first sending rate according to the network performance target of the first data and the network status of the first Transmission Control Protocol TCP connection, the method further includes: ACK. Make sure the network status is the initial network status. This determines the network status when the data is first sent.
  • the method before determining the first sending rate according to the network performance target of the first data and the network status of the first Transmission Control Protocol TCP connection, the method further includes: sending the first to the sending end through the first TCP connection. A TCP data packet of data; an ACK sent by the receiving end to confirm that the TCP data packet was received correctly; and determining the network state of the first TCP connection according to the ACK.
  • sending the first data through the first TCP connection according to the first sending rate includes: determining whether the TCP data packet of the first data is completely sent; when the TCP data packet of the first data is not completely sent , Sending the unsent first data TCP data packet through the first TCP connection according to the first sending rate.
  • sending the first data through the first TCP connection according to the first sending rate further includes: calculating the maximum sending amount according to the first sending rate; and sending the first data through the first TCP according to the first sending rate and the maximum sending amount.
  • the connection sends an unsent first TCP data packet.
  • the method further includes: storing a network performance target of the first data, a network state of the first TCP connection, a first global measurement value, and a first sending rate in a database.
  • the rate model based on deep reinforcement learning includes a policy network and a transmission rate decision network; the first transmission rate is determined according to a network performance target of the first data, a network state of the first TCP connection, and a global measurement value Including: taking the network state of the first TCP connection as the input of the policy network, and taking the output of the policy network, the network performance target of the first data, and the first global measurement value as the input of the transmission rate decision network, and outputting the first transmission rate .
  • the first sending rate can be determined through a rate model based on deep reinforcement learning, so as to make better use of historical data and improve communication efficiency.
  • the aforementioned deep reinforcement learning-based rate model further includes a value network; the method further includes: according to the network performance target of the second data stored in the database, the network status of the second TCP connection, and the second global The corresponding relationship between the measured value and the second transmission rate is to determine the loss function value through the value network; update the value network according to the loss function value to determine the updated policy gradient; make decisions on the policy network and the transmission rate according to the updated policy gradient The network is updated. In this way, it is possible to optimize the rate model based on the newly generated data, and further improve the use of historical data, so that the determined transmission rate is more reasonable and the communication efficiency is higher.
  • a data transmission device configured to transmit the first data.
  • the method includes a determining unit configured to determine a first sending rate according to a network performance target of the first data and a network state of the first TCP connection; the first TCP connection is used to send the first data; and the sending unit is configured to send the first data according to the first sending rate. Send the first data.
  • the network performance target of the first data is determined according to the characteristics of the first data.
  • the network performance goals include one or more of an effective throughput goal, a latency goal, and a packet loss rate goal.
  • the network performance target may be a target weight ratio of the effective throughput, the delay, and the packet loss rate.
  • the determining unit is specifically configured to determine the first sending rate according to a specified decision period according to a network performance target of the first data and a network state of the first TCP connection.
  • the network status of the first TCP connection is determined according to the acknowledgement character ACK corresponding to the received first TCP connection.
  • the network status includes: the smooth value of the queue delay, the rate at which TCP packets are sent during the decision period, the rate at which TCP packets are received during the decision period, the smooth value of the round-trip time RTT, and the send The interval of TCP data packets and the interval of sending ACK;
  • the network status is used to evaluate the delay, sending interval, packet loss rate, and bandwidth.
  • the determining unit is specifically configured to determine the first sending rate according to the network performance target of the first data, the network state of the first TCP connection, and the first global measurement value; the global measurement value corresponds to the sending end Measurements of one or more TCP connections.
  • the global measurement value includes one or more of the following: log (effective throughput of the sender / maximum effective throughput of the sender), log (minimum of one or more TCP connections at the sender RTT / smoothing value of the round-trip time of one or more TCP connections on the sending end), log (transmitting packet loss rate).
  • the method further includes: a storage unit, configured to store a network performance target of the first data, a network state of the first TCP connection, a first global measurement value, and a first sending rate in a database.
  • the rate model based on deep reinforcement learning includes a policy network and a sending rate decision network;
  • the determining unit is configured to use the network state of the first TCP connection as an input of the policy network, and use the output of the policy network, the network performance target of the first data, and the global measurement value as the input of the transmission rate decision network, and the output is the first transmission rate.
  • the rate model based on deep reinforcement learning further includes a value network; the device further includes an update unit, which is specifically configured to: according to the network performance target of the second data stored in the database, the second TCP connection The corresponding relationship between the network state, the second global measurement value, and the second sending rate is used to determine the loss function value through the value network; the value network is updated according to the loss function value to determine the updated policy gradient; according to the updated policy gradient pair The policy network and the sending rate decision network are updated.
  • a network device in a third aspect, includes a transceiver, a processor, and a memory; the transceiver is used to send and receive data, the memory is used to store a program; and the processor is used to execute the program stored in the memory to control the network device to execute The first aspect described above or any one of the possible designs of the first aspect.
  • a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the first aspect or any possible design of the first aspect is implemented. The method described.
  • a computer program product including instructions is provided, and when the instructions are run on a computer, the computer is caused to execute the method described in the first aspect or any one of the possible designs of the first aspect.
  • a chip in a sixth aspect, includes a processor and a memory; the memory is configured to store a program; the processor is configured to execute the program stored in the memory to perform the method described in the first aspect or any possible design of the first aspect .
  • the transmission rate is determined by the network performance target and the network status, the determination of the transmission rate is optimized, and the efficiency of data transmission is improved.
  • the network performance target may be determined according to the characteristics of the data to be transmitted. This adapts to different application requirements. Furthermore, historical experience can be used to obtain the appropriate network state and network performance target transmission rate, without going through the slow start phase of traditional TCP congestion control algorithms, improving the effective throughput of the sender, reducing startup time, and increasing throughput rate.
  • FIG. 1 is a schematic diagram of a congestion mechanism
  • FIG. 2 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • FIG. 3 is a schematic flowchart of a data transmission method according to an embodiment of the present application.
  • FIG. 4 is an example of a rate model according to an embodiment of the present application.
  • FIG. 6 is a schematic flowchart of another data transmission method according to an embodiment of the present application.
  • 7a is another example of a rate model according to an embodiment of the present application.
  • FIG. 8 is another example of a rate model according to an embodiment of the present application.
  • FIG. 9 is a schematic flowchart of another data transmission method according to an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a data transmission device according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a network device according to an embodiment of the present application.
  • FIG. 2 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • the scenario includes a transmitting end 100 and a receiving end 200.
  • the sending end 100 sends data packets to the TCP receiving end through M network nodes, where M is an integer greater than or equal to 1.
  • the sending end 100 may be a server or proxy server that provides data to the terminal, such as a data center node or a content delivery network (CDN) node.
  • CDN content delivery network
  • the sending end includes, but is not limited to, a web server and a video server. (video server), instant messaging server (IM server), and game server (game server).
  • the sending end 100 provides the receiving end 200 with data such as web pages, video files, IM communications, and games.
  • the receiving end 200 may be a terminal that provides users with functions such as web browsing, video playback, communication, entertainment, and leisure, such as, but not limited to, mobile phones, computers, and wearable devices.
  • the network node (not shown in the figure) is a data packet forwarding device located between the sending end 100 and the receiving end 200.
  • the network node has an IP address and supports the transmission layer protocol transmission control protocol / internet protocol.
  • Protocol TCP / IP
  • FIG. 3 is a schematic flowchart of a data transmission method according to an embodiment of the present application. As shown in FIG. 3, the method may specifically include the following steps:
  • the sending end determines a first sending rate according to a network performance target of the first data and a network state of the first TCP connection.
  • the network performance target of the first data may be determined according to the characteristics of the first data, and the first TCP connection is used to send the first data.
  • the network state of the TCP connection may refer to a current network state corresponding to the TCP connection, or a network state corresponding to the TCP connection when transmitting the first data.
  • the same sender can establish a TCP connection with one or more receivers, and send data through the TCP connection.
  • the first data may be data requested by the receiving end to the sending end.
  • the correspondence between the characteristics of the data and the network performance target may be defined in advance.
  • the characteristics of the data may include one or more of the characteristics of the application providing the data, the characteristics of the receiving end receiving the data, or the characteristics of the data itself.
  • different applications or different services of the same application have different requirements for network performance.
  • Correspondences between application information such as the type of application or application service, application or application service identification, and network performance goals can be established. For example, for live applications The requirement for network performance is low latency. For cloud disk applications, the requirement for network performance is to maintain high throughput. For video applications, the online watch service requires higher latency than the offline cache service; different receivers The requirements for network performance are also different.
  • the correspondence between the requirements of the receiving end and the network performance goals can be established in advance. For example, if the receiving end has better performance, it can accept higher throughput; the data itself also has different requirements for network performance.
  • the correspondence between the characteristics of the data itself and the network performance goals can be established in advance. For example, data with a relatively large amount of data requires a higher throughput.
  • the sending end may determine the network performance target of the first data in multiple ways.
  • the application server providing the first data may predefine the network performance target of the first data, and send the network performance target of the first data to the sending end.
  • the sending end may determine the network performance target of the first data based on the characteristics of the first data and based on the correspondence between the locally predetermined characteristics and the network performance target.
  • the sending end may receive the network performance target sent by the receiving end.
  • the network performance target can be used to indicate the requirements of one or more of effective throughput, delay, and packet loss rate.
  • the network performance goal may include one or more of an effective throughput goal, a delay goal, and a packet loss rate goal; or, the network performance goal may be an effective throughput goal, a delay goal, and a packet loss rate goal.
  • Ratio or a weighted value of the requirements for effective throughput, delay, and packet loss; or one or more of the effective throughput, delay, and packet loss.
  • the receiving end can send a data request to the transmitting end.
  • the transmitting end After the transmitting end receives the data request from the transmitting end, the transmitting end establishes a TCP connection with the receiving end in order to send over the TCP connection.
  • the first TCP connection may refer to a TCP connection established when the sending end and the receiving end transmit the first data.
  • the network status of the TCP connection is mainly used to evaluate information such as the delay, sending interval, packet loss rate, and bandwidth of the TCP connection.
  • the sending end may determine the first sending rate according to the network performance target of the first data and the network state of the first TCP connection according to a specified decision period.
  • the network state of the TCP connection may refer to the current network in the decision period of the TCP connection status.
  • the network status of the TCP connection can also be used to evaluate delay jitter or bit error rate and so on.
  • the network status of the TCP connection can include the smooth value of the queue delay (for assessing the delay), the rate at which TCP packets are sent during the decision period (for evaluating the sending interval, and the packet loss rate), The rate of TCP packets received during the period (used to evaluate the sending interval and packet loss rate), the smooth value of the round-trip time RTT (used to evaluate the delay), the interval of the TCP packets (used to evaluate the bandwidth), or the ACK Interval (for evaluating bandwidth) and so on.
  • the above network status is merely an example, and the network status of the TCP connection in the embodiment of the present application may further include other network statuses for evaluating any one of the information such as the delay, sending interval, packet loss rate, or bandwidth of the TCP connection.
  • One or more of the other network states may be combined with one or more of the foregoing network states to implement information such as time delay, sending interval, packet loss rate, and bandwidth of the TCP connection.
  • the network status of the TCP connection can be determined based on the ACK sent by the receiving end.
  • the sender records the ACK arrival time, and at the same time extracts the ACK sequence number, ACK confirmation sequence number, sending time, and ACK transmission time from the ACK.
  • the following information is extracted from the ACK with the sequence number: 1, sequence number; The time when the packet was sent, 129; the total number of bytes sent, 1436; the time when the last packet was confirmed to be received, 62; the total number of bytes that were confirmed to be received, 1436; Number of bytes, 1436; ACK transmission time, 192.
  • the network status of the TCP connection corresponding to the ACK can be determined at least in the following manner.
  • the smoothing value of the queue delay can be calculated according to the time when the data packet is sent and the time when the last data packet is received.
  • the rate at which TCP packets are sent during the decision period can be determined based on the total number of bytes sent during the decision period and when the ACK was received.
  • the decision period may refer to the time from the time when the ACK is received to the time after the last decision.
  • the rate at which TCP packets are received during the decision period can be determined based on the total number of bytes that have been acknowledged received during the decision period and when the ACK was sent.
  • the decision period may refer to the time from sending the ACK to the time after the last decision.
  • ⁇ is a smoothing factor, which determines the weight of the old SRTT value, 0 ⁇ ⁇ 1.
  • the RTT can be calculated according to the time when the data packet is sent and the ACK arrival time.
  • the interval at which TCP packets are sent is the interval between the current and previous TCP packets.
  • the interval for sending an ACK is the sending interval for this ACK and the previous ACK.
  • the network status of the TCP connection can be determined to be an initial value, and the initial value is a preset value, for example, the initial value may be 0.
  • congestion control is performed according to the network performance target and the network status, and a first sending rate that meets the network performance target requirements is selected according to the network status to pass the first TCP
  • the connection sends the first data.
  • the sending end may determine the first sending rate in multiple ways.
  • the sending end may determine the rate model in advance, and by using the network performance target of the first data and the network state of the first TCP connection as the input of the predetermined rate model, the output is the first sending rate.
  • the rate model may be a rate model based on deep reinforcement learning (for example, a model based on a deep deterministic policy gradient (DDPG) algorithm) or other machine learning models.
  • DDPG deep deterministic policy gradient
  • the rate model based on deep reinforcement learning includes a policy network 401 and a transmission rate decision network 402; when calculating the first transmission rate, the network state of the first TCP connection may be used as an input to the policy network 401, and Take the output of the policy network 401 and the network performance target of the first data as the input of the transmission rate decision network 402, and the output is the first transmission rate.
  • the network state can be expressed as S (t)
  • the output of the policy network 401 is A (t)
  • the sending rate is a (t)
  • the network performance target is g
  • the policy network 401 can be expressed as: h (S (t)
  • the sending rate decision network 402 can be expressed as: among them Is a set of parameters used to define f, that is, the specific f follows Change.
  • the obtained rate model can be optimized based on data generated during the operation of the sender.
  • the sending end may record the determined network performance target, network status, and sending rate in a database according to the corresponding relationship.
  • the corresponding relationship recorded in the database can be shown in Table 1.
  • Table 1 is only an example. In a specific implementation, the database may include more or less data. The corresponding relationship in Table 1 can also be implemented through multiple tables.
  • the sending end can update the rate model according to the data in the database every time a certain amount of data is added in the database.
  • the embodiment of the present application may further include: the passing value of the correspondence between the network performance target of the second data stored in the database, the network state of the second TCP connection, the second global measurement value, and the second sending rate.
  • the network determines the loss function value; updates the value network according to the loss function value to determine the updated policy gradient; and updates the policy network and the sending rate decision network according to the updated policy gradient.
  • the corresponding relationship between the network performance target of the second data, the network status of the second TCP connection, the second global measurement value, and the second transmission rate may include the network performance target of the first data, and the first TCP connection. Corresponding to the network state, the first global measurement value, and the first sending rate.
  • the DDPG algorithm-based rate model includes a policy network 501, a transmission rate decision network 502, and a value network 503.
  • the policy network 501 and the transmission rate decision network 502 are the same networks as the policy network 401 and the transmission rate decision network 402 shown in FIG. 4.
  • the training process is as follows:
  • parameters of the policy network 501 and the transmission rate decision network 502 are updated.
  • the sending end may determine the first sending rate according to a predetermined network state and a mapping of the network performance target to the sending rate. For example, multiple sets of network status, network performance targets, and sending rate samples can be determined in advance, and different methods can be used to adjust the sending rate for different network states and different network performance targets, and the adjustment method is modified until the objective function reaches the optimal, and finally generates A mapping of network status and network performance goals to sending rates.
  • the sending end sends the first data to the receiving end according to the first sending rate.
  • the sending end may calculate the maximum sending amount of the current sending according to the first sending rate, and send the first data to the receiving end.
  • the sending end judges whether all the TCP data packets corresponding to the first data have been sent. If the sending is not completed, the unsent first data is sent through the first TCP connection according to the first sending rate. TCP packets.
  • the sending end determines whether an ACK of the TCP data packet of the first data is received.
  • the network status of the next data transmission is determined according to the ACK.
  • the sender does not receive the ACK, it is determined whether all the TCP data packets corresponding to the sending window have been sent completely; if not, it continues to send the TCP data packets of the first data that has not been sent. If it is completed, stop sending until the ACK of the TCP data packet of the first data is received.
  • the transmission rate is determined by the network performance target determined according to the characteristics of the data to be transmitted, which improves the ability to respond to different network performance requirements and improves the efficiency of data transmission.
  • the sender uses historical experience to directly obtain the appropriate transmission rate of the appropriate network state and network performance target. It does not go through the slow start phase of the traditional TCP congestion control algorithm, which improves the effective throughput of the sender, reduces startup time, and improves throughput.
  • FIG. 6 is a schematic diagram of a data transmission method according to an embodiment of the present application.
  • the sending end may also determine a sending rate in combination with the global measurement value.
  • the method may specifically include the following steps:
  • the global measurement value is a measurement value of one or more TCP connections corresponding to the sending end.
  • the one or more TCP connections may include a first TCP connection.
  • the first global measurement value may be a current global measurement value of the transmitting end, or a global measurement value corresponding to the transmitting end when transmitting data including the first data through one or more TCP connections.
  • the first global measurement value may be a global measurement value of the sending end in the decision period.
  • the network performance target of the first data, the first TCP connection, and the network status of the first TCP connection refer to the description in the foregoing S310, and details are not described herein again.
  • the global measurement value can be used to evaluate all TCP connections on the sending end.
  • the global measurement value includes one or more of the following: log (the ratio of the effective throughput of the sender and the maximum effective throughput of the sender), log (the minimum RTT of one or more TCP connections on the sender and one or more of the sender's The ratio of the smoothed values of the RTTs of multiple TCP connections), log (the packet loss rate at the sending end).
  • the global measurement value can be determined according to the ACK of the global TCP connection of the sending end. For example, the sender records the ACK arrival time, and at the same time extracts the ACK sequence number, ACK confirmation sequence number, the time sent by the sender, and the time the ACK was sent from the ACK. The sender records each ACK as described above. The sender can The ACK of each TCP connection determines the global measurement value.
  • Determining the first sending rate according to the network performance target of the first data, the network state of the first TCP connection, and the global measurement value may include multiple methods, which are specifically described below.
  • the sending end may determine the rate model in advance, and by using the network performance target of the first data, the network state of the first TCP connection, and the global measurement value as the input of the predetermined rate model, the output is the first sending rate.
  • the rate model may be a rate model based on deep reinforcement learning, for example, a rate model based on the DDPG algorithm, or other machine learning models.
  • the congestion control environment includes a sending end such as a sending end 711, a sending end 712, and a sending end 713, and a receiving end such as a receiving end 721, a receiving end 722, and a receiving end 723.
  • the sending end and the receiving end also include other network elements for transmitting data and instructions.
  • the network performance target may be determined according to the application requirements corresponding to the first data.
  • the performance target may be a target weight ratio of an impact factor. Factors can include latency, effective throughput, and packet loss rate.
  • Global measurement values can be obtained based on the measurement values of multiple TCP connections.
  • the multiple TCP connections can include Two TCP connections between the sending end 711 and the receiving end 721 and the sending end 711 and the receiving end 722.
  • the first sending rate may be calculated based on the deep reinforcement learning rate model based on the network state of the first TCP connection, the first network performance target, and the first global measurement value.
  • the receiving end 721 continues to send data through the first TCP connection.
  • the rate model based on deep reinforcement learning may include multiple implementations.
  • the rate model based on deep reinforcement learning may include a policy network 701 and a transmission rate decision network 702; when calculating the first transmission rate, the network state of the first TCP connection may be used as an input to the policy network 701, The output of the policy network 701, the network state of the first TCP connection, and the first global measurement value are used as the input of the transmission rate decision network 702, and the output is the first transmission rate.
  • the network status can be identified as S (t), the output of the policy network 401 is A (t), the sending rate is a (t), the network performance target is g, and the global measurement value is m (t).
  • the sending rate decision network 402 can be expressed as: among them Is a set of parameters used to define f, that is, the specific f follows Change.
  • the obtained rate model can be optimized based on data generated during the operation of the sender.
  • the sending end may store the network performance target of the first data, the network state of the first TCP connection, the first global measurement value, and the first sending rate in a database. Since the sender can provide multiple data services for multiple receivers, the local database of the sender can store data corresponding to multiple TCP connections.
  • the sending end can update the rate model according to the data in the database every time a certain amount of data is added in the database.
  • the rate model based on deep reinforcement learning includes a policy network 801, a transmission rate decision network 802, and a value network 803.
  • the policy network 801 and the transmission rate decision network 802 are the same networks as the policy network 701 and the transmission rate decision network 702 shown in FIG. 7.
  • the training process is as follows:
  • the network measurement value m using the output network Q (at) and the target network output Q (a ') of the value network 803, calculate the loss function (loss) as follows:
  • the value of the policy gradient Q is obtained; according to the value of the policy gradient Q and the policy gradient of the policy network itself, parameters of the policy network 801 and the transmission rate decision network 802 are updated.
  • the policy network 801 or the transmission rate decision network 802 may be composed of parameters of an evaluation network with a parameter ⁇ and a target network with a parameter ⁇ Q.
  • the update of the transmission rate decision network 801 will be described as an example.
  • the transmission rate decision network 801 is composed of an evaluation network “ ⁇ (st
  • the number of samples, the function J is the gradient for ⁇ , and the parameters are updated according to the following formula:
  • the sending end may determine the first sending rate according to a predetermined network state and a mapping of the network performance target to the sending rate. For example, multiple sets of network status, network performance targets, and sending rate samples can be determined in advance, and different methods can be used to adjust the sending rate for different network states and different network performance targets, and the adjustment method is modified until the objective function reaches the optimal, and finally generates A mapping of network status and network performance goals to sending rates.
  • the sending end sends the first data to the receiving end according to the first sending rate.
  • step S620 reference may be made to the related description in the foregoing S320, and details are not described herein again.
  • FIG. 9 is a schematic diagram of a data transmission method according to an embodiment of the present application.
  • the performance goal g of the Web server-side transmission rate is based on the optimization goal, and the optimization goal is to improve the throughput of the Web server. Passed to the rate calculation engine through a socket.
  • the database of the web server stores each network state s, transmission rate a, network measurement value m, and network performance target g of the historical TCP flow.
  • the rate calculation engine is generated according to the multi-objective optimization deep reinforcement learning technology, and determines the sending rate a according to the network state s, the measurement value m, and the network performance target g.
  • the specific calculation of the sending rate a is calculated and output by the rate calculation engine in the deep reinforcement learning network.
  • the rate calculation engine is a multi-target deep reinforcement learning rate model based on the DDPG algorithm.
  • the input of the rate model is the network state s and the current network measurement value m.
  • S903 The Web server determines whether data is sent on the TCP connection. If there is no data to be sent, the transmission ends; if there is data to be sent, the data is sent according to the above-mentioned sending rate a, and the corresponding data is sent by the sending engine to the network via the network interface.
  • the terminal After receiving the data through the TCP connection, the terminal generates an ACK and sends it to the server via the network.
  • the server determines whether the maximum sending amount of the current sending is completed, and if not, continues to send data at the rate.
  • Sequence number (seq_num) 1 The time when the packet was sent (send_ts) 129 Total bytes sent (sent_bytes) 1436 Time when the last packet was acknowledged (delivered_time) 62 Confirmed Total Bytes Received (delivered) 1436 Number of bytes received this time (send_ack_bytes) 1436 ACK sending time (send_ack_time) 192
  • the global measurement value m is calculated and stored.
  • s [[0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.41] [0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0. 0.-1.41] [0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.41] [0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.41] [0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.41] [0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.41] [0.0.0.0.0.0.0.0.
  • max_g_tput is the current maximum effective throughput of the sender
  • FIG. 10 is a schematic structural diagram of a data transmission device according to an embodiment of the present application.
  • the apparatus may be configured to execute the method performed by the transmitting end in the embodiments shown in FIG. 3 or FIG. 6.
  • the device includes:
  • a determining unit 1001 configured to determine a first sending rate according to a network performance target of the first data and a network state of the first TCP connection; the first TCP connection is used to send the first data;
  • the sending unit 1002 is configured to send the first data according to the first sending rate.
  • the network performance target of the first data may be determined according to characteristics of the first data.
  • the determining unit 1001 is specifically configured to:
  • the first sending rate is determined according to the network performance target of the first data and the network status of the first TCP connection according to a specified decision period.
  • the network status of the first TCP connection is determined according to the received acknowledgement character ACK corresponding to the first TCP connection.
  • the network status includes: a smooth value of a queue delay, a rate at which a TCP data packet is sent during a decision period, a rate at which a TCP data packet is received during a decision period, a smooth value of a round-trip time RTT, sending The interval of TCP data packets and the interval of sending ACK;
  • the network state is used to evaluate delay, sending interval, packet loss rate, and bandwidth.
  • the determining unit 1001 is specifically configured to:
  • the first sending rate is determined according to a network performance target of the first data, a network state of the first TCP connection, and a first global measurement value; the global measurement value is a measurement value of one or more TCP connections corresponding to the sending end.
  • the global measurement value includes one or more of the following: log (effective throughput of the sending end / maximum effective throughput of the sending end), log (minimum of one or more TCP connections at the sending end) RTT / smoothing value of the round-trip time of one or more TCP connections on the sending end), log (transmitting packet loss rate).
  • the method further includes:
  • the storage unit 1003 is configured to correspondingly store a network performance target of the first data, a network state of the first TCP connection, the global measurement value, and the first sending rate in a database.
  • the rate model based on deep reinforcement learning includes a policy network and a sending rate decision network;
  • the determining unit 1001 is specifically configured to use the network state of the first TCP connection as an input of the policy network, and use the output of the policy network, the network performance target of the first data, and the global measurement value as The input of the transmission rate decision network, and the output is the first transmission rate.
  • the rate model based on deep reinforcement learning further includes a value network
  • the device further includes an update unit, the update unit is configured to:
  • FIG. 11 is a schematic structural diagram of a network device according to an embodiment of the present invention.
  • the network device 1100 specifically includes a transceiver 1101, a processor 1102, and a memory 1103.
  • the transceiver 1101, the processor 1102, and the memory 1103 may be connected through a bus.
  • the network device may be used to implement the function of the sending end in the embodiment shown in FIG. 3 or FIG. 6.
  • the transceiver 1101 is configured to support transmitting and receiving information between the transmitting end and the receiving end or other network elements in the foregoing embodiment.
  • the transceiver 1101 may include an Ethernet interface.
  • the data and signaling messages are processed by the processor 1102, and sent by the transceiver 1101 to the receiving end.
  • the data and signaling from the receiving end are received via the transceiver 1101 and processed by the processor 1102 to obtain the data and signaling sent by the transmitting end.
  • the processor 1102 may control the sending device 1100 to execute the processing process related to the sending end in the embodiment shown in FIG. 3 or FIG. 6 and / or other processes used for the technology described in this application.
  • the processor 1102 is configured to execute S310 and S610 in the embodiments shown in FIG. 3 and FIG. 6, and the transceiver 1101 is configured to execute S320 and S620 in the embodiments shown in FIG. 3 and FIG. 6.
  • the memory 1103 is used to store program codes and data of the terminal.
  • An embodiment of the present application provides a chip.
  • the chip includes a processor and a memory; the memory is configured to store a program; the processor is configured to execute the program stored in the memory to perform the method described in the foregoing method embodiment.
  • all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof.
  • software When implemented in software, it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions.
  • the computer When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable medium to another computer-readable medium, for example, the computer instructions may be transmitted from a website site, computer, server, or data center through a cable (Such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) for transmission to another website site, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, and the like that includes one or more available medium integration.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), or the like.
  • a magnetic medium for example, a floppy disk, a hard disk, a magnetic tape
  • an optical medium for example, a DVD
  • a semiconductor medium for example, a solid state disk (SSD)

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Environmental & Geological Engineering (AREA)
  • Computer Security & Cryptography (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本申请实施例提供了一种数据传输方法及装置。该方法包括：根据第一数据的网络性能目标和第一输层协议传输控制协议(transmission control protocol，TCP)连接的网络状态确定第一发送速率，第一TCP连接用于发送第一数据；根据第一发送速率发送第一数据。通过设置网络性能目标使得决策的输出不仅取决于网络状态，还取决于于网络性能目标，网络拥塞控制更灵活，提高了基于TCP的数据传输效率。

Description

数据传输方法及装置
本申请要求于2018年09月04日提交中国专利局、申请号为201811026124.1、申请名称为“数据传输方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及通信技术领域，尤其涉及一种数据传输方法及装置。
背景技术
随着互联网用户数量的增加，网络的拥塞问题越来越严重。加州大学劳伦伯克利国家实验室在1988年提出了拥塞避免机制，通过滑动窗口模型来进行传输控制。滑动窗口的大小是指接收方还有多大的缓冲区可以用于接收数据。发送方可以通过滑动窗口的大小来确定能够发送的最大数据量。此外，当加载到网络中的荷载超过了其处理能力时，就会发生拥塞现象，如图1所示，网络的容量限制了发送方能够发送的最大数据量。因此，发送方始终维持两个窗口大小：接收方确认窗口和拥塞窗口，并且取两个窗口的最小值作为可以发送的字节数。
随着互联网的发展，具有高带宽的主干网络、卫星通信网络以及无线网络已成为常见的网络场景。这些网络场景均具有高带宽时延积、网络状态动态多变等特点，对传输控制协议(transmission control protocol，TCP)提出了新的挑战。面对高带宽高时延且动态变化网络场景，传统TCP协议的拥塞控制机制策略固定，仅适应于特定的网络场景或流量分布，无法根据较大的空闲带宽，动态的调整发送窗口大小；此外，衡量网络性能的常用指标，诸如可用带宽、吞吐量、时延、丢包率等根据网络状态动态变化，特别是在较大往返时间(round trip time，RTT)的情况下能够获取的网络状态信息有限，无法及时获得路径的拥塞状态和可用带宽信息，难以精确建模并确定最佳的发送速率。
发明内容
本申请实施例提供了一种数据传输方法及装置，可以提高基于TCP的数据传输效率。
第一方面，提供了一种数据传输方法，其特征在于，包括：根据第一数据的网络性能目标和第一传输控制协议TCP连接的网络状态确定第一发送速率；第一TCP连接用于发送第一数据；根据第一发送速率发送第一数据。
其中，该TCP连接的网络状态可以指该TCP连接对应的当前网络状态，或该TCP连接进行第一数据的传输时对应的网络状态。
通过设置网络性能目标，使得决策的输出不仅取决于网络状态，还取决于于网络性能目标，使得网络拥塞控制更灵活，发送速率的决策更好，提高了基于TCP的数据 传输效率。
在一个可能的设计中，第一数据的网络性能目标根据第一数据的特征确定。以此实现，根据第一数据的不同特征来确定不同的网络性能目标，进而根据网络性能目标进行数据传输控制，能够适用不同应用需求的场景。
在一种可能的设计中，前述网络性能目标包括有效吞吐量目标、时延目标和丢包率目标中的一项或多项。示例性的，网络性能目标可以为有效吞吐量、时延和丢包率三者的目标权重比例。通过本申请实施例，可以通过有效吞吐量、时延和丢包率等来定义网络性能目标，使得网络性能目标能够适用更多的需求。
在另一种可能的设计中，前述根据第一数据的网络性能目标和第一TCP连接的网络状态确定第一发送速率包括：按照指定决策周期根据第一数据的网络性能目标和第一TCP连接的网络状态确定第一发送速率。通过本申请实施例可以实现周期性的更新发送速率，降低对系统性能的要求，提高数据传输的稳定性以及通信效率。
在另一种可能的设计中，前述第一TCP连接的网络状态根据接收到的第一TCP连接对应的确认字符ACK确定。以此可以实现TCP连接的网络状态的确定，准确性更高。
在另一种可能的设计中，前述网络状态包括：队列时延的平滑值、在决策期间内发送TCP数据包的速率、在决策期间内接收TCP数据包的速率、RTT的平滑值、发送TCP数据包的间隔、发送ACK的间隔；其中，网络状态用于评估时延、发送间隔、丢包率以及带宽。通过上述参数对时延、发送间隔、丢包率以及带宽的评估更准确，最终得到的发送速率更合理，提高了数据传输效率。
在另一种可能的设计中，前述根据第一数据的网络性能目标和第一TCP连接的网络状态确定第一发送速率包括：根据第一数据的网络性能目标、第一TCP连接的网络状态和第一全局测量值确定第一发送速率；全局测量值为发送端对应的一个或多个TCP连接的测量值。其中，该是一个或多个TCP连接可以包括第一TCP连接，该第一全局测量值可以为发送端当前的全局测量值，或发送端在通过一个或多个TCP连接传输包括第一数据在内的数据时对应的全局测量值。通过本申请实施例可以进一步结合发送的全局测量值来确定发送速率，使得确定的发送速率更合理，能够更好的利用全网资源。
在另一种可能的设计中，前述全局测量值包括下述一项或多项：log(发送端的有效吞吐量/发送端的最大的有效吞吐量)，log(发送端一个或多个TCP连接的最小RTT/发送端一个或多个TCP连接的往返时间RTT的平滑值)，log(发送端的丢包率)。通过上述参数对全局TCP连接的评估更准确，最终得到的发送速率更合理，提高了数据传输效率。
在另一种可能的设计中，在根据第一数据的网络性能目标和第一传输控制协议TCP连接的网络状态确定第一发送速率之前，还包括：接收端发送的用于确认建立TCP连接的ACK。确定网络状态为初始网络状态。以此在确定首次发送数据时的网络状态。
在另一种可能的设计中，在根据第一数据的网络性能目标和第一传输控制协议TCP连接的网络状态确定第一发送速率之前，还包括：通过第一TCP连接向发送端发送第一数据的TCP数据包；接收端发送的用于确认TCP数据包被正确接收的ACK； 根据ACK确定第一TCP连接的网络状态。
在另一种可能的设计中，前述根据第一发送速率通过第一TCP连接发送第一数据包括：判断第一数据的TCP数据包是否发送完毕；当第一数据的TCP数据包未发送完毕时，根据第一发送速率通过第一TCP连接发送未发送的第一数据TCP数据包。
在另一种可能的设计中，前述根据第一发送速率通过第一TCP连接发送第一数据还包括：根据第一发送速率计算最大发送量；根据第一发送速率和最大发送量通过第一TCP连接发送未发送的第一数据TCP数据包。
在另一种可能的设计中，还包括：将第一数据的网络性能目标、第一TCP连接的网络状态、第一全局测量值以及第一发送速率对应存储在数据库中。
在另一种可能的设计中，基于深度强化学习的速率模型包括策略网络和发送速率决策网络；根据第一数据的网络性能目标、第一TCP连接的网络状态和全局测量值确定第一发送速率包括：将第一TCP连接的网络状态作为策略网络的输入，并将策略网络的输出、第一数据的网络性能目标和第一全局测量值作为发送速率决策网络的输入，输出为第一发送速率。以此可以实现通过基于深度强化学习的速率模型确定第一发送速率，以更好地利用历史数据，提高通信效率。
在另一种可能的设计中，前述基于深度强化学习的速率模型还包括价值网络；该方法还包括：根据数据库存储的第二数据的网络性能目标、第二TCP连接的网络状态、第二全局测量值以及第二发送速率四者之间的对应关系通过价值网络确定损失函数值；根据损失函数值更新价值网络，确定更新后的策略梯度；根据更新后的策略梯度对策略网络和发送速率决策网络进行更新。以此可以实现根据新产生的数据，对速率模型进行优化，进一步提高了对历史数据的利用，使得确定的发送速率更合理，通信效率更高。
第二方面，提供了一种数据传输装置。包括：确定单元，用于根据第一数据的网络性能目标和第一TCP连接的网络状态确定第一发送速率；第一TCP连接用于发送第一数据；发送单元，用于根据第一发送速率发送第一数据。
在一种可能的设计中，第一数据的网络性能目标根据第一数据的特征确定。
在一种可能的设计中，网络性能目标包括有效吞吐量目标、时延目标和丢包率目标中的一项或多项。示例性的，网络性能目标可以为有效吞吐量、时延和丢包率三者的目标权重比例。
在另一种可能的设计中，确定单元具体用于：按照指定决策周期根据第一数据的网络性能目标和第一TCP连接的网络状态确定第一发送速率。
在另一种可能的设计中，第一TCP连接的网络状态根据接收到的第一TCP连接对应的确认字符ACK确定。
在另一种可能的设计中，网络状态包括：队列时延的平滑值、在决策期间内发送TCP数据包的速率、在决策期间内接收TCP数据包的速率、往返时间RTT的平滑值、发送TCP数据包的间隔、发送ACK的间隔；
其中，网络状态用于评估时延、发送间隔、丢包率以及带宽。
在另一种可能的设计中，确定单元具体用于：根据第一数据的网络性能目标、第一TCP连接的网络状态和第一全局测量值确定第一发送速率；全局测量值为发送端对 应的一个或多个TCP连接的测量值。
在另一种可能的设计中，全局测量值包括下述一项或多项：log(发送端的有效吞吐量/发送端的最大的有效吞吐量)，log(发送端一个或多个TCP连接的最小RTT/发送端一个或多个TCP连接的往返时间RTT的平滑值)，log(发送端的丢包率)。
在另一种可能的设计中，还包括：存储单元，用于将第一数据的网络性能目标、第一TCP连接的网络状态、第一全局测量值以及第一发送速率对应存储在数据库中。
在另一种可能的设计中，基于深度强化学习的速率模型包括策略网络和发送速率决策网络；
确定单元用于：将第一TCP连接的网络状态作为策略网络的输入，并将策略网络的输出、第一数据的网络性能目标和全局测量值作为发送速率决策网络的输入，输出为第一发送速率。
在另一种可能的设计中，基于深度强化学习的速率模型还包括价值网络；装置还包括更新单元，更新单元具体用于：根据数据库存储的第二数据的网络性能目标、第二TCP连接的网络状态、第二全局测量值以及第二发送速率四者之间的对应关系通过价值网络确定损失函数值；根据损失函数值更新价值网络，确定更新后的策略梯度；根据更新后的策略梯度对策略网络和发送速率决策网络进行更新。
第三方面，提供了一种网络设备。该网络设备包括收发器、处理器和存储器；所述收发器用于收发数据，所述存储器用于存放程序；所述处理器用于执行所述存储器存储的所述程序，以控制所述网络设备执行上述第一方面或第一方面中任一种可能的设计所述的方法。
第四方面，提供了一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器执行时实现上述第一方面或第一方面中任一种可能的设计所述的方法。
第五方面，提供了一种包含指令的计算机程序产品，当所述指令在计算机上运行时，使得计算机执行上述第一方面或第一方面中任一种可能的设计所述的方法。
第六方面，提供了一种芯片。该芯片包括处理器和存储器；存储器用于存放程序；所述处理器用于执行所述存储器存储的所述程序，以执行上述第一方面或第一方面中任一种可能的设计所述的方法。
本申请在该实施例中，通过网络性能目标和网络状态来确定发送速率，优化了发送速率的确定，提高了数据传输的效率，该网络性能目标可以是根据需要发送的数据的特征确定，以此适应不同应用需求的场景。进一步地，还可利用历史经验，获得合适网络状态和网络性能目标的发送速率的大小，不经历传统TCP拥塞控制算法的慢启动阶段，提高发送端的有效吞吐量、减少了启动时间、提高了吞吐率。
附图说明
图1为一种拥塞机制示意图；
图2为本申请实施例提供的一种应用场景示意图；
图3为本申请实施例提供的一种数据传输方法示意性流程图；
图4为本申请实施例提供的速率模型的一个示例；
图5为本申请实施例提供的速率模型的另一个示例；
图6为本申请实施例提供的另一种数据传输方法示意性流程图；
图7a为本申请实施例提供的速率模型的另一个示例；
图7b为本申请实施例提供的速率模型的另一个示例；
图8为本申请实施例提供的速率模型的另一个示例；
图9为本申请实施例提供的另一种数据传输方法示意性流程图；
图10为本申请实施例提供的一种数据传输装置结构示意图；
图11为本申请实施例提供的一种网络设备结构示意图。
具体实施方式
下面通过附图和实施例，对本申请的技术方案做进一步的详细描述。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本申请的实施例例如能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
图2为本申请实施例提供的一种应用场景示意图。如图2所示，该场景中包括发送端100以及接收端200。发送端100通过M个网络节点将数据报文发送到TCP接收端，M为大于或等于1的整数。其中，发送端100可以为数据中心节点或者内容分发网络(content delivery network，CDN)节点等为终端提供数据的服务器或代理服务器，例如，发送端包括但不限于网络服务器(web server)、视频服务器(video server)、即时通信服务器(IM server)和游戏服务器(game server)等。发送端100向接收端200提供网络页面、视频文件、IM交流和游戏等数据。接收端200可以是为用户提供网络浏览、视频播放、通信交流及娱乐休闲等功能的终端，例如但不限于手机、电脑、可穿戴设备等。网络节点(图中未示出)为位于发送端100和接收端200之间的数据报文转发设备，网络节点具有IP地址，支持传输层协议传输控制协议/因特网互联协议(transmission control protocol/internet protocol，TCP/IP)，可以是数字用户线路接入复用器(digital subscriber line access multiplexer，DSLAM)、交换机、路由器或光线路终端设备(optical line terminal，OLT)等网络节点。
图3为本申请实施例提供的一种数据传输方法示意性流程图。如图3所示，该方法具体可以包括如下步骤：
S310，发送端根据第一数据的网络性能目标和第一TCP连接的网络状态确定第一发送速率。其中，第一数据的网络性能目标可以根据第一数据的特征确定，第一TCP连接用于发送第一数据。
其中，该TCP连接的网络状态可以指该TCP连接对应的当前网络状态，或该TCP连接进行第一数据的传输时对应的网络状态。
同一个发送端可以与一个或多个接收端分别建立TCP连接，并通过TCP连接发 送数据。该第一数据可以为接收端向发送端请求发送的数据。
在本申请实施例中，可以预先定义数据的特征与网络性能目标的对应关系。该数据的特征可以包括提供该数据的应用的特征、接收该数据的接收端的特征或者数据本身的特征中的一项或多项。其中，不同应用或同一应用的不同服务对网络性能的需求不同，可以建立应用或应用服务的类型，应用或应用服务的标识等应用的信息与网络性能目标的对应关系，例如，对于直播类应用，对网络性能的需求为低时延，对于云盘类应用，对网络性能的需求为保持较高的吞吐，对于视频应用，在线观看服务比离线缓存服务的时延要求高；不同的接收端对网络性能的需求也是不同的，可以预先建立接收端需求与网络性能目标的对应关系，例如，接收端性能比较好，则可以接受较高的吞吐量；数据本身也对网络性能的需求不同，可以预先建立数据本身特征与网络性能目标的对应关系，例如，数据量比较大的数据，对吞吐量要求较高。
基于此，发送端确定第一数据的网络性能目标可以包括多种方式。例如，提供第一数据的应用服务器可以预定义第一数据的网络性能目标，并将第一数据的网络性能目标发送至发送端。再例如，发送端可以根据第一数据的特征，基于本地预先确定的特征与网络性能目标的对应关系，确定第一数据的网络性能目标。再例如，发送端可以接收接收端发送的网络性能目标。
其中，网络性能目标可以用于指示有效吞吐量、时延和丢包率中一项或多项的需求。例如，该网络性能目标可以包括有效吞吐量目标、时延目标和丢包率目标中的一项或多项；或者，网络性能目标可以为有效吞吐量目标、时延目标和丢包率目标的比值；或者为有效吞吐量、时延和丢包率需求的权重值；或者有效吞吐量、时延和丢包率中优先的一个或多个。
在接收端与发送端需要通过TCP连接传输数据时，接收端可以向发送端发送数据请求，在发送端接收到发送端的数据请求后，发送端与接收端建立TCP连接，以便通过该TCP连接发送数据。其中，第一TCP连接可以指发送端与接收端传输第一数据时建立的TCP连接。
在本申请实施例中，TCP连接的网络状态主要用于评估TCP连接的时延、发送间隔、丢包率以及带宽等信息。另外，发送端可以按照指定决策周期根据第一数据的网络性能目标和第一TCP连接的网络状态确定第一发送速率，此时，TCP连接的网络状态可以指TCP连接的决策周期内的当前网络状态。TCP连接的网络状态还可以用于评估时延的抖动或者误码率等等。
在一个示例中，TCP连接的网络状态可以包括队列时延的平滑值(用于评估时延)、在决策期间内发送TCP数据包的速率(用于评估发送间隔、丢包率)、在决策期间内接收TCP数据包的速率(用于评估发送间隔、丢包率)、往返时间RTT的平滑值(用于评估时延)、发送TCP数据包的间隔(用于评估带宽)或发送ACK的间隔(用于评估带宽)等等。其中，上述网络状态仅为示例，本申请实施例中TCP连接的网络状态还可以包括用于评估TCP连接的时延、发送间隔、丢包率或带宽等信息中任意一项的其他网络状态。该其他网络状态中的一个或多个可以与前述网络状态中的一种或多种组合，以实现评估TCP连接的时延、发送间隔、丢包率和带宽等信息。
另外，TCP连接的网络状态可以根据接收端发送的ACK确定。例如，发送端记 录ACK到达时间，同时从ACK中提取ACK的序号、ACK确认序号、发送端发送的时间以及ACK发送的时间，例如：从序号为1的ACK中提取如下信息：序号，1；该数据包发送时的时间，129；已发送的总字节数，1436；上一个数据包确认收到的时间，62；已确认收到的总字节数，1436；本次确认收到的字节数，1436；ACK发送的时间，192。
示例性的，至少可以通过如下方式确定该ACK对应的TCP连接的网络状态。
其中，队列时延的平滑值可以根据该数据包发送时的时间以及上一个数据包确认接收到的时间计算得到。
在决策期间内发送TCP数据包的速率可以根据决策期间内已发送的总字节数和接收到ACK的时间确定。其中，该决策期间内可以指接收到ACK的时间到上一次决策后的时间。
在决策期间内接收TCP数据包的速率可以根据决策期间内已确认收到的总字节数和发送ACK的时间确定。其中，该决策期间内可以指发送ACK的时间到上一次决策后的时间。
RTT的平滑值SRTT的计算方式如下：SRTT(新的SRTT)＝αSRTT(旧的SRTT值)+(1-α)RTT(新的RTT值)。基于此，SRTT的初始值就是0值。这里α是一个平滑因子，它决定了旧的SRTT值所占的权重，0≤α<1。其中，RTT可以根据该数据包发送时的时间和ACK到达时间计算得到。
发送TCP数据包的间隔为本次与前一次发送TCP数据包的间隔。
发送ACK的间隔为本次ACK与前一次ACK的发送间隔。
在通过TCP连接首次发送数据时，由于无法确定网络状态，所述可以确定该TCP连接的网络状态为初始值，该初始值为预设值，例如，初始值可以为0。
在确定第一数据的网络性能目标和第一TCP连接的网络状态后，根据该网络性能目标和网络状态，进行拥塞控制，根据网络状态选择符合网络性能目标要求的第一发送速率通过第一TCP连接发送第一数据。
其中，发送端确定第一发送速率可以包括多种方式。
在一个示例中，发送端可以预先确定速率模型，通过将第一数据的网络性能目标和第一TCP连接的网络状态作为预先确定的速率模型的输入，输出为第一发送速率。例如，该速率模型可以为基于深度强化学习的速率模型(例如，基于深度确定性策略梯度(deep deterministic policy gradient，DDPG)算法的模型)，也可以是其他机器学习模型。
以速率模型为基于深度强化学习的速率模型为例进行说明。例如，如图4所示，基于深度强化学习的速率模型包括策略网络401和发送速率决策网络402；在计算第一发送速率时可以将第一TCP连接的网络状态作为策略网络401的输入，并将该策略网络401的输出和第一数据的网络性能目标作为发送速率决策网络402的输入，输出为第一发送速率。
例如，网络状态可以表示为S(t)，该策略网络401的输出为A(t)，发送速率为a(t)，网络性能目标为g，策略网络401可以表达为：A(t)＝h(S(t)|β)，其中β是用来定义h的一组参数，即具体的h随β的改变而改变。发送速率决策网络402可以表达为：
Figure PCTCN2019086028-appb-000001
Figure PCTCN2019086028-appb-000002
其中
Figure PCTCN2019086028-appb-000003
是用来定义f的一组参数，即具体的f随
Figure PCTCN2019086028-appb-000004
的改变而改变。
另外，还可以根据发送端运行过程中产生的数据，对得到的速率模型进行优化。例如，发送端可以将确定的网络性能目标、网络状态和发送速率，按照对应关系记录在数据库中。例如，数据库中记录的对应关系可以如表1所示。
表1
Figure PCTCN2019086028-appb-000005
需要说明的是，表1仅为示例，在具体实现中，数据库中可以包括更多或更少的数据。表1中的对应关系也可以通过多个表来实现。
发送端可以按照一定的时间间隔或规律，例如，数据库中的数据每增加一定数量时，根据数据库中的数据对速率模型进行更新。基于此，本申请实施例还可以包括：根据数据库存储的第二数据的网络性能目标、第二TCP连接的网络状态、第二全局测量值以及第二发送速率四者之间的对应关系通过价值网络确定损失函数值；根据损失函数值更新价值网络，确定更新后的策略梯度；根据更新后的策略梯度对策略网络和发送速率决策网络进行更新。其中，第二数据的网络性能目标、第二TCP连接的网络状态、第二全局测量值以及第二发送速率四者之间的对应关系可以包括前述第一数据的网络性能目标、第一TCP连接的网络状态、第一全局测量值以及第一发送速率的对应关系。
以基于DDPG算法的速率模型为例进行说明。例如，如图5所示，以基于DDPG算法的速率模型包括策略网络501、发送速率决策网络502和价值网络503。其中，策略网络501、发送速率决策网络502与图4所示的策略网络401、发送速率决策网络402为相同的网络。从数据库中取指定大小的数据进行训练，训练过程如下：
根据网络性能目标g，利用价值网络中的评估网络的输出Q(at)和目标网络的输出Q(a’)，按如下公式计算损失函数(loss)：
loss＝E[(G(g)+Q(a’)-Q(a t) 2]
根据损失函数更新价值网络503的参数，根据更新后的价值网络503确定价值网络503的策略梯度；
根据价值网络503的策略梯度，结合策略网络501自身的策略梯度，更新策略网络501以及发送速率决策网络502的参数。
在另一个示例中，发送端可以根据预先确定的网络状态和网络性能目标到发送速率的映射确定第一发送速率。例如，可以预先确定多组网络状态、网络性能目标以及发送速率样本，针对不同的网络状态以及不同的网络性能目标采取不同的方式调节发 送速率，修改调节方式，直到目标函数达到最优，最终生成一个网络状态和网络性能目标到发送速率的映射。
S320，发送端根据第一发送速率向接收端发送第一数据。
发送端在确定第一发送速率后，可以根据该第一发送速率计算本次发送的最大发送量，并向接收端发送第一数据。
在确定第一数据的发送速率后，发送端判断第一数据对应的TCP数据包是否全部发送完成，如未发送完成，根据第一发送速率通过第一TCP连接发送未被发送的第一数据的TCP数据包。
发送端判断是否收到第一数据的TCP数据包的ACK。
若收到，则根据该ACK确定下一次发送数据的网络状态。
若发送端未收到ACK，则判断发送窗口对应的TCP数据包是否全部发送完成；如果没有完成，则继续发送未被发送的第一数据的TCP数据包。如果已完成，则停止发送，直至收到第一数据的TCP数据包的ACK。
本申请在该实施例中，通过根据需要发送的数据的特征确定的网络性能目标来确定发送速率，提高了应对不同网络性能需求的能力，提高了数据传输的效率。发送端利用历史经验，直接获得合适网络状态和网络性能目标的发送速率的大小，不经历传统TCP拥塞控制算法的慢启动阶段，提高发送端的有效吞吐量、减少了启动时间、提高了吞吐率。
图6为本申请实施例提供的一种数据传输方法的示意图。在本实施例中发送端还可以结合全局测量值确定一发送速率。该方法具体可以包括如下步骤：
S610，根据第一数据的网络性能目标、第一TCP连接的网络状态和第一全局测量值确定第一发送速率。其中，全局测量值为发送端对应的一个或多个TCP连接的测量值。该是一个或多个TCP连接可以包括第一TCP连接。该第一全局测量值可以为发送端当前的全局测量值，或发送端在通过一个或多个TCP连接传输包括第一数据在内的数据时对应的全局测量值。当第一发送速率按照决策周期进行时，该第一全局测量值可以为决策周期内的发送端的全局测量值。
其中，S610中的第一数据、第一数据的网络性能目标、第一TCP连接、第一TCP连接的网络状态可以参见前述S310中的描述，此处不再赘述。
其中，全局测量值可以用于对发送端的全部TCP连接进行评估。该全局测量值包括下述一项或多项：log(发送端的有效吞吐量和发送端的最大的有效吞吐量的比值)，log(发送端一个或多个TCP连接的最小RTT和发送端一个或多个TCP连接的RTT的平滑值的比值)，log(发送端的丢包率)。
另外，全局测量值可以根据发送端的全局TCP连接的ACK确定。例如，发送端记录ACK到达时间，同时从ACK中提取ACK的序号、ACK确认序号、发送端发送的时间以及ACK发送的时间，发送端通过记录每一个如上所述的ACK，发送端可以根据多个TCP连接的ACK确定全局测量值。
根据第一数据的网络性能目标、第一TCP连接的网络状态和全局测量值确定第一发送速率可以包括多种方式，具体如下所述。
在一个示例中，发送端可以预先确定速率模型，通过将第一数据的网络性能目标、 第一TCP连接的网络状态和全局测量值作为预先确定的速率模型的输入，输出为第一发送速率。例如，该速率模型可以为基于深度强化学习的速率模型，例如，基于DDPG算法的速率模型，也可以是其他机器学习模型。
以基于DDPG算法的速率模型为例进行说明。例如，如图7a所示，拥塞控制环境包括发送端711、发送端712和发送端713等发送端以及接收端721、接收端722和接收端723等接收端。其中，发送端与接收端之间还包括其他用于传输数据和指令的网元。在发送端711向接收端721通过第一TCP连接发送第一数据时，可以根据该第一数据对应的应用需求确定网络性能目标，例如，该性能目标可以为影响因子的目标权重比例，该影响因子可以包括时延、有效吞吐量以及丢包率等等。发送端711每次接收到ACK时，都计算该ACK对应的网络状态以及测量值，根据多个TCP连接的测量值可以得到全局测量值，例如，结合7a所示，该多个TCP连接可以包括发送端711与接收端721和发送端711与接收端722的两个TCP连接。在计算发送速率时，可以根据第一TCP连接的网络状态、第一网络性能目标以及第一全局测量值基于深度强化学习的速率模型计算得到第一发送速率，发送端711根据第一发送速率向接收端721通过第一TCP连接继续发送数据。
其中，基于深度强化学习的速率模型可以包括多种实现方式。例如，如图7b所示，基于深度强化学习的速率模型可以包括策略网络701和发送速率决策网络702；在计算第一发送速率时可以将第一TCP连接的网络状态作为策略网络701的输入，并将该策略网络701的输出和第一TCP连接的网络状态和第一全局测量值作为发送速率决策网络702的输入，输出为第一发送速率。
例如，网络状态可以标识为S(t)，策略网络401的输出为A(t)，发送速率为a(t)，网络性能目标为g，全局测量值为m(t)。策略网络401可以表达为：A(t)＝h(S(t)|β)，其中β是用来定义h的一组参数，即具体的h随β的改变而改变。发送速率决策网络402可以表达为：
Figure PCTCN2019086028-appb-000006
其中
Figure PCTCN2019086028-appb-000007
是用来定义f的一组参数，即具体的f随
Figure PCTCN2019086028-appb-000008
的改变而改变。
另外，还可以根据发送端运行过程中产生的数据，对得到的速率模型进行优化。例如，发送端可以将将第一数据的网络性能目标、第一TCP连接的网络状态、第一全局测量值以及第一发送速率对应存储在数据库中。由于发送端可以是为多个接收端提供多种数据服务的，所以发送端本地数据库中可以存储有多个TCP连接对应的数据。
发送端可以按照一定的时间间隔或规律，例如，数据库中的数据每增加一定数量时，根据数据库中的数据对速率模型进行更新。
以基于DDPG算法的速率模型为例进行说明。例如，基于深度强化学习的速率模型包括策略网络801、发送速率决策网络802和价值网络803。其中，策略网络801、发送速率决策网络802与图7所示的策略网络701、发送速率决策网络702为相同的网络。从数据库中取指定大小的数据进行训练，训练过程如下：
根据网络性能目标g，网络测量值m，利用价值网络803中的评估网络的输出Q(at)和目标网络的输出Q(a’)，按如下公式计算损失函数(loss)：
loss＝E[(G(g)·M(m t)+Q(a’)-Q(a t) 2]
更新价值网络803中的评估网络和目标网络的参数θμ和θQ；
取得价值的策略梯度Q；根据价值的策略梯度Q，结合策略网络自身的策略梯度，更新策略网络801以及发送速率决策网络802的参数。
其中，策略网络801或发送速率决策网络802可以由参数为θμ的评估网络和参数为θQ的目标网络的参数构成。以发送速率决策网络801的更新为例进行说明。其中，该送速率决策网络801由评估网络“μ(st|θμ)”和目标网络“Q(st，at|θQ)”构成，at为策略网络801的输出，st为网络状态，N为训练样本的数量，函数J为针对θμ的梯度，根据如下公式进行参数更新：
Figure PCTCN2019086028-appb-000009
在另一个示例中，发送端可以根据预先确定的网络状态和网络性能目标到发送速率的映射确定第一发送速率。例如，可以预先确定多组网络状态、网络性能目标以及发送速率样本，针对不同的网络状态以及不同的网络性能目标采取不同的方式调节发送速率，修改调节方式，直到目标函数达到最优，最终生成一个网络状态和网络性能目标到发送速率的映射。
S620，发送端根据第一发送速率向接收端发送第一数据。
该步骤S620可以参见前述S320中的相关描述，不再赘述。
图9为本申请实施例提供的一种数据传输方法的示意图。如图9所示，该方法可以部署于Web服务器中，其中，可以将网络性能目标g设置为g＝(0.7，0.2，0.1)，保证较高的吞吐的前提下，保持较低的时延和允许一定的丢包率。Web服务器端发送速率的决策依据的性能目标g，优化目标为提高Web服务器的吞吐量。通过套接字(Socket)传递给速率计算引擎。Web服务器的数据库中存储了历史TCP流的每一个网络状态s、发送速率a、网络测量值m以及网络性能目标g。
在数据传输过程中具体流程如下：
S901，当Web服务器需要向终端发送数据时，首先与终端建立TCP连接，Web服务器端收到终端发送的确认建立TCP连接的ACK信号时，将当前网络状态设置为初始化。例如，在12Mbps带宽60毫秒RTT的链路中TCP连接建立后，网络状态初始化为s＝[[0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.][0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.][0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.][0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.][0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.][0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.]]由于网络状态具有时序关系，因此本实例中每一维可以取16个时序值。
S902，速率计算引擎依据多目标优化的深度强化学习技术生成，根据网络状态s、测量值m信息以及网络性能目标g决策发送速率a。发送速率a的具体计算由深度强化学习网络中速率计算引擎计算输出，该速率计算引擎为基于DDPG算法的多目标深度强化学习的速率模型，该速率模型输入为网络状态s、当前网络测量值m以及网络性能目标g，输出即为下一次发送速率a＝2.68Mbps。根据a的值，计算本次发送的最大发送量wnd＝a/0.008/1436*RTT。
S903，Web服务器端判断该TCP连接是否有数据发送。如果没有数据发送，则发送结束；如有数据发送，根据上述发送速率a发送数据，由发送引擎发送相应数据经网络接口发送至网络中。
S904，终端通过TCP连接收到数据后，生成ACK，经由网络发送给服务器。
S905，当服务器未收到ACK时，则判断本次发送的最大发送量是否完成，如果没有完成，继续按该速率进行数据发送。
S906，当服务器收到ACK时，记录ACK到达时间，同时从ACK信号中提取ACK的序号、ACK确认序号、发送端发送的时间以及ACK信号发送的时间。
例如：序号为1的ACK信号中提取如下表2所示的信息：
表2
序号(seq_num) 1
该数据包发送时的时间(send_ts) 129
已发送的总字节数(sent_bytes) 1436
上一个数据包确认收到的时间(delivered_time) 62
已确认收到的总字节数(delivered) 1436
本次确认收到的字节数(send_ack_bytes) 1436
ACK发送的时间(send_ack_time) 192
根据上述表2所示的ACK的信息，计算得到全局测量值m存储。生成新的网络状态并做归一化处理后，s＝[[0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.41][0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.-1.41][0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.41][0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.41][0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.41][0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1.41]]，判断当前距离上次决策是否间隔10ms，如果没有，则不进行速率更新，否则将新的网络状态发送至速率计算引擎计算取得新的发送速率a＝4.15Mbps，进行下一轮数据发送。
其中测量值m的计算为m＝[goodput/max_g_tput，min_rtt/rtt_ewma，loss rate]＝[0.68，0.98，0.86]
其中，goodput为发送端当前链路的有效吞吐量(该发送端当前链路可以对应多个TCP连接)，具体地，goodput＝发送端当前已确认的发送数据包大小(delivered)/当前时间-发送端启动时间；
max_g_tput为发送端当前最大的有效吞吐量；
min_rtt为发送端当前TCP流中的最小RTT，具体地，rtt＝当前时间-数据包发送时间；
rtt_ewma为发送端当前TCP流的RTT的平滑值，具体地，rtt_ewma＝0.875*rtt_ewma+0.125*rtt；
loss rate为发送端当前丢包率，即loss rate＝(已发送的数据包的大小-已确认的数据包的大小)/已发送的数据包的大小。
图10为本申请实施例提供的一种数据传输装置结构示意图。该装置可以用于执行图3或图6所示实施例中发送端所执行的方法。如图10所示，该装置包括：
确定单元1001，用于根据第一数据的网络性能目标和第一TCP连接的网络状态确定第一发送速率；所述第一TCP连接用于发送所述第一数据；
发送单元1002，用于根据所述第一发送速率发送所述第一数据。
在一个实施例中，第一数据的网络性能目标可以根据第一数据的特征确定。
在另一个实施例中，所述确定单元1001具体用于：
按照指定决策周期根据第一数据的网络性能目标和第一TCP连接的网络状态确定第一发送速率。
在另一个实施例中，其特征在于，所述第一TCP连接的网络状态根据接收到的所述第一TCP连接对应的确认字符ACK确定。
在另一个实施例中，所述网络状态包括：队列时延的平滑值、在决策期间内发送TCP数据包的速率、在决策期间内接收TCP数据包的速率、往返时间RTT的平滑值、发送TCP数据包的间隔、发送ACK的间隔；
其中，所述网络状态用于评估时延、发送间隔、丢包率以及带宽。
在另一个实施例中，所述确定单元1001具体用于：
根据第一数据的网络性能目标、第一TCP连接的网络状态和第一全局测量值确定第一发送速率；所述全局测量值为发送端对应的一个或多个TCP连接的测量值。
在另一个实施例中，所述全局测量值包括下述一项或多项：log(发送端的有效吞吐量/发送端的最大的有效吞吐量)，log(发送端一个或多个TCP连接的最小RTT/发送端一个或多个TCP连接的往返时间RTT的平滑值)，log(发送端的丢包率)。
在另一个实施例中，还包括：
存储单元1003，用于将所述第一数据的网络性能目标、所述第一TCP连接的网络状态、所述全局测量值以及所述第一发送速率对应存储在数据库中。
在另一个实施例中，基于深度强化学习的速率模型包括策略网络和发送速率决策网络；
所述确定单元1001具体用于：将第一TCP连接的网络状态作为所述策略网络的输入，并将所述策略网络的输出、所述第一数据的网络性能目标和所述全局测量值作为发送速率决策网络的输入，输出为所述第一发送速率。
在另一个实施例中，所述基于深度强化学习的速率模型还包括价值网络；
所述装置还包括更新单元，所述更新单元用于：
根据数据库存储的第二数据的网络性能目标、第二TCP连接的网络状态、全局测量值以及第二发送速率四者之间的对应关系通过价值网络确定损失函数值；
根据所述损失函数值更新所述价值网络，确定更新后的策略梯度；
根据所述更新后的策略梯度对策略网络和发送速率决策网络进行更新。
上述数据传输装置涉及的相关技术特征在上文中已经结合附图，例如但不限于图3和图6，进行了详细的描述，此处不再赘述。
图11为本发明实施例提供的一种网络设备结构示意图。该网络设备1100具体包括：包括收发器1101，处理器1102，存储器1103。收发器1101、处理器1102和存储器1103可以通过总线连接。该网络设备可以用于实现图3或图6所示实施例中发送端的功能。
其中，收发器1101用于支持发送端与上述实施例中的接收端或其他网元之间收发信息，例如，该收发器1101可以包括以太网接口。在发送端与接收端之间通信过程中，数据和信令消息由处理器1102进行处理，并由收发器1101发送给接收端。来自接收 端的数据和信令的经由收发器1101接收，由处理器1102进行处理得到发送端发送的数据和信令。处理器1102可以控制发送设备1100执行图3或图6所示实施例中涉及发送端的处理过程和/或用于本申请所描述的技术的其他过程。例如，处理器1102用于执行图3和图6所示的实施例中的S310和S610，收发器1101用于执行图3和图6所示的实施例中的S320和S620。存储器1103用于存储终端的程序代码和数据。
本申请实施例提供了一种芯片。该芯片包括处理器和存储器；存储器用于存放程序；所述处理器用于执行所述存储器存储的所述程序，以执行上述方法实施例中所述的方法。
在上述各个本申请实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读介质向另一个计算机可读介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如，固态硬盘(solid state disk，SSD))等。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (20)

  1. 一种数据传输方法，其特征在于，包括：
    根据第一数据的网络性能目标和第一传输控制协议TCP连接的网络状态确定第一发送速率；所述第一TCP连接用于发送所述第一数据；
    根据所述第一发送速率发送所述第一数据。
  2. 根据权利要求1所述的方法，其特征在于，所述第一数据的网络性能目标根据所述第一数据的特征确定。
  3. 根据权利要求1或2所述的方法，其特征在于，所述第一TCP连接的网络状态根据接收到的所述第一TCP连接对应的确认字符ACK确定。
  4. 根据权利要求1-3任意一项所述的方法，其特征在于，所述网络状态包括：队列时延的平滑值、在决策期间内发送TCP数据包的速率、在决策期间内接收TCP数据包的速率、往返时间RTT的平滑值、发送TCP数据包的间隔、发送ACK的间隔；
    其中，所述网络状态用于评估时延、发送间隔、丢包率以及带宽。
  5. 根据权利要求1-4任意一项所述的方法，其特征在于，所述根据第一数据的网络性能目标和第一TCP连接的网络状态确定第一发送速率包括：
    根据第一数据的网络性能目标、第一TCP连接的网络状态和第一全局测量值确定第一发送速率；所述第一全局测量值为发送端对应的一个或多个TCP连接的测量值。
  6. 根据权利要求5所述的方法，其特征在于，所述全局测量值包括下述一项或多项：log(发送端的有效吞吐量/发送端的最大的有效吞吐量)，log(发送端一个或多个TCP连接的最小RTT/发送端一个或多个TCP连接的往返时间RTT的平滑值)，log(发送端的丢包率)。
  7. 根据权利要求5或6任意一项所述的方法，其特征在于，还包括：
    将所述第一数据的网络性能目标、所述第一TCP连接的网络状态、所述第一全局测量值以及所述第一发送速率对应存储在数据库中。
  8. 根据权利要求5-7任意一项所述的方法，其特征在于，基于深度强化学习的速率模型包括策略网络和发送速率决策网络；
    所述根据第一数据的网络性能目标、第一TCP连接的网络状态和第一全局测量值确定第一发送速率包括：
    将第一TCP连接的网络状态作为所述策略网络的输入，并将所述策略网络的输出、所述第一数据的网络性能目标和所述第一全局测量值作为发送速率决策网络的输入，输出为所述第一发送速率。
  9. 根据权利要求8所述的方法，其特征在于，所述基于深度强化学习的速率模型还包括价值网络；
    所述方法还包括：
    根据数据库存储的第二数据的网络性能目标、第二TCP连接的网络状态、第二全局测量值以及第二发送速率四者之间的对应关系通过价值网络确定损失函数值；
    根据所述损失函数值更新所述价值网络，确定更新后的策略梯度；
    根据所述更新后的策略梯度对策略网络和发送速率决策网络进行更新。
  10. 一种数据传输装置，其特征在于，包括：
    确定单元，用于根据第一数据的网络性能目标和第一传输控制协议TCP连接的网络状态确定第一发送速率；所述第一TCP连接用于发送所述第一数据；
    发送单元，用于根据所述第一发送速率发送所述第一数据。
  11. 根据权利要求10所述的装置，其特征在于，所述第一数据的网络性能目标根据所述第一数据的特征确定。。
  12. 根据权利要求10或11所述的装置，其特征在于，所述第一TCP连接的网络状态根据接收到的所述第一TCP连接对应的确认字符ACK确定。
  13. 根据权利要求10-12任意一项所述的装置，其特征在于，所述网络状态包括：队列时延的平滑值、在决策期间内发送TCP数据包的速率、在决策期间内接收TCP数据包的速率、往返时间RTT的平滑值、发送TCP数据包的间隔、发送ACK的间隔；
    其中，所述网络状态用于评估时延、发送间隔、丢包率以及带宽。
  14. 根据权利要求10-13任意一项所述的装置，其特征在于，所述确定单元具体用于：
    根据第一数据的网络性能目标、第一TCP连接的网络状态和第一全局测量值确定第一发送速率；所述全局测量值为发送端对应的一个或多个TCP连接的测量值。
  15. 根据权利要求14所述的装置，其特征在于，所述全局测量值包括下述一项或多项：log(发送端的有效吞吐量/发送端的最大的有效吞吐量)，log(发送端一个或多个TCP连接的最小RTT/发送端一个或多个TCP连接的往返时间RTT的平滑值)，log(发送端的丢包率)。
  16. 根据权利要求14或15任意一项所述的装置，其特征在于，还包括：
    存储单元，用于将所述第一数据的网络性能目标、所述第一TCP连接的网络状态、所述第一全局测量值以及所述第一发送速率对应存储在数据库中。
  17. 根据权利要求14-16任意一项所述的装置，其特征在于，基于深度强化学习的速率模型包括策略网络和发送速率决策网络；
    所述确定单元具体用于：
    将第一TCP连接的网络状态作为所述策略网络的输入，并将所述策略网络的输出、所述第一数据的网络性能目标和所述第一全局测量值作为发送速率决策网络的输入，输出为所述第一发送速率。
  18. 根据权利要求17所述的装置，其特征在于，所述基于深度强化学习的速率模型还包括价值网络；
    所述装置还包括更新单元，所述更新单元用于：
    根据数据库存储的第二数据的网络性能目标、第二TCP连接的网络状态、第二全局测量值以及第二发送速率四者之间的对应关系通过价值网络确定损失函数值；
    根据所述损失函数值更新所述价值网络，确定更新后的策略梯度；
    根据所述更新后的策略梯度对策略网络和发送速率决策网络进行更新。
  19. 一种计算机可读存储介质，包括计算机可读指令，当计算机读取并执行所述计算机可读指令时，使得计算机执行如权利要求1-9任意一项所述的方法。
  20. 一种计算机程序产品，包括计算机可读指令，当计算机读取并执行所述计算机可读指令，使得计算机执行如权利要求1-9任意一项所述的方法。
PCT/CN2019/086028 2018-09-04 2019-05-08 数据传输方法及装置 WO2020048147A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19858201.7A EP3780542B1 (en) 2018-09-04 2019-05-08 Data transmission method and device
US17/098,030 US11558302B2 (en) 2018-09-04 2020-11-13 Data transmission method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811026124.1A CN110875915B (zh) 2018-09-04 2018-09-04 数据传输方法及装置
CN201811026124.1 2018-09-04

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/098,030 Continuation US11558302B2 (en) 2018-09-04 2020-11-13 Data transmission method and apparatus

Publications (1)

Publication Number Publication Date
WO2020048147A1 true WO2020048147A1 (zh) 2020-03-12

Family

ID=69715991

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/086028 WO2020048147A1 (zh) 2018-09-04 2019-05-08 数据传输方法及装置

Country Status (4)

Country Link
US (1) US11558302B2 (zh)
EP (1) EP3780542B1 (zh)
CN (1) CN110875915B (zh)
WO (1) WO2020048147A1 (zh)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11330074B2 (en) * 2020-08-12 2022-05-10 Fortinet, Inc. TCP (transmission control protocol) fast open for classification acceleration of cache misses in a network processor
CN111970208B (zh) * 2020-08-27 2024-03-05 腾讯科技（深圳）有限公司 网络传输方法、装置、设备及计算机可读存储介质
CN112235322A (zh) * 2020-12-11 2021-01-15 广东睿江云计算股份有限公司 一种tcp传输方法及其系统
US20220207210A1 (en) * 2020-12-31 2022-06-30 Fortinet, Inc. Compiler plugin for special-purpose computer processors with dual support for design verification and release packaging
CN112770357B (zh) * 2021-01-08 2022-04-26 浙江大学 一种基于深度强化学习的无线网络拥塞控制方法
CN113411228B (zh) * 2021-06-04 2023-04-07 网宿科技股份有限公司 一种网络状况的确定方法及服务器
CN113518246B (zh) * 2021-09-10 2022-02-01 腾讯科技（深圳）有限公司 数据传输方法、装置、计算机设备、存储介质及程序产品
US11616730B1 (en) * 2021-10-01 2023-03-28 Compira Labs Ltd. System and method for adapting transmission rate computation by a content transmitter
WO2023078542A1 (en) * 2021-11-03 2023-05-11 Huawei Technologies Co., Ltd. An apparatus and a method for congestion control for multiple heterogeneous flows
CN115412437A (zh) * 2022-08-17 2022-11-29 Oppo广东移动通信有限公司 数据处理方法及装置、设备、存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040044783A1 (en) * 2002-08-27 2004-03-04 Marko Nordberg Method and arrangement for tuning rate of media stream
CN1848810A (zh) * 2006-05-15 2006-10-18 武汉虹旭信息技术有限责任公司 一种流媒体发送速率控制方法
CN102457486A (zh) * 2010-10-19 2012-05-16 华为技术有限公司 一种媒体内容的传输方法及装置
CN102546832A (zh) * 2012-02-29 2012-07-04 北京快网科技有限公司 基于tcp协议的报文发送方法

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7099954B2 (en) * 2002-06-27 2006-08-29 Microsoft Corporation Congestion control mechanism for streaming media
WO2004084497A1 (en) * 2003-03-21 2004-09-30 Matsushita Electric Industrial Co., Ltd. Method and communication system for signalling information for optimising rate control schemes in wireless networks
US7826358B2 (en) * 2006-12-29 2010-11-02 Ellacoya Networks, Inc. Hierarchical virtual queuing
US7821936B2 (en) * 2007-07-27 2010-10-26 Avaya Inc. Systems and methods for partitioning end-to-end performance effects using network tomography
EP2359536B1 (en) * 2008-11-24 2016-03-16 Ankeena Networks, Inc., Adaptive network content delivery system
US8416684B2 (en) * 2010-12-14 2013-04-09 Verizon Patent And Licensing, Inc. Time and data rate policing
US9060252B2 (en) * 2012-07-31 2015-06-16 International Business Machines Corporation Rate adaptive transmission of wireless broadcast packets
CN110730105B (zh) * 2015-05-29 2021-09-28 腾讯科技（深圳）有限公司 图片数据传输方法、装置、设备及存储介质
US10887239B2 (en) * 2015-09-08 2021-01-05 Parallel Wireless, Inc. RAN for multimedia delivery
CN107171969B (zh) * 2016-03-07 2020-06-02 华为技术有限公司 一种数据传输方法、装置及系统
CN105847175A (zh) * 2016-04-21 2016-08-10 中国科学院信息工程研究所 数据中心网络中的应用层调度方法
CN117313825A (zh) * 2017-10-27 2023-12-29 渊慧科技有限公司 使用分布式优先化回放的强化学习
CN108235382A (zh) * 2017-11-27 2018-06-29 上海华为技术有限公司 一种传输速率调整的方法、节点设备以及服务器

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040044783A1 (en) * 2002-08-27 2004-03-04 Marko Nordberg Method and arrangement for tuning rate of media stream
CN1848810A (zh) * 2006-05-15 2006-10-18 武汉虹旭信息技术有限责任公司 一种流媒体发送速率控制方法
CN102457486A (zh) * 2010-10-19 2012-05-16 华为技术有限公司 一种媒体内容的传输方法及装置
CN102546832A (zh) * 2012-02-29 2012-07-04 北京快网科技有限公司 基于tcp协议的报文发送方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3780542A4

Also Published As

Publication number Publication date
US11558302B2 (en) 2023-01-17
US20210067453A1 (en) 2021-03-04
CN110875915B (zh) 2021-08-20
EP3780542A4 (en) 2021-03-31
EP3780542B1 (en) 2023-07-26
EP3780542A1 (en) 2021-02-17
CN110875915A (zh) 2020-03-10

Similar Documents

Publication Publication Date Title
WO2020048147A1 (zh) 数据传输方法及装置
JP4233884B2 (ja) サービス品質のプロービングを行う方法
JP5867160B2 (ja) 通信制御装置、通信制御方法および通信制御プログラム
CN111865781B (zh) 用于路径优化的方法、设备和计算机程序产品
WO2021103706A1 (zh) 控制数据包发送方法、模型训练方法、装置及系统
Wang et al. TCP congestion control algorithm for heterogeneous Internet
JP6274113B2 (ja) データ送信装置、データ送信方法、及びそのプログラム
US11533656B2 (en) Method of traffic and congestion control for a network with quality of service
Havey et al. Receiver driven rate adaptation for wireless multimedia applications
KR20180010531A (ko) 통신 시스템에서 전송 제어 프로토콜의 전송 버퍼 제어 방법 및 장치
JP5308364B2 (ja) 送信装置、送信方法及びプログラム
CN111314961A (zh) Tcp传输方法、装置和系统
US8289868B2 (en) Network device and method of measuring upstream bandwidth employed thereby
US10063489B2 (en) Buffer bloat control
JP6418169B2 (ja) 遅延最小値算出装置、情報送信装置、遅延最小値算出方法およびコンピュータプログラム
JPWO2014171543A1 (ja) データ送信装置、データ送信方法、及びそのプログラム
Torres-Jr et al. Packet reordering metrics to enable performance comparison in IP‐networks
JP6200870B2 (ja) データ転送制御装置、方法及びプログラム
Talau et al. Improving TCP performance over a common IoT scenario using the Early Window Tailoring method
KR102131427B1 (ko) Sctp에서의 혼잡 제어 방법 및 장치
Lee A path selection model considering path latency in the communication network with geographically correlated failures
JP6897769B2 (ja) データ送信装置、方法およびプログラム
JP2018067788A (ja) 受信トラヒックの高速化装置、高速化方法、および高速化プログラム
Mahmoodi et al. Optimal design of forward error correction for fairness maximisation among transmission control protocol flavours over wireless networks
JP4828555B2 (ja) ノード装置および帯域制御方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19858201

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019858201

Country of ref document: EP

Effective date: 20201103

NENP Non-entry into the national phase

Ref country code: DE