CN115037416A

CN115037416A - Data forward error correction processing method, device, electronic equipment and storage medium

Info

Publication number: CN115037416A
Application number: CN202210638549.8A
Authority: CN
Inventors: 马海寿
Original assignee: Guangzhou Cubesili Information Technology Co Ltd
Current assignee: Guangzhou Cubesili Information Technology Co Ltd
Priority date: 2022-06-07
Filing date: 2022-06-07
Publication date: 2022-09-09

Abstract

The application relates to the technical field of real-time audio and video processing, and provides a data forward error correction processing method and device, electronic equipment and a storage medium. The method and the device can adapt to the change of the network state in real time, effectively utilize network transmission resources and improve the transmission quality of data such as real-time audio and video streams and the like in various network states. The method comprises the following steps: the method comprises the steps of obtaining the storage state of an original data packet of a forward error correction queue, determining the number of redundant packets according to network delay and network packet loss information if the storage state of the original data packet meets the forward error correction operation condition represented by the network delay, generating redundant data packets corresponding to the original data packet in the forward error correction queue according to the number of the redundant packets, and sending the redundant data packets to a receiving end under the condition that the network transmission state meets the redundant packet sending condition so that the receiving end can apply the redundant data packets to recover the original data packet when the received original data packet has packet loss.

Description

Data forward error correction processing method, device, electronic equipment and storage medium

Technical Field

The present application relates to the field of real-time audio and video processing technologies, and in particular, to a data forward error correction processing method and apparatus, an electronic device, and a computer-readable storage medium.

Background

Forward Error Correction (FEC) restores a lost packet at a receiving end according to an original data packet and a redundant data packet by adding the redundant data packet for Error Correction to the original data packet at the transmitting end, so as to cope with a network environment with the lost packet and ensure transmission delay of data such as audio/video streaming media.

In the prior art, an original data packet and a redundant data packet in forward error correction processing are fixed, and the technical problems exist that the utilization rate of network transmission resources is not high when the network environment is good, and related parameters of the forward error correction processing cannot accurately update the network change when the network environment changes, so that the data transmission quality is influenced.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a data forward error correction processing method, apparatus, electronic device and computer readable storage medium.

In a first aspect, the present application provides a data forward error correction processing method. The method comprises the following steps:

acquiring the storage state of an original data packet of a forward error correction queue;

if the storage state of the original data packet meets the forward error correction operation condition represented by network time delay, determining the number of redundant packets according to the network time delay and network packet loss information;

generating a redundant data packet corresponding to the original data packet in the forward error correction queue according to the redundant packet number;

if the network transmission state meets the sending condition of the redundant data packet, sending the redundant data packet to a receiving end so that the receiving end can apply the redundant data packet to recover the original data packet when the original data packet received by the receiving end has packet loss.

In one embodiment, the original data packet storage state comprises a length of time that a data packet has been stored; before determining the number of redundant packets according to the network delay and the network packet loss information if the storage state of the original data packet meets the forward error correction operation condition represented by the network delay, the method further comprises:

determining a storage duration threshold according to the network delay smooth value; and if the storage duration of the data packet is larger than the storage duration threshold, determining that the storage state of the original data packet meets the forward error correction operation condition.

In one embodiment, the original packet storage status includes the number of stored packets; if the storage state of the original data packet meets the forward error correction operation condition represented by the network delay, before determining the number of redundant packets according to the network delay and the network packet loss information, the method further comprises:

obtaining a threshold value of the number of stored packets according to the ratio of the network delay smooth value to the unit time length of the data packet; and if the number of the stored data packets is larger than the threshold value of the number of the stored packets, determining that the storage state of the original data packet meets the forward error correction operation condition.

In an embodiment, the determining the number of redundant packets according to the network delay and the network packet loss information includes:

acquiring network delay statistical information and network packet loss statistical information; if the network delay statistical information represents that the network delay meets the delay threshold condition, setting the redundant packet number as a first preset value under the condition that the network packet loss statistical information represents that the network packet loss meets the network packet loss threshold condition, and determining the redundant packet number according to the network packet loss statistical information and a second preset value under the condition that the network packet loss statistical information represents that the network packet loss does not meet the network packet loss threshold condition; if the network delay statistical information represents that the network delay does not meet the delay threshold condition, setting the number of the redundant packets to be a third preset value under the condition that the network packet loss statistical information represents that the network packet loss meets the network packet loss threshold condition, and determining the number of the redundant packets according to the network packet loss statistical information and a second preset value under the condition that the network packet loss statistical information represents that the network packet loss does not meet the network packet loss threshold condition.

In an embodiment, the determining the number of redundant packets according to the network packet loss statistical information and a second preset value includes:

obtaining the maximum continuous packet loss number according to the network packet loss statistical information; and obtaining the redundant packet number according to the smaller value of the maximum continuous packet loss number and a second preset value.

In one embodiment, before sending the redundant data packet to a receiving end if the network transmission status satisfies the condition for sending the redundant packet, the method further includes:

acquiring network bandwidth and data transmission flow statistical information obtained by detection; and judging whether the network transmission state meets the sending condition of the redundant packet or not according to the network bandwidth and the data transmission flow statistical information.

In an embodiment, the determining whether the network transmission state meets the redundant packet sending condition according to the network bandwidth and the data transmission traffic statistic information includes:

if the network bandwidth is greater than or equal to the data transmission flow mean value in the data transmission flow statistical information, determining that the network transmission state meets the redundant packet sending condition; if the network bandwidth is smaller than the data transmission flow average value, determining that the network transmission state meets the sending condition of the redundant packet under the condition that the data transmission flow average value is smaller than or equal to a data transmission flow threshold value, or the data retransmission flow average value in the data transmission flow statistical information is smaller than or equal to a data retransmission flow threshold value.

In a second aspect, the present application further provides a data forward error correction processing apparatus. The device comprises:

the acquisition module is used for acquiring the storage state of the original data packet of the forward error correction queue;

the determining module is used for determining the number of redundant packets according to the network delay and the network packet loss information if the storage state of the original data packet meets the forward error correction operation condition represented by the network delay;

a generating module, configured to generate a redundant data packet corresponding to an original data packet in the fec queue according to the number of redundant packets;

and the sending module is used for sending the redundant data packet to a receiving end if the network transmission state meets the redundant packet sending condition so as to apply the redundant data packet to recover the original data packet when the original data packet received by the receiving end has packet loss.

In a third aspect, the present application further provides an electronic device. The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor realizes the following steps when executing the computer program:

acquiring the storage state of an original data packet of a forward error correction queue; if the storage state of the original data packet meets the forward error correction operation condition represented by network time delay, determining the number of redundant packets according to the network time delay and network packet loss information; generating a redundant data packet corresponding to the original data packet in the forward error correction queue according to the redundant packet number; and if the network transmission state meets the redundant packet sending condition, sending the redundant data packet to a receiving end so that the receiving end can apply the redundant data packet to recover the original data packet when the original data packet received by the receiving end has packet loss.

In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:

acquiring the storage state of an original data packet of a forward error correction queue; if the storage state of the original data packet meets the forward error correction operation condition represented by network time delay, determining the number of redundant packets according to the network time delay and network packet loss information; generating redundant data packets corresponding to the original data packets in the forward error correction queue according to the number of the redundant packets; if the network transmission state meets the sending condition of the redundant data packet, sending the redundant data packet to a receiving end so that the receiving end can apply the redundant data packet to recover the original data packet when the original data packet received by the receiving end has packet loss.

The data forward error correction processing method, the data forward error correction processing device, the electronic equipment and the storage medium acquire the storage state of an original data packet of a forward error correction queue, determine the number of redundant packets according to network delay and network packet loss information if the storage state of the original data packet meets forward error correction operation conditions represented by the network delay, generate redundant data packets corresponding to the original data packet in the forward error correction queue according to the number of the redundant packets, and transmit the redundant data packets to a receiving end under the condition that the network transmission state meets the transmission conditions of the redundant packets, so that the receiving end applies the redundant data packets to recover the original data packet when the received original data packet has packet loss. The scheme can determine the time for carrying out forward error correction operation according to the storage state of an original data packet and network time delay, determine the number of redundant packets according to the network time delay and network packet loss information, generate redundant data packets corresponding to the original data packets in a forward error correction queue according to the redundant data packets, and send the redundant data packets to a receiving end for packet loss recovery when the network transmission state meets the redundant packet sending condition, so that the method can adapt to the network state change in real time, effectively utilize network transmission resources, and improve the transmission quality of data such as real-time audio and video streams and the like in various network states.

Drawings

FIG. 1 is a diagram illustrating an application scenario of a data FEC processing method in an embodiment;

FIG. 2 is a flow chart illustrating a data FEC processing method according to an embodiment;

FIG. 3 is a flow diagram illustrating the determination of redundant packet counts in one embodiment;

FIG. 4 is a diagram of an application scenario of a data FEC processing method in an application example;

FIG. 5 is a block diagram of a data FEC processing apparatus in an embodiment;

FIG. 6 is a diagram illustrating an internal structure of an electronic device according to an embodiment;

fig. 7 is an internal structural view of an electronic apparatus in another embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The data forward error correction processing method provided by the present application may be applied to an application scenario as shown in fig. 1, where the application scenario may include a sending end 110 of data and a receiving end 120 of data, and the sending end 110 communicates with the receiving end 120 through a network. The sending end 110 and the receiving end 120 may be terminals or servers. Taking the transmitting end 110 as an example, the transmitting end 110 may be, but is not limited to, a terminal such as a personal computer, a smart phone, an internet of things device, a portable wearable device, and the like, wherein the internet of things device may be an intelligent sound box, an intelligent television, an intelligent air conditioner, an intelligent vehicle-mounted device, and the like, and the portable wearable device may be an intelligent watch, an intelligent bracelet, a head-mounted device, and the like; the sender 110 may also be a stand-alone server or a server cluster composed of multiple servers.

The data forward error correction processing method provided by the application can realize real-time adaptation to network state change, effectively utilizes network transmission resources to transmit data, and improves the transmission quality of the data in various network states. Specifically, as the multimedia technology is continuously developed, various real-time audio streaming media applications are gradually increased, and under the network change scenes such as the scenes of complex wireless environment, limited bandwidth and the like, packet loss occurs in the audio and video streaming media data transmission, so that the problems of interruption, blockage, poor real-time performance and the like of the audio and video streaming media occur, and the actual service experience is influenced.

The data forward error correction processing method of the present application is further described below with reference to the embodiments of the present application and the accompanying drawings.

In one embodiment, as shown in fig. 2, a data forward error correction processing method is provided, which may be performed by the transmitting end 110 shown in fig. 1, and includes the following steps:

step S201, obtaining the original data packet storage status of the forward error correction queue.

In this step, the sending end 110 obtains the current storage state of the original data packet entering the forward error correction queue, which is called the original data packet storage state of the forward error correction queue.

Specifically, the transmitting end 110 obtains an original data packet to be transmitted, which is continuously provided by a data source. The data source may be an audio source or a video source, and the original data packet to be sent refers to an original data packet to be sent from the sending end 110 to the receiving end 120, that is, the sending end 110 may obtain an original data packet continuously provided by the audio source. A forward error correction queue (may be referred to as an FEC queue) is provided in the transmitting end 110 for storing the original data packets continuously provided by the data source.

Then, the sender 110 may add the original data packets into the FEC queue, so that a series of original data packets to be sent, which are continuously provided by the data source, may be added into the FEC queue one by one for storage. The transmitting end 110 may record the number of original packets entering the FEC queue, the time of the head of queue packet entering the FEC queue, and the like to form enqueue record information. The current storage state refers to a storage state of an original data packet currently stored in the FEC queue, where the storage state, that is, the storage state of the original data packet, may include a time length for the data packet to be stored, a number of the stored data packets, and the like, and the storage state may be obtained by the sending end 110 according to the enqueue record information. Moreover, the storage status may be continuously updated by the sender 110, and if the sender 110 confirms that some original data packet or original data packets in the FEC queue have been received by the receiver 120, the original data packet or original data packets will be deleted from the FEC queue by the sender 110, at this time, the sender 110 may update the storage status of the original data packets, such as the time length of the stored data packets, the number of the stored data packets, and the like.

Step S202, if the storage state of the original data packet meets the forward error correction operation condition represented by the network delay, determining the number of the redundant packets according to the network delay and the network packet loss information.

The step is mainly to judge the forward error correction operation time and determine the number of redundant packets. The forward error correction operation is not performed at regular time, and the number of redundant packets is not a fixed value. Specifically, when the forward error correction operation condition is satisfied, forward error correction operation is performed, that is, the number of redundant packets is determined and corresponding redundant data packets are generated. The forward error correction operation condition is represented by a network delay, and may be specifically represented based on Round-Trip Time (RTT), and when the storage state of the original data packet obtained in step S201 satisfies the forward error correction operation condition represented by the Round-Trip delay, the sending end 110 determines the number of redundant packets, that is, the sending end 110 needs to determine whether FEC operation can be performed in combination with the current storage state of the original data packet and the current network delay. After the sending end 110 determines that the FEC operation can be performed, the sending end 110 determines the number of redundant packets according to the current network delay and the current network packet loss information, and for the number of source packets (i.e., the number of original data packets) performing the FEC operation, the number of original data packets currently stored in the FEC queue is taken, so that the sending end 110 adapts to the network environment change in real time on the judgment of the forward error correction operation time and the determination of the number of source packets and the number of redundant packets.

Step S203 generates a redundant data packet corresponding to the original data packet in the forward error correction queue according to the number of redundant packets.

In this step, the sending end 110 may perform FEC operation according to the redundancy packet number and the source packet number determined in step S202. Specifically, the sending end 110 obtains a source packet number M _ block and a redundant packet number N _ redundancy, and encodes the original data packets in the forward error correction queue by using the cauchy matrix to obtain encoded data [ d 1.,. dm, r 1.,. rn ], where M represents the source packet number M _ block, N represents the redundant packet number N _ redundancy, [ d 1.,. dm ] represents the original data packets in the forward error correction queue, and [ r 1.,. rn ] represents the redundant data packets obtained by FEC operation.

Step S204, if the network transmission state meets the redundant packet sending condition, sending the redundant data packet to the receiving end, so that the receiving end applies the redundant data packet to recover the original data packet when the received original data packet has packet loss.

In this step, a redundant packet is sent, the sending end 110 may obtain a redundant data packet in step S203, and at this time, the sending end 110 does not directly send the redundant data packet to the receiving end 120, and needs to first determine whether the current network transmission state meets the redundant packet sending condition. The sending end 110 may determine that the network transmission state satisfies the sending condition of the redundant packet and send the redundant data packet to the receiving end when the sending end 110 detects that the current network bandwidth satisfies the preset condition, so that the sending end 110 adapts to the network environment change in real time at the sending stage of the redundant packet to reasonably and fully utilize the network transmission resource. In a specific application, the sending end 110 may send the original data packet according to a conventional sending manner, and the receiving end 120 receives the original data packet and generates corresponding Acknowledgement (ACK) information according to a receiving state of the original data packet and feeds back the generated ACK information to the sending end 110.

In the data fec processing method of this embodiment, the sending end 110 may determine the time for performing the fec operation according to the storage state of the original data packet and the network delay, determine the number of redundant packets according to the network delay and the network packet loss information, generate the redundant data packet corresponding to the original data packet in the fec queue according to the determined number of redundant packets, and send the redundant data packet to the receiving end 120 for packet loss recovery when the network transmission state meets the redundant packet sending condition, so that the method can adapt to the network state change in real time, effectively utilize the network transmission resource, and improve the transmission quality of data such as real-time audio and video streams in various network states.

In some embodiments, the determining the timing of the forward error correction operation before step S202 may be performed by the following steps, including:

determining a storage duration threshold according to the network delay smooth value; and if the storage duration of the data packet is greater than the storage duration threshold, determining that the storage state of the original data packet meets the forward error correction operation condition.

In this embodiment, the original packet storage state includes a packet storage duration, where the packet storage duration refers to the stored duration of the original packet currently stored in the FEC queue, and the packet storage duration may be calculated according to a difference (T1_ now-T _ first FEC queue) between a current time T1_ now and a time T _ first FEC queue of a head-of-queue packet entering the FEC queue.

In this embodiment, the forward error correction operation condition is greater than the stored duration threshold TH _ TimeOut, which is determined according to the network delay smoothed value RTT _ smooth, and specifically, the network delay smoothed value RTT _ smooth may be set as the stored duration threshold TH _ TimeOut, and in practical application, an upper limit TimeOut _ max and a lower limit TimeOut _ min may be set for the stored duration threshold TH _ TimeOut, that is, the stored duration threshold TH _ TimeOut belongs to [ TimeOut _ min, TimeOut _ max ]. The sending end 110 may obtain an RTT sample RTT _ cur ═ T2_ now-T _ stamp according to the time T2_ now when the ACK is received and the time T _ stamp when the ACK is acknowledged, and then may use a first-order exponential smoothing manner to calculate the network delay smoothing value RTT _ smooth, that is, RTT _ smooth (T) · RTT _ cur + (1- α) · RTT _ smooth (T-1), RTT _ smooth (T) · smooth (T-1) are RTT smoothing values at the time T and T-1, respectively, the smoothing exponent α may select different parameters according to a mode of an application scenario, for example, a large smoothing exponent such as 0.2 is selected according to a mode of a low-delay scenario, and an initial value of RTT _ smooth may select a first received RTT value.

In this embodiment, the sending end 110 may obtain the stored time length of the data packet and the storage time length threshold, then compare the stored time length of the data packet with the storage time length threshold, and if the stored time length of the data packet is greater than the storage time length threshold (i.e., T1_ now-T _ first FEC queue > TH _ timeout), the sending end 110 determines that the storage state of the original data packet satisfies the forward error correction operation condition, that is, determines that the FEC operation may be performed, so as to adapt to the change of the network environment in real time at the determination stage of the FEC operation time.

In some other embodiments, the determining the forward error correction operation timing before step S202 may be performed by the following steps:

In this embodiment, the original data packet storage status includes the number of stored data packets L _ fecQueue, where the number of stored data packets L _ fecQueue refers to the number of original data packets currently stored in the FEC queue.

In this embodiment, the forward error correction condition is greater than the stored packet number threshold T _ bolcsize, the stored packet number threshold TH _ bolcsize is determined according to a ratio of the network delay smooth value RTT _ smooth to the packet unit time Duration _ pkt, specifically, a ratio [ RTT _ smooth/Duration _ pkt ] of the network delay smooth value RTT _ smooth to the packet unit time Duration _ pkt may be set as the stored packet number threshold TH _ bolcsize, that is, TH _ bolcsize [ RTT _ smooth/Duration _ pkt ], and in practical application, an upper limit value bolcsize _ min and a lower limit value bolcsize _ min, that is, TH _ bolcsize [ bolcsize _ min, bolcsize _ max ] may be set for the stored packet number threshold TH _ bolcsize. The method for calculating the network delay smooth value RTT _ smooth may be performed according to the foregoing embodiment, where the Duration _ pkt of a data packet unit refers to a Duration of each original data packet to be sent.

In this embodiment, the sending end 110 may obtain the number of stored data packets and the threshold of the number of stored packets, then compare the number of stored data packets with the threshold of the number of stored packets, and if the number of stored data packets is greater than the threshold of the number of stored packets (i.e., L _ fecQueue > TH _ bolcsize), the sending end 110 determines that the storage state of the original data packet satisfies the forward error correction operation condition, that is, determines that the FEC operation may be performed, so as to adapt to the change of the network environment in real time at the determination stage of the FEC operation time.

Further, in an embodiment, the determining of the forward error correction operation timing before step S202 may further be performed through the following steps, specifically including:

and if the stored time of the data packet is larger than the threshold of the storage time, or the number of the stored data packets is larger than the threshold of the number of the stored data packets, determining that the storage state of the original data packet meets the forward error correction operation condition.

In this embodiment, the sending end 110 determines that one of the stored length of the data packet is greater than the threshold of the storage length (T1_ now-T _ firstFecQueue > TH _ timeout), and the number of the stored data packets is greater than the threshold of the number of the stored data packets (L _ fecQueue > TH _ borcsize) is satisfied, and then the sending end 110 determines that the storage state of the original data packet satisfies the fec calculation condition and performs the fec calculation, so that the storage length, the storage amount, and the network delay information are integrated to accurately determine the fec calculation time to better adapt to a variable network environment.

In some embodiments, as shown in fig. 3, the determining the number of redundant packets according to the network delay and the network packet loss information in step S202 may include the following steps:

step S301, network delay statistical information and network packet loss statistical information are obtained.

The network delay statistical information refers to statistical information of network delay within a certain time, such as maximum and minimum network delay, and average network delay. The network packet loss statistical information refers to statistical information of network packet loss numbers in a certain time, such as maximum and minimum network packet loss numbers, average network packet loss number values, and the like. The network delay statistical information and the network packet loss statistical information are obtained in the step, so that the subsequent steps determine the redundant packet number N _ redundancy for FEC operation according to the network delay statistical information and the network packet loss statistical information, and the source packet number M _ block for FEC operation is removed from the stored data packet number L _ fecQueue of the FEC queue.

Step S302, judging whether the network time delay statistical information represents that the network time delay meets the time delay threshold value condition.

The step can judge whether the network delay meets the delay threshold value condition according to the network delay statistical information. Specifically, the network delay statistical information may include a maximum network delay value RTT _ max within a certain time, the sending end 110 determines whether the maximum network delay value RTT _ max is smaller than a preset network delay threshold value TH _ RTT, if the maximum network delay value RTT _ max is smaller than the network delay threshold value TH _ RTT (i.e., RTT _ max is smaller than TH _ RTT), the sending end 110 determines that the network delay statistical information indicates that the network delay satisfies the delay threshold condition, and if the maximum network delay value RTT _ max is greater than or equal to the network delay threshold value TH _ RTT (i.e., RTT _ max is greater than or equal to TH _ RTT), the sending end 110 determines that the network delay statistical information indicates that the network delay does not satisfy the delay threshold condition. Wherein, the network delay threshold TH _ rtt can adopt an empirical set value; for the maximum network delay value RTT _ max, the sender 110 may obtain an RTT sample RTT _ cur according to the time T2_ now when the ACK is received and the time T _ stamp when the ACK acknowledgement packet is received, where RTT _ cur is T2_ now-T _ stamp, and the RTT sample RTT _ cur is used as one RTT sample in a sliding time window T _ win, and the maximum value in the sliding time window T _ win is used as the maximum network delay value RTT _ max.

Step S3031, if the network delay statistical information indicates that the network delay meets the delay threshold condition, the number of the redundant packets is set to a first preset value under the condition that the network packet loss statistical information indicates that the network packet loss meets the network packet loss threshold condition, and the number of the redundant packets is determined according to the network packet loss statistical information and a second preset value under the condition that the network packet loss statistical information indicates that the network packet loss does not meet the network packet loss threshold condition.

In the step, whether the network packet loss statistical information representation network packet loss can meet the network packet loss threshold condition or not can be further judged under the condition that the maximum network delay value RTT _ max is smaller than the network delay threshold value TH _ RTT.

Specifically, the network packet loss statistical information may include a maximum continuous packet loss number ContLoss _ max within a certain time, and the sending end 110 further determines whether the maximum continuous packet loss number ContLoss _ max is less than or equal to a maximum continuous packet loss number threshold, where the maximum continuous packet loss number threshold may be 1, that is, if the maximum continuous packet loss number ContLoss _ max is less than or equal to 1, the sending end 110 determines that the network packet loss statistical information indicates that the network packet loss satisfies the network packet loss threshold condition, and if the maximum continuous packet loss number ContLoss _ max is greater than 1, the sending end 110 determines that the network packet loss statistical information indicates that the network packet loss does not satisfy the network packet loss threshold condition.

Further, under the condition that the network packet loss statistical information indicates that the network packet loss meets the network packet loss threshold condition, the sending end 110 sets the number of the redundant packets to a first preset value, where the first preset value may be 0, that is, under the condition that the maximum continuous packet loss number, ContLoss _ max, is less than or equal to 1, the sending end 110 may set the number of the redundant packets to 0; under the condition that the network packet loss statistical information indicates that the network packet loss does not satisfy the network packet loss threshold condition, that is, under the condition that the maximum continuous packet loss number ContLoss _ max is greater than 1, the sending end 110 may further determine the number of redundant packets according to the network packet loss statistical information and a second preset value.

For the maximum consecutive packet Loss number ContLoss _ max, when the sending end 110 receives the ACK message, the sending end 110 may forward count the packet Loss number stored in the packet number threshold TH _ block size interval according to the maximum packet number Seq _ max of the ACK, obtain a maximum packet Loss number Loss _ sample, use the maximum packet Loss number sample as one of the maximum packet Loss number samples in the sliding time window T _ win, and take the maximum value in the sliding time window T _ win as the maximum consecutive packet Loss number ContLoss _ max.

Step S3032, if the network delay statistical information indicates that the network delay does not satisfy the delay threshold condition, setting the number of the redundant packets to a third preset value under the condition that the network packet loss statistical information indicates that the network packet loss satisfies the network packet loss threshold condition, and determining the number of the redundant packets according to the network packet loss statistical information and the second preset value under the condition that the network packet loss statistical information indicates that the network packet loss does not satisfy the network packet loss threshold condition.

The step can be performed under the condition that the maximum value RTT _ max of the network delay is greater than or equal to the network delay threshold TH _ RTT, and then it is determined whether the network packet loss represented by the network packet loss statistical information can satisfy the network packet loss threshold condition, and the determination process can be performed according to the manner described in the above step. Therefore, when the network packet loss statistical information indicates that the network packet loss meets the network packet loss threshold condition, that is, the sending end 110 sets the number of the redundant packets to a third preset value, where the third preset value may be 1, that is, when the maximum continuous packet loss number, ContLoss _ max, is less than or equal to 1, the sending end 110 may set the number of the redundant packets to 1; under the condition that the network packet loss statistical information indicates that the network packet loss does not satisfy the network packet loss threshold condition, that is, under the condition that the maximum continuous packet loss number ContLoss _ max is greater than 1, the sending end 110 may further determine the number of redundant packets according to the network packet loss statistical information and a second preset value.

Further, in an embodiment, the determining the number of redundant packets according to the network packet loss statistical information and the second preset value in the step S3031 and the step S3032 specifically includes:

obtaining the maximum continuous packet loss number according to the network packet loss statistical information; and obtaining the redundant packet number according to the smaller value of the maximum continuous packet loss number and the second preset value.

In this embodiment, under the condition that the network packet loss statistical information indicates that the network packet loss does not satisfy the network packet loss threshold condition, the sending end 110 may obtain the maximum continuous packet loss number ContLoss _ max in the network packet loss statistical information, and obtain the redundant packet number according to a smaller value of the maximum continuous packet loss number ContLoss _ max and a second preset value. Specifically, the second preset value may be a minimum redundant packet number N _ min set according to experience, and when the maximum consecutive packet loss number ContLoss _ max is greater than 1, the sending end 110 sets the redundant packet number N _ redundancy to a smaller value of the maximum consecutive packet loss number ContLoss _ max and the second preset value N _ min, that is, the redundant packet number N _ redundancy is min (ContLoss _ max, N _ min).

The embodiment can determine the number of the redundant packets and the number of the source packets according to the RTT state and the packet loss state of the network in the FEC operation stage, thereby adapting to the network environment change in real time and adjusting the FEC parameters.

In some embodiments, the determination of the sending time of the redundant data packet may be performed before step S204 by the following steps, including:

In this embodiment, the sending end 110 determines whether the current network transmission state satisfies the sending condition of the redundant packet after obtaining the redundant data packet, and does not send the redundant data packet if the current network transmission state does not satisfy the sending condition of the redundant packet. Specifically, the sending end 110 obtains the statistical information of the network bandwidth and the data transmission flow obtained by the detection, wherein, the sending end 110 can obtain the network bandwidth by detecting a conventional network bandwidth detection algorithm, such as a bottleneck bandwidth-Round-trip delay method (BBR), Google Congestion Control (GCC), etc., and can also count data transmission traffic statistical information within a time period T _ duration, such as a data transmission traffic mean value, a data retransmission traffic mean value, etc., in practical applications, the sending end 110 may obtain the uplink data network bandwidth, the average uplink data traffic, and the average uplink retransmission traffic, and judging whether the network transmission state meets the sending condition of the redundant packet or not according to the data uplink network bandwidth, the data uplink flow average value and the uplink retransmission flow average value, thereby reasonably and fully utilizing network transmission resources.

In an embodiment, further, the determining whether the network transmission state meets the redundant packet sending condition according to the network bandwidth and the data transmission traffic statistical information in the above embodiment may specifically include:

if the network bandwidth is greater than or equal to the data transmission flow mean value in the data transmission flow statistical information, determining that the network transmission state meets the sending condition of the redundant packet; and if the network bandwidth is smaller than the data transmission flow average value, determining that the network transmission state meets the redundant packet sending condition under the condition that the data transmission flow average value is smaller than or equal to the data transmission flow threshold value or the data retransmission flow average value in the data transmission flow statistical information is smaller than or equal to the data retransmission flow threshold value.

In this embodiment, the sending end 110 may first determine whether the network bandwidth of the data uplink is sufficient, may send a redundant data packet if the network bandwidth is sufficient, may further determine whether the uplink traffic and the uplink retransmission of the data are simultaneously limited if the network bandwidth is not sufficient, may not send the redundant data packet if the uplink traffic and the uplink retransmission are simultaneously limited, and may send the redundant data packet otherwise.

Specifically, if the BW _ probe of the uplink network bandwidth of the data detected by the sending end 110 is greater than or equal to the data transmission Flow mean value Flow _ avg in the data transmission Flow statistical information, that is, the BW _ probe is greater than or equal to the Flow _ avg, the sending end 110 determines that the network bandwidth of the uplink network bandwidth of the data is abundant, that is, the network transmission state meets the sending condition of the redundant packet; if BW _ probe is less than Flow _ avg, the sending end 110 determines that the network bandwidth of the data uplink is not sufficient, and at this time, the sending end 110 further determines whether the uplink Flow and the uplink retransmission of the data are simultaneously limited.

If the sending end 110 determines that the data transmission Flow mean value Flow _ avg is greater than the data transmission Flow threshold value uploadjlimit, that is, Flow _ avg > uploadjlimit, the sending end 110 determines that the uplink Flow is limited, where the data transmission Flow threshold value uploadjlimit is the data coding code rate encoderrate × the preset Multiple, and the experience value of the preset Multiple may be 2; if the sending end 110 determines that the average ResendFlow _ avg of the data retransmission Flow in the data transmission Flow statistic information is greater than a Resend Flow threshold value Resend _ limit, that is, ResendFlow _ avg > Resend _ limit, the sending end 110 determines that uplink retransmission is limited, where the Resend Flow threshold value Resend _ limit is Flow _ avg × Resend _ multiple, Resend _ multiple is a preset retransmission limit multiple, and the Resend Flow threshold value Resend _ limit belongs to [ a lower limit value Resend _ min of a Resend Flow threshold value Resend _ max of a Resend Flow threshold value ].

Therefore, if the sending end 110 determines that the average value of the data transmission Flow _ avg is less than or equal to the data transmission Flow threshold value uploadjlimit, or the average value of the data retransmission Flow resndflow _ avg is less than or equal to the data retransmission Flow threshold value resendjlimit, the sending end 110 may determine that the uplink Flow and the uplink retransmission of the data are not limited at the same time, and determine that the network transmission state meets the redundant packet sending condition.

In the aspect of practical application, the data forward error correction processing method provided by the application can be applied to processing of real-time audio and video data of network live broadcast. Specifically, as shown in fig. 1, the sending end 110 may be a server for live webcasting, the receiving end 120 may be a terminal used by a viewer in live webcasting, and the sending end 110 is configured to send real-time audio and video data of a main broadcast in live webcasting to the receiving end 120. By applying the data forward error correction processing method provided by the application, in live network broadcast, the sending end 110 can determine the opportunity for performing FEC operation according to the network state and the state of the source packet of real-time audio and video data, determine the source packet number and the redundant packet number of FEC coding according to the network packet loss state and the round-trip delay, generate corresponding redundant packets, and determine whether to send the FEC redundant packets to the receiving end 120 according to network bandwidth information, after the receiving end 120 receives the packets, the receiving end 120 can record the network packet loss state information, perform forward error correction decoding when packet loss events occur, and feed back the receiving state information to the sending end 110, the sending end 110 can timely adjust the network state and update FEC queue information, thereby realizing real-time adaptation to the network state change in the live network broadcast process, timely tracking the change of the network environment, adjusting the forward error correction parameters of the real-time audio and video data, and effectively utilizing network transmission resources to transmit the live network broadcast real-time audio and video data, the transmission quality of the real-time audio and video data of the live network in various network states is improved, the problems of interruption, blockage, poor real-time performance and the like of the live network audio and video caused by factors such as complex wireless environment, limited bandwidth and the like are avoided, and the service experience of the live network is improved.

The data forward error correction processing method provided by the present application is described below with reference to an application example, as shown in fig. 4, an application scenario diagram of the example is shown, where the application scenario diagram includes an audio stream transmitting end 410 and an audio stream receiving end 420, and the audio stream transmitting end 410 may include an audio source, a network status tracking unit, an FEC operation unit, and a transmitting unit.

An FEC operation unit in the audio stream sending end 410 stores original data packets (or called source packets and audio packets) output by an audio source, determines a time for performing FEC operation, determines a source packet number and a redundant packet number for the FEC operation, and finally performs the FEC operation to generate redundant data packets (or called redundant packets). Specifically, the FEC operation unit provides an FEC queue, stores source packets output by an audio source, adds the source packets output by the audio source into the FEC queue, and records the number L _ fecQueue of original data packets entering the FEC queue and the time T _ firstFecQueue of a head-of-line audio packet entering the FEC queue. And the FEC operation unit determines whether FEC operation can be carried out according to the storage state of the original data packet in the FEC queue and the round trip time RTT. When one of the conditions (1) and (2) below is satisfied, an FEC operation is performed. The length of the stored data packet in condition (1) is greater than the storage duration threshold, that is, the difference between the current time T1_ now and the time T _ first fecqueue of the head audio packet is greater than the storage duration threshold TH _ timeout, that is, T1_ now-T _ first fecqueue > TH _ timeout, where the storage duration threshold TH _ timeout is the network delay smooth value RTT _ smooth, and the network delay smooth value RTT _ smooth may be calculated by the network state tracking unit. The condition (2) is that the number of stored packets L _ fecQueue is greater than the stored packet number threshold TH _ bolcsize, i.e., L _ fecQueue > TH _ bolcsize, where the stored packet number threshold TH _ bolcsize ═ is [ network delay smoothing RTT _ smooth/Duration of one audio packet Duration _ pkt ].

After determining that the FEC operation can be performed, the FEC operation unit determines, according to the RTT state and the packet loss state of the network, a source packet number M _ block and a redundant packet number N _ redundancy for performing the FEC operation. The source packet number M _ block is the original data packet number L _ fecQueue of the FEC queue, that is, M _ block is L _ fecQueue. The determination method of the redundant packet number N _ redundancy is as follows:

when the maximum round-trip delay RTT _ max is smaller than the set round-trip delay threshold TH _ RTT, that is, RTT _ max is smaller than TH _ RTT, if the maximum continuous packet loss number ContLoss _ max is less than or equal to 1, the redundant packet number N _ redundancy is 0; if the maximum consecutive packet loss number ContLoss _ max > 1, the redundant packet number N _ redundancy is min (ContLoss _ max, N _ min).

When the maximum round-trip delay RTT _ max is greater than or equal to the set round-trip delay threshold TH _ RTT, namely RTT _ max is greater than or equal to TH _ RTT, if the maximum continuous packet loss Contloss _ max is less than or equal to 1, the redundant packet number N _ redundancy is 1; if the maximum number of consecutive lost packets, ContLoss _ max > 1, the number of redundant packets, N _ redundancy, is min (ContLoss _ max, N _ min).

The maximum round-trip delay RTT _ max and the maximum continuous packet loss count ContLoss _ max can be obtained from the network state tracking unit, and the round-trip delay threshold TH _ RTT and the minimum redundant packet count N _ min can be experience setting values.

After the FEC operation unit in the audio stream transmitting end 410 determines and obtains the source packet number M _ block and the redundant packet number N _ redundancy, the original data packet may be encoded using a cauchy matrix, and obtain encoded data [ d1,. multidot.,. dm, r1,. multidot.,. rn ], where M represents the source packet number M _ block, N represents the redundant packet number N _ redundancy, [ d1,. multidot.,. dm ] represents the original data packet, and [ r1,. multidot.,. rn ] represents the redundant data packet obtained by the FEC operation.

A sending unit in the audio stream sending end 410 may provide functions of sending traffic statistics and network bandwidth detection, and determine whether to send a redundant packet generated by FEC according to a network state, and the sending unit may add attribute information to a source packet and the redundant packet and send the source packet and the redundant packet to the audio stream receiving end 420 through a User Datagram Protocol (UDP).

Specifically, the sending unit supports the traffic statistics and the network Bandwidth detection function, may count an uplink audio traffic average value Flow _ avg and an uplink retransmission traffic average value ResendFlow _ avg in a statistical time period T _ duration, and may also detect and obtain a network Bandwidth BW _ probe using a network Bandwidth detection algorithm, such as a Bottleneck Bandwidth-Round-trip delay method (BBR), Google Congestion Control (GCC), and the like.

Wherein, the sending unit decides whether to send the redundant packet according to the network state. The sending unit firstly judges whether the bandwidth of the audio uplink network is abundant, and sends a redundant packet if the bandwidth is abundant; if the bandwidth is not sufficient, whether uplink audio flow is limited and uplink retransmission is limited needs to be further judged, if the uplink audio flow and the uplink retransmission are limited at the same time, the redundant packet is not sent, and if the uplink audio flow and the uplink retransmission are not limited at the same time, the redundant packet can be sent. Specifically, if the network bandwidth BW _ probe is smaller than the average upstream audio traffic Flow _ avg, it indicates that the upstream bandwidth is not sufficient, otherwise, it indicates that the upstream bandwidth is sufficient. And if the uplink audio Flow average value Flow _ avg is greater than a set uplink audio Flow threshold value uploadjlimit, it is indicated that the uplink audio Flow is limited, where the uplink audio Flow threshold value uploadjlimit is equal to the audio coding rate EncodeRate × the preset Multiple, and an empirical value of the preset Multiple may be 2. And if the uplink audio Flow average value Flow _ avg is greater than a set uplink retransmission Flow threshold value Resend _ limit, indicating that uplink retransmission is limited, wherein the uplink retransmission Flow threshold value Resend _ limit is the uplink audio Flow average value Flow _ avg multiplied by Resend _ multiple, and Resend _ multiple is a preset uplink retransmission limit multiple.

A transmitting unit in the audio stream transmitting end 410 may transmit a source packet and a redundant packet using a UDP protocol, the transmitted packet is marked as a source packet or a redundant packet, the source packet may add attribute information such as a packet sequence number and a transmission time, and the redundant packet may add attribute information such as a source packet number M _ block used in FEC operation, a redundant packet number N _ redundancy, a start source packet sequence number Seq _ start participating in FEC operation, and a Group sequence number Group _ id of FEC operation, so that the audio stream receiving end 420 may count a packet receiving state and perform FEC recovery after receiving the packet.

The audio stream receiving end 420 may generate Acknowledgement (ACK) information according to the packet receiving state, feed the ACK information back to the audio stream sending end 410, and determine whether FEC decoding can be performed according to the packet loss state to recover the source packet. Specifically, after the audio stream receiving end 420 successfully receives the packet, it counts the packet receiving status within a segment of packet sequence number interval according to the sequence number of the transmitted packet, and marks whether the packet with each sequence number is successfully received bit by bit, generates the packet receiving status RecvStatus, and feeds back the ACK information to the audio stream transmitting end 410. The audio stream receiving end 420 may add the received source packet and the redundant packet to the receiving queue, and if packet loss is found, combine attribute information such as a source packet number M _ block, a redundant packet number N _ redundancy, a starting source packet sequence number Seq _ start participating in FEC operation, and a Group sequence number Group _ id of FEC operation in the redundant packet, and if the lost packet satisfies a Group of FEC decoding recovery conditions, the source packet may be recovered by using a corresponding FEC decoding algorithm.

The function of the network state tracking unit in the audio stream sending end 410 is to track the packet loss and round trip time RTT state of the network, and update the FEC queue state in the FEC operation unit in time according to the ACK information.

Specifically, the network state tracking unit tracks a packet loss state and a round trip time RTT state of the network according to the received ACK information. Wherein, the calculation of the maximum continuous packet loss count ContLoss _ max by the network state tracking unit is as follows: when receiving the ACK information, counting the packet Loss number in the TH _ blockSize interval forward sequence by sequence according to the maximum packet sequence number Seq _ max of the ACK to obtain a maximum packet Loss number Loss _ sample, taking the maximum packet Loss number Loss _ sample as a sample in a sliding time window T _ win, and taking the maximum value in the sliding time window T _ win as the maximum continuous packet Loss number Contloss _ max. The network state tracking unit calculates the maximum round trip time delay RTT _ max in the following way: the network state tracking unit can obtain one RTT sample RTT _ cur which is T2_ now-T _ stamp according to the time T2_ now of receiving the ACK information and the time T _ stamp of the ACK confirmation packet, and the RTT _ cur is used as one sample in a sliding time window T _ win, and the maximum value in the sliding time window T _ win is used as the maximum round-trip delay RTT _ max. The network state tracking unit may also perform calculation of a network delay smoothing value RTT _ smooth: the network state tracking unit calculates a network delay smooth value RTT _ smooth by using a first-order exponential smoothing manner, that is, RTT _ smooth (t) is α · RTT _ cur + (1- α) · RTT _ smooth (t-1), RTT _ smooth (t) and RTT _ smooth (t-1) are RTT smooth values at time t and t-1, respectively, where the smoothing exponent α may select different parameters according to a mode of an application scenario, for example, a low latency scenario mode selects a larger smoothing exponent such as 0.2, and an initial value of RTT _ smooth may select a first received RTT value.

And the network state tracking unit in the audio stream sending end 410 updates the source packet state in the FEC queue according to the ACK information. Specifically, after the source packet has been successfully received by the audio stream receiving end 420, the source packet is deleted from the FEC queue, and at the same time, the time T _ first fecqueue of the head-of-line audio packet in the FEC queue is updated, and each time a source packet is successfully received, T _ first fecqueue is shifted backward by the Duration of one source packet, that is, T _ first fecqueue ═ T _ first fecqueue + Duration _ pkt.

In this application example, the audio stream sending end 410 may determine a time for performing FEC calculation according to a network state and a state of an audio source packet, determine a source packet number and a redundant packet number of FEC coding according to a network packet loss state and a round-trip delay, generate a redundant packet, and determine whether to send an FEC redundant packet according to network bandwidth information, after the audio stream receiving end 420 receives the packet, record network packet loss state information, perform forward error correction decoding in the event of packet loss, and feed back the received state information to the audio stream sending end 410, the audio stream sending end 410 adjusts a network state and updates FEC queue information in time, so as to adapt to a network state change in real time, effectively utilize network transmission resources, improve transmission quality of real-time audio streams in various network scenes, and improve actual service experience.

It should be understood that, although the steps in the flowcharts related to the embodiments as described above are sequentially displayed as indicated by arrows, the steps are not necessarily performed sequentially as indicated by the arrows. The steps are not limited to being performed in the exact order illustrated and, unless explicitly stated herein, may be performed in other orders. Moreover, at least a part of the steps in the flowcharts related to the embodiments described above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the execution order of the steps or stages is not necessarily sequential, but may be rotated or alternated with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the present application further provides a data forward error correction processing apparatus for implementing the data forward error correction processing method. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme described in the above method, so specific limitations in one or more embodiments of the data forward error correction processing apparatus provided below may refer to the limitations in the foregoing data forward error correction processing method, and details are not described here.

In one embodiment, as shown in fig. 5, a data forward error correction processing apparatus is provided, and the apparatus 500 may include:

an obtaining module 501, configured to obtain a storage state of an original data packet of a forward error correction queue;

a determining module 502, configured to determine, if the storage state of the original data packet meets a forward error correction operation condition represented by a network delay, a redundant packet number according to the network delay and network packet loss information;

a generating module 503, configured to generate a redundant data packet corresponding to the original data packet in the fec queue according to the number of redundant packets;

a sending module 504, configured to send the redundant data packet to a receiving end if the network transmission state meets a redundant packet sending condition, so that the receiving end applies the redundant data packet to recover the original data packet when the original data packet received by the receiving end has a packet loss.

In one embodiment, the original data packet storage state comprises a length of time that a data packet has been stored; the determining module 502 is further configured to determine a storage duration threshold according to the network delay smooth value; and if the storage duration of the data packet is larger than the storage duration threshold, determining that the storage state of the original data packet meets the forward error correction operation condition.

In one embodiment, the original packet storage status includes the number of stored packets; the determining module 502 is further configured to obtain a threshold of the number of stored packets according to a ratio of the network delay smooth value to the unit duration of the data packet; and if the number of the stored data packets is larger than the threshold value of the number of the stored packets, determining that the storage state of the original data packet meets the forward error correction operation condition.

In an embodiment, the determining module 502 is further configured to obtain network delay statistical information and network packet loss statistical information; if the network delay statistical information represents that the network delay meets the delay threshold condition, setting the redundant packet number as a first preset value under the condition that the network packet loss statistical information represents that the network packet loss meets the network packet loss threshold condition, and determining the redundant packet number according to the network packet loss statistical information and a second preset value under the condition that the network packet loss statistical information represents that the network packet loss does not meet the network packet loss threshold condition; if the network delay statistical information represents that the network delay does not meet the delay threshold condition, setting the number of the redundant packets to be a third preset value under the condition that the network packet loss statistical information represents that the network packet loss meets the network packet loss threshold condition, and determining the number of the redundant packets according to the network packet loss statistical information and a second preset value under the condition that the network packet loss statistical information represents that the network packet loss does not meet the network packet loss threshold condition.

In an embodiment, the determining module 502 is further configured to obtain a maximum number of consecutive packets lost according to the network packet loss statistical information; and obtaining the redundant packet number according to the smaller value of the maximum continuous packet loss number and a second preset value.

In an embodiment, the sending module 504 is further configured to obtain statistical information of network bandwidth and data transmission traffic obtained through detection; and judging whether the network transmission state meets the sending condition of the redundant packet or not according to the network bandwidth and the data transmission flow statistical information.

In an embodiment, the sending module 504 is further configured to determine that the network transmission status meets the redundant packet sending condition if the network bandwidth is greater than or equal to a data transmission traffic mean value in the data transmission traffic statistical information; if the network bandwidth is smaller than the data transmission flow average value, determining that the network transmission state meets the sending condition of the redundant packet under the condition that the data transmission flow average value is smaller than or equal to a data transmission flow threshold value, or the data retransmission flow average value in the data transmission flow statistical information is smaller than or equal to a data retransmission flow threshold value.

The modules in the data forward error correction processing apparatus can be wholly or partially implemented by software, hardware and a combination thereof. The modules may be embedded in a hardware form or may be independent of a processor in the electronic device, or may be stored in a memory in the electronic device in a software form, so that the processor calls and executes operations corresponding to the modules.

In one embodiment, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 6. The electronic device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the electronic device may be used to communicate with an external terminal through a network connection. The computer program is executed by a processor to implement a data forward error correction processing method.

In one embodiment, an electronic device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 7. The electronic device comprises a processor, a memory, a communication interface, a display screen and an input device which are connected through a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the electronic device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a data forward error correction processing method. The display screen of the electronic equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the electronic equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the configurations shown in fig. 6 and 7 are only block diagrams of some configurations relevant to the present disclosure, and do not constitute a limitation on the electronic devices to which the present disclosure may be applied, and a particular electronic device may include more or less components than those shown in the figures, or may combine certain components, or have a different arrangement of components.

In one embodiment, an electronic device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:

In one embodiment, the original data packet storage state comprises a length of time that a data packet has been stored; the processor, when executing the computer program, further performs the steps of:

In one embodiment, the original packet storage status includes the number of stored packets; the processor, when executing the computer program, further performs the steps of:

obtaining a storage packet number threshold according to the ratio of the network delay smooth value to the unit duration of the data packet; and if the number of the stored data packets is larger than the threshold value of the number of the stored packets, determining that the storage state of the original data packet meets the forward error correction operation condition.

In one embodiment, the processor when executing the computer program further performs the steps of:

In one embodiment, the processor, when executing the computer program, further performs the steps of:

if the network bandwidth is greater than or equal to the data transmission flow mean value in the data transmission flow statistical information, determining that the network transmission state meets the redundant packet sending condition; if the network bandwidth is smaller than the data transmission flow average value, determining that the network transmission state meets the redundant packet sending condition under the condition that the data transmission flow average value is smaller than or equal to a data transmission flow threshold value, or the data retransmission flow average value in the data transmission flow statistical information is smaller than or equal to a data retransmission flow threshold value.

In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:

In one embodiment, the original data packet storage state comprises a length of time that a data packet has been stored; the computer program when executed by the processor further realizes the steps of:

In one embodiment, the original packet storage status includes the number of stored packets; the computer program when executed by the processor further realizes the steps of:

In one embodiment, the computer program when executed by the processor further performs the steps of:

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory or other media used in the embodiments provided herein can include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), Magnetic Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.

All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A data forward error correction processing method, the method comprising:

generating redundant data packets corresponding to the original data packets in the forward error correction queue according to the number of the redundant packets;

and if the network transmission state meets the redundant packet sending condition, sending the redundant data packet to a receiving end so that the receiving end can apply the redundant data packet to recover the original data packet when the original data packet received by the receiving end has packet loss.

2. The method of claim 1, wherein the original packet storage status comprises a length of time that the packet has been stored; before determining the number of redundant packets according to the network delay and the network packet loss information if the storage state of the original data packet meets the forward error correction operation condition represented by the network delay, the method further comprises:

determining a storage duration threshold according to the network delay smooth value;

and if the storage duration of the data packet is larger than the storage duration threshold, determining that the storage state of the original data packet meets the forward error correction operation condition.

3. The method of claim 1, wherein the original packet storage status comprises a number of stored packets; before determining the number of redundant packets according to the network delay and the network packet loss information if the storage state of the original data packet meets the forward error correction operation condition represented by the network delay, the method further comprises:

obtaining a threshold value of the number of stored packets according to the ratio of the network delay smooth value to the unit time length of the data packet;

and if the number of the stored data packets is larger than the threshold value of the number of the stored packets, determining that the storage state of the original data packet meets the forward error correction operation condition.

4. The method of claim 1, wherein the determining the number of redundant packets according to the network latency and the network packet loss information comprises:

acquiring network delay statistical information and network packet loss statistical information;

if the network delay statistical information represents that the network delay meets the delay threshold condition, setting the redundant packet number as a first preset value under the condition that the network packet loss statistical information represents that the network packet loss meets the network packet loss threshold condition, and determining the redundant packet number according to the network packet loss statistical information and a second preset value under the condition that the network packet loss statistical information represents that the network packet loss does not meet the network packet loss threshold condition;

if the network delay statistical information represents that the network delay does not meet the delay threshold condition, setting the number of the redundant packets to be a third preset value under the condition that the network packet loss statistical information represents that the network packet loss meets the network packet loss threshold condition, and determining the number of the redundant packets according to the network packet loss statistical information and a second preset value under the condition that the network packet loss statistical information represents that the network packet loss does not meet the network packet loss threshold condition.

5. The method according to claim 4, wherein the determining the number of redundant packets according to the network packet loss statistic information and a second preset value comprises:

obtaining the maximum continuous packet loss number according to the network packet loss statistical information;

and obtaining the redundant packet number according to the smaller value of the maximum continuous packet loss number and a second preset value.

6. The method according to any one of claims 1 to 5, wherein before sending the redundant data packet to a receiving end if the network transmission status satisfies the redundant packet sending condition, the method further comprises:

acquiring network bandwidth and data transmission flow statistical information obtained by detection;

and judging whether the network transmission state meets the sending condition of the redundant packet or not according to the network bandwidth and the data transmission flow statistical information.

7. The method of claim 6, wherein the determining whether the network transmission status satisfies the redundant packet sending condition according to the network bandwidth and data transmission traffic statistics information comprises:

if the network bandwidth is greater than or equal to the data transmission flow mean value in the data transmission flow statistical information, determining that the network transmission state meets the redundant packet sending condition;

if the network bandwidth is smaller than the data transmission flow average value, determining that the network transmission state meets the redundant packet sending condition under the condition that the data transmission flow average value is smaller than or equal to a data transmission flow threshold value, or the data retransmission flow average value in the data transmission flow statistical information is smaller than or equal to a data retransmission flow threshold value.

8. A data forward error correction processing apparatus, characterized in that the apparatus comprises:

a generating module, configured to generate a redundant data packet corresponding to an original data packet in the forward error correction queue according to the number of redundant packets;

9. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor realizes the steps of the method of any of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.