WO2017157303A1 - 实时通信中的抗丢包方法、装置和系统 - Google Patents

实时通信中的抗丢包方法、装置和系统 Download PDF

Info

Publication number
WO2017157303A1
WO2017157303A1 PCT/CN2017/076758 CN2017076758W WO2017157303A1 WO 2017157303 A1 WO2017157303 A1 WO 2017157303A1 CN 2017076758 W CN2017076758 W CN 2017076758W WO 2017157303 A1 WO2017157303 A1 WO 2017157303A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
frame
fec
packet loss
receiving end
Prior art date
Application number
PCT/CN2017/076758
Other languages
English (en)
French (fr)
Inventor
丁仕林
林亚
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2017157303A1 publication Critical patent/WO2017157303A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1812—Hybrid protocols; Hybrid automatic repeat request [HARQ]
    • H04L1/1819—Hybrid protocols; Hybrid automatic repeat request [HARQ] with retransmission of additional or different redundancy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
    • H04L1/0009—Systems modifying transmission characteristics according to link quality, e.g. power backoff by adapting the channel coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • H04L1/0041—Arrangements at the transmitter end
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems

Definitions

  • the present invention relates to the field of communications technologies, and in particular, to a method, device and system for anti-dropping in real-time communication.
  • UDP User Datagram Protocol
  • FEC Forward Error Correction
  • retransmission retransmission
  • interleaving the interleaving delay is large, and when the packet loss is caused by congestion, the blind retransmission will aggravate the network congestion. Therefore, the application of retransmission and interleaving in the real-time communication system is limited.
  • FEC is usually used for anti-dropping in real-time communication. Since FEC achieves anti-dropping by increasing redundancy, it will cause congestion to be further aggravated when the network is congested. Moreover, in the era of mobile Internet, another negative effect of increasing redundancy through FEC is that expensive communication costs are incurred because too much FEC will consume a large amount of network traffic. If the redundancy of the FEC is high, the packet loss can be recovered to the maximum extent, but the bandwidth of the redundant data is very high. On the contrary, if the redundancy of the FEC is low, the full recovery of all packets cannot be guaranteed, resulting in video. The display shows problems such as a flower screen and reduces the user experience.
  • the common method to solve this problem is to determine the FEC redundancy level according to the packet loss rate of the network statistics.
  • the packet loss rate of the network feedback is generally based on historical statistics, it cannot fully reflect the current network condition, so the FEC redundancy at this time.
  • the level corresponds to the network average status of historical time. Under the current network conditions, all packages cannot be guaranteed to be restored. Therefore, FEC cannot achieve the expected effect, and thus the problem cannot be fundamentally solved.
  • the main purpose of the embodiments of the present invention is to provide an anti-lost method, device and system in real-time communication, which aims to realize adaptive processing of packet loss occurring in a data transmission network in a real-time communication process, and enhance the real-time communication process. Anti-lost ability.
  • the anti-drop method in real-time communication includes:
  • the network data packet includes the encoded video frame data and the FEC redundant data, where the FEC redundant data is determined according to a network packet loss rate, and the corresponding multiple sets of FEC redundancy levels are determined;
  • the corresponding level of FEC redundancy data is reselected from the preset multiple sets of redundancy level FEC redundancy data. Send to the receiving end.
  • the method further includes:
  • the method further includes:
  • the feedback data is decoding error feedback data
  • the corresponding independently decodable frame corresponding to the decoding error frame is sent to the receiving end.
  • the method before the step of sending, by the sending end, the network data packet to the receiving end, the method further includes:
  • the captured video frame is encoded by the video encoder, and the encoded video frame data is FEC-encoded to generate multiple sets of redundant level FEC redundant data, and the key frame encoder performs key frame on the collected video frame data.
  • the encoding generates an independently decodable frame, wherein the bandwidth occupied by the frame data generated by the key frame encoder is smaller than the bandwidth occupied by the frame data generated by the video encoder.
  • the packet loss model includes the number of media packets currently lost, and the currently lost FEC.
  • the number of packets and/or the packet loss sequence number; the decoding error model includes the frame number of the decoded error, the proportion of the error macroblock, and/or the location of the error macroblock.
  • the embodiment of the invention further provides an anti-drop packet device in real-time communication, wherein the device includes:
  • the sending module is configured to send a network data packet to the receiving end, where the network data packet includes the encoded video frame data and the FEC redundant data, and the FEC redundant data is determined according to the network packet loss rate, and the corresponding multiple groups of FEC redundancy are determined. Remaining level
  • a receiving module configured to receive feedback data sent by the receiving end after parsing and restoring the network data packet
  • the packet loss processing module when the feedback data is packet loss feedback data, reselecting the corresponding level from the preset multiple sets of redundancy level FEC redundancy data according to the current packet loss model of the packet loss feedback data FEC redundant data is sent to the receiving end.
  • the device further comprises:
  • the determining module is configured to determine the timeliness of the retransmitted FEC redundant data. If the current aging requirement is met, the packet loss processing module sends the reselected corresponding FEC redundant data to the receiving end.
  • the device further comprises:
  • the decoding error processing module is configured to: when the feedback data is decoding error feedback data, obtain an independently decodable frame corresponding to the decoding error frame and send the data to the receiving end according to the current decoding error model of the decoding error feedback data.
  • the device further comprises:
  • the encoding module encodes the collected video frame by the video encoder, performs FEC encoding on the encoded video frame data, generates multiple sets of redundant level FEC redundant data, and simultaneously collects the collected video frame data by the key frame encoder.
  • the key frame coding is performed to generate an independently decodable frame, wherein the bandwidth occupied by the frame data generated by the key frame encoder is smaller than the bandwidth occupied by the frame data generated by the video encoder.
  • the packet loss model includes the number of media packets currently lost, and the currently lost FEC.
  • the number of packets and/or the packet loss sequence number; the decoding error model includes the frame number of the decoded error, the proportion of the error macroblock, and/or the location of the error macroblock.
  • the embodiment of the invention further provides an anti-lost system in real-time communication, which comprises: a receiving end and a transmitting end, wherein:
  • the transmitting end includes the device as described above;
  • the receiving end is configured to receive a network data packet sent by the sending end, parse the FEC redundant data in the network data packet, recover the lost video frame data in the network data packet, and determine all video frames. Whether the data is restored, if yes, decoding the data; otherwise, sending the packet loss feedback data to the sending end, and receiving the re-selected by the sending end in the preset multiple sets of redundancy level FEC redundant data. Corresponding level of FEC redundant data, if the re-received corresponding level of FEC redundant data is lost, sending decoding error feedback data to the transmitting end; otherwise, buffering the decoded video frame data; The decoded data is sent to the display device for display.
  • the embodiment of the invention discloses an anti-drop packet method, device and system in real-time communication. After parsing and restoring the network data packet to the receiving end, the network data packet is sent, and the network data packet includes the encoded video frame data and the FEC. Redundant data, FEC redundant data is determined according to the network packet loss rate, and there are corresponding groups of FEC redundancy levels; receiving feedback data sent by the receiving end; when the feedback data is packet loss feedback data, according to the current lost data of the packet loss feedback data The packet model reselects the FEC redundant data of the corresponding level from the preset multiple sets of redundancy level FEC redundant data, and sends the FEC redundant data to the receiving end; in addition, when the feedback data is the decoding error feedback data According to the current decoding error model, the corresponding independently decodable frame corresponding to the decoding error frame is sent to the receiving end.
  • the embodiment of the invention solves the problem that the processing of network packet loss cannot meet the requirements of high quality and low bandwidth and the anti-dropping capability is poor when real-time communication is performed in the prior art. It realizes the adaptive processing of network packet loss occurring in the data transmission network in the real-time communication process, and enhances the anti-dropping capability in the real-time communication process.
  • FIG. 1 is a schematic flow chart of a first embodiment of an anti-drop method in real-time communication according to the present invention
  • FIG. 2 is a schematic flow chart of a second embodiment of an anti-drop method in real-time communication according to the present invention.
  • FIG. 3 is a schematic flow chart of a third embodiment of an anti-drop method in real-time communication according to the present invention.
  • FIG. 4 is a schematic flowchart of a fourth embodiment of an anti-drop method in real-time communication according to the present invention.
  • FIG. 5 is a schematic diagram of functional modules of a first embodiment of an anti-drop packet device in real-time communication according to the present invention
  • FIG. 6 is a schematic diagram of functional modules of a second embodiment of an anti-drop packet device in real-time communication according to the present invention.
  • FIG. 7 is a schematic diagram of functional modules of a third embodiment of an anti-drop packet device in real-time communication according to the present invention.
  • FIG. 8 is a schematic diagram of functional modules of a fourth embodiment of the anti-drop packet device in the real-time communication of the present invention.
  • FIG. 9 is a schematic structural diagram of a first embodiment of an anti-lost system in real-time communication according to the present invention.
  • FIG. 10 is a schematic structural diagram of a real-time communication system
  • FIG. 11 is a schematic structural diagram of a transmitting end in a real-time communication system
  • FIG. 12 is a schematic structural diagram of a receiving end in a real-time communication system
  • FIG. 13 is a schematic flowchart of feedback information sent by a receiving end of a receiving end in a real-time communication system.
  • the main solution of the embodiment of the present invention is: sending a network data packet to the receiving end, the network data packet includes the encoded video frame data and the FEC redundant data, and the FEC redundant data is determined according to the network packet loss rate to have corresponding FEC redundancy. Level; receiving feedback data sent by the receiving end; when the feedback data is packet loss feedback data, according to the current packet loss model of the packet loss feedback data, from the preset multiple groups The FEC redundant data of the corresponding level is reselected in the FEC redundant data of the redundancy level, and the FEC redundant data is sent to the receiving end; in addition, when the feedback data is the decoding error feedback data, the decoding is obtained according to the current decoding error model.
  • the corresponding independently decodable frame of the error frame is sent to the receiving end. Therefore, the problem that the processing of network packet loss cannot meet the requirements of high quality and low bandwidth at the same time in the real-time communication in the prior art, and the anti-dropping capability is poor is solved. It realizes the adaptive processing of packet loss occurring in the data transmission network in the real-time communication process, and enhances the anti-dropping capability in the real-time communication process.
  • a first embodiment of the present invention provides an anti-drop method in real-time communication, including:
  • Step S1 Send a network data packet to the receiving end, where the network data packet includes the encoded video frame data and the FEC redundant data, and the FEC redundant data is determined according to the network packet loss rate, and the corresponding multiple sets of FEC redundancy levels are determined. .
  • the executor of the method of the present invention may be an information transmitting device, a device, or a terminal.
  • the present embodiment is exemplified by a transmitting terminal (hereinafter referred to as a transmitting end), and is of course not limited to other encoding and sending services capable of providing real-time communication information. device.
  • the transmitter is provided with three encoders, including: a video encoder, a key frame encoder and an FEC encoder.
  • the transmitting end initializes the video encoder and the key frame encoder, and the initialization parameters include a target bit rate, a resolution, a frame rate, and the like.
  • the transmitting end collects the video frame data of the target video, and sends the collected video frame data to the video encoder for video encoding to generate the encoded video frame data; meanwhile, the collected video frame data is sent to the key frame encoder.
  • the key frame is encoded to generate an independently decodable frame that occupies a smaller bandwidth than the video frame data encoded by the video encoder.
  • the key frame encoder may adopt a different frame rate, code rate, resolution, encoding manner from the video encoder, or adopt other strategies to achieve a lower encoding rate; the frame rate of the key frame encoder may be Adaptive adjustment according to network conditions, so that the frame rate of the key frame encoder is lower than the frame rate of the video encoder, reducing resource occupation.
  • the frame rate of the key frame encoder can be dynamically adjusted according to the network packet loss rate.
  • the dynamic adjustment method is as follows: the frame rate of the video encoder is defined as FRA, the frame rate of the key frame encoder is FRB, and the network packet loss rate is L.
  • the encoding frame rate of the key frame encoder is determined by the following formula (1):
  • the calculation method of the frame number encoded by the key frame encoder is:
  • the frame number encoded by the key frame encoder is obtained on the basis of the frame number outputted by the video encoder.
  • the frame number encoded by the key frame encoder should be (1, 4, 7, 10, 13).
  • the transmitting end sends a network data packet to the receiving end, and the network data packet includes the encoded video frame data and the FEC redundant data.
  • the transmitting end collects the video frame data of the target video, encodes the video frame data through the video encoder, and generates the encoded video frame data.
  • the FEC encoder performs FEC encoding on the video frame data to generate three sets of redundancy. Level of FEC redundant data.
  • the FEC coding adopts an RS algorithm. In the RS algorithm, N represents the original packet size, K represents the sum of the original packet and the number of redundant packets, and K-N represents the number of anti-lost packets in a single packet.
  • the N and K values of the three sets of redundancy are respectively selected to be ⁇ 8, 9 ⁇ , ⁇ 4, 5 ⁇ , ⁇ 4, 6 ⁇ , and the three levels are sequentially defined. For level 1, level 2, level 3.
  • Step S2 Receive feedback data sent by the receiving end after parsing and restoring the network data packet.
  • the transmitting end receives the receiving end.
  • the feedback data sent after the network packet is restored is parsed.
  • the feedback data may be packet loss feedback data or error decoding feedback data.
  • the packet loss feedback data includes a current packet loss model, where the packet loss model includes the number of currently lost media packets, the number of currently lost FEC packets, and And data such as a packet loss sequence number;
  • the error decoding feedback data includes a current error decoding model, and the error decoding model includes data such as a frame number of the decoding error, a ratio of the error macroblock, and/or a location of the error macroblock; and the receiving end Feedback operations that send feedback data use immediate feedback.
  • step S3 when the feedback data is packet loss feedback data, according to the current packet loss model of the packet loss feedback data, the corresponding level of FEC redundancy is reselected from the preset multiple sets of redundancy level FEC redundancy data. The remaining data is sent to the receiving end.
  • the sending end when the received feedback data sent by the receiving end is the packet loss feedback data, the sending end is configured from the preset multiple sets of redundancy level FEC redundant data according to the current packet loss model of the packet loss feedback data. Reselect the appropriate level of FEC redundancy data.
  • K-N which indicates the number of packets that can be resisted in a single packet. Since the receiving end is in the state of packet loss, the number of lost packets in the group is greater than the value of K-N.
  • the transmitting end After successfully reselecting the corresponding level of FEC redundancy data, the transmitting end sends the FEC redundant data to the receiving end.
  • the present invention provides a method for performing packet compensation when packet loss occurs in real-time communication, which ensures that all packet loss can be recovered as much as possible, and reduces waste of redundancy in FEC data.
  • the second embodiment of the present invention provides an anti-drop packet method in real-time communication.
  • the steps of reselecting the corresponding FEC redundancy data to be sent to the receiving end include:
  • step S3 may include:
  • Step S31 when the feedback data is packet loss feedback data, reselecting the corresponding level of FEC redundancy from the preset multiple sets of redundancy level FEC redundancy data according to the current packet loss model of the packet loss feedback data. Remaining data;
  • the sending end when the received feedback data sent by the receiving end is the packet loss feedback data, the sending end is configured from the preset multiple sets of redundancy level FEC redundant data according to the current packet loss model of the packet loss feedback data. Reselect the appropriate level of FEC redundancy data.
  • K-N which indicates the number of packets that can be resisted in a single packet. Since the receiving end is in the state of packet loss, the number of lost packets in the group is greater than the value of K-N.
  • step S32 the timeliness of the retransmitted FEC redundant data is determined. If the preset aging requirement is met, step S33 is performed; otherwise, step S34 is performed.
  • the transmitting end determines the timeliness of the retransmitted FEC redundant data.
  • the timeliness judgment method is:
  • the decoded frame buffer unit water level of the receiving end is T (wherein, the decoding frame buffer unit is a buffer unit set by the receiving end for storing decoded frame data), and the retransmitted FEC redundant data recovery data is used.
  • the video decoding takes ⁇ t, and the timeliness is judged by the following formula (2):
  • step S33 is performed; otherwise, it is considered that the aging requirement is not satisfied, and step S34 is performed.
  • Step S33 the reselected corresponding FEC redundancy data is sent to the receiving end.
  • the reselected corresponding FEC redundancy data is sent to the receiving end.
  • step S34 the reselected corresponding FEC redundancy data is not sent to the receiving end.
  • the reselected corresponding FEC redundancy data is not sent to the receiving end.
  • the present invention provides a method for performing packet loss processing when packet loss occurs in real-time communication, which ensures that all packet loss can be recovered as much as possible, and reduces waste of redundancy in FEC data, thereby The anti-drop packet capability in the real-time communication process is enhanced, and the bandwidth occupation is reduced.
  • the FEC redundant data is time-critically judged before the FEC redundant data is transmitted, thereby reducing unnecessary network bandwidth overhead.
  • the third embodiment of the present invention provides an anti-drop method in real-time communication.
  • the step of receiving the feedback data sent by the receiving end further includes:
  • Step S5 When the feedback data is decoding error feedback data, according to the current decoding error model of the decoding error feedback data, the independently decodable frame corresponding to the decoding error frame is sent to the receiving end.
  • the transmitting end acquires a decoding error frame according to a current decoding error model of the decoding error data. Independently decodable frames.
  • the transmitting end collects the video frame data of the target video, and encodes the video frame data by the key frame encoder to generate an independently decodable frame, and the independently decodable frame occupies a smaller bandwidth than the video frame data encoded by the video encoder.
  • the frame rate of the key frame encoder is dynamically adjusted according to the network packet loss rate by: defining a frame rate of the video encoder FRA, a frame rate of the key frame encoder FRB, and a network packet loss rate of L.
  • the encoding frame rate of the key frame encoder is determined by the following formula (1):
  • the decoding error model mainly includes data such as decoding the wrong frame number, the proportion of the error macroblock, and/or the location of the error macroblock; and the method for obtaining the independently decodable frame corresponding to the decoding error frame is:
  • the frame number of the key frame encoder is converted according to the encoding frame rate FRB, and the frame number encoded by the key frame encoder is obtained on the basis of the frame number outputted by the video encoder.
  • the frame number encoded by the key frame encoder should be (1, 4, 7, 10, 13).
  • the transmitting end After successfully obtaining the independently decodable frame corresponding to the decoding error frame, the transmitting end sends the independently decodable frame to the receiving end.
  • the present invention provides a decoding error processing method when a decoding error occurs in a communication process, so that the media information can be resolved after the decoding error, and the media information can be effectively transmitted.
  • the fourth embodiment of the present invention provides an anti-drop packet method in real-time communication.
  • the step of transmitting the network data packet to the receiving end is further included before the step of transmitting the network data packet to the receiving end.
  • Step S6 encoding the collected video frame by the video encoder, performing FEC encoding on the encoded video frame data, generating multiple sets of redundancy level FEC redundant data, and simultaneously acquiring the collected video frame data by the key frame encoder.
  • the key frame coding is performed to generate an independently decodable frame, wherein the bandwidth occupied by the frame data generated by the key frame encoder is smaller than the bandwidth occupied by the frame data generated by the video encoder.
  • the transmitting end encodes the collected video frame by using a video encoder, performs FEC encoding on the encoded video frame data, generates multiple sets of redundant level FEC redundant data, and determines a group by using a network packet loss rate.
  • the FEC redundant data is transmitted according to the corresponding level of FEC redundancy data.
  • the transmitting end collects the video frame data of the target video, encodes the video frame data through the video encoder, and generates the encoded video frame data.
  • the FEC encoder performs FEC encoding on the video frame data to generate three sets of redundancy. Level FEC redundant data, and then determine the corresponding level of FEC redundant packets through the network packet loss rate.
  • the above-mentioned FEC redundancy level should be determined according to the principle of conservative selection.
  • the purpose is to prevent the network from being congested and the quality of the call to be deteriorated when the amount of FEC redundant data is too large. At the same time, the network traffic consumption is also reduced as much as possible.
  • the FEC coding adopts an RS algorithm.
  • N represents the original packet size
  • K represents the sum of the original packet and the number of redundant packets
  • K-N represents the number of anti-lost packets in a single packet.
  • the N and K values of the three sets of redundancy are respectively selected to be ⁇ 8, 9 ⁇ , ⁇ 4, 5 ⁇ , ⁇ 4, 6 ⁇ , and the three levels are sequentially defined. For level 1, level 2, level 3.
  • the method for determining a set of FEC redundancy levels according to the network packet loss rate is to define a network packet loss rate of L and a redundancy level of Level, and calculate a redundancy level by using the following formula:
  • the transmitting end encodes the collected video frame data to generate an independently decodable frame.
  • the transmitting end collects the video frame data of the target video, and encodes the video frame data by the key frame encoder to generate an independently decodable frame, and the independently decodable frame occupies a smaller bandwidth than the video frame data encoded by the video encoder.
  • the key frame encoder may adopt a different frame rate, code rate, resolution, coding mode from the video encoder, or adopt other strategies to achieve a lower coding rate.
  • the parameters such as the frame rate of the key frame encoder can be adaptively adjusted according to the network condition, so that the frame rate of the key frame encoder is lower than the frame rate of the video encoder, and the resource occupation is reduced.
  • the frame rate of the key frame encoder is dynamically adjusted according to the network packet loss rate:
  • the frame rate of the video encoder is defined as FRA
  • the frame rate of the key frame encoder is FRB
  • the network packet loss rate is L.
  • the encoding frame rate of the keyframe encoder is determined by equation (1):
  • the present invention provides a method for generating multiple sets of redundancy level FEC redundant data and independently decodable frames, which provides guarantee for packet loss processing and decoding error processing in real-time communication.
  • the first embodiment of the present invention provides an anti-drop packet device in real-time communication, including:
  • the sending module 100 is configured to send a network data packet to the receiving end, where the network data packet includes the encoded video frame data and the FEC redundant data, where the FEC redundant data is determined according to a network packet loss rate, and the corresponding FEC redundancy is determined. level.
  • the executor of the present invention may be an information transmitting device, a device, or a terminal.
  • the present embodiment is exemplified by a transmitting terminal (hereinafter referred to as a transmitting end), and is of course not limited to other devices capable of providing encoding and sending services for real-time communication information. .
  • the transmitter is provided with three encoders, including: a video encoder and a key frame encoder. And FEC encoder.
  • the transmitting end initializes the video encoder and the key frame encoder, and the initialization parameters include a target bit rate, a resolution, a frame rate, and the like.
  • the transmitting end collects the video frame data of the target video, and sends the collected video frame data to the video encoder for video encoding to generate the encoded video frame data; meanwhile, the collected video frame data is sent to the key frame encoder.
  • the key frame is encoded to generate an independently decodable frame that occupies a smaller bandwidth than the video frame data encoded by the video encoder.
  • the key frame encoder may adopt a different frame rate, code rate, resolution, encoding manner from the video encoder, or adopt other strategies to achieve a lower encoding rate; the frame rate of the key frame encoder may be Adaptive adjustment according to network conditions, so that the frame rate of the key frame encoder is lower than the frame rate of the video encoder, reducing resource occupation.
  • the frame rate of the key frame encoder can be dynamically adjusted according to the network packet loss rate by: defining a frame rate of the video encoder FRA, a frame rate of the key frame encoder FRB, and a network packet loss rate of L.
  • the encoding frame rate of the key frame encoder is determined by the following formula (1):
  • the calculation method of the frame number encoded by the key frame encoder is:
  • the frame number encoded by the key frame encoder is obtained on the basis of the frame number outputted by the video encoder.
  • the frame number encoded by the key frame encoder should be (1, 4, 7, 10, 13).
  • the sending module 100 sends a network data packet to the receiving end, where the network data packet includes the encoded video frame data and the FEC redundant data.
  • the transmitting end collects the video frame data of the target video, encodes the video frame data through the video encoder, and generates the encoded video frame data.
  • the FEC encoder performs FEC encoding on the video frame data to generate three sets of redundancy.
  • Level FEC Redundant Data where FEC encoding uses the RS algorithm.
  • N represents the original packet size
  • K represents the sum of the original packet and the number of redundant packets
  • KN represents the number of anti-lost packets in a single packet. .
  • the N and K values of the three sets of redundancy are respectively selected to be ⁇ 8, 9 ⁇ , ⁇ 4, 5 ⁇ , ⁇ 4, 6 ⁇ , and the three levels are sequentially defined. For level 1, level 2, level 3.
  • the receiving module 200 is configured to receive feedback data sent by the receiving end after parsing and restoring the network data packet.
  • the receiving module 200 receives the feedback data sent by the receiving end after parsing and restoring the network data packet.
  • the feedback data may be packet loss feedback data or error decoding feedback data.
  • the packet loss feedback data includes a current packet loss model, where the packet loss model includes the number of currently lost media packets, the number of currently lost FEC packets, and And data such as a packet loss sequence number;
  • the error decoding feedback data includes a current error decoding model, and the error decoding model includes data such as a frame number of the decoding error, a ratio of the error macroblock, and/or a location of the error macroblock; and the receiving end Feedback operations that send feedback data use immediate feedback.
  • the packet loss processing module 300 is configured to: when the feedback data is packet loss feedback data, reselect from the preset multiple sets of redundancy level FEC redundancy data according to the current packet loss model of the packet loss feedback data. The corresponding level of FEC redundancy data is sent to the receiving end.
  • the packet loss processing module 300 is redundant from the preset multiple sets of redundancy levels according to the current packet loss model of the packet loss feedback data. Re-select the corresponding level of FEC redundancy data in the remaining data.
  • K-N which indicates the number of packets that can be resisted in a single packet. Since the receiving end is in the state of packet loss, the number of lost packets in the group is greater than the value of K-N.
  • the transmitting end After successfully reselecting the corresponding level of FEC redundancy data, the transmitting end sends the FEC redundant data to the receiving end.
  • the present invention provides a device for performing packet compensation when packet loss occurs in a real-time communication process, which ensures that all packet loss can be recovered as much as possible, and reduces waste of redundancy in the FEC data. Enhanced anti-drop capability in real-time communication and reduces bandwidth usage.
  • the second embodiment of the present invention provides an anti-drop packet device in real-time communication.
  • the device further includes:
  • the determining module 400 is further configured to determine the timeliness of the retransmitted FEC redundant data. If the current aging requirement is met, the packet loss processing module sends the reselected corresponding FEC redundant data to the receiving end.
  • the specific plan is as follows:
  • the packet loss processing module 300 reselects the corresponding level from the preset multiple sets of redundancy level FEC redundancy data according to the current packet loss model of the packet loss feedback data. FEC redundant data;
  • the sending end when the received feedback data sent by the receiving end is the packet loss feedback data, the sending end is configured from the preset multiple sets of redundancy level FEC redundant data according to the current packet loss model of the packet loss feedback data. Reselect the appropriate level of FEC redundancy data.
  • K-N which indicates the number of packets that can be resisted in a single packet. Since the receiving end is in the state of packet loss, the number of lost packets in the group is greater than the value of K-N.
  • the determining module 400 determines the timeliness of the retransmitted FEC redundant data.
  • the transmitting end determines the timeliness of the retransmitted FEC redundant data.
  • the timeliness judgment method is:
  • the decoded frame buffer unit water level of the receiving end is T (wherein, the decoding frame buffer unit is a buffer unit set by the receiving end for storing decoded frame data), and the retransmitted FEC redundant data recovery data is used.
  • the video decoding takes ⁇ t, and the timeliness is judged by the following formula (2):
  • the packet loss processing module 300 sends the reselected corresponding FEC redundancy data to the receiving end; otherwise, it is considered that the time-sensitive requirement is not met, and the packet loss processing module 300 The reselected corresponding FEC redundancy data is not transmitted to the receiving end.
  • the present invention provides a device for performing packet loss processing when packet loss occurs in a real-time communication process, ensuring that all packet loss can be restored as much as possible, and reducing waste of redundancy in FEC data, thereby The anti-drop packet capability in the real-time communication process is enhanced, and the bandwidth occupation is reduced.
  • the FEC redundant data is time-critically judged before the FEC redundant data is transmitted, thereby reducing unnecessary network bandwidth overhead.
  • a third embodiment of the present invention provides an anti-drop packet device in real-time communication.
  • the apparatus further comprises:
  • the decoding error processing module 500 is configured to: when the feedback data is decoding error feedback data, obtain an independently decodable frame corresponding to the decoding error frame and send the signal to the receiving end according to the current decoding error model of the decoding error feedback data.
  • the decoding error processing module 500 acquires an independently decodable frame corresponding to the decoding error frame according to the current decoding error model of the decoding error data.
  • the transmitting end collects the video frame data of the target video, and encodes the video frame data by the key frame encoder to generate an independently decodable frame; the independently decodable frame occupies a smaller bandwidth than the video frame data encoded by the video encoder.
  • the frame rate of the key frame encoder is dynamically adjusted according to the network packet loss rate by: defining a frame rate of the video encoder FRA, a frame rate of the key frame encoder FRB, and a network packet loss rate of L.
  • the encoding frame rate of the key frame encoder is determined by the following formula (1):
  • the decoding error model mainly includes data such as decoding the wrong frame number, the proportion of the error macroblock, and/or the location of the error macroblock; and the method for obtaining the independently decodable frame corresponding to the decoding error frame is:
  • the frame number of the key frame encoder is converted according to the encoding frame rate FRB, and the frame number encoded by the key frame encoder is obtained on the basis of the frame number outputted by the video encoder.
  • the frame number encoded by the key frame encoder should be (1, 4, 7, 10, 13).
  • the decoding error processing module 500 sends the independently decodable frame to the receiving end after successfully acquiring the corresponding independently decodable frame of the decoding error frame.
  • the present invention provides a decoding error processing apparatus for implementing a decoding error in a communication process, so that the media information can be resolved after the decoding error, and the media information can be effectively transmitted.
  • the fourth embodiment of the present invention provides an anti-drop packet device in real-time communication.
  • the device further includes:
  • the encoding module 600 is configured to encode the collected video frame by the video encoder, perform FEC encoding on the encoded video frame data, generate multiple sets of redundancy level FEC redundant data, and simultaneously collect the collected by the key frame encoder.
  • the video frame data is subjected to key frame coding to generate an independently decodable frame, wherein the bandwidth occupied by the frame data generated by the key frame encoder is smaller than the bandwidth occupied by the frame data generated by the video encoder.
  • the encoding module 600 encodes the collected video frame by using a video encoder, performs FEC encoding on the encoded video frame data, generates multiple sets of redundancy level FEC redundant data, and determines a packet loss rate through the network.
  • the FEC redundant data is sent in the corresponding level of FEC redundancy data.
  • the transmitting end collects the video frame data of the target video, encodes the video frame data through the video encoder, and generates the encoded video frame data.
  • the FEC encoder performs FEC encoding on the video frame data to generate three sets of redundancy.
  • Level FEC redundant data and then determine the corresponding level of FEC redundant packets through the network packet loss rate.
  • the above-mentioned FEC redundancy level should be determined according to the principle of conservative selection. The purpose is to prevent the network from being congested and the quality of the call to be deteriorated when the amount of FEC redundant data is too large. At the same time, the network traffic consumption is also reduced as much as possible.
  • the FEC coding adopts an RS algorithm.
  • N represents the original packet size
  • K represents the sum of the original packet and the number of redundant packets
  • K-N represents the number of anti-lost packets in a single packet.
  • the N and K values of the three sets of redundancy are respectively selected to be ⁇ 8, 9 ⁇ , ⁇ 4, 5 ⁇ , ⁇ 4, 6 ⁇ , and the three levels are sequentially defined. For level 1, level 2, level 3.
  • the method for determining a set of FEC redundancy levels according to the network packet loss rate is to define a network packet loss rate of L and a redundancy level of Level, and calculate a redundancy level by using the following formula:
  • the encoding module 600 encodes the collected video frame data to generate an independently decodable frame.
  • the transmitting end collects the video frame data of the target video, and encodes the video frame data by the key frame encoder to generate an independently decodable frame, and the independently decodable frame occupies a smaller bandwidth than the video frame data encoded by the video encoder.
  • the key frame encoder may adopt a different frame rate, code rate, resolution, coding mode from the video encoder, or adopt other strategies to achieve a lower coding rate.
  • the parameters such as the frame rate of the key frame encoder can be adaptively adjusted according to the network condition, so that the frame rate of the key frame encoder is lower than the frame rate of the video encoder, and the resource occupation is reduced.
  • the frame rate of the key frame encoder is dynamically adjusted according to the network packet loss rate:
  • the frame rate of the video encoder is defined as FRA
  • the frame rate of the key frame encoder is FRB
  • the network packet loss rate is L.
  • the encoding frame rate of the keyframe encoder is determined by equation (1):
  • the present invention provides an apparatus for generating multiple sets of redundancy level FEC redundant data and independently decodable frames, which provides guarantee for packet loss processing and decoding error processing in real-time communication.
  • the first embodiment of the present invention provides an anti-lost system in real-time communication, and the structure of the real-time communication system shown in FIG. 10, the anti-lost system in the real-time communication. Including: information sender A and receiver B, where:
  • the sending end A includes a sending module 100, a receiving module 200, a packet loss processing module 300, a determining module 400, a decoding error processing module 500, and an encoding module 600;
  • the sending module 100 is configured to send a network data packet to the receiving end, where the network data packet includes the encoded video frame data and the FEC redundant data, and the FEC redundant data is determined according to the network packet loss rate, and the corresponding FEC redundancy is determined.
  • the receiving module 200 is configured to receive the feedback data sent by the receiving end;
  • the packet loss processing module 300 is configured to: when the feedback data is the packet loss feedback data, according to the current packet loss model of the packet loss feedback data, Reselecting the FEC redundancy data of the corresponding level from the preset multiple sets of redundancy level FEC redundancy data, and transmitting to the receiving end;
  • the determining module 400 is configured to determine the timeliness of the retransmitted FEC redundant data, if And satisfying the current aging requirement, the re-selected corresponding FEC redundancy data is sent by the packet loss processing module to the receiving end;
  • the decoding error processing module 500 is configured to: when the feedback data is decoding error feedback data, according to the Decoding the current decoding error model of the error feedback data, and obtaining an independently decodable frame corresponding to the decoding error frame is sent to the receiving end;
  • the encoding module 600 is configured Video frame coded data acquisition and FEC encoding is performed, generating a
  • the transmitting end A is further configured to initialize a video frame encoder and a key frame encoder; the video frame encoder is used to encode the collected video frame data to obtain encoded frame data; and the video frame data is sent to the Receiving end B;
  • the receiving end B is configured to receive the network data packet sent by the sending end, parse the FEC redundant data in the network data packet, recover the lost video frame data in the network data packet, and determine all video frame data. Whether to recover, if yes, decoding the data; otherwise, sending packet loss feedback data to the transmitting end; in the decoding process, determining whether a decoding error occurs when decoding the video frame data, and if so, sending Decoding error feedback data to the transmitting end; otherwise, buffering the decoded video frame data; and transmitting the buffered decoded data to the display device for display.
  • the sender A initializes the video frame encoder and the key frame encoder, and initializes the parameter package. Includes target bit rate, resolution, frame rate, and more.
  • the video frame data of the target video is collected in the transmitting end A; the video frame encoder in the encoding module 600 performs encoding processing on the video frame data to generate corresponding encoded frame data; and sends the encoded frame data to the receiving end B.
  • the FEC encoder of the encoding module 600 in the transmitting end A performs FEC encoding processing on the encoded frame data to generate a plurality of sets of redundancy level FEC data packets and saves the FEC redundant data from the plurality of sets of redundancy levels described above.
  • a set of FEC redundant data is selected and sent to the receiving end B.
  • the encoding module 600 in the transmitting end A performs key frame encoding processing on the encoded frame data to generate a corresponding independently decodable frame.
  • the sending module 100 in the sending end A sends the network data packet to the receiving end B, and includes the corresponding encoded frame data and the FEC redundant data generated after the video frame data is encoded and processed.
  • the FEC parser in the receiving end B recovers the lost media data packet; it is determined whether all media data packets have been recovered.
  • the decoding data is performed.
  • the receiving end B sends the decoding error feedback data to the transmitting end A, where the decoding error feedback data includes the current decoding error model, and the current decoding error model mainly includes the decoding error.
  • Data such as the frame number, the proportion of the error macroblock, and/or the location of the error macroblock.
  • the receiving end B sends the packet loss feedback data to the sending end A, where the packet loss feedback data includes the current packet loss model, and the current packet loss model includes the number of currently lost media packets, the number of currently lost FEC packets, and/or Data such as the packet loss sequence number; if the corresponding level of FEC redundant data received by the receiving end B is lost, the decoding error feedback data is sent to the transmitting end A.
  • the decoded frame data obtained after the network packet is restored is cached, and the corresponding data is taken out from the decoded frame buffer and sent to the display device for display and playback.
  • the decoding error processing module acquires the corresponding independently decodable frame from the preset independent decoding frame according to the current decoding error model; and the corresponding independently decodable frame Send to receiving end B.
  • the packet loss processing module 300 After the receiving module 200 in the sending end A receives the packet loss feedback data, the packet loss processing module 300 reselects a group of corresponding levels of FEC from the FEC redundant data of the preset multiple sets of redundancy levels according to the current packet loss model. Redundant data; the determining module 400 determines whether the FEC redundant data meets the timeliness requirement, and if so, sends it to the receiving end B.
  • the receiving end B After receiving the corresponding data sent by the sending end A, the receiving end B performs the data packet recovery and the decoding data packet operation again. If all the media data packets are not completely recovered or a decoding error occurs, the receiving end B performs the above corresponding feedback operation; The media data packet has been recovered. If no decoding error occurs, the decoded frame data obtained after parsing the restored network data packet is buffered, and then the corresponding data is taken out from the decoded frame buffer and sent to the display device for display and playback.
  • the transmitting end A may further include: a video collecting unit, an encoder A, an FEC encoder, an FEC packet buffer unit, an encoder B, a key frame buffer unit, a sending unit, and a decision unit; B may also include: an FEC parser, a decoder, a feedback unit, a decoded frame buffer unit, and a display unit.
  • the transmitting end A initializes the video frame encoder and the key frame encoder, and the initialization parameters include a target bit rate, a resolution, a frame rate, and the like.
  • the video collection unit of the transmitting end A collects video frame data of the target video; the encoder A encodes the video frame data to generate corresponding encoded frame data.
  • the FEC encoder of the transmitting end A performs FEC encoding processing on the encoded frame data to generate a plurality of sets of redundancy level FEC data packets and saves them to the FEC packet buffer unit; FEC redundant data from the plurality of sets of redundancy levels mentioned above. Select a corresponding set of FEC redundant data.
  • the encoder B performs key frame encoding processing on the encoded frame data to generate a corresponding independently decodable frame.
  • the sending unit in the transmitting end A sends the network data packet to the receiving end B, and includes the corresponding encoded frame data generated by the above-mentioned video frame data and the FEC redundant data.
  • the FEC parser in the receiving end B recovers the lost media data packet; determines whether all media data packets have been recovered.
  • the decoder decodes the data, and when a decoding error occurs, the feedback list
  • the element sends decoding error feedback data to the transmitting end A, and the decoding error feedback data includes a current decoding error model, and the current decoding error model includes data such as a frame number of the decoding error, a ratio of the error macroblock, and/or a position of the error macroblock.
  • the feedback unit sends the packet loss feedback data to the sender A, where the packet loss feedback data includes the current packet loss model, and the current packet loss model includes the number of currently lost media packets, the number of currently lost FEC packets, and/or loss. Packet serial number and other data. If the corresponding level of FEC redundant data received by the receiving end B is lost, the feedback unit sends the decoding error feedback data to the transmitting end A.
  • the decoded frame data obtained by parsing and restoring the network data packet is buffered to the decoding frame buffer unit, and then the corresponding data is taken out from the decoding frame buffer unit and sent to the display unit for display and Play.
  • the decision unit in the transmitting end A determines the decision type according to the feedback data, and responds to the decoding error request when the feedback data is the decoding error feedback data; when the feedback data is the packet loss Respond to the packet loss request when the data is fed back.
  • the decision unit in the sending end A After receiving the decoding error feedback data, the decision unit in the sending end A obtains the corresponding independently decodable frame from the independent decoded frame buffered in the key frame buffer unit according to the current decoding error model; the sending unit may be independent The decoded frame is sent to the receiving end B.
  • the decision unit in the sender A After receiving the packet loss feedback data, the decision unit in the sender A reselects a set of corresponding levels of FEC redundancy from the plurality of sets of redundancy level FEC redundancy data buffered in the FEC packet buffer unit according to the current packet loss model. Remaining data; determining whether the FEC redundant data meets the timeliness requirement, and if so, the sending unit sends the FEC redundant data to the receiving end B.
  • the receiving end B After receiving the corresponding data sent by the sending end A, the receiving end B performs the data packet recovery and the decoding data packet operation again. If all the media data packets are not completely recovered or a decoding error occurs, the feedback unit performs the above corresponding feedback operation; if all the media The data packet has been recovered, and no decoding error occurs.
  • the decoded frame data obtained after parsing the restored network data packet is buffered to the decoding frame buffer unit, and the corresponding data is taken out from the decoding frame buffer unit and sent to the display device for display and playback.
  • the present invention provides an anti-lost system in real-time communication, and now adaptively processes packet loss occurring in a data transmission network in a real-time communication process, and enhances real-time. Anti-dropping capability during communication.
  • Embodiments of the present invention also provide a storage medium.
  • the foregoing storage medium may be configured to store program code for performing the following steps:
  • Step S1 Send a network data packet to the receiving end, where the network data packet includes the encoded video frame data and the FEC redundant data, and the FEC redundant data is determined according to the network packet loss rate, and the corresponding multiple sets of FEC redundancy levels are determined. ;
  • step S3 when the feedback data is packet loss feedback data, according to the current packet loss model of the packet loss feedback data, the corresponding level of FEC redundancy is reselected from the preset multiple sets of redundancy level FEC redundancy data. The remaining data is sent to the receiving end.
  • the foregoing storage medium may include, but not limited to, a USB flash drive, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, and a magnetic memory.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • a mobile hard disk e.g., a hard disk
  • magnetic memory e.g., a hard disk
  • the embodiment of the invention is applied to the field of communication technology, and in real-time communication in the prior art,
  • the processing of network packet loss cannot meet the requirements of high quality and low bandwidth at the same time, and the problem of poor packet loss resistance. It realizes the adaptive processing of network packet loss occurring in the data transmission network in the real-time communication process, and enhances the anti-dropping capability in the real-time communication process.

Abstract

本发明公开了一种实时通信中的抗丢包方法、装置和系统，其方法包括：向接收端发送网络数据包，网络数据包包括编码后的视频帧数据和FEC冗余数据，FEC冗余数据确定有相应的FEC冗余级别；接收接收端解析还原网络数据包后发送的反馈数据；当反馈数据为丢包反馈数据时，根据当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据，发送至接收端；此外，当反馈数据为解码错误反馈数据，根据当前解码错误模型，获取解码错误帧相应的可独立解码帧发送至接收端。本发明解决了现有技术中对网络丢包的处理无法同时满足高质量和低带宽的需求，抗丢包能力较差的问题。增强实时通信过程中的抗丢包能力。

Description

实时通信中的抗丢包方法、装置和系统 技术领域
本发明涉及通信技术领域，尤其涉及一种实时通信中的抗丢包方法、装置和系统。
背景技术
在实时通信中，音视频媒体通常使用UDP(User Datagram Protocol，用户数据报协议)进行传输，由于UDP是一种无连接的传输协议，是一种尽力而为的服务方式，无法提供可靠的传输服务，因此网络中不可避免地会发生丢包，从而导致声音不连续、视频花屏停顿等现象，严重影响通信质量和用户体验，因此抗丢包的技术是关系到用户体验的一项重要技术。
目前用于抗丢包的技术一般有：FEC(Forward Error Correction，前向纠错)、重传、交织。一般来说，交织的时延较大，而当丢包由于拥塞引起时，盲目的重传会加重网络拥塞，因此，重传和交织在实时通讯系统中应用受到一些限制。
基于上述考虑，实时通信中通常采用FEC进行抗丢包，而由于FEC是通过增加冗余来达到抗丢包的目的，因此，它会在网络拥塞时导致拥塞进一步加重。并且，在移动互联网时代，通过FEC增加冗余的另一个负面影响是，由于FEC过多将耗费大量的网络流量，因此，会产生昂贵的通信费用。如果FEC的冗余比较高，则可以最大限度的恢复丢包，但会导致冗余数据的带宽占用很高；相反，如果FEC的冗余比较低，则不能保证完全恢复所有包，会导致视频显示出现花屏卡顿等问题，降低用户体验。
解决该问题的常用方法是根据网络统计的丢包率来决定FEC冗余级别，但由于网络反馈的丢包率一般是基于历史的统计，并不能完全反映当前的网络状况，故此时FEC冗余级别对应的是历史时间的网络平均状况，在当前网络状况下并不能保证恢复所有包，所以，FEC无法起到预期效果，从而无法从根本上解决该问题。
发明内容
本发明实施例的主要目的在于提出一种实时通信中的抗丢包方法、装置和系统，旨在实现在实时通信过程中对数据传输网络中发生的丢包进行自适应处理，增强实时通信过程中的抗丢包能力。
为实现上述目的，本发明实施例提供的实时通信中的抗丢包方法，包括：
向接收端发送网络数据包，所述网络数据包包括编码后的视频帧数据和FEC冗余数据，所述FEC冗余数据根据网络丢包率确定有相应的多组FEC冗余级别；
接收所述接收端解析还原所述网络数据包后发送的反馈数据；
当所述反馈数据为丢包反馈数据时，根据所述丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据，发送至所述接收端。
优选地，所述重新选择相应FEC冗余数据发送至所述接收端的步骤之前还包括：
判断重新发送的FEC冗余数据的时效性，若满足当前时效要求，则执行将重新选择的相应的FEC冗余数据发送至所述接收端的步骤。
优选地，所述接收所述接收端发送的反馈数据的步骤之后还包括：
当所述反馈数据为解码错误反馈数据，根据所述解码错误反馈数据的当前解码错误模型，获取解码错误帧相应的可独立解码帧发送至所述接收端。
优选地，所述发送端向接收端发送网络数据包的步骤之前还包括：
通过视频编码器对采集的视频帧进行编码，将编码后的视频帧数据进行FEC编码，生成多组冗余级别的FEC冗余数据，同时通过关键帧编码器对采集的视频帧数据进行关键帧编码生成可独立解码帧，其中，关键帧编码器生成的帧数据的带宽占用小于视频编码器生成的帧数据的带宽占用。
优选地，所述丢包模型包括当前丢失的媒体包个数、当前丢失的FEC 包个数和/或丢包序号；所述解码错误模型包括解码错误的帧号、错误宏块的比例和/或错误宏块的位置。
本发明实施例还提出一种实时通信中的抗丢包装置，其中，所述装置包括：
发送模块，设置为向接收端发送网络数据包，所述网络数据包包括编码后的视频帧数据和FEC冗余数据，所述FEC冗余数据根据网络丢包率确定有相应的多组FEC冗余级别；
接收模块，设置为接收所述接收端解析还原所述网络数据包后发送的反馈数据；
丢包处理模块，当所述反馈数据为丢包反馈数据时，根据所述丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据，发送至所述接收端。
优选地，所述装置还包括：
判断模块，设置为判断重新发送的FEC冗余数据的时效性，若满足当前时效要求，则由所述丢包处理模块将重新选择的相应的FEC冗余数据发送至所述接收端。
优选地，所述装置还包括：
解码错误处理模块，设置为当所述反馈数据为解码错误反馈数据，根据所述解码错误反馈数据的当前解码错误模型，获取解码错误帧相应的可独立解码帧发送至所述接收端。
优选地，其中，所述装置还包括：
编码模块，通过视频编码器对采集的视频帧进行编码，将编码后的视频帧数据进行FEC编码，生成多组冗余级别的FEC冗余数据，同时通过关键帧编码器对采集的视频帧数据进行关键帧编码生成可独立解码帧，其中，关键帧编码器生成的帧数据的带宽占用小于视频编码器生成的帧数据的带宽占用。
优选地，所述丢包模型包括当前丢失的媒体包个数、当前丢失的FEC 包个数和/或丢包序号；所述解码错误模型包括解码错误的帧号、错误宏块的比例和/或错误宏块的位置。
本发明实施例还提出一种实时通信中的抗丢包系统，其中，包括：接收端和发送端，其中：
所述发送端包括如上所述的装置；
所述接收端，设置为接收所述发送端发送的网络数据包；对所述网络数据包中的FEC冗余数据进行解析，恢复丢失的所述网络数据包中视频帧数据；判断所有视频帧数据是否恢复，若是，则对所述数据进行解码；否则，发送丢包反馈数据至所述发送端，接收所述发送端在预设的多组冗余级别的FEC冗余数据中重新选择的相应级别的FEC冗余数据，若重新接收的相应级别的FEC冗余数据发生丢包，则发送解码错误反馈数据至所述发送端；否则，对解码后的视频帧数据进行缓存；将缓存的解码数据发送至显示设备进行显示。
本发明实施例公开了一种实时通信中的抗丢包方法、装置和系统，通过向接收端解析还原所述网络数据包后发送网络数据包，网络数据包包括编码后的视频帧数据和FEC冗余数据，FEC冗余数据根据网络丢包率确定有相应的多组FEC冗余级别；接收接收端发送的反馈数据；当反馈数据为丢包反馈数据时，根据丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据，将FEC冗余数据发送至接收端；此外，还通过当反馈数据为解码错误反馈数据，根据当前解码错误模型，获取解码错误帧相应的可独立解码帧发送至接收端。本发明实施例解决了现有技术中在进行实时通信时，对网络丢包的处理无法同时满足高质量和低带宽的需求，抗丢包能力较差的问题。实现了在实时通信过程中对数据传输网络中发生的网络丢包进行自适应处理，增强实时通信过程中的抗丢包能力。
附图说明
图1是本发明实时通信中的抗丢包方法第一实施例的流程示意图；
图2是本发明实时通信中的抗丢包方法第二实施例的流程示意图；
图3是本发明实时通信中的抗丢包方法第三实施例的流程示意图；
图4是本发明实时通信中的抗丢包方法第四实施例的流程示意图；
图5是本发明实时通信中的抗丢包装置第一实施例的功能模块示意图；
图6是本发明实时通信中的抗丢包装置第二实施例的功能模块示意图；
图7是本发明实时通信中的抗丢包装置第三实施例的功能模块示意图；
图8是本发明实时通信中的抗丢包装置第四实施例的功能模块示意图；
图9是本发明实时通信中的抗丢包系统第一实施例的结构示意图。
图10是实时通信系统的结构示意图；
图11是实时通信系统中发送端的结构示意图；
图12是实时通信系统中接收端的结构示意图；
图13是实时通信系统中接收端的接收发送端发送的反馈信息的流程示意图。
本发明目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
具体实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
本发明实施例的主要解决方案是：向接收端发送网络数据包，网络数据包包括编码后的视频帧数据和FEC冗余数据，FEC冗余数据根据网络丢包率确定有相应的FEC冗余级别；接收接收端发送的反馈数据；当反馈数据为丢包反馈数据时，根据丢包反馈数据的当前丢包模型，从预设的多组 冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据，将FEC冗余数据发送至接收端；此外，还包括当反馈数据为解码错误反馈数据，根据当前解码错误模型，获取解码错误帧相应的可独立解码帧发送至接收端。由此，解决了有技术中在进行实时通信时，对网络丢包的处理无法同时满足高质量和低带宽的需求，抗丢包能力较差的问题。实现了在实时通信过程中对数据传输网络中发生的丢包进行自适应处理，增强实时通信过程中的抗丢包能力。
如图1所示，本发明第一实施例提出一种实时通信中的抗丢包方法，包括：
步骤S1，向接收端发送网络数据包，所述网络数据包包括编码后的视频帧数据和FEC冗余数据，所述FEC冗余数据根据网络丢包率确定有相应的多组FEC冗余级别。
本发明方法的执行主体可以为一种信息发送设备、装置或终端，本实施例以发送终端(以下简称发送端)进行举例，当然也不限定于其他能够为实时通信信息提供编码和发送服务的设备。
其中，发送端设置有三个编码器，包括：视频编码器、关键帧编码器和FEC编码器。
其中，发送端对视频编码器和关键帧编码器进行初始化，初始化参数包括目标码率、分辨率、帧率等。
其中，发送端采集目标视频的视频帧数据，将采集的视频帧数据送入视频编码器进行视频编码，生成编码后的视频帧数据；同时，将采集的视频帧数据送入关键帧编码器进行关键帧编码，生成可独立解码帧，该可独立解码帧比视频编码器编码后的视频帧数据占用更小带宽。
其中，关键帧编码器可采用与视频编码器不同的帧率、码率、分辨率、编码方式，或者采用其他策略，以达到较低的编码码率；关键帧编码器的帧率等参数可根据网络状况自适应调整，以使关键帧编码器的帧率低于视频编码器的帧率，减小资源占用。
其中，关键帧编码器的帧率可以根据网络丢包率动态调整，其动态调整的方法为：定义视频编码器的帧率为FRA，关键帧编码器的帧率为FRB，网络丢包率为L。关键帧编码器的编码帧率按如下公式(1)确定：
Figure PCTCN2017076758-appb-000001
其中，关键帧编码器编码的帧序号计算方法为：
1、获取视频帧编码器的编码帧率FRA和网络丢包率L，根据公式(1)计算的出关键帧编码器的编码帧率FRB；
2、根据关键帧编码器的编码帧率FRB换算，在与视频编码器输出的帧序号对应的基础上，得到关键帧编码器编码的帧序号。
例如：假定FRA＝9，L＝8％，视频编码器输出的帧序号为(1、2、3、4、5…..)，则根据公式(1)可得，FRB＝3。那么，关键帧编码器编码的帧序号应该为(1、4、7、10、13……)。
为了解决现有技术中在进行实时通信时抗丢包能力较差的问题，本实施例采用如下方案：
首先，发送端向接收端发送网络数据包，上述网络数据包包括编码后的视频帧数据和FEC冗余数据。
其中，发送端采集目标视频的视频帧数据，通过视频编码器对视频帧数据进行编码，生成编码后的视频帧数据；同时，通过FEC编码器对视频帧数据进行FEC编码，生成三组冗余级别的FEC冗余数据。其中，FEC编码采用RS算法，在RS算法中，N表示原始分组大小，K表示原始包与冗余包个数之和，K-N表示单个分组中可抗丢包的个数。在上述生成三组冗余级别的FEC冗余数据中，生成三组冗余的N、K值分别选择{8，9}、{4，5}、{4，6}，三个级别依次定义为级别1、级别2、级别3。
步骤S2，接收所述接收端解析还原所述网络数据包后发送的反馈数据。
具体地，发送端在成功向接收端发送网络数据包后，接收上述接收端 解析还原所述网络数据包后发送的反馈数据。
其中，上述反馈数据可以为丢包反馈数据或错误解码反馈数据；上述丢包反馈数据中包括当前丢包模型，该丢包模型包括当前丢失的媒体包个数、当前丢失的FEC包个数和/或丢包序号等数据；上述错误解码反馈数据中包括当前错误解码模型，该错误解码模型包括解码错误的帧号、错误宏块的比例和/或错误宏块的位置等数据；上述接收端发送反馈数据的反馈操作采用即时反馈。
步骤S3，当所述反馈数据为丢包反馈数据时，根据所述丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据，发送至所述接收端。
具体地，首先，当接收的上述接收端发送的反馈数据为丢包反馈数据时，发送端根据丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据。
其中，上述重新选择相应级别的FEC冗余数据的方法如下：
1、从当前丢包模型中获取原始分组大小N，原始包与冗余包个数之和K和该组中丢包个数；
2、计算K-N的值，表示单个分组中可抗丢包的个数，由于接收端为丢包状态，故此时该组中丢包个数大于K-N的值；
3、从预设的多个FEC冗余级别中，确定一个使该组中丢包个数小于或等于K-N的值的冗余级别，即将冗余级别提升到能抗该程度丢包的冗余级别；
4、选择该FEC冗余级别中的FEC冗余数据。
发送端在成功重新选择相应级别的FEC冗余数据后，将该FEC冗余数据发送至接收端。
通过上述方案，本发明提供了一种实时通信过程中出现丢包时进行数据包补偿的方法，尽可能地保证了所有丢包都能被恢复，同时减少了FEC数据中冗余的浪费，从而增强了实时通信过程中的抗丢包能力，并减少带 宽占用。
如图2所示，本发明第二实施例提出一种实时通信中的抗丢包方法，基于上述图1所述的实施例，重新选择相应FEC冗余数据发送至所述接收端的步骤之前还包括：
判断重新发送的FEC冗余数据的时效性，若满足预设时效要求，则执行重新选择相应的FEC冗余数据发送至所述接收端的步骤。
由此，在本实施例中，上述步骤S3可以包括：
步骤S31，当所述反馈数据为丢包反馈数据时，根据所述丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据；
具体地，首先，当接收的上述接收端发送的反馈数据为丢包反馈数据时，发送端根据丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据。
其中，上述重新选择相应级别的FEC冗余数据的方法如下：
1、从当前丢包模型中获取原始分组大小N，原始包与冗余包个数之和K和该组中丢包个数；
2、计算K-N的值，表示单个分组中可抗丢包的个数，由于接收端为丢包状态，故此时该组中丢包个数大于K-N的值；
3、从预设的多个FEC冗余级别中，确定一个使该组中丢包个数小于或等于K-N的值的冗余级别，即将冗余级别提升到能抗该程度丢包的冗余级别；
4、选择该FEC冗余级别中的FEC冗余数据。
步骤S32，判断重新发送的FEC冗余数据的时效性，若满足预设时效要求，则执行步骤S33；否则，执行步骤S34。
具体地，发送端在成功重新选择相应FEC冗余数据发送至所述接收端后，对重新发送的FEC冗余数据进行时效性判断。
其中，时效性判断方法为：
定义网络往返时延为TTL，接收端的解码帧缓存单元水位为T(其中，解码帧缓存单元为接收端用于存储解码帧数据而设置的缓存单元)，上述重新发送的FEC冗余数据恢复数据和视频解码需要时间为△t，通过如下公式(2)对时效性进行判断：
TTL+△t<T     (2)；
若满足公式(2)，则认为满足时效性要求，执行步骤S33；否则，认为不满足时效性要求，执行步骤S34。
步骤S33，将重新选择的相应的FEC冗余数据发送至所述接收端。
具体地，若判断到重新发送的FEC冗余数据满足时效性要求，则将重新选择的相应的FEC冗余数据发送至接收端。
步骤S34，不发送重新选择的相应的FEC冗余数据至所述接收端。
具体地，若判断到重新发送的FEC冗余数据不满足时效性要求，则不发送重新选择的相应的FEC冗余数据至接收端。
通过上述方案，本发明提供了一种实时通信过程中出现丢包时进行丢包处理的方法，尽可能地保证了所有丢包都能被恢复，同时减少了FEC数据中冗余的浪费，从而增强了实时通信过程中的抗丢包能力，并减少带宽占用；另外，在发送FEC冗余数据之前，对该FEC冗余数据进行了时效性判断，减少了不必要的网络带宽开销。
如图3所示，本发明第三实施例提出一种实时通信中的抗丢包方法，根据上述图1所述的实施例，接收所述接收端发送的反馈数据的步骤之后还包括：
步骤S5，当所述反馈数据为解码错误反馈数据，根据所述解码错误反馈数据的当前解码错误模型，获取解码错误帧相应的可独立解码帧发送至所述接收端。
具体地，首先，当接收的上述接收端发送的反馈数据为解码错误数据时，发送端根据解码错误数据的当前解码错误模型，获取解码错误帧相应 的可独立解码帧。
其中，发送端采集目标视频的视频帧数据，通过关键帧编码器对视频帧数据进行编码，生成可独立解码帧，该可独立解码帧比视频编码器编码后的视频帧数据占用更小带宽。
其中，关键帧编码器的帧率根据网络丢包率动态调整，其方法为：定义视频编码器的帧率为FRA，关键帧编码器的帧率为FRB，网络丢包率为L。关键帧编码器的编码帧率按如下公式(1)确定：
Figure PCTCN2017076758-appb-000002
其中，解码错误模型主要包括解码错误的帧号、错误宏块的比例和/或错误宏块的位置等数据；获取解码错误帧相应的可独立解码帧的方法为：
1、获取视频帧编码器的编码帧率FRA和网络丢包率L，根据公式(1)计算的出关键帧编码器的编码帧率FRB；
2、关键帧编码器的帧序号根据其编码帧率FRB换算，在与视频编码器输出的帧序号对应的基础上，得到关键帧编码器编码的帧序号。
例如：假定FRA＝9，L＝8％，视频编码器输出的帧序号为(1、2、3、4、5…..)，则根据公式(1)可得，FRB＝3。那么，关键帧编码器编码的帧序号应该为(1、4、7、10、13……)。
3、根据当前解码错误模型中的解码错误的帧号，获取对应帧号中的可独立解码帧。
发送端在成功获取解码错误帧相应的可独立解码帧后，将该可独立解码帧发送至接收端。
通过上述方案，本发明提供了一种实施通信过程中出现解码错误时的解码错误处理方法，使媒体信息在解码错误后能得到解决，也使媒体信息能得到有效传递。
如图4所示，本发明第四实施例提出一种实时通信中的抗丢包方法，根据上述图1所述的实施例，所述发送端向接收端发送网络数据包的步骤之前还包括：
步骤S6，通过视频编码器对采集的视频帧进行编码，将编码后的视频帧数据进行FEC编码，生成多组冗余级别的FEC冗余数据，同时通过关键帧编码器对采集的视频帧数据进行关键帧编码生成可独立解码帧，其中，关键帧编码器生成的帧数据的带宽占用小于视频编码器生成的帧数据的带宽占用。
具体地，发送端通过视频编码器对采集的视频帧进行编码，对编码后的视频帧数据进行FEC编码，生成多组冗余级别的FEC冗余数据，再通过网络丢包率，确定一组相应级别的FEC冗余数据，对该FEC冗余数据进行发送。其中，发送端采集目标视频的视频帧数据，通过视频编码器对视频帧数据进行编码，生成编码后的视频帧数据；同时，通过FEC编码器对视频帧数据进行FEC编码，生成三组冗余级别的FEC冗余数据，再通过网络丢包率，确定相应级别的FEC冗余分组。上述FEC冗余级别的确定，应遵循尽可能保守选择的原则，其目的是为了防止FEC冗余数据量过大时，可能导致网络拥塞，恶化通话质量；同时也尽可能降低网络流量的消耗。
其中，FEC编码采用RS算法，在RS算法中，N表示原始分组大小，K表示原始包与冗余包个数之和，K-N表示单个分组中可抗丢包的个数。在上述生成三组冗余级别的FEC冗余数据中，生成三组冗余的N、K值分别选择{8，9}、{4，5}、{4，6}，三个级别依次定义为级别1、级别2、级别3。
其中，根据网络丢包率确定一组相应级别的FEC冗余级别的方法为，定义网络丢包率为L，冗余级别为Level，通过如下计算公式计算出冗余级别：
Figure PCTCN2017076758-appb-000003
同时，发送端对采集的视频帧数据进行编码生成可独立解码帧。
其中，发送端采集目标视频的视频帧数据，通过关键帧编码器对视频帧数据进行编码，生成可独立解码帧，该可独立解码帧比视频编码器编码后的视频帧数据占用更小带宽。
其中，关键帧编码器可采用与视频编码器不同的帧率、码率、分辨率、编码方式，或者采用其他策略，以达到较低的编码码率。关键帧编码器的帧率等参数可根据网络状况自适应调整，以使关键帧编码器的帧率低于视频编码器的帧率，减小资源占用。
其中，关键帧编码器的帧率根据网络丢包率动态调整的方法为：定义视频编码器的帧率为FRA，关键帧编码器的帧率为FRB，网络丢包率为L。关键帧编码器的编码帧率按公式(1)确定：
Figure PCTCN2017076758-appb-000004
通过上述方案，本发明提供了一种生成多组冗余级别的FEC冗余数据和可独立解码帧的方法，为实时通信过程中的丢包处理和解码错误处理的进行提供了保证。
如图5所示，本发明第一实施例提出一种实时通信中的抗丢包装置，包括：
发送模块100，设置为向接收端发送网络数据包，所述网络数据包包括编码后的视频帧数据和FEC冗余数据，所述FEC冗余数据根据网络丢包率确定有相应的FEC冗余级别。
本发明的执行主体可以为一种信息发送设备、装置或终端，本实施例以发送终端(以下简称发送端)进行举例，当然也不限定于其他能够为实时通信信息提供编码和发送服务的设备。
其中，发送端设置有三个编码器，包括：视频编码器、关键帧编码器 和FEC编码器。
其中，发送端对视频编码器和关键帧编码器进行初始化，初始化参数包括目标码率、分辨率、帧率等。
其中，发送端采集目标视频的视频帧数据，将采集的视频帧数据送入视频编码器进行视频编码，生成编码后的视频帧数据；同时，将采集的视频帧数据送入关键帧编码器进行关键帧编码，生成可独立解码帧，该可独立解码帧比视频编码器编码后的视频帧数据占用更小带宽。
其中，关键帧编码器可采用与视频编码器不同的帧率、码率、分辨率、编码方式，或者采用其他策略，以达到较低的编码码率；关键帧编码器的帧率等参数可根据网络状况自适应调整，以使关键帧编码器的帧率低于视频编码器的帧率，减小资源占用。
其中，关键帧编码器的帧率可根据网络丢包率动态调整，其方法为：定义视频编码器的帧率为FRA，关键帧编码器的帧率为FRB，网络丢包率为L。关键帧编码器的编码帧率按如下公式(1)确定：
Figure PCTCN2017076758-appb-000005
其中，关键帧编码器编码的帧序号计算方法为：
1、获取视频帧编码器的编码帧率FRA和网络丢包率L，根据公式(1)计算的出关键帧编码器的编码帧率FRB；
2、根据关键帧编码器的编码帧率FRB换算，在与视频编码器输出的帧序号对应的基础上，得到关键帧编码器编码的帧序号。
例如：假定FRA＝9，L＝8％，视频编码器输出的帧序号为(1、2、3、4、5…..)，则根据公式(1)可得，FRB＝3。那么，关键帧编码器编码的帧序号应该为(1、4、7、10、13……)。
为了解决现有技术中在进行实时通信时抗丢包能力较差的问题，本实施例采用如下方案：
首先，发送模块100向接收端发送网络数据包，上述网络数据包包括编码后的视频帧数据和FEC冗余数据。
其中，发送端采集目标视频的视频帧数据，通过视频编码器对视频帧数据进行编码，生成编码后的视频帧数据；同时，通过FEC编码器对视频帧数据进行FEC编码，生成三组冗余级别的FEC冗余数据其中，FEC编码采用RS算法，在RS算法中，N表示原始分组大小，K表示原始包与冗余包个数之和，K-N表示单个分组中可抗丢包的个数。在上述生成三组冗余级别的FEC冗余数据中，生成三组冗余的N、K值分别选择{8，9}、{4，5}、{4，6}，三个级别依次定义为级别1、级别2、级别3。
接收模块200，设置为接收所述接收端解析还原所述网络数据包后发送的反馈数据。
具体地，在成功向接收端发送网络数据包后，接收模块200接收上述接收端解析还原所述网络数据包后发送的反馈数据。
其中，上述反馈数据可以为丢包反馈数据或错误解码反馈数据；上述丢包反馈数据中包括当前丢包模型，该丢包模型包括当前丢失的媒体包个数、当前丢失的FEC包个数和/或丢包序号等数据；上述错误解码反馈数据中包括当前错误解码模型，该错误解码模型包括解码错误的帧号、错误宏块的比例和/或错误宏块的位置等数据；上述接收端发送反馈数据的反馈操作采用即时反馈。
丢包处理模块300，设置为当所述反馈数据为丢包反馈数据时，根据所述丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据，发送至所述接收端。
具体地，首先，当接收的上述接收端发送的反馈数据为丢包反馈数据时，丢包处理模块300根据丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据。
其中，上述重新选择相应级别的FEC冗余数据的方法如下：
1、从当前丢包模型中获取原始分组大小N，原始包与冗余包个数之和K和该组中丢包个数；
2、计算K-N的值，表示单个分组中可抗丢包的个数，由于接收端为丢包状态，故此时该组中丢包个数大于K-N的值；
3、从预设的多个FEC冗余级别中，确定一个使该组中丢包个数小于或等于K-N的值的冗余级别，即将冗余级别提升到能抗该程度丢包的冗余级别；
4、选择该FEC冗余级别中的FEC冗余数据。
发送端在成功重新选择相应级别的FEC冗余数据后，将该FEC冗余数据发送至接收端。
通过上述方案，本发明提供了一种实时通信过程中出现丢包时进行数据包补偿的装置，尽可能地保证了所有丢包都能被恢复，同时减少了FEC数据中冗余的浪费，从而增强了实时通信过程中的抗丢包能力，并减少带宽占用。
如图6所示，本发明第二实施例提出一种实时通信中的抗丢包装置，根据上述图5所述的实施例，该装置还包括：
判断模块400，还设置为判断重新发送的FEC冗余数据的时效性，若满足当前时效要求，则由所述丢包处理模块将重新选择的相应的FEC冗余数据发送至所述接收端。具体方案如下：
当所述反馈数据为丢包反馈数据时，丢包处理模块300根据所述丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据；
具体地，首先，当接收的上述接收端发送的反馈数据为丢包反馈数据时，发送端根据丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据。
其中，上述重新选择相应级别的FEC冗余数据的方法如下：
1、从当前丢包模型中获取原始分组大小N，原始包与冗余包个数之和K和该组中丢包个数；
2、计算K-N的值，表示单个分组中可抗丢包的个数，由于接收端为丢包状态，故此时该组中丢包个数大于K-N的值；
3、从预设的多个FEC冗余级别中，确定一个使该组中丢包个数小于或等于K-N的值的冗余级别，即将冗余级别提升到能抗该程度丢包的冗余级别；
4、选择该FEC冗余级别中的FEC冗余数据。
判断模块400判断重新发送的FEC冗余数据的时效性。
具体地，发送端在成功重新选择相应FEC冗余数据发送至所述接收端后，对重新发送的FEC冗余数据进行时效性判断。
其中，时效性判断方法为：
定义网络往返时延为TTL，接收端的解码帧缓存单元水位为T(其中，解码帧缓存单元为接收端用于存储解码帧数据而设置的缓存单元)，上述重新发送的FEC冗余数据恢复数据和视频解码需要时间为△t，通过如下公式(2)对时效性进行判断：
TTL+△t<T    (2)；
若满足公式(2)，则认为满足时效性要求，丢包处理模块300将重新选择的相应的FEC冗余数据发送至所述接收端；否则，认为不满足时效性要求，丢包处理模块300不发送重新选择的相应的FEC冗余数据至所述接收端。
通过上述方案，本发明提供了一种实时通信过程中出现丢包时进行丢包处理的装置，尽可能地保证了所有丢包都能被恢复，同时减少了FEC数据中冗余的浪费，从而增强了实时通信过程中的抗丢包能力，并减少带宽占用；另外，在发送FEC冗余数据之前，对该FEC冗余数据进行了时效性判断，减少了不必要的网络带宽开销。
如图7所示，本发明第三实施例提出一种实时通信中的抗丢包装置， 根据上述图5所述的实施例，该装置还包括：
解码错误处理模块500，设置为当所述反馈数据为解码错误反馈数据，根据所述解码错误反馈数据的当前解码错误模型，获取解码错误帧相应的可独立解码帧发送至所述接收端。
具体地，首先，当接收的上述接收端发送的反馈数据为解码错误数据时，解码错误处理模块500根据解码错误数据的当前解码错误模型，获取解码错误帧相应的可独立解码帧。
其中，发送端采集目标视频的视频帧数据，通过关键帧编码器对视频帧数据进行编码，生成可独立解码帧；该可独立解码帧比视频编码器编码后的视频帧数据占用更小带宽。
其中，关键帧编码器的帧率根据网络丢包率动态调整，其方法为：定义视频编码器的帧率为FRA，关键帧编码器的帧率为FRB，网络丢包率为L。关键帧编码器的编码帧率按如下公式(1)确定：
Figure PCTCN2017076758-appb-000006
其中，解码错误模型主要包括解码错误的帧号、错误宏块的比例和/或错误宏块的位置等数据；获取解码错误帧相应的可独立解码帧的方法为：
1、获取视频帧编码器的编码帧率FRA和网络丢包率L，根据公式(1)计算的出关键帧编码器的编码帧率FRB；
2、关键帧编码器的帧序号根据其编码帧率FRB换算，在与视频编码器输出的帧序号对应的基础上，得到关键帧编码器编码的帧序号。
例如：假定FRA＝9，L＝8％，视频编码器输出的帧序号为(1、2、3、4、5…..)，则根据公式(1)可得，FRB＝3。那么，关键帧编码器编码的帧序号应该为(1、4、7、10、13……)。
3、根据当前解码错误模型中的解码错误的帧号，获取对应帧号中的可独立解码帧。
解码错误处理模块500在成功获取解码错误帧相应的可独立解码帧后，将该可独立解码帧发送至接收端。
通过上述方案，本发明提供了一种实施通信过程中出现解码错误时的解码错误处理装置，使媒体信息在解码错误后能得到解决，也使媒体信息能得到有效传递。
如图8所示，本发明第四实施例提出一种实时通信中的抗丢包装置，根据上述图5所述的实施例，该装置还包括：
编码模块600，设置为通过视频编码器对采集的视频帧进行编码，将编码后的视频帧数据进行FEC编码，生成多组冗余级别的FEC冗余数据，同时通过关键帧编码器对采集的视频帧数据进行关键帧编码生成可独立解码帧，其中，关键帧编码器生成的帧数据的带宽占用小于视频编码器生成的帧数据的带宽占用。
具体地，编码模块600通过视频编码器对采集的视频帧进行编码，对编码后的视频帧数据进行FEC编码，生成多组冗余级别的FEC冗余数据，再通过网络丢包率，确定一组相应级别的FEC冗余数据，对该FEC冗余数据进行发送。
其中，发送端采集目标视频的视频帧数据，通过视频编码器对视频帧数据进行编码，生成编码后的视频帧数据；同时，通过FEC编码器对视频帧数据进行FEC编码，生成三组冗余级别的FEC冗余数据，再通过网络丢包率，确定相应级别的FEC冗余分组。上述FEC冗余级别的确定，应遵循尽可能保守选择的原则，其目的是为了防止FEC冗余数据量过大时，可能导致网络拥塞，恶化通话质量；同时也尽可能降低网络流量的消耗。
其中，FEC编码采用RS算法，在RS算法中，N表示原始分组大小，K表示原始包与冗余包个数之和，K-N表示单个分组中可抗丢包的个数。在上述生成三组冗余级别的FEC冗余数据中，生成三组冗余的N、K值分别选择{8，9}、{4，5}、{4，6}，三个级别依次定义为级别1、级别2、级别3。
其中，根据网络丢包率确定一组相应级别的FEC冗余级别的方法为，定义网络丢包率为L，冗余级别为Level，通过如下计算公式计算出冗余级别：
Figure PCTCN2017076758-appb-000007
同时，编码模块600对采集的视频帧数据进行编码生成可独立解码帧。
其中，发送端采集目标视频的视频帧数据，通过关键帧编码器对视频帧数据进行编码，生成可独立解码帧，该可独立解码帧比视频编码器编码后的视频帧数据占用更小带宽。
其中，关键帧编码器可采用与视频编码器不同的帧率、码率、分辨率、编码方式，或者采用其他策略，以达到较低的编码码率。关键帧编码器的帧率等参数可根据网络状况自适应调整，以使关键帧编码器的帧率低于视频编码器的帧率，减小资源占用。
其中，关键帧编码器的帧率根据网络丢包率动态调整的方法为：定义视频编码器的帧率为FRA，关键帧编码器的帧率为FRB，网络丢包率为L。关键帧编码器的编码帧率按公式(1)确定：
Figure PCTCN2017076758-appb-000008
通过上述方案，本发明提供了一种生成多组冗余级别的FEC冗余数据和可独立解码帧的装置，为实时通信过程中的丢包处理和解码错误处理的进行提供了保证。
如图9所示，本发明第一实施例提出一种实时通信中的抗丢包系统，结合图10所示的实时通信系统结构示意图，上述实时通信中的抗丢包系统 包括：信息发送端A和接收端B，其中：
发送端A，包括发送模块100、接收模块200、丢包处理模块300、判断模块400、解码错误处理模块500和编码模块600；
其中，发送模块100设置为向接收端发送网络数据包，所述网络数据包包括编码后的视频帧数据和FEC冗余数据，所述FEC冗余数据根据网络丢包率确定有相应的FEC冗余级别；接收模块200设置为接收所述接收端发送的反馈数据；丢包处理模块300设置为当所述反馈数据为丢包反馈数据时，根据所述丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据，发送至所述接收端；判断模块400设置为判断重新发送的FEC冗余数据的时效性，若满足当前时效要求，则由所述丢包处理模块将重新选择的相应的FEC冗余数据发送至所述接收端；解码错误处理模块500设置为当所述反馈数据为解码错误反馈数据，根据所述解码错误反馈数据的当前解码错误模型，获取解码错误帧相应的可独立解码帧发送至所述接收端；编码模块600设置为对采集并编码后的视频帧数据进行FEC编码，生成多组冗余级别的FEC冗余数据，并对采集的视频帧数据进行关键帧编码生成可独立解码帧。
另外，发送端A还用于初始化视频帧编码器和关键帧编码器；用所述视频帧编码器对采集到的视频帧数据进行编码，得到编码帧数据；发送所述视频帧数据至所述接收端B；
接收端B，用于接收所述发送端发送的网络数据包；对所述网络数据包中的FEC冗余数据进行解析，恢复丢失的所述网络数据包中视频帧数据；判断所有视频帧数据是否恢复，若是，则对所述数据进行解码；否则，发送丢包反馈数据至所述发送端；在解码过程中，判断对所述视频帧数据进行解码时是否发生解码错误，若是，则发送解码错误反馈数据至所述发送端；否则，对解码后的视频帧数据进行缓存；将缓存的解码数据发送至显示设备进行显示。
具体地，在实时通信过程中：
发送端A对视频帧编码器和关键帧编码器进行初始化，初始化参数包 括目标码率、分辨率、帧率等。
然后，发送端A中采集目标视频的视频帧数据；编码模块600中的视频帧编码器对上述视频帧数据进行编码处理，生成相应编码帧数据；将上述编码帧数据发送至接收端B。
然后，发送端A中的编码模块600的FEC编码器对上述编码帧数据进行FEC编码处理，生成多组冗余级别的FEC数据包并保存；从上述的多组冗余级别的FEC冗余数据中选择一组FEC冗余数据，发送至接收端B。
然后，在编码模块600中的视频帧编码器对上述视频帧数据进行编码处理的同时，发送端A中的编码模块600对上述编码帧数据进行关键帧编码处理，生成相应的可独立解码帧。
发送端A中的发送模块100向接收端B发送网络数据包，包括上述视频帧数据进行编码处理后生成的相应编码帧数据和上述FEC冗余数据。
然后，接收端B中的FEC解析器对丢失的媒体数据包进行恢复；判断是否所有媒体数据包都已恢复。
若已全部恢复，则进行解码数据，当发生解码错误时，接收端B向发送端A发送解码错误反馈数据，该解码错误反馈数据中包含当前解码错误模型，当前解码错误模型主要包括解码错误的帧号、错误宏块的比例和/或错误宏块的位置等数据。
否则，接收端B向发送端A发送丢包反馈数据，该丢包反馈数据中包含当前丢包模型，当前丢包模型包括当前丢失的媒体包个数、当前丢失的FEC包个数和/或丢包序号等数据；若接收端B重新接收的相应级别的FEC冗余数据发生丢包，则发送解码错误反馈数据至发送端A。
若所有媒体数据包都已恢复，没有发生解码错误，则将解析还原网络数据包后得到的解码帧数据进行缓存，再从解码帧缓存中取出相应数据送到显示设备进行显示和播放。
发送端A中的接收模块200接收到解码错误反馈数据后，解码错误处理模块根据上述当前解码错误模型，从预设的独立解码帧中获取相应的可独立解码帧；将相应的可独立解码帧发送至接收端B。
发送端A中的接收模块200接收到丢包反馈数据后，丢包处理模块300根据当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择一组相应级别的FEC冗余数据；判断模块400判断该FEC冗余数据是否满足时效性要求，若满足，则发送至接收端B。
接收端B接收到发送端A发送的相应数据后，再次进行数据包恢复和解码数据包操作，若所有媒体数据包未完全恢复或发生解码错误，则接收端B执行上述对应反馈操作；若所有媒体数据包都已恢复，没有发生解码错误，则将解析还原网络数据包后得到的解码帧数据进行缓存，再从解码帧缓存中取出相应数据送到显示设备进行显示和播放。
另外，参照图11和图12，发送端A还可以包括：视频采集单元，编码器A、FEC编码器、FEC分组缓存单元、编码器B、关键帧缓存单元、发包单元和决策单元；接收端B还可以包括：FEC解析器、解码器、反馈单元、解码帧缓存单元和显示单元。
具体地，在实时通信过程中：
发送端A对视频帧编码器和关键帧编码器进行初始化，初始化参数包括目标码率、分辨率、帧率等。
然后，发送端A的视频采集单元采集目标视频的视频帧数据；编码器A对上述视频帧数据进行编码处理，生成相应编码帧数据。
然后，发送端A的FEC编码器对上述编码帧数据进行FEC编码处理，生成多组冗余级别的FEC数据包并保存至FEC分组缓存单元；从上述的多组冗余级别的FEC冗余数据中选择一组相应的FEC冗余数据。
然后，在编码器A对上述视频帧数据进行编码处理的同时，编码器B对上述编码帧数据进行关键帧编码处理，生成相应的可独立解码帧。
然后，发送端A中的发包单元向接收端B发送网络数据包，包括上述视频帧数据进行编码处理后生成的相应编码帧数据和上述FEC冗余数据。
接收端B中的FEC解析器对丢失的媒体数据包进行恢复；判断是否所有媒体数据包都已恢复。
若已全部恢复，则解码器进行解码数据，当发生解码错误时，反馈单 元向发送端A发送解码错误反馈数据，该解码错误反馈数据中包含当前解码错误模型，当前解码错误模型包括解码错误的帧号、错误宏块的比例和/或错误宏块的位置等数据。
否则，反馈单元向发送端A发送丢包反馈数据，该丢包反馈数据中包含当前丢包模型，当前丢包模型包括当前丢失的媒体包个数、当前丢失的FEC包个数和/或丢包序号等数据。若接收端B重新接收的相应级别的FEC冗余数据发生丢包，则反馈单元发送解码错误反馈数据至发送端A。
若所有媒体数据包都已恢复，没有发生解码错误，则将解析还原网络数据包后得到的解码帧数据缓存至解码帧缓存单元，再从解码帧缓存单元取出相应数据送到显示单元进行显示和播放。
参照图13，发送端A中的决策单元接收到接收端B发送的反馈数据后，根据反馈数据判断决策类型，当反馈数据为解码错误反馈数据时，响应解码错误请求；当反馈数据为丢包反馈数据时，响应丢包请求。
发送端A中的决策单元接收到解码错误反馈数据后，根据上述当前解码错误模型，从缓存在关键帧缓存单元中的独立解码帧中获取相应的可独立解码帧；发包单元将相应的可独立解码帧发送至接收端B。
发送端A中的决策单元接收到丢包反馈数据后，根据当前丢包模型，从缓存在FEC分组缓存单元中的多组冗余级别的FEC冗余数据中重新选择一组相应级别的FEC冗余数据；判断该FEC冗余数据是否满足时效性要求，若满足，则发包单元发送该FEC冗余数据至接收端B。
接收端B接收到发送端A发送的相应数据后，再次进行数据包恢复和解码数据包操作，若所有媒体数据包未完全恢复或发生解码错误，则反馈单元执行上述对应反馈操作；若所有媒体数据包都已恢复，没有发生解码错误，则将解析还原网络数据包后得到的解码帧数据缓存至解码帧缓存单元，再从解码帧缓存单元取出相应数据送到显示设备进行显示和播放。
通过上述方案，本发明提供了一种实时通信中的抗丢包系统，现了在实时通信过程中对数据传输网络中发生的丢包进行自适应处理，增强实时 通信过程中的抗丢包能力。
本发明的实施例还提供了一种存储介质。可选地，在本实施例中，上述存储介质可以被设置为存储用于执行以下步骤的程序代码：
步骤S1，向接收端发送网络数据包，所述网络数据包包括编码后的视频帧数据和FEC冗余数据，所述FEC冗余数据根据网络丢包率确定有相应的多组FEC冗余级别；
步骤S2，接收所述接收端解析还原所述网络数据包后发送的反馈数据；
步骤S3，当所述反馈数据为丢包反馈数据时，根据所述丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据，发送至所述接收端。
可选地，在本实施例中，上述存储介质可以包括但不限于：U盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述仅为本发明的优选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。本领域普通技术人员可以理解上述方法中的全部或部分步骤可通过程序来指令相关硬件完成，所述程序可以存储于计算机可读存储介质中，如只读存储器、磁盘或光盘等。可选地，上述实施例的全部或部分步骤也可以使用一个或多个集成电路来实现，相应地，上述实施例中的各模块/模块可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。本申请不限制于任何特定形式的硬件和软件的结合。
工业实用性
本发明实施例，应用于通信技术领域，现有技术中在进行实时通信时， 对网络丢包的处理无法同时满足高质量和低带宽的需求，抗丢包能力较差的问题。实现了在实时通信过程中对数据传输网络中发生的网络丢包进行自适应处理，增强实时通信过程中的抗丢包能力。

Claims (11)

  1. 一种实时通信中的抗丢包方法，所述方法包括：
    向接收端发送网络数据包，所述网络数据包包括编码后的视频帧数据和前向纠错FEC冗余数据，所述FEC冗余数据根据网络丢包率确定有相应的多组FEC冗余级别；
    接收所述接收端解析还原所述网络数据包后发送的反馈数据；
    当所述反馈数据为丢包反馈数据时，根据所述丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据，发送至所述接收端。
  2. 根据权利要求1所述的方法，其中，所述接收所述接收端发送的反馈数据的步骤之后还包括：
    当所述反馈数据为解码错误反馈数据，根据所述解码错误反馈数据的当前解码错误模型，获取解码错误帧相应的可独立解码帧发送至所述接收端。
  3. 根据权利要求1所述的方法，其中，所述发送端向接收端发送网络数据包的步骤之前还包括：
    通过视频编码器对采集的视频帧进行编码，将编码后的视频帧数据进行FEC编码，生成多组冗余级别的FEC冗余数据，同时通过关键帧编码器对采集的视频帧数据进行关键帧编码生成可独立解码帧，其中，关键帧编码器生成的帧数据的带宽占用小于视频编码器生成的帧数据的带宽占用。
  4. 根据权利要求2或3所述的方法，其中，所述丢包模型包括当前丢失的媒体包个数、当前丢失的FEC包个数和/或丢包序号；所述解码错误模型包括解码错误的帧号、错误宏块的比例和/或错误宏块 的位置。
  5. 根据权利要求1所述的方法，其中，所述重新选择相应FEC冗余数据发送至所述接收端的步骤之前还包括：
    判断重新发送的FEC冗余数据的时效性，若满足当前时效要求，则执行将重新选择的相应的FEC冗余数据发送至所述接收端。
  6. 一种实时通信中的抗丢包装置，所述装置包括：
    发送模块，设置为向接收端发送网络数据包，所述网络数据包包括编码后的视频帧数据和FEC冗余数据，所述FEC冗余数据根据网络丢包率确定有相应的多组FEC冗余级别；
    接收模块，设置为接收所述接收端解析还原所述网络数据包后发送的反馈数据；
    丢包处理模块，设置为当所述反馈数据为丢包反馈数据时，根据所述丢包反馈数据的当前丢包模型，从预设的多组冗余级别的FEC冗余数据中重新选择相应级别的FEC冗余数据，发送至所述接收端。
  7. 据权利要求6所述的装置，其中，所述装置还包括：
    判断模块，设置为判断重新发送的FEC冗余数据的时效性，若满足当前时效要求，则由所述丢包处理模块将重新选择的相应的FEC冗余数据发送至所述接收端。
  8. 据权利要求6所述的装置，其中，所述装置还包括：
    解码错误处理模块，设置为当所述反馈数据为解码错误反馈数据，根据所述解码错误反馈数据的当前解码错误模型，获取解码错误帧相应的可独立解码帧发送至所述接收端。
  9. 据权利要求6所述的装置，其中，所述装置还包括：
    编码模块，设置为通过视频编码器对采集的视频帧进行编码，将编码后的视频帧数据进行FEC编码，生成多组冗余级别的FEC冗余数据，同时通过关键帧编码器对采集的视频帧数据进行关键帧编码生成可独立解码帧，其中，关键帧编码器生成的帧数据的带宽占用小于视频编码器生成的帧数据的带宽占用。
  10. 据权利要求8或9中所述的装置，其中，所述丢包模型包括当前丢失的媒体包个数、当前丢失的FEC包个数和/或丢包序号；所述解码错误模型包括解码错误的帧号、错误宏块的比例和/或错误宏块的位置。
  11. 一种实时通信中的抗丢包系统，其中，包括：接收端和发送端，其中：
    所述发送端包括权利要求6-10中任一项所述的装置；
    所述接收端，用于接收所述发送端发送的网络数据包；对所述网络数据包中的FEC冗余数据进行解析，恢复丢失的所述网络数据包中视频帧数据；判断所有视频帧数据是否恢复，若是，则对所述数据进行解码；否则，发送丢包反馈数据至所述发送端，接收所述发送端在预设的多组冗余级别的FEC冗余数据中重新选择的相应级别的FEC冗余数据，若重新接收的相应级别的FEC冗余数据发生丢包，则发送解码错误反馈数据至所述发送端；否则，对解码后的视频帧数据进行缓存；将缓存的解码数据发送至显示设备进行显示。
PCT/CN2017/076758 2016-03-15 2017-03-15 实时通信中的抗丢包方法、装置和系统 WO2017157303A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610147944.0 2016-03-15
CN201610147944.0A CN107196746B (zh) 2016-03-15 2016-03-15 实时通信中的抗丢包方法、装置和系统

Publications (1)

Publication Number Publication Date
WO2017157303A1 true WO2017157303A1 (zh) 2017-09-21

Family

ID=59852031

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/076758 WO2017157303A1 (zh) 2016-03-15 2017-03-15 实时通信中的抗丢包方法、装置和系统

Country Status (2)

Country Link
CN (1) CN107196746B (zh)
WO (1) WO2017157303A1 (zh)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111262658A (zh) * 2020-02-11 2020-06-09 平安科技（深圳）有限公司 基于fec的重传方法、装置及可读存储介质
WO2020244327A1 (zh) * 2019-06-05 2020-12-10 西安万像电子科技有限公司 数据处理方法及装置
CN112422235A (zh) * 2020-11-19 2021-02-26 广州科天视畅信息科技有限公司 一种基于冗余算法的网络丢包恢复方法
CN112671509A (zh) * 2020-12-18 2021-04-16 杭州叙简科技股份有限公司 一种基于fec冗余的数据包抗丢包方法和装置
CN114039703A (zh) * 2021-11-26 2022-02-11 北京字节跳动网络技术有限公司 数据传输方法、装置、设备和介质
CN114124910A (zh) * 2021-11-04 2022-03-01 湖北美和易思教育科技有限公司 直播数据传输方法、系统及可读存储介质
CN114286132A (zh) * 2021-12-22 2022-04-05 天翼视讯传媒有限公司 一种无线网络中视频传输容错的方法
CN114301884A (zh) * 2021-08-27 2022-04-08 腾讯科技（深圳）有限公司 音频数据的发送方法、接收方法、装置、终端及存储介质
CN114448569A (zh) * 2022-01-27 2022-05-06 阿里巴巴（中国）有限公司 数据传输方法、设备及计算机存储介质
CN114554198A (zh) * 2022-04-27 2022-05-27 广州番禺职业技术学院 基于纠删码的视频关键帧冗余传输方法和系统
CN114584257A (zh) * 2022-01-26 2022-06-03 百果园技术(新加坡)有限公司 一种基于前向纠错编码的冗余分配方法及装置
CN114928573A (zh) * 2022-05-20 2022-08-19 中国科学院计算技术研究所 一种数据的传输方法及系统
WO2022199558A1 (zh) * 2021-03-22 2022-09-29 华为技术有限公司 一种数据传输方法、相关装置以及设备
CN115189809A (zh) * 2022-07-07 2022-10-14 福州大学 基于qoe的异构网络实时视频传输arq与fec模式选择方法
CN115189810A (zh) * 2022-07-07 2022-10-14 福州大学 一种面向低时延实时视频fec编码传输控制方法
CN117118573A (zh) * 2023-10-23 2023-11-24 腾讯科技（深圳）有限公司 数据网络传输方法、装置、设备及存储介质

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112492118B (zh) * 2018-06-21 2023-11-17 深圳市道通智能航空技术股份有限公司 数据传输控制方法、信息发送端、接收端及飞行器图传系统
CN111130686B (zh) * 2018-11-01 2022-12-06 海思光电子有限公司 业务数据处理方法及装置
CN109660319B (zh) * 2018-12-29 2021-07-23 北京金山安全软件有限公司 网络数据传输方法、装置及电子设备
CN109951254B (zh) * 2019-03-21 2020-11-06 腾讯科技（深圳）有限公司 一种数据处理方法及装置、计算机可读存储介质
CN110740334B (zh) * 2019-10-18 2021-08-31 福州大学 一种帧级别的应用层动态fec编码方法
CN111181698B (zh) * 2019-10-31 2022-12-20 腾讯云计算（北京）有限责任公司 数据处理方法、装置、设备及介质
CN111312264B (zh) * 2020-02-20 2023-04-21 腾讯科技（深圳）有限公司 语音传输方法、系统、装置、计算机可读存储介质和设备
CN111464458B (zh) * 2020-04-03 2024-01-02 广州市百果园信息技术有限公司 抗丢包方法、装置、设备及存储介质
CN111954002A (zh) * 2020-07-29 2020-11-17 西安万像电子科技有限公司 图像编码方法、装置、编码端设备及存储介质
CN114070459B (zh) * 2020-08-04 2023-11-07 成都鼎桥通信技术有限公司 数据传输方法、装置、终端设备和存储介质
CN114499747B (zh) * 2020-11-09 2023-06-20 成都鼎桥通信技术有限公司 音视频数据的处理方法、装置、电子设备及存储介质
CN112532349B (zh) * 2020-11-24 2022-02-18 广州技象科技有限公司 基于解码异常的数据处理方法及装置
CN115085859B (zh) * 2021-03-15 2023-11-24 海能达通信股份有限公司 一种抗丢包方法、装置及计算机可读存储介质
CN112911295A (zh) * 2021-04-16 2021-06-04 北京杰瑞创通科技有限公司 自适应动态抗网络丢包智能信源编码装置及方法
CN114051164B (zh) * 2021-11-16 2022-10-25 重庆紫光华山智安科技有限公司 解码异常码流生成方法、装置、电子设备及可读存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101061659A (zh) * 2004-12-02 2007-10-24 汤姆逊许可公司 自适应前向纠错
US20080002565A1 (en) * 2006-06-20 2008-01-03 Paul Spencer Incremental redundancy using high-order modulation and coding schemes
CN101719809A (zh) * 2009-11-25 2010-06-02 中兴通讯股份有限公司 一种媒体数据包丢包恢复的方法及系统
CN104620525A (zh) * 2012-07-09 2015-05-13 瑞典爱立信有限公司 用于在广播递送期间分发信息的方法和装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101061659A (zh) * 2004-12-02 2007-10-24 汤姆逊许可公司 自适应前向纠错
US20080002565A1 (en) * 2006-06-20 2008-01-03 Paul Spencer Incremental redundancy using high-order modulation and coding schemes
CN101719809A (zh) * 2009-11-25 2010-06-02 中兴通讯股份有限公司 一种媒体数据包丢包恢复的方法及系统
CN104620525A (zh) * 2012-07-09 2015-05-13 瑞典爱立信有限公司 用于在广播递送期间分发信息的方法和装置

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020244327A1 (zh) * 2019-06-05 2020-12-10 西安万像电子科技有限公司 数据处理方法及装置
CN111262658B (zh) * 2020-02-11 2022-09-16 平安科技（深圳）有限公司 基于fec的重传方法、装置及可读存储介质
CN111262658A (zh) * 2020-02-11 2020-06-09 平安科技（深圳）有限公司 基于fec的重传方法、装置及可读存储介质
CN112422235A (zh) * 2020-11-19 2021-02-26 广州科天视畅信息科技有限公司 一种基于冗余算法的网络丢包恢复方法
CN112422235B (zh) * 2020-11-19 2022-11-11 广州科天视畅信息科技有限公司 一种基于冗余算法的网络丢包恢复方法
CN112671509A (zh) * 2020-12-18 2021-04-16 杭州叙简科技股份有限公司 一种基于fec冗余的数据包抗丢包方法和装置
WO2022199558A1 (zh) * 2021-03-22 2022-09-29 华为技术有限公司 一种数据传输方法、相关装置以及设备
CN114301884A (zh) * 2021-08-27 2022-04-08 腾讯科技（深圳）有限公司 音频数据的发送方法、接收方法、装置、终端及存储介质
CN114301884B (zh) * 2021-08-27 2023-12-05 腾讯科技（深圳）有限公司 音频数据的发送方法、接收方法、装置、终端及存储介质
CN114124910A (zh) * 2021-11-04 2022-03-01 湖北美和易思教育科技有限公司 直播数据传输方法、系统及可读存储介质
CN114039703A (zh) * 2021-11-26 2022-02-11 北京字节跳动网络技术有限公司 数据传输方法、装置、设备和介质
CN114039703B (zh) * 2021-11-26 2023-10-20 北京字节跳动网络技术有限公司 数据传输方法、装置、设备和介质
CN114286132A (zh) * 2021-12-22 2022-04-05 天翼视讯传媒有限公司 一种无线网络中视频传输容错的方法
CN114584257A (zh) * 2022-01-26 2022-06-03 百果园技术(新加坡)有限公司 一种基于前向纠错编码的冗余分配方法及装置
CN114584257B (zh) * 2022-01-26 2024-02-13 百果园技术(新加坡)有限公司 一种基于前向纠错编码的冗余分配方法及装置
CN114448569A (zh) * 2022-01-27 2022-05-06 阿里巴巴（中国）有限公司 数据传输方法、设备及计算机存储介质
CN114554198B (zh) * 2022-04-27 2022-08-26 广州番禺职业技术学院 基于纠删码的视频关键帧冗余传输方法和系统
CN114554198A (zh) * 2022-04-27 2022-05-27 广州番禺职业技术学院 基于纠删码的视频关键帧冗余传输方法和系统
CN114928573A (zh) * 2022-05-20 2022-08-19 中国科学院计算技术研究所 一种数据的传输方法及系统
CN115189809A (zh) * 2022-07-07 2022-10-14 福州大学 基于qoe的异构网络实时视频传输arq与fec模式选择方法
CN115189810A (zh) * 2022-07-07 2022-10-14 福州大学 一种面向低时延实时视频fec编码传输控制方法
CN115189809B (zh) * 2022-07-07 2024-03-19 福州大学 基于qoe的异构网络实时视频传输arq与fec模式选择方法
CN115189810B (zh) * 2022-07-07 2024-04-16 福州大学 一种面向低时延实时视频fec编码传输控制方法
CN117118573A (zh) * 2023-10-23 2023-11-24 腾讯科技（深圳）有限公司 数据网络传输方法、装置、设备及存储介质
CN117118573B (zh) * 2023-10-23 2024-02-06 腾讯科技（深圳）有限公司 数据网络传输方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN107196746A (zh) 2017-09-22
CN107196746B (zh) 2020-11-27

Similar Documents

Publication Publication Date Title
WO2017157303A1 (zh) 实时通信中的抗丢包方法、装置和系统
US10652580B2 (en) Video data processing method and apparatus
US8341684B2 (en) Method and device for transmitting data
US8964115B2 (en) Transmission capacity probing using adaptive redundancy adjustment
JP3699910B2 (ja) データ伝送装置、データ伝送方法及びプログラム
JP6699834B2 (ja) ビデオデータ冗長制御方法および装置
CN106341738B (zh) 流媒体网络传输的带宽计算方法、服务器端和系统
CN111800218B (zh) 一种数据流的传输方法和设备
US20120272124A1 (en) Method and System for Recovering Lost Media Data Packets
US20080134005A1 (en) Adaptive Forward Error Correction
JP4699187B2 (ja) 受信装置、通信システム、および受信装置の制御プログラム
JP5109787B2 (ja) データ伝送システム、プログラム及び方法
RU2009116472A (ru) Динамическая модификация свойств видео
KR20060018794A (ko) 에러 정정용 데이터의 생성 방법 및 생성 장치와 생성프로그램 및 그 프로그램을 저장한 컴퓨터 판독 가능한기록 매체
US9781488B2 (en) Controlled adaptive rate switching system and method for media streaming over IP networks
WO2009113924A1 (en) Device and method for adaptation of target rate of video signals
JP3806133B2 (ja) データ伝送装置、データ伝送方法及びプログラム
JP4320024B2 (ja) 誤り訂正パケットを用いた伝送率制御方法およびそれを用いた通信装置
CN103632671A (zh) 数据编解码方法、装置及数据通信系统
JP5117512B2 (ja) 再送回数を動的に適合させる方法及び装置
JP2008085798A (ja) 音声伝送装置
WO2013189428A2 (zh) 基于分层编码的抗丢包实时通信方法、系统及相关设备
JP4834013B2 (ja) 送信装置、送信プログラム、受信装置及び受信プログラム
Liang et al. TCP-RTM: Using TCP for real time multimedia applications
JPWO2005039180A1 (ja) メディア信号の送信方法と受信方法ならびに送受信方法及び装置

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17765845

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17765845

Country of ref document: EP

Kind code of ref document: A1