WO2017211154A1

WO2017211154A1 - Forward error correction feedback information transmission method, apparatus and device, and storage medium

Info

Publication number: WO2017211154A1
Application number: PCT/CN2017/083526
Authority: WO
Inventors: 郑可馨
Original assignee: 中兴通讯股份有限公司
Priority date: 2016-06-07
Filing date: 2017-05-08
Publication date: 2017-12-14
Also published as: CN107483144B; CN107483144A

Abstract

Provided are a forward error correction feedback information transmission method, apparatus and device, and a storage medium. The method comprises: a forward error correction feedback information transmission apparatus receiving an opposite-end media data packet, determining a packet loss rate and generating forward error correction feedback information with regard to the packet loss rate; encapsulating the forward error correction feedback information into the media data packet, wherein the media data packet also comprises media data sent to an opposite end; and then, sending the media data packet to the opposite end, so that the opposite end can perform corresponding forward error correction control processing according to the forward error correction feedback information in the media data packet.

Description

Forward error correction feedback information transmission method, device, device and storage medium

Cross-reference to related applications

The present application is filed on the basis of the Chinese Patent Application Serial No. No. No. No. No. No. No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No No

Technical field

The present invention relates to the field of multimedia transmission technologies, and in particular, to a forward error correction feedback information transmission method, apparatus, device, and storage medium.

Background technique

With the development of information technology, the use of multimedia is more and more extensive. The so-called multimedia refers to a kind of human-computer interactive information exchange and communication medium combining two or more media. The media used includes text, images, photos, sounds, animations, and movies, as well as the interactive features provided by the program. When multimedia data is transmitted over the Internet, it is usually encapsulated and transmitted over the network using the RTP protocol (Real-time Transport Protocol). Since the requirement of real-time multimedia data is relatively high, the application of RTP for media data transmission is usually carried in the UDP protocol (User Datagram Protocol), and since the characteristics of the UDP data itself are fast transmission speed, However, its reliability is low, and the so-called low reliability is because it confirms the transmitted data and there is a case of packet loss.

In view of the above problem, the solution given in the related art is to use the forward error correction to recover the packet loss. Specifically, the sending direction sends the redundant packet generated according to the media data to the receiving party, and sends the redundant packet to the peer end. After receiving the packet, the peer end performs forward error correction processing according to the redundant packet when the packet is found to be lost. However, there is still a problem with this, that is, it cannot adapt The network with the packet loss rate changes, because if the forward coding parameters are certain, then the ability of error correction recovery is certain. For example, if the forward coding strategy is set in advance, only two packets can be corrected at most, then when the packet is lost. When the rate is high, it may not be possible to complete the error correction process and restore the original data.

A mechanism for using packet loss feedback is proposed in the related art for the above problem. The specific method is to use the RTCP protocol (Real-time Transport Control Protocol) to transmit feedback information. For example, the RTCP receiver reports feedback of the receiver packet loss situation, and can also define the private RTCP application message feedback packet loss situation. In the packet loss environment, the RTCP packet also has a certain packet loss rate. When the feedback message is lost, the sender can use other means, such as timing retransmission, or design a responsive RTCP private feedback message to solve the feedback message loss retransmission. The problem, but the delay caused by the repeated transmission back and forth will cause the forward error correction to have no good real-time performance, resulting in slow adjustment of the forward error correction coding redundancy, slow data recovery, and poor user experience.

In summary, in the related art, using the RTCP protocol for forward error correction feedback has a problem of slow adjustment of the forward error correction coding and slow data recovery.

Summary of the invention

The embodiment of the invention provides a method, a device, a device and a storage medium for transmitting forward error correction feedback information, and solves the problem that the forward error correction coding using the RTCP protocol has slow adjustment of the forward error correction coding and slow data recovery.

The embodiment of the invention provides a forward error correction feedback information transmission method, including:

Receiving a peer media data packet, determining a packet loss rate, and generating forward error correction feedback information for the packet loss rate;

Encapsulating the forward error correction feedback information into a media data packet, where the media data packet includes media data to be sent to the opposite end;

Sending the media data packet to the opposite end.

In an embodiment of the present invention, the method further includes: performing forward error correction coding according to the media data of the media data packet to generate a redundancy packet, and transmitting the redundancy packet to the opposite end.

In an embodiment of the present invention, after the sending the media data packet to the opposite end, the method further includes: saving the forward error correction feedback information, monitoring a change of a packet loss rate, and recovering a lost data packet.

In an embodiment of the present invention, the method further includes: determining, according to the change of the packet loss rate and the recovery of the lost data packet, whether the forward error correction feedback information needs to be updated, and if yes, updating the forward direction. The feedback information is error-corrected, and the updated forward error correction feedback information is sent to the opposite end through the media data packet.

In an embodiment of the invention, the encapsulating the forward error correction feedback information into the media data packet comprises: encapsulating the forward error correction feedback information into a source of a header of a real-time transport protocol message (CSRC, Contributing) Source field, or encapsulate the forward error correction feedback information into an extended header of a real-time transport protocol message.

The present invention also provides another forward error correction feedback information transmission method, including:

Receiving a media data packet sent by the peer end, where the media data packet includes media data and forward error correction feedback information generated by the peer end for the packet loss rate;

Parsing the media data packet to obtain the forward error correction feedback information;

Corresponding forward error correction coding control processing is performed according to the forward error correction feedback information.

In an embodiment of the present invention, the method further includes: receiving a redundant packet sent by the opposite end, and determining whether there is a media packet loss packet, and if yes, performing forward error correction decoding by using the redundant packet.

In an embodiment of the present invention, the same feedback information sequence number is set for the forward error correction feedback information with the same packet loss rate;

The performing the forward error correction control processing according to the forward error correction feedback information includes: adjusting the forward error correction coding parameter according to the unprocessed forward error correction feedback information; and marking the feedback information sequence number and The forward error correction feedback information with the same feedback information sequence number corresponding to the forward error correction feedback information is the processed forward error correction feedback information.

The embodiment of the invention further provides a forward error correction feedback information transmission device, including information generation Module, package module, and packet sending module:

The information generating module is configured to receive a peer media data packet, determine a packet loss rate, and generate forward error correction feedback information for the packet loss rate;

The encapsulating module is configured to encapsulate the forward error correction feedback information into a media data packet, where the media data packet includes media data to be sent to the opposite end;

The data packet sending module is configured to send the media data packet to the peer end.

In an embodiment of the present invention, the method further includes a first redundant packet processing module, configured to perform forward error correction coding according to the media data of the media data packet to generate a redundant packet, and send the redundant packet to The opposite end.

In an embodiment of the present invention, the method further includes a saving processing module configured to save the forward error correction feedback information, monitor a change of a packet loss rate, and recover a lost data packet.

In an embodiment of the present invention, the determining module further includes: determining, according to the change of the packet loss rate and the recovery of the lost data packet, whether the forward error correction feedback information needs to be updated, and if so, updating The forward error correction feedback information is sent to the opposite end by the updated forward error correction feedback information through the media data packet.

The present invention also provides another forward error correction feedback information transmission apparatus, including a data packet receiving module, a parsing module, and a processing module:

The data packet receiving module is configured to receive a media data packet sent by the peer end, where the media data packet includes media data and forward error correction feedback information generated by the peer end for a packet loss rate;

The parsing module is configured to parse the media data packet to obtain the forward error correction feedback information;

The processing module is configured to perform corresponding forward error correction coding control processing according to the forward error correction feedback information.

In an embodiment of the present invention, the second redundant packet processing module is further configured to receive the redundant packet sent by the opposite end, and determine whether there is a media packet loss packet, and if so, use the The redundant packet performs forward error correction decoding.

In an embodiment of the present invention, the method further includes a setting module configured to set the same feedback information sequence number for the forward error correction feedback information with the same packet loss rate;

The processing module is configured to adjust its forward error correction coding parameter according to the unprocessed forward error correction feedback information; and mark the feedback information sequence number equal to the feedback information sequence number corresponding to the forward error correction feedback information The error correction feedback information is processed forward error correction feedback information.

Embodiments of the present invention provide a computer storage medium having stored therein computer executable instructions configured to perform the forward error correction feedback information transmission method described above.

Embodiments of the present invention provide a forward error correction feedback information transmission apparatus including a memory, a processor, and a computer program stored on the memory and operable on the processor, the processor being configured to implement the program The above forward error correction feedback information transmission method.

The embodiment of the present invention provides a forward error correction feedback information transmission method, device, device, and storage medium, wherein the forward error correction feedback information transmission device receives the media data packet at the opposite end, determines a packet loss rate, and generates a packet loss rate. Forward error correction feedback information, and the forward error correction feedback information is encapsulated into a media data packet, where the media data packet further includes media data sent to the opposite end; and then the media data packet is sent to the opposite end, so that the opposite end can Corresponding forward error correction coding control processing is performed according to forward error correction feedback information in the media data packet. First, the method in the present invention is directed to a bidirectional transmission channel, and the media data packet is also sent to the opposite end according to the media data packet received by the opposite end. In the prior art, the outbound transmission forward error correction feedback information cannot be used. To ensure the reliability of the forward error correction feedback information transmission, the prior art adopts the timing retransmission or the design response replies message, but the forward error correction does not have good real-time performance, resulting in forward error correction coding. Redundancy adjustment is slow, data recovery is slow, and user experience is poor. With the solution of the present invention, the in-band transmission forward error correction feedback information is used. Since the transmission channel is a bidirectional transmission channel, when the devices A and B interact, A will receive the media data packet of B, and A will also B sends the number of media According to the packet, the forward error correction feedback information is generated according to the packet loss condition of the A receiving B data packet, and the forward error correction feedback information is encapsulated into the media data packet to be sent to the B, and then the data packet is sent to the B. Although the media packet sent by A to B may have packet loss, the possibility that all media packets are lost is very small. As a result, the forward error correction feedback sent by A to B is real-time. There is basically no delay, and the problem of using the RTCP protocol for forward error correction feedback is that the forward error correction coding has poor adjustment, slow data recovery, and poor user experience; real-time adjustment of forward error correction coding redundancy is achieved. Accelerate data recovery and improve user experience.

DRAWINGS

1 is a flowchart of a method for transmitting forward error correction feedback information according to an embodiment of the present invention;

2 is a schematic structural diagram of a forward error correction feedback information encapsulation structure according to an embodiment of the present invention;

3 is a flowchart of a method for transmitting forward error correction feedback information according to an embodiment of the present invention;

4 is a schematic structural diagram of a forward error correction feedback information transmission apparatus according to an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of another forward error correction feedback information transmission according to an embodiment of the present invention; FIG.

FIG. 6 is a flowchart of a specific example working process in an embodiment of the present invention.

detailed description

The present invention will be further described in detail below with reference to the accompanying drawings.

The embodiment provides a forward error correction feedback information transmission method. The solution in the present invention relates to the sender and the receiver of the forward error correction feedback information. The solution in this embodiment is mainly directed to the forward feedback information sender. For the steps to be performed, please refer to Figure 1, which includes:

S101: Receive a peer media data packet, determine a packet loss rate, and generate forward error correction feedback information for the packet loss rate.

S102: Encapsulating the forward error correction feedback information into a media data packet.

S103: Send the media data packet to the opposite end.

Since the forward error correction feedback information is mainly for the case where the media packet of the opposite end is received by itself The feedback is made, therefore, in the method of the present embodiment. Step S101 is executed to generate forward error correction feedback information according to the packet loss situation of the peer media data packet. For example, device B interacts with device A for media data, device A receives the media data packet of device B, and device A detects the packet loss when receiving the media data packet of device B. The packet loss situation usually uses the packet loss rate. Indicates that forward error correction feedback information is generated according to the packet loss condition.

In other embodiments of the present invention, the forward error correction feedback information may include: a current network packet loss rate, a feedback information sequence number, and a forward error correction control identifier. The current network packet loss rate is easy to understand, which is the current packet loss rate of the network; the feedback information sequence number is used to mark the current error correction feedback information according to the packet loss rate. The forward error correction feedback information for the same packet loss rate usually has the same feedback information sequence number. The forward error correction control identifier is mainly used to guide the peer to perform forward error correction coding, for example, to guide the peer to start or stop the forward error correction coding. For example, the device A detects the packet loss of the media packet of the receiving device B. The rate is particularly low, and there is basically no packet loss rate. The forward error correction control identifier in the forward error correction feedback information sent to B can be set to represent the corresponding identifier for stopping the error correction.

After generating the forward error correction feedback information, step S102 needs to be performed to encapsulate the forward error correction feedback information. In a specific implementation manner of the embodiment, the media data packet is also transmitted through the RTP protocol, and the forward error correction feedback information is encapsulated to the media data packet. The header of the RTP packet usually encapsulates the forward error correction feedback information in the RTP extension header, and may also encapsulate the forward error correction feedback information in the CSRC field of the RTP data packet. Please refer to FIG. 2, which is a schematic structural diagram of the forward error correction feedback information of the RTP data packet header. The load of the RTP packet is the media data that needs to be transmitted to the peer end, for example, voice data, video data, and the like. Then, step S103 is executed to send the media data packet to the opposite end.

Since the RTP packet itself carries the media data, the media data packet itself has the risk of packet loss. Therefore, the forward error correction coding is usually generated according to the content of the media data portion of the transmitted media data packet to generate a redundant packet. The residual packet is mainly used when the corresponding media packet is lost. Recover lost packets. In a specific implementation manner of this embodiment, forward error correction information may also be added to the redundant packet. Usually, the redundant packet is also transmitted by using the RTP protocol, and the packet header of the redundant packet may be selected according to specific requirements. Carry forward error correction feedback information. In order to avoid excessive loss of media packets, the forward error correction feedback information can also be encapsulated into the header of the redundant packet.

In a specific implementation manner of the embodiment, the forward error correction feedback information sender sends the forward error correction feedback information after transmitting the data packet with the forward error correction feedback information to the opposite end. Save it. It will also continue to monitor the packet loss of the receiving peer media data packet. When receiving the packet loss of the media packet at the peer end, it will also use the redundant packet of the peer end to recover the data, and also monitor the recovery of the lost packet. According to the detected change of the packet loss rate of the received peer media data packet and the recovery of the lost packet, it is determined whether the saved forward error correction feedback information needs to be updated. For the specific judgment method, it may be based on the artificial setting. For example, the recovery situation of the lost packet reaches a certain level, for example, 5% of the packet loss can be recovered, and the current state may be considered to be very good, and it is not necessary to Do any adjustments, so the forward error correction feedback information is not updated. For the packet loss rate, the forward error correction feedback information may be updated when the packet loss rate is detected, or may be the timing period detection. When the period arrives, the forward error correction feedback information is performed according to the packet loss situation in the current period. Update, in this way, it is also possible to artificially set a certain fixed value, for example, when the packet loss rate changes by more than 2%, the forward error correction feedback information is updated regardless of whether the update period is reached or not. In other embodiments of the present invention, the update process mainly includes: changing the current network packet loss rate of the previously saved forward error correction feedback information to a corresponding packet loss rate, and modifying the feedback information sequence number, which may be used for convenience. The method of increasing the order; modifying the forward error correction control identifier to the corresponding identifier according to the corresponding requirement.

By adopting the scheme of the embodiment, the in-band transmission mode is used to transmit the forward error correction feedback information, which can increase the real-time performance of the forward error correction feedback information and improve the user experience.

This embodiment provides another forward error correction feedback information transmission method. The solution in the present invention relates to the sender and the receiver of the forward error correction feedback information. The solution in this embodiment is mainly for the former. The steps performed by the feedback information receiver are described. Referring to FIG. 3, the method includes:

S301: Receive a media data packet sent by the peer end.

S302: Parsing the media data packet to obtain forward error correction feedback information.

S303: Perform corresponding forward error correction coding control processing according to the forward error correction feedback information.

In step S301, the received media data packet sent by the opposite end includes media data and forward error correction feedback information generated by the opposite end. Usually, the media data packet is transmitted through the RTP protocol, the media data is encapsulated in the data portion of the media data packet, and the forward error correction feedback information encapsulates the packet header of the RTP packet. In other embodiments of the present invention, the forward error correction feedback information may include: a current network packet loss rate, a feedback information sequence number, and a forward error correction control identifier. The current network packet loss rate is the current packet loss rate of the network; the feedback information sequence number is used to mark the current error correction feedback information according to the packet loss rate. The forward error correction feedback information for the same packet loss rate usually has the same feedback information sequence number. The forward error correction control indicator is mainly used to guide the local end to perform forward error correction coding processing. After receiving the media data packet sent by the peer end, only step S302 is performed, and the media data packet and the forward error correction feedback information are obtained by parsing the media data packet.

In step S303, the corresponding forward error correction control processing according to the forward error correction feedback information may be specifically: adjusting the forward error correction coding parameter according to the forward error correction feedback information that is not processed, and marking the feedback The forward error correction feedback information whose information sequence number is equal to the feedback information sequence corresponding to the forward error correction feedback information is the processed forward error correction feedback information. The marking method may be based on the feedback information serial number of the forward error correction feedback information. For example, the feedback information of the forward error correction feedback information A is 1 and the feedback information serial number of the feedback information serial number 1 is received for the first time. Therefore, the forward error correction feedback information with the feedback information sequence number 1 is not processed yet, and then the forward error correction coding parameter is adjusted according to the forward error correction feedback information of the feedback information sequence number 1. Because the packet error rate corresponding to the forward error correction feedback information of the feedback information sequence number 1 may be 10%, and the 10% packet loss rate is relatively high, it may be necessary to adjust the forward error correction coding parameters, such as grouping and redundancy. Adjustment. As explained above, for the same packet loss rate The forward error correction feedback information has the same feedback information sequence number, and the packet loss rate in the forward error correction feedback information may be 10% in a period of time, and the sequence number of the feedback message is 1 because The forward error correction feedback information with the feedback message sequence number 1 is processed, so there is no need to repeat the processing. After a period of time, the peer finds that the packet loss rate of the media packet receiving the local end becomes 20%, so the forward error correction feedback information is updated, the feedback information sequence number is changed to 2, and the packet loss rate is modified to 20 %, after receiving the media data packet carrying the forward error correction feedback information, the local end adjusts its own forward error correction coding parameter, for example, so that the coding can target more packets and increase redundancy. and many more.

In a specific implementation manner of this embodiment, the local end further receives a redundant packet generated by the peer end according to the media data, and the redundancy packet is mainly used to recover lost when the corresponding media data packet is lost. data. In a specific implementation manner of this embodiment, the redundancy packet may also include forward error correction information. Usually, the redundancy packet is also transmitted by using the RTP protocol. To avoid excessive loss of the media data packet, the forward correction may be performed. The error feedback information is also encapsulated into the header of the redundant packet. After receiving the redundant packet of the peer end, it is determined whether there is a packet loss. If there is a packet loss, the redundancy packet is used for forward error correction to recover data. As for how to determine whether there is a packet loss, the RTP protocol itself has a mechanism for detecting packet loss. When the RTP protocol is used to send media packets, the packet sequence number is carried in the RTP header, which can be used to detect packet loss.

With the solution in this embodiment, the received media data packet is parsed, the forward error correction feedback information is obtained, and the corresponding forward error correction coding control process is performed according to the forward error correction feedback information. For example, adjusting the forward error correction coding parameter, for example, performing corresponding forward error correction coding adjustment according to the forward error correction feedback information in the received media data packet, can well adapt to the network with the packet loss rate change, and has more Good real-time, user-friendly.

The present embodiment provides a forward error correction feedback information transmission apparatus. Referring to FIG. 4, the apparatus includes an information generation module 41, a package module 42 and a data packet transmission module 43. The information generation module 41 is configured to receive the peer media data packet. Determine the packet loss rate and generate forward error correction for the packet loss rate Feed information. The encapsulation module 42 is configured to encapsulate the forward error correction feedback information into a media data packet, where the media data packet further includes media data to be sent to the opposite end. The data packet sending module 43 is configured to send the encapsulated media data packet to the opposite end, and enable the peer end to perform corresponding forward error correction coding control processing according to the forward error correction feedback information in the media data packet.

The forward error correction feedback information includes: the current network packet loss rate, the feedback information sequence number, and the forward error correction control identifier; the current network packet loss rate is the current packet loss rate of the network; the feedback information sequence number is configured according to the packet loss rate. Mark the current error correction feedback information. The forward error correction feedback information for the same packet loss rate usually has the same feedback information sequence number. The forward error correction control flag is mainly used to guide the peer to perform forward error correction coding control, for example, to guide the peer to start or stop using forward error correction.

The forward error correction feedback information transmission apparatus 4 further includes a first redundancy packet processing module 44 configured to perform forward error correction coding to generate a redundancy packet according to the media data of the media data packet, and send the redundancy packet to the opposite end. .

The redundant packet is generated by forward error correction coding according to the content of the media data portion of the media data packet, and the redundancy packet is mainly used to recover the lost data packet when the corresponding media data packet is lost. The redundant packet may also include forward error correction information. Usually, the redundant packet is also transmitted using the RTP protocol. In order to avoid excessive loss of the media packet, the forward error correction feedback information may also be encapsulated into the header of the redundant packet.

The forward error correction feedback information transmission apparatus 4 further includes a save processing module 45 configured to save the forward error correction feedback information after transmitting the data packet with the forward error correction feedback information to the opposite end once.

And continue to monitor the packet loss situation of the receiving end media data packet, when receiving the media packet loss of the peer end, use the redundant packet of the opposite end to perform data recovery, and monitor the recovery of the lost packet. The forward error correction feedback information transmission device 4 further includes a determination module 46, according to the change of the packet loss rate of the received peer media data packet and the recovery of the packet loss detected by the save processing module 45. In the case, it is determined whether the saved forward error correction feedback information needs to be updated. For the specific judgment method, it can be based on the artificial setting.

The forward error correction feedback information transmission apparatus 4 in this embodiment is configured to perform the forward error correction feedback information transmission method in the embodiment, that is, any of the steps in the embodiment, the forward correction in this embodiment. The error feedback information transmission device 4 has an execution module corresponding thereto, which may not be shown for some modules, but it is present.

This embodiment further provides another forward error correction feedback information transmission device. Referring to FIG. 5, the device includes a data packet receiving module 51, a parsing module 52, and a processing module 53. The data packet receiving module 51 is configured to receive a media data packet sent by the opposite end, where the media data packet includes media data and forward error correction feedback information generated by the opposite end. The parsing module 52 is configured to parse the media data packet to obtain forward error correction feedback information. The processing module 53 is configured to perform a corresponding forward error correction coding control process based on the forward error correction feedback information. Similarly, the error correction feedback information includes a current packet loss rate, a feedback information sequence number, and a forward error correction control identifier.

The forward error correction feedback information transmission device 5 further includes a second redundant packet processing module 54 configured to receive the redundant packet sent by the opposite end, and determine whether there is a media packet loss packet, and if so, use the redundant packet Forward error correction decoding. The forward error correction feedback information transmission device 5 further includes a setting module 55 configured to set the same feedback information sequence number for the forward error correction feedback information with the same packet loss rate, and a processing module 56. The processing module 56 is based on the unprocessed The forward error correction feedback information adjusts its forward error correction coding parameter; and marks the forward error correction feedback information whose feedback information sequence number is equal to the feedback information sequence corresponding to the forward error correction feedback information as the processed forward correction Wrong feedback information.

In a specific implementation manner of the embodiment, the forward error correction feedback information transmission device 5 can further adjust its own forward error correction coding parameter according to the forward error correction feedback information; and mark the feedback information sequence number and the former The forward error correction feedback information having the same number of feedback information corresponding to the error correction feedback information is the processed forward error correction feedback information. Adjusting its own forward error correction coding parameters can be, Modification of the encoding allows the encoding to target more packets, as well as increased redundancy and the like.

The forward error correction feedback information transmission apparatus 5 in this embodiment is configured to perform the forward error correction feedback information transmission method in the embodiment, that is, any of the steps in the embodiment, and the forward correction in the embodiment. The error feedback information transmission device 5 has an execution module corresponding thereto, which may not be shown for some modules, but it is present.

In the following, the device A and the device B are used for the actual use. Since the solution in the present invention is directed to the two-way transmission channel, both the device A and the device B serve both the transmitting end of the media data packet and the receiving of the media data packet. end. That is to say, the device A and the device B simultaneously set the forward error correction feedback information transmission device 4 and the forward error correction feedback information transmission device 5 in this embodiment. Please refer to Figure 6, the specific interaction process is:

S601: Device A sends a media data packet a to device B;

S602: The device B receives the media data packet a, and generates forward error correction feedback information b according to the packet loss rate;

S603: The device B needs to send the media data packet b to the device A, and then carries the forward error correction feedback information b in the media data packet b, and sends the media data packet b to the device A;

S604: The device A receives the media data packet b, and obtains forward error correction feedback information b, and performs corresponding adjustment on the forward error correction coding.

S605: The device A generates forward error correction feedback information a* according to the packet loss condition of the media packet b of the receiving device B.

S606: The device A needs to send the media data packet a* to the device B, and then carries the forward error correction feedback information a* in the media data packet, and sends the media data packet a* to the device B;

S607: The device B receives the media data packet a*, and obtains the forward error correction feedback information a*, and adjusts the forward error correction coding accordingly;

S608: The device B determines, according to the packet loss condition of the received media data packet and the packet loss recovery situation, whether to update the forward error correction feedback information b, and if necessary, update to b* and send it Send to device A.

S609: The device A determines, according to the packet loss condition of the received media data packet and the packet loss recovery situation, whether to update the forward error correction feedback information a*, and if necessary, updates to a** and sends it to the device B. .

Then repeat the above steps (the timing of the steps 608 and 609 is not strictly required, as long as there is already forward error correction feedback information a* and forward error correction feedback information b, it is possible to perform the corresponding step 608 or 609) In this way, real-time adjustment of forward error correction coding redundancy is realized, data recovery is accelerated, and user experience is improved.

Each module in the forward error correction feedback information transmission apparatus provided by the embodiment of the present invention can be implemented by a processor in the device; of course, it can also be implemented by a logic circuit; in the process of implementation, the processor can be a CPU (Central Processor, Central Processing Unit), MPU (Micro Processor Unit), DSP (Digital Signal Processor) or FPGA (Field Programmable Gate Array).

In the embodiment of the present invention, if the foregoing forward error correction feedback information transmission method is implemented in the form of a software function module, and is sold or used as a separate product, it may also be stored in a computer readable storage medium. Based on such understanding, the technical solution of the embodiments of the present invention may be embodied in the form of a software product in essence or in the form of a software product stored in a storage medium, including a plurality of instructions. A computer device (which may be a personal computer, server, or network device, etc.) is caused to perform all or part of the methods described in various embodiments of the present invention. The foregoing storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM (Read Only Memory), a magnetic disk, or an optical disk. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.

It is to be understood that the phrase "one embodiment" or "an embodiment" or "an" Thus, "in one embodiment" or "in an embodiment" or "an" In addition, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present invention, the size of the sequence numbers of the above processes does not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not be directed to the embodiments of the present invention. The implementation process constitutes any limitation. The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

It is to be understood that the term "comprises", "comprising", or any other variants thereof, is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes those elements. It also includes other elements that are not explicitly listed, or elements that are inherent to such a process, method, article, or device. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, such as: multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, or direct coupling, or communication connection of the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical, mechanical or other forms. of.

The units described above as separate components may or may not be physically separated. The components displayed as the unit may be, or may not be, physical units; they may be located in one place or on multiple network units; some or all of the units may be selected according to actual needs to implement the solution of the embodiment. purpose.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated into one unit; The unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

It will be understood by those skilled in the art that all or part of the steps of implementing the foregoing method embodiments may be performed by hardware related to program instructions. The foregoing program may be stored in a computer readable storage medium, and when executed, the program includes The foregoing steps of the method embodiment; and the foregoing storage medium includes: a removable storage device, a ROM, a magnetic disk, or an optical disk, and the like, which can store program codes.

Alternatively, the above-described integrated unit of the present invention may be stored in a computer readable storage medium if it is implemented in the form of a software function module and sold or used as a standalone product. Based on such understanding, the technical solution of the embodiments of the present invention may be embodied in the form of a software product in essence or in the form of a software product stored in a storage medium, including a plurality of instructions. A computer device (which may be a personal computer, server, or network device, etc.) is caused to perform all or part of the methods described in various embodiments of the present invention. The foregoing storage medium includes various media that can store program codes, such as a mobile storage device, a ROM, a magnetic disk, or an optical disk.

The above is a further detailed description of the present invention in connection with the specific embodiments, and the specific embodiments of the present invention are not limited to the description. It will be apparent to those skilled in the art that the present invention may be made without departing from the spirit and scope of the invention.

Industrial applicability

In the embodiment of the present invention, the forward error correction feedback information transmission device receives the media packet of the opposite end, determines a packet loss rate, generates forward error correction feedback information for the packet loss rate, and encapsulates the forward error correction feedback information. The media data packet includes media data sent to the peer end in the media data packet; and then the media data packet is sent to the peer end, so that the peer end can perform corresponding forward according to the forward error correction feedback information in the media data packet. The error correction control processing can be effectively solved. The forward error correction feedback using the RTCP protocol can be effectively solved. The problem of slow adjustment of the forward error correction coding, slow data recovery, and poor user experience; real-time adjustment of forward error correction coding Redundancy accelerates data recovery and improves user experience.

Claims

A forward error correction feedback information transmission method includes:

Receiving a peer media data packet, determining a packet loss rate, and generating forward error correction feedback information for the packet loss rate;

Encapsulating the forward error correction feedback information into a media data packet, where the media data packet includes media data to be sent to the opposite end;

Sending the media data packet to the opposite end.
The forward error correction feedback information transmission method according to claim 1, further comprising: performing forward error correction coding to generate a redundancy packet according to the media data of the media data packet, and transmitting the redundancy packet to The opposite end.
The forward error correction feedback information transmission method according to claim 2, wherein after transmitting the media data packet to the opposite end, the method further comprises: saving the forward error correction feedback information, and monitoring a change of a packet loss rate. And the recovery of lost packets.
The forward error correction feedback information transmission method according to claim 3, further comprising: determining whether the forward error correction feedback information needs to be updated according to a change of a packet loss rate and a recovery condition of the lost data packet. If yes, the forward error correction feedback information is updated, and the updated forward error correction feedback information is sent to the opposite end through the media data packet.
The forward error correction feedback information transmission method according to any one of claims 1 to 4, wherein the packaging the forward error correction feedback information to the media data packet comprises: packaging the forward error correction feedback information to The CSRC field of the header of the protocol message is transmitted in real time, or the forward error correction feedback information is encapsulated into an extension header of the real-time transport protocol message.
A forward error correction feedback information transmission method includes:

Receiving a media data packet sent by the peer end, where the media data packet includes media data and forward error correction feedback information generated by the peer end for the packet loss rate;

Parsing the media data packet to obtain the forward error correction feedback information;

Corresponding forward error correction control processing is performed according to the forward error correction feedback information.
The forward error correction feedback information transmission method according to claim 6, further comprising: receiving a redundant packet sent by the opposite end, and determining whether there is a media packet loss, and if so, using the redundancy The packet performs forward error correction decoding.
The forward error correction feedback information transmission method according to claim 6 or 7, wherein the same feedback information sequence number is set for the forward error correction feedback information having the same packet loss rate;

The performing the forward error correction control processing according to the forward error correction feedback information includes: adjusting the forward error correction coding parameter according to the unprocessed forward error correction feedback information; and marking the feedback information sequence number and The forward error correction feedback information with the same feedback information sequence number corresponding to the forward error correction feedback information is the processed forward error correction feedback information.
A forward error correction feedback information transmission device includes:

The information generating module is configured to receive the peer media data packet, determine a packet loss rate, and generate forward error correction feedback information for the packet loss rate;

Encapsulating module: configured to encapsulate the forward error correction feedback information into a media data packet, where the media data packet includes media data to be sent to the opposite end;

The data packet sending module is configured to send the media data packet to the peer end.
The forward error correction feedback information transmission apparatus according to claim 9, further comprising a first redundant packet processing module configured to perform forward error correction coding to generate a redundancy packet according to the media data of the media data packet, And sending the redundant packet to the peer end.
The forward error correction feedback information transmission apparatus according to claim 10, further comprising a save processing module configured to save the forward error correction feedback information, monitor a change in a packet loss rate, and recover a lost data packet.
The forward error correction feedback information transmission apparatus according to claim 11, further comprising a determining module configured to determine whether the forward error correction is required according to a change of a packet loss rate and a recovery condition of the lost data packet The feedback information is updated, and if so, the forward correction is updated The feedback information is incorrectly sent, and the updated forward error correction feedback information is sent to the opposite end through the media data packet.
The forward error correction feedback information transmission apparatus according to any one of claims 9 to 12, wherein the encapsulation module is configured to encapsulate the forward error correction feedback information into a CSRC field of a header of a real-time transmission protocol message Or encapsulating the forward error correction feedback information into an extended header of a real-time transport protocol message.
A forward error correction feedback information transmission device, comprising:

a data packet receiving module, configured to receive a media data packet sent by the peer end, where the media data packet includes media data and forward error correction feedback information generated by the peer end for a packet loss rate;

a parsing module configured to parse the media data packet to obtain the forward error correction feedback information;

Processing module: configured to perform corresponding forward error correction control processing according to the forward error correction feedback information.
The forward error correction feedback information transmission apparatus according to claim 14, further comprising a second redundant packet processing module configured to receive the redundant packet sent by the opposite end, and determine whether there is a media packet loss packet If yes, forward error correction decoding is performed using the redundant packet.
The forward error correction feedback information transmission apparatus according to claim 14 or 15, further comprising a setting module configured to set the same feedback information sequence number for the forward error correction feedback information having the same packet loss rate;

The processing module is configured to adjust its forward error correction coding parameter according to the unprocessed forward error correction feedback information; and mark the feedback information sequence number equal to the feedback information sequence number corresponding to the forward error correction feedback information The error correction feedback information is processed forward error correction feedback information.
A computer storage medium having stored therein computer executable instructions for performing forward error correction feedback according to any one of claims 1 to 5, or 6 to 8 Information transmission method.
A forward error correction feedback information transmission device includes a memory, a processor, and a computer program stored on the memory and operable on the processor, the processor implementing the program to implement the following steps:

Receiving a peer media data packet, determining a packet loss rate, and generating forward error correction feedback information for the packet loss rate;

Encapsulating the forward error correction feedback information into a media data packet, where the media data packet includes media data to be sent to the opposite end;

Sending the media data packet to the opposite end.
A forward error correction feedback information transmission device includes a memory, a processor, and a computer program stored on the memory and operable on the processor, the processor implementing the program to implement the following steps:

Receiving a media data packet sent by the peer end, where the media data packet includes media data and forward error correction feedback information generated by the peer end for the packet loss rate;

Parsing the media data packet to obtain the forward error correction feedback information;

Corresponding forward error correction control processing is performed according to the forward error correction feedback information.