CN110087140A

CN110087140A - A kind of method, apparatus, medium and equipment for transmitting stream medium data

Info

Publication number: CN110087140A
Application number: CN201810078872.8A
Authority: CN
Inventors: 冯松佳
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2018-01-26
Filing date: 2018-01-26
Publication date: 2019-08-02
Anticipated expiration: 2038-01-26
Also published as: CN110087140B

Abstract

The embodiment of the present application provides a kind of method, apparatus, medium and equipment for transmitting stream medium data, wherein, the method of transmission stream medium data applied to transmitting terminal includes: to determine the coding redundancy degree of current second package to be sent according to the flow media data packet number and redundant data packets number of adjacent upper one the first package sent of receiving end feedback；According to the coding redundancy degree of second package, forward error correction FEC redundancy encoding is carried out to the flow media data packet of second package, obtains the redundant data packets of second package；Send the flow media data packet and redundant data packets of second package.Embodiment provided by the embodiments of the present application, can in the case where occupying less network bandwidth as far as possible, provide be enough to restore lose flow media data packet coding redundancy degree, can avoid as flow media data packet loss and caused by video cardton problem.

Description

A kind of method, apparatus, medium and equipment for transmitting stream medium data

Technical field

This application involves multimedia transport technology field more particularly to a kind of method, apparatus for transmitting stream medium data, it is situated between Matter and equipment.

Background technique

This part intends to provides background or context for the presently filed embodiment stated in claims.Herein Description recognizes it is the prior art not because not being included in this section.

Streaming Media refers to the media formats played by the way of stream transmission in internet, also known as streaming video.Quotient The Video coding that one video transmission server needs of household play is packaged into data packet and is transmitted to internet, and user can pass through visitor The video that family end carries out video decoding to data packet corresponding in internet and broadcast decoder obtains.

Currently, real-time transport protocol (Real-time Transport Protocol, RTP) is mostly used to transmit Streaming Media number According to guarantee the real-time of transmission.It is real that RTP generallys use UDP (User Datagram Protocol, User Datagram Protocol) Existing transmission of the data on IP (Internet Protocol, network protocol) network, since the transmission process of udp protocol does not have Retransmission mechanism, so the loss of transmission data is easily caused, and stream medium data packet loss of the Streaming Media in transmission process The problem of will cause video cardton.

Summary of the invention

The application provides a kind of method, apparatus, medium and equipment for transmitting stream medium data, for solving in the prior art The problem of video cardton caused by the existing stream medium data packet loss due to Streaming Media in transmission process.

In a first aspect, the embodiment of the present application provides a kind of method for transmitting stream medium data, comprising:

According to the flow media data packet number and redundant digit of adjacent upper one the first package sent of receiving end feedback According to packet number, the coding redundancy degree of current second package to be sent is determined；

According to the coding redundancy degree of second package, forward error correction is carried out to the flow media data packet of second package FEC redundancy encoding obtains the redundant data packets of second package；

Send the flow media data packet and redundant data packets of second package.

Optionally, the coding redundancy degree of current second package to be sent of the determination, specifically includes:

The flow media data packet number of first package that the receiving end is fed back and redundant data packets number and value work For first and value；And

By the flow media data packet number of first package of preservation and redundant data packets number and value as second and be worth；

According to described first and value with described second and value, determine the packet loss of first package；

Determine the product of the flow media data packet number of second package and the packet loss of first package；

The product is rounded up into resulting result as the coding redundancy degree of second package.

Optionally, the packet loss of the determination first package, specifically includes:

Determine described first and value with described second and value difference；

By the absolute value of the difference divided by described second and value it is resulting as a result, packet loss as first package Rate.

Optionally, before to the flow media data packet progress FEC redundancy encoding of second package, further includes:

Determine that the flow media data packet number of the second package of storage is equal to preset value.

The length that length in second package is less than the flow media data packet of preset length is carried out to mend 0 operation, so that Its length is equal to preset length.

Optionally, the flow media data packet includes extension header, which includes belonging to the flow media data packet Serial number of the serial number, the valid data length, the flow media data packet of the flow media data packet of package in affiliated package and The discrimination flag of the flow media data packet；The redundant data packets include extension header, which includes the redundant data The serial number of package belonging to wrapping, the valid data length of the redundant data packets, serial number of the redundant data packets in affiliated package with And the discrimination flag of the redundant data packets.

Second aspect, the embodiment of the present application provide a kind of device for transmitting stream medium data, comprising:

First determining module, the Streaming Media number of adjacent upper one the first package sent for being fed back according to receiving end According to packet number and redundant data packets number, the coding redundancy degree of current second package to be sent is determined；

Coding module, for the coding redundancy degree according to second package, to the stream medium data of second package Packet carries out forward error correction FEC redundancy encoding, obtains the redundant data packets of second package；

Sending module, for sending the flow media data packet and redundant data packets of second package.

The third aspect, the embodiment of the present application provide a kind of nonvolatile computer storage media, and the computer storage is situated between Matter is stored with executable program, which executes any transmission Streaming Media number for realizing that first aspect provides According to method the step of.

Fourth aspect, the embodiment of the present application provide a kind of computer equipment, including memory, processor and are stored in storage Computer program on device, the processor realize any transmission stream medium data that first aspect provides when executing described program Method the step of.

5th aspect, the embodiment of the present application provide a kind of method for transmitting stream medium data, comprising:

Receive the flow media data packet and redundant data packets of the package that transmitting terminal is sent, the redundant data packets are preceding to entangling The redundant data packets obtained after wrong FEC redundancy encoding；

Determine the flow media data packet number and redundant data packets number of the package received；

The flow media data packet number of the determining package and redundant data packets number are fed back into the transmitting terminal.

Optionally, it is described the flow media data packet number of the determining package and redundant data packets number are fed back to it is described Transmitting terminal specifically includes:

Added in the Receiver Report RR packet of real time control protocol RTCP the first extended field, the second extended field and Third extended field；

First extended field is written into the serial number of the package, institute is written into the flow media data packet number of the package The second extended field is stated, and, the third extended field is written into the redundant data packets number of the package, is written RR Packet；

Said write RR packet is fed back into the transmitting terminal.

Optionally, the flow media data packet includes extension header, which includes belonging to the flow media data packet The discrimination flag of the serial number of package and the flow media data packet, the redundant data packets include extension header, the extension header The discrimination flag of serial number and the redundant data packets including the affiliated package of the redundant data packets, then the reception transmitting terminal is sent Package flow media data packet and redundant data packets, specifically include:

In response to data packet arrival notice, according to the discrimination flag in the extension header of the data packet, the data packet is determined It is flow media data packet or redundant data packets；

If the data packet is flow media data packet, which is stored to the first chained list buffer area；

If the data packet is redundant data packets, which is stored to the second chained list buffer area.

Optionally, the flow media data packet number and redundant data packets number for the package that the determination receives, specifically Include:

Determine the flow media data packet number of serial number of the extension header comprising the package in the first chained list buffer area, with And

Determine the redundant data packets number of serial number of the extension header comprising the package in the second chained list buffer area.

If it is determined that the package flow media data packet number and the sum of redundant data packets number be not less than preset value, alternatively, If the first chained list buffer area is without available space, by the flow media data packet number and redundant data of the determining package Packet number feeds back to the transmitting terminal.

Optionally, the extension header of the flow media data packet further include the valid data length of the flow media data packet with And serial number of the flow media data packet in affiliated package, the extension header of the redundant data packets further include the redundant data packets Serial number in affiliated package of valid data length and the redundant data packets, then the method also includes:

By the flow media data packet of the package of the first chained list buffer area storage, and, second chained list is slow The redundant data packets for depositing the package of area's storage are stored to third chained list buffer area；

Remove the extension header of the flow media data packet and redundant data packets in the third chained list buffer area；

According to the extension header of each flow media data packet and the expanding packet of redundant data packets in third chained list buffer area Flow media data packet and redundant data packets after head, and removal extension header carry out fec decoder processing to the package.

Optionally, the method for transmission stream medium data provided by the embodiments of the present application, further includes:

The flow media data packet that fec decoder processing has been carried out in the first chained list buffer area is deleted, and,

Delete the redundant data packets that fec decoder processing has been carried out in the second chained list buffer area；And

Empty the third chained list buffer area.

6th aspect, the embodiment of the present application provide a kind of device for transmitting stream medium data, comprising:

Receiving module, the flow media data packet and redundant data packets of the package for receiving transmitting terminal transmission, the redundancy Data packet is obtained redundant data packets after forward error correction FEC redundancy encoding；

Determining module, for determining the flow media data packet number and redundant data packets number of the package received；

Feedback module, for the flow media data packet number of the package determined and redundant data packets number to be fed back to institute State transmitting terminal.

7th aspect, the embodiment of the present application provide a kind of nonvolatile computer storage media, and the computer storage is situated between Matter is stored with executable program, which executes any transmission Streaming Media number for realizing that the 5th aspect provides According to method the step of.

Eighth aspect, the embodiment of the present application provide a kind of computer equipment, including memory, processor and are stored in storage Computer program on device, the processor realize any transmission stream medium data that the 5th aspect provides when executing described program Method the step of.

9th aspect, the embodiment of the present application provide a kind of system for transmitting stream medium data, comprising:

Receiving end, the flow media data packet and redundant data packets of the first package for receiving transmitting terminal transmission are described superfluous Remaining data packet is obtained redundant data packets after forward error correction FEC redundancy encoding；Determine the stream of first package received Media data packet number and redundant data packets number；By the flow media data packet number and redundant data of determining first package Packet number feeds back to the transmitting terminal；

Transmitting terminal, the flow media data packet number and redundant data packets number of the first package for being fed back according to receiving end, Determine the coding redundancy degree of current second package to be sent；According to the coding redundancy degree of second package, to described second The flow media data packet of package carries out FEC redundancy encoding, obtains the redundant data packets of second package；Send second packet The flow media data packet and redundant data packets of group, wherein the first package is upper one adjacent with second package and has sent out Send package.

Method, apparatus, medium and the equipment of transmission stream medium data provided by the embodiments of the present application, have below beneficial to effect Fruit: according to receiving end feedback adjacent upper one the first package sent flow media data packet number and redundant data packets number, It determines the coding redundancy degree of current second package to be sent, and after carrying out FEC redundancy encoding to the second package, sends the second packet The flow media data packet and redundant data packets of group, so that the coding redundancy degree that dynamic adjusts current package to be sent is realized, energy It is enough to provide the coding redundancy degree for being enough to restore the flow media data packet lost in the case where occupying less network bandwidth as far as possible, Can avoid as flow media data packet loss and caused by video cardton problem.

Detailed description of the invention

The following detailed description is read with reference to the accompanying drawings, the above-mentioned and other purposes of the application exemplary embodiment, Feature and advantage will become prone to understand.In the accompanying drawings, the several of the application are shown by way of example rather than limitation Embodiment, in which:

Fig. 1 is the application scenarios schematic diagram of the embodiment of the present application；

Fig. 2 is the method flow schematic diagram of the transmission stream medium data provided by the embodiments of the present application applied to transmitting terminal；

Fig. 3 is the method flow schematic diagram of the coding redundancy degree of the second package of determination provided by the embodiments of the present application；

Fig. 4 is the form schematic diagram of data packet provided by the embodiments of the present application；

Fig. 5 is the configuration diagram of transmitting terminal provided by the embodiments of the present application；

Fig. 6 is video coded data message form schematic diagram provided by the embodiments of the present application；

Fig. 7 is the method flow schematic diagram of the transmission stream medium data provided by the embodiments of the present application applied to receiving end；

Flow media data packet number and redundant data packets number are fed back to transmitting terminal to be provided by the embodiments of the present application by Fig. 8 Method flow schematic diagram；

Fig. 9 is the side of flow media data packet and redundant data packets that reception transmitting terminal provided by the embodiments of the present application is sent Method flow diagram；

Figure 10 is the method flow schematic diagram that receiving end provided by the embodiments of the present application carries out FEC redundancy decoding process；

Figure 11 is the schematic device of the transmission stream medium data provided by the embodiments of the present application applied to transmitting terminal；

Figure 12 is the schematic device of the transmission stream medium data provided by the embodiments of the present application applied to receiving end；

Figure 13 is that the calculating of the method for the transmission stream medium data provided by the embodiments of the present application for realizing transmitting terminal is set Standby hardware structural diagram；

Figure 14 is that the calculating of the method for the transmission stream medium data provided by the embodiments of the present application for realizing receiving end is set Standby hardware structural diagram.

Specific embodiment

In the prior art, in order to alleviate because of stream medium data packet loss caused by video cardton problem, transmission Streaming Media When FEC (forward error correction, Forward Eeeor Correction) redundant data packets can be added so that Streaming Media receiving end passes through FEC redundant data packets and the flow media data packet received recover the stream medium data lost during streaming media Packet.However, it is found by the inventors that the coding redundancy degree of FEC is fixed value, in network rings in existing FEC redundancy encoding technology In the preferable situation in border, it may use the coding redundancy degree smaller than fixed value that can recover all stream medium datas of loss Packet, if will lead to the problem of occupying higher network bandwidth using fixed value at this time；In the case where network environment is poor, Streaming Media The loss number of data packet is more, it may be necessary to the stream of all loss can be just recovered using the coding redundancy degree bigger than fixed value Media data packet, if still having stream medium data packet loss at this time using fixed value.

For this purpose, the embodiment of the present application provides a kind of method for transmitting stream medium data, this method may include: according to reception Hold feedback adjacent upper one the first package sent flow media data packet number and redundant data packets number, determine currently to The coding redundancy degree of the second package sent；According to the coding redundancy degree of second package, to the stream matchmaker of second package Volume data packet carries out forward error correction FEC redundancy encoding, obtains the redundant data packets of second package；Send second package Flow media data packet and redundant data packets.

The method of transmission stream medium data provided by the embodiments of the present application, adjacent upper one for being fed back according to receiving end is The flow media data packet number and redundant data packets number of the first package sent determine that the coding of current second package to be sent is superfluous Remaining, and to the second package carry out FEC redundancy encoding after, send the second package flow media data packet and redundant data packets, from And the coding redundancy degree that dynamic adjusts current package to be sent is realized, it can be the case where occupying less network bandwidth as far as possible Under, provide be enough to restore lose flow media data packet coding redundancy degree, can avoid due to flow media data packet loss and Caused by video cardton problem.

Below with reference to Fig. 1, the application scenarios of the scheme of transmission stream medium data provided by the embodiments of the present application are said Bright, as shown in Figure 1, including receiving end 101 and transmitting terminal 102, receiving end 101 receives the stream matchmaker for the package that transmitting terminal 102 is sent Volume data packet and redundant data packets；Determine the flow media data packet number and redundant data packets number of the package received；It will determine Package flow media data packet number and redundant data packets number feed back to transmitting terminal 102.Transmitting terminal 102 is according to receiving end 101 The flow media data packet number and redundant data packets number of adjacent upper one the first package sent are fed back, is determined current to be sent The second package coding redundancy degree；According to the coding redundancy degree of the second package, the flow media data packet of the second package is carried out FEC redundancy encoding obtains the redundant data packets of the second package；The stream medium data of the second package is sent to the receiving end 101 Packet and redundant data packets.Wherein, receiving end 101 and transmitting terminal 102 are by that can pass through local area network, wide area network or mobile interchange The communication networks such as net are communicated；Receiving end and transmitting terminal can for portable equipment (such as: mobile phone, plate, laptop Deng), or PC (PC, Personal Computer).In some cases, transmitting terminal can have receiving end Function, receiving end can have the function of transmitting terminal, i.e. receiving end and transmitting terminal can be interchanged.It can hold, receiving end is client End, transmitting terminal is server.Application scenarios including a receiving end are only shown in Fig. 1, in practical applications, also may include Multiple receiving ends, i.e., multiple receiving ends carry out the transmission of stream medium data with same transmitting terminal respectively.

It, below will be to the possible embodiment of the application to keep the purposes, technical schemes and advantages of the application clearer It is further described.It should be noted that above-mentioned application scenarios are merely for convenience of understanding spirit herein and principle and showing Out, presently filed embodiment is unrestricted in this regard.On the contrary, presently filed embodiment can be applied to it is applicable Any scene.

The embodiment of the present application provides a kind of method for transmitting stream medium data, is applied to transmitting terminal, as shown in Figure 2, comprising:

Step 201, according to receiving end feedback adjacent upper one the first package sent flow media data packet number with And redundant data packets number, determine the coding redundancy degree of current second package to be sent.

Transmitting terminal successively sends the data packet of each package according to the playing sequence of the corresponding video frame of each package, packet The data packet of group includes flow media data packet and redundant data packets, specifically, transmitting terminal sends the data packet of package to receiving end. The data packet that transmitting terminal is sent reaches receiving end by the transmission of network, due to the uncertainty of network environment, reaches receiving end Data packet may have lost, in order to adjust the coding redundancy degree of current second package to be sent in real time, receiving end needs The flow media data packet number and redundant data packets number of adjacent upper one the first package sent are fed back to transmitting terminal.

Wherein, the first package is the upper package that has sent adjacent with the second package, i.e. the first package is that transmission is suitable Tagmeme before the second package and the package that has sent adjacent with the second package, the second package be transmitting terminal it is current to The package of transmission.

When it is implemented, transmitting terminal in response to receiving end RTSP (Real Time Streaming Protocol, in real time Flow host-host protocol) address access request, start to acquire video data, every acquisition for completing one-frame video data, i.e., to collecting The frame video data carry out Video coding, obtain the corresponding video data encoder of frame video data；It is assisted according to Preset Transfer View, is packaged into flow media data packet for the corresponding video data encoder of frame video data；The stream medium data that encapsulation is obtained Packet carries out FEC redundancy encoding, will obtain redundant data packets；By the flow media data packet that encapsulation obtains and the redundancy that coding obtains Data packet group sends the flow media data packet and redundant data packets of the package at a package.Receiving end can be according to receiving Package flow media data packet and redundant data packets carry out corresponding fec decoder processing and video decoding process, and decode Obtained video plays out.Wherein, Preset Transfer agreement can be RTP, i.e., flow media data packet and redundancy in package Data packet is RTP data packet.

When it is implemented, H.264 video coding technique, which can be used, carries out Video coding to collected video data, it can also Video coding is carried out to collected video data using other video coding techniques, here without limitation.

Step 202, according to the coding redundancy degree of second package, the flow media data packet of second package is carried out FEC redundancy encoding obtains the redundant data packets of second package.

When it is implemented, it is directed to collected every frame video data, the corresponding Video coding number of each pair of frame video data A flow media data packet is obtained according to encapsulation, then the flow media data packet is stored to transmitting terminal, can specifically be stored to transmitting terminal The first buffer area.In practical application, institute's code (Reed-solomon codes, RS code) in can be used in FEC redundancy encoding Carry out redundancy encoding, it is possible to use low density parity check code (Low Density Parity Check Code, LDPC), LT Digital fountain code or Raptor code etc. carry out redundancy encoding, here without limitation.

Optionally, before to the flow media data packet progress FEC redundancy encoding of second package, further includes: determination is deposited The flow media data packet number of second package of storage is equal to preset value, wherein preset value is the Streaming Media number for including in the second package According to the sum of packet, alternatively, preset value is the value set according to practical application request, here without limitation.

Optionally, before to the flow media data packet progress FEC redundancy encoding of second package, further includes: to described The length that length is less than the flow media data packet of preset length in second package carries out mending 0 operation, presets so that its length is equal to Length.

Specifically, when carrying out redundancy encoding using the RS code in FEC redundancy encoding, it is desirable that each Streaming Media number encoded It is identical according to the length of packet, it is therefore desirable to which that the flow media data packet for being less than preset length to length in the second package carries out zero padding behaviour Make, so that the length of each flow media data packet is identical in the second package.When flow media data packet is RTP data packet, preset Length is the maximum length of RTP data packet, which is 1448 bytes.Wherein, zero padding operation is i.e. in flow media data packet Valid data trailing zero, to obtain the flow media data packet of preset length.

Optionally, the redundant data packets of the second package are saved to the second buffer area of transmitting terminal；Determine second packet When the flow media data packet and redundant data packets of group are sent, second buffer area is discharged, to save transmitting terminal caching Space.

Step 203, the flow media data packet and redundant data packets of second package are sent.

When it is implemented, the flow media data packet and redundant data packets of each package have transmission sequence, it can be according to packet The flow media data packet of group and redundant data packets transmission sequence it is successive, be that sequence is arranged in flow media data packet and redundant data packets Number, wherein the transmission sequence of the smaller flow media data packet of serial number and redundant data packets is more forward, also, the Streaming Media of package The transmission sequence of data packet is prior to redundant data packets.

It should be noted that the first package and the second package are the package of non-first transmission sequence, transmitting terminal is sending the When the package of one transmission sequence, the flow media data packet number and redundant data packets number of receiving end feedback are not yet received, is sent out at this time Sending end can carry out FEC redundancy encoding to the flow media data packet of the package of the first transmission sequence according to pre-arranged code redundancy, should Pre-arranged code redundancy can be set according to practical application request, here without limitation.

The embodiment of the present application, the stream medium data for adjacent upper one the first package sent that can be fed back according to receiving end Packet number and redundant data packets number determine the coding redundancy degree of current second package to be sent, and it is superfluous to carry out FEC to the second package After remaining coding, the flow media data packet and redundant data packets of the second package are sent, so that it is current to be sent to realize dynamic adjustment The coding redundancy degree of package can provide the Streaming Media for being enough to restore to lose in the case where occupying less network bandwidth as far as possible The coding redundancy degree of data packet, can avoid as flow media data packet loss and caused by video cardton problem.

In the embodiment of the present application, if the flow media data packet number and redundant data packets number of receiving end feedback are very few, illustrate Current network environment is poor, and the coding redundancy degree of the current package to be sent determined at this time is larger；If the stream matchmaker of receiving end feedback Volume data packet number and redundant data packets number are more, then illustrate that current network environment is preferable, the current package to be sent determined at this time Coding redundancy degree it is smaller, according to receiving end feedback flow media data packet number and redundant data packets number, it can be achieved that dynamic adjust The coding redundancy degree of current package to be sent, in this way can in the case where occupying less network bandwidth as far as possible, provide be enough it is extensive The coding redundancy degree for the flow media data packet lost again.

Optionally, the content provided according to Fig. 3 determines the coding redundancy degree of current second package to be sent:

Step 301, the flow media data packet number and redundant data packets number of first package receiving end fed back And value as first and value.

When it is implemented, may cause to wrap due to network environment is poor etc. during transmitting terminal sends package The loss of interior data packet is organized, the flow media data packet number and redundant data packets of the first package of receiving end feedback are determined in this step Several and value, and should and be worth as first and value, the coding redundancy degree for subsequent the second package of determination provides basis.

Step 302, by the flow media data packet number of first package of preservation and redundant data packets number and value, as Second and value.

When it is implemented, saving the Streaming Media of the package sent in transmitting terminal while transmitting terminal sends package Data packet and redundant data packets, the media data packet and redundant data packets of the package saved at this time will not by network environment influence and It loses.In this step, determine the first package that transmitting terminal saves flow media data packet and redundant data packets and value, and should With value as second and value.Wherein, if when sending the first package data-bag lost occurs for transmitting terminal, first and value less than second And value.

When it is implemented, not being defined successively to step 301 and step 302 execution sequence, step 302 can also be first carried out It executes step 301 afterwards or the two is performed simultaneously.

Step 303, according to described first and value with described second and value, determine the packet loss of first package.

When it is implemented, can calculate first and value with second and value difference, by the absolute value of the difference divided by the second He It is worth resulting as a result, packet loss as the first package.

Step 304, the product of the flow media data packet number of second package and the packet loss of first package is determined.

Step 305, the product is rounded up into resulting result as the coding redundancy degree of second package.

When it is implemented, obtain the second package flow media data packet number, and determine obtain flow media data packet number with The product of the packet loss of first package.It rounds up to the product, and the resulting knot that rounds up will be carried out to the product Fruit, the coding redundancy degree as the second package.Alternatively, will can also be rounded downwards to the product resulting as a result, as The coding redundancy degree of two packages, here without limitation.

Optionally, extension header is added in the flow media data packet and redundant data packets of current package to be sent； The extension header of the flow media data packet includes that the serial number of the affiliated package of the flow media data packet, the flow media data packet have Imitate the discrimination flag of the serial number and the flow media data packet of data length, the flow media data packet in affiliated package；It is described The extension header of redundant data packets includes that the serial number of the affiliated package of the redundant data packets, the valid data of the redundant data packets are long The discrimination flag of degree, serial number and the redundant data packets of the redundant data packets in affiliated package.Transmitting terminal sends package Flow media data packet and redundant data packets after adding extension header.Receiving end can determine the packet received according to extension header The flow media data packet number and redundant data packets number of group.In addition, receiving end can also be according to the Streaming Media number of the package received Fec decoder processing is carried out to the package according to the extension header in packet and redundant data packets, to recover the package in transmission process The flow media data packet of middle loss.

Specifically, the format of data packet (including flow media data packet and redundant data packets) is as shown in figure 4, below to Fig. 4 In each field be introduced:

Pkt_Size field is used to fill the valid data length of data packet, the data packet after data packet is zero padding When, the payload before the valid data zero padding of the data packet, such receiving end can recover this according to the Pkt_Size field Payload in data packet is used as valid data, optionally, Pkt_Size word to avoid zero will supplemented in the data packet The length of section is 2 bytes.

Blk_Num field is used to fill the serial number of the affiliated package of the data packet, wherein the flow media data packet of same package It is identical with the value of the Blk_Num field in redundant data packets, it should in the flow media data packet and redundant data packets of different packages The value of Blk_Num field is different, and receiving end can distinguish packet belonging to received each data packet according to Blk_Num field Group, optionally, the length of Blk_Num field are 4 bytes.

Blk_Seq_Num field is for filling serial number of the data packet in its affiliated package, wherein according to the number of package According to the transmission sequence of packet from front to back, serial number is set for each data packet in the package, wherein the serial number in package is got over Greatly, the data packet transmission sequence more rearward, also, the serial number of redundant data packets be greater than flow media data packet serial number, and respectively The serial number of a data packet is different and the serial number of latter data packet is incremented by 1 on the basis of the serial number of last data packet, starting The serial number of data packet can be 0 or 1, here without limitation, at this point, can all there is uniquely all data packets in same package Serial number, receiving end can according to the Blk_Seq_Num field and Blk_Num field, identify package after being transmitted to receiving end, The data packet lost in the package, optionally, the length of Blk_Seq_Num field are 1 byte.

Pkt_Redundancy field is used to fill the discrimination flag of the data packet, when the discrimination flag filled in the field When for particular value, indicates that the data packet is redundant data packets, when the discrimination flag filled in the field is nonspecific value, indicate The data packet is flow media data packet, and optionally, particular value is the coding redundancy degree of the affiliated package of the data packet, the particular value It can be other numerical value, for example be 0xff.

The title of above-mentioned each field is merely illustrative title, can also be set as other names according to by the title of a field Claim, as long as each field name is different.

Transmitting terminal framework provided by the embodiments of the present application is introduced below with reference to Fig. 5.

The framework of the transmitting terminal of the embodiment of the present application is as shown in Figure 5, wherein the hardware structure of transmitting terminal is shown in fig. 5 Tiny6410 hardware structure, the software architecture of transmitting terminal by the H.264 video coding process of Fig. 5, Streaming Media transmitting terminal process with And linux kernel composition, linux kernel includes the driving of corresponding hardware module in hardware structure.The hardware structure of transmitting terminal and The step of software architecture cooperates, and realizes the method for transmitting terminal transmission flow media data packet provided by the embodiments of the present application.When So, Fig. 5 is only the example architecture for being suitable for a kind of transmitting terminal of the embodiment of the present application, be can also be used other real suitable for the application The framework for applying example realizes the method that stream medium data is transmitted in the transmission end of the embodiment of the present application.

Each module in the Tiny6410 hardware structure in transmitting terminal is simply introduced below:

Tiny6410 is the development board of FriendlyARM company customization, is integrated on the development board based on ARM11 kernel S3C6410 microprocessor, OV9650 imaging sensor, RT3070USB WIFI, 256M DDR RAM, 256M Nand Flash Deng additionally providing a variety of Peripheral Interface (not shown)s such as USB Host, SD/MMC and CameraIF.The choosing of video acquisition part With OV9650 imaging sensor, OV9650 imaging sensor supports highest 30fps VGA (640x480) YUV mode, substantially may be used To reach the demand of real-time video monitoring；Because S3C6410 microprocessor internal is integrated with a hardware coder (Multi Format Codec, MFC), when hardware encoding, can be not take up the cpu resource of S3C6410 microprocessor, therefore select hardware encoding Device carries out vision hardware coding to collected video data, specifically can carry out H.264 format to collected video data Vision hardware coding；Transmitting terminal wireless network transmissions part has selected RT3070USB WIFI module to build wireless aps (Access Point, access point), transmission rate reaches as high as 150Mbps.

The software architecture in transmitting terminal is introduced below:

H.264 video coding process passes through Unix Domain in response to the address the RTSP access request of receiving end Socket interprocess communication issues video acquisition and Video coding is instructed to H.264 video coding process；H.264 Video coding into After the every acquisition for completing one-frame video data of journey and coding, the video data encoder of the frame video data is sent to Streaming Media hair Sending end process, the video data encoder received is packaged into rtp streaming media data packet by Streaming Media transmitting terminal process, and utilizes this The method for the transmission flow media data packet that body asks embodiment to provide carries out the transmission of flow media data packet.

H.264 video coding process and the video coded data message format of Streaming Media transmitting terminal interprocess communication can be such as figures Shown in 6, wherein Command field is the custom command code of 1 byte, and Datalen field is the integer variable of 4 bytes.

In the case where H.264 video coding process sends the message to Streaming Media transmitting terminal process, in Fig. 6 Command field starts to send video coded data message mark for filling, alternatively, Command field prepares for filling End of identification, that is, H.264 video coding process terminates to send message to Streaming Media transmitting terminal process, and Datalen field is for filling out It fills in currently transmitted message and removes the effective video coded data length after header, payload field is currently transmitted for carrying Message in effective video coded data.

In the case where Streaming Media transmitting terminal process sends the message to H.264 video coding process, in Fig. 6 Command field is for filling successfully response identification, and the mark is for illustrating that Streaming Media transmitting terminal process receives H.264 video The message that coding pass is sent, alternatively, Command field is unsuccessfully identified for filling to send, the mark is for illustrating Streaming Media Transmitting terminal process does not receive the message that H.264 video coding process is sent, since Streaming Media transmitting terminal process is to H.264 video Without carrying effective video data encoder length and payload in the message of coding pass feedback, thus Datalen field with And payload field is sky.

Wherein, different value distinguishings can be used in the different marks of Command field filling, here not to specific value It is defined.

H.264 video coding process uses multi-threaded architecture, including Video4Linux2 video acquisition thread, H.264 Hard Video coding compression thread and video data export thread.Data interaction between each thread uses Fifo (first in, first out) machine To realize, each thread parallel executes system.

Wherein, V4L2 (i.e. Video4Linux2) is the specification of the video acquisition driver of mainstream under linux system, directly It connects and calls V4L2API interface that can collect the video requency frame data in OV9650 imaging sensor.In order to simplify video acquisition Operation, V4L2 video acquisition thread encapsulate V4L2 specification, are reduced to 4 steps: (1) video acquisition initializes: V4L2 order is called to complete to open the imaging sensor in Fig. 5, the device attribute of query image sensor, setting video frame lattice Formula, requesting the V4L2 integrated in the linux kernel of Fig. 5 driving distribution frame buffering, (frame buffering refers to individually opens up for video requency frame data One piece of memory headroom), driving frame buffering be mapped to user's space (for running in Fig. 5 H.264 video coding process and stream matchmaker The space of body transmitting terminal process) etc. work；(2) it obtains a video frame: V4L2 order being called to obtain from the imaging sensor in Fig. 5 Take a video frame；(3) video frame enqueue: V4L2 order is called to complete the work that frame buffering returns to fifo queue；(4) video Acquisition is closed: V4L2 order being called to close the imaging sensor in Fig. 5.

H.264 it is H.264 hard to use the hardware coder progress in S3C6410 microprocessor for hard Video coding compression thread Part coding compression, so as to reduce the occupancy of microprocessor CPU resource and improve Video coding speed.Hardware compression device mentions Supplied multiple api interfaces for execute H.264 hardware encoding compress, compression process include hardware coder initialization, H.264 firmly The execution of part coding compression, the video data encoder H.264 after hardware encoding generate and hardware coder discharges etc..

Video data output thread is mainly responsible for the interaction that video coded data message is carried out with Streaming Media transmitting terminal process, This thread will mainly have been subjected to the video data encoder after Video coding and be sent to Streaming Media transmitting terminal process.

Wherein, H.264 Streaming Media transmitting terminal process can be encoded lattice based on the LIVE555 multimedia transmission frame of open source The transmission of flow media data of formula.When it is implemented, BasicTaskScheduler object and task tune can be created in the frame Device is tried, environment is set using；RTSP service and session are created, and session is added in RTSP service, to realize Streaming Media Data transmission environments are built.Later, in order to realize Streaming Media transmitting terminal process transmission flow media data packet and redundant data packets, Thread can be configured to export in the frame from video data obtain the class of video data encoder (for example can be named as PipeStreamSource class), such video data encoder that will acquire is transmitted to for notifying such to continue to obtain video volume The interface (for example being named as doGetNextFrame () interface) and error detection component of code data (for example are named as H264FrameSource component), error detection component analyzes the video data encoder of such forwarding according to H.264 code flow structure Whether meet coding criterion H.264, corresponding SDP description is generated if meeting, if not meeting, abandons the video got Coded data.The video data encoder for meeting coding criterion is sent to package assembling and (for example is named as H264RTPSink group Part), video data encoder is packaged into rtp streaming media data packet by package assembling, and envelope is turned obtained rtp streaming media by package assembling Data packet is sent to FEC redundancy encoding component (for example being named as H264FecEncode component), and FEC redundancy encoding component utilizes The method of the transmission flow media data packet of transmitting terminal provided by the embodiments of the present application sends the flow media data packet and redundancy of package Data packet.

Wherein, FEC redundancy encoding component can customize 3 api functions of encapsulation, comprising: 1, api function AFecInit (RS_ K, RS_R, FecBuf), for completing FEC redundancy encoding structure initialization, and it is corresponding superfluous for obtained redundant data packets distribution Remaining data packet buffer area (corresponding above-mentioned second buffer area), RS_K is preset value in the function, and RS_R is coding redundancy degree, FecBuf is redundant data packets buffer area, wherein the size of redundant data packets buffer area can be set dynamically according to coding redundancy degree, The redundant data packets buffer size is specially the product of coding redundancy degree Yu redundant data packet length；2, api function AFecEnc (SourceFec, RS_K*PktSize, PktSize) carries out FEC redundancy encoding for the flow media data packet to package, and will The FEC redundant data packets of generation are stored in the redundant data packets buffer area of distribution, wherein PktSize is flow media data packet Valid data length, RS_K is preset value；3, api function AFecUninit (FecHandle pHandle), at one When the flow media data packet and redundant data packets of package are all sent completely, redundant data packets buffer area is discharged.

It is high to reliability, requirement of real-time that the embodiment of the present application can be applied to home security, vehicle-mounted monitoring, target following etc. System.

The embodiment of the present application provides a kind of method for transmitting stream medium data, is applied to transmitting terminal, as shown in fig. 7, comprises:

Step 701, the flow media data packet and redundant data packets of the package that transmitting terminal is sent, the redundant data packets are received For the redundant data packets obtained after FEC redundancy encoding.

When it is implemented, corresponding receiving end can after transmitting terminal sends the flow media data packet and redundant data packets of package Receiving stream media data packet and redundant data packets, are influenced by network environment, transmitting terminal send package flow media data packet and During redundant data packets, it is possible that the case where packet loss, at this point, receiving end possibly can not receive all stream matchmakers of package Volume data packet and redundant data packets.Wherein, redundant data packets are flow media data packet and coding redundancy of the transmitting terminal according to package Degree carries out the redundant data packets obtained after FEC redundancy encoding.

Optionally, receiving end saves the flow media data packet and redundant data packets of the package received.

Optionally, the package that transmitting terminal is sent is equivalent to the first package in step 201.

Step 702, the flow media data packet number and redundant data packets number of the package received are determined.

Step 703, the flow media data packet number of the determining package and redundant data packets number are fed back into the hair Sending end.

When it is implemented, the flow media data packet number of the determining package and redundant data packets number individually can be fed back to Transmitting terminal, can also be by the way that in the RR of RTCP (Real-time Control Protocol, real time control protocol), (recipient is reported Accuse) flow media data packet number and redundant data packets number that package is added in packet, by RR packet by the flow media data packet of package Several and redundant data packets number feeds back to transmitting terminal.

Optionally, however, it is determined that the package flow media data packet number and the sum of redundant data packets number not less than default Value, then feed back to transmitting terminal for the flow media data packet number of the determining package and redundant data packets number, wherein this is default It is worth identical as preset value mentioned above.

Optionally, the content provided according to Fig. 8, by the flow media data packet number and redundant data of the determining package Packet number feeds back to the transmitting terminal:

Step 801, the first extended field, the second extended field and third extended field are added in the RR packet of RTCP.

Specifically the first extended field, the second extended field and third can be added on the head of RR packet extend word.

Step 802, first extended field is written into the serial number of the package, by the flow media data packet of the package Second extended field is written in number, and, the third extended field is written into the redundant data packets number of the package, is obtained RR packet is written.

When it is implemented, the serial number of package is used to distinguish different packages, in practical applications, as long as by the sequence of package Number, the redundant data packets number of the flow media data packet number of the package and the package be respectively written into three extended fields, than Such as, also first extended field can be written in the package flow media data packet number, described in the serial number write-in by the package Second extended field, and, the third extended field is written into the redundant data packets number of the package.

It should be noted that transmitting terminal and receiving end need to make an appointment be written RR packet different extended fields be written it is interior Hold, such transmitting terminal just can the corresponding flow media data packet number of Exact Solutions precipitation and redundant data packets after receiving write-in RR packet Number.

Step 803, said write RR packet is fed back into the transmitting terminal.

When it is implemented, transmitting terminal is fed back to by RR packet is written by rtcp protocol, so that transmitting terminal is according to write-in RR packet In extended field, parse the flow media data packet number and redundant data packets number of corresponding package, as receiving end feedback packet The flow media data packet number and redundant data packets number of group.

The embodiment of the present application is fed back to the flow media data packet number of package and redundant data packets number using the RR in RTCP Transmitting terminal, flow media data packet number and redundant data packets number without individually sending package feed back to transmitting terminal, to a certain degree On save Internet resources.

Optionally, the flow media data packet includes extension header, which includes belonging to the flow media data packet The discrimination flag of the serial number of package and the flow media data packet, the redundant data packets include extension header, the extension header The discrimination flag of serial number and the redundant data packets including the affiliated package of the redundant data packets, wherein package in extension header Serial number and discrimination flag meaning above will introduce, be not described herein.

Optionally, the content that can be provided according to Fig. 9, receives the flow media data packet and redundancy of the package that transmitting terminal is sent Data packet:

Step 901, it is determined in response to data packet arrival notice according to the discrimination flag in the extension header of the data packet The data packet is flow media data packet or redundant data packets.

When it is implemented, parsing the data packet when monitoring socket descriptor of receiving end receives data packet arrival notice Extension header in discrimination flag, discrimination flag indicate the data packet be flow media data packet in the case where, execute step 902, in the case where discrimination flag indicates that the data packet is redundant data packets, execute step 903.

More specifically, if the serial number of affiliated package is less than or equal to the sequence of specified package in the extension header of the data packet Number, then by the data packet discarding, specified package is the serial number for the package that receiving end feeds back to transmitting terminal recently, wherein anti-recently Feed transmitting terminal package be all packages for having fed back to transmitting terminal in the maximum package of serial number.

Step 902, which is stored to the first chained list buffer area.

Can according to flow media data packet its affiliated package the ascending sequence of serial number, by the flow media data packet of package It stores to the first chained list buffer area.

Step 903, which is stored to the second chained list buffer area.

Sequence that can be ascending in the serial number of its affiliated package according to redundant data packets, the redundant data packets of package are deposited It stores up to the second chained list buffer area.

Wherein, the first chained list buffer area and the second chained list buffer area are respectively to have list structure and orderly buffer area, The flow media data packet received can be slow to the first chained list according to the ascending sequential storage of its serial number in affiliated package Area is deposited, the redundant data packets received can be according to the ascending sequential storage of its serial number in affiliated package to the second chained list Buffer area, to carry out quickly searching traversing operation in corresponding chained list buffer area.

Optionally it is determined that the flow media data packet number and redundant data packets number of the package received, specifically include: Determine the flow media data packet number of serial number of the extension header comprising the package in the first chained list buffer area, and, it determines Extension header includes the redundant data packets number of the serial number of the package in second chained list buffer area.

When it is implemented, traversing the first chained list buffer area and the second chain table cache respectively according to the extension header of data packet Area, to count the flow media data packet number and the second chained list buffer area of the serial number in the first chained list buffer area comprising same package In comprising the same package serial number redundant data packets number.

Optionally, if the first chained list buffer area is without available space, by the stream medium data of the determining package Packet number and redundant data packets number feed back to transmitting terminal.

When it is implemented, receiving end can not continue stored stream media data when the first chained list buffer area is without available space Packet can be fed back at this time in order to guarantee that transmitting terminal can determine the coding redundancy degree of current package to be sent in real time to transmitting terminal The flow media data packet number and redundant data packets number of package.

Optionally, the extension header of the flow media data packet further include the valid data length of the flow media data packet with And serial number of the flow media data packet in affiliated package, the extension header of the redundant data packets further include the redundant data packets Serial number in affiliated package of valid data length and the redundant data packets, above to the affiliated package in extension header Interior serial number and valid data length is illustrated, and is not described herein.

As shown in Figure 10, the method for transmission stream medium data provided by the embodiments of the present application, further includes:

Step 1001, by the flow media data packet of the package of the first chained list buffer area storage, and, described the The redundant data packets of the package of two chained list buffer areas storage are stored to third chained list buffer area.

Wherein, third chained list buffer area be with list structure and orderly buffer area, can according to flow media data packet and The redundant data packets sequence ascending in the serial number of its affiliated package, the flow media data packet of package and redundant data packets are deposited Storage is to third chained list buffer area, to carry out quickly searching traversing operation in third chained list buffer area, thus to third chained list Flow media data packet and redundant data packets in buffer area carry out fec decoder processing.

Optionally, before step 1001, the serial number of the affiliated package in the first chained list buffer area in extension header is determined Less than the flow media data packet of the serial number of package in step 1001；The extension header for the flow media data packet determined is removed, and Video decoding process is carried out to the flow media data packet after removal extension header, specifically when Video coding is compiled using H.264 video When code algorithm, video decoding is accordingly using H.264 video decoding algorithm.

Step 1002, the expanding packet of the flow media data packet and redundant data packets in the third chained list buffer area is removed Head.

When it is implemented, since extension header is the packet header additionally added, in streaming media data packet and redundant digit Before carrying out fec decoder processing according to packet, the extension header by the extension header of flow media data packet and redundant data packets is needed Removal.

Step 1003, according to the extension header and redundant data packets of each flow media data packet in third chained list buffer area Extension header, and removal extension header after flow media data packet and redundant data packets, to the package carry out fec decoder Processing.

When it is implemented, for the data packet after removal extension header each in any package, (data packet is Streaming Media number According to packet or redundant data packets), the extension header of the data packet is parsed, according to the sequence of the affiliated package of data packet in extension header Number, determine package belonging to the data packet；Determine that the data packet is flow media data packet according to the discrimination flag in extension header Or redundant data packets；According to the valid data length of the data packet and the length of the data packet, whether the data packet is determined Zero padding operation is carried out, if so, Z-operation is carried out to the data packet, to obtain raw data packets；The data packet is recorded in institute Belong to the serial number in package.According to the serial number of each data packet in any package, it is determined whether lost data packets, if so, root According to the flow media data packet and redundant data packets after the removal extension header of any package, FEC solution is carried out to any package Code processing, and corresponding video decoding is carried out to fec decoder treated data.When Video coding is using H.264 Video coding When algorithm, video decoding is accordingly using H.264 video decoding algorithm.

Optionally, it after carrying out FEC video decoding process, deletes in the first chained list buffer area and has carried out fec decoder The flow media data packet of processing, and, delete the redundant data that fec decoder processing has been carried out in the second chained list buffer area Packet；And empty the third chained list buffer area.The spatial cache of receiving end can be discharged in this way.Optionally, the first chained list is slow The size for depositing area, the second chained list buffer area and third chained list buffer area is identical, for example is preset value and flow media data packet 2 times of the product of size.Certainly, the size of the first chained list buffer area, the second chained list buffer area and third chained list buffer area Can be different, here without limitation.

Optionally, in the case where the first chained list buffer area is without available space, to extension header in the first chained list buffer area The smallest flow media data packet of serial number of affiliated package carry out video decoding, and delete extension header in the first chained list buffer area Affiliated package the smallest flow media data packet of serial number, meanwhile, delete the second chained list buffer area in corresponding redundant data packets, Corresponding redundant data packets are the redundant data packets that the flow media data packet deleted with the first chained list buffer area belongs to same package.

It, can be by the ffplay mould under FFmpeg multi-media decoding and encoding frame when the embodiment of the present application carries out fec decoder processing Block can construct fec decoder component, under ffplay module specifically as decoded base to realize transport stream in the embodiment of the present application The step of media data.

The embodiment of the present application also provides a kind of system for transmitting stream medium data, comprising:

The embodiment of the present application also provides a kind of device for transmitting stream medium data, is applied to transmitting terminal, as shown in figure 11, packet It includes:

First determining module 1101, the stream matchmaker of adjacent upper one the first package sent for being fed back according to receiving end Volume data packet number and redundant data packets number determine the coding redundancy degree of current second package to be sent；

Coding module 1102, for the coding redundancy degree according to second package, to the Streaming Media of second package Data packet carries out forward error correction FEC redundancy encoding, obtains the redundant data packets of second package；

Sending module 1103, for sending the flow media data packet and redundant data packets of second package.

Optionally, first determining module 1101, is specifically used for:

Optionally, the device of transmission stream medium data provided by the embodiments of the present application, further includes:

Second determining module 1104, for being carried out in flow media data packet of the coding module to second package Before FEC redundancy encoding, determine that the flow media data packet number of the second package of storage is equal to preset value.

Zero padding module 1105, it is superfluous for carrying out FEC in flow media data packet of the coding module to second package Before remaining coding, the length that length in second package is less than the flow media data packet of preset length is carried out to mend 0 operation, with Its length is set to be equal to preset length.

The embodiment of the present application also provides a kind of device for transmitting stream medium data, is applied to receiving end, as shown in figure 12, packet It includes:

Receiving module 1201, the flow media data packet and redundant data packets of the package for receiving transmitting terminal transmission are described Redundant data packets are obtained redundant data packets after forward error correction FEC redundancy encoding；

Determining module 1202, for determining the flow media data packet number and redundant data packets of the package received Number；

Feedback module 1203, for feeding back the flow media data packet number of the package determined and redundant data packets number To the transmitting terminal.

Optionally, the feedback module 1203, is specifically used for:

Said write RR packet is fed back into the transmitting terminal.

Optionally, the flow media data packet includes extension header, which includes belonging to the flow media data packet The discrimination flag of the serial number of package and the flow media data packet, the redundant data packets include extension header, the extension header The discrimination flag of serial number and the redundant data packets including the affiliated package of the redundant data packets, then the receiving module 1201, It is specifically used for:

Optionally, the determining module 1202, is specifically used for:

Optionally, the feedback module 1203, is specifically used for:

Optionally, the extension header of the flow media data packet further include the valid data length of the flow media data packet with And serial number of the flow media data packet in affiliated package, the extension header of the redundant data packets further include the redundant data packets Serial number in affiliated package of valid data length and the redundant data packets, then transport stream matchmaker provided by the embodiments of the present application The device of volume data, further includes:

Memory module 1204, the flow media data packet of the package for storing the first chained list buffer area, with And the redundant data packets of the package of the second chained list buffer area storage are stored to third chained list buffer area；

Module 1205 is removed, for removing flow media data packet in the third chained list buffer area and redundant data packets Extension header；

Decoder module 1206, for according to the extension header of each flow media data packet in third chained list buffer area and superfluous The extension header of remaining data packet, and removal extension header after flow media data packet and redundant data packets, to the package into The processing of row fec decoder.

Removing module 1207 deletes the flow media data packet that fec decoder processing has been carried out in the first chained list buffer area, And delete the redundant data packets that fec decoder processing has been carried out in the second chained list buffer area；And empty the third Chained list buffer area.

The embodiment of the present application provides a kind of nonvolatile computer storage media, which, which is stored with, to hold Line program, which, which executes, realizes any transport stream matchmaker applied to transmitting terminal provided by the above embodiment The step of method of volume data.

The embodiment of the present application also provides a kind of computer equipment, on a memory including memory, processor and storage Computer program, the processor realize any transmission applied to transmitting terminal provided by the above embodiment when executing described program The step of method of stream medium data.

Computer equipment provided by the embodiments of the present application, for executing any in above-described embodiment applied to transmitting terminal The method for transmitting stream medium data, as shown in figure 13, for the hardware configuration signal of computer equipment described in the application implementation Figure, the computer equipment are specifically as follows desktop computer, portable computer, smart phone, tablet computer etc..Specifically, The computer equipment may include memory 1301, the computer program of processor 1302 and storage on a memory, the place Reason device realizes the method for any transmission stream medium data applied to transmitting terminal in above-described embodiment when executing described program Step.Wherein, memory 1301 may include read-only memory (ROM) and random access memory (RAM), and to processor The program instruction and data stored in 1302 offer memories 1301.

Further, computer equipment described in the embodiment of the present application can also include input unit 1303 and output Device 1304 etc..Input unit 1303 may include keyboard, mouse, touch screen etc.；Output device 1304 may include that display is set It is standby, such as liquid crystal display (Liquid Crystal Display, LCD), cathode-ray tube (Cathode Ray Tube, CRT), Touch screen etc..Memory 1301, processor 1302, input unit 1303 and output device 1304 can by bus or other Mode connects, in Figure 13 for being connected by bus.

Processor 1302 calls the program instruction of the storage of memory 1301 and executes above-mentioned reality according to the program instruction of acquisition The method that any transmission stream medium data applied to transmitting terminal of example offer is provided.

The embodiment of the present application provides a kind of nonvolatile computer storage media, which, which is stored with, to hold Line program, which, which executes, realizes any transport stream matchmaker applied to receiving end provided by the above embodiment The step of method of volume data.

The embodiment of the present application also provides a kind of computer equipment, on a memory including memory, processor and storage Computer program, the processor realize any transmission applied to receiving end provided by the above embodiment when executing described program The step of method of stream medium data.

Computer equipment provided by the embodiments of the present application, for executing any in above-described embodiment applied to receiving end The method for transmitting stream medium data, as shown in figure 14, for the hardware configuration signal of computer equipment described in the application implementation Figure, the computer equipment are specifically as follows desktop computer, portable computer, smart phone, tablet computer etc..Specifically, The computer equipment may include memory 1401, the computer program of processor 1402 and storage on a memory, the place Reason device realizes the method for any transmission stream medium data applied to receiving end in above-described embodiment when executing described program Step.Wherein, memory 1401 may include read-only memory (ROM) and random access memory (RAM), and to processor The program instruction and data stored in 1402 offer memories 1401.

Further, computer equipment described in the embodiment of the present application can also include input unit 1403 and output Device 1404 etc..Input unit 1403 may include keyboard, mouse, touch screen etc.；Output device 1404 may include that display is set It is standby, such as liquid crystal display (Liquid Crystal Display, LCD), cathode-ray tube (Cathode Ray Tube, CRT), Touch screen etc..Memory 1401, processor 1402, input unit 1403 and output device 1404 can by bus or other Mode connects, in Figure 14 for being connected by bus.

Processor 1402 calls the program instruction of the storage of memory 1401 and executes above-mentioned reality according to the program instruction of acquisition The method that any transmission stream medium data applied to receiving end of example offer is provided.

It should be noted that although being referred to several modules of the device of transmission stream medium data in the above detailed description, Be it is this division be only exemplary it is not enforceable.In fact, according to presently filed embodiment, above-described two The feature and function of a or more module can embody in a module.Conversely, the feature of an above-described module It can be to be embodied by multiple modules with further division with function.

The application is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present application Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates, Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or The function of being specified in multiple boxes.

These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one The step of function of being specified in a box or multiple boxes.

Although the preferred embodiment of the application has been described, it is created once a person skilled in the art knows basic Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as It selects embodiment and falls into all change and modification of the application range.

Obviously, those skilled in the art can carry out various modification and variations without departing from the essence of the application to the application Mind and range.In this way, if these modifications and variations of the application belong to the range of the claim of this application and its equivalent technologies Within, then the application is also intended to include these modifications and variations.

Claims

1. a kind of method for transmitting stream medium data characterized by comprising

According to the flow media data packet number and redundant data packets of adjacent upper one the first package sent of receiving end feedback Number determines the coding redundancy degree of current second package to be sent；

According to the coding redundancy degree of second package, forward error correction FEC is carried out to the flow media data packet of second package Redundancy encoding obtains the redundant data packets of second package；

Send the flow media data packet and redundant data packets of second package.

2. the method according to claim 1, wherein the coding of current second package to be sent of the determination is superfluous Remaining specifically includes:

The flow media data packet number of first package that the receiving end is fed back and redundant data packets number and value as the One and value；And

3. according to the method described in claim 2, it is characterized in that, the packet loss of the determination first package, specific to wrap It includes:

By the absolute value of the difference divided by described second and value it is resulting as a result, packet loss as first package.

4. the method according to claim 1, wherein the flow media data packet to second package carries out FEC Before redundancy encoding, further includes:

5. the method according to claim 1, wherein the flow media data packet to second package carries out FEC Before redundancy encoding, further includes:

The length that length in second package is less than the flow media data packet of preset length is carried out to mend 0 operation, so that it is grown Degree is equal to preset length.

6. -5 any method according to claim 1, which is characterized in that the flow media data packet includes extension header, The extension header includes the serial number of the affiliated package of the flow media data packet, the valid data length of the flow media data packet, the stream The discrimination flag of serial number and the flow media data packet of the media data packet in affiliated package；The redundant data packets include expanding Packet header is opened up, which includes the serial number of the affiliated package of the redundant data packets, the valid data length of the redundant data packets, is somebody's turn to do The discrimination flag of serial number and the redundant data packets of the redundant data packets in affiliated package.

7. a kind of method for transmitting stream medium data characterized by comprising

The flow media data packet and redundant data packets of the package that transmitting terminal is sent are received, the redundant data packets are forward error correction The redundant data packets obtained after FEC redundancy encoding；

8. the method according to the description of claim 7 is characterized in that the flow media data packet number by the determining package And redundant data packets number feeds back to the transmitting terminal, specifically includes:

The first extended field, the second extended field and third are added in the Receiver Report RR packet of real time control protocol RTCP Extended field；

First extended field is written into the serial number of the package, by the flow media data packet number write-in of the package described the Two extended fields, and, the third extended field is written into the redundant data packets number of the package, is written RR packet；

Said write RR packet is fed back into the transmitting terminal.

9. the method according to the description of claim 7 is characterized in that the flow media data packet includes extension header, the extension Packet header includes the serial number of the affiliated package of the flow media data packet and the discrimination flag of the flow media data packet, the redundant data Packet includes extension header, which includes the differentiation of the serial number and the redundant data packets of the affiliated package of the redundant data packets Mark, the then flow media data packet and redundant data packets of the package that the reception transmitting terminal is sent, specifically includes:

In response to data packet arrival notice, according to the discrimination flag in the extension header of the data packet, determine that the data packet is stream Media data packet or redundant data packets；

10. according to the method described in claim 9, it is characterized in that, the Streaming Media number for the package that the determination receives According to packet number and redundant data packets number, specifically include:

Determine the flow media data packet number of serial number of the extension header comprising the package in the first chained list buffer area, and,

11. according to the method described in claim 9, it is characterized in that, the flow media data packet by the determining package Several and redundant data packets number feeds back to the transmitting terminal, specifically includes:

If it is determined that the package flow media data packet number and the sum of redundant data packets number be not less than preset value, alternatively, if institute The first chained list buffer area is stated without available space, then by the flow media data packet number and redundant data packets number of the determining package Feed back to the transmitting terminal.

12. according to the method described in claim 9, it is characterized in that, the extension header of the flow media data packet further includes this Serial number of the valid data length and the flow media data packet of flow media data packet in affiliated package, the redundant data packets Extension header further include the serial number of the valid data length and the redundant data packets of the redundant data packets in affiliated package, Then the method also includes:

By the flow media data packet of the package of the first chained list buffer area storage, and, the second chained list buffer area The redundant data packets of the package of storage are stored to third chained list buffer area；

According to the extension header of each flow media data packet and the extension header of redundant data packets in third chained list buffer area, with And flow media data packet and redundant data packets after removal extension header, fec decoder processing is carried out to the package.

13. a kind of device for transmitting stream medium data characterized by comprising

First determining module, the flow media data packet of adjacent upper one the first package sent for being fed back according to receiving end Several and redundant data packets number determines the coding redundancy degree of current second package to be sent；

Coding module, for the coding redundancy degree according to second package, to the flow media data packet of second package into Row forward error correction FEC redundancy encoding, obtains the redundant data packets of second package；

14. a kind of device for transmitting stream medium data characterized by comprising

Receiving module, the flow media data packet and redundant data packets of the package for receiving transmitting terminal transmission, the redundant data Wrap the redundant data packets to obtain after forward error correction FEC redundancy encoding；

Feedback module, for the flow media data packet number of the package determined and redundant data packets number to be fed back to the hair Sending end.

15. a kind of system for transmitting stream medium data characterized by comprising

Receiving end, the flow media data packet and redundant data packets of the first package for receiving transmitting terminal transmission, the redundant digit Redundant data packets according to packet to be obtained after forward error correction FEC redundancy encoding；Determine the Streaming Media of first package received Number-of-packet and redundant data packets number；By the flow media data packet number and redundant data packets number of determining first package Feed back to the transmitting terminal；

The transmitting terminal, the flow media data packet number and redundant data packets number of the first package for being fed back according to receiving end, Determine the coding redundancy degree of current second package to be sent；According to the coding redundancy degree of second package, to described second The flow media data packet of package carries out forward error correction FEC redundancy encoding, obtains the redundant data packets of second package；Send institute State the flow media data packet and redundant data packets of the second package, wherein the first package is adjacent with second package upper One has sent package.