WO2016110275A1 - 一种基于媒体内容的fec机制 - Google Patents

一种基于媒体内容的fec机制 Download PDF

Info

Publication number
WO2016110275A1
WO2016110275A1 PCT/CN2016/070525 CN2016070525W WO2016110275A1 WO 2016110275 A1 WO2016110275 A1 WO 2016110275A1 CN 2016070525 W CN2016070525 W CN 2016070525W WO 2016110275 A1 WO2016110275 A1 WO 2016110275A1
Authority
WO
WIPO (PCT)
Prior art keywords
fec
signaling
media
coding
different
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/070525
Other languages
English (en)
French (fr)
Inventor
徐异凌
张文军
孙军
汤旭国
黄巍
李博
管云峰
柳宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiao Tong University
Original Assignee
Shanghai Jiao Tong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201510010097.9A external-priority patent/CN105827361B/zh
Priority claimed from CN201510080576.8A external-priority patent/CN105991226B/zh
Priority claimed from CN201510673091.XA external-priority patent/CN106603192B/zh
Priority claimed from CN201510673115.1A external-priority patent/CN106603193B/zh
Priority to KR1020177021722A priority Critical patent/KR102083302B1/ko
Priority to JP2017536358A priority patent/JP2018505597A/ja
Priority to KR1020197034659A priority patent/KR102251278B1/ko
Application filed by Shanghai Jiao Tong University filed Critical Shanghai Jiao Tong University
Priority to US15/542,079 priority patent/US10469202B2/en
Priority to CA2998900A priority patent/CA2998900C/en
Publication of WO2016110275A1 publication Critical patent/WO2016110275A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • H04L1/0056—Systems characterized by the type of code used
    • H04L1/007—Unequal error protection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
    • H04L1/0009—Systems modifying transmission characteristics according to link quality, e.g. power backoff by adapting the channel coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
    • H04L1/0015—Systems modifying transmission characteristics according to link quality, e.g. power backoff characterised by the adaptation strategy
    • H04L1/0017—Systems modifying transmission characteristics according to link quality, e.g. power backoff characterised by the adaptation strategy where the mode-switching is based on Quality of Service requirement
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
    • H04L1/0023—Systems modifying transmission characteristics according to link quality, e.g. power backoff characterised by the signalling
    • H04L1/0028—Formatting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • H04L1/0041—Arrangements at the transmitter end
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • H04L1/0056—Systems characterized by the type of code used
    • H04L1/0057—Block codes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • H04L1/0076—Distributed coding, e.g. network coding, involving channel coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/1066—Session management
    • H04L65/1101—Session protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/762—Media network packet handling at the source 
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/35—Unequal or adaptive error protection, e.g. by providing a different level of protection according to significance of source information or by adapting the coding according to the change of transmission channel characteristics
    • H03M13/353—Adaptation to the channel
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/35—Unequal or adaptive error protection, e.g. by providing a different level of protection according to significance of source information or by adapting the coding according to the change of transmission channel characteristics
    • H03M13/356—Unequal error protection [UEP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • H04L1/0045—Arrangements at the receiver end
    • H04L1/0047—Decoding adapted to other signal detection operation
    • H04L1/0048—Decoding adapted to other signal detection operation in conjunction with detection of multiuser or interfering signals, e.g. iteration between CDMA or MIMO detector and FEC decoder

Definitions

  • the present invention relates to the field of multimedia transmission technologies, and more particularly to a FEC (Forward Error Correction) mechanism based on media content.
  • FEC Forward Error Correction
  • IP/TCP or UDP messages are used in the Internet to transmit media data
  • broadcasts are transmitted through MPEG2-TS.
  • UDP packets may be lost after passing through multiple network devices.
  • the broadcast TS stream may be misinterpreted due to the influence of the transmission environment, causing the terminal side to be damaged or the sound to pause.
  • FEC Forward Error Correction
  • the media data is error-corrected and encoded by the server side, and the redundant information is added and sent, and the terminal side performs reverse FEC decoding to recover the lost message.
  • the basic principle is: at the transmitting end, by encoding the kbit information as a packet, adding (n-k) bits of redundancy check information to form a codeword of length n bit.
  • the error bit can be checked and corrected by decoding, thereby resisting the interference caused by the channel, effectively reducing the bit error rate of the system, and improving the reliability of the communication system. Sex.
  • FEC processing reduces the bit error rate of the system with the cost of redundancy overhead.
  • Excessive FEC coding also puts pressure on the real-time and network status of the system.
  • the conventional FEC structure uniformly encodes all information, and there is no discrimination for user information. They are not suitable for mixed content or distribution under mixed networks, so they propose a two-tier structure. Two-layer structure, as shown in Figure 1.
  • the first layer divides the source packet block into more small blocks for FEC protection, and the second layer is a whole block for FEC protection.
  • the first layer is finely divided to provide less latency, and the second layer guarantees recovery performance and less redundancy.
  • the content may be divided into timed and non-timed, so this two-layer structure can be adopted.
  • the timing content is in mode 1, which ensures the delay, and the non-timing content is guaranteed by the mode 1 and mode 2 at the same time.
  • FEC1 For users with different performance channels, users with good channel performance only need FEC1 to guarantee delay and power consumption. For users with poor channel performance, FEC1 and FEC2 are simultaneously guaranteed to ensure accuracy.
  • UEP Unequal Error Protection
  • the Digital Fountain Code does not require feedback and automatic retransmission mechanism during transmission, avoiding the delay of signal round-trip and the feedback explosion problem in broadcast applications.
  • the basic idea of the digital fountain is that the originator divides the original data into Data symbols, encoding these data symbols, outputting an encoded symbol stream of arbitrary length, and the receiving end only needs to receive correctly (n is slightly larger than k) encoding symbols can recover all k data symbols with a high probability.
  • the digital fountain code itself has UEP performance and can protect data of different importance. Compared with the traditional fixed rate rate channel coding method, digital fountain code has the following obvious advantages:
  • the user's decoding performance is independent of the channel's erasure probability and bandwidth.
  • the channel loss rate is high and the condition is not good, it will not affect the decoding of the receiving end. That is, the receiving end can receive a sufficient amount of encoded data to decode normally, and has stronger adaptability.
  • the compilation code has low complexity. Ideally, the fountain code generates a linear codec complexity for each code symbol, which is beneficial to simplify the design and software implementation of the transceiver codec.
  • the codeless rate feature of the fountain code makes users with different packet loss rates or bandwidths have no influence on each other, and high-quality users are not restrained by inferior users.
  • the digital fountain code can support multiple service modes such as interrupted transmission and asynchronous access.
  • the object of the present invention is to provide a FEC mechanism based on media content, thereby solving the current real-time performance of the FEC system, which has poor accuracy, good accuracy, poor real-time performance, and excessive FEC coding. Data congestion problem.
  • the present invention adopts the following technical solutions:
  • a FEC mechanism based on media content the FEC mechanism is implemented by any of the following four methods:
  • Method 1 classify the media content and assign different importance, and then according to the packets belonging to different importance degree frames, combined with the channel condition and the user experience, change the coding scheme, and protect according to the importance degree of the frame;
  • Method 2 classify the media content and assign different importance. In the case of not dividing the original media data stream, combined with the channel condition and the user experience experience, according to the importance degree of the frame included in the media data packet, Transfer the data packet to the corresponding FEC encoder for different degrees of protection;
  • Method 3 classify the media content and assign different importance. Dynamically adjust the importance degree of the frame included in the media data packet and correspondingly according to the current channel condition without offloading the original media data stream.
  • the coding scheme transmits the data packet to the corresponding FEC encoder for different degrees of protection, and finally one source data stream is only encoded into one FEC code stream;
  • Method 4 The media content is graded and given different importance. Combined with the channel condition and user experience, digital fountain codes with unequal error protection performance are used to protect data of different importance.
  • the four methods include: adding unequal error protection flag bits in signaling information transmitted together with a media processing unit (MPU).
  • MPU media processing unit
  • the FEC mechanism adds a special indication field to form a more personalized protection scheme. Further, the special indication field is added before the MFU header.
  • the four methods include: adding unequal error protection flag bits in the signaling information transmitted together with the media processing unit (MPU), and adding a special indication field to form a more personalized protection scheme.
  • MPU media processing unit
  • the method 1 to method 3 classify the media content and assign different importance, and then use signaling and indication field control, and adopt different FEC coding intensity coding.
  • the method 4 classifying the media content and assigning different importance, and then using the signaling and indication field control, adopting the copy extended window fountain code with UEP performance.
  • the sender process is:
  • the server generates MMTP flow and signaling according to the media resource
  • the method 1 identifying the FEC stream and the adopted FEC encoding structure and the FEC code by signaling, in order to support the mechanism, modifying the FEC_FLOW_DESCRIPTOR field in the signaling: the original signaling only supports one FEC stream in the complex Fixing information with multiple media resources now adds a different importance to a media resource In the sexual part, the FEC signaling control is performed separately, so that the FEC mechanism can be further refined.
  • the server can dynamically adjust the FEC strength of different parts of the media resource according to the user network state, and obtain a balance point between the network bandwidth and the user experience.
  • the sender process is:
  • the server generates MMTP flow and signaling according to the media resource
  • the method 2 identifying the FEC stream and the adopted FEC encoding structure and the FEC code by signaling, in order to support the mechanism, modifying the fec_flow_descriptor field in the signaling: the original signaling only supports one FEC stream in the complex With the repair information of multiple media resources, different importance parts for one media resource are added, and the modified signaling performs FEC coding control for different importance parts of one media resource to obtain different FEC streams, so that The FEC mechanism can be further refined.
  • the server dynamically adjusts the FEC strength of different parts of the media resource according to the state of the user network. After receiving the signaling, the receiving end recovers the media resource according to the corresponding indication, and obtains a balance between the network bandwidth and the user experience. point.
  • the fec_flow_descriptor field in the signaling is modified: a new fec_coding_structure is added to the existing three fec_coding_structures in the AL-FEC message, and the function of the fec_coding_structure is used to describe the currently adopted FEC encoding scheme. , including the selected coding algorithm, whether to use the private coding scheme, the maximum protection time window time and value, etc., the field is transmitted to the receiving end in the AL-FEC signaling; the value of the newly added flag of fec_coding_structure is present There are a range of reserved options.
  • the value of the flag bit of the newly added fec_coding_structure is selected to be 0100.
  • the sender process is:
  • the server generates MMTP flow and signaling according to the media resource
  • the MMT packet is transmitted to the FEC encoder, and different FEC encoding matrices are used to perform FEC encoding on the MMT packet to generate a corresponding FEC code for different priorities;
  • the method identifies the FEC stream and the adopted FEC encoding structure and the FEC code by signaling, and in order to support the mechanism, the fec_flow_descriptor field in the signaling is modified: three existing in the AL-FEC message Based on fec_coding_structure, a new fec_coding_structure function is added.
  • the function of fec_coding_structure is used to describe the currently used FEC encoding scheme, including the selected encoding algorithm, whether to use a private encoding scheme, and the maximum protection time window time and value information.
  • the field is transmitted to the receiving end in the AL-FEC signaling; the value of the flag bit of the newly added fec_coding_structure is selected within the range of the existing reserved; the original signaling only supports multiplexing of multiple media resources in one FEC stream.
  • the repair information is now added to the different importance parts of a media resource, and the FEC signaling control is respectively performed to make the FEC mechanism more detailed.
  • the server can dynamically adjust the FEC of different parts of the media resource according to the user network state. Strength, a balance between network bandwidth and user experience.
  • the value of the flag bit of the newly added fec_coding_structure is selected to be 0110.
  • the sender process is:
  • the server generates MMTP flow and signaling according to the media resource
  • the method 4 identifying the FEC stream by using signaling and adopting the FEC encoding structure and the D-EWF code, in order to support the mechanism, the FEC_FLOW_DESCRIPTOR field in the signaling needs to be modified: the original signaling only supports one FEC stream. Multiplexing the repair information of multiple media resources, and now adding a different importance part for one media resource, respectively performing FEC signaling control, so that the FEC mechanism can be more refined; the server dynamically adjusts the media according to the state of the user network. Resource D-EWF code encoding strength, to achieve a balance between the available bandwidth of the network and the user experience.
  • the method 4 by introducing a spreading factor, a virtual extension distribution range, and a window technology of the D-EWF code, the data obtained by the virtual extension is divided into windows, and each window adopts an optimized robust solitary wave distribution. Perform LT code to enhance the protection of importance data.
  • the present invention has the following beneficial effects:
  • the technical solution of the present invention can be used for data congestion caused by over-encoding in the current FEC system, by grading media content, giving different importance, using signaling and/or indicator bit control, and adopting different FEC coding strengths. Further, a duplicate extended window fountain (D-EWF) code with unequal error protection (UEP) performance is used to maximize the quality of the media content while reducing the amount of data caused by FEC.
  • D-EWF duplicate extended window fountain
  • UDP unequal error protection
  • Figure 1 is a two-layer structure of FEC for media resources in MMT
  • Figure 2 is a diagram of each frame dependency relationship in an image group
  • Figure 3 is a schematic diagram of the importance of a general MPU component and each part
  • FIG. 4 is a schematic structural diagram of an MMT AL-FEC transmitting end in the first embodiment of the present invention.
  • FIG. 5 is a structural diagram of an improved MMT AL-FEC transmitting end in the second embodiment of the present invention.
  • FIG. 6 is a structural diagram of an improved MMT AL-FEC transmitting end in Embodiment 3 of the present invention.
  • FIG. 7 is a structural diagram of an improved MMT AL-FEC transmitting end in Embodiment 4 of the present invention.
  • FIG. 8 is a block diagram of D-EWF code encoding in Embodiment 4 of the present invention.
  • FIG. 9 is a flowchart of D-EWF code encoding in Embodiment 4 of the present invention.
  • the FEC two-layer structure for media resources the first layer divides the source packet block into more small blocks for FEC protection, and the second layer is a whole block for FEC protection.
  • the first layer is finely divided to provide less latency, and the second layer guarantees recovery performance and less redundancy, but this flexibility is not sufficient.
  • each frame dependency in an image group shows that the degree of dependence and importance of different frames in an image group are different, I frame is the most important, the previous P frame is more important than the latter P frame, B Frames are of the lowest importance, so hierarchical FEC can be based on importance. This is also the basis for FEC.
  • Embodiment 1 is a diagrammatic representation of Embodiment 1:
  • the MMT transmission protocol is taken as an example, and the method 1 in the content of the invention is implemented:
  • the MFU packages under the MPU package have different importance and lack the protection of unequal errors, so that a personalized transmission scheme cannot be set.
  • frames of different importance such as I frame and B frame
  • the encoding does not involve unequal error protection.
  • the Two-stage structure scheme and LA-FEC can achieve unequal error protection to a certain extent, the flexibility is low and the complexity is high.
  • the existing two methods may have large redundancy, poor adaptability, and can only be considered for a single situation and cannot be considered from the user experience.
  • the importance degree is differentiated, and the coding scheme is changed in combination with the channel condition and the user experience. For example, when the channel condition is poor or the user storage capacity is limited, the sender will protect the I frame data more, so that the user receives the I frame with a greater probability, and the B frame and the P frame will be more received. Discarding, this is no longer the subsequent processing in the receiving end to discard the B, P frame, but solve the problem from the sender, saving bandwidth resources, and can use resources to protect more important frames.
  • the unequal error protection can only be arranged according to the importance of the frame.
  • the unequal error protection flag is added to the signaling transmitted with the MPU.
  • FIG. 4 it is an improved MMT AL-FEC transmitting end architecture in this embodiment, which mainly modifies the streamed output in the MMT protocol, distributes the data streams to different source buffers according to different priorities, and performs FEC respectively.
  • the sender architecture is shown in Figure 4.
  • the server generates MMTP streams and signaling according to the media resources.
  • the FEC_FLOW_DESCRIPTOR field in the signaling is modified in this embodiment to identify the FEC stream and the FEC encoding structure and the FEC code.
  • the signaling is modified as follows:
  • the original signaling only supports the repair information of multiplexing multiple media resources in one FEC stream.
  • this embodiment adjusts the order and definition of the fec_flow_descriptor field, and uses a reserved field to indicate this change.
  • the modified number_of_assets indicates all the media resources for performing FEC
  • the packet_id is the identifier of each corresponding media resource
  • the number_of_fec_flows indicates how many FEC streams are corresponding to each media resource, that is, how many levels.
  • the FEC signaling control is performed separately, and different FEC streams are obtained, so that the FEC mechanism can be further refined, and the server can dynamically adjust the FEC strength of different parts of the media resource according to the state of the user network.
  • the receiving end restores the media resource according to the corresponding indication. Get a balance between network bandwidth and user experience.
  • Video transmission should be based on the user experience.
  • the degree of FEC protection for I and B frames can be designed for user conditions such as visual experience and cache status.
  • the following table shows a case of content classification coding transmission in MMT that simply uses RS code.
  • the coding scheme can be flexibly changed according to the network conditions and user requirements: the FEC strength of various frames can be reasonably configured to ensure that while saving bandwidth, try to save bandwidth. Maximize the acceptance of the frame rate to get the best user experience.
  • Embodiment 2 is a diagrammatic representation of Embodiment 1:
  • This embodiment takes the MMT transmission protocol as an example, and implements method 2 in the content of the invention:
  • the MFU packages under the MPU package have different importance and lack the protection of unequal errors, so that a personalized transmission scheme cannot be set.
  • frames of different importance such as I frame and B frame
  • the encoding does not involve unequal error protection.
  • the Two-stage structure scheme and LA-FEC can achieve unequal error protection to a certain extent, the flexibility is low and the complexity is high.
  • the existing two methods may have large redundancy, poor adaptability, and can only be considered for a single situation and cannot be considered from the user experience.
  • the importance degree is differentiated, and the coding scheme is changed in combination with the channel condition and the user experience. For example, when the channel condition is poor or the user storage capacity is limited, the sender will protect the I frame data more, so that the user receives the I frame with a greater probability, and the B frame and the P frame will be more received. Discarding, this is no longer the subsequent processing in the receiving end to discard the B, P frame, but solve the problem from the sender, saving bandwidth resources, and can use resources to protect more important frames.
  • the unequal error protection can only be arranged according to the importance of the frame.
  • the unequal error protection flag is added to the signaling transmitted with the MPU.
  • FIG. 5 it is an improved MMT AL-FEC transmitting end architecture of the embodiment, which mainly modifies the streamed output in the MMT protocol, and sends the MMT data packets to different FEC encoders according to priorities, and performs FEC respectively.
  • the transmitting end architecture is as shown in FIG.
  • the server generates MMTP streams and signaling according to the media resources.
  • the MMT packets are passed to different FEC encoders to generate corresponding FEC codes.
  • the fec_flow_descriptor field in the signaling is modified in this embodiment by using the FEC stream to identify the FEC stream and the FEC encoding structure and the FEC code.
  • the original signaling only supports multiplexing multiple media in one FEC stream.
  • the repair information of the resource is now added to the different importance parts of one media resource.
  • the modified signaling performs FEC coding control for different importance parts of one media resource, and obtains different FEC streams, so that the FEC mechanism can be more
  • the server dynamically adjusts the FEC strength of different parts of the media resource according to the state of the user network. After receiving the signaling, the receiving end recovers the media resource according to the corresponding indication, and obtains a balance point between the network bandwidth and the user experience.
  • this embodiment adds a new fec_coding_structure to the existing three fec_coding_structures in the AL-FEC message.
  • the function of the fec_coding_structure is used to describe the currently adopted FEC encoding scheme, including the selected encoding algorithm, whether to use a private encoding scheme, maximum protection time window time and value, etc., which is transmitted in the AL-FEC signaling to Receiving end.
  • the value of the flag bit of the newly added fec_coding_structure can be selected within the range of the existing reserved. In this scheme, it is recommended to select 0100.
  • Num_of_priority_for_mmtps The number of priorities in a media resource.
  • Private_fec_flag An indication bit indicating whether a private FEC encoding scheme is used.
  • Private_flag indicates a bit indicating whether a private_field exists to describe the private FEC encoding scheme used.
  • Private_field_length A length field used to describe the length of a field of a private FEC encoding scheme.
  • Private_field Used to describe the details of a private FEC scheme.
  • Priority_id Priority id, used to indicate the priority of the MMT packet.
  • Fec_code_id_for_repair_flow used to describe the FEC encoding scheme used.
  • Repair_flow_id 8-bit integer used to indicate the generated FEC repair flow, which corresponds to the packet id in the header of the FEC repair package.
  • Maximum_k_for_repair_flow A 24-bit integer describing the maximum number of source symbols in a source sysmbol block.
  • Maximum_p_for_repair_flow A 24-bit integer describing the maximum number of repair symbols in a repair sysmbol block.
  • Protection_window_time The protection window time, indicating the maximum time difference between sending the first source or repair packet in the FEC encoding and sending the last source or repair packet, in milliseconds.
  • Protection_window_size The protection window value indicating the maximum count value between the payload of the first FEC packet sent in the FEC encoded stream and the payload of the last FEC packet transmitted.
  • the media content is classified and given different importance.
  • the data packets are transmitted to the corresponding FEC encoder for different degrees of protection.
  • Video transmission should be based on the user experience.
  • the degree of FEC protection for I and B frames can be designed for user conditions such as visual experience and cache status.
  • Embodiment 3 is a diagrammatic representation of Embodiment 3
  • This embodiment takes the MMT transmission protocol as an example, and implements method 3 in the content of the invention:
  • the MFU packages under the MPU package have different importance and lack the protection of unequal errors, so that a personalized transmission scheme cannot be set.
  • frames of different importance such as I frame and B frame
  • the encoding does not involve unequal error protection.
  • the Two-stage structure scheme and LA-FEC can achieve unequal error protection to a certain extent, the flexibility is low and the complexity is high.
  • the existing two methods may have large redundancy, poor adaptability, and can only be considered for a single situation and cannot be considered from the user experience.
  • the importance degree is differentiated, and the channel condition and the user experience experience are combined, and the coding scheme is adaptively changed.
  • the sender will The I frame data is further protected, so that the user receives the I frame with a greater probability, and the B frame and the P frame are discarded to a greater extent at the time of receiving, so that the B and P frames are discarded in the subsequent processing of the receiving end. Instead, the problem is solved from the sender, saving bandwidth resources and using resources to protect more important frames.
  • the FEC encoding scheme can be adaptively changed according to the current network condition. For example, if the network condition deteriorates, the FEC seeding can be changed to change the encoding matrix of the FEC encoding to increase the protection strength of the transmitted data. .
  • the unequal error protection can only be arranged according to the importance of the frame.
  • the unequal error protection flag is added to the signaling transmitted with the MPU.
  • the MMT AL-FEC transmitting end architecture improved in this embodiment mainly modifies the streamed output in the MMT protocol, and sends the MMT data packet to the FEC encoder according to the priority to perform FEC encoding, according to the MMT. Different priorities of the packets are selected, and different coding matrices are selected, and the FEC codes generated by the FEC encoders are integrated into one FEC code stream.
  • the sender architecture is shown in Figure 6.
  • the server generates MMTP streams and signaling according to the media resources.
  • the MMT packet is transmitted to the FEC encoder, and the FEC code is used to generate the corresponding FEC code for the MMT packet by using different FEC coding matrices for different priorities.
  • the FEC_flow_descriptor field in the signaling is modified in this embodiment to identify the FEC stream and the adopted FEC encoding structure and the FEC code.
  • the original signaling only supports the repair information of multiplexing multiple media resources in one FEC stream.
  • a new fec_coding_structure is added.
  • the function of the fec_coding_structure is used to describe the currently used FEC encoding scheme, including the selected encoding algorithm. Private coding scheme, maximum protection time window time and value, etc. This field is transmitted to the receiving end in the AL-FEC signaling.
  • the value of the newly added flag of fec_coding_structure can be selected within the scope of the existing reserved. In this scheme, it is recommended to select 0110.
  • Seed A random seed, which is used as an initial condition for generating a pseudo-random number, for generating a pseudo-random sequence to construct a generation matrix and a check matrix of FEC coding.
  • Num_of_priority_for_mmtps The number of priorities in a media resource.
  • Priority_mapping Priority mapping, which is used to indicate the mapping between different data packets and resource priorities in a media resource.
  • Private_fec_flag An indication bit indicating whether a private FEC encoding scheme is used.
  • Private_flag indicates a bit indicating whether a private_field exists to describe the private FEC encoding scheme used.
  • Private_field_length A length field used to describe the length of a field of a private FEC encoding scheme.
  • Private_field Used to describe the details of a private FEC scheme.
  • Priority_id Priority id, used to indicate the priority of the MMT packet.
  • Fec_code_id_for_repair_flow used to describe the FEC encoding scheme used.
  • Repair_flow_id 8-bit integer used to indicate the generated FEC repair flow, which corresponds to the packet id in the header of the FEC repair package.
  • Maximum_k_for_repair_flow A 24-bit integer describing the maximum number of source symbols in a source sysmbol block.
  • Maximum_p_for_repair_flow A 24-bit integer describing the maximum number of repair symbols in a repair sysmbol block.
  • Protection_window_time The protection window time, indicating the maximum time difference between sending the first source or repair packet in the FEC encoding and sending the last source or repair packet, in milliseconds.
  • Protection_window_size The protection window value indicating the maximum count value between the payload of the first FEC packet sent in the FEC encoded stream and the payload of the last FEC packet transmitted.
  • the modified signaling performs FEC encoding control on different priorities for different importance parts of one media resource, and finally generates only one FEC code stream for one media data stream, so that the FEC mechanism can be more Refining and reducing the increase of network traffic caused by excessive FEC coding
  • the server can dynamically adjust the FEC strength of different parts of the media resource according to the state of the user network.
  • the receiver After receiving the signaling, the receiver can parse the seed from it. And the priority of each resource packet, and the check matrix is generated according to the seed to perform FEC decoding to recover the media resource. Get a balance between network bandwidth and user experience.
  • Video transmission should be based on the user experience.
  • the degree of FEC protection for I and B frames can be designed for user conditions such as visual experience and cache status.
  • the FEC 3 can adaptively change the FEC coding mode according to the current network condition, and perform adaptive adjustment on the basis of different strength FEC protection for different priority data, if the network condition deteriorates, the packet loss rate increases.
  • the FEC seed changes the FEC generation matrix and increases the FEC strength to counter the effects of network degradation.
  • the existing AL-FEC scheme in 4MMT only supports generating different FEC code streams for different priority MMT packets.
  • the solution proposed in this solution supports only generating one FEC code stream for one media resource data stream, which can greatly reduce The traffic brought by FEC increases, reducing network pressure.
  • Embodiment 4 is a diagrammatic representation of Embodiment 4:
  • This embodiment takes the MMT transmission protocol as an example, and implements method 4 in the content of the invention:
  • the FEC two-layer structure for media resources the first layer divides the source packet block into more small blocks for FEC protection, and the second layer is a whole block for FEC protection.
  • the first layer is finely divided to provide less latency, and the second layer guarantees recovery performance and less redundancy.
  • P1 and P2 are repair symbol blocks generated by the FEC encoders 1, 2, respectively.
  • the source K data is divided into two importance levels according to importance, namely s 1 and s 2 .
  • s 1 be the most important part of MIB (Most ImPortant Bits)
  • s 2 be the least important part of LIB (Least ImPortant Bits).
  • the MFU packages under the MPU package have different importance and lack the protection of unequal errors, so that a personalized transmission scheme cannot be set.
  • different levels of importance are distinguishable (with indicators) as shown in Figure 3.
  • the protection of unequal errors is not involved in the coding.
  • the two-layer coding scheme and LA-FEC shown in Figure 1 can achieve unequal error protection to a certain extent, the flexibility is low and the complexity is high.
  • the existing two methods have large redundancy, poor adaptability, and can only be considered for a single situation and cannot be considered from the user experience.
  • FIG. 7 it is an improved MMT AL-FEC sender architecture, which mainly modifies the streamed output in the MMT protocol, and transmits the data stream to the source buffer, and performs D- according to the importance of different contents indicated by the identifier bits. EWF coding.
  • the sender architecture is shown in Figure 7.
  • the server generates MMTP flow and signaling according to the media resource
  • the FEC_FLOW_DESCRIPTOR field in the signaling is modified as follows in the embodiment to support the FEC stream and the FEC encoding structure and the D-EWF code.
  • This embodiment adjusts the order and definition of the fec_flow_descriptor field, and utilizes a reserved field to indicate the introduction of the UEP mechanism.
  • the modified number_of_assets indicates all the media resources for performing FEC
  • the packet_id is the identifier of each corresponding media resource
  • the number_of_fec_flows indicates how many FEC streams are corresponding to each media resource, that is, how many levels.
  • D-EWF copy extended window fountain code
  • the server can dynamically adjust the coding strength of the D-EWF according to the state of the user network. After receiving the signaling, the receiving end recovers the media resource according to the corresponding indication. Get a balance between network bandwidth and user experience.
  • the copying and expanding window fountain code is used to strengthen the protection of important media content.
  • the specific implementation process is as follows:
  • the transmission source data has two important levels.
  • the source K data is divided into two importance levels according to importance, that is, s 1 and s 2 .
  • s 1 be the most important part MIB (Most ImPortant Bits)
  • s 2 be the least important part LIB (Least ImPortant Bits)
  • ⁇ 1 and ⁇ 2 are s 1 and s respectively.
  • 2 is the total data weight
  • the obtained two layers of data are expanded by the expansion factors ⁇ 1 and ⁇ 2 to obtain virtual extension layer data: s 1 ', s 2 ', where k 1 ', k 2 ' represents s 1 ', s 2 'data symbols
  • K '.
  • the index replacement process in the LT coding is performed: a number ⁇ is randomly generated, and when 0 ⁇ ⁇ ⁇ 1 selects the first window W 1 , the degree d ⁇ (1) is generated by the degree distribution d 1 .
  • the d 1 data is randomly selected in the W 1 virtual data.
  • j represents the index in W 1 virtual data k 1 ⁇ 1 , j ⁇ 0,...,k 1 ⁇ 1 -1 ⁇
  • ⁇ 1 ⁇ ⁇ ⁇ 1 that is, when the second window W 2 is selected
  • the degree d 2 is generated by the degree distribution ⁇ (2)
  • d 2 pieces of data are randomly selected from the W 2 dummy data.
  • j represents the index in the W 2 virtual data (k 1 ⁇ 1 +k 2 ⁇ 2 ), j ⁇ ⁇ 0,...,k 1 ⁇ 1 +k 2 ⁇ 2 -1 ⁇ , m represents the original k 2
  • FIG. 9 shows the D-EWF code encoding process.
  • the above solution solves the problem from the source side, so that the less important media content is received during the transmission process.
  • the degree of protection is lower, more bandwidth resources are allocated to important content, and more important content is protected.
  • Video transmission should be based on the user experience.
  • D-EWF encoding strength of important and unimportant media content more detailed error protection can be applied to the user's situation, such as visual experience, cache status and so on.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Quality & Reliability (AREA)
  • Multimedia (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Detection And Prevention Of Errors In Transmission (AREA)
  • Error Detection And Correction (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Communication Control (AREA)

Abstract

本发明提供了一种基于媒体内容的FEC机制，该机制将媒体内容进行分级，并赋予不同的重要性，再根据属于不同重要程度帧的包，结合信道状况和用户体验感受，改变编码方案，按照所属帧的重要程度进行保护。采用了本发明的技术方案，可以针对目前FEC系统中过度编码造成的数据拥塞，通过对媒体内容分级，赋予不同的重要性，利用信令和指示位控制，采用不同的FEC编码强度；进一步采用具有不等差错保护性能的复制扩展窗喷泉码，实现最大限度保证媒体内容质量的同时，减少FEC造成的极大的数据量。

Description

一种基于媒体内容的FEC机制 技术领域
本发明涉及多媒体传输技术领域，更具体地，涉及一种基于媒体内容的FEC(前向纠错)机制。
背景技术
在异构网络媒体服务系统中，内容通过因特网协议或广播协议分发到终端，因特网中使用IP/TCP或UDP报文来传输媒体数据，广播通过MPEG2-TS来传输内容。UDP报文在经过多个网络设备后可能会出现丟失，广播TS流可能因为传输环境的影响，产生误码，从而造成终端侧的画面破损或者声音停顿。
FEC(Forward Error Correction，前向纠错)技术是一种广泛应用于通信系统中的编码技术。通过服务器侧对媒体数据进行纠错编码，加入冗余信息一并发送，终端侧进行反向FEC解码，对丟失的报文进行恢复。以典型的分组码为例，其基本原理是：在发送端，通过将kbit信息作为一个分组进行编码，加入(n-k)bit的冗余校验信息，组成长度为n bit的码字。码字经过信道到达接收端之后，如果错误在可纠范围之内，通过译码即可检查并纠正错误bit，从而抵抗信道带来的干扰，有效降低系统的误码率，提高通信系统的可靠性。
但是FEC处理，是以冗余开销代价来降低系统的误码率,过度FEC编码对系统实时性和网络状态也会造成压力。
在ISO-23008-1/10/13标准中，传统的FEC结构对于所有信息统一进行编码，对于用户信息没有区分度。对于混合内容或者混合网络下分发不适合，因此他们提出了两层结构。两层结构，如附图1所示。
第一层将source packet block分为较多的小块分别做FEC保护，第二层是一个整块做FEC保护。第一层划分较细致可以提供较小的时延，第二层保证了恢复性能和较小的冗余。
对于混合内容传输，其内容可能分为timed和non-timed，因此可以采用这种两层结构。时序内容用方式1，保证了时延，非时序内容同时借助方式1和方式2，保证了准确性。
对于用户处于不同性能信道，信道性能好的用户只需要FEC1保证时延和功耗，对于信道性能差的用户同时做FEC1和FEC2保证了准确性。
这在一定程度上的解决了问题，对于信道性能较差的用户(GroupB)，两层结构确实提高了 恢复性能，但是会引入极大的时延。对于信道性能较好的用户(GroupA)，不一定是做FEC1，越小的分包带来越小的时延。没有考虑信息的不等重要程度。对于信息或者用户要根据两层结构进行分类，以及对于将大block拆分为小blcok的具体策略是一个复杂的问题。
同时，不等差错保护(Unequal Error Protection，UEP)是联合信源信道编码的一种。其核心思想是，依据码流的各部分数据的重要性不同，对各部分数据采用不同的信道保护机制，即对重要码流进行重点保护。尽管UEP降低了非重要码流的抗噪声性能，但有利于系统抗误码总体性能的提升。
作为一种FEC(前向纠错)编码技术，数字喷泉码(Digtial Fountain Code)在传输过程中，不需要反馈及自动重发机制，避免了信号往返的延时以及广播应用中的反馈爆炸问题。数字喷泉的基本思想是：发端将原始数据分割成
Figure PCTCN2016070525-appb-000001
个数据符号，对这些数据符号进行编码，输出一个任意长度的编码符号码流，接收端只需正确地接收
Figure PCTCN2016070525-appb-000002
(n稍大于k)个编码符号就可以很大的概率恢复出所有的k个数据符号。
数字喷泉码本身就具有UEP性能，可以实现对不同重要性的数据的保护。相比与传统的固定码率的信道编码方法，数字喷泉码具有以下明显的优势：
1、理想的可扩展性。由于单向广播没有反馈，发送方不受用户数量增长的任何影响。使得发送方能够为任意数量的用户提供服务。
2、适应时变信道，高效利用信道容量。用户的译码性能与信道的删除概率和带宽无关。当信道丢包率较高、状况不好时，不会对接收端的译码造成影响，即接收端接收足够数量的编码数据就可以正常译码，具有更强的适应性。
3、编译码复杂度低。在理想情况下，喷泉码生成每个编码符号具有线性编译码复杂度，有利于简化收发端编译码器的设计和软件化实现。
4、对异质用户的适用性能良好。喷泉码的无码率特性使得具有不同丢包率或带宽的用户之间互不影响，优质用户不受劣质用户牵制。除此之外，数字喷泉码可以支持中断续传、异步接入等多种服务模式。
发明内容
针对现有技术中的缺陷，本发明的目的是提供一种基于媒体内容的FEC机制,从而解决目前FEC系统中实时性好，而准确性差，准确性好而实时性差，以及过度FEC编码造成的数据拥塞问题。
为实现上述目的，本发明采用以下技术方案：
一种基于媒体内容的FEC机制，所述FEC机制采用以下四种方法中任一种实现：
方法一：将媒体内容进行分级，并赋予不同的重要性，再根据属于不同重要程度帧的包，结合信道状况和用户体验感受，改变编码方案，按照所属帧的重要程度进行保护；
方法二：将媒体内容进行分级，并赋予不同的重要性，在不对原有媒体数据流进行分流的情况下，结合信道状况和用户体验感受，按照媒体数据包中所包含的帧的重要程度，把数据包传送到相应的FEC编码器，进行不同程度的保护；
方法三：将媒体内容进行分级，并赋予不同的重要性，在不对原有媒体数据流进行分流的情况下，根据当前信道状况，动态调整媒体数据包中所包含的帧的重要程度和相应的编码方案，把数据包传送到相应的FEC编码器，进行不同程度的保护，最终一个源数据流只会被编码为一个FEC码流；
方法四：将媒体内容进行分级，并赋予不同的重要性，再结合信道状况和用户体验，采用本身就具有不等差错保护性能的数字喷泉码对不同重要性的数据进行保护。
进一步的，所述四种方法：在与媒体处理单元(MPU)一同传输的信令信息中加入不等差错保护标志位。
进一步的，所述四种方法：FEC机制加入专门的指示字段，以形成更多个性化的保护方案。更进一步的，所述专门的指示字段加在MFU包头前。
进一步的，所述四种方法：在与媒体处理单元(MPU)一同传输的信令信息中加入不等差错保护标志位，同时加入专门的指示字段，以形成更多个性化的保护方案。
进一步的，所述方法一～方法三：将媒体内容进行分级，并赋予不同的重要性，然后利用信令和指示字段控制，采用不同的FEC编码强度编码。
进一步的，所述方法四：将媒体内容进行分级，并赋予不同的重要性，然后利用信令和指示字段控制，采用具有UEP性能的复制扩展窗喷泉码。
进一步的，所述方法一：针对MMT AL-FEC发送端架构，发送端流程为：
e)服务器端根据媒体资源生成MMTP流和信令；
f)分析媒体资源中每帧的重要性，把这些MMT负载分到不同的原数据流，传递到相应的FEC机制做保护；
g)FEC编码完后，返回相应的修复字符以及FEC数据负载标识和原数据负载标识；
h)所有的修复字符打包成FEC修复包，发送到传输层。
进一步的，所述方法一：通过信令识别FEC流以及采用的FEC编码结构和FEC code，为了支持这种机制，修改了信令中的FEC_FLOW_DESCRIPTOR字段：原来的信令仅仅支持一个FEC流中复用多个媒体资源的修复信息，现在增加了一种针对一个媒体资源的不同重要 性部分，分别进行FEC的信令控制，使FEC机制能够更加细化，服务端可以根据用户网络状态动态地调整媒体资源不同部分的FEC强度，在网络带宽和用户体验间取得一个平衡点。
进一步的，所述方法二：针对MMT AL-FEC发送端架构，发送端流程为：
e)服务器端根据媒体资源生成MMTP流和信令；
f)分析媒体资源中每帧的重要性，把MMT包传送到不同的FEC编码器，采用相应的FEC机制做保护，生成相应的FEC码；
g)FEC编码完后，返回相应的修复字符以及FEC数据负载标识和原数据负载标识；
h)所有的修复字符打包成FEC修复包，发送到传输层。
进一步的，所述方法二：通过信令识别FEC流以及采用的FEC编码结构和FEC code，为了支持这种机制，修改了信令中的fec_flow_descriptor字段：原来的信令仅仅支持一个FEC流中复用多个媒体资源的修复信息，现增加针对一个媒体资源的不同重要性部分，修改后的信令针对一个媒体资源的不同重要性部分，分别进行FEC的编码控制，得到不同的FEC流，使FEC机制能够更加细化，服务端根据用户网络状态动态地调整媒体资源不同部分的FEC强度，收端收到信令后，根据相应的指示恢复媒体资源，在网络带宽和用户体验间取得一个平衡点。
进一步的，所述修改了信令中的fec_flow_descriptor字段：在AL-FEC message中现有的三种fec_coding_structure的基础上增加了一种新的fec_coding_structure，fec_coding_structure的功能是用来描述当前采用的FEC编码方案，包括选择的编码算法，是否采用私有编码方案，最大的保护时间窗时间和值等信息，该字段位于AL-FEC信令中被传送到接收端；新增加的fec_coding_structure的标志位的值在现有的reserved的范围内选择。
进一步的，新增加的fec_coding_structure的标志位的值选取为0100。
进一步的，所述方法三：针对MMT AL-FEC发送端架构，发送端流程为：
f)服务器端根据媒体资源生成MMTP流和信令；
g)根据MMT流中每帧数据的重要性的不同，把MMT包传递到FEC编码器，对不同的优先级，采用不同的FEC编码矩阵对MMT包进行FEC编码生成相应的FEC码；
h)将上步中对同一数据流的不同优先级的MMT包进行FEC编码后形成的FEC码进行整合生成一个FEC码流；
i)FEC编码完后，返回相应的修复字符以及FEC数据负载标识和原数据负载标识；
j)所有的修复字符打包成FEC修复包，发送到传输层。
进一步的，所述方法通过信令识别FEC流以及采用的FEC编码结构和FEC code，为了支持这种机制，修改了信令中的fec_flow_descriptor字段：在AL-FEC message中现有的三种 fec_coding_structure的基础上增加了一种新的fec_coding_structure，fec_coding_structure的功能是用来描述当前采用的FEC编码方案，包括选择的编码算法，是否采用私有编码方案，最大的保护时间窗时间和值等信息，该字段位于AL-FEC信令中被传送到接收端；新增加的fec_coding_structure的标志位的值在现有的reserved的范围内选择；原来的信令仅仅支持一个FEC流中复用多个媒体资源的修复信息，现在增加了一种针对一个媒体资源的不同重要性部分，分别进行FEC的信令控制，使FEC机制能够更加细化，服务端可以根据用户网络状态动态地调整媒体资源不同部分的FEC强度，在网络带宽和用户体验间取得一个平衡点。
进一步的，新增加的fec_coding_structure的标志位的值选取为0110。
进一步的，所述方法四：针对MMT AL-FEC发送端架构，发送端流程为：
e)服务器端根据媒体资源生成MMTP流和信令；
f)把MMT负载传递到FEC机制，根据标识位中对内容重要性的区分，采用D-EWF码进行不等差错保护；
g)D-EWF码编码完后，返回相应的修复字符以及FEC数据负载标识和源数据负载标识；
h)所有的修复字符打包成FEC修复包，发送到传输层。
进一步的，所述方法四：通过信令识别FEC流以及采用FEC编码结构和D-EWF码，为支持这种机制，需修改信令中的FEC_FLOW_DESCRIPTOR字段：原来的信令仅仅支持一个FEC流中复用多个媒体资源的修复信息，现增加一种针对一个媒体资源的不同重要性部分，分别进行FEC的信令控制，使FEC机制能够更加细化；服务端根据用户网络状态动态地调整媒体资源D-EWF码编码强度，在网络可用带宽和用户体验间取得一个平衡点。
进一步的，所述方法四：通过引入扩展因子，虚拟扩展度分布范围，结合D-EWF码的窗技术，将虚拟扩展得到的数据进行分窗，同时各个窗分别采用优化的鲁棒孤波分布进行LT码，实现对重要性数据的加强保护。
与现有技术相比，本发明具有如下的有益效果：
采用了本发明的技术方案，可以针对目前FEC系统中过度编码造成的数据拥塞，通过对媒体内容分级，赋予不同的重要性，利用信令和/或指示位控制，采用不同的FEC编码强度，进一步的，采用具有不等差错保护(UEP)性能的复制扩展窗喷泉(D-EWF)码，实现最大限度保证媒体内容质量的同时，减少FEC造成的极大的数据量。
附图说明
通过阅读参照以下附图对非限制性实施例所作的详细描述，本发明的其它特征、目的和优点将会变得更明显：
图1是MMT中针对媒体资源的FEC两层结构；
图2是一个图像组中各帧依赖关系图；
图3是一个通用的MPU组成部分及各部分的重要性示意图；
图4是本发明实施例一中改进的MMT AL-FEC发送端架构图；
图5是本发明实施例二中改进的MMT AL-FEC发送端架构图；
图6是本发明实施例三中改进的MMT AL-FEC发送端架构图；
图7是本发明实施例四中改进的MMT AL-FEC发送端架构图；
图8是本发明实施例四中D-EWF码编码框图；
图9是本发明实施例四中D-EWF码编码流程图。
具体实施方式
下面结合具体实施例对本发明进行详细说明。以下实施例将有助于本领域的技术人员进一步理解本发明，但不以任何形式限制本发明。应当指出的是，对本领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干变形和改进。这些都属于本发明的保护范围。
如图1所示：MMT中针对媒体资源的FEC两层结构，第一层将source packet block分为较多的小块分别做FEC保护，第二层是一个整块做FEC保护。第一层划分较细致可以提供较小的时延，第二层保证了恢复性能和较小的冗余，但是这种灵活性不够。
如图2所示：一个图像组中各帧依赖关系，此图说明一个图像组中不同帧的依赖程度和重要性不同，I帧是最重要的，前面的P帧比后面P帧重要，B帧重要性最低，因此可以根据重要性不同进行分层FEC。这也是进行FEC的基础。
实施例一：
本实施例以MMT传输协议为例，采用发明内容中的方法一进行实施：
MMT方案中，MPU包下的MFU包拥有不同的重要性，缺少不等差错的保护，这样就不能设定个性化的传输方案。在MMT的包中，不同重要程度的帧(如I帧和B帧)是可以区分开的(有指示标志)如附图2，附图3。然而编码中并没有涉及到不等差错的保护，虽然Two-stage结构方案和LA-FEC在一定程度上可以实现不等差错保护，但是灵活性低，复杂度高。尤其对于变化的信道状况和具有不同特征(如缓存不同)的接收用户，现有两种方式会存在冗余大，适应性差，只能针对单一状况，无法从用户体验考虑等问题。
解决方式：
1根据属于不同重要程度帧的包，其重要程度有所区分，同时结合信道状况和用户体验感受，将编码方案改变。例如当信道状况很差或是用户存储能力有限时候，发送端将更针对I帧数据进行保护，使得用户以更大的概率接收到I帧，B帧和P帧将更大程度的在接收时候舍弃，这样不再是接收端后续的处理中舍弃B，P帧，而是从发送端就解决了这一问题，节省了带宽资源，可以利用资源保护更多的重要帧。
2如果不加入专门的指示字段，那么只能根据先有的状况来编码，不等差错保护只能按照所属帧的重要程度来安排。在与MPU同传输的信令中加入不等差错保护标志位。
3但是如果想更个性化的方案，随着信道质量改变保护方案，根据用户特征来定制，那么则需要加入专门的指示字段。由于不同的MFU有不同的重要性，则在MFU包头前加入指示，可以做更多个性化的保护方案。
如图4所示，是本实施例中改进的MMT AL-FEC发送端架构，主要修改了MMT协议中流化后的输出，把数据流按照不同优先级分发到不同的源缓冲区，分别进行FEC。
发送端架构如附图4所示。
发送端流程：
i)服务器端根据媒体资源生成MMTP流和信令。
j)分析媒体资源中每帧的重要性，把这些MMT负载分到不同的原数据流，传递到相应的FEC机制做保护。
k)FEC编码完后，返回相应的修复字符以及FEC数据负载标识和原数据负载标识。
l)所有的修复字符打包成FEC修复包，发送到传输层。
通过信令识别FEC流以及采用的FEC编码结构和FEC code，为了支持这种机制，本实施例中修改了信令中的FEC_FLOW_DESCRIPTOR字段。
信令修改如下：
Figure PCTCN2016070525-appb-000003
Figure PCTCN2016070525-appb-000004
原来的信令仅仅支持一个FEC流中复用多个媒体资源的修复信息，为了达到这个目的，本实施例调整修改了fec_flow_descriptor字段顺序和定义，并且利用一个reserved字段来指示这种变化。修改后number_of_assets指示所有进行FEC的媒体资源，packet_id是每个相应媒体资源的标识，number_of_fec_flows指示每一个媒体资源对应多少个FEC流，即分多少级。针对一个媒体资源的不同重要性部分，分别进行FEC的信令控制，得到不同的FEC流，使FEC机制能够更加细化，服务端可以根据用户网络状态动态地调整媒体资源不同部分的FEC强度，收端收到信令后，根据相应的指示恢复媒体资源。在网络带宽和用户体验间取得一个平衡点。
上述解决方式的优势：
1资源的节省，如果在接收端，人为的根据情况舍弃已经接收的帧(例如B帧)则造成了传输资源的浪费，上述方案从源端解决问题，让不想要的包在传输过程中更大概率丢掉，而更大程度的保护了重要的包。
2个性化的传输方案。视频传输应该是基于用户体验的，做更细致的不等差错保护就可以针对用户状况，如视觉体验，缓存状况等设计I帧和B帧的FEC保护的程度。
下表给出一个简单采用RS码的MMT中内容分级编码传送案例，可以根据网络状况及用户需求灵活改变编码方案：可以通过合理地配置各种帧的FEC强度，保证在节省带宽的同时，尽量最大化收端的接受帧率来获取一个最好的用户体验。
Figure PCTCN2016070525-appb-000005
实施例二：
本实施例以MMT传输协议为例，采用发明内容中的方法二进行实施：
MMT方案中，MPU包下的MFU包拥有不同的重要性，缺少不等差错的保护，这样就不能设定个性化的传输方案。在MMT的包中，不同重要程度的帧(如I帧和B帧)是可以区 分开的(有指示标志)如附图2，附图3。然而编码中并没有涉及到不等差错的保护，虽然Two-stage结构方案和LA-FEC在一定程度上可以实现不等差错保护，但是灵活性低，复杂度高。尤其对于变化的信道状况和具有不同特征(如缓存不同)的接收用户，现有两种方式会存在冗余大，适应性差，只能针对单一状况，无法从用户体验考虑等问题。
解决方式：
1根据属于不同重要程度帧的包，其重要程度有所区分，同时结合信道状况和用户体验感受，将编码方案改变。例如当信道状况很差或是用户存储能力有限时候，发送端将更针对I帧数据进行保护，使得用户以更大的概率接收到I帧，B帧和P帧将更大程度的在接收时候舍弃，这样不再是接收端后续的处理中舍弃B，P帧，而是从发送端就解决了这一问题，节省了带宽资源，可以利用资源保护更多的重要帧。
2如果不加入专门的指示字段，那么只能根据现有的状况来编码，不等差错保护只能按照所属帧的重要程度来安排。在与MPU同传输的信令中加入不等差错保护标志位。
3但是如果想更个性化的方案，随着信道质量改变保护方案，根据用户特征来定制，那么则需要加入专门的指示字段。由于不同的MFU有不同的重要性，则在MFU包头前加入指示，可以做更多个性化的保护方案。
如图5所示，是本实施例改进的MMT AL-FEC发送端架构，主要修改了MMT协议中流化后的输出，把MMT数据包按照优先级发送到不同的FEC编码器，分别进行FEC。
发送端架构如附图5所示。
发送端流程：
a)服务器端根据媒体资源生成MMTP流和信令。
b)根据MMT流中每帧数据的重要性的不同，把MMT包传递到不同的FEC编码器，生成相应的FEC码。
c)FEC编码完后，返回相应的修复字符以及FEC数据负载标识和原数据负载标识。
d)所有的修复字符打包成FEC修复包，发送到传输层。
通过信令识别FEC流以及采用的FEC编码结构和FEC code，为了支持这种机制，本实施例中修改了信令中的fec_flow_descriptor字段：原来的信令仅仅支持一个FEC流中复用多个媒体资源的修复信息，现增加针对一个媒体资源的不同重要性部分，修改后的信令针对一个媒体资源的不同重要性部分，分别进行FEC的编码控制，得到不同的FEC流，使FEC机制能够更加细化，服务端根据用户网络状态动态地调整媒体资源不同部分的FEC强度，收端收到信令后，根据相应的指示恢复媒体资源，在网络带宽和用户体验间取得一个平衡点。
表1：新的fec_coding_structure的说明
Figure PCTCN2016070525-appb-000006
原来的信令仅仅支持一个FEC流中复用多个媒体资源的修复信息，为了达到这个目的，本实施例在AL-FEC message中现有的三种fec_coding_structure的基础上增加了一种新的fec_coding_structure，fec_coding_structure的功能是用来描述当前采用的FEC编码方案，包括选择的编码算法，是否采用私有编码方案，最大的保护时间窗时间和值等信息，该字段位于AL-FEC信令中被传送到接收端。新增加的fec_coding_structure的标志位的值可以在现有的reserved的范围内选择，本方案中建议选取为0100。
表中字段语义如下：
num_of_priority_for_mmtps：一个媒体资源中优先级的数目。
private_fec_flag：指示位，指明是否使用私有的FEC编码方案。
private_flag：指示位，指明是否存在一个private_field用来描述所使用的私有FEC编码方案。
private_field_length：长度字段，用于描述私有FEC编码方案的字段的长度。
private_field：用于描述私有FEC方案的详细信息。
priority_id：优先级id，用于指示MMT包的优先级。
fec_code_id_for_repair_flow：用于描述所使用的FEC编码方案。
repair_flow_id：8位整数，用于指示生成的FEC repair flow，与FEC repair包的包头中的packet id有对应关系。
maximum_k_for_repair_flow：24位整数，描述在一个source sysmbol块中source symbol的最大数目。
maximum_p_for_repair_flow：24位整数，描述在一个repair sysmbol块中repair symbol的最大数目。
protection_window_time：保护窗时间，指示在FEC编码中发送第一个source或者repair包与发送最后一个source或者repair包之间的最大时间差，单位为毫秒。
protection_window_size：保护窗值，指示在FEC编码流中发送第一个FEC包的负载与发送最后一个FEC包负载之间的最大计数值。
本实施例将媒体内容进行分级，并赋予不同的重要性，在不对原有媒体数据流进行分流的情况下，结合信道状况和用户体验感受，按照媒体数据包中所包含的帧的重要程度，把数据包传送到相应的FEC编码器，进行不同程度的保护。
上述解决方式的优势：
1资源的节省，如果在接收端，人为的根据情况舍弃已经接收的帧(例如B帧)则造成了传输资源的浪费，上述方案从源端解决问题，让不想要的包在传输过程中更大概率丢掉，而更大程度的保护了重要的包。
2个性化的传输方案。视频传输应该是基于用户体验的，做更细致的不等差错保护就可以针对用户状况，如视觉体验，缓存状况等设计I帧和B帧的FEC保护的程度。
实施例三：
本实施例以MMT传输协议为例，采用发明内容中的方法三进行实施：
MMT方案中，MPU包下的MFU包拥有不同的重要性，缺少不等差错的保护，这样就不能设定个性化的传输方案。在MMT的包中，不同重要程度的帧(如I帧和B帧)是可以区分开的(有指示标志)如附图2，附图3。然而编码中并没有涉及到不等差错的保护，虽然Two-stage结构方案和LA-FEC在一定程度上可以实现不等差错保护，但是灵活性低，复杂度高。尤其对于变化的信道状况和具有不同特征(如缓存不同)的接收用户，现有两种方式会存在冗余大，适应性差，只能针对单一状况，无法从用户体验考虑等问题。
解决方式：
1根据属于不同重要程度帧的包，其重要程度有所区分，同时结合信道状况和用户体验感受，自适应的改变编码方案。例如当信道状况很差或是用户存储能力有限时候，发送端将 更针对I帧数据进行保护，使得用户以更大的概率接收到I帧，B帧和P帧将更大程度的在接收时候舍弃，这样不再是接收端后续的处理中舍弃B，P帧，而是从发送端就解决了这一问题，节省了带宽资源，可以利用资源保护更多的重要帧。而在时变信道中，可以根据当前网络状况的变化自适应的改变FEC编码方案，例如如果出现网络状况恶化的情况，可以通过改变FEC seed来改变FEC编码的编码矩阵，增加传输数据的保护强度。
2如果不加入专门的指示字段，那么只能根据现有的状况来编码，不等差错保护只能按照所属帧的重要程度来安排。在与MPU同传输的信令中加入不等差错保护标志位。
3但是如果想更个性化的方案，随着信道质量改变保护方案，根据用户特征来定制，那么则需要加入专门的指示字段。由于不同的MFU有不同的重要性，则在MFU包头前加入指示，可以做更多个性化的保护方案。
如图6所示，是本实施例改进的MMT AL-FEC发送端架构，主要修改了MMT协议中流化后的输出，把MMT数据包按照优先级发送到FEC编码器分别进行FEC编码，根据MMT包的优先级不同，选择不同的编码矩阵，并将各个优先级经过FEC编码器处理后生成的FEC码整合成一个FEC码流。
发送端架构如附图6所示。
发送端流程：
a)服务器端根据媒体资源生成MMTP流和信令。
b)根据MMT流中每帧数据的重要性的不同，把MMT包传递到FEC编码器，对不同的优先级，采用不同的FEC编码矩阵对MMT包进行FEC编码生成相应的FEC码。
c)将上步中对同一数据流的不同优先级的MMT包进行FEC编码后形成的FEC码进行整合生成一个FEC码流。
d)FEC编码完后，返回相应的修复字符以及FEC数据负载标识和原数据负载标识。
e)所有的修复字符打包成FEC修复包，发送到传输层。
通过信令识别FEC流以及采用的FEC编码结构和FEC code，为了支持这种机制，本实施例中修改了信令中的fec_flow_descriptor字段。
表1:fec_coding_structure的说明
Figure PCTCN2016070525-appb-000007
Figure PCTCN2016070525-appb-000008
原来的信令仅仅支持一个FEC流中复用多个媒体资源的修复信息，本实施例增加新的fec_coding_structure，fec_coding_structure的功能是用来描述当前采用的FEC编码方案，包括选择的编码算法，是否采用私有编码方案，最大的保护时间窗时间和值等信息，该字段位于AL-FEC信令中被传送到接收端。新增加的fec_coding_structure的标志位的值可以在现有的reserved的范围内选择，本方案中建议选取为0110。
表中的字段语义如下：
seed：随机种子，是作为产生伪随机数的初始条件，用于产生伪随机序列以构造FEC编码的生成矩阵和校验矩阵。seed的选取有多种算法，常用的如获取系统当前时间，获取当前进程ID等方法。
num_of_priority_for_mmtps：一个媒体资源中优先级的数目。
priority_mapping：优先级映射，用于指示媒体资源中不同数据包与资源优先级的映射关系。
private_fec_flag：指示位，指明是否使用私有的FEC编码方案。
private_flag：指示位，指明是否存在一个private_field用来描述所使用的私有FEC编码方案。
private_field_length：长度字段，用于描述私有FEC编码方案的字段的长度。
private_field：用于描述私有FEC方案的详细信息。
priority_id：优先级id，用于指示MMT包的优先级。
fec_code_id_for_repair_flow：用于描述所使用的FEC编码方案。
repair_flow_id：8位整数，用于指示生成的FEC repair flow，与FEC repair包的包头中的packet id有对应关系。
maximum_k_for_repair_flow：24位整数，描述在一个source sysmbol块中source symbol的最大数目。
maximum_p_for_repair_flow：24位整数，描述在一个repair sysmbol块中repair symbol的最大数目。
protection_window_time：保护窗时间，指示在FEC编码中发送第一个source或者repair包与发送最后一个source或者repair包之间的最大时间差，单位为毫秒。
protection_window_size：保护窗值，指示在FEC编码流中发送第一个FEC包的负载与发送最后一个FEC包负载之间的最大计数值。
修改后的信令针对一个媒体资源的不同重要性部分，采用不同的编码矩阵分别对不同的优先级进行FEC的编码控制，对于一个媒体数据流最终只生成一个FEC码流，使FEC机制能够更加细化，同时减少因过度FEC编码带来的网络流量增加，采用该方案服务端可以根据用户网络状态动态地调整媒体资源不同部分的FEC强度，收端收到信令后，可以从中解析出seed以及各个资源包的优先级，根据seed生成校验矩阵进行FEC解码恢复媒体资源。在网络带宽和用户体验间取得一个平衡点。
上述解决方式的优势：
1资源的节省，如果在接收端，人为的根据情况舍弃已经接收的帧(例如B帧)则造成了传输资源的浪费，上述方案从源端解决问题，让不想要的包在传输过程中更大概率丢掉，而更大程度的保护了重要的包。
2个性化的传输方案。视频传输应该是基于用户体验的，做更细致的不等差错保护就可以针对用户状况，如视觉体验，缓存状况等设计I帧和B帧的FEC保护的程度。
3能根据当前网络状况自适应的改变FEC编码的方式，在对不同优先级的数据进行不同强度的FEC保护的基础上，如果出现网络状况恶化丢包率增加的情况下，可自适应的调整FEC的seed，改变FEC的生成矩阵，增加FEC强度来抵抗网络恶化带来的影响。
4MMT中现有的AL-FEC方案只支持对不同的优先级的MMT包生成不同的FEC码流，本方案提出的方案支持对一个媒体资源数据流只生成一个FEC码流，可以极大的减少FEC带来的流量增加，减小网络压力。
实施例四：
本实施例以MMT传输协议为例，采用发明内容中的方法四进行实施：
如图1所示：MMT中针对媒体资源的FEC两层结构，第一层将source packet block分为较多的小块分别做FEC保护，第二层是一个整块做FEC保护。第一层划分较细致可以提供较小的时延，第二层保证了恢复性能和较小的冗余。图中P1、P2分别是FEC编码器1、2生成的修复符号块。
如图8所示：具体考虑传输信源数据具有两个重要等级的情况。将信源K个数据，按重要性分为2个重要性等级，即s1、s2。设s1为最重要部分MIB(Most ImPortant Bits,最重要信息比特)，s2为最不重要部分LIB(Least ImPortant Bits，最不重要数据比特)。
MMT方案中，MPU包下的MFU包拥有不同的重要性，缺少不等差错的保护，这样就不能设定个性化的传输方案。在MMT的包中，不同重要程度内容是可以区分开的(有指示标志)如附图3。然而编码中并没有涉及到不等差错的保护，虽然图1所示的两层编码方案和LA-FEC在一定程度上可以实现不等差错保护，但是灵活性低，复杂度高。尤其对于变化的信道状况和具有不同特征(如缓存不同)的接收用户，现有两种方式均存在冗余大，适应性差，只能针对单一状况，无法从用户体验考虑等问题。
解决方式：
1如果想更个性化的方案，随着信道质量改变保护方案，根据用户特征来定制，那么则需要加入专门的指示字段。由于不同的MFU有不同的重要性，则在MFU包头前加入指示，可以做更多个性化的保护方案。
如图7所示，是改进的MMT AL-FEC发送端架构，主要修改了MMT协议中流化后的输出，把数据流传输到源缓冲区，根据标识位指示的不同内容的重要程度进行D-EWF编码。
发送端架构如附图7所示。
发送端流程：
a)服务器端根据媒体资源生成MMTP流和信令；
b)把MMT负载传递到FEC机制，根据标识位中对内容重要性的区分，采用D-EWF码进行不等差错保护；
c)D-EWF码编码完后，返回相应的修复字符以及FEC数据负载标识和源数据负载标识；
d)所有的修复字符打包成FEC修复包，发送到传输层。
通过信令识别FEC流以及采用的FEC编码结构和D-EWF码，为了支持这种机制，本实施例中对信令中的FEC_FLOW_DESCRIPTOR字段做了如下修改：
Figure PCTCN2016070525-appb-000009
本实施例调整修改了fec_flow_descriptor字段顺序和定义，并且利用一个reserved字段来指示UEP机制的引入。修改后number_of_assets指示所有进行FEC的媒体资源，packet_id是每个相应媒体资源的标识，number_of_fec_flows指示每一个媒体资源对应多少个FEC流，即分多少级。针对一个媒体资源的不同重要性部分，采用本身就具有UEP性能的复制扩展窗喷泉码(D-EWF)进行保护。服务端可以根据用户网络状态动态地调整D-EWF的编码强度，收端收到信令后，根据相应的指示恢复媒体资源。在网络带宽和用户体验间取得一个平衡点。
2针对不同重要性程度的媒体内容，采用复制扩展窗喷泉码对重要媒体内容加强保护，其具体实施过程如下：
考虑传输信源数据具有两个重要等级的情况。如图3所示，将信源K个数据，按重要性分为2个重要性等级，即s1、s2。设s1为最重要部分MIB(Most ImPortant Bits,最重要信息比特)，s2为最不重要部分LIB(Least ImPortant Bits，最不重要数据比特)，π1、π2分别为s1、s2占总数据比重，s1、s2对应数据长度分别为k1＝π1·K，k2＝π2·K，其中π1+π2＝1。
对得到的两层数据，按扩展因子ψ1、ψ2进行扩展，得到虚拟扩展层数据：s1'、s2'，其中k1'、k2'代表s1'、s2'数据符号长度k1'＝k1·ψ1、k2'＝k2·ψ2。虚拟扩展数据的比重用γ1、γ2来描述，虚拟扩展层数据总和：K'＝k1'+k2'。
将虚拟扩展层数据s1'、s2'划入2个窗中，即W1＝s1'、W2＝s1'+s2'，W1的虚拟总数据为k1'，W2的虚拟总数据为|W2|＝K'。将第一个窗W1的度分布从k1增大到k1'，第二个窗W2的度分布从K增大到K'，采用鲁棒孤波度分布Ωrs(k,c,δ)，其概率分布分别为：
Figure PCTCN2016070525-appb-000010
Figure PCTCN2016070525-appb-000011
其中，
Figure PCTCN2016070525-appb-000012
c>0，k为数据符号个数，δ是译码失败的概率。设c＝0.1，δ＝0.5，对W1，W2分别采用鲁棒孤波度分布函数
Figure PCTCN2016070525-appb-000013
设第一个窗W1的选择概率为Γ1，W2的选择概率为Γ2＝1-Γ1。
在划分窗口之后，进行LT编码中的索引替换过程：随机生成一个数ξ，当0<ξ≤Γ1即选择第1个窗W1时，由度分布Ω(1)产生度d1，从W1虚拟数据中随机选择d1个数据。j代表W1虚拟数据k1·ψ1中的索引，j∈{0,…,k1·ψ1-1}，m代表原始k1的索引m∈{0,…,k1-1}，通过下列转换，由j得到索引m＝jmodk1，0≤j≤k1·ψ1-1；
当Γ1<ξ≤1即选择第2个窗W2时，由度分布Ω(2)产生度d2，从W2虚拟数据中随机选择d2个数据。j代表W2虚拟数据(k1·ψ1+k2·ψ2)中的索引，j∈{0,…,k1·ψ1+k2·ψ2-1}，m代表原始k2的索引m∈{0,…,K-1}，通过下列转换，由j得到索引m：
Figure PCTCN2016070525-appb-000014
索引m得到原始符号进行异或操作，得到编码码字，重复上述过程，直至得到足够的编码码字。图9给出了D-EWF码编码流程。
上述解决方式的优势：
1资源的节省，如果在接收端，人为的根据情况舍弃已经接收的媒体内容，则造成了传输资源的浪费，上述方案从源端解决问题，让重要性较低的媒体内容在传输过程中受到的保护程度较低，将更多地带宽资源分配给重要的内容，而更大程度的保护了重要的内容。
2个性化的传输方案。视频传输应该是基于用户体验的，通过改变重要和不重要媒体内容的D-EWF编码强度，做更细致的不等差错保护就可以针对用户状况，如视觉体验、缓存状况等。
以上对本发明的部分具体实施例进行了描述。需要理解的是，本发明并不局限于上 述特定实施方式，本领域技术人员可以在权利要求的范围内做出各种变形或修改，这并不影响本发明的实质内容。

Claims (18)

  1. 一种基于媒体内容的FEC机制，其特征在于，所述FEC机制采用以下四种方法中任一种实现：
    方法一：将媒体内容进行分级，并赋予不同的重要性，再根据属于不同重要程度帧的包，结合信道状况和用户体验感受，改变编码方案，按照所属帧的重要程度进行保护；
    方法二：将媒体内容进行分级，并赋予不同的重要性，在不对原有媒体数据流进行分流的情况下，结合信道状况和用户体验感受，按照媒体数据包中所包含的帧的重要程度，把数据包传送到相应的FEC编码器，进行不同程度的保护；
    方法三：将媒体内容进行分级，并赋予不同的重要性，在不对原有媒体数据流进行分流的情况下，根据当前信道状况，动态调整媒体数据包中所包含的帧的重要程度和相应的编码方案，把数据包传送到相应的FEC编码器，进行不同程度的保护，最终一个源数据流只会被编码为一个FEC码流；
    方法四：将媒体内容进行分级，并赋予不同的重要性，再结合信道状况和用户体验，采用本身就具有不等差错保护性能的数字喷泉码对不同重要性的数据进行保护。
  2. 根据权利要求1所述的基于媒体内容的FEC机制，其特征在于：所述四种方法：在与媒体处理单元(MPU)一同传输的信令信息中加入不等差错保护标志位。
  3. 根据权利要求1所述的基于媒体内容的FEC机制，其特征在于：所述四种方法：FEC机制加入专门的指示字段，以形成更多个性化的保护方案。
  4. 根据权利要求3所述的基于媒体内容的FEC机制，其特征在于：所述专门的指示字段加在MFU包头前。
  5. 根据权利要求1所述的基于媒体内容的FEC机制，其特征在于：所述四种方法：在与媒体处理单元(MPU)一同传输的信令信息中加入不等差错保护标志位，同时加入专门的指示字段，以形成更多个性化的保护方案。
  6. 根据权利要求1所述的基于媒体内容的FEC机制，其特征在于：
    所述方法一～方法三：将媒体内容进行分级，并赋予不同的重要性，然后利用信令和指示字段控制，采用不同的FEC编码强度编码；
    所述方法四：将媒体内容进行分级，并赋予不同的重要性，然后利用信令和指示字段控制，采用具有UEP性能的复制扩展窗喷泉码。
  7. 根据权利要求1-6任一项所述的基于媒体内容的FEC机制，其特征在于：所述方法一：针对MMT AL-FEC发送端架构，发送端流程为：
    a)服务器端根据媒体资源生成MMTP流和信令；
    b)分析媒体资源中每帧的重要性，把这些MMT负载分到不同的原数据流，传递到相应的FEC机制做保护；
    c)FEC编码完后，返回相应的修复字符以及FEC数据负载标识和原数据负载标识；
    d)所有的修复字符打包成FEC修复包，发送到传输层。
  8. 根据权利要求7所述的基于媒体内容的FEC机制，其特征在于：所述方法一：通过信令识别FEC流以及采用的FEC编码结构和FEC code，为了支持这种机制，修改了信令中的FEC_FLOW_DESCRIPTOR字段：原来的信令仅仅支持一个FEC流中复用多个媒体资源的修复信息，现在增加了一种针对一个媒体资源的不同重要性部分，分别进行FEC的信令控制，使FEC机制能够更加细化，服务端可以根据用户网络状态动态地调整媒体资源不同部分的FEC强度，在网络带宽和用户体验间取得一个平衡点。
  9. 根据权利要求1-6任一项所述的基于媒体内容的FEC机制，其特征在于：所述方法二：针对MMT AL-FEC发送端架构，发送端流程为：
    a)服务器端根据媒体资源生成MMTP流和信令；
    b)分析媒体资源中每帧的重要性，把MMT包传送到不同的FEC编码器，采用相应的FEC机制做保护，生成相应的FEC码；
    c)FEC编码完后，返回相应的修复字符以及FEC数据负载标识和原数据负载标识；
    d)所有的修复字符打包成FEC修复包，发送到传输层。
  10. 根据权利要求9所述的基于媒体内容的FEC机制，其特征在于：所述方法二：通过信令识别FEC流以及采用的FEC编码结构和FEC code，为了支持这种机制，修改了信令中的fec_flow_descriptor字段：原来的信令仅仅支持一个FEC流中复用多个媒体资源的修复信息，现增加针对一个媒体资源的不同重要性部分，修改后的信令针对一个媒体资源的不同重要性部分，分别进行FEC的编码控制，得到不同的FEC流，使FEC机制能够更加细化，服务端根据用户网络状态动态地调整媒体资源不同部分的FEC强度，收端收到信令后，根据相应的指示恢复媒体资源，在网络带宽和用户体验间取得一个平衡点。
  11. 根据权利要求9所述的基于媒体内容的FEC机制，其特征在于：所述修改了信令中的fec_flow_descriptor字段：在AL-FEC message中现有的三种fec_coding_structure的基础上增加了一种新的fec_coding_structure，fec_coding_structure的功能是用来描述当前采用的FEC编码方案，包括选择的编码算法，是否采用私有编码方案，最大的保护时间窗时间和值等信息，该字段位于AL-FEC信令中被传送到接收端；新增加的fec_coding_structure的标志位的值在现有的reserved的范围内选择。
  12. 根据权利要求11所述的基于媒体内容的自适应FEC机制，其特征在于：新增加的fec_coding_structure的标志位的值选取为0100。
  13. 根据权利要求1-6任一项所述的基于媒体内容的FEC机制，其特征在于：所述方法三：针对MMT AL-FEC发送端架构，发送端流程为：
    a)服务器端根据媒体资源生成MMTP流和信令；
    b)根据MMT流中每帧数据的重要性的不同，把MMT包传递到FEC编码器，对不同的优先级，采用不同的FEC编码矩阵对MMT包进行FEC编码生成相应的FEC码；
    c)将上步中对同一数据流的不同优先级的MMT包进行FEC编码后形成的FEC码进行整合生成一个FEC码流；
    d)FEC编码完后，返回相应的修复字符以及FEC数据负载标识和原数据负载标识；
    e)所有的修复字符打包成FEC修复包，发送到传输层。
  14. 根据权利要求13所述的基于媒体内容的FEC机制，其特征在于：所述方法通过信令识别FEC流以及采用的FEC编码结构和FEC code，为了支持这种机制，修改了信令中的fec_flow_descriptor字段：在AL-FEC message中现有的三种fec_coding_structure的基础上增加了一种新的fec_coding_structure，fec_coding_structure的功能是用来描述当前采用的FEC编码方案，包括选择的编码算法，是否采用私有编码方案，最大的保护时间窗时间和值等信息，该字段位于AL-FEC信令中被传送到接收端；新增加的fec_coding_structure的标志位的值在现有的reserved的范围内选择；
    原来的信令仅仅支持一个FEC流中复用多个媒体资源的修复信息，现在增加了一种针对一个媒体资源的不同重要性部分，分别进行FEC的信令控制，使FEC机制能够更加细化，服务端可以根据用户网络状态动态地调整媒体资源不同部分的FEC强度，在网络带宽和用户体验间取得一个平衡点。
  15. 根据权利要求14所述的基于媒体内容的FEC机制，其特征在于：新加的fec_coding_structure的标志位的值选取为0110。
  16. 根据权利要求1-6任一项所述的基于媒体内容的FEC机制，其特征在于：所述方法四：针对MMT AL-FEC发送端架构，发送端流程为：
    a)服务器端根据媒体资源生成MMTP流和信令；
    b)把MMT负载传递到FEC机制，根据标识位中对内容重要性的区分，采用D-EWF码进行不等差错保护；
    c)D-EWF码编码完后，返回相应的修复字符以及FEC数据负载标识和源数据负载标识；
    d)所有的修复字符打包成FEC修复包，发送到传输层。
  17. 根据权利要求16所述的基于媒体内容的自适应FEC机制，其特征在于：所述方法四：通过信令识别FEC流以及采用FEC编码结构和D-EWF码，为支持这种机制，需修改信令中的FEC_FLOW_DESCRIPTOR字段：原来的信令仅仅支持一个FEC流中复用多个媒体资源的修复信息，现增加一种针对一个媒体资源的不同重要性部分，分别进行FEC的信令控制，使FEC机制能够更加细化；服务端根据用户网络状态动态地调整媒体资源D-EWF码编码强度，在网络可用带宽和用户体验间取得一个平衡点。
  18. 根据权利要求1-6、16-17任一项所述的基于媒体内容的自适应FEC机制，其特征在于：所述方法四：通过引入扩展因子，虚拟扩展度分布范围，结合D-EWF码的窗技术，将虚拟扩展得到的数据进行分窗，同时各个窗分别采用优化的鲁棒孤波分布进行LT码，实现对重要性数据的加强保护。
PCT/CN2016/070525 2015-01-08 2016-01-08 一种基于媒体内容的fec机制 Ceased WO2016110275A1 (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
CA2998900A CA2998900C (en) 2015-01-08 2016-01-08 Fec mechanism based on media contents
US15/542,079 US10469202B2 (en) 2015-01-08 2016-01-08 Fec mechanism based on media content
KR1020177021722A KR102083302B1 (ko) 2015-01-08 2016-01-08 일종 미디어 컨텐츠에 기반한 fec 메커니즘
KR1020197034659A KR102251278B1 (ko) 2015-01-08 2016-01-08 일종 미디어 컨텐츠에 기반한 fec 메커니즘
JP2017536358A JP2018505597A (ja) 2015-01-08 2016-01-08 メディアコンテンツに基づくｆｅｃメカニズム

Applications Claiming Priority (8)

Application Number Priority Date Filing Date Title
CN201510010097.9A CN105827361B (zh) 2015-01-08 2015-01-08 一种基于媒体内容的fec方法
CN201510010097.9 2015-01-08
CN201510080576.8 2015-02-13
CN201510080576.8A CN105991226B (zh) 2015-02-13 2015-02-13 一种基于不等差错保护的前向纠错方法
CN201510673115.1A CN106603193B (zh) 2015-10-16 2015-10-16 一种基于媒体内容的fec方法
CN201510673115.1 2015-10-16
CN201510673091.X 2015-10-16
CN201510673091.XA CN106603192B (zh) 2015-10-16 2015-10-16 一种基于媒体内容的自适应fec方法

Publications (1)

Publication Number Publication Date
WO2016110275A1 true WO2016110275A1 (zh) 2016-07-14

Family

ID=56355544

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/070525 Ceased WO2016110275A1 (zh) 2015-01-08 2016-01-08 一种基于媒体内容的fec机制

Country Status (5)

Country Link
US (1) US10469202B2 (zh)
JP (2) JP2018505597A (zh)
KR (2) KR102083302B1 (zh)
CA (1) CA2998900C (zh)
WO (1) WO2016110275A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018177348A1 (zh) * 2017-03-29 2018-10-04 上海交通大学 一种基于媒体内容的自适应fec编码矩阵设计方法
CN109245850A (zh) * 2017-07-11 2019-01-18 上海交通大学 基于媒体内容的自适应系统码fec编译码方法
KR20200024319A (ko) * 2017-07-11 2020-03-06 상하이 지아오통 유니버시티 미디어 콘텐츠 기반의 자가 적응 시스템 코드 fec의 코딩 및 디코딩 방법, 장치, 시스템 및 매체

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10404411B2 (en) * 2016-02-19 2019-09-03 Mediatek Inc. Method and system of adaptive application layer FEC for MPEG media transport
JP7269112B2 (ja) * 2019-06-26 2023-05-08 日本放送協会 受信装置及びプログラム
CN114257858B (zh) * 2022-03-02 2022-07-19 浙江宇视科技有限公司 一种基于情感计算的内容同步方法和装置
CN118802067A (zh) * 2024-02-05 2024-10-18 中移（杭州）信息技术有限公司 数据传输方法、装置、设备、介质和程序产品

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101478373A (zh) * 2009-01-16 2009-07-08 北京航空航天大学 一种信源信道编码联合优化的自适应差错控制方法
CN101902296A (zh) * 2010-06-23 2010-12-01 中兴通讯股份有限公司 一种喷泉码的编解码方法和装置
WO2014005077A1 (en) * 2012-06-29 2014-01-03 Vid Scale, Inc. Frame prioritization based on prediction information
CN103795996A (zh) * 2012-11-01 2014-05-14 上海贝尔股份有限公司 3d视频传递方法和设备

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8233532B2 (en) * 2007-09-21 2012-07-31 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Information signal, apparatus and method for encoding an information content, and apparatus and method for error correcting an information signal
US8776161B2 (en) * 2008-02-12 2014-07-08 Ciena Corporation Systems and methods for video processing in network edge devices
EP2324635A1 (en) 2008-08-12 2011-05-25 Telefonaktiebolaget L M Ericsson (PUBL) Subdivision of media streams for channel switching
US9215082B2 (en) * 2009-10-06 2015-12-15 Thomson Licensing Method and apparatus for hop-by-hop reliable multicast in wireless networks
US8879640B2 (en) * 2011-02-15 2014-11-04 Hong Kong Applied Science and Technology Research Institute Company Limited Memory efficient implementation of LDPC decoder
CN102227103B (zh) 2011-06-23 2013-07-31 天津大学 一种基于不等差错保护喷泉码的信道编码传输方法
KR101874116B1 (ko) * 2011-10-24 2018-07-04 삼성전자주식회사 멀티미디어 시스템에서 순방향 오류 정정 부호화 방법 및 장치
KR101995221B1 (ko) * 2011-11-24 2019-07-16 삼성전자주식회사 통신 시스템에서 패킷 송수신 장치 및 방법
EP3288187B1 (en) * 2011-11-30 2023-01-04 Samsung Electronics Co., Ltd. Apparatus and method for transmiting layered source data
US9235467B2 (en) * 2013-03-15 2016-01-12 Pmc-Sierra Us, Inc. System and method with reference voltage partitioning for low density parity check decoding
WO2015053788A1 (en) * 2013-10-11 2015-04-16 Hewlett-Packard Development Company, L.P. Utilizing collected data from a software-defined networking network to diagnose a user experience
US9722651B2 (en) * 2015-01-09 2017-08-01 Qualcomm Incorporated Adaptive channel coding using polarization
US10637607B2 (en) * 2016-09-15 2020-04-28 Huawei Technologies Co., Ltd. Method and apparatus for encoding data using a polar code

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101478373A (zh) * 2009-01-16 2009-07-08 北京航空航天大学 一种信源信道编码联合优化的自适应差错控制方法
CN101902296A (zh) * 2010-06-23 2010-12-01 中兴通讯股份有限公司 一种喷泉码的编解码方法和装置
WO2014005077A1 (en) * 2012-06-29 2014-01-03 Vid Scale, Inc. Frame prioritization based on prediction information
CN103795996A (zh) * 2012-11-01 2014-05-14 上海贝尔股份有限公司 3d视频传递方法和设备

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018177348A1 (zh) * 2017-03-29 2018-10-04 上海交通大学 一种基于媒体内容的自适应fec编码矩阵设计方法
CN108667557A (zh) * 2017-03-29 2018-10-16 上海交通大学 一种基于媒体内容的自适应fec编码矩阵设计方法
CN108667557B (zh) * 2017-03-29 2021-04-02 上海交通大学 一种基于媒体内容的自适应fec编码矩阵设计方法
CN109245850A (zh) * 2017-07-11 2019-01-18 上海交通大学 基于媒体内容的自适应系统码fec编译码方法
KR20200024319A (ko) * 2017-07-11 2020-03-06 상하이 지아오통 유니버시티 미디어 콘텐츠 기반의 자가 적응 시스템 코드 fec의 코딩 및 디코딩 방법, 장치, 시스템 및 매체
CN109245850B (zh) * 2017-07-11 2021-04-02 上海交通大学 基于媒体内容的自适应系统码fec编译码方法
EP3654556A4 (en) * 2017-07-11 2021-04-21 Shanghai Jiao Tong University Media content-based adaptive method, device and system for fec coding and decoding of systematic code, and medium
KR102383892B1 (ko) * 2017-07-11 2022-04-08 상하이 지아오통 유니버시티 미디어 콘텐츠 기반의 자가 적응 시스템 코드 fec의 코딩 및 디코딩 방법, 장치, 시스템 및 매체

Also Published As

Publication number Publication date
JP2018505597A (ja) 2018-02-22
KR102251278B1 (ko) 2021-05-13
CA2998900A1 (en) 2016-07-14
JP2021153298A (ja) 2021-09-30
JP7343915B2 (ja) 2023-09-13
CA2998900C (en) 2022-08-30
US10469202B2 (en) 2019-11-05
KR20190133294A (ko) 2019-12-02
US20180069654A1 (en) 2018-03-08
KR20170102524A (ko) 2017-09-11
KR102083302B1 (ko) 2020-03-02

Similar Documents

Publication Publication Date Title
JP7343915B2 (ja) メディアコンテンツに基づくｆｅｃメカニズム
CN106603192B (zh) 一种基于媒体内容的自适应fec方法
KR102048452B1 (ko) 멀티미디어 시스템에서 순방향 오류 정정 패킷을 생성하는 방법과 그 오류 정정 패킷을 송수신하는 방법 및 장치
CN104205698B (zh) 使用前向纠错方案的分组发送/接收装置和方法
US8315238B2 (en) Network processing node and method for manipulating packets
CN102640508B (zh) 基于优先级的无线视频传输的方法
KR20130040096A (ko) 이동 통신 시스템에서 패킷 송수신 장치 및 방법
KR102383892B1 (ko) 미디어 콘텐츠 기반의 자가 적응 시스템 코드 fec의 코딩 및 디코딩 방법, 장치, 시스템 및 매체
US10958376B2 (en) Method and apparatus for transmitting and receiving packet in communication system
CN106603193B (zh) 一种基于媒体内容的fec方法
CN101917625A (zh) 一种基于联合信源-网络编码的可分级视频流传输方法
RU2646346C2 (ru) Устройство и способ передачи и приема пакета с прямой коррекцией ошибок
CN108667557B (zh) 一种基于媒体内容的自适应fec编码矩阵设计方法
CN105827361A (zh) 一种基于媒体内容的fec机制
CN109687934B (zh) 基于媒体内容的自适应系统码fec方法、装置及系统
CN108306705A (zh) 一种基于媒体内容的自适应fec机制及系统
CN105991226B (zh) 一种基于不等差错保护的前向纠错方法
CN101938322B (zh) 带宽自适应数据分级保护方法与装置
CN101931492A (zh) 数据块前向纠错算法的确定方法与装置
CN105978884A (zh) 一种用于突发擦除信道的实时流编码方法
CN120075887A (zh) 数据传输方法、装置、计算机可读介质及电子设备
Cooklev Dynamic bandwidth allocation and channel coding in providing QoS for wireless local area networks
JP2008092346A (ja) 送信装置及び受信装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16734935

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2017536358

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 20177021722

Country of ref document: KR

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 15542079

Country of ref document: US

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 14/11/2017)

ENP Entry into the national phase

Ref document number: 2998900

Country of ref document: CA

122 Ep: pct application non-entry in european phase

Ref document number: 16734935

Country of ref document: EP

Kind code of ref document: A1