CN112333470B - FEC (forward error correction) system based on video frame - Google Patents
FEC (forward error correction) system based on video frame Download PDFInfo
- Publication number
- CN112333470B CN112333470B CN202011160364.8A CN202011160364A CN112333470B CN 112333470 B CN112333470 B CN 112333470B CN 202011160364 A CN202011160364 A CN 202011160364A CN 112333470 B CN112333470 B CN 112333470B
- Authority
- CN
- China
- Prior art keywords
- frame
- packet
- module
- data
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000011084 recovery Methods 0.000 claims abstract description 44
- 238000009432 framing Methods 0.000 claims abstract description 43
- 238000004806 packaging method and process Methods 0.000 claims abstract description 29
- 230000006835 compression Effects 0.000 claims abstract description 4
- 238000007906 compression Methods 0.000 claims abstract description 4
- 239000012634 fragment Substances 0.000 claims description 51
- 239000011159 matrix material Substances 0.000 claims description 27
- 230000005540 biological transmission Effects 0.000 claims description 15
- 238000000034 method Methods 0.000 claims description 8
- 230000001965 increasing effect Effects 0.000 claims description 6
- 238000012856 packing Methods 0.000 claims 5
- 102100037812 Medium-wave-sensitive opsin 1 Human genes 0.000 claims 1
- 230000003247 decreasing effect Effects 0.000 claims 1
- 230000009466 transformation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000010154 cross-pollination Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
- H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64784—Data processing by the network
- H04N21/64792—Controlling the complexity of the content stream, e.g. by dropping packets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Detection And Prevention Of Errors In Transmission (AREA)
Abstract
The invention discloses a FEC error correction system based on video frame, comprising: the system comprises a video acquisition coding module, a frame proportion module, a frame disassembling module, an I frame packaging module, an I frame slicing module, a P frame packaging module, an I frame FEC coding module, a mixed FEC coding module, a network mixed sending module, a network mixed receiving module, an I frame FEC decoding module, a mixed FEC decoding module, a framing module and a video decoding display module; the video acquisition coding module acquires a video, and codes the video into a video compressed stream through video compression, wherein the video compressed stream consists of a video frame group, and the video frame group comprises an I frame and a plurality of P frames. The invention generates the FEC error correction packet according to the frame, can shorten the recovery time delay, promote the FEC work efficiency, effectively utilize the network resource; mixed FEC encoding is carried out by using P frame and I frame slices, so that the reliability of the I frame is improved; the I frame FEC decoding module and the hybrid FEC decoding module are mutually matched, and the packet loss recovery capability is strong.
Description
Technical Field
The invention relates to the field of data processing, in particular to an FEC (forward error correction) system based on video frames.
Background
The current FEC (forward Error correction) forward Error correction code for video network packets carries out Error correction, which brings several problems, I frames of a video stream are large (generally 10 times to 100 times of P frames), P frames represent picture transformation amount, and when the picture transformation amount is small, the P frames become small, and a uniform FEC method introduces fixed time delay, which results in slow packet loss recovery. A complete image is generated by a single I frame, and a P frame needs to be decoded by the I frame, which means that the priority of the I frame is higher than that of the P frame, and the priority of FEC is needed to ensure that packet loss recovery needs to be performed on the I frame, so a frame FEC method for video stream is needed.
Disclosure of Invention
The invention aims to provide a FEC error correction system based on video frames, which generates FEC error correction packets according to frames, shortens recovery time delay, improves FEC working efficiency, and improves I frame reliability by using P frames and I frame slices for mixed FEC encoding.
The technical scheme is as follows:
the invention provides a FEC error correction system based on video frame, comprising: the system comprises a video acquisition coding module, a frame proportion module, a frame disassembling module, an I frame packaging module, an I frame slicing module, a P frame packaging module, an I frame FEC coding module, a mixed FEC coding module, a network mixed sending module, a network mixed receiving module, an I frame FEC decoding module, a mixed FEC decoding module, a framing module and a video decoding display module;
the video acquisition coding module: the method comprises the steps that collected videos are compressed through videos and are encoded into video compressed streams, the video compressed streams consist of video frame groups, each video frame group comprises an I frame and a plurality of P frames, the I frames can be independently decoded, the P frames depend on the previous frame for decoding, and a video collection and encoding module sends the video compressed streams to a frame proportion module and a frame disassembling module;
a frame proportion module: confirming the proportion K of the I frame and the P frame according to GOP (interval of two I frames) of a previous video frame group of the video compressed stream, and sending the proportion K to an I frame slicing module;
a frame disassembling module: sending the I frame to an I frame packaging module, and sending the P frame to a P frame packaging module;
i frame packaging module: receiving I frame data of a frame disassembling module, performing subpackage processing, subpackaging according to 1400 bytes for network transmission, and sending the subpackaged data to a network mixed sending module and an I frame FEC encoding module;
an I-frame slicing module: receiving I frame subpackage data of a frame I frame packaging module, grouping the I frame subpackage data according to the proportion K of an I frame and a P frame to obtain K parts of fragment data, marking a sequence number ki (in a range of 1-K) on each fragment, and sending the fragment data, the sequence number and the proportion K to a hybrid FEC encoding module;
a P frame packaging module: performing packet processing on P frame data for network transmission, sending the packetized data to a network hybrid sending module, marking the packetized data of each P frame by a P frame packetizing module with a frame sequence number ki, wherein the frame sequence number ki is a position label of a video frame group, numbering P frames from 1, and sending the P frame data and the frame sequence number ki to a hybrid FEC encoding module;
i frame FEC encoding module: receiving the data of the sub-packets of the I frame packet module to carry out FEC forward error correction coding, wherein the data of I frames is larger, two-dimensional single matrix cross-pollination coding is adopted, the matrix which is close to a square is used for reducing the number of FEC data of the I frame, the square value of N is selected to be just larger than the number of the sub-packets, the Y axis is reduced by one, whether N (N-1) is larger than the number of the sub-packets or not is judged, if so, the N (N-1) matrix is selected, otherwise, the N (N) matrix is selected, and the final matrix is set to be N M; the data of the video sub-packets are sequentially put into an N x M matrix from left to right from top to bottom, the XOR is carried out on the video packet data of each line according to 1400 bytes to finally obtain a line error correction code, the XOR is carried out on the video packet data of each column according to 1400 bytes to finally obtain a column error correction code, if no video fragment data exists on the N x M matrix, the position is skipped, if the video packet data is the first packet of the video sub-packets, the XOR is carried out according to the packet length, and the line error correction code, the column error correction code and the frame I data are sent to a network mixed sending module;
a hybrid FEC encoding module: receiving P frame sub-packet data, a serial number and a P frame length of a P frame packaging module, receiving I frame fragment data, a serial number and a proportion K of the I frame fragment data and the P frame sub-packet data of the I frame fragment module, matching the I frame fragment data and the P frame sub-packet data according to the serial number, using a single-row error correcting code, firstly carrying out XOR calculation on the sub-packet data of the P frame to obtain f, carrying out XOR calculation on the sub-packet of the I frame fragment data by using f to obtain the error correcting code of the P frame, and carrying out XOR calculation on the actual length of the last packet of the sub-packet data of the P frame; the hybrid FEC encoding module sends the generated error correcting code and the length of the P frame and the frame number and the proportion K of the P frame to a network hybrid sending module;
a network hybrid transmission module: receiving the packet data and the proportion K of an I frame packet module and a P frame packet module, using RT P for packet transmission, gradually increasing the seq sequence numbers of RTP (real-time transport protocol), gradually increasing the timestamp of RTP, and using PT (payload type) of independent RTP as a video RTP stream to send the video RTP stream to a network hybrid receiving module; receiving the line error correction code and column error correction code of an I frame FEC encoding module, the size of I frame data and the frame number of a P frame, sequentially numbering the line error correction code and the column error correction code by x, using RTP to pack and transmit, wherein the timestamp of the RTP is the same as the timestamp of the RTP corresponding to a video frame and is used for associating the error correction code with the I frame at a receiving end, storing error correction code data by a load (P payload) of the RTP, placing the sequential numbering x of the error correction code, the length of the I frame data and the seq number of a first packet of the I frame in filling information of the RTP, and sending the PT (payload type) of the independent RTP as an I frame FEC stream to a network hybrid receiving module; receiving the length of an error correcting code and a P frame of a hybrid FEC encoding module, using RTP for packaging transmission, wherein the time stamp of the RTP is related to the RTP time stamp of a corresponding video frame and is used for associating the error correcting code and the P frame at a receiving end, loading error correcting code data in the RTP, and filling information of the RTP is used for placing the data length of the P frame, the seq number of the first packet of the P frame, the frame number of the P frame, the I frame time stamp and the proportion K, and sending the data length, the seq number of the first packet of the P frame, the frame number of the P frame, the I frame time stamp and the proportion K to a network hybrid receiving module as an independent RTP stream, and sending the data length, the P (payload type) of the independent RTP to the network hybrid receiving module as a hybrid FEC stream;
a network hybrid receiving module: receiving a video RTP stream, an I frame FEC stream and a mixed FEC stream of a network mixed sending module, grouping according to a timestamp of the RTP, acquiring a load (Payload) of the video RTP stream, obtaining frame sub-packet data, and sending the frame sub-packet data to a framing module; judging whether the packet FEC packet is an I-frame FEC stream or a mixed FEC stream, if the packet FEC packet is the I-frame FEC stream, extracting a sequence number x, an I-frame data length and an I-frame first packet seq sequence number according to RTP filling information of the I-frame FEC stream, restoring row error correction codes and column error correction codes from row error correction codes and column error correction codes of a load (Payload) of the RTP of the I-frame FEC stream, sending the row error correction codes, the column error correction codes and I-frame timestamps to an I-frame FEC decoding module, calculating the I-frame packet number P as I-frame data length/1400 +1 according to the I-frame data length and the I-frame first packet seq sequence number, and if the I-frame data length is not zero, subtracting one from the packet number, wherein if the I-frame data length is zero, the last packet length is 1400, and if the I-frame data length is not zero, the last packet length is a remainder; the method comprises the steps of obtaining seq sequence numbers of all RTP packets of an I frame according to the seq sequence number of a first packet of the I frame and the number of the I frame, searching a lost packet list for the seq sequence numbers and RTP streams of the I frame of the packet, obtaining I frame sub-packet data from a load (Payload) of a video RTP stream, and sending the I frame sub-packet data, the last sub-packet length, the lost packet list and an I frame timestamp to an I frame FEC decoding module, a hybrid FEC decoding module and a framing module; if the mixed FEC flow is generated, taking out the data length of P frames and the frame sequence number of a P frame head packet seq and the frame sequence number of the P frame and the time stamp and proportion K of the I frame according to the RTP filling information of the mixed FEC flow, taking out the error correction code data of the RTP load of the mixed FEC flow, and sending the error correction code data and the time stamp to a mixed FEC decoding module; calculating the P frame packet number P which is equal to the P frame data length/1400 +1 according to the P frame data length and the first packet seq serial number of the P frame, if the P frame data length takes the remainder 1400 as zero, subtracting one from the packet number, and the last packet length is 1400, if the P frame data length takes the remainder 1400 as non-zero, and the last packet length is the remainder; the method comprises the steps of obtaining seq sequence numbers of all RTP packets of a P frame according to the seq sequence number of the first packet of the P frame and the number of the P frame, searching a lost packet list according to the seq sequence numbers and the RTP stream of the P frame of the group, obtaining P frame sub-packet data from a load (Payload) of a video RTP stream, and sending the P frame sub-packet data, the last sub-packet length, the lost packet list, a P frame time stamp, a frame sequence number of the P frame, an I frame time stamp and a proportion K of a video frame group to a hybrid FEC decoding module and a framing module;
an I-frame FEC decoding module: recovering the lost packet of the I frame, receiving the row error correction code and column error correction code of the network hybrid receiving module and the I frame timestamp, and the number of I frame sub-packet data and I frame sub-packets, the last sub-packet length, the lost packet list and the I frame timestamp, associating row error correction codes and column error correction codes with I frame time stamps, and generating N x M matrix by using I frame FEC encoding module according to the number of packets and I frame packet data and lost packet list, if the row/column of the lost packet list matrix has only one lost packet, performing XOR calculation on other row/column packet data and the row/column error correcting code to obtain the packet data of the position, if the packet is the last packet, performing exclusive OR according to the packet length, sending the packet data to a framing module and a mixed FEC decoding module, and then performing secondary reduction on the I frame packet data to the I frame FEC decoding module; the I frame FEC decoding module receives the recovery sub-packets of the hybrid FEC decoding module to carry out secondary recovery; the frame I FEC decoding module recovers sub-packet secondary reduction, deletes the frame I sub-packet in a lost packet list, checks whether a packet loss exists in a column and a row of the frame I sub-packet, if only one packet loss exists, performs XOR calculation on other sub-packet data of the row and the row/column error correction code to obtain sub-packet data of the position, sends the sub-packet data to the framing module and the hybrid FEC decoding module, and then uses the frame I sub-packet data to perform secondary reduction on the frame I FEC decoding module; enhancing packet loss recovery capability through the frame I FEC decoding module and the mixed FEC decoding module;
a hybrid FEC decoding module: receiving P frame packet data and the last packet length of a network hybrid receiving module, a lost packet list, a P frame timestamp, a frame number of a P frame, a I frame timestamp of the video frame group and a proportion K; the hybrid FEC decoding module receives error correcting code data and P frame time stamps of the network hybrid receiving module, and receives the number of I frame sub-packet data and I frame sub-packets of the network hybrid receiving module, the last sub-packet length, a lost packet list and I frame time stamps; acquiring a corresponding I frame according to the I frame timestamp of the video frame group of the P frame, equally dividing the number of I frame sub-packets according to the proportion K to obtain a K sub-packet list, acquiring I frame sub-packet data corresponding to the P frame according to the frame number of the P frame as a sub-packet list subscript, checking whether the sub-packet of the I frame lost packet list is lost or not, if the sub-packet is not lost and the lost packet list of the P frame loses one packet, performing XOR on the sub-packet data of other P frames and the I frame sub-packet and error correction code data to obtain the lost packet of the P frame, and sending a recovery data packet of the P frame and the P frame timestamp to a framing module; if the lost packet list of the P frame has no packet loss and the lost packet list of the I frame fragment has a packet loss, carrying out exclusive OR on P frame sub-packet data and other sub-packet data of the I frame fragment and error correction code data to obtain recovery sub-packet data of the I frame, and sending the sub-packet data of the I frame and a time stamp of the I frame to an FEC decoding module and a framing module of the I frame; the hybrid FEC decoding module obtains I frame recovery sub-packet data and I frame time stamps of the I frame FEC decoding module, a corresponding I frame is found according to the I frame time stamps, a packet is deleted from an I frame lost packet list, the recovery sub-packet data corresponds to a fragment in a K fragment list, whether the fragment has a packet loss is judged, if no packet is lost, the frame number of a P frame is matched according to the position offset of the fragment, a corresponding P frame is found, if the P frame only loses one packet, the other sub-packet data of the P frame and the sub-packet data of the I frame fragment are subjected to XOR with error correction code data to obtain the packet loss of the P frame, and the recovery data packet of the P frame and the P frame time stamps are sent to the framing module;
a framing module: receiving I frame sub-packet data, the last sub-packet length, a lost packet list and an I frame timestamp of a network hybrid receiving module, and receiving P frame sub-packet data, the last sub-packet length, the lost packet list, the P frame timestamp and the I frame timestamp; receiving I frame recovery packet data of an I frame FEC decoding module, and receiving I frame recovery packet data and P frame recovery packet data of a hybrid FEC decoding module; the framing module deletes lost packets of the I frame lost packet list after receiving the I frame recovery packet data, checks the I frame lost packet list, and if the I frame has no packet loss, merges the I frame packet data to obtain I frame data and sends the I frame data to the video decoding display module; if the P frame has no packet loss, the I frame timestamp of the P frame finds the corresponding I frame, and other frames between the I frame and the timestamp of the P frame have no packet loss, the P frame sub-packaging data is combined to obtain P frame data which is sent to the video decoding display module;
the video decoding display module: and decoding and displaying the I frame and the P frame data generated by the framing module.
The invention has the beneficial effects that: the FEC error correction packet is generated according to the frame, so that the recovery time delay can be shortened, the FEC working efficiency can be improved, and the network resources can be effectively utilized; mixed FEC encoding is carried out by using P frame slices and I frame slices, so that the reliability of the I frame is improved; the I frame FE C decoding module and the hybrid FEC decoding module are mutually matched, and the packet loss recovery capability is strong.
Drawings
Fig. 1 is a flow chart of an FEC error correction system based on video frames according to the present invention.
Detailed Description
The embodiments of the present invention will be described in detail below, and the embodiments described below by referring to the drawings are exemplary and intended to be used for explaining the present invention, and should not be construed as limiting the present invention.
As shown in fig. 1, the present invention provides a FEC error correction system based on video frames, comprising: the system comprises a video acquisition coding module 1, a frame proportion module 2, a frame disassembling module 3, an I frame packaging module 4, an I frame slicing module 5, a P frame packaging module 6, an I frame F EC coding module 7, a mixed FEC coding module 8, a network mixed sending module 9, a network mixed receiving module 10, an I frame FE C decoding module 11, a mixed FEC decoding module 12, a framing module 13 and a video decoding display module 14;
the video acquisition and coding module 1 acquires a video, and codes the video into a video compressed stream through video compression, wherein the video compressed stream consists of a video frame group, the video frame group comprises an I frame and a plurality of P frames, the I frame can be independently decoded, the P frames depend on the previous frame for decoding, and the video acquisition and coding module 1 transmits the video compressed stream to the frame proportion module 2 and the frame disassembling module 3;
the frame proportion module 2 confirms the proportion K of I frame and P frame according to GOP (interval of two I frames) of the previous video frame group of the video compressed stream, and sends the proportion K to the I frame slicing module 5;
the frame disassembling module 3 sends the I frame to the I frame packaging module 4, and sends the P frame to the P frame packaging module 6;
the I frame packaging module 4: receiving I frame data of the frame disassembling module 3, performing subpackage processing, subpackaging according to 1400 bytes for network transmission, and sending the subpackaged data to a network mixed sending module 9 and an I frame FEC encoding module 7;
i-frame slicing module 5: receiving I frame subpackaged data of a frame I frame packaging module 4, grouping the I frame subpackaged data according to the proportion K of the I frame and the P frame to obtain K parts of fragment data, marking a sequence number ki (in a range of 1-K) on each fragment, and sending the fragment data, the sequence number and the proportion K to a hybrid FEC encoding module 8;
the P frame packaging module 6: the P frame data is subjected to sub-packet processing for network transmission, the sub-packet data is sent to a network hybrid sending module 9, a P frame packaging module 6 marks the sub-packet data of each P frame with a frame sequence number ki, the frame sequence number ki is a position label of a video frame group, the P frames start numbering from 1, and the P frame data and the frame sequence number ki are sent to a hybrid FEC encoding module 8;
the I-frame FEC encoding module 7 receives the packetized data of the I-frame packet module 4 and performs FEC forward error correction encoding, and the specific flow is as follows:
the frame I data is larger, two-dimensional single matrix alternating-current coding is adopted, a matrix close to a square shape is used for reducing the number of the frame I fec data, the square value of N is just larger than the number of sub-packets, the Y axis is reduced by one, whether N (N-1) is larger than the number of the sub-packets or not is judged, if yes, the N (N-1) matrix is selected, otherwise, the N (N-N) matrix is selected, and the final matrix is set to be N M;
the data of the video sub-package sequentially from left to right, from top to bottom enter into an N x M matrix, each line of video sub-package data is subjected to XOR according to 1400 bytes to finally obtain a line error correction code, each column of video sub-package is subjected to XOR according to 1400 bytes to finally obtain a column error correction code, if no video fragment data exists on the N x M matrix, the position is skipped, if the position is the most packet of the video sub-package, XOR is carried out according to the packet length, and the line error correction code, the column error correction code and the I frame data are sent to a network mixed sending module 9;
the hybrid FEC encoding module 8 performs hybrid FEC encoding on the I frame and the P frame, and the specific flow is as follows:
the hybrid FEC encoding module 8 receives P frame sub-packet data, a serial number and a P frame length of the P frame packaging module 6, receives I frame fragment data, a serial number and a proportion K of the I frame fragment data and the P frame sub-packet data of the I frame slicing module 5, matches the I frame fragment data and the P frame sub-packet data according to the serial number, uses a single-row error correcting code, firstly carries out XOR calculation on the P frame sub-packet data to obtain f, uses the f to carry out XOR calculation on the I frame fragment sub-packet to obtain the P frame error correcting code, and carries out XOR calculation on the actual length of the last packet of the P frame sub-packet data; the hybrid FEC encoding module 8 sends the generated error correcting code and the length of the P frame and the frame number and the proportion K of the P frame to the network hybrid sending module 9;
the network hybrid transmission module 9 transmits a video RTP stream, a hybrid FEC stream, and an I frame FEC stream, and the specific flow is as follows:
the network hybrid sending module 9 receives the packet data and the proportion K of the I frame packet module 4 and the P frame packet module 6, and uses RTP to perform packet transmission, wherein seq serial numbers of RTP are gradually increased one by one, timestamps of RTP are gradually increased, and PT of independent RTP is used as a video RTP stream to be sent to the network hybrid receiving module 10;
the network hybrid sending module 9 receives the row error correction code and column error correction code of the I-frame FEC coding module 7, the size of the I-frame data and the frame number of the P-frame, sequentially numbers x the row error correction code and the column error correction code, and uses RTP for packet transmission, the RTP timestamp of the RTP is the RTP timestamp of the corresponding video frame, and is used for associating the receiving end error correction code with the I-frame, the RTP load stores the error correction code data, the filling information of the RTP places the sequential number x of the error correction code, the length of the I-frame data and the seq number of the first packet of the I-frame, and uses PT of the independent RTP as an I-frame FEC stream to send to the network hybrid receiving module 10;
the network hybrid sending module 9 receives the error correction code and the length of the P frame of the hybrid FEC encoding module 8, and uses RTP for packet transmission, where the RTP timestamp is associated with the RTP timestamp of the corresponding video frame, and is used for receiving the end error correction code and the P frame, and the RTP load of the RTP stores the error correction code data, and the filling information of the RTP stores the frame length of the P frame, the seq number of the first packet of the P frame, the frame number of the P frame, the I frame timestamp, and the ratio K, and sends the frame length, the frame number of the P frame, the frame number of the I frame, and the ratio K to the network hybrid receiving module 10 as an independent RTP stream, and sends the PT of the independent RTP as a hybrid FE C stream to the network hybrid receiving module 10;
the network hybrid receiving module 10 receives the video RTP stream, the I-frame FEC stream, and the hybrid FEC stream of the network hybrid sending module 9, groups the streams according to the RTP timestamp, obtains a load (Payload) of the video RTP stream, obtains frame packetization data, and sends the frame packetization data to the framing module 13, where the specific flow is as follows:
judging whether the packet FEC packet is an I-frame FEC stream or a mixed FEC stream, if the packet FEC packet is the I-frame FEC stream, extracting a sequence number x, an I-frame data length and an I-frame first packet seq number according to RTP filling information of the I-frame FEC stream, recovering a row error correction code and a column error correction code from loaded row error correction code and column error correction code of R TP of the I-frame FEC stream according to the sequence number, sending the row error correction code, the column error correction code and an I-frame timestamp to an I-frame FEC decoding module 11, calculating according to the I-frame data length and the I-frame first packet seq number, calculating the I-frame packet number P as the I-frame data length/1400 +1, if the I-frame data length is zero, subtracting one packet number, and if the I-frame data length is not zero, determining the last packet length as a remainder;
the seq sequence numbers of all RTP packets of the I frame are obtained according to the seq sequence number of the first packet of the I frame and the frame number of the I frame, a lost packet list is searched for by the seq sequence numbers and the RTP stream of the I frame of the packet, I frame sub-packet data are obtained by the load of the video RTP stream, and the I frame sub-packet data, the last sub-packet length, the lost packet list and the I frame timestamp are sent to an I frame FEC decoding module 11, a hybrid FEC decoding module 12 and a framing module 13;
if the mixed FEC stream is generated, the data length of the P frame, the sequence number of the first packet s eq of the P frame, the frame sequence number of the P frame, the timestamp of the I frame, and the ratio K are extracted according to the RTP padding information of the mixed FEC stream, the error correction code data of the RTP load of the mixed FEC stream is extracted, and the error correction code data and the timestamp of the I frame are sent to the mixed FEC decoding module 12; calculating the P frame packet number P which is equal to the P frame data length/1400 +1 according to the P frame data length and the P frame first packet s eq sequence number, if the P frame data length takes the remainder 1400 as zero, subtracting one from the packet number, wherein the last sub-packet length is 1400, and if the P frame data length takes the remainder 1400 as non-zero, the last sub-packet length is the remainder;
the seq sequence numbers of all RTP packets of the P frame are obtained according to the seq sequence number of the first packet of the P frame and the number of the P frame, a lost packet list is searched for by the seq sequence numbers and the RTP stream of the P frame of the packet, P frame sub-packet data are obtained by the load of the video RTP stream, and the P frame sub-packet data, the last sub-packet length, the lost packet list, the P frame timestamp, the frame sequence number of the P frame, the I frame timestamp and the proportion K of the video frame group are sent to the hybrid FEC decoding module 12 and the framing module 13;
the I frame FEC decoding module 11 recovers the lost packet of the I frame, and the specific flow is as follows:
receiving a row error correction code, a column error correction code and an I-frame timestamp of the network hybrid receiving module 10, the number of I-frame packet data and I-frame packets, the last packet length, a lost packet list and the I-frame timestamp, and associating the row error correction code, the column error correction code, the I-frame packet data and the lost packet list according to the I-frame timestamp;
the number of the sub-packets generates an N x M matrix by using a method of an I-frame FEC encoding module 7, if only one packet is lost in a row/column of a lost packet list matrix, the exclusive OR calculation is carried out on other sub-packet data in the row/column and the row/column error correcting code to obtain sub-packet data in the position, if the last sub-packet is obtained, the exclusive OR calculation is carried out according to the sub-packet length, the sub-packet data is sent to a framing module 13 and a mixed FEC decoding module 12, and then the I-frame sub-packet data is used for carrying out secondary reduction on an I-frame FEC decoding module 11;
the I-frame FEC decoding module 11 receives the recovery packet of the hybrid FEC decoding module 12 to perform secondary restoration;
the I frame FEC decoding module 11 recovers the sub-packet secondary reduction, deletes the I frame sub-packet in the lost packet list, checks whether a packet loss exists in the column and the row of the I frame sub-packet, if only one packet loss exists, performs XOR calculation on the other sub-packet data of the row and the row/column error correction code to obtain the sub-packet data of the position, sends the sub-packet data to the framing module 13 and the hybrid FEC decoding module 12, and then uses the I frame sub-packet data to perform the secondary reduction on the I frame FEC decoding module 11;
the packet loss recovery capability is mutually enhanced through the I frame FEC decoding module 11 and the hybrid FEC decoding module 12;
the hybrid FEC decoding module 12 recovers the lost packet of the I frame and the lost packet of the P frame, and the specific flow is as follows:
the hybrid FEC decoding module 12 receives the P frame packet data and the last packet length and the lost packet list of the network hybrid receiving module 10, the P frame timestamp, the frame number of the P frame, and the I frame timestamp and the ratio K of the video frame group;
the hybrid FEC decoding module 12 receives the error correction code data and the P frame timestamp of the network hybrid receiving module 10, and receives the number of I frame packet data and I frame packet of the network hybrid receiving module 10, the last packet length, the lost packet list, and the I frame timestamp;
the hybrid FEC decoding module 12 obtains a corresponding I frame according to the I frame timestamp of the video frame group of the P frame, equally divides the number of I frame packets according to the proportion K to obtain a K fragment list, obtains I frame fragment data corresponding to the P frame according to the frame number of the P frame as a fragment list subscript, checks whether the fragment of the I frame lost packet list loses a packet, and if the fragment does not lose a packet and the lost packet list of the P frame loses a packet, obtains the packet loss of the P frame by performing xor on the other packet data of the P frame and the packet data of the I frame fragment and error correction code data, and sends a recovery data packet of the P frame and the P frame timestamp to the framing module 13; if the packet loss list of the P frame has no packet loss and the packet loss list of the I frame fragment has a packet loss, performing exclusive OR on P frame packet data and other packet data of the I frame fragment and error correction code data to obtain recovery packet data of the I frame, and sending the packet data of the I frame and a time stamp of the I frame to an I frame FEC decoding module 11 and a framing module 13;
the hybrid FEC decoding module 12 obtains I-frame recovery sub-packet data and I-frame timestamp of the I-frame FEC decoding module 11, finds a corresponding I-frame according to the I-frame timestamp, deletes a sub-packet from the I-frame lost packet list, where the recovery sub-packet data corresponds to a fragment in the K-fragment list, determines whether the fragment has a packet loss, if no packet is lost, matches a frame number of a P-frame according to a position offset of the fragment, finds a corresponding P-frame, and if the P-frame has only one packet loss, performs xor on the other sub-packet data of the P-frame and the sub-packet data of the I-frame fragment and error correction code data to obtain a packet loss of the P-frame, and sends a recovery data packet and timestamp of the P-frame to the framing module 13;
the framing module 13 generates I-frame and P-frame data, and the specific process is as follows:
the framing module 13 receives the I frame packet data, the last packet length, the lost packet list and the I frame timestamp of the network hybrid receiving module 10, and receives the P frame packet data, the last packet length, the lost packet list, the P frame timestamp and the I frame timestamp;
the framing module 13 receives the I-frame recovery packetized data of the I-frame FEC decoding module 11, and receives the I-frame recovery packetized data and the P-frame recovery packetized data of the hybrid FEC decoding module 12;
the framing module 13 deletes the lost sub-packets of the I-frame lost packet list when receiving the I-frame recovery sub-packet data, checks the I-frame lost packet list, and if the I-frame has no packet loss, merges the I-frame sub-packet data to obtain I-frame data and sends the I-frame data to the video decoding display module 14;
the framing module 13 deletes the lost sub-packet of the P frame lost packet list when receiving the P frame recovery sub-packet data, and if the P frame has no packet loss, the I frame timestamp of the P frame finds the corresponding I frame, and other frames between the I frame and the timestamp of the P frame have no lost sub-packet, combines the P frame sub-packet data to obtain the P frame data and sends the P frame data to the video decoding display module 14;
the video decoding and displaying module 14 decodes and displays the I-frame data and the P-frame data generated by the framing module 13.
Claims (2)
1. A FEC correction system based on video frames, comprising: the system comprises a video acquisition coding module (1), a frame proportion module (2), a frame disassembling module (3), an I frame packaging module (4), an I frame slicing module (5), a P frame packaging module (6), an I frame FEC coding module (7), a mixed FEC coding module (8), a network mixed sending module (9), a network mixed receiving module (10), an I frame FEC decoding module (11), a mixed FEC decoding module (12), a framing module (13) and a video decoding display module (14);
the video acquisition and coding module (1) acquires a video, and codes the video into a video compressed stream through video compression, wherein the video compressed stream consists of a video frame group, the video frame group comprises an I frame and a plurality of P frames, the I frame can be independently decoded, the P frame depends on the previous frame for decoding, and the video acquisition and coding module (1) transmits the video compressed stream to the frame proportion module (2) and the frame disassembly module (3);
the frame proportion module (2) confirms the proportion K of I frames and P frames according to GOPs of a previous video frame group of the video compression stream, and sends the proportion K to the I frame slicing module (5);
the frame disassembling module (3) sends the I frame to the I frame packaging module (4) and sends the P frame to the P frame packaging module (6);
i frame packing module (4): receiving I frame data of a frame disassembling module (3), performing subpackage processing, subpackaging according to 1400 bytes for network transmission, and sending the subpackaged data to a network mixed sending module (9) and an I frame FEC encoding module (7);
i frame slicing module (5): receiving I frame subpackage data of an I frame packaging module (4), grouping the I frame subpackage data according to the proportion K of the I frame and the P frame to obtain K parts of fragment data, marking a serial number ki on each fragment, and sending the fragment data, the serial number and the proportion K to a hybrid FEC encoding module (8);
the P frame packing module (6) performs packing processing on P frame data for network transmission, sends the packed data to the network mixed sending module (9), the P frame packing module (6) marks the frame serial number ki on the packed data of each P frame, the frame serial number ki is a position mark of a video frame group, the P frame starts numbering from 1, and the P frame data and the frame serial number ki are sent to the mixed FEC encoding module (8);
the I frame FEC encoding module (7) receives the packetized data of the I frame packaging module (4) to perform FEC forward error correction encoding, and the specific flow is as follows:
the frame I data is larger, two-dimensional single matrix alternating-current coding is adopted, a matrix close to a square shape is used for reducing the number of the frame I fec data, the square value of N is just larger than the number of sub-packets, the Y axis is reduced by one, whether N (N-1) is larger than the number of the sub-packets or not is judged, if yes, the N (N-1) matrix is selected, otherwise, the N (N-N) matrix is selected, and the final matrix is set to be N M;
the data of the video sub-package are sequentially put into an N x M matrix from left to right from top to bottom, the data of each row of video sub-package are subjected to XOR according to 1400 bytes to finally obtain a row error correcting code, each column of video sub-package is subjected to XOR according to 1400 bytes to finally obtain a column error correcting code, if no video fragment data exists on the N x M matrix, the position is skipped, if the position is the last packet of the video sub-package, the XOR is carried out according to the packet length, and the row error correcting code, the column error correcting code and the I frame data are sent to a network mixed sending module (9);
the mixed FEC encoding module (8) performs mixed FEC encoding on the I frame and the P frame, and the specific flow is as follows:
the hybrid FEC encoding module (8) receives P frame sub-packet data, a serial number and a P frame length of a P frame packaging module (6), receives I frame fragment data, a serial number and a proportion K of an I frame slicing module (5), matches the I frame fragment data and the P frame sub-packet data according to the serial number, uses a single-row error correcting code, firstly carries out XOR calculation on the P frame sub-packet data to obtain f, uses f to carry out XOR calculation on the I frame fragment sub-packet to obtain an error correcting code of the P frame, and carries out XOR calculation on the actual length of the last packet of the P frame sub-packet data; the mixed FEC encoding module (8) sends the generated error correcting codes and the length of the P frames and the frame sequence number and the proportion K of the P frames to a network mixed sending module (9);
the network mixed sending module (9) sends a video RTP stream, a mixed FEC stream and an I frame FEC stream, and the specific flow is as follows:
the network mixed sending module (9) receives the packet data and the proportion K of the I frame packet module (4) and the P frame packet module (6), RTP is used for carrying out packet transmission, seq serial numbers of RTP are gradually increased one by one, a timestamp of RTP is gradually increased, and PT of independent RTP is used as a video RTP stream to be sent to the network mixed receiving module (10);
the network hybrid sending module (9) receives the line error correction code and the column error correction code of the I frame FEC encoding module (7), the size of the I frame data and the frame number of the P frame, carries out sequential numbering x on the line error correction code and the column error correction code, uses RTP to carry out packing transmission, the timestamp of the RTP and the timestamp of the RTP corresponding to the video frame are used for associating the error correction code with the I frame at a receiving end, the load of the RTP stores error correction code data, the sequential numbering x of the error correction code, the length of the I frame data and the sequence number of a first packet seq of the I frame are placed in the filling information of the RTP, and the PT of the independent RTP is used as an I frame FEC stream to be sent to the network hybrid receiving module (10);
a network mixed sending module (9) receives the error correcting code and the length of the P frame of the mixed FEC encoding module (8), RTP is used for packaging and transmitting, the timestamp of RTP is matched with the RTP timestamp of a corresponding video frame and is used for correlating the error correcting code and the P frame at a receiving end, the load of RTP is used for placing error correcting code data, the filling information of RTP is used for placing the data length of the P frame, the seq serial number of the first packet of the P frame, the frame serial number of the P frame, the timestamp of the I frame and the proportion K of the first packet of the P frame, and the filling information of RTP is used as an independent RTP stream to be sent to the network mixed receiving module (10), and the PT of the independent RTP is used as a mixed FEC stream to be sent to the network mixed receiving module (10);
the network hybrid receiving module (10) receives the video RTP stream, the I frame FEC stream and the hybrid FEC stream of the network hybrid sending module (9), groups the streams according to the RTP timestamp, obtains the load data of the video RTP stream, obtains the frame subpackage data, and sends the frame subpackage data to the framing module (13), and the specific flow is as follows:
judging whether the packet FEC packet is an I-frame FEC stream or a mixed FEC stream, if the packet FEC packet is the I-frame FEC stream, extracting a sequence number x, an I-frame data length and an I-frame first packet seq number according to RTP filling information of the I-frame FEC stream, restoring row error correction code and column error correction code from loaded row error correction code and column error correction code of RTP of the I-frame FEC stream according to the sequence number, sending the row error correction code, the column error correction code and an I-frame timestamp to an I-frame FEC decoding module (11), calculating an I-frame packet number P according to the I-frame data length and the I-frame first packet seq number, wherein the I-frame packet number P is the I-frame data length/1400 +1, if the I-frame data length is zero, the packet number is decreased by one, the last packet length is 1400, and if the I-frame data length is nonzero, the last packet length is a remainder;
the method comprises the steps of obtaining seq sequence numbers of all RTP packets of an I frame according to the seq sequence number of a first packet of the I frame and the number of the I frame, searching a lost packet list according to the seq sequence numbers and RTP streams of the I frame of the packet, obtaining I frame sub-packet data according to the load of the video RTP stream, and sending the I frame sub-packet data, the last sub-packet length, the lost packet list and an I frame timestamp to an I frame FEC decoding module (11), a hybrid FEC decoding module (12) and a framing module (13);
if the mixed FEC flow is adopted, the data length of P frame, the sequence number of the first packet seq of the P frame, the frame sequence number of the P frame and the time stamp and proportion K of the I frame are taken out according to the RTP filling information of the mixed FEC flow, the error correction code data of the load of the RTP of the mixed FEC flow is taken out, and the error correction code data and the time stamp of the I frame are sent to a mixed FEC decoding module (12); calculating the P frame packet number P according to the P frame data length and the first packet seq serial number of the P frame, wherein the P frame packet number P is equal to the P frame data length/1400 +1, if the P frame data length is equal to the remainder 1400, the packet number is reduced by one, the last packet length is 1400, if the P frame data length is equal to the remainder 1400 and is not zero, and the last packet length is the remainder;
the seq sequence numbers of all RTP packets of the P frame are obtained according to the seq sequence number of the first packet of the P frame and the number of the P frame, a lost packet list is searched for by the seq sequence numbers and the RTP stream of the P frame of the packet, P frame sub-packet data is obtained according to the load of the video RTP stream, and the P frame sub-packet data, the last sub-packet length, the lost packet list, the P frame time stamp, the frame sequence number of the P frame, the I frame time stamp and the proportion K of the video frame group are sent to a hybrid FEC decoding module (12) and a framing module (13);
an I frame FEC decoding module (11) recovers lost packets of I frames, and the specific flow is as follows:
receiving a row error correction code, a column error correction code and an I frame timestamp of a network hybrid receiving module (10), the number of I frame packet data and I frame packet, the last packet length, a lost packet list and the I frame timestamp, and associating the row error correction code, the column error correction code, the I frame packet data and the lost packet list according to the I frame timestamp;
the number of the sub-packets generates an N x M matrix by using an I-frame FEC encoding module (7), if only one packet is lost in a row/column of a lost packet list matrix, the other sub-packet data in the row/column and the row/column error correction code are subjected to XOR calculation to obtain the sub-packet data in the position, if the last sub-packet is obtained, the XOR calculation is carried out according to the sub-packet length, the sub-packet data is sent to a framing module (13) and a hybrid FEC decoding module (12), and then the I-frame sub-packet data is used for carrying out secondary reduction on an I-frame FEC decoding module (11);
the I frame FEC decoding module (11) receives the recovery packets of the hybrid FEC decoding module (12) to carry out secondary recovery;
the I frame FEC decoding module (11) recovers the sub-packet secondary reduction, deletes the I frame sub-packet in a lost packet list, checks whether a packet loss exists in a column and a row of the I frame sub-packet, if only one packet loss exists, performs XOR calculation on other sub-packet data of the row and the column error correction code to obtain sub-packet data of the position, sends the sub-packet data to the framing module (13) and the hybrid FEC decoding module (12), and then uses the I frame sub-packet data to perform secondary reduction on the I frame FEC decoding module (11);
the packet loss recovery capability is mutually enhanced through an I frame FEC decoding module (11) and a hybrid FEC decoding module (12);
the mixed FEC decoding module (12) recovers the lost sub-packets of the I frames and the lost sub-packets of the P frames, and the specific flow is as follows:
a hybrid FEC decoding module (12) receives P frame packet data and the last packet length of a network hybrid receiving module (10), a lost packet list, a P frame timestamp, a frame sequence number of a P frame, an I frame timestamp and a proportion K;
the hybrid FEC decoding module (12) receives error correcting code data and P frame time stamps of the network hybrid receiving module (10), and receives the number of I frame packet data and I frame packets, the last packet length, a lost packet list and the I frame time stamps of the network hybrid receiving module (10);
the hybrid FEC decoding module (12) acquires a corresponding I frame according to the I frame timestamp of the video frame group of the P frame, equally divides the number of I frame sub-packets according to the proportion K to obtain a K sub-packet list, acquires I frame sub-packet data corresponding to the P frame according to the frame number of the P frame as a sub-packet list subscript, checks whether the sub-packet in the I frame lost packet list loses a packet or not, if the sub-packet does not lose the packet and the lost packet list of the P frame loses one packet, performs XOR on the sub-packet data of other sub-packet data of the P frame and the I frame sub-packet and error correction code data to obtain the packet loss of the P frame, and sends a recovery data packet and the timestamp of the P frame to the framing module (13); if the lost packet list of the P frame has no packet loss and the lost packet list of the I frame fragment has a packet loss, carrying out exclusive OR on P frame sub-packet data and other sub-packet data of the I frame fragment and error correction code data to obtain recovery sub-packet data of the I frame, and sending the sub-packet data and the timestamp of the I frame to an I frame FEC decoding module (11) and a framing module (13);
the hybrid FEC decoding module (12) obtains I frame recovery sub-packet data and an I frame timestamp of the I frame FEC decoding module (11), finds a corresponding I frame according to the I frame timestamp, deletes sub-packets from an I frame lost packet list, corresponds to sub-packets in a K fragment list by the recovery sub-packet data, judges whether the fragments have packet loss, matches the frame number of a P frame according to the position offset of the fragments if the fragments have no packet loss, finds a corresponding P frame, performs XOR on the sub-packet data of other sub-packet data of the P frame and the I frame fragment and error correction code data to obtain the packet loss of the P frame if the P frame has only lost one packet, and sends the recovery data packet of the P frame and the P frame timestamp to the framing module (13);
the framing module (13) generates I-frame and P-frame data, and the specific flow is as follows:
the framing module (13) receives I frame sub-packet data, the last sub-packet length, a lost packet list and an I frame time stamp of the network hybrid receiving module (10), and receives P frame sub-packet data, the last sub-packet length, the lost packet list, a P frame time stamp and the I frame time stamp;
the framing module (13) receives the I frame recovery packet data of the I frame FEC decoding module (11), and receives the I frame recovery packet data and the P frame recovery packet data of the mixed FEC decoding module (12);
the framing module (13) deletes the lost sub-packets of the I frame lost packet list when receiving the I frame recovery sub-packet data, checks the I frame lost packet list, and if the I frame does not have lost packets, merges the I frame sub-packet data to obtain I frame data and sends the I frame data to the video decoding display module (14);
the framing module (13) deletes lost sub-packets of the P frame lost packet list when receiving P frame recovery sub-packet data, if the P frame has no packet loss, and finds out a corresponding I frame according to the I frame time stamp of the P frame, and other frames between the I frame and the time stamp of the P frame have no lost sub-packets, then combines the P frame sub-packet data to obtain P frame data and sends the P frame data to the video decoding display module (14);
and the video decoding and displaying module (14) decodes and displays the I frame and the P frame data generated by the framing module (13).
2. A video frame based FEC correction system according to claim 1, wherein: the ratio K is a positive integer, i of the frame number ki is a positive integer ranging from 1 to K, the sequence number x is a positive integer, and the PT represents Payload type.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011160364.8A CN112333470B (en) | 2020-10-27 | 2020-10-27 | FEC (forward error correction) system based on video frame |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011160364.8A CN112333470B (en) | 2020-10-27 | 2020-10-27 | FEC (forward error correction) system based on video frame |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112333470A CN112333470A (en) | 2021-02-05 |
| CN112333470B true CN112333470B (en) | 2022-05-27 |
Family
ID=74311987
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011160364.8A Active CN112333470B (en) | 2020-10-27 | 2020-10-27 | FEC (forward error correction) system based on video frame |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112333470B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114554198B (en) * | 2022-04-27 | 2022-08-26 | 广州番禺职业技术学院 | Video key frame redundancy transmission method and system based on erasure codes |
| CN114944890B (en) * | 2022-07-21 | 2022-10-25 | 北京全路通信信号研究设计院集团有限公司 | Transponder message transmission processing method and system |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6490705B1 (en) * | 1998-10-22 | 2002-12-03 | Lucent Technologies Inc. | Method and apparatus for receiving MPEG video over the internet |
| WO2004079982A1 (en) * | 2003-03-05 | 2004-09-16 | Nokia Corporation | Method and system for forward error correction |
| CN1741435A (en) * | 2004-08-24 | 2006-03-01 | 华为技术有限公司 | Method for recovering drop-out data unit |
| CN101166273A (en) * | 2006-10-16 | 2008-04-23 | 华为技术有限公司 | Method, device and system for transmitting video data |
| CN101212280A (en) * | 2006-12-25 | 2008-07-02 | 索尼株式会社 | Data communication system, data transmitting apparatus, data transmitting method, and method for determining packet size and redundancy |
| CN101854224A (en) * | 2009-04-01 | 2010-10-06 | 华为技术有限公司 | Error correcting coding method, device and system, and forwarding control method and device |
| CN102143367A (en) * | 2010-01-30 | 2011-08-03 | 华为技术有限公司 | Method, device and system for checking error correction |
| CN107592959A (en) * | 2015-03-02 | 2018-01-16 | 三星电子株式会社 | Transmitter and its reduction method |
| CN108174234A (en) * | 2018-01-12 | 2018-06-15 | 珠海全志科技股份有限公司 | A kind of flow-medium transmission method and system |
| US10034023B1 (en) * | 2012-07-30 | 2018-07-24 | Google Llc | Extended protection of digital video streams |
| CN108429921A (en) * | 2017-02-14 | 2018-08-21 | 北京金山云网络技术有限公司 | A kind of video coding-decoding method and device |
| CN110943800A (en) * | 2019-11-06 | 2020-03-31 | 厦门亿联网络技术股份有限公司 | Method, device and system for sending data packet, storage medium and electronic device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7539187B2 (en) * | 2004-07-07 | 2009-05-26 | Qvidium Technologies, Inc. | System and method for low-latency content-sensitive forward error correction |
| CN101512989B (en) * | 2006-07-25 | 2013-08-14 | 汤姆森特许公司 | Recovery of outburst packet lose in wireless network based on Internet protocol using interleaving play and crossover packet forward error correction |
| US7746882B2 (en) * | 2006-08-22 | 2010-06-29 | Nokia Corporation | Method and device for assembling forward error correction frames in multimedia streaming |
-
2020
- 2020-10-27 CN CN202011160364.8A patent/CN112333470B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6490705B1 (en) * | 1998-10-22 | 2002-12-03 | Lucent Technologies Inc. | Method and apparatus for receiving MPEG video over the internet |
| WO2004079982A1 (en) * | 2003-03-05 | 2004-09-16 | Nokia Corporation | Method and system for forward error correction |
| CN1741435A (en) * | 2004-08-24 | 2006-03-01 | 华为技术有限公司 | Method for recovering drop-out data unit |
| CN101166273A (en) * | 2006-10-16 | 2008-04-23 | 华为技术有限公司 | Method, device and system for transmitting video data |
| CN101212280A (en) * | 2006-12-25 | 2008-07-02 | 索尼株式会社 | Data communication system, data transmitting apparatus, data transmitting method, and method for determining packet size and redundancy |
| CN101854224A (en) * | 2009-04-01 | 2010-10-06 | 华为技术有限公司 | Error correcting coding method, device and system, and forwarding control method and device |
| CN102143367A (en) * | 2010-01-30 | 2011-08-03 | 华为技术有限公司 | Method, device and system for checking error correction |
| US10034023B1 (en) * | 2012-07-30 | 2018-07-24 | Google Llc | Extended protection of digital video streams |
| CN107592959A (en) * | 2015-03-02 | 2018-01-16 | 三星电子株式会社 | Transmitter and its reduction method |
| CN108429921A (en) * | 2017-02-14 | 2018-08-21 | 北京金山云网络技术有限公司 | A kind of video coding-decoding method and device |
| CN108174234A (en) * | 2018-01-12 | 2018-06-15 | 珠海全志科技股份有限公司 | A kind of flow-medium transmission method and system |
| CN110943800A (en) * | 2019-11-06 | 2020-03-31 | 厦门亿联网络技术股份有限公司 | Method, device and system for sending data packet, storage medium and electronic device |
Non-Patent Citations (1)
| Title |
|---|
| 基于RTP协议的网络视频监控系统的实现;裘英等;《微计算机应用》;20060715(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112333470A (en) | 2021-02-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2103026B1 (en) | A method to support forward error correction for real-time audio and video data over internet protocol networks | |
| CN102143367B (en) | Method, device and system for checking error correction | |
| CN112333470B (en) | FEC (forward error correction) system based on video frame | |
| EP2437421B1 (en) | Method, device and communication system for retransmitting based on forward error correction | |
| KR101829923B1 (en) | Apparatus and method for encoding in data communication system | |
| US8718098B2 (en) | Method for compressing and decompressing time stamp and equipment thereof | |
| CN101171806B (en) | Method and apparatus for transmitting and receiving packet data using predefined length indicant in mobile communication system | |
| KR101933465B1 (en) | Apparatus and method for transmitting/receiving a packet in a mobile communication system | |
| EP2362653A1 (en) | Transport stream packet header compression | |
| CN104247436A (en) | Apparatus and method for transmitting and receiving signaling information in a digital broadcasting system | |
| CN101834700A (en) | Unidirectional reliable transmission method and transceiving device based on data packets | |
| CA2477389A1 (en) | Transmitting streams over asynchronous networks | |
| CN101166273A (en) | Method, device and system for transmitting video data | |
| CN101453463B (en) | Data packet packing, transmitting and receiving method for mobile multimedia broadcast system | |
| CN106453160A (en) | Beidou data transmission method and system | |
| CN102665140B (en) | RTP (real-time transport protocol) packaging method of AVS (audio video coding standard) video frame | |
| CN101080925B (en) | Apparatus and method for demultiplexing in a digital broadcasting receiver | |
| CN113810769A (en) | Data transmission method and system | |
| EP2842253B1 (en) | Apparatus and method for transmitting a packet in a communication system | |
| CN110099036B (en) | Data encapsulation method based on fusion transmission system | |
| CN111385058A (en) | Data transmission method and device | |
| CN115604481B (en) | Method, device and system for improving parallelism of encoding and decoding and transmission | |
| CN101453465A (en) | IP packet compression, decompression method for mobile multimedia broadcast system | |
| CN101426005B (en) | IP data packet packaging, sending and receiving method for wireless broadcast system | |
| US20090196307A1 (en) | Transmitter apparatus, receiver apparatus, moving image and audio transmission evaluation method, and computer readable storage medium storing program thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |