CN102404568A - P-frame interpolating method aimed at whole frame loss error concealment in video coding and decoding - Google Patents

P-frame interpolating method aimed at whole frame loss error concealment in video coding and decoding Download PDF

Info

Publication number
CN102404568A
CN102404568A CN2011103448270A CN201110344827A CN102404568A CN 102404568 A CN102404568 A CN 102404568A CN 2011103448270 A CN2011103448270 A CN 2011103448270A CN 201110344827 A CN201110344827 A CN 201110344827A CN 102404568 A CN102404568 A CN 102404568A
Authority
CN
China
Prior art keywords
frame
motion vector
vectors
motion
vector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN2011103448270A
Other languages
Chinese (zh)
Inventor
郑世宝
杨华
孙豪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiaotong University
Original Assignee
Shanghai Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiaotong University filed Critical Shanghai Jiaotong University
Priority to CN2011103448270A priority Critical patent/CN102404568A/en
Publication of CN102404568A publication Critical patent/CN102404568A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention discloses a P-frame interpolating method aimed at whole frame loss error concealment in video coding and decoding, comprising: modifying the decoder working flow, acquiring all motion vectors of the next frame before concealing the current lost frame; acquiring residual information of corresponding block, eliminating unreliable motion vectors in all motion vectors of the last frame and the next frame, obtaining available motion vectors; by mapping the available motion vectors, establishing two reference motion vector tables of 4*4 blocks concealed in the lost frame to be respectively corresponding to the last frame and the next frame; according to the field consistence judging method, screening the motion vector with the least difference with the likelihood motion vector in the last frame and the next frame, obtaining the motion vector of the present block of the interpolated frame from the two optimal motion vectors, and completing the interpolation referring to the last frame and with zero residual.

Description

Be directed against the P frame interpolation method of whole LOF code error shielding in the coding and decoding video
Technical field
The present invention relates to the P frame interpositioning of decoding end in the coding and decoding video, specifically is to the P frame interpolation method of putting in order the LOF code error shielding in a kind of coding and decoding video.
Background technology
At present, adopted coding techniques efficiently, can realize very high compression ratio with the video encoding standard of new generation that H.264/AVC is encoded to example.Thereby it is also used comprising in the middle of the various video application scenarios such as internet video Streaming Media, 3G mobile TV widely.To video transmission and the coding standard that proposes, it inevitably runs into the corrupted data in the transmission course or loses as a kind of, when video flowing packing transmission, and this damage and lose and be directly reflected as losing of packet.Because the high compression ratio of video coding; Make it face the especially fragility that seems under the situation of video packet loss: the loss of data in the middle of a certain frame; Not only can have influence on the normal demonstration of present frame, and mistake is spread on time domain, make subsequent frame mistake also occur.So in the face of the packet loss in the video transmission time, the particular importance that just seems of the error concealment technology in the decoder, it can reduce the visual impact that mistake is brought to greatest extent, hinders wrong diffusion on time domain simultaneously.
Packet loss generally speaking can make part macro block or the part band in the frame lose, but compares under the serious situation at packet loss, losing of whole frame data can occur.On the other hand; The Channel Transmission standard of some low code checks (300-600kbps) (such as the T-DMB mobile TV standard of widely using in European and Korea S); Tend to beat a frame in the H.264 video of CIF specification in a RTP bag, the situation of whole LOF will take place when like this packet loss taking place.
Literature search through to prior art is found; Chinese patent, publication number CN1455596, patent name are " spreading to the video error coverage method of motion vector sought based on motion vector outward "; This patent is to whole LOF problem; Utilize the method for motion vector mapping (perhaps being called extrapolation), choose motion vector according to the overlapping area size, and then the reconstructing lost piece.The problem that this method exists is that at first, when choosing motion vector, the order of accuarcy of motion vector might not be directly proportional with the overlapping area after the mapping, secondly, does not have to utilize the information of back one frame to cover quality with further raising.
Summary of the invention
The objective of the invention is to overcome the weak point of above-mentioned prior art; Proposed in a kind of coding and decoding video to the P frame interpolation method of putting in order the LOF code error shielding; Be a kind of two-way whole LOF error concealment technology, this technology can be utilized the information of back one frame further to improve and cover quality.
The present invention realizes through following technical scheme:
To the P frame interpolation method of whole LOF code error shielding, comprise the steps: in the coding and decoding video of the present invention
Step 1 is revised the decoder functions flow process, before covering lost frames, has added a module to obtain the motion vector of next frame;
Step 2 for the motion vector information of former frame with back one frame, is obtained the residual information of its corresponding blocks, removes a part according to residual information and does not have a motion vector of reliability, and the residual movement vector is defined as the available motion vector, and its corresponding blocks is defined as block available;
Step 3 according to the available motion Vector Message of former frame with back one frame, utilizes the motion vector reflection method that all available motion vectors are mapped on the lost frames, sets up the reference motion vector tabulation of the current 4x4 piece of covering in the lost frames;
Step 4; Filter out the optimum movement vector of former frame and back one frame respectively according to field consistency decision method; From two optimum movement vectors, draw the motion vector of current block in the interpolation frame with linear interpolation method, serving as to insert in zero mode is accomplished with reference to former frame and residual error.
Said step 1; Be specially: in the flow process of decoding, before covering lost frames, added a module to obtain the motion vector of next frame; Call the interface function of covering lost frames then, the frame that will cover completion at last inserted output buffers and exports waiting before next frame.
Said step 2; Wherein removing a part of motion vector of not having reliability according to residual information is specially: the residual information that utilizes motion vector place piece; Judgement respective motion vectors reliability in the method; If the residual error of this motion vector place piece thinks that greater than a predetermined threshold value motion vector of this piece does not possess reliability in the method, from the available motion set of vectors, remove.
Said step 3; Be specially: the method through the motion vector mapping is mapped to all available motion vectors on the lost frames; Find out in former frame and back one frame with lost frames in the motion vector of the 4x4 piece that overlaps of the current 4x4 of covering piece, former frame is set up reference motion vector respectively with back one frame tabulates.
Said step 4; Be specially: for some current blocks, be mapped on the former frame, judge the difference of this motion vector and its likelihood motion vector according to field consistency decision method according to the some motion vectors in the reference motion vector tabulation of former frame; Travel through the motion vector in all reference motion vector tabulations; Find the minimum motion vector of difference,, find the optimum movement vector in one frame of back with quadrat method as the optimum movement vector in the former frame; From two optimum movement vectors, draw the motion vector of current block in the interpolation frame then with linear interpolation method, serving as to insert in zero mode is accomplished with reference to former frame and residual error.
To sum up, in the technical scheme of the present invention, at first revise the decoder functions flow process; Before covering current lost frames; Obtain all motion vectors of next frame, utilize the motion vector reflection method to obtain the reference motion vector tabulation then, filter out the optimum movement vector of former frame and back one frame at last according to field consistency decision method respectively; From two optimum movement vectors, draw the motion vector of current block in the interpolation frame with linear interpolation method, serving as to insert in zero mode is accomplished with reference to former frame and residual error.The present invention can utilize the information of back one frame to cover quality with further raising, and the average output Y-PSNR (PSNR) in each sequence all is superior to other conventional methods, has better visual effect.
Description of drawings
Fig. 1 is the inventive method flow chart;
Adopt the former frame motion vector mapping method sketch map of lost frames among Fig. 2 the present invention;
Fig. 3 midfield of the present invention consistency decision method sketch map;
In Fig. 4 specific embodiment of the invention, the PSNR parameter comparison diagram of each frame of table sequence;
In Fig. 5 specific embodiment of the invention, the output image subjective quality comparison diagram of table sequence the whole bag of tricks the 50th frame.
Embodiment
Below in conjunction with accompanying drawing embodiments of the invention are elaborated: present embodiment provided detailed execution mode and concrete operating process, but protection scope of the present invention is not limited to following embodiment being to implement under the prerequisite with technical scheme of the present invention.
As shown in Figure 1, be the inventive method flow chart, the method detailed process is:
Step 1 is revised the decoder functions flow process, before covering current lost frames, obtains all motion vectors of next frame;
Being in the middle of the decoding process of example with standard H.264, when certain P LOF, the subsequent P frame since the dependence on the time domain then can't be solved.So based on the method for traditional decoding process, for covering of certain P frame, can only be with reference to the information of previous frame, such as motion vector or brightness and colourity.In fact, decoder is in when work, and and if only if when decoder receives the next frame of lost frames, because the frame number of two consecutive frames is discontinuous, decoder just can detect losing of former frame, and then call frame is covered the interface of function.And this moment, in fact decoder has received the motion vector and the residual information of a frame behind the lost frames, has only lost just and can't decode owing to former frame.So can obtain the motion vector field of a frame behind the lost frames,, cover lost frames in conjunction with the information such as motion vector field of lost frames former frame.After the decoding process of revising H.264,, except the information that can obtain previous frame, can also obtain the partial information of back one frame, to be used for covering of lost frames for lost frames.Concrete implementation method is, before covering lost frames, added a module to obtain the motion vector of next frame, calls the interface function of covering lost frames then, and the frame that will cover completion at last inserted output buffers and exports waiting before next frame.
Step 2; For the motion vector information of former frame with back one frame, obtain the residual information of its corresponding blocks, remove a part according to residual information and do not have the motion vector of reliability in the method; The residual movement vector is defined as the available motion vector, and its corresponding blocks is defined as block available.
Utilize the residual information of motion vector place piece, judgement respective motion vectors reliability in the method.With the former frame is example; Adjudicate its size for the residual error
Figure BDA0000105551580000042
of
Figure BDA0000105551580000041
individual piece; If
Figure BDA0000105551580000043
thinks that the motion vector of this piece does not possess reliability in the method, removes from the available motion set of vectors.Wherein Th is the threshold value of residual energy, chooses according to the experiment effect of concrete video sequence.
Step 3 according to the available motion vector of former frame with back one frame, utilizes the motion vector reflection method that all available motion vectors are mapped on the lost frames, obtains the reference motion vector tabulation of the current 4x4 piece of covering in the lost frames;
Fundamental method is the mapping and the extrapolation of motion vector, is example with the available motion vector of former frame, and is as shown in Figure 2, supposes current n P LOF, finds its former frame n-1.Block available
Figure BDA0000105551580000044
to i 4x4 on the n-1 frame is reverse with its motion vector, is mapped to along reverse motion vector that to produce its motion vector of piece on the n frame identical with the motion vector of
Figure BDA0000105551580000046
.After so being mapped to the piece on all n-1 frames on the n frame, each the 4x4 piece on the n frame is inevitable to be covered by several mapping blocks.For j piece on the n frame, it by the area of i mapping block covering on the n-1 frame is
Figure BDA0000105551580000047
Wherein f i ( p ) = 1 p ∈ EB n i 0 p ∉ EB n i . For j piece on the n frame, it is satisfied
Figure BDA0000105551580000049
All i of condition #Individual piece covers.Can in the middle of all available motion vectors, obtain these motion vectors
Figure BDA0000105551580000051
that cover piece for j piece, k all qualified reference motion vectors
Figure BDA0000105551580000052
are formed its reference motion vector tabulation
Figure BDA0000105551580000053
Step 4; Filter out the optimum movement vector of former frame and back one frame respectively according to field consistency decision method; From two optimum movement vectors, draw the motion vector of current block in the interpolation frame with linear interpolation method, serving as to insert in zero mode is accomplished with reference to former frame and residual error;
Be example equally with the former frame; For j piece on the lost frames, the current block j in the lost frames is mapped on the former frame n-1 for the direction of each reference vector in the reference listing
Figure BDA0000105551580000054
according to this vector.Usually; It will cover 4 pieces:
Figure BDA0000105551580000055
and
Figure BDA0000105551580000056
current block j be respectively w1 with their coincidence area; W2, w3 and w4.As shown in Figure 3; For the part that covers; The likelihood motion vector
Figure BDA0000105551580000057
that estimates this part according to the weighted average of motion vector is noticed current
Figure BDA0000105551580000058
consistency according to the field;
Figure BDA0000105551580000059
should be approaching with likelihood motion vector
Figure BDA00001055515800000510
, quality that can use consistency discrimination standard is judged
Figure BDA00001055515800000511
.Choose and the optimum movement vector of the minimum reference motion vector of the difference of likelihood motion vector
Figure BDA00001055515800000513
with poor
Figure BDA00001055515800000512
of Orthogonal Method definition vector as former frame; The choosing method of the optimum movement vector of back one frame is similar with it, promptly from its reference motion vector tabulation, chooses optimum movement vector.Adopt approach based on linear interpolation at last,, obtain final reconstructed motion vector according to the optimal reconstruction vector of former frame with back one frame MV Final ( B n j ) = α · MV Best Pre ( B n j ) + ( 1 - α ) · MV Best Next ( B n j ) , Choose α=0.5.
Embodiment
Based on above-mentioned description to Fig. 1 method flow, present embodiment adopts reference codec JM15.1 H.264 to carry out software emulation.The encoder section of this model can be implemented coding to the image sequence of YUV, the output code flow formatted file, its part of decoding can with compatibility H.264 the code stream decoding of standard become the YUV sequence.The RTP Packet Loss Simulation device that adopts JM model self to provide is implemented Packet Loss Simulation to the code stream sequence.
Present embodiment is at first revised the decoder functions flow process.When it is decoded, its cover lost frames before, added a module obtaining the motion vector of next frame, and then called the interface function of covering lost frames, the frame that will cover completion at last inserted output buffers and exports waiting before next frame.
With table, foreman, coastguard; Stephan; Test video sequence such as Paris and headwithglasses is utilized the JM encoder to compile to be ASCII stream file ASCII H.264, and its intermediate-resolution has adopted the CIF (352x288) that uses always in the mobile TV, and test condition is 30 frame per seconds; The cycle that GOP is made as IPPP and I frame is 15 frames, and QP is made as 28.Code stream is encapsulated in the RTP bag and with a frame and encloses in the packet with the whole LOF of emulation.The packet loss instrument that utilizes the JM model to carry is then implemented packet loss by 5% ratio to code stream, for wrong situation about on time domain, spreading after the better observation LOF, is limited in and only loses a frame among the GOP.
To packet loss sequential decoding, wherein decoder utilizes method of the present invention and some conventional methods automatically mistake to be implemented to cover.
Traditional method of covering the whole LOF of P frame comprises directly duplicates former frame method (FC); The above-mentioned bi-directional motion vector Extrapolation method (BME) that has had the motion vector Extrapolation method (MP) of patent and expanded a back frame motion vector; Add method of the present invention, four kinds of methods have been implemented to cover to the mistake of whole LOF altogether.
The output of each representative series through table 1 shows, and the average output Y-PSNR (PSNR) of the inventive method in each sequence all is superior to other conventional methods.
The present invention exports PSNR and all is superior to additive method in each sequence of table 1
Figure BDA0000105551580000061
Find out advantage of the present invention for clearer; Chosen the table sequence PSNR of four kinds of every frames of method has been carried out comparative analysis; Result relatively is as shown in Figure 4, notices after each generation LOF that the effect of covering of the present invention all will continue to lead over additive method on whole GOP.
Close injecting method at last and cover the subjective assessment of effect, as shown in Figure 5, observe the output performance under distinct methods of the 50th frame of table sequence, can find that in some thin portions the effect of covering of the present invention can obtain better visual effect.

Claims (5)

1. be directed against the P frame interpolation method of putting in order the LOF code error shielding in a coding and decoding video, it is characterized in that, comprise the steps:
Step 1 is revised the decoder functions flow process, before covering lost frames, has added a module to obtain the motion vector of next frame;
Step 2 for the motion vector information of former frame with back one frame, is obtained the residual information of its corresponding blocks, removes a part according to residual information and does not have a motion vector of reliability, and the residual movement vector is defined as the available motion vector, and its corresponding blocks is defined as block available;
Step 3 according to the available motion Vector Message of former frame with back one frame, utilizes the motion vector reflection method that all available motion vectors are mapped on the lost frames, sets up the reference motion vector tabulation of the current 4x4 piece of covering in the lost frames;
Step 4; Filter out the optimum movement vector of former frame and back one frame respectively according to field consistency decision method; From two optimum movement vectors, draw the motion vector of current block in the interpolation frame with linear interpolation method, serving as to insert in zero mode is accomplished with reference to former frame and residual error.
2. be directed against the P frame interpolation method of whole LOF code error shielding in the coding and decoding video according to claim 1; It is characterized in that said step 1 is specially: in the flow process of decoding; Before covering lost frames; Added a module to obtain the motion vector of next frame, called the interface function of covering lost frames then, the frame that will cover completion at last inserted output buffers and exports waiting before next frame.
3. be directed against the P frame interpolation method of whole LOF code error shielding in the coding and decoding video according to claim 1; It is characterized in that; Said step 2; Wherein removing a part of motion vector of not having reliability according to residual information is specially: utilize the residual information of motion vector place piece, judgement respective motion vectors reliability in the method is if the residual error of this motion vector place piece is greater than a predetermined threshold value; Think that the motion vector of this piece does not possess reliability in the method, removes from the available motion set of vectors.
4. be directed against the P frame interpolation method of whole LOF code error shielding in the coding and decoding video according to claim 1; It is characterized in that; Said step 3; Be specially: the method through motion vector mapping is mapped to all available motion vectors on the lost frames, find out in former frame and back one frame with lost frames in the motion vector of the 4x4 piece that overlaps of the current 4x4 of covering piece, former frame is set up reference motion vector respectively with back one frame tabulates.
5. be directed against the P frame interpolation method of whole LOF code error shielding in the coding and decoding video according to claim 1; It is characterized in that said step 4 is specially: for some current blocks; Some motion vectors according in the reference motion vector tabulation of former frame are mapped on the former frame; According to the difference of this motion vector of field consistency decision method judgement and its likelihood motion vector, travel through the motion vector in all reference motion vectors tabulations, find the minimum motion vector of difference; As the optimum movement vector in the former frame; Find the optimum movement vector of back in one frame with quadrat method, from two optimum movement vectors, draw the motion vector of current block in the interpolation frame then with linear interpolation method, serving as to insert in zero mode is accomplished with reference to former frame and residual error.
CN2011103448270A 2011-08-18 2011-11-04 P-frame interpolating method aimed at whole frame loss error concealment in video coding and decoding Pending CN102404568A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2011103448270A CN102404568A (en) 2011-08-18 2011-11-04 P-frame interpolating method aimed at whole frame loss error concealment in video coding and decoding

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
CN201110237139.4 2011-08-18
CN201110237139 2011-08-18
CN2011103448270A CN102404568A (en) 2011-08-18 2011-11-04 P-frame interpolating method aimed at whole frame loss error concealment in video coding and decoding

Publications (1)

Publication Number Publication Date
CN102404568A true CN102404568A (en) 2012-04-04

Family

ID=45886288

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2011103448270A Pending CN102404568A (en) 2011-08-18 2011-11-04 P-frame interpolating method aimed at whole frame loss error concealment in video coding and decoding

Country Status (1)

Country Link
CN (1) CN102404568A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102752602A (en) * 2012-06-19 2012-10-24 清华大学 Video sequence loss frame recovery method and device
CN103905825A (en) * 2012-12-27 2014-07-02 中国电信股份有限公司 Motion compensation method and apparatus of video image
CN104918052A (en) * 2014-03-14 2015-09-16 想象技术有限公司 Error tracking and mitigation for motion compensation-based video compression
CN107027029A (en) * 2017-03-01 2017-08-08 四川大学 High-performance video coding improved method based on frame rate conversion

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101175217A (en) * 2007-11-12 2008-05-07 昊迪移通(北京)技术有限公司 Code error shielding technology aiming at mobile network video transmission
CN101227619A (en) * 2008-01-29 2008-07-23 华为技术有限公司 Method for realizing image error covering in video decipher and video decoder

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101175217A (en) * 2007-11-12 2008-05-07 昊迪移通(北京)技术有限公司 Code error shielding technology aiming at mobile network video transmission
CN101227619A (en) * 2008-01-29 2008-07-23 华为技术有限公司 Method for realizing image error covering in video decipher and video decoder

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HAO SUN,ETC.: "An Efficient Frame Loss Error Concealment Scheme Based on Tentative Projection for H.264/AVC", 《SPRINGER-VERLAG BERLIN HEIDELBERG 2010》 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102752602A (en) * 2012-06-19 2012-10-24 清华大学 Video sequence loss frame recovery method and device
CN102752602B (en) * 2012-06-19 2014-11-26 清华大学 Video sequence loss frame recovery method and device
CN103905825A (en) * 2012-12-27 2014-07-02 中国电信股份有限公司 Motion compensation method and apparatus of video image
CN103905825B (en) * 2012-12-27 2018-02-13 中国电信股份有限公司 The motion compensation process and device of video image
CN104918052A (en) * 2014-03-14 2015-09-16 想象技术有限公司 Error tracking and mitigation for motion compensation-based video compression
CN107027029A (en) * 2017-03-01 2017-08-08 四川大学 High-performance video coding improved method based on frame rate conversion
CN107027029B (en) * 2017-03-01 2020-01-10 四川大学 High-performance video coding improvement method based on frame rate conversion

Similar Documents

Publication Publication Date Title
US20070211800A1 (en) Method and Apparatus for Frame Rate Up Conversion with Multiple Reference Frames and Variable Block Sizes
US20090279615A1 (en) Error concealment for frame loss in multiple description coding
TW201811024A (en) Method and apparatus for selective filtering of cubic-face frames
TWI647946B (en) Image encoding and decoding methods and devices thereof
CN102404568A (en) P-frame interpolating method aimed at whole frame loss error concealment in video coding and decoding
RU2010127313A (en) SYSTEM AND METHOD FOR COMPRESSING INTERACTIVE STREAM VIDEO
KR960703481A (en) Method and device for encoding image signal and image signal decoding device
CN105306945A (en) Scalable synopsis coding method and device for monitor video
CN104410864B (en) Error concealing method based on residual energy in HEVC
CN101355708B (en) Self-adapting method for shielding error code
CN101163250B (en) Boundary gradient based video stream fault tolerance method
CN102026001B (en) Method for evaluating importance of video frame based on motion information
CN112422989B (en) Video coding method
CN106534871B (en) The coding method of Video Codec and coding/decoding method
US20110222603A1 (en) Method and System Making It Possible to Protect A Compressed Video Stream Against Errors Arising During a Transmission
CN101242540B (en) Error control method for video decoder chip
CN103139571A (en) Video fault-tolerant error-resisting method based on combination of forward error correction (FEC) and WZ encoding and decoding
PH12021550833A1 (en) Image coding/decoding method, coder, decoder, and storage medium
CN112702602A (en) Video coding and decoding method and storage medium
CN115943422A (en) Video processing method, device, equipment, decoder, system and storage medium
US20130230105A1 (en) Intra-prediction encoding/decoding apparatus and method for omitting the transmission of prediction mode information using characteristics of reference pixels
CN102547282A (en) Extensible video coding error hiding method, decoder and system
CN104363454B (en) A kind of video coding and decoding method and system of high code rate image
CN103152581B (en) The error recovery method that a kind of spatial domain is multiple description coded
CN102665076B (en) Construction method for lapped transform post-filter

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20120404