CN101917628A - Whole-frame error concealment method based on adaptive block sizes - Google Patents
Whole-frame error concealment method based on adaptive block sizes Download PDFInfo
- Publication number
- CN101917628A CN101917628A CN 201010266739 CN201010266739A CN101917628A CN 101917628 A CN101917628 A CN 101917628A CN 201010266739 CN201010266739 CN 201010266739 CN 201010266739 A CN201010266739 A CN 201010266739A CN 101917628 A CN101917628 A CN 101917628A
- Authority
- CN
- China
- Prior art keywords
- piece
- losing
- motion vector
- frame
- overlapping region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a whole-frame error concealment method based on adaptive block sizes, comprising the following steps: selecting the forward or backward motion vector extrapolation direction; selecting the size of 16*16 blocks or dividing the size of the lost macroblocks into the sizes of four 8*8 blocks to conceal the lost macroblocks or skipping the lost macroblocks; selecting the size of 8*8 blocks or dividing the size of the lost 8*8 subblocks into the sizes of four 4*4 blocks to conceal the lost 8*8 subblocks or skipping the lost 8*8 subblocks; selecting the size of 4*4 blocks to conceal the lost 4*4 subblocks or skipping the lost 4*4 subblocks; and adopting the external boundary matching algorithm to conceal the skipped lost macroblocks, the lost 8*8 subblocks or the lost 4*4 subblocks. The method can restore the videos with detailed motion characteristics and reduce the blocking effect when reconstructing the images.
Description
Technical field
The present invention relates to the video monitoring technology, particularly a kind of whole-frame error concealment method based on adaptive block sizes.
Background technology
Mobile video monitor transmits video content by mobile network and portable terminal for the cellphone subscriber, and the user can watch the monitoring place in real time by mobile phone.It not only possesses all functions of traditional monitoring trade, can also accomplish multiterminal video recording storage backup simultaneously, and real-time message is reported to the police, and the warning picture of capturing is sent on the user mobile phone immediately, is convenient in time handle accident.
For mobile video monitor, because mobile network's usable bandwidth resources is limited, the picture dimension of frame of video is less during coding, and the Frame behind the coding usually is compressed in the individual data bag and sends.If this packet is lost, then mean losing of whole video frame video image in transmission course; Owing to be subjected to inter prediction error code diffusion influence, video quality will severe exacerbation during data-bag lost, and carry out the frame of video error concealment in decoding end and seem particularly important this moment.
Existing a kind of frame of video error concealment method is the frame copy method, and its data with the former frame corresponding blocks copy in the present frame, yet this method can produce bigger distortion for the moving region.
Existing another kind of frame of video error concealment method is the motion vector copy method, H.264 it incite somebody to action, and the motion vector of losing piece (macro block, 8x8 piece or 4x4 piece) of encoded video streams is assumed to the motion vector that the former frame correspondence is lost piece, recover the data lose by motion compensation, yet that this method is covered effect to the video sequence of non-uniform movement or irregular movement is relatively poor.
Existing another frame of video error concealment method is the motion vector Extrapolation method; being about to former frame loses piece one 8x8 piece and is extrapolated to lost frames according to its motion vector; add up all outer pushing blocks and lose the area of piece overlapping region; pass judgment on each outer pushing block and the degree of correlation of losing piece by the size that compares the overlapping region; selecting the motion vector of the outer pushing block of overlapping region maximum is the motion vector of losing piece; be compensated piece to realize error concealment; this method is based on this precondition proposition of adjacent video sequence motion continuity; yet this precondition of motion continuity is not always set up; when sports ground takes place sharply to change; this method usually can cause the partial pixel of losing in the piece to be covered by a plurality of outer pushing block; it is overlap problem; and partial pixel is not covered by any outer pushing block in addition; be occlusion issue, these all will influence the effect of covering of this motion vector Extrapolation method.
In addition, above-mentioned existing three kinds of frame of video error concealment methods adopt unified piece size to carry out error concealment to all video sequences are fixing, the image that causes rebuilding exists obvious blocking artifact maybe can't recover the shortcoming of careful motion, can not provide high objective quality and subjective vision effect for the user.
Therefore, be necessary to provide a kind of improved error concealment method to overcome the defective of prior art.
Summary of the invention
The purpose of this invention is to provide a kind of whole-frame error concealment method based on adaptive block sizes, can solve in the mobile video monitor when frame of video is lost, there is obvious blocking artifact in existing frame of video error concealment method reconstructed image and can't recovers the shortcoming of careful motion, can be the user higher objective quality and subjective vision effect is provided, and overcome the bigger distortion of problem first kind of existing method can produce to(for) the moving region, second kind of existing method covered the relatively poor problem of effect to the video sequence of non-uniform movement or irregular movement, and the overlap problem and the occlusion issue of the third existing method generation.
To achieve these goals, the invention provides a kind of whole-frame error concealment method based on adaptive block sizes, comprise the steps: that (1) is according to the former frame of lost frames and the type of coding of next frame, and all the outer pushing blocks of the 4x4 piece of the 4x4 piece of the 4x4 of described former frame piece and next frame described former frame when being extrapolated to described lost frames respectively and area and all the outer pushing blocks of described next frame and the area of described lost frames overlapping region of described lost frames overlapping region, select forward or backward movement vector extrapolation direction; (2) will be in former frame or next frame be in the adjacent neighborhood macro block in the macro block space of same position and be extrapolated to the described macro block of losing along the motion vector extrapolation direction of selecting with the macro block of losing of lost frames, obtain the macro block of extrapolating, according to extrapolation macro block and the area of described number of losing the macro block overlapping region, overlapping region and the close degree between the extrapolation macroblock motion vector, selection 16x16 piece size is covered or is that the described macro block of losing is covered or skipped to 4 8x8 piece sizes with the described macroblock partitions of losing.
In one embodiment of the invention, described step (1) is specially: (11) are intracoded frame, when next frame is inter-frame encoding frame, choose backward motion vector extrapolation direction when the former frame of lost frames; (12) former frame when lost frames is inter-frame encoding frame, when next frame is intracoded frame, chooses forward motion vector extrapolation direction; (13) former frame when lost frames be inter-frame encoding frame, when next frame is inter-frame encoding frame, respectively the 4x4 piece of the 4x4 piece of former frame and next frame is lost 4x4 piece according to what self movable information was extrapolated to described lost frames, add up all the outer pushing blocks of described former frame and area and all the outer pushing blocks of described next frame and the area of described lost frames overlapping region of described lost frames overlapping region, determine motion vector extrapolation direction according to the former frame or the next frame of maximum overlapping region area correspondence.
In another embodiment of the present invention, selecting in the described step (2) 16x16 piece size to cover or will describedly lose macroblock partitions is that 4 8x8 piece sizes are covered or skipped the described macro block of losing and are specially: (21) are one, when overlapping region area maximum is zero, skip the described macro block of losing when the overlapping region; (22) when the overlapping region be that one, overlapping region area maximum are during greater than the first macro block threshold values, the described macro block of losing is covered with 16x16 piece size, and the described motion vector of losing macro block is the motion vector of the extrapolation macro block of maximum overlapping region area correspondence; (23) when the overlapping region be a plurality of, a plurality of overlapping regions area sum is greater than the second macro block threshold values, and when the motion vector of all extrapolation macro blocks is close, the described macro block of losing is covered described weighted average of losing the motion vector of macro block for the motion vector of all extrapolation macro blocks with 16x16 piece size; (24) when the overlapping region be a plurality of, a plurality of overlapping regions area sum is greater than the second macro block threshold values, and the motion vector of all extrapolation macro blocks is that 4 8x8 pieces are covered respectively with the described macroblock partitions of losing when not close.
In another embodiment of the present invention, described method also comprises: (3) lose 8x8 piece for what cover by 8x8 piece size, the adjacent neighborhood 8x8 piece of 8x8 block space that will be in same position in former frame or next frame with it is extrapolated to the described 8x8 of losing piece along the motion vector extrapolation direction of selecting, 8x8 piece obtains extrapolating, according to extrapolation 8x8 piece and described number of losing 8x8 piece overlapping region, close degree between the area of overlapping region and the extrapolation 8x8 block motion vector is selected 8x8 piece size to cover or the described 8x8 of losing piece further is divided into 4 4x4 piece sizes and is covered or skip the sub-piece of the described 8x8 of losing.
In an embodiment more of the present invention, select in the described step (3) 8x8 piece size to cover or will the described 8x8 of losing piece further be divided into 4 4x4 piece sizes to cover or skip the described 8x8 of losing piece and be specially: (31) are one, when overlapping region area maximum is zero, skip the sub-piece of the described 8x8 of losing when the overlapping region; (32) when the overlapping region be that one, overlapping region area maximum are during greater than a 8x8 piece threshold values, the described 8x8 of losing piece is covered with 8x8 piece size, and the motion vector of the described 8x8 of losing piece is the motion vector of the extrapolation 8x8 piece of maximum overlapping region area correspondence; (33) when the overlapping region be a plurality of, a plurality of overlapping regions area sum is during greater than the 2nd 8x8 piece threshold values, and when the motion vector of all extrapolation 8x8 pieces is close, the described 8x8 of losing piece is covered described weighted average of losing the motion vector of 8x8 piece for all extrapolation 8x8 block motion vectors with 8x8 piece size; (34) when the overlapping region be a plurality of, a plurality of overlapping regions area sum is greater than the second macro block threshold values, and the motion vector of all extrapolation 8x8 pieces is divided into 4 4x4 pieces with the described 8x8 of losing piece and covers respectively when not close.
In another embodiment of the present invention, described method also comprises: (4) lose 4x4 piece for what cover by 4x4 piece size, the adjacent neighborhood 4x4 piece of 8x8 block space that will be in same position in former frame or next frame with it is extrapolated to the described 4x4 of losing piece along the motion vector extrapolation direction of selecting, 4x4 piece obtains extrapolating, according to extrapolation 4x4 piece and described number of losing 4x4 piece overlapping region, close degree between the area of overlapping region and the extrapolation 4x4 block motion vector selects 4x4 piece size to cover or skip the described 4x4 of losing piece.
In an embodiment more of the present invention, select 4x4 piece size to cover or skip the described 4x4 of losing piece in the described step (4) and be specially: (41) are one, when overlapping region area maximum is zero, skip the described 4x4 of losing piece when the overlapping region; (42) when the overlapping region be that one, overlapping region area maximum are during greater than a 4x4 piece threshold values, the described 4x4 of losing piece is covered with 4x4 piece size, and the motion vector of the described 4x4 of losing piece is the motion vector of the extrapolation 4x4 piece of maximum overlapping region area correspondence; (43) when the overlapping region be a plurality of, a plurality of overlapping regions area sum is greater than the 2nd 4x4 piece threshold values, and when the motion vector of all extrapolation 4x4 pieces is close, the described 4x4 of losing piece is covered described weighted average of losing the motion vector of 4x4 piece for all extrapolation 4x4 block motion vectors with 4x4 piece size; (44) when the overlapping region be a plurality of, a plurality of overlapping regions area sum is greater than the 2nd 4x4 piece threshold values, and the motion vector of all extrapolation 4x4 pieces is skipped the described 4x4 of losing piece when not close.
In another embodiment of the present invention, described motion vector determining step close and that motion vector is not close is specially: when the difference between two motion vectors when presetting threshold values, two motion vectors are close; When the difference between two motion vectors is not less than when presetting threshold values, two motion vectors are not close.
In an embodiment more of the present invention, described method also comprises: (5) are for described losing macro block, lose 8x8 piece or losing 4x4 piece of skipping, lose macro block, lose 8x8 piece or lose the motion vector of 4x4 piece according to what recover to cover, utilize the capable error concealment of border matching process.
In another embodiment of the present invention, described step (5) is specially: (51) form candidate set with mean value vector and median vector, the zero vector of the motion vector of neighborhood piece in the motion vector of neighborhood piece in the mean value vector of the motion vector of the motion vector of former frame neighborhood piece, former frame neighborhood piece and median vector, the present frame covered, the present frame covered; (52) each motion vector that described candidate vector is concentrated is lost the replacement block of piece with the reference block conduct of this motion vector points, and calculates the difference of the boundary pixel of neighborhood piece in the boundary pixel of described replacement block and the present frame; (53) motion vector with the boundary pixel difference minimum of the boundary pixel of replacement block and neighborhood piece is defined as losing the motion vector of piece the best, obtains motion compensation block by this motion vector in reference frame,, thus the error concealment of losing piece finished.。
Compared with prior art, the whole-frame error concealment method that the present invention is based on adaptive block sizes has following advantage:
1) adopt adaptive block sizes to carry out error concealment to all video sequences, can recover to have the video of careful motion feature, when reconstructed image, can alleviate blocking artifact, efficient recovery can be arranged, can obtain the objective and subjective effect of better quality the frame of video of losing;
2) suppressed the error code diffusion effectively;
3) computational complexity is moderate, has practicality widely.
4) can overcome first kind of existing method can produce the problem of bigger distortion, second kind of existing method for the moving region video sequence of non-uniform movement or irregular movement is covered overlap problem and the occlusion issue that the relatively poor problem of effect and the third existing method produce.
By following description also in conjunction with the accompanying drawings, it is more clear that the present invention will become, and these accompanying drawings are used to explain embodiments of the invention.
Description of drawings
Fig. 1 is the flow chart that the present invention is based on the whole-frame error concealment method of adaptive block sizes.
Fig. 2 is shown in Figure 1 based on frame, macro block, 8x8 piece and 4x4 piece location diagram in the whole-frame error concealment method of adaptive block sizes.
Fig. 3 a is the schematic diagram based on forward motion vector extrapolation in the whole-frame error concealment method of adaptive block sizes shown in Figure 1.
Fig. 3 b is the schematic diagram based on backward motion vector extrapolation in the whole-frame error concealment method of adaptive block sizes shown in Figure 1.
Fig. 4 is shown in Figure 1 based on macroblock motion vector extrapolation schematic diagram in the whole-frame error concealment method of adaptive block sizes.
Fig. 5 is shown in Figure 1 based on 8x8 block motion vector extrapolation schematic diagram in the whole-frame error concealment method of adaptive block sizes.
Fig. 6 is shown in Figure 1 based on 4x4 block motion vector extrapolation schematic diagram in the whole-frame error concealment method of adaptive block sizes.
Embodiment
With reference now to accompanying drawing, describe embodiments of the invention, the similar elements label is represented similar elements in the accompanying drawing.
Fig. 2 is that present embodiment is based on macro block, 8x8 piece and 4x4 piece location diagram in the whole-frame error concealment method of adaptive block sizes.As shown in Figure 2, f
nExpression lost frames n, it is by macro block
Form each macro block
Comprise 4 8x8 pieces
, each 8x8 piece
Comprise 4 4x4 pieces
Wherein, i
16And j
16Identified macro block
At frame f
nIn be the coordinate of unit with the macro block, i
8And j
8Identified 8x8 piece
At frame f
nIn be the coordinate of unit with 8x8 piece, i
4And j
4Identified 4x4 piece
At frame f
nIn be the coordinate of unit with 4x4 piece.Macro block
, 8x8 piece
With 4x4 piece
Motion vector be expressed as respectively
With
The following describes the whole-frame error concealment method of present embodiment based on adaptive block sizes.As Fig. 1, described method comprises the steps:
Step S1 is when putting in order LOF, according to lost frames f
nFormer frame f
N-1With next frame f
N+1Type of coding, and described former frame f
N-14x4 piece
With next frame f
N+14x4 piece
Be extrapolated to described lost frames f respectively
nShi Suoshu former frame f
N-1All outer pushing blocks and described lost frames f
nThe area of overlapping region and described next frame f
N+1All outer pushing blocks and described lost frames f
nThe area of overlapping region is selected forward or backward movement vector extrapolation direction.For video sequence, former frame f
N-1With next frame f
N+1With lost frames f
nDegree of correlation and incomplete same, particularly when scene changes, lost frames f
nOften only stronger with wherein a frame correlation, for guaranteeing the continuity of motion, need choose when covering and lost frames f
nThe frame that correlation is stronger carries out the motion vector extrapolation, and this step S1 selects and lost frames f earlier
nThe former frame f that correlation is stronger
N-1Or next frame f
N+1, and then determine to select the forward motion vector extrapolation (with former frame f
N-1Be extrapolated to lost frames f
n) or backward motion vector extrapolation (with next frame f
N+1Be extrapolated to lost frames f
n).Following step is that forward motion vector extrapolation direction is an example with selected motion vector extrapolation direction, provide the performing step of present embodiment adaptive block sizes whole-frame error concealment, motion vector extrapolation direction be the adaptive block sizes whole-frame error concealment performing step of backward motion vector extrapolation direction during with forward motion vector extrapolation direction to cover step similar, no longer repetition.
Step S2 is as Fig. 4, for lost frames f
nLose macro block
, will be at former frame f
N-1In be in the macro block of same position with it
The neighborhood macro block in space adjacent (adjacent up and down)
The motion vector extrapolation direction selected along step S1 is extrapolated to the described macro block of losing
, obtain the macro block of extrapolating
(neighborhood macro block
16 affiliated 4x4 pieces
Extrapolation 4x4 piece
(frame of broken lines shown in Fig. 4 the right) constitutes the neighborhood macro block
The extrapolation macro block
, the extrapolation macro block
Motion vector be extrapolation 4x4 piece
The motion vector weighted average), according to the neighborhood macro block
The extrapolation macro block
With the described macro block of losing
The area of the number of overlapping region (the oblique line frame table shows the overlapping region), overlapping region and extrapolation macro block
Close degree between the motion vector selects 16x16 piece size to cover or with the described macro block of losing
Be divided into 4 8x8 piece sizes and cover or skip the described macro block of losing
Step S3, as Fig. 5, for step S2 selected lose 8x8 piece by what 8x8 piece size was covered
, will be at former frame f
N-1In be in the 8x8 piece of same position with it
The neighborhood 8x8 piece in space adjacent (adjacent up and down)
The motion vector extrapolation direction of selecting along step S1 is extrapolated to the described 8x8 of losing piece
, 8x8 piece obtains extrapolating
(neighborhood 8x8 piece
4 affiliated 4x4 pieces
Extrapolation 4x4 piece
(frame of broken lines shown in Fig. 5 the right) constitutes neighborhood 8x8 piece
Extrapolation 8x8 piece
, extrapolation 8x8 piece
Motion vector be extrapolation 4x4 piece
The motion vector weighted average), according to neighborhood 8x8 piece
Extrapolation 8x8 piece
With the described 8x8 piece of losing
The area of the number of overlapping region (the oblique line frame table shows the overlapping region), overlapping region and extrapolation 8x8 piece
Close degree between the motion vector selects 8x8 piece size to cover or with the described 8x8 of losing piece
Further be divided into 4 4x4 piece sizes and cover or skip the described 8x8 of losing piece
Step S4, as Fig. 6, for step S3 selected lose 4x4 piece by what 4x4 piece size was covered
, will be at former frame f
N-1In be in the 4x4 piece of same position with it
The neighborhood 4x4 piece in space adjacent (adjacent up and down)
The motion vector extrapolation direction selected along step S1 is extrapolated to the described 4x4 of losing piece
, 4x4 piece obtains extrapolating
(frame of broken lines shown in Fig. 6 the right) is according to neighborhood 4x4 piece
Extrapolation 4x4 piece
With the described 4x4 piece of losing
The area of the number of overlapping region (the oblique line frame table shows the overlapping region), overlapping region and extrapolation 4x4 piece
Close degree between the motion vector selects 4x4 piece size to cover or skip the described 4x4 of losing piece
As shown from the above technical solution, present embodiment has following advantage based on the whole-frame error concealment method of adaptive block sizes:
1) to all video sequence, characteristics self adaptation according to video sequence adopts 16x16 piece size, 8x8 piece size and 4x4 piece size to carry out error concealment, can recover to have the video of careful motion feature, when reconstructed image, can alleviate blocking artifact, to the frame of video of losing efficient recovery can be arranged, can obtain the objective and subjective effect of better quality;
2) owing to recovered lost frames effectively, therefore, the recovery value that the frame of video after the lost frames can make full use of lost frames carries out decoding and rebuilding, has avoided the influence of lost frames to subsequent frame, thereby has suppressed the error code diffusion effectively;
3) computational complexity is moderate, has practicality widely;
4) owing to avoided adopting the deficiency of fixed block size restoration motion vector at the different video sequence, therefore can accurately recover to lose the operation vector of piece, can overcome first kind of existing method and can produce the problem of bigger distortion, second kind of existing method for the moving region video sequence of non-uniform movement or irregular movement is covered overlap problem and the occlusion issue that the relatively poor problem of effect and the third existing method produce.
Preferably, described step S1 is specially:
Step S11 is as lost frames f
nFormer frame f
N-1Be intracoded frame, next frame f
N+1(has only next frame f this moment during for inter-frame encoding frame
N+1Movable information can with), choose backward motion vector extrapolation direction; Former frame f
N-1Interior neighborhood 4x4 piece
Step S12 is as lost frames f
nFormer frame f
N-1Be inter-frame encoding frame, next frame f
N+1(has only former frame f this moment during for intracoded frame
nMovable information can with), choose forward motion vector extrapolation direction;
Step S13 is as lost frames f
nFormer frame f
N-1Be inter-frame encoding frame, next frame f
N+1(this moment former frame f during for inter-frame encoding frame
N-1With next frame f
N+1Movable information all available), shown in Fig. 3 a and Fig. 3 b, respectively with former frame f
N-1In 4x4 piece
With next frame f
N+1In 4x4 piece
Movable information according to self is extrapolated to lost frames f
nLose the 4x4 block
, 4x4 piece obtains extrapolating
(frame of broken lines shown in Fig. 3 a and Fig. 3 b the right) adds up described former frame f
N-1All outer pushing blocks and described lost frames f
nArea of overlapping region (the oblique line frame table among Fig. 3 a and Fig. 3 b shows the overlapping region) and described next frame f
N+1All outer pushing blocks and described lost frames f
nThe area of overlapping region, the former frame f of maximum overlapping region area correspondence
N-1Or next frame f
N+1For with lost frames f
nThe frame that degree of correlation is higher is determined motion vector extrapolation direction, even described former frame f according to the higher frame of degree of correlation
N-1All outer pushing blocks and described lost frames f
nThe area of overlapping region is greater than described next frame f
N+1All outer pushing blocks and described lost frames f
nThe area of overlapping region, then described former frame f
N-1For with lost frames f
nThe frame that degree of correlation is higher is selected forward motion vector extrapolation direction this moment, otherwise selects backward motion vector extrapolation direction (among Fig. 3 a and Fig. 3 b, arrow is represented motion vector, frame of broken lines representative extrapolation 4x4 piece).
Note, as lost frames f
nFormer frame f
N-1Be intracoded frame, next frame f
N+1During for intracoded frame, the probability that this situation takes place is minimum, does not therefore consider.
Wherein, among the described step S2, select 16x16 piece size to cover or with the described macro block of losing
Be divided into 4 8x8 piece sizes and cover or skip the described macro block of losing
Be specially:
Step S21 is when the overlapping region is that one, overlapping region area maximum (show and lose macro block when being zero
With all extrapolation macro blocks
Equal zero lap zone), skip the described macro block of losing
Step S22 is when the overlapping region is that one, overlapping region area maximum are greater than the first macro block threshold values
In time, (shows and loses macro block
Similar with the movement locus of one of them extrapolation macro block), with the described macro block of losing
Cover the described macro block of losing with 16x16 piece size
Motion vector be the motion vector of the extrapolation macro block of maximum overlapping region area correspondence;
Step S23, when the overlapping region is a plurality of, a plurality of overlapping regions area sum is greater than the second macro block threshold values
(wherein,
), and the motion vector of all extrapolation macro blocks is when close, with the described macro block of losing
Cover the described macro block of losing with 16x16 piece size
Motion vector be the weighted average of motion vector of all extrapolation macro blocks;
Step S24, when the overlapping region is a plurality of, a plurality of overlapping regions area sum is greater than the second macro block threshold values
, and the motion vector of all extrapolation macro blocks (shows and loses macro block when not close
Comprise a plurality of motion of objects), with the described macro block of losing
Being divided into 4 8x8 piece sizes covers respectively.
Wherein, among the described step S3, select to lose the 8x8 piece
Size or with the described 8x8 piece of losing
Further be divided into 4 4x4 piece sizes and cover or skip the described 8x8 of losing piece
Be specially:
Step S31 is when the overlapping region is one, when overlapping region area maximum is zero, skips the described 8x8 of losing piece
Step S32 is when the overlapping region is that one, overlapping region area maximum are greater than a 8x8 piece threshold values
In time, (shows and loses 8x8 piece
Similar with the movement locus of one of them extrapolation 8x8 piece), with the described 8x8 piece of losing
Cover the described 8x8 piece of losing with 8x8 piece size
Motion vector be the motion vector of the extrapolation 8x8 piece of maximum overlapping region area correspondence;
Step S33, when the overlapping region is a plurality of, a plurality of overlapping regions area sum is greater than the 2nd 8x8 piece threshold values
(wherein,
), and the motion vector of all extrapolation 8x8 pieces is when close, with the described 8x8 piece of losing
Cover the described 8x8 piece of losing with 8x8 piece size
Motion vector is the weighted average of all extrapolation 8x8 block motion vectors;
Step S34, when the overlapping region is a plurality of, a plurality of overlapping regions area sum is greater than the second macro block threshold values
, and the motion vector of all extrapolation 8x8 pieces (shows and loses 8x8 piece when not close
Comprise a plurality of motion of objects), with the described 8x8 piece of losing
Being divided into 4 4x4 pieces covers respectively.
Wherein, among the described step S4, select 4x4 piece to cover or skip the described 4x4 of losing piece
Be specially:
Step S41 is when the overlapping region is one, when overlapping region area maximum is zero, skips the described 4x4 of losing piece
Step S42 is when the overlapping region is that one, overlapping region area maximum are greater than a 4x4 piece threshold values
In time, (shows and loses 4x4 piece
Similar with the movement locus of one of them extrapolation 4x4 piece), with the described 4x4 piece of losing
Cover the described 4x4 piece of losing with 4x4 piece size
Motion vector be the motion vector of the extrapolation 4x4 piece of maximum overlapping region area correspondence;
Step S43, when the overlapping region is a plurality of, a plurality of overlapping regions area sum is greater than the 2nd 4x4 piece threshold values
(wherein,
), and the motion vector of all extrapolation 4x4 pieces is when close, with the described 4x4 piece of losing
Cover the described 4x4 piece of losing with 4x4 piece size
Motion vector be the weighted average of all extrapolation 4x4 block motion vectors;
Step S44, when the overlapping region is a plurality of, a plurality of overlapping regions area sum is greater than the 2nd 4x4 piece threshold values
, and the motion vector of all extrapolation 4x4 pieces (shows the described 4x4 of losing piece when not close
Comprise a plurality of motion of objects), skip the described 4x4 of losing piece
Wherein, described motion vector is close is specially with the not close determining step of motion vector:
When the difference between two motion vectors when presetting threshold values, two motion vectors are close;
When the difference between two motion vectors is not less than when presetting threshold values, two motion vectors are not close.
In addition, this method also comprises step:
Step S5, to above-mentioned steps S21, what S31 and S41 skipped processing loses macro block, 8x8 piece and 4x4 piece, according to the movable information of losing macro block, 4x4 piece, 8x8 piece that has recovered, utilize the border matching process to carry out error concealment, smooth connection with macroblock boundaries, 8x8 piece or the 4x4 piece place of guarantee recovering image is specially:
Step S51 forms candidate set with mean value vector and median vector, the zero vector of the motion vector of neighborhood piece in the motion vector of neighborhood piece in the mean value vector of the motion vector of the motion vector of former frame neighborhood piece, former frame neighborhood piece and median vector, the present frame covered, the present frame covered;
Step S52 to each motion vector that described candidate vector is concentrated, loses the replacement block of piece with the reference block conduct of this motion vector points, and calculates the difference of the boundary pixel of neighborhood piece in the boundary pixel of described replacement block and the present frame;
Step S53, boundary pixel difference based on adjacent block is more little, the theory that image pixel is level and smooth more, the motion vector of the boundary pixel difference minimum of the boundary pixel of replacement block and neighborhood piece is defined as losing the motion vector of piece the best, after obtaining losing the motion vector of piece, obtain motion compensation block by this motion vector in reference frame, the compensation replacement block is the recovery value of losing piece, thereby finishes the error concealment of losing piece.
As from the foregoing, this method is extrapolated to lost frames with the 4x4 piece of consecutive frame, adds up the area of all extrapolation 4x4 pieces and lost frames overlapping regions, and in conjunction with the type of coding of consecutive frame, selects the suitable motion vector direction of extrapolating.For losing macro block, the movable information of consecutive frame is extrapolated to along selected direction loses macro block, according to the close degree of extrapolation macro block, select 16x16 piece size or be that the processing of losing macro block is covered or skipped to 4 8x8 piece sizes macroblock partitions with size of losing the macro block overlapping region and extrapolation macroblock motion vector.For 8x8 piece, in like manner can select 8x8 piece size or 8x8 piece further is divided into 4 4x4 piece sizes to cover or skip the processing of losing 8x8 piece.For losing 4x4 piece, in like manner can select 4x4 piece size to cover or skip the processing of losing 4x4 piece.
After selecting suitable macro block, 8x8 piece, 4x4 piece size, obtain corresponding motion vector, be compensated replacement block according to this motion vector then in reference frame, the compensation replacement block is the recovery value of losing piece, thereby finishes losing the error concealment of piece.
For the macro block of skipping in the said process, 8x8 piece, 4x4 piece, according to the movable information of the 4x4 piece that recovers to cover, 8x8 piece or macro block and adopt the external boundary matching algorithm to cover.
Above invention has been described in conjunction with most preferred embodiment, but the present invention is not limited to the embodiment of above announcement, and should contain various modification, equivalent combinations of carrying out according to essence of the present invention.
Claims (10)
1. the whole-frame error concealment method based on adaptive block sizes comprises the steps:
(1) according to the former frame of lost frames and the type of coding of next frame, and all the outer pushing blocks of the 4x4 piece of the 4x4 piece of the 4x4 of described former frame piece and next frame described former frame when being extrapolated to described lost frames respectively and area and the outer pushing block of described next frame and the area of described lost frames overlapping region of described lost frames overlapping region, select forward or backward movement vector extrapolation direction;
(2) will be in former frame or next frame be in the adjacent neighborhood macro block in the macro block space of same position and be extrapolated to the described macro block of losing along the motion vector extrapolation direction of selecting with the macro block of losing of lost frames, obtain the macro block of extrapolating, according to extrapolation macro block and the area of described number of losing the macro block overlapping region, overlapping region and the close degree between the extrapolation macroblock motion vector, selection 16x16 piece size is covered or is that the described macro block of losing is covered or skipped to 4 8x8 piece sizes with the described macroblock partitions of losing.
2. the whole-frame error concealment method based on adaptive block sizes as claimed in claim 1 is characterized in that, described step (1) is specially:
(11) former frame when lost frames is intracoded frame, when next frame is inter-frame encoding frame, chooses backward motion vector extrapolation direction;
(12) former frame when lost frames is inter-frame encoding frame, when next frame is intracoded frame, chooses forward motion vector extrapolation direction;
(13) former frame when lost frames be inter-frame encoding frame, when next frame is inter-frame encoding frame, respectively the 4x4 piece of the 4x4 piece of former frame and next frame is lost 4x4 piece according to what self movable information was extrapolated to described lost frames, add up all the outer pushing blocks of described former frame and area and all the outer pushing blocks of described next frame and the area of described lost frames overlapping region of described lost frames overlapping region, determine motion vector extrapolation direction according to the former frame or the next frame of maximum overlapping region area correspondence.
3. the whole-frame error concealment method based on adaptive block sizes as claimed in claim 1, it is characterized in that, select in the described step (2) 16x16 piece size to cover or be that 4 8x8 piece sizes are covered or skipped the described macro block of losing and are specially the described macroblock partitions of losing:
(21) be one, when overlapping region area maximum is zero, skip the described macro block of losing when the overlapping region;
(22) when the overlapping region be that one, overlapping region area maximum are during greater than the first macro block threshold values, the described macro block of losing is covered with 16x16 piece size, and the described motion vector of losing macro block is the motion vector of the extrapolation macro block of maximum overlapping region area correspondence;
(23) when the overlapping region be a plurality of, a plurality of overlapping regions area sum is greater than the second macro block threshold values, and when the motion vector of all extrapolation macro blocks is close, the described macro block of losing is covered described weighted average of losing the motion vector of macro block for the motion vector of all extrapolation macro blocks with 16x16 piece size;
(24) when the overlapping region be a plurality of, a plurality of overlapping regions area sum is greater than the second macro block threshold values, and the motion vector of all extrapolation macro blocks is that 4 8x8 pieces are covered respectively with the described macroblock partitions of losing when not close.
4. the whole-frame error concealment method based on adaptive block sizes as claimed in claim 1 is characterized in that, also comprises:
(3) lose 8x8 piece for what cover by 8x8 piece size, the adjacent neighborhood 8x8 piece of 8x8 block space that will be in same position in former frame or next frame with it is extrapolated to the described 8x8 of losing piece along the motion vector extrapolation direction of selecting, 8x8 piece obtains extrapolating, according to extrapolation 8x8 piece and described number of losing 8x8 piece overlapping region, close degree between the area of overlapping region and the extrapolation 8x8 block motion vector is selected 8x8 piece size to cover or the described 8x8 of losing piece further is divided into 4 4x4 piece sizes and is covered or skip the sub-piece of the described 8x8 of losing.
5. the whole-frame error concealment method based on adaptive block sizes as claimed in claim 4, it is characterized in that, select in the described step (3) 8x8 piece size to cover or the described 8x8 of losing piece further is divided into 4 4x4 piece sizes to cover or skip the described 8x8 of losing piece and be specially:
(31) be one, when overlapping region area maximum is zero, skip the described 8x8 of losing piece when the overlapping region;
(32) when the overlapping region be that one, overlapping region area maximum are during greater than a 8x8 piece threshold values, the described 8x8 of losing piece is covered with 8x8 piece size, and the motion vector of the described 8x8 of losing piece is the motion vector of the extrapolation 8x8 piece of maximum overlapping region area correspondence;
(33) when the overlapping region be a plurality of, a plurality of overlapping regions area sum is during greater than the 2nd 8x8 piece threshold values, and when the motion vector of all extrapolation 8x8 pieces is close, the described 8x8 of losing piece is covered described weighted average of losing the motion vector of 8x8 piece for all extrapolation 8x8 block motion vectors with 8x8 piece size;
(34) when the overlapping region be a plurality of, a plurality of overlapping regions area sum is greater than the second macro block threshold values, and the motion vector of all extrapolation 8x8 pieces is divided into 4 4x4 pieces with the described 8x8 of losing piece and covers respectively when not close.
6. the whole-frame error concealment method based on adaptive block sizes as claimed in claim 4 is characterized in that, also comprises:
(4) lose 4x4 piece for what cover by 4x4 piece size, the adjacent neighborhood 4x4 piece of 8x8 block space that will be in same position in former frame or next frame with it is extrapolated to the described 4x4 of losing piece along the motion vector extrapolation direction of selecting, 4x4 piece obtains extrapolating, according to extrapolation 4x4 piece and the described number of 4x4 piece overlapping region, the area of overlapping region and the close degree between the extrapolation 4x4 block motion vector of losing, select 4x4 piece size to cover or skip the sub-piece of the described 4x4 of losing.
7. the whole-frame error concealment method based on adaptive block sizes as claimed in claim 6 is characterized in that, selects 4x4 piece size to cover or skip the described 4x4 of losing piece in the described step (4) and is specially:
(41) be one, when overlapping region area maximum is zero, skip the described 4x4 of losing piece when the overlapping region;
(42) when the overlapping region be that one, overlapping region area maximum are during greater than a 4x4 piece threshold values, the described 4x4 of losing piece is covered with 4x4 piece size, and the motion vector of the described 4x4 of losing piece is the motion vector of the extrapolation 4x4 piece of maximum overlapping region area correspondence;
(43) when the overlapping region be a plurality of, a plurality of overlapping regions area sum is greater than the 2nd 4x4 piece threshold values, and when the motion vector of all extrapolation 4x4 pieces is close, the described 4x4 of losing piece is covered described weighted average of losing the motion vector of 4x4 piece for all extrapolation 4x4 block motion vectors with 4x4 piece size;
(44) when the overlapping region be a plurality of, a plurality of overlapping regions area sum is greater than the 2nd 4x4 piece threshold values, and the motion vector of all extrapolation 4x4 pieces is skipped the described 4x4 of losing piece when not close.
8. as claim 3 or 5 or 7 described whole-frame error concealment methods, it is characterized in that described motion vector is close to be specially with the not close determining step of motion vector based on adaptive block sizes:
When the difference between two motion vectors when presetting threshold values, two motion vectors are close;
When the difference between two motion vectors is not less than when presetting threshold values, two motion vectors are not close.
9. the whole-frame error concealment method based on adaptive block sizes as claimed in claim 6 is characterized in that, also comprises:
(5) for described losing macro block, lose 8x8 piece or losing 4x4 piece of skipping, lose macro block, lose 8x8 piece or lose the motion vector of 4x4 piece according to what recover to cover, utilize the capable error concealment of border matching process.
10. the whole-frame error concealment method based on adaptive block sizes as claimed in claim 9 is characterized in that, described step (5) is specially:
(51) mean value vector and median vector, the zero vector of the motion vector of neighborhood piece in the motion vector of neighborhood piece in the mean value vector of the motion vector of the motion vector of former frame neighborhood piece, former frame neighborhood piece and median vector, the present frame covered, the present frame covered are formed candidate set;
(52) each motion vector that described candidate vector is concentrated is lost the replacement block of piece with the reference block conduct of this motion vector points, and calculates the difference of the boundary pixel of neighborhood piece in the boundary pixel of described replacement block and the present frame;
(53) motion vector with the boundary pixel difference minimum of the boundary pixel of replacement block and neighborhood piece is defined as losing the motion vector of piece the best, obtains motion compensation block by this motion vector in reference frame, thereby finishes the error concealment of losing piece.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010266739 CN101917628B (en) | 2010-08-30 | 2010-08-30 | Whole-frame error concealment method based on adaptive block sizes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010266739 CN101917628B (en) | 2010-08-30 | 2010-08-30 | Whole-frame error concealment method based on adaptive block sizes |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101917628A true CN101917628A (en) | 2010-12-15 |
| CN101917628B CN101917628B (en) | 2012-07-18 |
Family
ID=43324995
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010266739 Active CN101917628B (en) | 2010-08-30 | 2010-08-30 | Whole-frame error concealment method based on adaptive block sizes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101917628B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111556334A (en) * | 2020-03-27 | 2020-08-18 | 李惠芳 | Internet video smoothing method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050157800A1 (en) * | 2004-01-19 | 2005-07-21 | Samsung Electronics Co., Ltd. | Error concealing device and method thereof for video frame |
| CN1893663A (en) * | 2005-09-02 | 2007-01-10 | 华为技术有限公司 | Transmission protection method of multi-media communication |
| CN101123731A (en) * | 2007-08-31 | 2008-02-13 | 广东威创日新电子有限公司 | Covering method for video image error |
| CN101355708A (en) * | 2007-07-25 | 2009-01-28 | 中兴通讯股份有限公司 | Self-adapting method for shielding error code |
| CN101692712A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Error concealment method based on state of signal channel and visual perception side information |
-
2010
- 2010-08-30 CN CN 201010266739 patent/CN101917628B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050157800A1 (en) * | 2004-01-19 | 2005-07-21 | Samsung Electronics Co., Ltd. | Error concealing device and method thereof for video frame |
| CN1893663A (en) * | 2005-09-02 | 2007-01-10 | 华为技术有限公司 | Transmission protection method of multi-media communication |
| CN101355708A (en) * | 2007-07-25 | 2009-01-28 | 中兴通讯股份有限公司 | Self-adapting method for shielding error code |
| CN101123731A (en) * | 2007-08-31 | 2008-02-13 | 广东威创日新电子有限公司 | Covering method for video image error |
| CN101692712A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Error concealment method based on state of signal channel and visual perception side information |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111556334A (en) * | 2020-03-27 | 2020-08-18 | 李惠芳 | Internet video smoothing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101917628B (en) | 2012-07-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | Error concealment of lost motion vectors with overlapped motion compensation | |
| Zhai et al. | A low complexity motion compensated frame interpolation method | |
| KR101055738B1 (en) | Method and apparatus for encoding/decoding video signal using prediction information of intra-mode macro blocks of base layer | |
| CN100531400C (en) | Video error coverage method based on macro block level and pixel motion estimation | |
| KR20060109248A (en) | Method and apparatus for processing information of an intra-mode block on encoding/decoding of a video signal | |
| CA2565670A1 (en) | Method and apparatus for motion compensated frame rate up conversion | |
| WO2010093430A1 (en) | System and method for frame interpolation for a compressed video bitstream | |
| CN109451310B (en) | Rate distortion optimization method and device based on significance weighting | |
| KR20110128366A (en) | Preprocessor method and apparatus | |
| Wu et al. | A temporal error concealment method for H. 264/AVC using motion vector recovery | |
| EP3591973A1 (en) | Method and apparatus for decoding video data, and method and apparatus for encoding video data | |
| Kazemi et al. | A review of temporal video error concealment techniques and their suitability for HEVC and VVC | |
| CN1269359C (en) | Video error blanketing method based on motion vector extrapolation and motion vector search | |
| Ling et al. | A novel spatial and temporal correlation integrated based motion-compensated interpolation for frame rate up-conversion | |
| KR100883591B1 (en) | Method and apparatus for encoding/decoding video signal using prediction information of intra-mode macro blocks of base layer | |
| CN101917628B (en) | Whole-frame error concealment method based on adaptive block sizes | |
| CN108833920B (en) | DVC side information fusion method based on optical flow and block matching | |
| CN101175217A (en) | Code error shielding technology aiming at mobile network video transmission | |
| KR101138922B1 (en) | Apparatus and method for upconverting frame rates of decoded video frames | |
| Lin et al. | Video motion vector recovery method using decoding partition information | |
| KR101144283B1 (en) | Apparatus and method for concealing errors included in the decoded video | |
| Chen et al. | Spatio-temporal video error concealment using priority-ranked region-matching | |
| Garcia-V et al. | Image processing for error concealment | |
| Xu et al. | Frame rate up-conversion with true motion estimation and adaptive motion vector refinement | |
| Chen | Advancing video compression with error resilience and content analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| ASS | Succession or assignment of patent right |
Owner name: WUHAN FIBERHOME ZHONGZHI DIGITAL TECHNOLOGY CO., L Free format text: FORMER OWNER: WUHAN HONGXIN COMMUNICATION TECHNOLOGY CO., LTD. Effective date: 20111129 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20111129 Address after: 430074 Wuhan, Hongshan Province District Road, Department of mail, No. 88 hospital Applicant after: Wuhan Fiberhome Digtal Technology Co., Ltd. Address before: 430074 Wuhan, Hongshan Province District Road, Department of mail, No. 88 hospital Applicant before: Wuhan Hongxin Communication Technology Co., ltd. |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |