KR20060085003A - A temporal error concealment method in the h.264/avc standard - Google Patents

Abstract

The present invention relates to a method for performing visual error concealment in consideration of the characteristics of the H.264 / AVC coding standard established by video standards such as terrestrial and satellite DMB. In particular, the macroblock in H.264 / AVC includes The present invention relates to a method of visual error concealment in H.264 that is suitable for providing improved objective image quality with a small amount of computation by performing error concealment with a variable size using various information.

The constituent means of the method of concealing the error in the time direction of the present invention H.264 comprises the steps of: identifying four macroblock modes located on the top, bottom, left, and right sides of the lost macroblock, using the determined top, bottom, left, and right macroblock modes. Determining an error concealment mode having a number and a size of?, Calculating a cost value for the motion vector and the reference frame information by the OBMA method according to the determined error concealment mode, and performing error concealment; do.

H.264, error concealment

Description

H. 264 / AEC method for concealment of visual errors in ACC The {A TEMPORAL ERROR CONCEALMENT METHOD IN THE H.264 / AVC STANDARD}

1 is a schematic diagram of a macroblock mode in H.264.

2 is a schematic diagram of a motion vector and reference frame information used in a conventional error concealment method in H.264 / AVC.

3 is a flowchart illustrating a process of selecting an error concealment mode according to the present invention.

4 is a schematic diagram showing adjacent pixels used to calculate a cost value in each error concealment mode in the present invention.

The present invention relates to a method for performing visual error concealment in consideration of the characteristics of the H.264 / AVC coding standard established by video standards such as terrestrial and satellite DMB. In particular, the macroblock in H.264 / AVC includes The present invention relates to a method of visual error concealment in H.264 that is suitable for providing improved objective image quality with a small amount of computation by performing error concealment with a variable size using various information.

The H264 / ACC standard was established in collaboration with ITU-T's Video Coding Experts Group (VCEG) and ISO / IEC's Moving Picture Expert Group (MPEG). The H.264 / AVC video coding scheme adds new coding tools that were not found in the previous video coding standards (MPEG2, MPEG4 Part 2, H.263). In comparison, it has a higher compression efficiency.

Newly added coding tools are as follows.

First, H.264 performs integer conversion of 4 × 4 units, and there is no mismatch between the encoder and the decoder. This is compared with several existing video coding standards that had a problem of mismatch between encoder and decoder by using discrete cosine transform of 8 × 8 units. In addition, when a macroblock is selected as an intra 16 × 16 mode, DC values of a 4 × 4 transform block, which is a transform performing unit, perform a Hadamard transform once again, and through this hierarchical transform, High compression efficiency. In addition, the quantization parameter in H.264 is used as one index indicating quantization data and has a value of 52 steps from 0 to 51.

Second, in H.264, two entropy coding methods are used. In other words, the entropy coding technique converts most of the syntax except the transform coefficients into one codeword table according to the characteristics of the data, and then encodes it into EGC (Exponential Golomb Code). There is a method of encoding by CAVLC (Context Adaptive Variable Length Coding) and a method of adaptively selecting estimated symbol probability distribution models by adding context modeling to an existing arithmetic coding scheme.

Third, block-based motion estimation, transformation, and quantization techniques bring many advantages to the encoder design, but involve blocking artifacts. H.264 eliminates blocking by using loop filters in the encoding and decoding process to efficiently improve such blocking.

Fourth, the H.264 coding scheme includes intra prediction codes and processes in the spatial domain. The prediction coding method in space is performed with 16 × 16 size and 4 × 4 size for the luminance component, and has a prediction mode in consideration of four and nine directions in space, and the color difference component has a luminance of 16 × 16 size. It has four prediction modes as component. Intra prediction of 16 × 16 size and 4 × 4 size in the spatial domain with respect to the luminance component has high compression efficiency in the detailed region as well as the flat region of the image.

Fifth, in H.264, previously reconstructed frames are used as reference pictures. The use of such a multi-reference image has a higher compression efficiency than the case of using only a single reference image, and is more prominent in the case of encoding an area that is covered by a repetitive image or another object.

Sixth, in H.264, the size of the block is varied and the motion is estimated to be 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, and 4 × 4. This contrasts with H.263 and MPEG-4 technology, which previously compensated for motion only in 16 × 16 and 8 × 8 sizes. Motion estimation of block sizes of various sizes, especially 8 × 4, 4 × 8, and 4 × 4 blocks, which are less than 8 × 8 sub macroblocks, can achieve high compression efficiency even in finer image areas. do.

In addition, coding tools such as adaptive frame / field coding and network abstraction layer (NAL) have been added. These newly added coding tools enable the H.264 coding standard to have higher compression efficiency than previous video coding standards. In the case of encoding the main profile using five reference frames and a 32-pixel motion estimation region, the H.264 standard is known to reduce the bit rate by about 48% to 78% compared to MPEG-2. Thomas Wiegand et al., "Rate-Constrained Coder Control and Comparison of Video Coding Standard."

Among these, various block-size motion estimation techniques require more computation compared to existing coding standards, but are a coding tool that significantly affects coding efficiency by estimating motion in more granular units.

Another feature of estimating motion using these various block sizes is that it generates more motion vectors than existing video standards. That is, while the existing video standard has only one motion vector for one macroblock having a size of 16 × 16, H.264 can have a maximum of 16 motion vectors for one macroblock of size 16 × 16. . This indicates that the amount of motion vector information of neighboring macroblocks that can be used in error concealment in H.264 is larger than that of the existing coding standard.

In addition, by using several reference images, each macroblock includes information of a reference frame together with a motion vector. Accordingly, one macroblock in H.264 includes up to 16 different motion vectors, up to 4 reference frame information, and macroblock mode information defining a unit of motion estimation.

Since MPEG-2 was enacted, much research has been done on visual error concealment. Keiu proposed a technique for concealing errors by copying a reference block using an estimated motion vector by estimating the motion of a lost macroblock [pp. Of "Cell-loss concealment techniques for layered video codecs in an ATM network". 666-677]. However, when the estimation of the motion vector is not accurate, there is a drawback that the difference between the neighboring blocks is large.

In a similar way, Suh and Kwon, pp. 750-753 and mean values of surrounding macroblock motion vectors, respectively, are described in pp. 203 of “Error Concealment Techniques for H.264 Video Transmission”. 276-279], a technique for concealing errors is proposed. It must be assumed that the missing macroblock and neighboring macroblocks have a high correlation between motion vectors.

Jung has proposed a modified BMA method that improves the BMA (Boundary Matching Algorithm) method for estimating motion vectors using neighboring boundary pixels of damaged macroblocks. See "Robust error concealment algorithm using iterative weighted boundary matching criterion." 384-387]. However, this has the disadvantage of having a high complexity.

The various forward error concealment schemes above are suitable for previous coding standards for performing motion estimation on a 16 × 16 scale. Zheng concealed the error by reconstructing the motion vector pixel-by-pixel using more motion vector information in H.264, which estimates motion with various block sizes, but this also does not consider multi-reference frame and macroblock mode information. [A motion vector recovery algorithm for digital video using lagrange interpolation ”pp. 383-389.

Hereinafter, a BMA (Boundary Matching Algorithm) technique and an OBMA (Overlapping BMA) technique corresponding to conventional error concealment methods will be described.

Boundary Matching Algorithm (BMA) is the most widely used method for error concealment through visual motion estimation in the video coding standard. This method uses a feature of a natural image having a large correlation between adjacent pixels. The motion vector minimizes the difference between the neighboring pixels of the macroblock where the error occurs and the edge pixels of the block estimated from the motion vectors of the neighboring macroblocks. The error is concealed by selecting.

Equation (1) to Equation (4) below calculates the difference between adjacent pixels in the up, down, left, and right sides of the lost macroblock as the cost value, and the final cost value in each motion vector value calculated using this is Same as 5).

Cost _BM = BM _T + BM _B + BM _L + BM _R (5)

In this case, P _{x, y} denotes a pixel value at the position (x, y) of the current frame, and P ^r _{x, y} denotes a pixel value at the position (x, y) of the reference frame. M _x and M _y are motion vector information obtained from neighboring macroblocks adjacent to the block in which an error occurs.

On the other hand, in the case of the overlapping BMA (OBMA), the edge component in the screen can be considered to some extent by considering the motion vector in the reference frame instead of the difference value between adjacent pixels and using the difference value between the same positions as the cost value. In the case of using OBMA, the cost values in the up, down, left, and right sides of the lost macroblock are as shown in Equations (6) to (9) below, and the final cost value for one motion vector is shown in Equation (10) below.

The following is a general error concealment technique in H.264.

In most coding standards before H.264 / AVC, the number of motion vectors that a macroblock can have is limited to one by estimating motion in units of 16 × 16. On the other hand, in H.264, one macroblock can have up to 16 motion vectors by performing motion estimation with various block sizes. In addition, by using multiple reference frames, each macroblock also includes information on the reference frame.

Since the minimum unit of frame reference is performed by 8 × 8 blocks, the maximum number of reference frames that one macroblock can have is four.

FIG. 1 shows the granular shape of a macroblock used for motion estimation in the current H.264 / AVC standard.

As shown in FIG. 1, mode 1 estimates the motion of a macroblock with a size of 16x16, mode 2 estimates the motion for each reference frame with two block sizes of 16x8, and in mode 3, two block sizes of 8x16. In the case of Mode 8, one of Mode 4 to Mode 7 corresponding to each 8 × 8 sub-macroblock may be selected. In this case, Mode 1, Mode 2, Mode 3 and Mode 8 are 16x16 macroblock sizes, and Modes 4 to 7 are 8x8 submacroblock sizes.

2 is a block diagram illustrating an error concealment technique using motion vector information and reference frame information of a conventional neighboring macroblock.

As illustrated in FIG. 2, H.264 conceals an error using information of surrounding macroblocks having reference frame information of 8 × 8 units and motion vector information of 4 × 4 units. That is, the motion error and the reference frame information from the 8 × 8 block located in the top, bottom, left and right of the lost macroblock is concealed.

Information included in the 8 × 8 block can be represented by the formula V _i = (M _x , M _y , Ref). Where M _x is the value of the motion vector in the x direction in the 8 × 8 block, M _y is the motion vector value in the y direction in the 8 × 8 block, and Ref represents the reference frame at the size of 8 × 8 block. .

At this time, the motion vector is a motion vector of the a, x in the 8 × 8 block size, because it can have a size of at least 4 × 4 units and the y-direction as shown in Figure 2 has the formula Vi = Mean (V _i (1 ) Is obtained by taking the average of the motion vectors in units of 4 × 4 using + V _i (2) + V _i (3) + V _i (4)).

Therefore, as shown in Fig. 2 and the formulas V _i = (M _x , M _y , Ref) and Vi = Mean (V _i (1) + V _i (2) + V _i (3) + V _i (4)) Similarly, H.264 uses motion vector candidates and reference frames from eight 8 × 8 sized blocks around lost macroblocks, together with up to nine different vectors by considering the zero motion vectors of the immediately preceding reference frame together. The error is concealed using the motion vector and reference frame information.

The visual error concealment technique in H.264 is basically based on BMA, and the motion vector candidate group used is as follows.

V _{Mx, My, Ref} = [ZM, V ₁ , V ₂ , ....., V ₈ ]

where, V _i = Mean (V _i (1) + V _i (2) + V _i (3) + V _i (4)) (11)

ZM in Equation (11) refers to a case of copying a block at the same position as the previous frame. As shown in Equation (11), in H.264, up to 9 motion vectors are used for one macroblock in concealment of the visual error.

If a slice error occurs, the information of the right macroblock cannot be used. For the left macroblock, seven motion vectors are used if the error is already concealed. If the information of the left macroblock is also unavailable, 5 Errors are concealed using the two motion vectors.

When the motion vector candidate group is determined from the neighboring macroblocks, the final motion vector of the error macroblock is selected through Equation (12) below, and the error is concealed using the motion vector candidate group.

V _x , v _y, and ref in Equation (12) represent a motion vector and a reference picture that minimize cost values. When the motion vector and the reference image that minimize the cost value are determined, H.264 finally conceals the error using the following equation.

MB _conceal (x _i , y _j , t) = MB (x _i + v _x , y _j + v _y , t-1-ref)

i, j = 0, 1, ... 15 ref = 0 to 4 (13)

Equation (13) means to obtain a block located in the motion vector determined in the determined reference frame after the final motion vector and the reference frame from the neighboring macroblock information of the lost macroblock. In case of using 5 reference frames, ref value can have one of values from 0 to 4.

According to the conventional error concealment method described above, the macroblock of H.264 / AVC contains more information than the previous video coding standards. The technique does not properly consider the characteristics of the H.264 / AVC coding standard by performing error concealment in units of 16 × 16 based on BMA used in many existing video coding standards.

In addition, the complexity of the error concealment technique should be considered as important as the objective picture quality after the error concealment. This is because if a macroblock in an image is lost, the image is operated as a reference frame of the next image after concealing an error of the lost macroblock through an error concealment process. However, the conventional error concealment method in the conventional H.264 / AVC does not solve the high complexity efficiently.

The present invention was devised to solve the above-mentioned problems of the prior art, and includes information on neighboring macroblocks positioned up, down, left, and right in a mode of a lost macroblock in consideration of the characteristics of the H.264 / AVC coding standard. By estimating the error rate, and performing error concealment with different block sizes, H.264 can improve the objective image quality with less computational complexity than the conventional error concealment technique in H.264 / AVC. It is an object of the present invention to provide a method for concealing error in time direction.

The constituent means constituting the method for concealing a visual error in H.264 according to the present invention having the technical problem as described above comprises the steps of: identifying four macroblock modes located at the top, bottom, left, and right sides of the lost macroblock; Determining an error concealment mode having a predetermined number and size using a macroblock mode, and performing error concealment by calculating cost values for motion vectors and reference frame information by an OBMA method according to the determined error concealment mode Characterized in that comprises a.

The determining of the error concealment mode may include one or more of the upper and lower blocks of the lost macroblock having a macroblock mode other than mode 3 or mode 8, and one or more of the left and right blocks other than mode 2 or mode 8. In the case of having a macroblock mode, it is determined that the error concealment mode 1 conceals an error in one 16 × 16 unit.

In the determining of the error concealment mode, both the left and right blocks of the lost macroblock have a macroblock of mode 2 or mode 8, and at least one of the upper and lower blocks has a macroblock mode other than mode 3 or mode 8. In the case of the case, the error concealment mode 2 concealing the error with two block sizes of 16 × 8 is characterized in that it is determined.

The determining of the error concealment mode may include: both upper and lower blocks of the lost macroblock have a macroblock mode of mode 3 or mode 8, and at least one of the left and right blocks is a macroblock mode other than mode 2 or mode 8. In the case of having, the error concealment mode 3 concealing an error with two 8 × 16 block sizes is determined.

The determining of the error concealment mode may include: when both upper and lower blocks of the lost macroblock have a macroblock mode of mode 3 or mode 8, and both left and right blocks have a macroblock mode of mode 2 or mode 8 It is characterized in that the error concealment mode 4 concealing the error in four 8 × 8 block size.

Hereinafter, with reference to the accompanying drawings will be described in detail the operation and preferred embodiment of the method for concealing the error in the time direction in the present invention H.264 / AVC consisting of the above configuration means.

When the compressed bitstream is transmitted through a channel including noise, macroblocks in the image may be lost. The reconstruction method of the lost macroblocks can be divided into an error concealment method in space using an intra-spatial correlation in a natural image and a method of error concealment using a temporal correlation in a video.

An object of the present invention is to provide a forward error concealment scheme in consideration of coding characteristics of H.264 / AVC among an error concealment technique and a spatial error concealment scheme. This means that although several existing video coding standards have only one motion vector for one macroblock, a reference frame can be obtained with a maximum of 16 different motion vectors and a minimum of 8 × 8 size generated by estimating motion at least 4 × 4 in size. By determining, the error error concealment is performed using the coding feature of H.264 / AVC having up to four different reference frame information and macroblock mode information defining the size of a block used for motion estimation.

In H.264, motion is estimated with various block sizes, unlike previous coding standards. Modes of macroblocks having various block sizes used for the motion estimation are as shown in FIG. 1, which has been described above.

To sum up again, Mode 1, Mode 2, Mode 3, and Mode 8 represent 16 × 16 macroblock sizes, and Modes 4 through 7 represent 8 × 8 submacroblock sizes. That is, one macroblock estimates motion in the shape of a block of mode 1, mode 2, mode 3, and mode 8. When mode 8 is selected, each sub-macroblock moves in the shape of a block from modes 4 to 7 again. Estimate

In H.264, the motion vector cost function (MVcost), the reference frame cost function (REFcost) and the rate-distortion cost function (RDcost) are used to select the mode of the macroblock.

MVcost = WeightedCost (f, mvbits [(cx << s) -px] + mvbits [(cy << s) -py])

where, f: lambda factor (14)

The motion vector cost function is as shown in Equation (14) above. Here, cx and cy represent the motion vector of the current macroblock, s represents the transition amount of the motion vector, and px and py represent the predicted motion vector. The WeightedCost function returns a cost value by taking a bit amount used for encoding a motion vector prediction error using an f value.

REFcost = WeightedCost (f, refbits (ref))

Where f: lambda factor (15)

The reference frame cost function is expressed by Equation (15). Here ref refers to the selected reference frame, and the WeightedCost function returns the cost value by taking the amount of bits required to encode the selected reference frame value using the f value. In general, it has a very small value compared to the motion vector cost.

RDcost = Distortion + lambda × Rate (16)

The rate-distortion cost function is shown in equation (16). When the rate-distortion optimization technique is used, the rate-distortion cost function is used in the selection of the sub macroblock mode in mode 8 and the final selection of the entire macroblock mode and has a relatively high complexity compared to the two cost functions. The distortion value in Equation (16) can be calculated by obtaining the SNR in each macroblock mode, and the rate value is the bit rate of the compressed bitstream at the end of encoding of the macroblock. Indicates.

The macroblock mode decision in H.264 is divided into two types depending on whether the rate-distortion optimization technique is used. If the rate-distortion optimization technique is used, the macroblock mode that minimizes the rate-distortion cost value is determined. If the rate-distortion optimization technique is not used, the sum of the motion vector cost value and the reference frame cost value is used. Select the minimum macroblock mode.

The rate-distortion cost value and the motion vector cost value among the cost functions in H.264 are in proportion to each other because the magnitude of the prediction error through the motion estimation directly affects the bit rate. In other words, the macroblock mode is related to the magnitude of the prediction error after motion estimation regardless of the rate-distortion optimization technique. For example, if a macroblock is selected as mode 2, it can be expected that performing motion estimation with two 16 × 8 block sizes generates a minimum prediction error.

Accordingly, the present invention conceals the error more efficiently by predicting the mode of the macroblock in which the error occurs (loss) as the mode of the adjacent macroblock and variably selecting the size of the block used for the error concealment.

Hereinafter, with reference to the accompanying FIG. 3 will be described a procedure for a method of error concealment according to the present invention.

First, four macroblock modes located on the top, bottom, left, and right sides of the lost (error generated) macroblock are checked (S10). That is, as shown in FIG. 2, it is checked to which macroblock mode among the mode 1, the mode 2, the mode 3, and the mode 8 of the four macroblocks existing in the vicinity of the lost macroblock.

After checking the modes of the macroblocks located on the top, bottom, left, and right of the lost macroblock as described above, the analysis is performed on them and the contrast is performed (S20). That is, it is determined whether the macroblocks existing on the top, bottom, left, and right sides of the lost macroblock are all in mode 2 or mode 8, all in mode 3 or mode 8, or whether mode 2, mode 3, and mode 8 are mixed.

After checking and analyzing the modes of the macroblocks present in up, down, left, and right around the lost macroblock, the error concealment mode having a predetermined number and size is determined using the identified up, down, left, and right macroblock modes (S30 ~). S31, S40 to S41, S50 to S51, S60 to S61).

The error concealment mode is selected from one of four, as shown in FIG. 4, and each error concealment mode conceals an error with a different block size.

The four error concealment modes conceal errors in different block units, calculate cost values in the same manner as OBMA, and have different numbers of reference pixels according to each error concealment mode.

Four error concealment modes will now be described with reference to FIG. 4.

First, when at least one of the upper and lower blocks of the lost macroblock has a macroblock mode other than mode 3 or mode 8, and at least one of the left and right blocks has a macroblock mode other than mode 2 or mode 8, one 16 The error concealment mode 1 concealing the error in units of x 16 is determined (S30 to S31).

That is, the error concealment mode 1 has a macroblock mode other than the mode 3 or the mode 8 in which at least one of the upper and lower blocks of the macroblock in which the error occurs (lost) has a division component in the vertical direction, and at least one of the left and right blocks. It is selected when one has a macroblock mode other than Mode 2 or Mode 8 with the division component in the horizontal direction, in which case the error is concealed in units of 16 × 16, which is the same as that adopted in H.264. do.

Second, when both left and right blocks of the lost macroblock have a macroblock of mode 2 or mode 8, and at least one of the upper and lower blocks has a macroblock mode other than mode 3 or mode 8, two 16 × 8 blocks The error concealment mode 2 concealing the error in magnitude is determined (S40 to S41).

That is, the error concealment mode 2 has a macroblock mode of mode 2 or mode 8 in which both the left and right blocks of the macroblock in which the error occurs (lost) have a dividing component in the horizontal direction, and at least one of the upper and lower blocks are vertical It is selected if it has a macroblock mode other than Mode 3 or Mode 8 with components divided in the direction. In this case, error concealment is performed with two 16 × 8 block sizes.

Third, when both the upper and lower blocks of the lost macroblock have a macroblock mode of mode 3 or mode 8, and at least one of the left and right blocks has a macroblock mode other than mode 2 or mode 8, two 8 × 16 blocks are used. An error concealment mode 3 concealing an error with a block size of is determined as S50 to S51.

That is, the error concealment mode 3 has a macroblock mode of mode 3 or mode 8 in which all of the upper and lower blocks of the macroblock in which the error occurs (lost) have a division component in the vertical direction, and at least one of the left and right blocks are horizontal It is selected if it has a macroblock mode other than Mode 2 or Mode 8 with components divided in the direction. In this case, error concealment is performed in two 8 × 16 block sizes.

Fourth, when all of the upper and lower blocks of the lost macroblock have a macroblock mode of mode 3 or mode 8, and both the left and right blocks have a macroblock mode of mode 2 or mode 8, an error occurs with four 8 × 8 block sizes. Error concealment mode 4 concealing is determined as S60 to S61.

That is, the error concealment mode 4 has a macroblock mode of mode 3 or mode 8 in which both upper and lower blocks of an errored (lost) macroblock have a division component in the vertical direction, and both left and right blocks are in a horizontal direction. It is selected if you have Mode 2 or Mode 8 with the divisible components of. In this case, error concealment is performed with four 8 × 8 block sizes.

After one of the above four error concealment modes is determined, error concealment is performed by calculating a cost value for the motion vector and the reference frame information by the OBMA method according to the determined error concealment mode (S70).

The motion vector and reference frame information to be considered when the error concealment mode 1 is selected by the steps S30 to S31 are considered in the same manner as in the conventional method of H.264 / AVC using Equation (11). The cost value for each motion vector and reference frame candidate group is calculated using the following equation. In the following equations, OBM _T , OBM _B , OBM _L , and OBM _R represent the differences between pixels adjacent to the top, bottom, left, and right sides of the lost macroblock, respectively.

When the error concealment mode 2 is selected by the steps S40 to S41, the error concealment is performed in two 16 × 8 block sizes, and the motion vector and the reference frame considered for each subdivided block (block 0, block 1). The cost value for the candidate group is calculated using the following Equations (17) and (18).

At this time, the motion vector candidate group considered for each block is as follows.

Block 0: {ZM, V ₁ , V ₂ , V ₃ , V ₇ }

Block 1: {ZM, V ₄ , V ₅ , V ₆ , V ₈ }

Equations (17) and (18) represent cost value calculations for each motion vector and reference frame in each block of the error concealment mode 2, and P _{x, y} is represented at the (x, y) position of the current frame. P ^r _{x, y} represents a pixel value at the (x, y) position of the reference frame.

In addition, the V ₁ to V ₈ refers to eight 8 × 8 sized blocks located around the lost macroblock shown in FIG. 2. That is, the motion vector candidate group considered for block 0 uses the V ₁ , V ₂ , V ₃ , V ₇ blocks and ZM vectors that exist around block 0, and the motion vector candidate group considered for block 1 is a ZM vector. And V ₄ , V ₅ , V ₆ , and V ₈ blocks.

When the error concealment mode 3 is selected by the steps S50 to S51, the error concealment is performed with two 8 × 16 block sizes, and the motion vector and reference considered for each subdivided block (block 0, block 1) The cost value for the frame candidate group is calculated using Equations (19) and (20) below.

At this time, the motion vector candidate group considered for each block (block 0, block 1) is as follows.

Block 0: {ZM, V ₁ , V ₃ , V ₄ , V ₅ }

Block 1: {ZM, V ₂ , V ₆ , V _7, V ₈ }

Equation (19) and Equation (20) represent cost value calculations for each motion vector and reference frame in each block of error concealment mode 3, and P _{x, y} is represented at the position (x, y) of the current frame. P ^r _{x, y} represents a pixel value at the (x, y) position of the reference frame.

In addition, the V ₁ to V ₈ refers to eight 8 × 8 sized blocks located around the lost macroblock shown in FIG. 2. That is, the motion vector candidate group considered for block 0 uses the V ₁ , V ₃ , V ₄ , V ₅ blocks and ZM vectors that exist around block 0, and the motion vector candidate group considered for block 1 is a ZM vector. And V ₂ , V ₆ , V _7, V ₈ blocks.

Finally, when the error concealment mode 4 is selected by the steps of S60 to S61, the error concealment is performed in four 8 × 8 block sizes, and each subdivided block (block 0, block 1, block 2, block 3) The cost value for the motion vector and the reference frame candidate group considered for Equation 2 is calculated using Equation (21) to Equation (24) below.

At this time, the motion vector candidate group considered for each block (block 0, block 1, block 2, block 3) is as follows.

Block 0: {ZM, V ₁ , V ₃ }

Block 1: {ZM, V ₂ , V _7, }

Block 2: {ZM, V ₄ , V ₅ }

Block 3: {ZM, V ₆ , V ₈ }

Equation (21) to Equation (24) represent cost value calculations for each motion vector and reference frame in each block of the error concealment mode 4, and P _{x, y} denotes the (x, y) position at the current frame. P ^r _{x, y} represents a pixel value at the (x, y) position of the reference frame.

In addition, the V ₁ to V ₈ refers to eight 8 × 8 sized blocks located around the lost macroblock shown in FIG. 2. That is, the motion vector candidate group considered for block 0 uses V ₁ , V ₃ blocks and ZM vectors that exist in the vicinity of block 0, and the motion vector candidate group considered for block 1 includes ZM vectors, V ₂ , and V _7. A motion vector candidate group considering a block and using block 2 uses ZM vectors and V ₄ and V ₅ blocks, and a motion vector candidate group considering block 3 uses ZM vectors and V ₆ and V ₈ blocks.

By the above procedure, it is possible to conceal an error for a lost macroblock.

In error concealment techniques, complexity must be considered as an important performance as well as objective picture quality. This is because a frame containing an error block during decoding is used as a reference frame in decoding the next frame after the error concealment process is finished. Therefore, a high complexity error concealment technique can significantly affect real-time decoding.

The present invention performs error concealment with four block sizes according to the mode of the neighboring macroblocks. If each error concealment mode has a different amount of calculation and information of all macroblocks of up, down, left, and right is available, the calculation amount according to each error concealment mode for one macroblock is shown in Table 1 below.

ECMode Addition Subtraction Comparison ECMode1 9 576 9 ECMode2 10 320 10 ECMode3 10 320 10 ECMode4 12 192 12

Table 1

If all error blocks are selected as error concealment mode 1, all error blocks perform error concealment in units of 16 × 16, and in this case, they have the same amount of computation as the conventional error concealment technique in H.264.

In the case of the error concealment mode 2, the error concealment mode 3, and the error concealment mode 4, the motion vector group under consideration is limited according to the error concealment mode and thus has a smaller amount of calculation than the error concealment mode 1.

In Table 1, the difference between the number of subtractions is more than just the number of subtractions. This takes into account the complexity used in interpolation because motion estimation in H.264 is performed in units of 1/4 pixels, and subtraction is performed by interpolating the reference frame when the candidate motion vector has 1/2 or 1/4 units. In reality, the complexity of each mode shows a lot of difference.

Therefore, the present invention adaptively selects the error concealment block size using the block mode of the neighboring macroblock and restricts the motion vector candidate group according to each error concealment block mode. Has low complexity.

According to the visual error concealment method of the present invention H.264 / AVC having the above-described configuration and operation and preferred embodiments, the motion vector and the reference frame information considered according to each error concealment mode are lost. By estimating the information contained by the macroblocks located above, below, left, and right of the macroblock, and performing error concealment with different block sizes, the computational error is less than that of the conventional error concealment method in H.264 / AVC It also has the effect of providing improved image quality.

Claims (5)

In the method of concealing the error in time in H.264, Identifying four macroblock modes located on the top, bottom, left, and right sides of the lost macroblock; Determining an error concealment mode having a predetermined number and size using the identified up, down, left, and right macroblock modes; And calculating the cost values for the motion vector and the reference frame information by the OBMA method according to the determined error concealment mode to perform error concealment. The method of claim 1, wherein the determining of the error concealment mode, If at least one of the upper and lower blocks of the lost macroblock has a macroblock mode other than mode 3 or mode 8, and at least one of the left and right blocks has a macroblock mode other than mode 2 or mode 8, one 16 × 16 An error concealment method in H.264, characterized in that the error concealment mode 1 conceals an error in units. The method of claim 1, wherein the determining of the error concealment mode, When both left and right blocks of the lost macroblock have a macroblock of mode 2 or mode 8, and at least one of the upper and lower blocks has a macroblock mode other than mode 3 or mode 8, two 16 × 8 block sizes are used. A method for visual error concealment in H.264, characterized in that the error concealment mode 2 concealing errors is determined. The method of claim 1, wherein the determining of the error concealment mode, If both of the upper and lower blocks of the lost macroblock have a macroblock mode of mode 3 or mode 8 and at least one of the left and right blocks has a macroblock mode other than mode 2 or mode 8, two 8 × 16 block sizes The error concealment method in H.264 characterized in that the error concealment mode 3 concealing the error is determined. The method of claim 1, wherein the determining of the error concealment mode, If both the upper and lower blocks of the lost macroblock have a macroblock mode of mode 3 or mode 8, and both the left and right blocks have a macroblock mode of mode 2 or mode 8, conceal an error with four 8 × 8 block sizes. The error concealment method in H.264 characterized in that the error concealment mode 4 is determined.

Images