KR20030065694A - Error concealment method of Itra coded frame by spatio-temporal information - Google Patents

Abstract

PURPOSE: A method of hiding an error in intra frame using spatio-temporal information is provided to effectively hide even an error that is generated in continuous macro blocks. CONSTITUTION: A macro block I having an error is detected from intra frame(S1). A macro block B of B-frame corresponding to the detected macro block is found from neighboring B-frame of the previous group of picture(S2). A backward motion vector is found from information about the macro block of B-frame, to search for a macro black B' that has been moved from the position of the macro block of B-frame by the backward motion vector(S3). A macro block is selected from macro blocks of B-frame including the macro block B' and a forward motion vector is found from information of the selected macro block, to search for a reference macro block P of a corresponding anchor frame(S6). Each pixel data of the reference macro block of the anchor frame is copied as pixel data of the macro block I having the error(S8).

Description

Error concealment method of Itra coded frame by spatio-temporal information}

The present invention relates to an error concealment technique for concealing an error in an image encoding technique according to an MPEG-2 algorithm, and more particularly, to a method of concealing an error generated in an intra frame using space-time information. will be.

In general, digital TV / HDTV signals use signals compressed at high compression rates using the MPEG-2 algorithm due to limited channel bandwidth. In MPEG-2, a compressed signal is formed into a hierarchical bit stream by motion compensation, hybrid discrete cosine transform, variable length coding, and adaptive quantization. Send it to the transmission channel. At this time, the compressed data sent to the transmission channel may be lost or damaged due to transmission error such as channel noise or physical channel damage. The transmission error that damages the compressed data is mainly caused by random bit error and There is an erasure error.

Random bit errors are generated by defects in the physical channel. When using fixed-length coding, only one codeword is affected, but variable-length coding can cause loss of synchronization of coded information. Eraser errors are caused by packet loss in packet networks, burst errors in storage media due to physical faults, system faults for short periods of time, and random bits due to loss or corruption of consecutive segments of bits. Cause much more serious damage.

The conventional error concealment technique mainly used in MPEG-2 to prevent deterioration of image quality from data loss or damage due to such transmission error is to recover or estimate information lost by transmission error in the decoder. This conventional error concealment technique can be largely divided into two types of temporal error concealment and spatial error concealment.

The temporal error concealment scheme uses temporal redundancy in one sequence, which is used in intercoding frames with motion information, while the spatial error concealment scheme is used in one frame. By using spatial redundancy, it is used for an intra coding frame having no motion information.

In addition, the temporal error concealment technique is divided into a simple temporal error concealment scheme and a motion compensation temporal error concealment scheme. Simple temporal error concealment is a method of using a value in a previously transmitted anchor frame that matches the part where an error is found in the current frame. Therefore, this method is simple to implement, but there is a drawback that if there is movement between the current frame and the previous anchor frame, visual errors due to the movement between the frames may be seen. The motion compensation temporal error concealment method is a method that compensates for the shortcomings of the simple temporal error concealment method. If the motion vector of the current frame exists, the error processing method is performed by using the motion vector of the neighboring macroblock of the error part. to be. The motion compensation temporal error concealment method is more effective than the simple temporal error concealment method. However, if the motion vector of the neighboring macroblock is damaged, a large error may occur in the reconstructed image due to incorrect prediction of the motion vector. have.

There are two methods of spatial error concealment: a method of concealing an error in block units of a macroblock in which an error occurs, and a method of concealing an error in units of one macroblock. A method of concealing errors on a block-by-block basis is to conceal the error by using pixels adjacent to each block of the four blocks of pixels constituting the macro block in which the error occurs. This technique achieves good results when there are no errors in the four surrounding macroblocks of the macroblock where the error occurs. However, if the surrounding macroblocks are damaged, the error cannot be concealed properly. For example, if an error of a continuous macroblock occurs (slice error), an error also occurs in the left and right macroblocks, and the error is correctly hidden because only the pixels of the upper and lower macroblocks conceal the error. You cannot hide it. The method of concealing errors in units of macroblocks is a method of restoring each pixel of the entire macroblock with four macroblocks around which an error occurs and is mainly used in MPEG-2 video. That is, when an error occurs in a slice, which is a video transmission unit in MPEG-2, and several consecutive macroblocks are damaged, the left and right macroblocks cannot be used for error concealment. Therefore, the error is concealed using only the macroblocks above and below the macroblock in which the error occurs.

This spatial error concealment method can achieve good results when the neighboring macroblocks of the macroblock in which the error occurs are present properly. However, in the block-based error concealment method, when the color of the neighboring block is the same as the color of the current block, the error is almost exactly restored, but when there is a color difference between the neighboring blocks, blur may occur. In addition, the error concealment method based on macroblocks does not show the blur phenomenon in the error concealment method based on the block, but when there is a boundary in one macroblock, a blur phenomenon may also occur. There is a problem that image quality deteriorates when an error occurs.

The present invention has been proposed to solve the above-described problems, and an object thereof is to provide an error concealment method for an intra frame using spatiotemporal information for concealing that an error has occurred in an intra frame using spatiotemporal information.

1 is a schematic diagram of a typical MPEG-2 video encoder to which the present invention may be applied;

2 is a schematic diagram of an exemplary MPEG-2 video decoder to which the present invention may be applied;

3 is a structure of a GOP according to an embodiment of the present invention,

4 is a flowchart showing an error concealment method according to the present invention;

5 is a schematic diagram illustrating a concept of finding an error occurrence corresponding block of an I frame in a B frame according to the present invention;

6 is a schematic diagram illustrating the application of a backward motion vector to a corresponding block of a B frame according to the present invention;

7 is a schematic diagram for explaining finding a reference macroblock of a P frame with a forward motion vector of a macroblock of a B frame according to the present invention;

8 is a schematic diagram illustrating a concept of concealing an error of an I frame by compensating a distance to a reference macroblock of a P frame according to the present invention.

☞ Explanation of symbols for main parts of drawing

102,124,212: frame memory 104,122,210: adder

106: Discrete Cosine Converter 108: Quantizer

110: variable-length encoding and multiplexer 112, 202: buffer

114: rate controller 116: complexity calculator

118,206 Inverse Quantizer 120,208 Inverse Discrete Cosine Converter

126,214: Adaptive Prediction 128: Motion Estimation

204 demultiplexing and variable length decoder

In order to achieve the above object, the method of the present invention provides a method for detecting an error detected in an intra frame while decoding a bitstream in which an image frame consisting of a series of picture groups consecutive on a time series is encoded and transmitted by MPEG-2 encoding. CLAIMS 1. A method for concealing, the method comprising: detecting a macroblock I (i, j) in which an error occurs in an intra frame; Finding the macroblock B (i, j) of the B frame corresponding to the macroblock I (i, j) in the adjacent B frame of the previous GOP; Backward motion vector by the information for the macroblock of the B frame, _{B (i, j) MV B} (i, j) to find the B-frame macro-block B (i, j) reverse motion vector macro of the moved position as from the position Finding block B '(i, j); Macroblock B corrected by finding and averaging each backward motion vector from information on macroblocks of frame B including macroblock B '(i, j) and correcting macroblock B' (i, j) obtaining ((i, j); distance d between base corrected macroblocks of macroblocks of frame B including the corrected macroblock B " (i, j) and the macroblock B '(i, j) obtaining (m, n); Finding a reference macroblock P (i, j) of the anchor frame by finding a forward motion vector MV _{F (i, j)} in the information on the base macroblock; Obtaining a corrected reference macroblock P '(i, j) by correcting the position of the reference macroblock of the anchor frame by the distance; And copying each pixel data of the corrected reference block as pixel data of the macroblock I (i, j) in which the error occurs.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram of a typical MPEG-2 video encoder to which the present invention may be applied, the encoder comprising a frame memory 102, an adder 104, a discrete cosine transformer 106 (DCT), and an adaptive quantizer 108 (Q). Variable length coding and multiplexer 110 (VLC & MUX), buffer 112, rate controller 114, complexity calculator 116, dequantizer 118 (DeQ), inverse discrete The cosine transformer 120 includes an IDCT, an adder 122, a frame memory 124, an adaptive predictor 126, and a motion estimation 128.

MPEG video algorithms are based on two basic techniques: block based motion compensation to reduce temporal redundancy and discrete cosine transform to reduce spatial redundancy. Depends. In the case of MPEG-2 video source coding, a coding scheme based on motion-compensated mixed discrete cosine transform is used, which includes an I frame called an intra coding frame and a P and B called an inter coding frame. Three types of frames are used, including frames.

An I frame is coded using its own information without reference to another frame, used as a random access point in the bit stream, and used as a reference frame of P and B frames. P frames use forward motion prediction from previous I or P frames, and B frames use forward motion prediction and next I or P frames from previous I or P frames. Bidirectional motion prediction is used, such as backward motion prediction from. In this case, the I frame and the P frame are called anchor frames.

In addition, several frames are grouped into one group and called a picture group (GOP: Group Of Picture), and one GOP includes only one I frame. Each frame consists of a slice, one slice consists of several macroblocks, and each macroblock consists of four blocks (8 x 8 pixels). block).

Referring to FIG. 1, a series of pictures for forming a GOP are sequentially stored in the frame memory 102. In the case of intra coding a frame stored in the frame memory 102 into I frames, since I frame coding has no motion prediction, the discrete cosine transformer 106, the adaptive quantizer 108, and the variable length encoder 110 pass through the adder 104. Intra coding is performed in units of blocks or macroblocks.

The discrete cosine transform in the discrete cosine transformer 106 consists of 8 x 8 block units, which move the information of the frame to the frequency domain so that all the energy of the block is concentrated on the DC value of the block. Here, the discrete cosine transform coefficient obtained by the discrete cosine transform process is divided into a low frequency (DC) component and a high frequency (AC) component and independently quantized. The low frequency (DC) component is a low frequency (DC) component of the previous macroblock. The difference value as the predicted value is encoded, and the high frequency (AC) component is encoded after zigzag scanning for each block and arranged in a line. The quantizer 108 quantizes the discrete cosine transform coefficient using a weighting matrix and an appropriately sized quantization step in consideration of human visual characteristics and coding efficiency, taking into account the complexity and transmission rate of a frame to be encoded. To adjust the quantization step. In other words, in MPEG-2, the feed rate is adjusted by combining a feed-back method and a feed-forward method. To this end, the rate controller 114 performs the fullness and complexity calculation unit 116 of the buffer 112. The quantization step size is varied in consideration of the complexity of

On the other hand, when inter-coding frame data stored in the frame memory 102 into P or B frames, a motion vector predicted by a block matching algorithm (BMA) is used. For the P frame, the forward motion vector predicted from the previous I or P frame is used, and for the B frame, the forward and backward motion vectors predicted from the previous I or P frame and the next I or P frame are used.

In addition, MPEG-2 uses two methods of motion compensation prediction, that is, a frame unit and a field unit. In the case of a frame unit, one frame becomes one entire picture and one motion vector is included for each macro block. . On the other hand, in a field unit, one field becomes an interlaced half frame, and two motion vectors are included in one macroblock, one per field. In the encoder, the prediction error generated by the motion compensation prediction method is calculated and inserted into the bit stream, and the prediction error is compressed using a discrete cosine transform, like an intra coding macroblock, with a reduction in time redundancy. At this time, the motion-related information is transmitted along with the spatial information based on the 16x16 block, and the motion information is compressed using variable length coding for maximum efficiency. As such, variable length coding is used when coding information such as MPEG-2 type video prediction errors and motion vectors, thereby being very sensitive to errors generated during transmission.

Referring back to FIG. 1, after intra coding of an I frame is performed, in the frame memory 102, an inverse quantized by an inverse quantizer 118 and an inverse discrete cosine transformed by an inverse discrete cosine transformer 120 to restore a reference frame. The data is stored. In the case of inter coding, motion estimation is generally performed in units of macroblocks. The motion estimation unit 128 finds a macroblock most similar to a macroblock to be coded in a previous reference frame, and obtains a motion vector. The reference block designated by the vector is found and output to the adder 104. The adder 104 outputs a difference (prediction error) between the corresponding macroblock and the reference block of the frame to be coded, and the discrete cosine transformer 106 performs discrete cosine transform on it. The quantizer 108 quantizes the discrete cosine transform coefficient, and the variable length encoder and multiplexer 110 outputs the quantized prediction error and additional information (motion vector, quantization information, etc.) output from the quantizer 108. After variable length coding, multiplexing is performed.

The bit stream coded in the MPEG-2 encoder is transmitted through the transmission medium and decoded in the MPEG-2 video decoder as shown in FIG.

Referring to FIG. 2, the MPEG-2 video decoder includes a buffer 202, a demultiplexer and a variable length decoder 204 (DeMUX & VLD), an inverse quantizer 206 (DeQ), an inverse discrete cosine transformer 208 (IDCT), The adder 210, the frame memory 212, and the adaptive prediction unit 214. The video data of the I frame received by the buffer 202 is demultiplexed and variable-length decoded by the demultiplexer and variable length decoder 204, and then dequantized by the dequantizer 206, and deintersected by the inverse discrete cosine transformer 208. The inverse discrete cosine transform is performed and stored in the frame memory 212 via the adder 210. The data of the intercoded P or B frame is transmitted to the adder 210 through the inverse quantizer 206 and the inverse discrete cosine transformer 208 like the I frame, and received by the received motion vector (MV). The adaptive prediction unit 214 adds the predicted reference block and restores the original source data.

3 is a structure of a group of picture (GOP) according to an embodiment of the present invention, one GOP is composed of I, B1, B2, P, B1, B2, P, B1, B2, P, B1, B2 It consists of two pictures. Here, the I picture is an I frame, the B1 and B2 pictures are B frames, and the P picture is a P frame. And it can be seen that two B frames are included between I or P frames.

Referring to FIG. 3, in the embodiment of the present invention, when an error occurs in an I frame that is a first frame of a current GOP received after a previous GOP in a GOP arranged sequentially on a time series, an I frame of the current GOP is displayed in the previous GOP. Using the motion vector of the nearest B1 or B2 frame in time, the corresponding macroblock of the previous anchor frame (the last P frame of the previous GOP) is copied to the I frame to conceal the error. That is, the present invention is to process the slice error in the I frame that does not have motion information based on MPEG-2. In general coding schemes, I frames do not use motion prediction, but in H.263 + and MPEG-4, there are options for predicting motion.

As described above, the method for concealing an error generated in an I frame according to the present invention includes detecting an error in a current I frame (S1) and finding a corresponding macroblock in an adjacent B frame (as shown in FIG. 4). S2), compensating the backward motion (S3), compensating the motion vector (S4), calculating the distance (S5), finding a reference macroblock in the anchor frame (S6), compensating for the calculated distance Step S7 is followed by copying from the anchor frame to the I frame to conceal the error (S8).

Referring to FIG. 4, in step S1, the macroblock I (i, j) in which an error occurs in an intra (I) frame is detected.

In step S2, as shown in FIG. 5, the macroblock B (i, j) of the B frame corresponding to the macroblock I (i, j) is found in the adjacent B frame of the previous GOP.

Referring to FIG. 5, since there are two B frames B1 and B2 between the I frame of the current GOP and the last P frame of the previous GOP, the macroblock and the B2 frame of the B1 frame corresponding to I (i, j) of the I frame. You can find the macroblock of. As described above, although the present invention can use either of the two B frames, it is more efficient to use the B2 frame that is closer to the I frame in time. The embodiment of the present invention uses a B2 frame, which is the last frame of the previous GOP.

In step S3, as shown in FIG. 6, the backward motion vector MV _{B (i, j)} is found from the information on the macroblock B (i, j) of the B frame, and the macroblock B (i, j) position of the B frame is found. Find the macroblock B '(i, j) at the position shifted by the backward motion vector from.

As described above, in encoding, the B frame not only predicts the forward motion from the P frame before the B frame but also predicts the backward motion from the I frame of the next GOP, which is an anchor frame. Therefore, the macroblock I (i, j) of the frame in which the error occurs is located at the position moved by the backward motion vector in the macroblock at the same position of the frame B, which is the previous frame. For this backward motion vector by the information for the macroblock in B-frame _{B (i, j) MV B} (i, j) found in the moved position vector by a reverse motion from the macroblock of the B frame, B (i, j) where the Find the macroblock B '(i, j).

As such, the macroblock B '(i, j) at the position corresponding to I (i, j) is expressed by Equation 1 below.

Subsequently, in step S4, each backward motion vector is found from the information on the macroblocks of the B frame including the macroblocks B '(i, j), averaged, and corrected to correct the macroblocks B' (i, j). Macroblock B " (i, j) is obtained. That is, macroblocks B '(i, j) corresponding to I (i, j) obtained in step S3 have four surroundings as shown in FIG. The backward motion of these four macroblocks is represented by macroblocks B (i-1, j-1), B (i, j-1), B (i-1, j), and B (i, j). In the present embodiment, macroblocks B '(i, j) may correspond to any one macroblock according to the backward motion vector value. In this case, it may be represented by one macroblock, and the mean may be a weighted average or another method such as median depending on the degree to which B '(i, j) is included. As it may be obtained.

The macroblock B " (i, j), thus corrected in detail, is obtained as in Equation 2 below.

In step S5, the distance d (m, n) of the corrected macroblock B " (i, j) and the base macroblock among the macroblocks of the B frame including the macroblock B '(i, j) is obtained. Four surrounding macroblocks B (i-1, j-1), B (i, j-1), B (i-1, j), B (containing macroblocks B '(i, j) The macro block to be used for reference to the previous P frame among i, j) is called a 'base macroblock', and in the embodiment of the present invention, B (i-1, j-1) is used as the base macroblock. The distance d (m, n) between the macroblock B (i-1, j-1) and the corrected macroblock B " (i, j) is obtained as shown in Equation 3 below.

In step S6, as shown in FIG. 7, the forward motion vector MV _{F (i, j)} is found in the information on the base macroblock to find the reference macroblock P (i, j) of the anchor frame.

Since the macroblock B "(i, j) of the B frame corresponding to the error I (i, j) is not a macroblock of the anchor frame, the position must be determined using the last P frame of the previous GOP, which is an anchor frame. This is because the B frame is encoded together with the forward motion prediction value and the prediction error from the P frame used together with the I frame. Therefore, the B frame including the reference point of the macroblock B " (i, j) shifted by an average value. A macro corresponding to I (i, j) in the P frame, which is the previous anchor frame, by using the forward motion vector MV _{F (i-1, j-1)} of the macroblock B (i-1, j-1) of Find block P (i, j).

Subsequently, in step S7, the macroblock P (i, j) of the anchor frame is corrected by the distance d (m, n) obtained in step S5, and the corrected macroblock P '(i, j) is obtained. As shown, each pixel data of the corrected macroblock P '(i, j) is copied as pixel data of the macroblock I (i, j) in which the error is generated, thereby macroblock I (i, j) of I frame. Conceal an error from

The error concealment of the macroblocks I (i, j) of the I-frame in which the error is generated from the macroblock of the P-frame is expressed by Equation 4 below.

As shown in Equation 4, the macroblock found from the B frame in the P frame is moved by the forward motion vector and moved again by the distance d (m, n) to determine the P frame corresponding to the macroblock I (i, j). Find the macroblock P '(i, j) and copy it to the I (i, j) position of the I frame.

As described above, according to the present invention, when an error occurs in an I frame that does not have motion information, an error occurrence macro of an I frame in a previous anchor frame using a bidirectional motion vector of a B frame of a previous GOP adjacent to the I frame. By copying the macroblock corresponding to the block to the corresponding position of the I frame to conceal the error, it is possible to prevent blur in the conventional spatial error concealment method, and in particular, errors occur in successive macroblocks (for example, slices). In this case, there is an effect that can effectively hide the error.

Claims (6)

A method for concealing an error detected in an intra frame while decoding a bitstream in which an image frame consisting of a series of picture groups consecutive in time series is encoded and transmitted by MPEG-2 encoding method, Detecting a macroblock I (i, j) having an error in an intra (I) frame; Finding a macroblock B (i, j) of a B frame corresponding to the macroblock I (i, j) in a neighboring B frame of a previous GOP; Backward motion vector by the information for the macroblock of the B frame, _{B (i, j) MV B} (i, j) to find the B-frame macro-block B (i, j) reverse motion vector macro of the moved position as from the position Finding a block B '(i, j); By selecting any one of the macroblocks of the frame B including the macroblock B '(i, j), the forward motion vector MV _F is found from the information of the macroblock, and the reference macroblock P ( i, j) finding a fourth step; And And copying each pixel data of the reference macroblock of the anchor frame as pixel data of the macroblock I (i, j) in which the error occurs, wherein the error concealment method of an intra frame using spatiotemporal information is included. . The method of claim 1, wherein the fourth step A macroblock corrected by finding and averaging each backward motion vector from information on each macroblock of a B frame including the macroblock B '(i, j) and correcting the macroblock B' (i, j) Obtaining B "(i, j), Obtaining a distance d (m, n) between a base macroblock among macroblocks of frame B including the corrected macroblock B " (i, j) and the macroblock B '(i, j); An error concealment method for an intra frame using spatiotemporal information, characterized in that it comprises. The error concealment method of an intra frame according to claim 2, wherein the average of the backward motion vectors is a weighted average. The error concealment of an intra frame using spatiotemporal information according to claim 2, wherein the base macroblock is the first macroblock among four macroblocks of a B frame including the macroblock B '(i, j). Way. The method of claim 2, wherein the fifth step The reference macroblock P (i, j) of the anchor frame is corrected by the distance d (m, n) to obtain a corrected reference macroblock P '(i, j), and each pixel data of the corrected reference block is obtained. And copying as pixel data of the macroblock I (i, j) in which the error occurs. The method according to any one of claims 1 to 5, wherein the B frame is any one of a B1 or B2 frame located between an I frame of a current GOP and a P frame of a previous GOP, and the anchor frame is the P frame. An error concealment method for intra frames using spatiotemporal information.