KR100388802B1 - apparatus and method for concealing error - Google Patents

Abstract

The error concealment apparatus and method are to prevent the degradation of the image quality by using the spatial / temporal correlation of the neighboring blocks lost at the receiver with respect to the error occurring on the channel. And a transport stream decoding unit for decoding a bit stream according to a control signal of the controller, and decoding the transport stream decoded by the transport stream decoding unit to a variable length and attaching an error flag when an error occurs. A variable length decoder to decode the signal decoded by the variable length decoder into a video signal, store the same in the memory, and display the same through the display unit, and conceal and compensate an error according to an error flag of the variable length decoder. Consists of error concealment , In an error concealment method to restore the lost block, a macro block by applying a cost function of the neighborhood around the lost block to determine the spatial and temporal correlation between the previous frame motion vectors Detecting candidate motion vectors having the smallest error among the steps; detecting motion vectors having the smallest expression within the search range of the motion vectors using the candidate motion vectors; and using the detected motion vectors, a previous frame is detected. The idea is to replace the lost blocks with blocks within it.

Description

Error concealment and method {apparatus and method for concealing error}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to moving picture decompression systems, such as digital television or digital videoconferencing systems, and more particularly, to error concealment apparatus and methods in moving picture decompression systems such as digital television or digital video conferencing systems.

Recently, products and communication media for digital video services have been rapidly developed.

Among them, video application technologies using video standardization standards such as MPEG, H.261, and H.263 are providing video services to general homes through ATM or over-the-air communication.

In the case of providing various video services through the current communication media, if an error occurs on the transmission channel, the quality of the reconstructed image may be greatly degraded.

The video coding schemes use discrete cosine transform, variable length coding, and motion compensation coding to increase the compression rate. The information is lost.

In addition, since the influence of the damaged portion is continued over several subsequent frames, serious image quality degradation occurs when the decoder decodes the bit stream transmitted through the encoding process as described above.

In order to solve these problems, error concealment techniques have been proposed to compensate for the lost information and recover the original image, and the error concealment techniques are largely space predictive error concealment technique, temporal prediction error concealment technique, and error concealment using additional information. Techniques, etc.

The spatial prediction concealment technique is a typical technique for compensating for image quality in a spatial domain with respect to a damaged block, and may include a projection-based error compensation technique.

However, this technique can guarantee good image quality compensation if the damaged block is small enough for the image size, but it requires a large amount of computation at the receiver.

In addition, some straight lines can be restored by a complex algorithm, but curves and textures are smoothed.

The temporal prediction concealment technique is a method of compensating a lost image block using a motion vector in a temporal domain.

This method has the advantage of low complexity and quick compensation, but the reliability of compensation is low.

Especially in case of fast movement or irregular movement between two consecutive frames, performance is not good.

In addition, there are methods of concealing an error when an error occurs by inserting additional information in preparation for an error in the user data part.

However, there is a disadvantage in that the amount of data increases due to the insertion of additional information.

1A and 1B illustrate a conceptual diagram of a time prediction technique of an error concealment method according to the prior art, in which information of a lost macro block E1 is best represented in a previous frame PF as shown in FIG. 1A. The information of the similar macro block E'1 is replaced by the motion information of the blocks C1 to C3 around the macro block E1 lost in the current frame CF, as shown in FIG. 1B. To replace the most similar macro block in the previous frame PF.

However, the error concealment apparatus and method according to the prior art show good performance when there is no change in the current frame and the previous frame, but when there is a large change in the movement between the frames or when the change of surrounding blocks and the boundary such as an edge or a corner is severe There is a problem that the performance is degraded.

Accordingly, the present invention has been made to solve the above problems, and provides an error concealment apparatus for preventing the deterioration of image quality by using the spatial / temporal correlation of the neighboring blocks lost at the receiver with respect to the error occurring on the channel. Its purpose is to.

Another object of the present invention is to provide an error concealment method corresponding to the above apparatus.

1A and 1B are conceptual views illustrating a time prediction technique in the error concealment method according to the prior art.

2 is a block diagram illustrating an error concealment apparatus according to the present invention.

3 is a diagram illustrating a linear predictive block boundary matching method according to an error concealment method according to the present invention.

4 illustrates a linear predictor of an error concealment apparatus according to the present invention.

5 is a flowchart showing an operation state by the error concealment method according to the present invention.

Explanation of symbols for main parts of the drawings

2: transport stream decoder 3: variable length decoder

4: video decoder 5: memory

6: error concealment unit 11: first register

12: first mixer 13: second register

14 second mixer 15 adder

16, 17: third and fourth register 18: subtractor

19: calculator 20: accumulator

A feature of the error concealment apparatus according to the present invention for achieving the above object is, in the error concealment device having a control unit, a memory and a display unit, a transceiving bit stream in accordance with the control signal of the control unit A variable length decoder which decodes the transport stream decoded by the transport stream decoder to a variable length and attaches an error flag when an error occurs, and a signal decoded by the variable length decoder as a video signal. After decoded and stored in the memory and displayed on the display unit, when the error flag signal is output from the variable length decoder, a linear prediction coefficient is obtained using information of the lost block and adjacent blocks, and then the linear Prediction coefficients and blocks of adjacent blocks The beam is linearly predicted using the beams to predict the brightness values of the boundary pixels in the lost block, and the absolute difference between the brightness values of the linear predicted pixels and the brightness values of the boundary pixels of the block in the previous frame corresponding to the lost block is minimized. And an error concealer for estimating a motion vector and replacing a lost block with a block in a previous frame obtained by using the estimated motion vector.

The error concealment method of restoring a lost block according to the present invention is characterized by (a) obtaining a linear prediction coefficient using pixel information of the lost block and adjacent blocks, and then, the block adjacent to the linear prediction coefficient and the lost block. Linearly predicting brightness values of boundary pixels in the lost block using the pixel information of the pixels, and (b) among the at least one or more candidate motion vectors, the brightness values of the linearly predicted pixels and the previous ones corresponding to the lost blocks. Determining a candidate motion vector as an initial motion vector having a minimum absolute difference with brightness values of boundary pixels of a block in a frame, and (c) setting the initial motion vector as an initial position and then all possible motion vectors in a search region. A boundary value of a block in a previous frame corresponding to the lost value and a brightness value of the linearly predicted pixels among And estimating a motion vector which is the absolute difference between the minimum brightness value, (d) is makin comprises the step of replacing the lost block with a block in the previous frame obtained using the estimated motion vector.

Hereinafter, a preferred embodiment of the error concealment apparatus and method according to the present invention will be described with reference to the accompanying drawings.

FIG. 2 is a block diagram illustrating an error concealment apparatus according to the present invention, wherein the control unit 1 collectively controls the operation of the entire circuit and a bit stream according to a control signal of the control unit 1. A transport stream decoder 2 for decoding, a variable length decoder 3 for decoding the transport stream decoded by the transport stream decoder 2 to a variable length and attaching an error flag when an error occurs; A video decoder 4 which decodes the variable-length decoded signal by the variable length decoder 3 into a video signal, stores it in the memory 5 and then displays it through the display unit 7, and the variable length decoder 3 And an error concealment section 6 for concealing the error in accordance with the error flag.

3 is a diagram illustrating a linear predictive block boundary matching method according to an error concealment method according to the present invention.

4 shows a linear predictor of an error concealment apparatus according to the present invention, wherein a first register 11, a first mixer 12, a second register 13, a second mixer 14, The adder 15, the third and fourth registers 16 and 17, the subtractor 18, the calculator 19, and the accumulator 20 are formed.

5 is a flow chart showing an operation state by the error concealment method according to the present invention.

The error concealment apparatus and method according to the present invention configured as described above will be described in detail with reference to the accompanying drawings.

Most video standardization standards process video into 16 * 16 blocks called macro blocks, and several macro blocks are grouped into slices.

Therefore, if an error occurs in the slice unit, several macro blocks are lost in succession.

To conceal and compensate for the lost block, the receiver goes through a channel decoder such as Reed-Solomon, a forward error correction code, on the received bit stream. The transport_error_indicator signal is set to 1 in the bit stream and output to the transport stream decoder 2 of the video decoder as shown in FIG.

Then, the transport stream decoder 2 attaches sequence_error_start_code to the video stream and outputs it to the variable length decoder 3. The variable length decoder 3 decodes the bit stream output from the transport stream decoder 2 according to the sequence_error_start_code of the transport stream decoder 2 to a variable length, and if the error does not occur, the variable length. The decoded signal is output to the video decoder 4, and if an error occurs, an error plug EF is attached to the decoded signal having a variable length and output to the video decoder 4. At this time, the error flag EF signal is also output to the error concealment unit 6.

That is, the variable length decoder 3 decodes the signal of the transport stream decoder 2 to a variable length and outputs an error flag (EF) signal when a syntax error or semantic error occurs in the video stream.

The video decoder 4 decodes the video stream in which the variable length decoder 3 does not generate an error, and stores the video stream in the memory 5, and outputs the address of the video stream in which the error occurs as error data ED. .

Accordingly, the error concealment unit 6 determines the size of the block lost in the current frame from the memory 5 according to the error flag EF of the variable length decoder 3 and the error data ED of the video decoder 4. An error concealment technique is performed by extracting a motion vector (MV) indicating a position and motion information of neighboring blocks normally reconstructed.

The error concealed video stream is decoded by the video decoder 4 and displayed on the display unit 7.

The error concealment technique will be described below with reference to FIGS. 3 and 5.

First, as shown in FIG. 3, linear prediction of an interface in a lost block using information of blocks closest to the top (C1), left (C2), bottom (C3), and right (C4) around the lost block. Pixels and motion vector Next, the neighboring blocks for which the pixels of the previous frame block are determined to be the most suitable are searched for (S1).

That is, the spatial correlation at the interface between the blocks lost through the linear predictor and the neighboring blocks Is 2 neighboring pixels The linear combination relationship between is as in Equation 1 below.

[Equation 1]

Where Is a linear prediction coefficient for linearly predicting boundary pixel values in a lost block, and is a kind of weight that depends on a line of pixel values in an adjacent block referred to for linearly predicting boundary pixel values in a lost block.

In this case, linear prediction coefficients that minimize the least-squares function of pixel values of neighboring blocks that are not lost to obtain an optimal linear prediction coefficient, that is, a weighting coefficient. Is equal to the following Equation 2 (S2).

[Equation 2]

Here, the pixel vectors in each neighboring block are expressed as Equation 3 below, and applied to Equation 2 to apply linear prediction coefficients. Obtain

[Equation 3]

Using the linear prediction coefficients obtained through Equations 1 to 3, loss from pixels (first and second registers) close to each boundary of a locally lost block and adjacent blocks, as shown in FIG. Linearly predict the boundary pixel values in the given block (third register). The predicted value calculates and stores the boundary pixel values (fourth register) and its absolute error in the block to be motion compensated as in Equation 4 below.

That is, the first and second registers 11 and 13 are pixels of adjacent blocks close to each boundary of the lost block. Outputs a temporary save per line.

The first and third mixers 12 and 14 are then subjected to the pixels of the first and second registers 11 and 13. Linear prediction coefficient corresponding to Is obtained by applying Equation (2), and each is mixed and outputs the resultant signal.

Accordingly, the adder 15 adds the mixed signals from the first and third mixers 12 and 14, respectively, and outputs the resultant signal.

The third register 16 then temporarily stores and outputs the signal added by the adder 15.

In addition, the fourth register 17 is the boundary pixel value in the block of the previous frame of the lost block. Temporarily stores the output.

Accordingly, the subtractor 18 subtracts the signals output from the third and fourth registers 16 and 17, respectively, and outputs the resultant signal.

The calculating unit 19 then calculates an absolute value from the signal of the subtractor 18 and stores the result signal in the accumulator 20.

Thereafter, the search region is determined by selecting an initial motion vector as an initial position of candidate vectors and average values of motion vectors of the lost block and neighboring blocks and motion vectors of neighboring blocks. S3).

Motion vector available by applying new cost function to the initial vector Next, the optimal motion vector having the smallest error is selected (S4).

At this time, the equation for estimating the optimal motion vector is shown in Equation 4 below.

[Equation 4]

here,

In Equation 4 Denotes the brightness of the pixels in the up, side, and bottom blocks adjacent to the lost block, Above Represents a brightness value of the pixels in the previous frame corresponding to Denotes a pixel value of the boundary within the lost block linearly predicted using the linear prediction coefficient and the pixels of the block adjacent to the lost block.

And Indicates the position of the lost macro block in the image, Represents the size of the macro block, Denotes the macro block in the vicinity of the missing block.

In Equation 1 silver Of the linear predictor in the macro block of position Second linear prediction coefficient (that is, the weighting coefficient), Denotes the number of lines of pixels (ie, the number of linear prediction coefficients) used for the comparison.

After determining the range of the search area using the motion vectors of the neighboring blocks of the damaged block, the search area of the previous frame of the lost block is searched and replaced with the most suitable block (S5).

For example, as shown in FIG. 1, the motion vector of the top macro block C1, the left macro block C2, and the bottom macro block C3 of the macro block E1 lost in the current frame is If are all present, it is a total of five initial vectors of the vector, the zero vector, and the motion vectors of the average of the three vectors.

Initial position of the vector which minimizes the expression (4) of the initial vector By setting Find the optimal motion vector that minimizes equation (4) within the search range.

At this time, the search range is determined in a range smaller than the size of the initial vector.

In other words Decide on

here, Is a constant controlling the search range, usually Is set to about 1/2.

The search range at this time is as shown in Equation 5 below.

[Equation 5]

if If is compensated for the lost block using the initial vector immediately without motion search.

As described above, the error concealment apparatus and method according to the present invention have an effect of preventing degradation of image quality by performing an algorithm using spatial / temporal correlation of neighboring blocks lost at the receiver for errors generated on a channel. have.

Claims (8)

In the error concealment device having a control unit, a memory and a display unit, A transport stream decoder for decoding a bit stream according to a control signal of the controller; A variable length decoder for decoding the transport stream decoded by the transport stream decoder into a variable length and attaching an error flag when an error occurs; A video decoder which decodes the signal decoded by the variable length decoder into a video signal, stores the same in the memory, and displays the same through the display unit; When an error flag signal is output from the variable length decoder, a linear prediction coefficient is obtained using information of blocks adjacent to the lost block, and then a boundary pixel in the lost block is obtained using the linear prediction coefficient and information of blocks adjacent to the lost block. Linearly predict a brightness value of the pixels, estimate a motion vector of which the absolute difference between the brightness value of the linearly predicted pixels and the brightness value of the boundary pixels of the block in the previous frame corresponding to the lost block is minimized. And an error concealment unit for replacing a lost block with a block in a previous frame obtained by using a motion vector. In the error concealment method for restoring lost blocks, (a) After obtaining a linear prediction coefficient by using pixel information of blocks adjacent to the lost block, linearly converting brightness values of boundary pixels in the lost block by using the linear prediction coefficient and pixel information of blocks adjacent to the lost block. Predicting, (b) Initially moving a candidate motion vector having a minimum absolute difference between the brightness value of the linearly predicted pixels among the one or more candidate motion vectors and the brightness value of boundary pixels of a block in a previous frame corresponding to the lost block. Determining by vector, (c) after setting the initial motion vector as an initial position, the brightness value of the linearly predicted pixels among the possible motion vectors in the search region and the brightness value of the boundary pixels of the block in the previous frame corresponding to the lost block. Estimating a motion vector with a minimum absolute difference, and (d) replacing the lost block with a block in a previous frame obtained by using the estimated motion vector. The method of claim 2, The candidate motion vector of step (b) is And at least one of a motion vector of the lost block and neighboring blocks and an average value of the motion vectors of the neighboring blocks and zero motion vectors. The method of claim 2, The motion vector at which the absolute difference of steps (b) and (c) is minimum is Error concealment method characterized in that the estimation using the following equation. here, Where Denotes the brightness of the pixels in the up, side, and bottom blocks adjacent to the lost block, Above Represents a brightness value of the pixels in the previous frame corresponding to Indicates the position of the lost macro block in the image, Represents the size of the macro block, Represents the surrounding macro block. And silver Of the linear predictor in the macro block of position Th linear prediction coefficient, Represents the number of lines of pixels (ie, the number of linear prediction coefficients) used in the comparison, Denotes the brightness value of the boundary pixel in the lost block linearly predicted using the linear prediction coefficient and the pixels of the block adjacent to the lost block. The method of claim 4, wherein At the interface between the lost block and the neighboring block. Is 2 neighboring pixels The spatial correlation between the error concealment method characterized in that estimated using the following equation. The method of claim 5, And the linear prediction coefficients that minimize the least squares function from the pixel values of the neighboring blocks that are not lost in order to obtain an optimal linear prediction coefficient are estimated using the following equation. The method of claim 6, And the pixel vectors in each of the neighboring blocks are estimated using the following equation. The method of claim 7, wherein The search range of the boundary values of a block to be motion compensated by predicting the boundary values of a lost block from pixels near each boundary of the pixel vectors in each neighboring block is calculated and stored using the following equation. Error concealment method. here, ego, ego, Is 1/2.