KR100943068B1 - Method for error concealment using curve interpolation - Google Patents

Abstract

The error recovery method according to the present invention is a method for restoring each pixel included in a lost block, wherein the difference value with neighboring pixels in the boundary region of the lost block is calculated based on the pixel to be restored. And detecting a minimum boundary pixel, which is a pixel having the smallest difference value, in each block adjacent to the boundary region of the lost block, based on the calculated difference value, and at least two minimum boundary pixels. Performing curve interpolation.

Image, decoding, error, reconstruction, loss

Description

Error restoration method using curve interpolation {METHOD FOR ERROR CONCEALMENT USING CURVE INTERPOLATION}

The present invention relates to video decoding, and more particularly, to a method for reconstructing an image included in a lost block among encoded data.

Various digital video compression techniques have been proposed for obtaining low data rates while maintaining high image quality when transmitting or storing video signals. Such compression technology achieves relatively high compression ratio through Discrete Cosine Transform (DCT) technique or Motion Compensation (MC) technique.

Such video data loses spatio-temporal information during the encoding, transformation, and quantization processes in encoding, and has a close relationship between front and rear data in the bit stream by variable length coding (VLC). It became. In particular, when video data is encoded using a prediction encoding technique, front and rear data are directly related to each other by referring to a frame encoded through prediction in a next frame. Therefore, in the process of transmitting the encoded data using the predictive encoding technique, when the encoded data is lost, a problem occurs in which an error continuously occurs in a frame reconstructed using the lost reference frame.

As a result, various methods have been studied for encoding and decoding video data in consideration of a transmission channel environment in which transmission errors may occur. That is, a method of generating a bitstream that is robust to errors is being studied to minimize the effects of errors in the encoding process, and even if data is lost due to transmission errors during decoding, the encoded data is smoothly restored and the error is concealed. There are various ways to do this.

1 is a conceptual diagram of a linear interpolation method used in the error concealment technique according to the prior art. Referring to Fig. 1, in the error concealment technique according to the prior art, the lost 16x16 macroblock 1 is recovered by linear interpolation using spatial correlation. Specifically, after checking whether the neighboring blocks (for example, the neighboring blocks 3, 5, 7, and 9 located on the top, bottom, left, and right sides) of the lost block 1 are decoded, each pixel included in the lost block is decoded. The pixels included in the decoded neighboring block are reconstructed. That is, among the pixels included in the macroblocks 3, 5, 7, and 9 adjacent to the top, bottom, left, and right and directly adjacent to the lost block 1, the pixels at the positions to be restored, that is, Y (x, y) The pixels Y1, Y2, Y3, and Y4 located in the horizontal and vertical directions of the pixel are linearly interpolated and restored.

This method uses a mean value of pixels located in the vicinity, and does not reflect the characteristics of the lost block, and blurring is formed in the reconstructed image. In particular, blurring is severely caused in an image including a small QCIF (176 × 144) format image and a block in which edge components are distinctly present. Accordingly, there is a need for a method of restoring lost data by minimizing blurring.

The present invention has been made in consideration of the above-described point, and an object of the present invention is to provide a method of reducing blurring occurring when reconstructing a lost block in an image encoded in blocks.

In order to achieve the above object, the error recovery method according to the present invention is a method of restoring each pixel included in a lost block, wherein the pixels adjacent to the boundary region of the lost block are based on the pixel to be restored. Calculating a difference value between and detecting a minimum boundary pixel, which is a pixel having the smallest difference value in each block adjacent to the boundary region of the lost block, based on the calculated difference value; Curve interpolation using at least two minimum boundary pixels.

The at least two minimum boundary pixels are two pixels having the smallest difference value among the minimum boundary pixels.

The performing of the curve interpolation may include checking whether a pixel to be restored includes an edge region of the image, and corresponding to a case in which the pixel to be restored does not include an edge region of the image, wherein the pixel to be restored and each minimum Calculating a distance between the boundary pixels and performing curve interpolation based on each of the calculated distances.

The performing of the curve interpolation may include checking whether a pixel to be restored includes an edge region of the image, and corresponding to a case where the pixel to be restored does not include an edge region of the image, between the at least two minimum boundary pixels. Acquiring the midpoint of the curve; and moving the vertical reference axis of the function adopted for determining the weight of the curve interpolation corresponding to the position of the vertical axis including the midpoint, and weighting the curve interpolation based on the moved vertical reference axis. Decision making process is included.

The curve interpolation is preferably sigmoid interpolation using a sigmoid function.

The process of performing sigmoid interpolation in response to the case in which the pixel to be reconstructed does not include an edge region of the image is performed based on a range of parameters of the sigmoid function. And mapping a distance from a pixel included in a boundary region to the parameter, and performing a calculation based on sigmoid interpolation using the mapped parameter.

According to the error recovery method according to the present invention, the blurring phenomenon generated in the process of restoring the pixels of the lost block can be reduced.

In addition, by restoring the lost pixel in consideration of the relationship between the pixels included in the block located near the lost block, the lost block can be restored close to the original image, and image quality deterioration due to the restoration of the lost block can be reduced. Can be reduced.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Specific details appear in the following description, which is provided to help a more general understanding of the present invention, and it is common knowledge in the art that such specific matters may be changed or modified within the scope of the present invention. It is self-evident to those who have.

The present invention can be applied to an apparatus for decoding video data as a method of restoring a block included in lost data to a block before it is lost as some data of the encoded data is lost. In particular, the present invention can be applied to an apparatus for processing error recovery of a video decoding apparatus, for example, an intra predictor of an H.264 decoder.

2 is an exemplary diagram of a lost block and neighboring blocks around it. Referring to FIG. 2, a lost block 10 and blocks 20, 30, 40, and 50 adjacent to the top, bottom, left, and right sides of the lost block 10 are shown. Each block is a 16 × 16 block, and includes 16 × 16 pixels. In general, restoration of a lost block is performed in a state where decoding of blocks 20, 30, 40, and 50 located around the lost block 10 is completed, and each pixel included in the lost block is sequentially processed. The method of restoration is adopted. Accordingly, in an exemplary embodiment of the present invention, with reference to the neighboring blocks 20, 30, 40, and 50 that have been decoded, the rightmost part of the plurality of pixels included in the lost block 10 may be included. The pixels 11 located on the right side of the pixel 11 are sequentially restored, and when the pixels located in the uppermost column are completed, the pixels located in the next lower column are restored.

Furthermore, since the process of restoring each pixel included in the lost block 10 is performed in the same manner, it is obvious that the lost block can be restored through the method of restoring the specific pixel 15. Accordingly, a method of restoring a specific pixel 15 included in the lost block 10 will be described below.

3 is a flowchart illustrating a procedure of an error restoration method according to an embodiment of the present invention. Referring to FIG. 3, in the error recovery method according to an embodiment of the present invention, calculating a difference value between a specific pixel 15 and pixels located around a lost block, respectively, at least 100 from the calculated difference value. Detecting a border pixel (200), and performing a curve interpolation using the minimum boundary pixel (300).

First, in operation 100, a difference value between a specific pixel 15 to be restored and pixels 25, 35, 45, and 55 neighboring to the lost block 10 is calculated. Here, the pixels 25, 35, 45, and 55 adjacent to the lost block 10 are lost among the pixels included in the neighboring blocks 20, 30, 40, and 50 of the lost block 10. It may be a pixel directly neighboring the upper, lower, left, and right corner portions of the block 10.

In operation 200, pixels 21, 31, 41, and 51 having the smallest value among pixels included in each block are detected based on the difference value calculated in operation 100. . For example, the first pixel 21 having the smallest difference value between the specific pixel 15 and the pixels 25 of the upper block 20 is replaced by the first minimum boundary pixel, the specific pixel 15 and the lower block 30. The second pixel 31 having the smallest difference value between the pixels 35 of the second pixel 31 has the smallest difference between the second minimum boundary pixel, the specific pixel 15 and the pixels 45 of the left block 40. The fourth pixel 51 having the smallest difference value between the third pixel 41 having the value and the third minimum boundary pixel, and the pixel 55 of the specific pixel 15 and the right block 50. Detect with minimum boundary pixel.

Next, in step 300, curve interpolation is performed using the first, second, third, and fourth minimum boundary pixels 21, 31, 41, and 51. For example, a parameter corresponding to a distance value between the specific pixel 15 and the first, second, third, and fourth minimum boundary pixels 21, 31, 41, and 51 is calculated, respectively, and the calculated parameters Interpolation can be performed by applying variables as variables used in sigmoid interpolation. Such interpolation may be performed as described in steps 330 and 340 below.

Preferably, step 300 may be performed based on sigmoid interpolation using a sigmoid function. To this end, step 300 may include steps 310 to 370 shown in FIG. 4.

4 is a flowchart illustrating a detailed procedure of a curve interpolation step included in an error recovery method according to an embodiment of the present invention.

Referring to FIG. 4, the curved interpolation according to an embodiment of the present invention first checks whether a specific pixel 15 under restoration includes an edge region of an object represented in an image through step 310. For example, step 310 extracts two pixels having the maximum value of the difference value among the first, second, third, and fourth minimum boundary pixels 21, 31, 41, and 51, and extracts the two pixels. The difference between the pixels is compared with a predetermined threshold to determine whether the difference between the two pixels exceeds the threshold. As a result of the check, if the difference between the two pixels exceeds the threshold, it is determined that the edge region is included in the pixel to be restored; otherwise (that is, the difference between the at least one pixel is less than or equal to the threshold). If it is determined that the edge area is not included in the pixel to be restored.

If the edge area is not included in the pixel to be restored, steps 330 and 340 are performed. If the edge area is included in the pixel to be restored, steps 350 to 370 are performed (step 320).

In steps 330 and 340, lost pixels are reconstructed using sigmoid interpolation based on the sigmoid function.

Sigmoid interpolation is based on the calculation of the sigmoid function (see the graph of FIG. 5) according to Equation 1 below, to obtain the weight according to the position of the pixel and to restore the pixel included in the lost block That's how.

However, when the number of parameters of the sigmoid function and the size of the lost macro block are different, appropriate mapping of the lost macro block to the parameters of the sigmoid function is required. To do this, in step 330, the parameters of the lost macroblock and the sigmoid function are mapped. For example, when the lost macroblock is 16 × 16 in size, the index of the lost block may be set to 0 to 15, and may be finally set to −1 to 16 including neighboring pixels of the neighboring block, respectively. On the other hand, it is assumed that the parameters of the sigmoid function are set to -4 to 4 (see Fig. 6). In this case, the calculation may be performed by performing the calculation of Equation 3 based on the proportional expression of Equation 2 below.

Where x _i Is the variable of the sigmoid function calculated through the mapping, and d _i is the distance between the pixel to be restored and the first, second, third, and fourth minimum boundary pixels 21, 31, 41, and 51. to be.

Next, in step 340, the sigmoid function is calculated for the minimum boundary pixels in the vertical and horizontal directions, and the interpolation of the pixels to be restored by averaging the calculated values is performed. This step 340 may be performed through the calculation of Equation 4 below.

는 수평방향으로 인접한 블록에 포함된 최소경계화소(즉, 제3 및 제4최소경계화소(41,51))에 대한 시그모이드 함수 값을 연산한 결과이며,

는 수직방향으로 인접한 블록에 포함된 최소경계화소(즉, 제1 및 제2최소경계화소(21,31))에 대한 시그모이드 함수 값을 연산한 결과이며,

는 연산된

및

값을 평균화한 값이다.here,

Is the result of calculating the sigmoid function values for the minimum boundary pixels (ie, the third and fourth minimum boundary pixels 41 and 51) included in the adjacent blocks in the horizontal direction.

Is the result of calculating the sigmoid function values for the minimum boundary pixels (ie, the first and second minimum boundary pixels 21 and 31) included in the vertically adjacent blocks.

Is calculated

And

The averaged value.

In operation 350, a middle point between the two pixels extracted in operation 310 (eg, the first and second minimum boundary pixels 21 and 31) is obtained. In operation 360, a vertical reference axis passing through the midpoint is generated, the vertical axis of the sigmoid function graph is moved corresponding to the vertical reference axis, and the function value is changed accordingly. Hereinafter, a process of steps 350 and 360 will be described in detail with reference to the accompanying drawings.

FIG. 7A illustrates a graph of the sigmoid function before the vertical axis is moved and the relationship between the lost blocks according to an embodiment of the present invention, and FIG. 7B is a sigmoid function having the vertical axis moved through step 360 of FIG. 4. A graph of and illustrates the relationship of the lost blocks. Referring to FIG. 7A, in operation 350, the two pixels extracted in operation 310 (eg, the first and second minimum boundary pixels 21 and 31) are connected to each other, and the midpoint O of the connected line segment is obtained. do. Then, a vertical reference axis 61 passing through the mid point O is formed. In operation 350, the sigmoid function graph is mapped in consideration of the lost block size. Next, in step 360, the vertical axis 71 is moved to the new vertical axis 75 in accordance with the vertical reference axis 61 in the mapped sigmoid function graph. As a result, as shown in FIG. 7B, the sigmoid function graph is formed around the new reference axis 75.

Finally, in step 370, the pixels 25, 35, 45, and 55 adjacent to the lost block 10 indicate the weighted sigmoid function value formed around the new reference axis 75. Apply to the value to restore the lost pixels.

According to the error recovery method of the present invention described above, it is possible to solve the problems caused by interpolating the pixels of the uniformly lost block without considering the situation of the neighboring block. In particular, the problem of causing serious blurring and deterioration of image quality can be solved.

Although the present invention has been described above by means of limited embodiments and drawings, the present invention is not limited thereto, and various modifications and changes may be made by those skilled in the art to which the present invention pertains.

1 is a conceptual diagram of linear interpolation used in the error concealment technique according to the prior art

2 is an exemplary diagram of a lost block and neighboring blocks around it;

3 is a flowchart illustrating a procedure of an error restoration method according to an embodiment of the present invention.

4 is a flowchart showing the detailed procedure of the curve interpolation step included in the error recovery method according to an embodiment of the present invention.

5 is a graph of a typical sigmoid function

6 is an exemplary diagram of a relationship between a size of a lost macroblock and a parameter of a sigmoid function according to an embodiment of the present invention.

7A illustrates an example of a relationship between a graph of a sigmoid function and a lost block before the vertical axis is moved in the error recovery method according to an embodiment of the present invention.

FIG. 7B is an exemplary diagram illustrating a relationship between a graph of a sigmoid function having a vertical axis shifted and a lost block in the error restoration method according to an embodiment of the present invention.

Claims (6)

In the method for restoring each pixel included in the lost block, Calculating a difference value with pixels neighboring the boundary region of the lost block, based on the pixel to be restored; Detecting a minimum boundary pixel, which is a pixel having the smallest difference value in each block adjacent to the boundary region of the lost block, based on the calculated difference value; Error recovery method comprising the step of performing curve interpolation using at least two minimum boundary pixels. The method of claim 1, The at least two minimum boundary pixels And two pixels having the smallest difference value among the minimum boundary pixels. The method of claim 1, The process of performing the curve interpolation, Determining whether a pixel to be restored includes an edge region of the image; In response to the case where the pixel to be reconstructed does not include an edge region of the image, calculating a distance between the pixel to be reconstructed and each minimum boundary pixel, and performing curve interpolation based on the calculated distances. Error recovery method comprising the. The method of claim 1, The process of performing the curve interpolation, Determining whether a pixel to be restored includes an edge region of the image; In response to the pixel to be reconstructed including an edge region of the image, obtaining a midpoint between the at least two minimum boundary pixels; Moving the vertical reference axis of the function adopted for determining the weight of the curve interpolation corresponding to the position of the vertical axis including the midpoint, and determining the weight of the curve interpolation based on the moved vertical reference axis. Error recovery method. The method according to any one of claims 1 to 4, The curve interpolation is an error recovery method, characterized in that sigmoid (Sigmoid) interpolation using a sigmoid function. The method of claim 5, The process of performing sigmoid interpolation in response to the case where the pixel to be reconstructed does not include the edge region of the image, Mapping a distance between the lost block and a pixel included in a boundary region of a neighboring block to the parameter based on a range of a parameter of a sigmoid function; And performing an operation based on sigmoid interpolation using the mapped parameter.

Images