KR20140013145A - An efficient interpolation algorithm using intra prediction information in h.264/avc - Google Patents

Abstract

The present invention relates to a interpolation algorithm using intra prediction information on H.264/AVC. The interpolation algorithm includes a step of determining the encoding mode of an encoded input image, a step of determining the orientation of an edge in the encoded input image, and a step of performing an adaptive interpolation according to the determined orientation of an edge.

Description

H.264 / ANC interpolation method using intra prediction information {An Efficient Interpolation Algorithm using Intra Prediction Information in H.264 / AVC}

The present invention relates to an interpolation method using H.264 / AHC intra prediction information. More particularly, the present invention relates to an H interpolation method capable of displaying image data efficiently in an image display device having various resolutions by increasing the image resolution through an efficient interpolation method. ．264 It relates to the interpolation method using intra prediction information.

Recently released vehicles are equipped with various video display devices such as a display device, a front display device, a rear seat display device and the like provided in an audio video navigation (AVN) head unit. In addition, the video display device mounted on the vehicle transmits and receives various image data through interworking with an external device such as a smart phone.

Meanwhile, the screen resolution supported by the driver's seat AVN head unit may be different from the screen resolution supported by the rear seat display device while the image generated by the driver's seat AVN head unit is transmitted to the rear seat display device through the in-vehicle network. When the screen resolution of the unit is smaller than the screen resolution of the rear seat display device, the screen resolution of the driver's seat AVN head unit may be increased through image interpolation.

That is, since each image display device installed in the vehicle may have different resolutions, an image interpolation technique for adjusting the resolution is required for each image display device to efficiently share image data with each other.

1 is a diagram illustrating an interpolated image screen according to a conventional image interpolation technique.

Referring to FIG. 1, (a) is an original image, and (b), (c), and (d) are interpolated images interpolated according to a linear interpolation technique, which is one of image interpolation techniques. .

Specifically, (b) is an interpolated image interpolated by nearest neighbor interpolation, and (c) is an image interpolated by bilinear interpolation. (D) is interpolated images interpolated according to bicubic or higher-order interpolation (Bi-cubic).

However, the above three techniques do not consider the directionality of the edge (Edge), which causes a serious deterioration of image quality. In order to overcome this problem, various techniques considering edges of images have been proposed, but there are problems in applying them to a vehicle environment having limited performance resources due to high computational complexity.

Accordingly, an object of the present invention is to apply an interpolation method adaptively according to the edge direction of an image using H.264 / AVC intra prediction information, thereby reducing computational complexity and simultaneously interpolation. The present invention provides an interpolation method using H.264 / AVC intra prediction information that can improve the display quality of the image.

Another object of the present invention is to provide an image interpolation apparatus using the interpolation method.

In order to achieve the above object, an interpolation method using H.264 / AHC intra prediction information of the present invention includes the steps of determining an encoding mode of an encoded input image and, when the determination result is an intra-picture encoding mode, the encoding. Determining an orientation of an edge in the input image and performing adaptive image interpolation according to the determined orientation of the edge.

The image interpolation apparatus of the present invention includes an intra frame predictor for predicting a next image block based on an image block that is already encoded among a plurality of encoded input images consecutive in time, and a motion compensation between the plurality of encoded input images. A motion compensation predictor for predicting and an intra / inter switch for selecting a prediction result by the intra frame predictor and a prediction result by the motion compensation predictor according to the prediction mode information extracted from the encoded input image,

When the prediction mode information is an intra picture encoding mode, the intra frame predictor may determine the direction of an edge in the encoded input image and perform adaptive image interpolation according to the determined direction of the edge.

According to the present invention, the computational complexity can be reduced by determining the edge of the image using the correlation between the intra coding mode and the edge.

1 is a diagram illustrating an interpolated image screen according to a conventional image interpolation technique.
2 and 3 show prediction directions of Intra8x8 mode and Intra16x16 mode.
4 is a diagram illustrating an arrangement of pixels using an intra coding mode to explain an image interpolation method according to an embodiment of the present invention.
5 is a block diagram of an image interpolation apparatus according to an embodiment of the present invention.
6 to 8 are diagrams showing an interpolated image according to the conventional bilinear interpolation technique and the interpolation technique proposed by the present invention.

Prior to examining the details for carrying out the present invention, in order to help the understanding of the present invention, the known contents of the prediction technique in the H.264 / AVC screen will be described in general.

H.264 / AVC intra-picture prediction technology is a prediction technology using spatial correlation of an image. In an image, one pixel has a high correlation with neighboring adjacent pixels, and a high coding efficiency can be obtained by encoding a block having a low correlation with an I-frame or a previous frame using the pixel.

To this end, H.264 / AVC supports three modes (Intra4 × 4, Intra8 × 8, Intra16 × 16) for luminance components and one prediction mode for chrominance component blocks.

Intra8 × 8 is only supported in H.264 / AVC FRExt. 2 and 3 show prediction directions of Intra8x8 mode and Intra16x16 mode.

The optimal prediction mode among these prediction modes is determined by the following equations (1) and (2).

는 최적의 예측 모드를 나태내고,

는 예측 모드를 나타낸다. 수학식 (2)에서

는 예측 모드에 의한 왜곡 (Distortion)을,

은 발생 Bitrate를 나타낸다.

는 라그란지 곱셈자(Lagrange multiplier) 이다.here,

Indicates the best prediction mode,

Denotes the prediction mode. In equation (2)

Is the distortion due to the prediction mode,

Represents the generated bitrate.

Is a Lagrange multiplier.

An interpolation method proposed by the present invention will be described in detail with reference to the attached H.264 / AVC intra prediction technique described above.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is said to "include" a certain component, it means that it can further include other components, without excluding other components unless specifically stated otherwise. Also, the terms " part, "" module," and " module ", etc. in the specification mean a unit for processing at least one function or operation and may be implemented by hardware or software or a combination of hardware and software have.

In the present invention, H.264 / AVC intra prediction information is used to determine edges of an image with low computational complexity, and to select an adaptive interpolation method according to the direction of edge to display the interpolated image quality. The method of raising is disclosed.

First, how edges are distributed according to an intra coding mode (or an intra mode) is examined.

4 is a diagram illustrating an arrangement of pixels using an intra coding mode (or an intra mode) to explain an image interpolation method according to an exemplary embodiment of the present invention.

Referring to FIG. 4, the C3, D2, and D3 pixels represent integer unit pixels closest to the C2 pixel in the horizontal, vertical, and diagonal directions, and the pixels represented by the rectangular boxes are the image interpolation method according to an embodiment of the present invention. According to the pixels to be interpolated.

Table 1 below shows the ratio of pixels whose difference value (| C2-C3 |) of the horizontal pixels and the difference value (| C2-D2 |) of the vertical pixels exceed 30 according to the intra picture coding mode (or intra mode). (%).

Intra Prediction Mode Mode = 0 (Vertical) Mode = 1 (Horizontal) | C2-D2 |> 30 0.48 6.94 | C2-C3 |> 30 8.75 0.57

The test sequence used was Hall Monitor (CIF), and the pixels at the boundary of each mode were excluded from the experiment. If the intra picture coding mode (intra mode) is vertical (mode = 0), | C2-D2 |> 0.48% for 30, but | C2-C3 indicating that there is an edge between C2 and C3 pixels. |> 30 is about 49 times higher.

On the contrary, when the intra coding mode (intra mode) is horizontal (Mode = 1), the probability that there is an edge between C2 and D2 is 12 times higher than the probability that there is an edge between C2 and C3.

By using the correlation between the intra coding mode (Intra mode) and the edge (Edge) to easily determine the edge (edge) of the image to reduce the computational complexity, we propose an adaptive interpolation method considering the edge (Edge).

Below is a pseudo code for an adaptive interpolation method according to an embodiment of the present invention.

If ( Intra Prediction Mode  == Horizontal )

{

      If (| C2 - D2 | > 30)

      {

         If ((| C2 - C3 | <| C2 - D2 |) && (| C2 - C3 | <| C2 - D3 |)

{

l _C2 = C2 , m _C2 = i _C2 , n _C2 = j _C2 , o _C2 = k _C2

t _C2 = D2 , u _C2 = i _D2 , v _C2 = j _D2 , w _C2 = k _D2

p _C2 = ( l _C2 + t _C2 +1) >> 1, q _C2 = ( m _C2 + u _C2 +1) >> 1

r _C2 = ( n _C2 + v _C2 +1) >> 1, s _C2 = ( o _C2 + w _C2 +1) >> 1

}

}

}

Else  if ( Intra Prediction Mode  == Vertical )

{

    If (| C2 - C3 | > 30)

    {

        If ((| C2 - D2 | <| C2 - C3 |) && (| C2 - D2 | <| C2 - D3 |)

{

i _C2 = C2 , m _C2 = l _C2 , q _C2 = p _C2 , u _C2 = t _C2

k _C2 = C3 , o _C2 = l _C3 , s _C2 = p _C3 , w _C2 = t _C3

j _C2 = ( i _C2 + k _C2 +1) >> 1, n _C2 = ( m _C2 + o _C2 +1) >> 1

r _C2 = ( q _C2 + s _C2 +1) >> 1, v _C2 = ( u _C2 + w _C2 +1) >> 1

}

}

}

Else

{H.264 / AVC Interpolation method is used . }

In the above pseudo code, a method of checking whether or not the intra prediction mode matches the directionality in each pixel may be added to further improve the accuracy of edge prediction.

The threshold value 30 used is generally used as a criterion for determining a strong edge, and the algorithm has a trade off relationship between the robustness of the algorithm and the subjective image quality improvement effect.

If the intra coding mode (intra mode) is DC, the computational complexity is reduced using bilinear interpolation, and in the remaining modes, interpolation is performed using the above pseudo code, and the pixels i, j, k, l, p, t Pixels at the position of are interpolated using a 6-tap filter used in H.264 / AVC inter prediction.

An image interpolation apparatus based on the interpolation method described above will be described.

5 is a block diagram of an image interpolation apparatus according to an embodiment of the present invention.

Referring to FIG. 5, the image interpolation apparatus 300 according to an embodiment of the present invention may include a variable length decoder 214 (VLD) and an inverse quantization unit 215 (IQ: inverse quantization). DCT unit 216 (IDCT: Inverse DCT), frame memory 218 (FS: Frame Store), motion compensation predictor 220 (MC: Motion Compensation), intra frame predictor 222 (IP: Intra Prediction ) And an intra / inter changeover switch 224.

The variable length decoder 214 receives input encoded data 200 (Bitstream) and decodes the input encoded data 200. In this case, the prediction error signal 201, the motion vector information, and the prediction mode information may be extracted in the decoding process.

The prediction error signal 201 is sequentially input to the inverse quantization unit 215 and the inverse DCT unit 216, and after the inverse quantization and inverse DCT processes are performed, the prediction error signal 201 is added to the prediction signal 206. Accordingly, decoded image blocks 203 are generated. The decoded image 203 is input to the frame memory 218.

The prediction signal 206 is generated by the motion compensation predictor 220 (MC) and the intra frame predictor 222 (IP) according to the motion vector and the prediction mode extracted from the encoded data 200.

The prediction type depends on whether the macroblock is encoded in "intra" mode or "inter" mode. In the "intra" mode, in order to predict the next macroblock, H.264 / AVC, which is a video encoding standard, is applied a prediction structure based on a macroblock already encoded in the same video. This prediction structure is performed according to the pseudo code described above. That is, the edge of the image is determined by using the correlation between the intra coding mode (intra mode) and the edge, and the adaptive interpolation operation considering the determined edge is performed. On the other hand, in the inter mode, the motion compensation prediction unit 220 (MC) uses motion compensation prediction between corresponding blocks of several consecutive pictures.

Only the intra coded video (I-type video) is encoded without referring to the decoded video first. This I-type video is provided with error tolerance (resilience of error) with respect to the encoded video sequence. Furthermore, in order to access an I-type video among the pictures included in the encoded video sequence, an entry point of the encoded data into the bit stream is defined.

The intra / inter changeover switch 224 controls switching between the intra mode (processing by the intra frame predictor 222) and the inter mode (processing by the motion compensation predictor 220).

The prediction signal generated from the process selected by the intra / inter switch 224 is added to the prediction error signal 201 described above to generate decoded image blocks 203.

First of all, when the generated image blocks 203 are processed by the intra picture encoding mode (intra mode), the edge of the image is determined using the correlation between the intra picture encoding mode (intra mode) and the edge. ) Can be easily determined to reduce the computational complexity.

Hereinafter, the experimental results according to the interpolation method proposed by the present invention will be discussed with reference to FIG. 6 below.

H.264 / AVC JM 12.4 is used to implement the interpolation algorithm using the H.264 / AVC intra picture coding mode proposed in the present invention. The test sequences used were Foreman , Bus and Flower in CIF size (352 x 288).

The interpolated image has a resolution of 1408 x 1152 by enlarging the original image to 1 / 4-pel increments. The main encoding parameters of the input bee stream used in the experiment are shown in the table below.

RDO On In-Screen Prediction Mode Intra4x4 (9 prediction modes) Rate control Off Used QP 28 GOP Only Intra

6 to 8 are views showing an interpolated image according to the conventional bilinear interpolation technique and the interpolation technique proposed by the present invention, in which FIGS. The image is interpolated according to the linear interpolation technique, and (c) is the image interpolated according to the interpolation technique proposed in the present invention.

As shown in FIG. 6, the conventional bilinear interpolation is observed in the diagonal edge of the building part in the image, but the blocking phenomenon is confirmed in the interpolated image according to the method proposed by the present invention. .

As shown in FIG. 7, in the bus sequence, it was confirmed that the clumping of the boundary was relaxed by the interpolation method proposed by the present invention at the boundary portion of the roof of the bus compared to the conventional bilinear interpolation technique.

As shown in FIG. 8, in the flower sequence, the edge portion of the street lamp is better preserved in the image to which the interpolation technique proposed in the present invention is applied.

Table 3 below is a table comparing the computational complexity of the interpolation method proposed in the present invention and the conventional bilinear interpolation method.

As shown in Table 3, it can be seen that the addition operation is reduced from 6% to 53% compared with the conventional bilinear interpolation method. However, although the shift and multiplication operations are slightly increased, it can be seen that the increase in the computational complexity is negligible compared to the improvement effect of the display quality.

Addition operation Shift operation Multiplication Foreman Bilinear 3041280 1520640 Proposed 2860512 2308266 554688 Bus Bilinear 3041280 1520640 Proposed 1450944 1571856 362736 Flower Bilinear 3041280 1520640 Proposed 2343168 2372800 445632

The foregoing description is merely illustrative of the technical idea of the present invention and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas falling within the scope of the same shall be construed as falling within the scope of the present invention.

Claims (6)

Determining an encoding mode of the encoded input image;
Determining an orientation of an edge in the encoded input image in the intra coding mode; And
Performing adaptive image interpolation according to the determined directionality of the edges
H.264 / ABC Interpolation method using intra prediction information.
The method of claim 1, wherein the determining of the directionality of the edge comprises:
The interpolation method using H.264 / ACC intra prediction information which determines the directionality of the edge using the correlation between the intra coding mode and the edge.
The method of claim 2, wherein the determining of the directionality of the edge comprises:
The interpolation method using H.264 / ACC intra prediction information which discriminates the vertical direction and the horizontal direction of the edge using the correlation.
The method of claim 3, wherein performing the adaptive image interpolation comprises:
When the directionality of the edge is vertical directionality, the adaptive image interpolation considering the vertical edge is performed,
And the adaptive image interpolation considering horizontal edge is performed when the directionality of the edge is horizontal.
An intra frame predictor for predicting a next image block based on an already encoded image block among a plurality of encoded input images consecutive in time;
A motion compensation predictor for predicting motion compensation between the plurality of encoded input images; And
And an intra / inter switching switch for selecting a prediction result by the intra frame predictor and a prediction result by the motion compensation predictor according to the prediction mode information extracted from the encoded input image.
When the prediction mode information is an intra coding mode, the intra frame predictor may include:
And determining the directionality of the edges in the encoded input image to perform adaptive image interpolation according to the determined directionality of the edges.
The method of claim 5, wherein the intra frame prediction unit,
The vertical direction and the horizontal direction of the edge are determined using the correlation between the intra coding mode and the edge.
The image interpolation apparatus performing the adaptive image interpolation considering the vertical edge when the directionality of the edge is vertical, and performing the adaptive image interpolation considering the horizontal edge when the directionality of the edge is horizontal directional .

Images