KR101433169B1 - Mode prediction based on the direction of intra predictive mode, and method and apparatus for applying quantifization matrix and scanning using the mode prediction - Google Patents

Abstract

1. Technical field to which the invention described in the claims belongs

The present invention relates to a method and apparatus for intraprediction according to a visible direction in intra-prediction in video coding, and then improving performance by applying a variable quantization matrix and scanning.

2. Technical Challenges to be Solved by the Invention

The present invention proposes a new intraprediction method that shows directionality in intra-prediction of video coding and a quantization and scanning method that improves the coding performance by using a certain pattern of DCT coefficient values.

3. The point of the solution of the invention

The present invention adopts an intra mode prediction method having a high prediction performance by using the intra mode prediction method in intra-prediction of video coding, and makes use of a constant pattern of DCT coefficient values in an intra mode in which directionality is observed And a method of quantizing using a quantization matrix and then applying a scanning method suitable for the pattern to improve the coding efficiency.

4. Important Uses of the Invention

The present invention is used for improving the coding efficiency for an image.

MPEG, intra coding, intra mode, quantization matrix

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a mode prediction method, a mode prediction method, and a quantization matrix,

The present invention relates to a video codec, and more particularly, to a method and apparatus for enhancing performance by performing intra prediction according to the directionality in intra prediction in video coding and then applying a variable quantization matrix and scanning.

The Video Coding Experts Group (VCEG), a research group of the International Telecommunication Union (ITU-T), is now working on the MPEG-4 part 10 Advanced Video Coding (AVC) We are trying to prepare standardization by including various tools with coding ability.

In H.264, intraprediction coding is used to improve the compression ratio by using the pixel correlation between blocks. In this case, intra prediction coding is called intra coding and there are four reference blocks that can be predicted: left (A), upper (B), upper right (C), and upper left (D) In addition, an appropriate prediction direction is selected and encoded for each block, and a complex image is predicted by a small block unit such as a 4x4 pixel, and a flat image is predicted by a large macroblock unit such as a 16x16 pixel.

In the 4x4 pixel intra-picture prediction coding, coding is performed according to the block order of 0 to 15 constituting 16x16 pixels as shown in Fig. For example, in the case of the 12th block shown in Fig. 2, 4 pixels in the left block (A in Fig. 1 corresponds to block 9 in Fig. 2) and an upper block (B in Fig. 4 pixels in the upper right block (C in Fig. 1 corresponds to block 7 in Fig. 2), and 1 pixel in the upper left block (D in Fig. 1 corresponds to block 3 in Fig. 2) And predictively encodes the value of 4x4 pixels in the block. The pixel of the current block to be encoded with reference to the surrounding value means the pixel value before applying the deblocking filter.

The number described in each prediction direction in Fig. 3 is a prediction mode number, and a smaller number is assigned as the frequency of occurrence of the prediction direction is higher. In the prediction direction, one of the nine prediction directions shown in FIG. 3 is selected for each 4x4 pixel block, and the selected prediction direction (prediction mode) is encoded in units of 4x4 pixels. The average value of the prediction mode 2 is obtained by finding the average value of 4 pixels of the left block (A pixel in FIG. 1) and 4 pixels of the upper block (B pixel of FIG. 1) .

In the case of a block located at the left boundary of the picture and a block located at the upper boundary of the picture, the left block and the upper block are located outside the picture, respectively. In this case, the use of the prediction direction is restricted because the block outside the picture boundary can not be referred to. For example, in the block at the top of the picture, the prediction directions of the mode 0, mode 3, mode 4, mode 5, mode 6, and mode 7 referring to the pixel B or C in FIG. 1 can not be used (modes 1, Mode 8 can be used because it does not include pixels of B and C). Exceptionally, in the case of mode 2 average prediction, prediction is performed to calculate an average value referring only to pixels that do not deviate from the boundary of a picture. That is, in the block located at the uppermost end of the picture, the average value of the four pixels of the left block is used as the predicted value, and in the block located at the left boundary of the picture, the average value of the four uppermost blocks is used as the predicted value.

The mode prediction method predicts the prediction direction of the current block from the direction of the encoded neighboring block using a point having a high correlation with the prediction direction of the neighboring block. The mode of block A in Fig. 1 is compared with the mode of block B, and the small number of prediction modes is used as a prediction value of the prediction direction to be applied to the current block.

In intra-picture prediction coding of 16x16 pixel units, one of four prediction modes (vertical prediction, horizontal prediction, average prediction, and plane prediction) is selected and predictive coding is performed.

H.264 uses 4x4 pixel orthogonal transform after intra prediction coding. Compared with the previous 8x8 pixel orthogonal transform, it is easy to implement because the number of data to be handled is small. In addition, since the minimum unit of intra prediction and motion compensation is 4x4 pixels, it is exactly matched with 4x4 pixel unit of orthogonal transformation. The orthogonal transformation introduced in H.264 uses integer unit operations. In the conventional DCT (Discrete Cosine Transform), since the real number unit operation is used, the result before the orthogonal transformation and the result after the orthogonal transformation are inconsistent. However, since H.264 uses integer unit operations, the results before and after the orthogonal transformation are completely matched. That is, it means that the decoded images are completely matched in all decoders. In addition, the integer precision orthogonal transformation requires only a small number of bits in the calculation process since it is unnecessary to calculate the decimal point.

The 4x4 DCT (Discrete Cosine Transform) can be defined as Equation (1).

이다.here,

to be.

And this matrix multiplication is factorized into an equivalent equation as shown in equation (2).

이다.here,

to be.

E is the scaling factor and d is approximated by 0.5. The second and fourth rows of C and the second and fourth columns of C ^T are scaled by a factor of 2, and the post-scaling matrix E reduces the scaling to compensate for this. In this way, the 4x4 DCT can be approximated as shown in Equation (3).

After DCT, quantization is performed. H.264 uses scalar quantization (for example, rounding a decimal number to the nearest integer). The quantization method uses a post-scaling method without using division and floating-point operations. The basic quantization operation can be expressed by Equation (4).

Here, Yij is a coefficient of the transformation described above, Qstep is a quantization step size, and Zij is a quantized coefficient.

A total of 52 Qstep values (Table 1) are supported in H.264 and are indicated by the quantization parameter QP. Qstep doubles every time QP increases by 6.

When the quantization is completed in the encoder, the signal is subjected to entropy coding after the scanning, and is transmitted to the decoder. A 4x4 block of each quantized transform coefficient is arranged in a zigzag order (Figure 4) in an array with 16 elements. In the 16x16 intra mode macroblock, the DC coefficients of each 4x4 luminance block are scanned first, and these DC coefficients form a 4x4 array scanned in the order of Fig. After this process, 15 AC coefficients remain in each luminance block, and the scan starts from the second position in Fig.

The present invention proposes a new intraprediction method that shows directionality in intra-prediction of video coding and a quantization and scanning method that improves the coding performance by using a certain pattern of DCT coefficient values.

Currently, H.264 has nine modes (Figure 5) for 4x4 blocks when performing intra coding. In order to increase the compression rate by saving the bit information when transmitting 9 mode information to each block, intra mode prediction is performed. When prediction is performed, 1 bit (bit) And is represented by 3 bits.

In order to perform intra mode prediction, only the mode of A (left) block and B (upper) block in Fig. 1 are reported, and a small number is set as a prediction mode. The reason for predicting a small number at this time is that, in H.264, the numbers are arranged in the order of the probability that each intra mode number is statistically statistically used for prediction.

Table 2 shows the probability that the intra mode for each video sequence is selected in the block. However, Table 2 shows that the intra mode probabilities of different image sequences are different. This shows the problem of the mode prediction method currently used in intraprediction.

Figure 6 shows the arrows with the number assigned to the optimal intra mode for intraprediction. In Fig. 6, we can see the direction of the intra mode in the boundary of the image (the backside slope, the cap shape boundary, the human face boundary ...). If the direction of the intra mode is used for prediction, better performance can be expected. That is, if prediction is performed by looking at the mode number of the block A (left), block B (upper), block C (upper right), block D (upper left) As shown in FIG.

In H.264, 4x4 DCT (Discrete Cosine Transform) is performed after intraprediction. In this case, the pixel values of the spatial domain are transformed into the frequency domain coefficient values. FIG. 7 shows which patterns are distributed in the frequency domain. The uppermost line in Figure 7 shows the frequencies in the vertical direction, and the leftmost line shows the frequencies in the horizontal direction. The upper left shows low frequency and the lower right shows high frequency.

In the natural image, the DCT (Discrete Cosine Transform) is used to collect the coefficient values at the lower left part of the low frequency side. However, after the intra prediction, DCT (Discrete Cosine Transform) coefficient values may be considered to be included in a certain pattern according to the intra mode. For example, if intra prediction is performed in the 0-th mode (downward direction), the probability that the residual data remains in the vertical direction becomes larger. As described above, the DCT (Discrete Cosine Transform) coefficient values are distributed according to the intra mode after the intra prediction. After the prediction using the directionality of the intra mode, the pattern becomes more distinct. This is because the probability that the boundary part of the image is continuous is high, so that it is necessary to use an adaptive tool for the intra mode prediction according to the directionality.

Figure 8 shows the distribution of DCT (Discrete Cosine Transform) coefficients after intra mode prediction according to directionality. This shows that the pattern is different according to the pattern of general DCT (Discrete Cosine Transform) coefficients and the direction of the intra mode. In addition, when intra mode prediction is performed, it can be seen that the pattern appears more clearly according to the presence or absence of directionality.

Currently, H.264 does not use a quantization method that effectively improves the performance of DCT (Discrete Cosine Transform) coefficient patterns after intraprediction.

In addition, the zigzag scan in Fig. 4 is a method of scanning first using a DCT (Discrete Cosine Transform) coefficient pattern concentrated on a low frequency in a general image. This improves performance in entropy coding. However, this is also not suitable for the pattern of DCT coefficients for directional intra mode, and other scanning methods are needed.

In the intra mode in which the directionality is predicted after the intra prediction of video coding, a certain pattern of the DCT coefficient values is visible. At this time, quantization is performed using a quantization matrix suitable for a certain pattern, It is effective.

Therefore, in the present invention, adaptive quantization matrix application and scanning method and apparatus using a pattern of DCT (Discrete Cosine Transform) coefficients for each intra mode, which is generated by using prediction of intra mode according to directionality which is not applied in existing H.264 I suggest.

Figure 9 shows an overall block diagram of directional intra prediction coding. The encoder looks at the input image, determines the directionality of each of the nine intra modes, predicts the optimal intra mode, and performs intra prediction encoding.

Intra prediction determines the directionality by referring to the intra prediction mode of the current block and the intra prediction mode of the neighboring block. In this case, the method of viewing the intra-prediction mode of the neighboring block determines how many neighboring blocks are to be viewed according to the intra mode of the current block (e.g., the neighboring block is the upper block if the current block is 0) If you look at the number, you will be judged to have directionality. After the intra prediction process is completed, a directional DCT (Discrete Cosine Transform) coefficient and a non-directional DCT (Discrete Cosine Transform) coefficient in the intraprediction mode are relatively different in distribution, And quantization is performed. In this case, since the DCT (Discrete Cosine Transform) coefficient values are uniformly distributed in the intra prediction mode, an adaptive quantization matrix can be used without sending additional information. The quantization matrix is shared among the encoder and the decoder so that a suitable quantization matrix is selected according to the direction of the intra mode so that the matrix is applied to the quantization process.

After the quantization process, the remaining quantization coefficients are subjected to zigzag scanning. In order to maximize the performance of the patterns distributed according to the direction of the intra-prediction mode, the order of scanning is changed from the order of a large number of coefficients to the order of a small number of coefficients , There is a performance improvement in entropy coding.

Fig. 10 shows the intra prediction mode where the Foreman QCIF sequence is coded with QP22.

It can be seen that the direction in which the direction of the intra prediction mode is seen is along the arrow of the intra prediction mode of the current block. For example, in the case of intra mode 4 (right downward mode), it can be seen that the mode of the block at the position of block D (upper left) in FIG. 1 can be predicted.

As a result of analyzing the intra-prediction mode distribution of the video sequence, the intra-mode directionality determination block diagram is shown in FIG. The algorithm is as follows.

Step 1) If the current block is the intra prediction mode "0", it is compared with the intra prediction mode of the above block.

Step 2) If the intra prediction mode of the block is "0 ", it is determined that the direction has the direction, and that the modes other than" 0 "

Step 3) When the current block is the intra prediction mode "1", it is compared with the intra prediction mode of the left block.

Step 4) If the intra prediction mode of the left block is "1 ", it is determined that there is directionality, and that modes other than" 1 "

Step 5) When the current block is the intra prediction mode "2", it is determined that the direction of the intra mode can not be found because it is the DC mode having no directionality.

Step 6) When the current block is the intra prediction mode "3", it is compared with the intra prediction mode of the upper right side block.

Step 7) If the intra prediction mode of the upper right block is "3 ", it is determined that there is directionality, and that modes other than" 3 "

Step 8) When the current block is the intra prediction mode "4", it is compared with the intra prediction mode of the upper left side block.

Step 9) If the intra prediction mode of the block on the upper left side is "4 ", it is determined that there is directionality, and that modes other than" 4 "

Step 10) If the current block is the intra prediction mode "5", it is compared with the intra prediction mode of the above block.

Step 11) If the intra-prediction mode of the block is "5 ", it is determined that the direction is present, and the mode other than" 5 "

Step 12) When the current block is the intra prediction mode "6 ", it is compared with the intra prediction mode of the left block.

Step 13) If the intra prediction mode of the left block is "6 ", it is determined that there is directionality, and that modes other than" 6 "

Step 14) If the current block is the intra prediction mode "7", it is compared with the intra prediction mode of the above block.

Step 15) If the intra prediction mode of the block is "7 ", it is determined that there is directionality, and that modes other than" 7 "

Step 16) If the current block is the intra prediction mode "8 ", it is compared with the intra prediction mode of the left block.

Step 17) If the intra prediction mode of the left block is "8 ", it is determined that there is directionality, and that modes other than" 8 "

If the presence or absence of the direction is determined, the optimal prediction mode according to the direction is determined and coded in the intra prediction part unlike the previous codec. And directionality is used to determine the next quantization matrix.

≪ Method of applying quantization matrix according to intramode direction >

After the presence or absence of the directionality of the intra mode is known, an appropriate quantization matrix is applied. It is classified into 18 matrices with different weights according to the intra mode and directionality, and one of them is selected and applied to the quantization.

Each of the 18 matrices is branched into directional and non-directional prediction modes of the intra nine prediction modes, and 18 matrices are created in advance by looking at the pattern of the DCT coefficient distribution. This is shared by the encoder and the decoder, so there is no need to send a matrix or select indexes separately.

As a result of analyzing the intra-prediction mode distribution of the video sequence, 18 matrices can be made as shown in FIG. 12 (example), and the algorithm to be created is as follows.

Step 1) The intra prediction mode is divided into 0, 5, 7, 1, 6, 8, 2, 3, 4 according to the direction characteristics.

Step 2) In the intraprediction modes 0, 5, and 7, the DCT coefficient distribution is biased toward the upper side of the block. Therefore, the quantization scale value above the matrix is compared with the quantization scale value on the left side.

Step 3) In the 1, 6, and 8 intraprediction modes, the DCT coefficient distribution is biased to the left of the block, so the quantization scale value on the left of the matrix is compared with the quantization scale value on the upper side.

Step 4) Since the DCT coefficient distributions are distributed evenly on the left and upper sides in the second, third and fourth intra prediction modes, the quantization scale values in the matrix are equally applied.

Step 5) Step (Step) The matrix divided into the steps 2, 3 and 4 can finely adjust the quantization scale value so that 9 matrices can be respectively generated according to the characteristics of the intra prediction mode.

Step 6) According to the intra prediction mode number, nine modes are created. Then, two directions are shown for each intra prediction mode.

Step 7) If the direction of the intra-prediction mode is seen, the distribution of the DCT coefficients is relatively larger than the directionality is not visible, so that the difference of the quantization scale values in the matrix is larger than that of the non-directional.

Step 8) If the direction of the intra prediction mode is not visible, the distribution of the DCT coefficients is relatively smaller than that of the direction, so that the difference of the quantization scale values in the matrix is smaller than that of the direction.

Step 9) After all the above steps, all 18 matrices are created.

When all of the quantization matrices are created, the encoder and decoder share 18 fixed quantization matrices, and after intra prediction, the corresponding matrix is selected according to the intra prediction mode and directionality as shown in Fig. , And the quantization matrix index used is a reference for the next scanning.

≪ Scanning Application Method According to Quantization Matrix >

When the quantization matrix is selected and the index is received, the scan is selected for the scan. The scan method is also shared by the encoder and the decoder, and the appropriate one is selected.

That is, the scanning method sets the order of the scale values in the quantization matrix from the smallest order to the greatest order to be scanned. Therefore, it is possible to increase the compression efficiency when the following Variable Length Coding (VLC) coding is performed.

As shown in Fig. 14 (example) according to the index of the quantization matrix, the block diagram for selecting the scan is as shown in Fig. 15, and the algorithm for determining the scan order is as follows.

Step 1) All of the 18 quantization matrices are branched to select the scanning method through the corresponding index.

Step 2) View the scale values in the quantization matrix and scan from smallest to largest.

In the above process, a proper scan is selected, scanning is performed, and variable length coding (VLC) coding is performed and transmitted to the decoder.

The decoder is shown in Figure 16, and the process is reversed to the encoding process.

In other words, when the decoder learns the intra-prediction mode, it determines the directionality and applies the appropriate scan through the intra-prediction mode with nine intra-prediction modes and selects the appropriate quantization matrix to perform inverse quantization do.

[Apparatus of the invention]

In the present invention, a mode prediction unit, a quantization matrix application unit, and a scanning unit according to intra directionality included in a moving picture encoder / decoder are shown in FIG. Receives an original image, performs intra prediction, and outputs an entropy-coded bitstream. The mode prediction unit, the quantization matrix application unit, and the scanning unit perform processing on a block-by-block basis. The intra prediction unit, the intra mode directionality determination unit, the DCT conversion unit, the quantization unit, A quantization matrix applying section according to mode direction, a scanning section, a scanning application section according to a quantization matrix, and an entropy coding section.

The intra prediction unit receives the original image and performs intra prediction. At this time, according to the direction of the intra mode direction presence / absence determination unit, more accurate prediction than the existing codec is performed. When the intra prediction is completed, the transformed DCT transformer is transformed into a frequency space. Then, the quantization unit performs quantization. When quantization is performed, the quantization matrix application unit according to the intra mode direction looks at the direction and selects an appropriate quantization matrix to be applied in the quantization. In the scanning unit, the coefficient values derived from the quantization unit are scanned according to the scanning order determined by the scanning application unit according to the quantization matrix. Finally, the entropy coding unit entropy-codes the scanned values to generate a bitstream.

Figure 1 (Figure 1) shows the location of blocks referenced in H.264 encoding.

FIG. 2 (FIG. 2) shows the coding sequence of the 4 × 4 block.

FIG. 3 (FIG. 3) shows an intra-picture prediction structure of a 4 × 4 block.

FIG. 4 (FIG. 4) shows a zigzag scan structure for a 4 × 4 block.

Fig. 5 (Fig. 5) shows nine modes of intra prediction.

Figure 6 (Figure 6) shows the directionality of the intra prediction mode.

Figure 7 (Figure 7) shows the DCT basic pattern.

8 shows an example of DCT coefficient distribution after directional intra mode prediction.

9 is a block diagram of a directional intra-coded encoder.

Fig. 10 (Fig. 10) shows the intra mode direction determination position.

Fig. 11 (Fig. 11) shows a block diagram for determining intra mode directionality.

Figure 12 (Figure 12) is an example of a quantization matrix.

Fig. 13 (Fig. 13) shows the application of the quantization matrix according to the directionality.

Fig. 14 (Fig. 14) is an example of a scanning method.

FIG. 15 is a block diagram illustrating a scanning method according to a quantization matrix. FIG.

16 is an overall block diagram of a directional intra prediction coding decoder.

Figure 17 (Figure 17) shows a device for quantization matrix application and scanning according to intra directional prediction.

Claims (7)

A video coding method comprising: Determining whether an intra mode of a current block and an intra mode of a neighboring block are directional; Setting an optimal intra mode for the current block based on the determined directionality; Determining a quantization matrix for the current block based on the directionality and the optimal intra mode; And scanning the scale value in the quantization matrix. The method according to claim 1, The step of determining the presence or absence of the directionality comprises: Determining whether the intra mode of the current block and the intra mode of any one of neighboring blocks determined according to the intra mode of the current block among the plurality of neighboring blocks are the same; If the intra mode of the current block and the intra mode of the neighboring block are the same, it is determined that the directionality exists. If the intra mode of the current block and the intra mode of the neighboring block are not the same, The method comprising the steps < RTI ID = 0.0 > of: < / RTI > 3. The method of claim 2, Wherein the plurality of neighboring blocks include a left block located on the left side of the current block, an upper block located on the upper side of the current block, a right corner block located on the left side of the upper block, a left corner located between the left block and the upper block, Wherein the block includes a block. 3. The method of claim 2, Wherein determining the quantization matrix comprises: Setting two quantization matrices according to the presence or absence of directionality for each intra mode; And selecting a quantization matrix corresponding to the optimal intra mode based on the presence or absence of the direction. 5. The method of claim 4, Wherein the intra modes are all nine, and the quantization matrix is set to eighteen. 5. The method of claim 4, Wherein the scanning comprises: Assigning an index according to the selected quantization matrix; Selecting a scanning method according to the index; And scanning the scale value in order from a small value to a large value. delete

Images