JP2015223003A - Image decoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image encoding/decoding technique with higher compression efficiency.SOLUTION: An image decoding method for performing intra-frame prediction in a block unit to decode an image includes: a first prediction step of generating pixel values of two reference pixels by prediction; and a second prediction step of predicting a pixel value of a decoding target pixel arranged between the two reference pixels using the pixel values of the two reference pixels generated in the first prediction step.

Description

  The present invention relates to a moving picture coding technique for coding a moving picture.

  Encoding methods such as MPEG (Moving Picture Experts Group) method have been established as a method for recording and transmitting large-capacity moving image information as digital data, and MPEG-1 standard, MPEG-2 standard, MPEG-4 standard, The H.264 / AVC (Advanced Video Coding) standard is an international standard encoding method.

  In these standards, the encoding target image is predicted in block units using the image information that has been encoded, and the prediction difference from the original image is encoded, thereby eliminating the redundancy of the moving image. The code amount is reduced. In particular, in H.264 / AVC, a dramatic improvement in the compression rate was realized by adopting an intra-screen predictive encoding method that uses peripheral pixels of the encoding target block.

  However, in the case of intra prediction in H.264 / AVC, the prediction method is simple and sufficient prediction accuracy cannot be realized. For example, in the intra-screen prediction based on H.264 / AVC, only one reference pixel is designated, and all pixels along the prediction direction are predicted using only the pixel value of one reference pixel as a reference value. There was room for improvement in prediction accuracy. Therefore, an intra-screen coding technique for improving the accuracy of intra-screen prediction and increasing the compression rate has been demanded.

  As a technique for improving the accuracy of intra-screen prediction, for example, in Patent Document 1, it is possible to increase the number of types of pixels that can be used for intra-screen prediction by enabling encoding after the entire image is inverted. It is disclosed.

  Non-Patent Document 1 discloses that prediction is performed using upper, lower, left and right blocks by changing the encoding order in units of blocks.

JP2006-352181

Taichiro Shiojira, Akiyuki Tanizawa, Takeshi Nakajo, “Intra coding based on block-based extrapolation / interpolation prediction”, PCSJ2006, November, 2006.

  However, in Patent Document 1, after the image is inverted, since simple prediction in one direction is performed using only the pixel value of one reference pixel as a reference value in the same manner as H.264 / AVC, the prediction accuracy is further increased. It cannot be improved.

  In Non-Patent Document 1, only a part of blocks can be predicted using the upper, lower, left and right blocks, and the prediction accuracy of other blocks is lower than that of H.264 / AVC.

  In these conventional techniques, for example, when the luminance value greatly changes along the prediction direction, there is a problem that the difference from the prediction value increases, the code amount increases, and the compression rate decreases.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an image encoding / decoding technique with higher compression efficiency.

  In order to achieve the above object, an embodiment of the present invention may be configured as described in the claims, for example.

  According to the present invention, there is provided an image encoding / decoding technique with better compression efficiency.

Explanatory drawing of the image coding apparatus which concerns on one Example of this invention. Explanatory drawing of the image decoding apparatus concerning one Example of this invention. Explanatory drawing of the prediction encoding process in a screen used by H.264 / AVC Explanatory drawing of the prediction encoding process in a screen based on one Example of this invention. Explanatory drawing of the prediction encoding process in a screen based on one Example of this invention. Explanatory drawing of the prediction decoding process in a screen used by H.264 / AVC Explanatory drawing of the prediction decoding process in a screen based on one Example of this invention. Explanatory drawing of the prediction decoding process in a screen based on one Example of this invention. Illustration of intra-screen prediction used in H.264 / AVC Explanatory drawing of the prediction in a screen concerning one Example of this invention Explanatory drawing of the prediction in a screen concerning one Example of this invention 1 is a flowchart of an image encoding apparatus according to an embodiment of the present invention. 1 is a detailed flowchart of an image encoding device according to an embodiment of the present invention. 1 is a flowchart of an image decoding apparatus according to an embodiment of the present invention. 1 is a detailed flowchart of an image decoding apparatus according to an embodiment of the present invention. Explanatory drawing of one Example of this invention Explanatory drawing of the prediction method different from the prediction in a screen concerning one Example of this invention. Configuration example of encoded stream according to one embodiment of the present invention Configuration example of encoded stream according to one embodiment of the present invention

    Embodiments of the present invention will be described below with reference to the drawings.

  Moreover, in each drawing, the component to which the same code | symbol was attached | subjected shall have the same function.

  In addition, the expression “sum of pixels” in each description and each drawing of the present specification refers to a result of adding pixel values of pixels.

  First, the operation of the intra-frame predictive coding process based on H.264 / AVC will be described with reference to FIG. In H.264 / AVC, encoding is performed on the encoding target image in the raster scan order (301). Note that the order of raster scanning generally refers to the order in which processing is performed from the upper left end to the right end of the screen, and then the processing is performed from the lower left end to the right end one step down and repeated.

  Here, the prediction for each pixel of the encoding target block is performed using the pixel value of the decoded image of the encoded block adjacent to the left, upper left, upper, and upper right of the encoding target block.

  In particular, among the encoded blocks, all the pixels aligned on the same straight line in the prediction direction with the pixel value of one of the 13 pixels shown in FIG. The pixel value of the one pixel is predicted as a reference value (302).

  For example, as shown in (303), all the pixels B, C, D, and E of the encoding target block are subjected to predictive encoding with reference to the same pixel, and a value A ′ obtained by decoding the pixel immediately above the pixel B (Prediction differences) b, c, d, and e are calculated. Further, in H.264 / AVC, an optimal one of eight types of prediction direction candidates such as vertical, horizontal, and diagonal can be selected in units of blocks, and the prediction difference and the value of the prediction direction are encoded. However, in H.264 / AVC, in addition to the prediction along the specific direction, “DC prediction” that predicts all the pixels included in the encoding target block by the average value of the reference pixels can be used. (304).

  Next, the operation of the intra-screen predictive decoding process according to H.264 / AVC will be described with reference to FIG. Similarly to the encoding process, the decoding process is also performed according to the raster scan order (601), and the pixel value of the decoded image is calculated using the decoded reference pixel and the prediction difference. That is, a decoded image is acquired by adding a prediction difference with a reference pixel along the prediction direction.

  For example, in (602), prediction differences b ′, c ′, d ′, and e ′ of the decoding target block (b, c, d, and e in FIG. 3 are decoded and include quantization errors), respectively. For each decoded pixel B ′, C ′, D ′, E ′ (B, C, D, E in FIG. 3 respectively) To obtain a decoded pixel).

  As described above, the intra-screen predictive encoding process based on H.264 / AVC employs a simple method based on a single direction in which only one reference pixel is designated and all pixels along the prediction direction are predicted with the value. ing.

  FIG. 9 shows a conceptual diagram of intra-screen predictive coding by H.264 / AVC. Here, the horizontal axis sets the coordinate value in the target block along the prediction direction, and the vertical axis sets the pixel value (luminance value) at that coordinate. Therefore, the curve in the graph represents the luminance curve in the target block. As already described, in H.264 / AVC, when performing intra prediction, only the blocks located on the left side and the upper side of the target block can be referred to. For this reason, H.264 / AVC employs a method of copying reference pixels in one direction. In this case, as shown in (901), if the luminance gradient in the target block is small, the prediction is easy to hit and the prediction difference is small.However, if the luminance gradient is large, the prediction difference increases as the distance from the reference pixel increases. As a result, the code amount increases.

  A conceptual diagram of intra prediction encoding according to the present embodiment is shown in FIG. In order to cope with the above-described problem due to H.264 / AVC, in this embodiment, a pixel located at the boundary of the target block is set as a new reference pixel (reference pixel 2) as in (1001), and a normal reference pixel ( Predicted in combination with reference pixel 1). That is, two reference pixels (reference pixel 1 and reference pixel 2) located on a straight line passing through the target block are selected, and the value of the pixel sandwiched between these reference pixels is interpolated using these two reference pixels. Predict by the interpolation process by. As a result, the prediction accuracy can be increased and the prediction error can be reduced particularly for blocks having a large luminance gradient.

  However, when encoding is performed according to the raster scan order as in H.264 / AVC, in most cases, only one (reference pixel 1) value of two reference pixels located at the boundary of the target block is obtained. I can't. In the method according to the present application, the value of the other reference pixel (reference pixel 2) among the reference pixels is predicted from the pixel value included in the encoded peripheral block.

  That is, in this embodiment, when the reference pixel 1 and the reference pixel 2 can be selected from the encoded pixels, the reference pixel 1 and the reference pixel 2 are selected from the encoded pixels. Here, when the reference pixel 2 cannot be selected from the encoded pixels, the reference pixel 2 is predicted from the encoded pixels first, so that interpolation using two reference pixels as shown in FIG. 10 is performed. Prediction is performed by interpolation processing by prediction.

  As a result, even when one of the two reference pixels is not an encoded pixel, it is possible to increase the prediction accuracy and reduce the prediction error for a block having a large luminance gradient.

  FIG. 4 conceptually shows an operation example of the intra prediction encoding process according to the present embodiment. Also in this case, encoding is performed on the encoding target image according to the order of raster scanning, and prediction is performed with reference to the encoded blocks adjacent to the left, upper left, upper, and upper right of the target block. (401) represents an intra-frame prediction encoding procedure in the vertical direction. Here, “Step 1: Prediction of pixels located at the boundary of the target block (for example, reference pixel 2 in (1001)) and calculation of prediction difference” and “Step 2: Reference pixels located at both ends of the target block are used. Prediction is executed by the two-step process of “bidirectional prediction”.

  In step 1, two reference pixels used for bidirectional prediction are selected. Here, if the reference pixel cannot be selected from the encoded pixels, the reference pixel is predicted from the encoded peripheral block. For example, in (402), the values E, F, G, and H of the pixels located in the lowermost row of the target block are set to four decoded pixels A ′ located in the same row of the block adjacent to the left of the target block. Prediction is based on the average value Z of B ′, C ′, and D ′. At the same time, the difference values between these four pixels and Z are encoded as prediction differences e, f, g, and h.

  Subsequently, in step 2, other pixel values included in the target block are predicted by performing interpolation processing by interpolation prediction using the two reference pixels selected or predicted in step 1. For example, in (403), the pixels J, K, and L belonging to the same column in the target block are predicted by linear interpolation using the reference pixel I ′ and the value of Z predicted in step 1, and the prediction difference j, k , L are described, and further, these prediction differences are encoded.

  FIG. 7 conceptually shows an operation example of the intra prediction decoding process according to the present embodiment. For decoding, the procedure of FIG. 4 may be reversed (701). First, the reference pixel Z is calculated using the block adjacent to the left side of the target block (702), and the pixel located at the boundary of the target block is calculated. Decoded images E ′, F ′, G ′, and H ′ are obtained by adding to the prediction differences e ′, f ′, g ′, and h ′. Subsequently, each pixel in the target block is predicted by linear interpolation by interpolation prediction of the reference pixel I ′ included in the block adjacent to the upper side of the target block and the value Z predicted by the above processing (703), and prediction The decoded pixels J ′, K ′, and L ′ are obtained by adding to the differences j ′, k ′, and l ′.

  In the intra prediction shown in FIG. 4, one of the reference pixels is predicted using blocks adjacent to the left and upper sides of the target pixel, and thus the method according to the present application is applied to the block located at the screen end. I can't. Therefore, the bi-directional prediction according to the present embodiment can be applied to the block located at the end of the screen by performing intra prediction encoding according to the procedure of FIG. 5, for example (501). That is, in Step 1 (prediction of pixels located at the boundary of the target block and calculation of the prediction difference), the decoded values A ′ and D ′ of the two pixels located at both ends of the block adjacent to the upper side of the target block are used. By performing extrapolation prediction, the pixel H located at the boundary of the same column in the target block is predicted, and the difference h between the pixel value H and the predicted value Z is encoded as a prediction difference (502).

  Subsequently, in step 2 (bidirectional prediction using reference pixels located at both ends of the target block), the pixels E, F, and G belonging to the same column in the target block are predicted in step 1 and the reference pixel D ′. Prediction is performed by performing linear interpolation by interpolation prediction using Z, and prediction differences e, f, and g are encoded (503). In other words, the intra-screen prediction process for the block located at the screen edge has a different procedure in step 1 compared to the other cases (FIG. 4).

  By following the above procedure for the block located at the screen edge, bi-directional prediction according to the present embodiment can be performed even when a block adjacent to the left side of the target block cannot be used. In this case, decoding can be performed by following the procedure of FIG. That is, the decoding may be performed by reversing the procedure of FIG. 5 (801). First, the reference pixel Z is calculated using the block adjacent to the upper side of the target block (802), and is located at the boundary of the target block. The decoded image H is obtained by adding to the pixel prediction difference h ′. Subsequently, each pixel in the target block is predicted by linear interpolation using the reference pixel D ′ included in the block adjacent to the upper side of the target block and the value Z predicted by the above processing (903), and the prediction difference e Decoded pixels E ′, F ′, and G ′ are obtained by adding to “, f”, and g ′.

  Even in the method according to the present application, a prediction method can be selected from a plurality of candidates. For example, among the nine prediction methods (1101) used in H.264 / AVC shown in FIG. One of the prediction directions can be used. For example, when predicting in the horizontal direction (prediction direction 1) (1102), the pixel belonging to the rightmost column of the target block is predicted using the adjacent block on the upper side of the block other than the screen edge, and this is used as the reference pixel. One-way prediction is performed.

  When prediction is performed along an oblique direction such as the prediction direction 4 (1103), the pixel belonging to the rightmost column and the pixel belonging to the lowermost row of the target block are adjacent to the upper side and the left side, respectively. Bidirectional prediction is realized by predicting from the block. On the other hand, for the block located at the edge of the screen, for example, when predicting in the horizontal direction (prediction direction 1) (1104), the pixel value belonging to the leftmost column of the target block is determined using the adjacent block on the left side. Bidirectional prediction is realized by prediction. In the case of the prediction direction 7 (1105), bi-directional prediction is realized by predicting a pixel value belonging to the lowermost row of the target block using an adjacent block on the upper side. In this case, by following the procedures of (1104) and (1105), bi-directional prediction according to the present embodiment can be realized even if the blocks adjacent to the upper side and the left side cannot be used.

  By using the predictive coding technique according to the present embodiment in combination with the conventional technique, it is considered that a high compression rate can be realized according to the property of the image. For example, predictive encoding using the present technology and the conventional technology separately in units of blocks enables encoding suitable for the properties of the image. As conventional techniques, for example, an intra-screen prediction encoding method (decoding method) based on H.264 / AVC shown in FIG. 3 (FIG. 6), or an inter-screen prediction method (encoding) used also in H.264 / AVC. It is effective to use a prediction method that refers to an image different from the target image.

  FIG. 1 shows an embodiment of a moving picture encoding apparatus according to this embodiment. The moving image encoding apparatus includes an input image memory (102) that holds an input original image (101), a block dividing unit (103) that divides the input image into small regions, and a motion that detects motion in units of blocks. Similarly to the search unit (104), the intra-screen prediction unit (105) that performs intra-screen prediction processing in a procedure other than the present example, such as intra-screen prediction by H.264 / AVC (described in FIG. 3), in units of blocks. And a new intra-screen prediction unit (106) that performs intra-screen prediction (described in FIGS. 4 and 5) according to the present embodiment in block units, and a block based on the amount of motion detected by the motion search unit (104). In order to generate a prediction difference, an inter-screen prediction unit (107) that performs inter-screen prediction in units, a mode selection unit (108) that determines a predictive encoding means (prediction method and block size) that matches the characteristics of the image, and A subtracting unit (109), a frequency converting unit (110) and a quantizing unit (111) for encoding the prediction difference, A variable length encoding unit (112) for encoding according to the occurrence probability of the symbol, an inverse quantization processing unit (113) and an inverse frequency transform unit for decoding the prediction difference encoded once ( 114), an addition unit (115) for generating a decoded image using the decoded prediction difference, and a reference image memory (116) for holding the decoded image and utilizing it for later prediction Have.

  The input image memory (102) holds one image as an encoding target image from the original image (101), and divides it into fine blocks by the block dividing unit (103), and the motion search unit (104 ) To the old screen prediction unit (105) and the new screen prediction unit (106). The motion search unit (104) calculates the motion amount of the corresponding block using the decoded image stored in the reference image memory (116), and passes it to the inter-screen prediction unit (107) as a motion vector. The old screen prediction unit (105), new screen prediction unit (106), and inter-screen prediction unit (107) perform intra-screen prediction processing and inter-screen prediction processing in units of several blocks, and select the mode. In section (108), the optimum predictive encoding means is selected. Subsequently, the subtraction unit (109) generates a prediction difference by the optimal prediction encoding means and passes it to the frequency conversion unit (110). The frequency transform unit (110) and the quantization processing unit (111) each perform frequency transform such as DCT (Discrete Cosine Transformation) in units of blocks of a specified size with respect to the transmitted prediction difference. Quantization processing is performed, and the result is passed to the variable length coding processing unit (112) and the inverse quantization processing unit (113). Further, in the variable length coding processing unit (112), the prediction difference information represented by the frequency transform coefficient is necessary for predictive decoding such as a prediction direction in intra prediction encoding and a motion vector in inter prediction encoding. Along with the information, variable length coding is performed based on the occurrence probability of the symbol to generate a coded stream.

  In addition, the inverse quantization processing unit (113) and the inverse frequency transform unit (114) perform inverse frequency transform such as inverse quantization and IDCT (Inverse DCT) on the frequency transform coefficients after quantization. The prediction difference is acquired and sent to the adding unit (115). Subsequently, the adder (115) generates a decoded image and stores it in the reference image memory (116).

  FIG. 2 shows an embodiment of the moving picture decoding apparatus according to this embodiment. The video decoding device includes, for example, a variable length decoding unit (202) that performs the reverse procedure of variable length encoding on the encoded stream (201) generated by the video encoding device shown in FIG. Procedures other than the present embodiment, such as an inverse quantization processing unit (203) and an inverse frequency transform unit (204) for decoding a prediction difference, and intra-screen prediction (described in FIG. 6) using, for example, H.264 / AVC The old intra-screen prediction unit (205) that performs intra-screen prediction processing in FIG. 8, the new intra-screen prediction unit (206) that performs intra-screen prediction (described in FIGS. 7 and 8) according to the present embodiment, and intra-screen prediction An inter-screen prediction unit (207) to perform, an addition unit (208) for obtaining a decoded image, and a reference image memory (209) for temporarily storing the decoded image are provided.

  The variable length decoding unit (202) performs variable length decoding on the encoded stream (201), and obtains information necessary for prediction processing, such as the frequency transform coefficient component of the prediction difference and the prediction direction and motion vector. For the former prediction difference information, the inverse quantization processing unit (203) is used. For the information required for the latter prediction processing, the old screen prediction unit (205) and the new screen prediction are used according to the prediction means. Part (206) or the inter-screen prediction unit (207). Subsequently, the inverse quantization processing unit (203) and the inverse frequency transform unit (204) perform decoding by performing inverse quantization and inverse frequency transform on the prediction difference information, respectively. On the other hand, the old intra-screen prediction unit (205), the new intra-screen prediction unit (206), or the inter-screen prediction unit (206) uses the reference image memory (based on the information sent from the variable length decoding unit (202)). 209), the prediction process is executed, and the adder (208) generates a decoded image and stores the decoded image in the reference image memory (209).

FIG. 12 shows a procedure for encoding one frame in the embodiment of the moving picture encoding apparatus shown in FIG. First, the following processing is performed for all blocks existing in the frame to be encoded (1201). In other words, the prediction coding process is performed once for all coding modes (combination of prediction method and block size) for the corresponding block, the prediction difference is calculated, and the one with the highest coding efficiency is selected from among them. To do. As a prediction processing method, in addition to the method according to the present embodiment (hereinafter referred to as “intra-screen predictive encoding process” (1205)), for example, the intra-screen prediction method (hereinafter referred to as “ By executing the “previous intra-frame predictive encoding process” (1206)) and the inter-screen predictive encoding process (1207) and selecting an optimum one from these, it is possible to efficiently encode according to the property of the image. When selecting the one with the highest coding efficiency from among the above-mentioned many coding modes (1208), for example, the RD-Optimization method for determining the optimum coding mode from the relationship between image quality distortion and code amount is used. Therefore, it can encode efficiently. For the RD-Optimization method, for example, the method described in Reference Document 1 below may be used.
[Reference 1] G. Sullivan and T. Wiegand: “Rate-Distortion Optimization for Vid
eo Compression ”, IEEE Signal Processing Magazine, vol.15, no.6, pp.74-90, 1998.
Subsequently, frequency conversion (1209) and quantization processing (1210) are performed on the prediction difference generated in the selected encoding mode, and further, variable length encoding is performed to generate an encoded stream (1211). ). On the other hand, the quantized frequency transform coefficient is subjected to inverse quantization processing (1212) and inverse frequency transform processing (1213) to decode the prediction difference, generate a decoded image, and store it in the reference image memory (1214). When the above processing is completed for all the blocks, encoding for one frame of the image is completed (1215).

  FIG. 13 shows a detailed processing procedure of the new intra-screen prediction encoding process (1205). Here, for example, as shown in (1101), the following processing is performed for all lines along the prediction direction (1301) for all the prediction directions defined in advance (1302). That is, if the target block is located at the screen edge (1303), the prediction of the boundary pixel and the calculation of the prediction difference are performed according to the procedure of step 1 in FIG. 5 (1304). On the other hand, if the target block is not located at the edge of the screen, prediction of the boundary pixel and calculation of the prediction difference are performed according to the procedure of step 1 in FIG. 4 (1305). Subsequently, bi-directional prediction is performed according to the procedure of step 2 of FIGS. 4 and 5 using the reference pixels included in the peripheral blocks and the values of the boundary pixels predicted by the above procedure (1305). If the above processing is completed for all prediction directions and all lines, the prediction encoding processing for one block is terminated (1307).

  FIG. 14 shows a procedure for decoding one frame in the embodiment of the moving picture decoding apparatus shown in FIG. First, the following processing is performed on all blocks in one frame (1401). That is, the variable length decoding process is performed on the input stream (1402), the inverse quantization process (1403), and the inverse frequency conversion process (1404) are performed to decode the prediction difference. Subsequently, depending on which method is used to predictively encode the target block, new intra-screen predictive decoding processing (1407), conventional intra-screen predictive decoding processing (1408), or inter-screen predictive decoding processing ( 1409), the decoded image is acquired and stored in the reference image memory. When the above processing is completed for all the blocks in the frame, decoding for one frame of the image is completed (1410).

  FIG. 15 shows a detailed processing procedure of the new intra-screen predictive decoding process (1407). Here, the following processing is performed for all lines along the prediction direction (1501). That is, if the target block is located at the edge of the screen (1402), prediction of the boundary pixel and calculation of the prediction difference are performed according to the procedure of step 1 in FIG. 8 (1403). On the other hand, if the target block is not located at the screen edge, prediction of the boundary pixel and calculation of the prediction difference are performed according to the procedure of step 1 in FIG. 7 (1404). Subsequently, bi-directional prediction is performed according to the procedure of Step 2 of FIGS. 7 and 8 using the reference pixels included in the peripheral blocks and the values of the boundary pixels predicted by the above procedure (1405). When the above process is completed for all lines, the predictive decoding process for one block is terminated (1406).

  In the embodiment, DCT is cited as an example of frequency transformation, but DST (Discrete Sine Transformation), WT (Wavelet Transformation), DFT (Discrete Fourier Transformation), KLT (Karhunen- Any orthogonal transformation used for removing correlation between pixels, such as Loeve Transformation (Karunen-Leave transformation), may be used, and the prediction difference itself may be encoded without any frequency transformation. Furthermore, variable length coding is not particularly required. Further, in the embodiment, the case where the luminance component is predicted in a block unit of 4 × 4 pixel size in particular is described, but for example, the block of any size such as 8 × 8 pixel size or 16 × 16 pixel size is described. On the other hand, the present embodiment may be applied. For example, the present embodiment may be applied to prediction for a color difference component in addition to the luminance component. In the embodiment, prediction is performed along 8 directions defined in H.264 / AVC, but the number of directions may be increased or decreased.

  Subsequently, an example of a prediction formula in the present embodiment will be described. Here, a case where a luminance component is predicted in units of 4 × 4 pixel size blocks will be described. First, as shown in FIG. 16 (1601), the coordinate of the pixel located at the upper left of the target block is set to (0, 0), and the x axis is set to the horizontal right side and the y axis is set to the vertical lower side. The luminance value at the coordinates (x, y) in the target block is represented as p [x, y], and the predicted value is represented as pred4x4 [x, y]. Also, two reference pixels used for bidirectional prediction are Ref1 and Ref2, respectively. The function Min (a, b) returns the smaller of the two integers a and b, and the function ROUND (a) returns an integer value obtained by rounding off the first decimal place of the real number a.

  In the following, the calculation formulas for the prediction value pred4x4 will be described in the case of performing prediction along eight types of directions except for the DC prediction (1604) among the nine types of prediction methods (1602) to (1610).

In the case of the prediction direction 0 (Vertical) (1602), prediction is performed according to the following Equation 1.
(Equation 1)
・ When adjacent blocks on the left and upper sides are available
Ref1 = p [x, -1]
Ref2 = (p [-1, 3] + p [-2, 3] + p [-3, 3] + p [-4, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2 Ref1) * (y + 1) / 4)
・ When the adjacent block on the upper side can be used, and the adjacent block on the left side cannot be used
Ref1 = p [x, -1]
Ref2 = 2 * p [x, -1]-p [x, -4]
pred4x4 [x, y] = ROUND (Ref1 + (Ref2 Ref1) * (y + 1) / 4)
In the case of the prediction direction 1 (Horizontal) (1603), the prediction is performed according to Equation 2 below.
(Equation 2)
・ When adjacent blocks on the left and upper sides are available
Ref1 = p [-1, y]
Ref2 = (p [3, -1] + p [3, -2] + p [3, -3] + p [3, -4] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2 Ref1) * (x + 1) / 4)
・ When the adjacent block on the left side can be used and the adjacent block on the upper side cannot be used
Ref1 = p [-1, y]
Ref2 = 2 * p [-1, y]-p [-4, y]
pred4x4 [x, y] = ROUND (Ref1 + (Ref2 Ref1) * (x + 1) / 4)
In the case of the prediction direction 3 (Diagonal Down Left) (1605), prediction is performed according to the following Equation 3.
(Equation 3)
・ When adjacent blocks on the left and upper sides can be used ・ When x = y = 3
Ref1 = (p [6, -1] + 3 * p [7, -1] + 2) >> 2
Ref2 = (p [-1,2] + 3 * p [-1,3] + 2) >> 2
pred4x4 [x, y] = ROUND ((Ref1 + Ref2) / 2)
・ Other cases (when x is not 3 and y is not 3)
Ref1 = (p [x + y, -1] + 2 * p [x + y + 1, -1] + p [x + y + 2, -1] + 2) >> 2
Ref2 = (p [-1, Min (3, x + y)] + 2 * p [-1, Min (3, x + y + 1)] + p [-1, Min (3, x + y + 2)] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2 Ref1) * (y + 1) / (x + y + 2))
・ When the adjacent block on the upper side can be used and the adjacent block on the left side cannot be used ・ When x = y = 3
Ref1 = (p [6, -1] + 3 * p [7, -1] + 2) >> 2
Ref2 = (p [3, -1] + p [3, -2] + p [3, -3] + p [3, -4] + 2) >> 2
pred4x4 [x, y] = ROUND ((Ref1 + Ref2) / 2)
・ Other cases (when x is not 3 and y is not 3)
Ref1 = (p [x + y, -1] + 2 * p [x + y + 1, -1] + p [x + y + 2, -1] + 2) >> 2
Ref2 = (p [3, -1] + p [3, -2] + p [3, -3] + p [3, -4] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2 Ref1) * (y + 1) / (x + y + 2))
In the case of the prediction direction 4 (Diagonal Down Right) (1606), prediction is performed according to the following Equation 4.
(Equation 4)
-When adjacent blocks on the left and upper sides are available-When x> y
Ref1 = (p [xy-2, -1] + 2 * p [xy-1, -1] + p [xy, -1] + 2) >> 2
Ref2 = (p [3, -1] + p [3, -2] + p [3, -3] + p [3, -4] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * x / 3)
・ If x <y
Ref1 = (p [-1, yx-2] + 2 * p [-1, yx-1] + p [-1, yx] + 2) >> 2
Ref2 = (p [-1, 3] + p [-2, 3] + p [-3, 3] + p [-4, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * y / 3)
・ When x = y
Ref1 = (p [0, -1] + 2 * p [-1, -1] + p [-1, 0] + 2) >> 2
Ref2 = (p [3, -1] + p [3, -2] + p [-1, 3] + p [-2, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
In the case of the prediction direction 5 (1607) (Vertical Right), prediction is performed according to the following Expression 5.
(Equation 5)
When zVR = 2 * xy,
・ When adjacent blocks on the left and upper sides can be used ・ When zVR = 0, 2, 4, 6
Ref1 = (p [x- (y >> 1) -1, -1] + p [x- (y >> 1), -1] + 1) >> 1
Ref2 = (p [-1, 3] + p [-2, 3] + p [-3, 3] + p [-4, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (y + 1) / 4)
・ When zVR = 1, 3, 5
Ref1 = (p [x- (y >> 1) -2, -1] + 2 * p [x- (y >> 1) -1, -1] + p [x- (y >> 1), -1] + 2)
>> 2
Ref2 = (p [-1, 3] + p [-2, 3] + p [-3, 3] + p [-4, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (y + 1) / 4)
・ When zVR = -1
Ref1 = (p [-1, 0] + 2 * p [-1, -1] + p [0, -1] + 2) >> 2
Ref2 = (p [-1, 3] + p [-2, 3] + p [-3, 3] + p [-4, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (y + 1) / 4)
・ Other (when zVR = -2, -3)
Ref1 = (-1, y-1] + 2 * p [-1, y-2] + p [-1, y-3] + 2) >> 2
Ref2 = (p [-1, 3] + p [-2, 3] + p [-3, 3] + p [-4, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (y + 1) / 4)
In the case of the prediction direction 6 (Horizontal Down) (1608), the prediction is performed according to Equation 6 below.
(Equation 6)
When zHD = 2 * yx,
・ When adjacent blocks on the left and upper sides can be used ・ When zHD = 0, 2, 4, 6
Ref1 = (p [-1, y- (x >> 1) -1] + p [-1, y- (x >> 1)] + 1) >> 1
Ref2 = (p [3, -1] + p [3, -2] + p [-1, 3] + p [-2, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
・ When zHD = 1, 3, 5
Ref1 = (p [-1, y- (x >> 1) -2] + 2 * p [-1, y- (x >> 1) -1] + p [-1, y- (x >> 1)] + 2) >> 2
Ref2 = (p [3, -1] + p [3, -2] + p [-1, 3] + p [-2, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
・ When zHD = -1
Ref1 = (p [-1, 0] + 2 * p [-1, -1] + p [0, -1] + 2) >> 2
Ref2 = (p [3, -1] + p [3, -2] + p [-1, 3] + p [-2, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
・ Other (when zHD = -2, -3)
Ref1 = (p [x-1, -1] + 2 * p [x-2, -1] + p [x-3, -1] + 2) >> 2
Ref2 = (p [3, -1] + p [3, -2] + p [-1, 3] + p [-2, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
In the case of the prediction direction 7 (Vertical Left) (1609), prediction is performed according to Equation 7 below.
(Equation 7)
-When adjacent blocks on the left and upper sides can be used-When y = 0, 2
Ref1 = (p [x + (y >> 1), -1] + p [x + (y >> 1) +1, -1] + 1) >> 1
Ref2 = (p [-1, 3] + p [-2, 3] + p [-3, 3] + p [-4, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (y + 1) / (x + y + 2))
・ Other than that (when y = 1, 3)
Ref1 = (p [x + (y >> 1), -1] + 2 * p [x + (y >> 1) +1, -1] + p [x + (y >> 1) +2, -1] + 2) >> 2
Ref2 = (p [-1, 3] + p [-2, 3] + p [-3, 3] + p [-4, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (y + 1) / (x + y + 2))
・ When the adjacent block on the upper side can be used and the adjacent block on the left side cannot be used ・ When y = 0, 2
Ref1 = (p [x + (y >> 1), -1] + p [x + (y >> 1) +1, -1] + 1) >> 1
Ref2 = 2 * p [x, -1]-p [x, -4]
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (y + 1) / (x + y + 2))
・ Other than that (when y = 1, 3)
Ref1 = (p [x + (y >> 1), -1] + 2 * p [x + (y >> 1) +1, -1] + p [x + (y >> 1) +2, -1] + 2) >> 2
Ref2 = 2 * p [x, -1]-p [x, -4]
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (y + 1) / (x + y + 2))
In the case of the prediction direction 8 (Horizontal Up) (1610), prediction is performed according to the following Expression 8.
(Equation 8)
When zHU = x + 2 * y,
・ When adjacent blocks on the left and upper sides can be used ・ When zHU = 0, 2, 4
Ref1 = (p [-1, y + (x >> 1)] + p [-1, y + (x >> 1) +1] + 1) >> 1
Ref2 = (p [3, -1] + p [3, -2] + p [-1, 3] + p [-2, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
・ When zHU = 1 or 3
Ref1 = (p [-1, y + (x >> 1)] + 2 * p [-1, y + (x >> 1) +1] + p [-1, y + (x >> 1) +2] + 2) >> 2
Ref2 = (p [3, -1] + p [3, -2] + p [-1, 3] + p [-2, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
・ When zHU = 5
Ref1 = (p [-1, 2] + 3 * p [-1, 3] + 2) >> 2
Ref2 = (p [3, -1] + p [3, -2] + p [-1, 3] + p [-2, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
・ Other (when zHU> 5)
Ref1 = p [-1, 3]
Ref2 = (p [3, -1] + p [3, -2] + p [-1, 3] + p [-2, 3] + 2) >> 2
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
・ When the adjacent block on the left side can be used and the adjacent block on the upper side cannot be used ・ When zHU = 0, 2, 4
Ref1 = (p [-1, y + (x >> 1)] + p [-1, y + (x >> 1) +1] + 1) >> 1
Ref2 = 2 * p [-1, y]-p [-4, y]
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
・ When zHU = 1 or 3
Ref1 = (p [-1, y + (x >> 1)] + 2 * p [-1, y + (x >> 1) +1] + p [-1, y + (x >> 1) +2] + 2) >> 2
Ref2 = 2 * p [-1, y]-p [-4, y]
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
・ When zHU = 5
Ref1 = (p [-1, 2] + 3 * p [-1, 3] + 2) >> 2
Ref2 = 2 * p [-1, y]-p [-4, y]
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
・ Other (when zHU> 5)
Ref1 = p [-1, 3]
Ref2 = 2 * p [-1, y]-p [-4, y]
pred4x4 [x, y] = ROUND (Ref1 + (Ref2-Ref1) * (x + 1) / 4)
In the above-described embodiment, prediction (for example, prediction of boundary pixels E, F, G, and H in step 1 in FIG. 4) that cannot be selected from the encoded (decoded) pixels of the two reference pixels is performed mainly. However, the average value of pixels belonging to the same row (or column) of the encoded adjacent block is not particularly limited. For example, a minimum value, a maximum value, or an intermediate value may be used. Any calculation formula such as extrapolation prediction or interpolation prediction using these pixels may be used. In particular, when performing interpolation prediction, any method such as linear interpolation and nonlinear interpolation may be used, and when performing extrapolation prediction, for example, linear / parabolic approximation using the least square method, Newton interpolation, Any model such as Lagrangian interpolation may be used for prediction.

  FIG. 17 shows a conceptual diagram when the boundary pixels E, F, G, and H are predicted by extrapolation prediction in Step 1 (402) of FIG. Here, coordinates are set on the horizontal axis and luminance values are set on the vertical axis. (1701) shows a case where linear approximation is performed, and the least square method is used by using the decoded pixels A ′, B ′, C ′, and D ′ located in the lowermost row of the adjacent block on the left side. Thus, a predicted straight line is calculated, and points corresponding to the coordinates of the boundary pixels E, F, G, and H on the straight line are set as predicted values. (1702) shows a case where the same extrapolation prediction is performed by a curve approximation method such as parabolic approximation, Newton interpolation, Lagrangian interpolation, etc. In this case, the decoded pixels A ′, B ′, C ′, and D ′ are set as follows. The boundary pixels E, F, G, and H are predicted based on the prediction curve calculated using the above. The boundary pixel predicted by the above processing is used as one of the reference pixels when the bidirectional prediction in step 2 in FIG. 4 is performed, for example.

Further, the information used for prediction of the reference pixel is not particularly limited to pixels belonging to the same row (or column) as long as it is information on adjacent blocks. Furthermore, in the embodiment, bi-directional prediction (for example, prediction of pixels J, K, and L in step 2 in FIG. 4) is performed by linear interpolation using two reference pixels, but two reference pixels sandwiching the target pixel As long as the method is used, the interpolation method is not particularly limited. For example, the predicted value of the target pixel may be represented by an arbitrary linear expression of two reference pixels. In this case, assuming that the two reference pixels are Ref1 and Ref2, the predicted value pred4x4 is calculated by the following formula 9.
(Equation 9)
pred4x4 [x, y] = a * Ref1 + b * Ref2
Here, a and b represent real constants, respectively, and these values may be set to constant values in advance or may be specified in units of blocks. Further, the intra prediction encoding method (decoding method) for the screen edge shown in FIG. 5 (FIG. 8) can be applied even if it is not a block at the screen edge. That is, this method may be applied to blocks other than the screen edge.

  FIG. 18 shows a configuration example of a portion related to the encoding parameter to be set in units of blocks in the encoded stream generated when this embodiment is used. Here, as in the processing unit in H.264 / AVC, a case is described in which the encoding mode is determined in units of macroblocks of a fixed length size. The macroblock can be divided into finer blocks, and predictive coding is performed in units of the divided blocks. At this time, the macroblock number (1801) for specifying the coordinates for each macroblock, the encoding mode number (1802) indicating the prediction method and the block size, and the motion vector and intra-frame prediction for inter-screen prediction, for example Information necessary for the prediction (1803) and prediction difference information (1804), such as the prediction direction at the time of prediction, are variable-length encoded and stored. In particular, the encoding mode number (1802) may be assigned sequentially to all prediction means, or may be expressed by a different bit for each prediction means.

  FIG. 19 shows another configuration example for the part related to the encoding parameter to be set in block units shown in FIG. In this example, a flag (1903) indicating whether to perform prediction using an adjacent pixel according to the present embodiment is added to the macroblock number (1901) and the prediction method (1902), and further necessary for prediction. Information (1904) and prediction difference information (1905) are stored.

DESCRIPTION OF SYMBOLS 101 ... Original image, 102 ... Original image memory, 103 ... Block division part, 104 ... Motion search part, 105 ... Old screen prediction part, 106 ... New screen prediction part, 107 ... Inter screen prediction part, 108 ... Mode selection 109: Subtraction unit, 110 ... Frequency conversion unit, 111 ... Quantization processing unit, 112 ... Variable length coding unit, 113 ... Inverse quantization processing unit, 114 ... Inverse frequency conversion unit, 115 ... Addition unit, 116 ... Reference image memory, 201 ... encoded stream, 202 ... variable length decoding unit, 203 ... dequantization processing unit, 204 ... inverse frequency transform unit, 205 ... prediction unit in old screen, 206 ... prediction unit in new screen, 207 ... Inter-screen prediction unit, 208 ... Adding unit, 209 ... Reference image memory

Claims (1)

An image decoding method for decoding an image by performing a prediction process,
A prediction mode selection step of selecting one prediction mode among a plurality of prediction modes for the prediction process;
A prediction step of performing prediction processing of the prediction mode selected in the prediction mode selection step,
In the prediction mode that can be selected in the prediction mode selection step,
In the intra prediction mode, the first reference pixel is generated using the pixels in the decoded block adjacent to the left side or the upper side of the decoding target block, and is the upper side or the left side of the decoding target block. A first screen for generating a prediction pixel by a calculation process using a pixel value of a decoded second reference pixel adjacent to a different side from the first reference pixel and a pixel value of the first reference pixel Intra prediction mode,
A second intra prediction mode that is an intra prediction mode different from the first intra prediction mode and that generates a prediction pixel using a decoded pixel value adjacent to the decoding target block;
There is
In the first intra-screen prediction mode, a prediction pixel is generated by an arithmetic process using an interpolation process between a pixel value of the second reference pixel and a pixel value of the first reference pixel.
Image decoding method.

Images