JP7483422B2 - Intra prediction device, image decoding device, and program - Google Patents

Description

The present invention relates to an intra prediction device, an image encoding device, an image decoding device, and a program.

Research has been conducted into video coding methods to compress the amount of data for still and moving images during transmission and storage. In recent years, video coding technology has seen the spread of ultra-high resolution video, such as 8K-SHV, and coding methods such as AVC/H.264 and HEVC/H.265 are known as methods for transmitting huge amounts of video data.

The evaluation software (VTM) for VVC (The Next-Next Generation Codec), a next-generation video coding method being standardized jointly by MPEG and ITU, uses intra-prediction (inter-frame prediction) that takes advantage of spatial correlation within a frame (see non-patent document 1). Using decoded reference pixels around the coding target block (coding unit), the encoder selects the optimal mode from a total of 67 prediction modes consisting of planar prediction, DC prediction, and 65 directional predictions, and the selected information is sent to the decoder.

As a method for improving the prediction accuracy of intra prediction, a prediction image synthesis method has been proposed in which a prediction image is generated using two intra prediction modes and each pixel of the two prediction images is added together to generate a new prediction image (see Non-Patent Document 2). Specifically, a prediction image is generated by adding a directional prediction image, which is a prediction image generated using one of the above-mentioned 65 directional prediction modes, and a planar prediction image, which is a prediction image generated using the planar mode.

JVET-M1001 "Versatile Video Coding (Draft 4)" JVET-M0458 "Non-CE3: Combined-Hypothesis Intra-Prediction"

However, directional prediction has the drawback that while the prediction accuracy is high for pixels close to the reference pixel, the prediction accuracy can decrease as the pixel moves away from the reference pixel.

However, the predicted image synthesis method described in Non-Patent Document 2 merely averages the directional predicted image and the planar predicted image on a pixel-by-pixel basis, and does not take into account the shortcomings of directional prediction, leaving room for improvement in terms of further improving the prediction accuracy of intra prediction.

The present invention aims to provide an intra prediction device, an image encoding device, an image decoding device, and a program that further improve the prediction accuracy of intra prediction using a prediction image synthesis technique.

The intra prediction device according to the first aspect is an intra prediction device that performs intra prediction on an image block obtained by dividing an original image, and includes a first predicted image generation unit that predicts the image block using a first intra prediction mode, which is a directional prediction, to generate a first predicted image, a second predicted image generation unit that predicts the image block using a second intra prediction mode, which is a non-directional prediction, to generate a second predicted image, a weighting factor determination unit that determines a weighting factor used when weighting and synthesizing the first predicted image and the second predicted image for each pixel based on the position of the pixel, and an image synthesis unit that weights and synthesizes the first predicted image and the second predicted image for each pixel using the weighting factor determined for each pixel by the weighting factor determination unit.

The image encoding device according to the second aspect is characterized in that it includes the intra prediction device according to the first aspect.

The image decoding device according to the third aspect is characterized in that it includes the intra prediction device according to the first aspect.

The program according to the fourth aspect is intended to cause a computer to function as the intra prediction device according to the first aspect.

The present invention provides an intra prediction device, an image encoding device, an image decoding device, and a program that further improve the prediction accuracy of intra prediction using a prediction image synthesis technique.

FIG. 1 is a diagram showing a configuration of an image encoding device according to an embodiment. FIG. 1 is a diagram showing prediction modes of intra prediction according to an embodiment. FIG. 2 is a diagram showing a configuration of an intra-prediction unit of an image encoding device according to an embodiment. FIG. 11 is a diagram illustrating an example of the operation of an intra prediction unit according to the embodiment. FIG. 11 is a diagram illustrating a first example of an operation of a weighting coefficient determination unit according to the embodiment. FIG. 11 is a diagram illustrating a second example of the operation of the weighting coefficient determination unit according to the embodiment. FIG. 2 is a diagram showing a configuration of an image decoding device according to an embodiment. FIG. 2 is a diagram showing a configuration of an intra-prediction unit of an image decoding device according to an embodiment. FIG. 11 is a diagram illustrating an example of an operation flow of an intra prediction unit according to the embodiment.

An image encoding device and an image decoding device according to an embodiment will be described with reference to the drawings. The image encoding device and the image decoding device according to the embodiment respectively encode and decode moving images such as MPEG. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

<Configuration of Image Encoding Device>
First, an image encoding device according to this embodiment will be described. Fig. 1 is a diagram showing the configuration of an image encoding device 1 according to this embodiment.

As shown in FIG. 1, the image coding device 1 includes a block division unit 100, a subtraction unit 110, a transformation and quantization unit 120, an entropy coding unit 130, an inverse quantization and inverse transformation unit 140, a synthesis unit 150, a memory 160, and a prediction unit 170.

The block division unit 100 divides an original image, which is an input image in units of frames (or pictures) constituting a moving image, into a plurality of image blocks, and outputs the image blocks obtained by the division to the subtraction unit 1.
The image data is output to the image decoder 10. The size of the image block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the image block is not limited to a square and may be a rectangle (rectangle). The image block is a unit (block to be coded) for coding by the image coding device 1 and a unit (block to be coded) for decoding by the image decoding device. Such an image block is sometimes called a CU (Coding Unit).

The subtraction unit 110 calculates a prediction residual that represents the difference (error) between the encoding target block input from the block division unit 100 and a predicted image obtained by predicting the encoding target block by the prediction unit 170. Specifically, the subtraction unit 110 calculates the prediction residual by subtracting each pixel value of the predicted image from each pixel value of the block, and outputs the calculated prediction residual to the transformation/quantization unit 120.

The transform/quantization unit 120 performs orthogonal transform processing and quantization processing on a block-by-block basis. The transform/quantization unit 120 has a transform unit 121 and a quantization unit 122.

The transform unit 121 performs an orthogonal transform process on the prediction residual input from the subtraction unit 110 to calculate an orthogonal transform coefficient, and outputs the calculated orthogonal transform coefficient to the quantization unit 122. Examples of orthogonal transform include discrete cosine transform (DCT), discrete sine transform (DST), and Karhunen-Loeve transform (KLT).

The quantization unit 122 quantizes the orthogonal transform coefficients input from the transformation unit 121 using a quantization parameter (Qp) and a quantization matrix, and outputs the quantized orthogonal transform coefficients to the entropy coding unit 130 and the inverse quantization and inverse transform unit 140. Note that the quantization parameter (Qp) is a parameter that is commonly applied to each orthogonal transform coefficient in a block, and is a parameter that determines the coarseness of quantization. The quantization matrix is a matrix whose elements are the quantization values used when quantizing each orthogonal transform coefficient.

The entropy coding unit 130 performs entropy coding on the orthogonal transform coefficients input from the quantization unit 122, performs data compression to generate coded data (bit stream), and outputs the coded data to the outside of the image coding device 1. For entropy coding, Huffman codes, CABAC (Context-based Adaptive Binary Arithmetic Coding), etc. can be used. Note that the entropy coding unit 130 receives information such as syntax related to prediction from the prediction unit 170, and also performs entropy coding on the input information.

The inverse quantization and inverse transform unit 140 performs inverse quantization processing and inverse orthogonal transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 140 includes an inverse quantization unit 141 and an inverse transform unit 142.

The inverse quantization unit 141 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores the orthogonal transform coefficients by inverse quantizing the orthogonal transform coefficients input from the quantization unit 122 using a quantization parameter (Qp) and a quantization matrix, and outputs the restored orthogonal transform coefficients to the inverse transform unit 142.

The inverse transform unit 142 performs an inverse orthogonal transform process corresponding to the orthogonal transform process performed by the transform unit 121. For example, if the transform unit 121 performs a discrete cosine transform, the inverse transform unit 142 performs an inverse discrete cosine transform. The inverse transform unit 142 performs an inverse orthogonal transform process on the orthogonal transform coefficients input from the inverse quantization unit 141 to restore the prediction residual, and outputs the restored prediction residual, which is the restored prediction residual, to the synthesis unit 150.

The synthesis unit 150 synthesizes the reconstructed prediction residual input from the inverse transform unit 142 with the predicted image input from the prediction unit 170 on a pixel-by-pixel basis. The synthesis unit 150 adds each pixel value of the reconstructed prediction residual to each pixel value of the predicted image to reconstruct (decode) the block to be coded, and outputs the decoded image on a block-by-block basis to the memory 160. Such a decoded image is sometimes called a reconstructed image.

The memory 160 stores the decoded image input from the synthesis unit 150. The memory 160 stores the decoded image on a frame-by-frame basis. The memory 160 outputs the stored decoded image to the prediction unit 170. Note that a loop filter may be provided between the synthesis unit 150 and the memory 160. Note that a part of the memory 160 may be included in the prediction unit 170.

The prediction unit 170 performs prediction on a block-by-block basis. The prediction unit 170 has an inter prediction unit 171, an intra prediction unit 172, and a switching unit 173.

The inter prediction unit 171 uses the decoded image stored in the memory 160 as a reference image to calculate a motion vector using a technique such as block matching, predicts the block to be coded, generates an inter prediction image, and outputs the generated inter prediction image to the switching unit 173.

The inter prediction unit 171 selects an optimal inter prediction method from among inter prediction using multiple reference images (typically bi-prediction) and inter prediction using one reference image (unidirectional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 171 outputs information related to the inter prediction (motion vectors, etc.) to the entropy coding unit 130.

The intra prediction unit 172 generates an intra prediction image by referring to decoded pixel values in the vicinity of the block to be coded among the decoded images stored in the memory 160, and outputs the generated intra prediction image to the switching unit 173. The intra prediction unit 172 also outputs syntax related to the selected prediction mode to the entropy coding unit 130. Hereinafter, the image block to be subjected to intra prediction is referred to as the intra prediction target block.

The intra prediction unit 172 selects an optimal prediction mode to be applied to the intra prediction target block from among multiple prediction modes, and predicts the intra prediction target block using the selected prediction mode.

Figure 2 is a diagram showing prediction modes of intra prediction according to this embodiment. As shown in Figure 2, there are 67 prediction modes from 0 to 66. Prediction mode mode "0" is planar prediction, prediction mode mode "1" is DC prediction, and prediction modes "2" to "66" are directional prediction. In directional prediction, the direction of the arrow indicates the prediction direction, the starting point of the arrow indicates the position of the pixel to be predicted, and the end point of the arrow indicates the position of the reference pixel used to predict this pixel to be predicted. Modes "2" to "18" are prediction modes that refer to reference pixels only on the left side of the target block for intra prediction. On the other hand, modes "50" to "66" are prediction modes that refer to reference pixels only on the upper side of the target block for intra prediction.

The switching unit 173 switches between the inter-prediction image input from the inter-prediction unit 171 and the intra-prediction image input from the intra-prediction unit 172, and outputs one of the predicted images to the subtraction unit 110 and the synthesis unit 150.

FIG. 3 is a diagram showing the configuration of the intra prediction unit 172 according to this embodiment. The intra prediction unit 172 corresponds to an intra prediction device provided in the image encoding device 1.

As shown in FIG. 3, the intra prediction unit 172 includes a memory 160a, a first predicted image generation unit 172a, a second predicted image generation unit 172b, a weighting coefficient determination unit 172c, and an image synthesis unit 172d.

Memory 160a is part of memory 160 shown in FIG. 1. Memory 160a stores reference pixels, which are decoded pixels that are referenced during intra prediction.

The first predicted image generating unit 172a predicts the intra prediction target block using a first intra prediction mode, which is a directional prediction, to generate a directional predicted image (first predicted image), and outputs the generated directional predicted image to the image synthesis unit 172d. Specifically, the first predicted image generating unit 172a refers to the reference pixels stored in the memory 160a, and generates a directional predicted image using one of 65 directional prediction modes.

The second predicted image generating unit 172b predicts the intra prediction target block using a second intra prediction mode, which is a non-directional prediction, to generate a predicted image (second predicted image), and outputs the generated predicted image to the image synthesis unit 172d. Specifically, the second predicted image generating unit 172b refers to the reference pixels stored in the memory 160a, and generates a directional predicted image using a predetermined non-directional second intra prediction mode.

The second intra prediction mode may be any non-directional prediction mode, but in this embodiment, an example will be described in which the second intra prediction mode is planar prediction.

The weighting coefficient determination unit 172c determines a weighting coefficient for each pixel position (coordinates) of the block to be intra-predicted, and outputs the determined weighting coefficient to the image synthesis unit 172d. The weighting coefficients output by the weighting coefficient determination unit 172c are used when weighting and synthesizing the directional prediction image generated by the first prediction image generation unit 172a and the planar prediction image generated by the second prediction image generation unit 172b. Specifically, the weighting coefficient determination unit 172c determines the weighting coefficient to be used when weighting and synthesizing the directional prediction image and the planar prediction image for each pixel, based on the position of the pixel.

The image synthesis unit 172d uses the weighting coefficients input from the weighting coefficient determination unit 172c to weight-synthesize the directional prediction image input from the first prediction image generation unit 172a and the planar prediction image input from the second prediction image generation unit 172b for each pixel, and outputs the prediction image after weighting synthesis as an intra-prediction image.

Figure 4 is a diagram showing an example of the operation of the intra prediction unit 172 according to this embodiment. In Figure 4, an example is shown in which the intra prediction target block is a square of 8 x 8 pixels, but the intra prediction target block does not have to be a square. Also, the pixels indicated by circles in Figure 4 represent reference pixels (decoded reference pixels).

As shown in FIG. 4A, the second predicted image generating unit 172b predicts the intra-prediction target block by planar prediction to generate a planar predicted image. Specifically, planar prediction generates a predicted pixel value by interpolation prediction using four reference pixels at the starting points of the four arrows on the top, bottom, left, and right in FIG. 4A.

As shown in Fig. 4B, the first predicted image generating unit 172a predicts the intra prediction target block in a directional prediction mode to generate a directional predicted image. Since directional prediction generates a predicted pixel value by extrapolating a reference pixel along a prediction direction, the prediction accuracy is high for pixels located close to the reference pixel, but the prediction accuracy may decrease as the pixel moves away from the reference pixel.

The weighting factor determination unit 172c determines the weighting factor used when weighting and synthesizing the directional prediction image and the planar prediction image for each pixel based on the position of the pixel. In this embodiment, the weighting factor determination unit 172c determines the weighting factor α (second weighting factor) to be applied to the planar prediction image and the weighting factor β (first weighting factor) to be applied to the directional prediction image.

As shown in FIG. 4(c), the image synthesis unit 172d applies a weighting factor β(x, y) to each pixel (x, y) of the directional prediction image generated by the first predicted image generation unit 172a, and applies a weighting factor α(x, y) to each pixel (x, y) of the planar prediction image generated by the second predicted image generation unit 172b, synthesizes the directional prediction image and the planar prediction image on a pixel-by-pixel basis, and outputs the synthesized prediction image as an intra-prediction image. Here, with the top left vertex of the prediction target block as a reference, x indicates the horizontal coordinate position, and y indicates the vertical coordinate position.

In this embodiment, the image synthesis unit 172d multiplies the pixel pred _α (x, y) of the planar predicted image by a weighting coefficient α(x, y) and multiplies the pixel pred _β (x, y) of the directional predicted image by a weighting coefficient β(x, y) according to the following equation (1), and outputs the synthesized predicted image as the pixel pred(x, y) of the intra predicted image.

pred(x,y)=(α(x,y)×pred _α (x,y) + β(x,y)×pred _β (x,y) + 32) >> 6
... (1)

Fig. 5 is a diagram showing a first example of the operation of the weighting coefficient determination unit 172c according to this embodiment. In Fig. 5, an example is shown in which the intra-prediction target block is a square of 4x4 pixels, but the intra-prediction target block does not have to be a square. Also, the pixels indicated by circles in Fig. 5 represent reference pixels (decoded reference pixels).

As shown in FIG. 5, the weighting coefficient determination unit 172c determines the weighting coefficient α(x, y) to be applied to the planar predicted image and the weighting coefficient β(x, y) to be applied to the directional predicted image based on the pixel position (x, y). Specifically, the weighting coefficient determination unit 172c increases the ratio of β(x, y) to α(x, y) as the pixel position (x, y) is closer to the reference pixel. In other words, the weighting coefficient determination unit 172c decreases the ratio of β(x, y) to α(x, y) as the pixel position (x, y) is farther from the reference pixel.

Directional prediction has the disadvantage that the accuracy of prediction can decrease as the pixel moves away from the reference pixel. According to this embodiment, the ratio of the weighting coefficient β(x, y) of a pixel in the directional predicted image to the weighting coefficient α(x, y) of a pixel in the planar predicted image is decreased as the pixel moves away from the reference pixel, so that the disadvantage of directional prediction can be compensated for by planar prediction.

Directional prediction also has the advantage that it has high prediction accuracy for pixels located close to the reference pixel. According to this embodiment, the ratio of the weighting coefficient β(x, y) of a pixel in the directional predicted image to the weighting coefficient α(x, y) of a pixel in the planar predicted image is increased as the pixel approaches the reference pixel, making it possible to take advantage of the advantages of directional prediction.

For example, the weighting coefficient determination unit 172c determines the weighting coefficients α(x, y) and β(x, y) using the following formula (2):

α(x,y) = 64 - β(x,y)
β(x,y) = 32 - (z << 5) >> shift
z=min(x,y)
shift = ( _log2 (width) + _log2 (height) + 1) >> 1
... (2)

where width indicates the width of the block to be predicted, and height indicates the height of the block to be predicted.

As shown in formula (2), the value of z changes according to the pixel position (x, y), and the values of weighting coefficients α(x, y) and β(x, y) change according to the change in the value of z. Specifically, the larger the values of x and y, the larger the value of z becomes, and accordingly the value of weighting coefficient β(x, y) becomes smaller and the value of weighting coefficient α(x, y) becomes larger.

The example in FIG. 5 shows a case where x = 1 and y = 1, i.e., at the pixel position closest to the reference pixel, α:β = 32:32, and the pixel values of the planar predicted image and the pixel values of the directional predicted image are combined in equal proportions. However, it is not necessary for the pixel values of each pixel to be combined in the same ratio at the pixel position closest to the reference pixel, and it may be determined that the proportion of the pixel values of the directional predicted image is larger at the pixel position closest to the reference pixel.

Fig. 6 is a diagram showing a second operation example of the weighting coefficient determination unit 172c according to this embodiment. In Fig. 6, an example is shown in which the intra-prediction target block is a square shape of 4x4 pixels, but the intra-prediction target block does not have to be a square. Also, the pixels indicated by circles in Fig. 6 represent reference pixels (decoded reference pixels).

As shown in FIG. 6, the weighting coefficient determination unit 172c determines the weighting coefficient α(x, y) to be applied to the planar predicted image and the weighting coefficient β(x, y) to be applied to the directional predicted image, taking into consideration not only the pixel position (x, y) but also the prediction direction in the directional prediction (i.e., the prediction mode of the directional prediction).

Specifically, the weighting coefficient determination unit 172c obtains the prediction mode used by the first predicted image generation unit 172a when generating a directional predicted image, and determines the weighting coefficients α(x, y) and β(x, y) in a different manner depending on the prediction mode.

As shown in FIG. 6(a), for prediction modes "2" to "18" that reference pixels on the left side of a block to be intra-predicted, the weighting factor determination unit 172c reduces the ratio of the weighting factor β(x, y) of a pixel of the directional prediction image to the weighting factor α(x, y) of a pixel of the planar prediction image as the pixel moves away from the left reference pixel.

For example, when the prediction mode is one of "2" to "18", the weighting coefficient determination unit 172c determines the weighting coefficients α(x, y) and β(x, y) according to the following formula (3).

α(x,y) = 64 - β(x,y)
β(x,y) = 32 - (x << 5) >> shift
shift = ( _log2 (width) + _log2 (height) + 1) >> 1
...(3)

As shown in equation (3), the weighting coefficient β(x
, y) becomes smaller, the value of the weighting factor α(x, y) becomes larger.

As shown in FIG. 6(c), for prediction modes "50" to "66" that reference upper reference pixels of the intra prediction target block, the weighting factor determination unit 172c decreases the ratio of the weighting factor β(x, y) of the pixel of the directional prediction image to the weighting factor α(x, y) of the pixel of the planar prediction image as the pixel moves away from the upper reference pixel.

For example, when the prediction mode is one of "50" to "66", the weighting coefficient determination unit 172c determines the weighting coefficients α(x, y) and β(x, y) according to the following formula (4).

α(x,y) = 64 - β(x,y)
β(x,y) = 32 - (y << 5) >> shift
shift = ( _log2 (width) + _log2 (height) + 1) >> 1
...(4)

As shown in equation (4), the larger the value of the horizontal pixel position y, the smaller the value of the weighting coefficient β(x, y) and the larger the value of the weighting coefficient α(x, y).

As shown in FIG. 6(b), the weighting factor determination unit 172c determines the weighting factors α(x,y) and β(x,y) by the above-mentioned formula (2) for prediction modes "19" to "49" that refer to the reference pixels on the left and above of the intra-prediction target block. That is, when the prediction mode is any of "50" to "66", the method of determining the weighting factors α(x,y) and β(x,y) is the same as that of the operation example 1.

In the second operation example, three patterns are used depending on the prediction direction of the directional prediction, and an example is described in which the proportion of pixel values of the planar predicted image increases as the pixel moves away from the reference pixel actually referenced in the directional prediction. However, the case may be divided into more than three patterns. In addition, the number of calculation formulas for determining the weighting coefficients α(x, y) and β(x, y) may be defined in the same number as the number of directional predictions.

<Configuration of Image Decoding Device>
Next, an image decoding device according to this embodiment will be described below. Fig. 7 is a diagram showing the configuration of an image decoding device 2 according to this embodiment.

As shown in FIG. 7, the image decoding device 2 has an entropy decoding unit 200, an inverse quantization/inverse transform unit 210, a synthesis unit 220, a memory 230, and a prediction unit 240.

The entropy decoding unit 200 decodes the encoded data generated by the image encoding device 1, and outputs the quantized orthogonal transform coefficients to the inverse quantization and inverse transform unit 210. The entropy decoding unit 200 also acquires syntax related to prediction (intra prediction and inter prediction), and outputs the acquired syntax to the prediction unit 240.

The inverse quantization and inverse transform unit 210 performs inverse quantization processing and inverse orthogonal transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 210 includes an inverse quantization unit 211 and an inverse transform unit 212.

The inverse quantization unit 211 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122 of the image encoding device 1. The inverse quantization unit 211 restores the orthogonal transform coefficients of the block to be decoded by inverse quantizing the quantized orthogonal transform coefficients input from the entropy decoding unit 200 using a quantization parameter (Qp) and a quantization matrix, and outputs the restored orthogonal transform coefficients to the inverse transform unit 212.

The inverse transform unit 212 performs inverse orthogonal transform processing corresponding to the orthogonal transform processing performed by the transform unit 121 of the image encoding device 1. The inverse transform unit 212 performs inverse orthogonal transform processing on the orthogonal transform coefficients input from the inverse quantization unit 211 to restore the prediction residual, and outputs the restored prediction residual (restored prediction residual) to the synthesis unit 220.

The synthesis unit 220 reconstructs (decodes) the original block by synthesizing the prediction residual input from the inverse transform unit 212 and the prediction image input from the prediction unit 240 on a pixel-by-pixel basis, and outputs the decoded image on a block-by-block basis to the memory 230.

The memory 230 stores the decoded image input from the synthesis unit 220. The memory 230 stores the decoded image on a frame-by-frame basis. The memory 230 outputs the decoded image on a frame-by-frame basis to the outside of the image decoding device 2. Note that a loop filter may be provided between the synthesis unit 220 and the memory 230. Also, a part of the memory 230 may be included in the prediction unit 240.

The prediction unit 240 performs prediction on a block-by-block basis. The prediction unit 240 has an inter prediction unit 241, an intra prediction unit 242, and a switching unit 243.

The inter prediction unit 241 predicts the block to be decoded by inter prediction using the decoded image stored in the memory 230 as a reference image. The inter prediction unit 241 generates an inter prediction image by performing inter prediction according to the syntax, motion vectors, etc. input from the entropy decoding unit 200, and outputs the generated inter prediction image to the switching unit 243.

The intra prediction unit 242 references the decoded image stored in the memory 230, and generates an intra prediction image by predicting the block to be decoded using intra prediction based on the syntax input from the entropy decoding unit 200, and outputs the generated intra prediction image to the switching unit 243.

The switching unit 243 switches between the inter-prediction image input from the inter-prediction unit 241 and the intra-prediction image input from the intra-prediction unit 242, and outputs one of the predicted images to the synthesis unit 220.

FIG. 8 is a diagram showing the configuration of the intra prediction unit 242 according to this embodiment. The intra prediction unit 242 corresponds to the intra prediction device provided in the image decoding device 2. The intra prediction unit 242 performs the same operation as the intra prediction unit 172 provided in the image encoding device 1.

As shown in FIG. 8, the intra prediction unit 242 has a memory 230a, a first predicted image generation unit 242a, a second predicted image generation unit 242b, a weighting coefficient determination unit 242c, and an image synthesis unit 242d.

Memory 230a is part of memory 230 shown in FIG. 7. Memory 230a stores reference pixels, which are decoded pixels that are referenced during intra prediction.

The first predicted image generating unit 242a predicts the intra prediction target block using a first intra prediction mode, which is a directional prediction, to generate a directional predicted image (first predicted image), and outputs the generated directional predicted image to the image synthesis unit 242d. Specifically, the first predicted image generating unit 242a refers to the reference pixels stored in the memory 230a, and generates a directional predicted image using the directional prediction mode indicated by the syntax input from the entropy decoding unit 200.

The second predicted image generating unit 242b predicts the intra prediction target block using a second intra prediction mode, which is a non-directional prediction, to generate a predicted image (second predicted image), and outputs the generated predicted image to the image synthesis unit 242d. Specifically, the second predicted image generating unit 242b refers to the reference pixels stored in the memory 230a, and generates a directional predicted image using a predetermined non-directional second intra prediction mode. In this embodiment, the second intra prediction mode is planar prediction.

The weighting coefficient determination unit 242c determines a weighting coefficient to be used when weighting and synthesizing the directional prediction image and the planar prediction image for each pixel based on the position of the pixel, and outputs the determined weighting coefficient to the image synthesis unit 242d. The operation of the weighting coefficient determination unit 242c is the same as the operation examples 1 and 2 described above.

The image synthesis unit 242d uses the weighting coefficients input from the weighting coefficient determination unit 242c to weight-synthesize the directional prediction image input from the first prediction image generation unit 242a and the planar prediction image input from the second prediction image generation unit 242b, and outputs the prediction image after weighting synthesis as an intra-prediction image. The image synthesis unit 242d performs the weighting synthesis process according to the above-mentioned formula (1).

<Example of intra prediction operation flow>
Next, an example of an operation flow of intra prediction according to this embodiment will be described. The operation of intra prediction is the same in the image encoding device 1 and the image decoding device 2, but here, the operation of intra prediction (intra prediction unit 242) in the image decoding device 2 will be described. Fig. 9 is a diagram showing an example of the operation flow of the intra prediction unit 242. Note that the flow described below is merely an example, and the type of flag (syntax) and the order of sending can be changed as appropriate.

First, the entropy decoding unit 200 decodes syntax indicating the intra prediction mode selected by the image encoding device 1. If this syntax indicates a mode other than the directional prediction mode (DC prediction or planar prediction) (step S1: NO), the intra prediction unit 242 performs intra prediction similar to the conventional method (step S7).

Secondly, the entropy decoding unit 200 decodes syntax indicating whether or not to apply a predictive image synthesis method. If this syntax indicates that a predictive image synthesis method is not to be applied (step S2: NO), the intra prediction unit 242 performs intra prediction similar to conventional methods. However, if the intra prediction mode selected by the image encoding device 1 is a directional prediction mode, the predictive image synthesis method may always be applied. In this case, there is no need to signal syntax indicating whether or not to apply a predictive image synthesis method.

Thirdly, when the intra prediction mode selected by the image encoding device 1 is directional prediction (step S1: YES) and a predictive image synthesis method is applied (step S2: YES), the weighting factor determination unit 242c determines weighting factors α(x, y) and β(x, y) for each pixel position (x, y) in the intra prediction target block (step S3).

Fourthly, the first predicted image generating unit 242a predicts the intra-prediction target block using the directional prediction mode selected by the image encoding device 1 to generate a directional predicted image (step S4). Also, the second predicted image generating unit 242b predicts the intra-prediction target block using planar prediction to generate a planar predicted image (step S5).

Fifth, the image synthesis unit 242d uses the weighting coefficients α(x, y) and β(x, y) determined in step S3 to weight-synthesize the directional prediction image generated in step S4 and the planar prediction image generated in step S5 for each pixel position (x, y), and outputs the prediction image after weighted synthesis as an intra-prediction image (step S6).

Summary of the embodiment
The image encoding device 1 and the image decoding device 2 according to the present embodiment determine weighting coefficients α(x,y) and β(x,y) based on pixel positions (x,y) in an intra prediction target block, and weight and synthesize a directional prediction image and a planar prediction image for each pixel position (x,y) using the determined weighting coefficients α(x,y) and β(x,y). This makes it possible to improve the prediction accuracy of intra prediction in a prediction image synthesis technique compared to a case where a directional prediction image and a planar prediction image are simply averaged.

<Other embodiments>
In the above-described embodiment, an example in which the second intra prediction mode is planar prediction has been mainly described. However, the second intra prediction mode may be planar prediction as defined in the H.264 standard. When the second intra prediction mode is planar prediction, the "planar prediction" in the above-described embodiment may be read as "planar prediction."

Alternatively, the second intra prediction mode may be DC prediction. When the second intra prediction mode is DC prediction, the "Planar prediction" in the above-described embodiment may be read as "DC prediction."

A program may be provided that causes a computer to execute each process performed by the image encoding device 1. A program may be provided that causes a computer to execute each process performed by the image decoding device 2. The program may be recorded on a computer-readable medium. Using a computer-readable medium, it is possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

The circuits that execute the processes performed by the image encoding device 1 may be integrated, and the image encoding device 1 may be configured as a semiconductor integrated circuit (chip set, SoC). The circuits that execute the processes performed by the image decoding device 2 may be integrated, and the image decoding device 2 may be configured as a semiconductor integrated circuit (chip set, SoC).

The above describes the embodiment in detail with reference to the drawings, but the specific configuration is not limited to the above, and various design changes can be made without departing from the spirit of the invention.

1: Image encoding device 2: Image decoding device 100: Block division unit 110: Subtraction unit 120: Transformation and quantization unit 121: Transformation unit 122: Quantization unit 130: Entropy encoding unit 140: Inverse quantization and inverse transformation unit 141: Inverse quantization unit 142: Inverse transformation unit 150: Synthesis unit 160: Memory 160a: Memory 170: Prediction unit 171: Inter prediction unit 172: Intra prediction unit 172a: First predicted image generation unit 172b: Second predicted image generation unit 172c: Weighting coefficient determination unit 172d: Image synthesis unit 173: Switching unit 200: Entropy decoding unit 210: Inverse quantization and inverse transformation unit 211: Inverse quantization unit 212: Inverse transformation unit 220: Synthesis unit 230: Memory 230a: Memory 240 : Prediction unit 241 : Inter prediction unit 242 : Intra prediction unit 242a : First predicted image generation unit 242b : Second predicted image generation unit 242c : Weighting coefficient determination unit 242d : Image synthesis unit 243 : Switching unit

Claims (7)

An intra prediction device provided in an image decoding device, which performs intra prediction on image blocks obtained by dividing an original image, comprising:
a first predicted image generating unit that predicts the image block in a first prediction mode to generate a first predicted image;
a second predicted image generating unit that predicts the image block in a second prediction mode to generate a second predicted image;
a weighting coefficient determination unit that determines a weighting coefficient used when weighting and synthesizing the first predicted image and the second predicted image for each pixel based on a position of the pixel;
an image synthesis unit that weights and synthesizes the first predicted image and the second predicted image for each pixel by using the weighting coefficient determined for each pixel by the weighting coefficient determination unit,
The intra prediction device is characterized in that the image synthesis unit always performs the weighting synthesis when a specified condition is satisfied, even if syntax indicating that the weighting synthesis is to be performed is not signaled from the encoding side. The intra prediction device according to claim 1 , wherein the predetermined condition is that the first prediction mode is directional intra prediction. The intra prediction device according to claim 1 , wherein the second prediction mode is planar prediction, plane prediction or DC prediction. The weighting coefficient determination unit
4. The intra prediction device according to claim 1, wherein when the first predicted image and the second predicted image are weighted and synthesized for each pixel, the weighting coefficients are determined so that the ratio of the weighting coefficient of the pixel of the second predicted image to the weighting coefficient of the pixel of the first predicted image increases as the distance between the position of the pixel and a decoded reference pixel adjacent to the image block increases. The weighting coefficient determination unit
3. The intra prediction device according to claim 2, wherein the weighting coefficient used when weighting and synthesizing the first predicted image and the second predicted image for each pixel is determined based on the position of the pixel and the prediction direction in the directional intra prediction. An image decoding device comprising an intra prediction device according to any one of claims 1 to 5. A program that causes a computer to function as an intra prediction device according to any one of claims 1 to 5.

Images