JP2021097368A - Image coding method and image decoding method - Google Patents

Abstract

To provide a more suitable image coding technology and an image decoding technology.SOLUTION: An image coding method of coding an image includes a prediction image generation step of generating a prediction image of synthesis prediction by performing a synthesis process for synthesizing a prediction image of the inter-prediction and a prediction image of the intra-prediction for a coded target block, and a coding step of encoding a difference between a pixel value of the prediction image generated in the prediction image generation step and a pixel value of the image of the coded target block, and a plurality of types of intra-prediction can be used for the intra-prediction used in the synthesis process, and the plurality of types of intra-prediction include a matrix-weighted intra-prediction. In the synthesis process, the prediction image of the inter prediction and the prediction image of the intra-prediction are weighted, and a weighting parameter of the weighting process is determined according to the type of the intra-prediction.SELECTED DRAWING: Figure 13

Description

The present invention relates to an image coding technique for encoding an image or an image decoding technique for decoding an image.

As a method of converting image and audio information into digital data, recording and transmitting it, H. 264 / AVC (Advanced Video Coding) and H.A. 265 / HEVC (High Efficiency Video Coding) standards and the like have been formulated so far. ISO / IEC MPEG and ITU-T VCEG are studying a next-generation method called VVC (Versatile Video Coding) that realizes a compression ratio exceeding these (see Non-Patent Document 1).

Xiaozhong Xu and Shan Liu, “Recent advances in video coding beyond the HEVC standard” SIP (2019), vol.8

As one of the technical candidates of VVC, it is considered to use a predicted image obtained by synthesizing an inter-predicted image and an intra-predicted image for the same coded / decoded target block.

However, the reduction of the code amount was not sufficient by simply using the predicted image obtained by averaging the inter-predicted image and the intra-predicted image.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a more suitable image coding technique and image decoding technique.

In order to achieve the above object, in one embodiment of the present invention, for example, a composite process of synthesizing an inter-prediction prediction image and an intra-prediction prediction image is performed on a coded target block to obtain a composite prediction prediction image. The intra is provided with a predicted image generation step to be generated and a coding step for encoding the difference between the predicted image generated in the predicted image generation step and the pixel value of the image of the coded target block. A plurality of types of intra-prediction can be used for the prediction, and the plurality of types of intra-prediction includes a matrix-weighted intra-prediction. The weighting process is performed on the predicted image of the intra prediction, and the weighting parameter of the weighting process may be configured to be determined according to the type of the intra prediction.

According to the present invention, it is possible to provide a more suitable image coding technique and image decoding technique.

It is explanatory drawing of an example of the image coding apparatus which concerns on Example 1 of this invention. It is explanatory drawing of an example of the image decoding apparatus which concerns on Example 2 of this invention. It is explanatory drawing of an example of the image coding method which concerns on Example 1 of this invention. It is explanatory drawing of an example of the image decoding method which concerns on Example 2 of this invention. It is explanatory drawing of an example of the data recording medium which concerns on Example 3 of this invention. It is explanatory drawing of an example of the type of intra prediction which concerns on one Example of this invention. It is explanatory drawing of an example of Planar prediction which concerns on one Example of this invention. It is explanatory drawing of an example of the process of synthetic prediction which concerns on one Example of this invention. It is explanatory drawing of an example of the process of synthetic prediction which concerns on one Example of this invention. It is explanatory drawing of an example of the process of synthetic prediction which concerns on one Example of this invention. It is explanatory drawing of an example of the process of synthetic prediction which concerns on one Example of this invention. It is explanatory drawing of an example of matrix weighting intra prediction which concerns on one Example of this invention. It is explanatory drawing of an example of the process of synthetic prediction which concerns on one Example of this invention.

Hereinafter, examples of the present invention will be described with reference to the drawings.

Further, in each drawing, the components having the same reference numerals have the same functions.

The "0 vc" or "0 vector" in each description and each drawing of the present specification means a vector in which the value of each component is 0, or conversion and setting to such a vector.

In addition, "not referenceable" in each description and each drawing of the present specification means that the block information cannot be acquired because the block position is outside the range of the screen. “Referenceable” means that block information can be acquired, and the block information includes information such as a pixel value, a vector, a reference frame number, and / or a prediction mode.

In addition, the expression "residual component" in each description and each drawing of the present specification also includes the same meaning as "prediction error".

In addition, the expression "area" in each description and each drawing of the present specification also includes the same meaning as "image".

In addition, the expression "transmission with flag" in each description and each drawing of the present specification also includes the meaning of "transmission included in the flag".

(Example 1)
First, Example 1 of the present invention will be described with reference to the drawings.

FIG. 1 shows an example of a block diagram of the image coding apparatus according to the first embodiment of the present invention.

The image coding device includes, for example, an image input unit 101, a block division unit 102, a mode management unit 103, an intra prediction unit 104, an inter prediction unit 105, a synthesis prediction unit 120, a block processing unit 106, a conversion / quantization unit 107, and a reverse. It includes a quantization / inverse conversion unit 108, an image composition / filter unit 109, a decoded image management unit 110, an entropy coding unit 111, and a data output unit 112.

The operation of each component of the image coding apparatus will be described in detail below.

The operation of each component of the image coding device may be, for example, an autonomous operation of each component as described below. Further, for example, it may be realized by cooperating with software stored in the control unit or the storage unit.

First, the image input unit 101 acquires and inputs the original image to be encoded. Next, the block division unit 102 divides the input original image into blocks of a certain size called a CTU (Coding Tree Unit), and further analyzes the input image to make each CTU according to its characteristics. And divide it into more detailed blocks. The block that is the unit of these coding is called a CU (Coding Unit). The division of the CTU into CUs is managed by a tree structure such as a quadtree, a ternary tree, or a binary tree. The inside of the CU may be further divided into subblocks for prediction and TUs (Transform Units) for frequency conversion, quantization, and the like.

The mode management unit 103 manages a mode that determines a coding method for each CU. The coding process is performed using a plurality of intra-prediction methods, inter-prediction methods, and synthetic prediction methods, and the most efficient mode for coding the CU is determined. The most efficient mode is a mode in which the coding error can be minimized for a certain amount of code. There may be a plurality of optimum modes, which may be selected as appropriate according to the situation. To determine which mode is efficient, the intra-prediction unit 104, the inter-prediction unit 105, or the composite prediction unit 120 determines the prediction processing of a plurality of modes, and the residual component and the code amount measurement of various flags using other processing units. , Also, it is performed in combination with the reproduction image error prediction at the time of decoding. Generally, the mode is determined in units of CU, but the CU may be divided into sub-blocks and the mode may be determined for each.

For the prediction method of the coded block (CU or subblock), there are intra (intra-frame) prediction, inter (inter-frame) prediction, and synthetic prediction that combines intra prediction and inter prediction, and these are the intra prediction units. It is performed by 104, an inter-prediction unit 105, and a synthetic prediction unit 120. The intra prediction uses the information of the same frame encoded before the coded block, and the inter prediction uses the information of the frame before or after the coded frame before the coded frame. Use. Here, although only one intra prediction unit 104, an inter prediction unit 105, and a composite prediction unit 120 are described for the sake of explanation, they may be provided for each coding mode and each frame.

The intra prediction unit 104 performs in-screen prediction processing. In the "prediction processing", a prediction image is generated. The in-screen prediction process predicts the pixels of the coded block using the information of the same frame encoded before the coded block. Intra prediction includes DC prediction, angler prediction, planner prediction, matrix prediction, cross-component prediction, multi-line prediction, in-screen block copy, and the like. In the transmission of the intra prediction mode, the most probable mode is estimated from the intra prediction mode of the encoded block.

The inter-screen prediction unit 105 performs inter-screen prediction processing. In the "prediction processing", a prediction image is generated. The inter-screen prediction process predicts the pixels of the coded block by using the information of the frame before or after the coded frame, which is coded before the coded frame. Inter-prediction includes motion compensation prediction, merge mode prediction, affine transformation prediction, triangle block division prediction, optical flow prediction, decoder-side movement prediction, and the like.

The composite prediction unit 120 performs composite prediction to generate a composite prediction image in which a prediction image by intra-prediction and a prediction image by inter-prediction are combined. As the prediction image by intra-prediction used in the composite prediction unit 120, the prediction image of various in-screen prediction processing that can be performed by the intra-prediction unit 104 may be used. As the prediction image by inter-prediction used in the composite prediction unit 120, the prediction image of various in-screen prediction processing that can be performed by the inter-prediction unit 105 may be used. Details of the synthesis method will be described later.

For each coded block, the block processing unit 106 synthesizes a prediction image generated by the intra prediction by the intra prediction unit 104, a prediction image generated by the inter prediction by the inter prediction unit 105, or a composition by the synthesis prediction unit 120. The residual component is calculated and output by taking the difference between the predicted image generated by the prediction and the original image of the coded block obtained from the block dividing unit 102.

The conversion / quantization unit 107 performs frequency conversion and quantization processing on the residual component input from the block processing unit 106, and outputs a coefficient sequence. For frequency conversion, DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or those converted so that they can be processed by integer arithmetic may be used. The coefficient sequence is sent to both the process of restoring the image to create the decoded image used for prediction and the process of outputting the data. Conversion and quantization may be skipped by specifying the mode.

The inverse quantization / inverse transformation unit 108 performs inverse quantization and inverse transformation in order to create a decoded image using the coefficient sequence obtained from the transformation / quantization unit 107 for prediction, and outputs the restored residual component. To do. Inverse quantization and inverse transformation may be performed in the opposite directions corresponding to the quantization and transformation of the transformation / quantization unit, respectively. Inverse quantization and inverse transformation may be skipped by specifying the mode.

The image composition / filter unit 109 includes a prediction image generated by the intra prediction unit 104, a prediction image generated by the inter prediction unit 105, or a prediction image generated by the composition prediction unit 120, and an inverse quantization / inverse conversion unit. The residual component restored by 108 is synthesized, and further processing such as a loop filter is performed to generate a decoded image.

The decoded image management unit 110 holds the decoded image and manages the image referred for the intra prediction, the inter prediction, the composite prediction, the mode information, and the like.

The entropy coding unit 111 performs entropy coding processing on the mode information and the coefficient string information and outputs the bit string. As the entropy coding method, a method such as CABAC (Context Adaptive Binary Arithmetic Code) may be used. A variable length code and a fixed length code may be used in combination. The specified table may be referred to for determining the context.

The data output unit 112 outputs the encoded data to the recording medium or the transmission line.

Next, the flow of the coding method in the image coding apparatus according to the first embodiment of the present invention will be described with reference to FIG.

First, in step 301, the original image to be encoded is input, the content of the image is analyzed, the division method is determined, and the image is divided into blocks. The analysis of the image content may be performed on the entire image, may be performed by combining a plurality of frames, or may be performed in units of blocks such as slices, tiles, bricks, and CTUs in which the image is divided. The block is generally divided into CTUs of a certain size and then divided into CUs by a tree structure.

Next, in step 302, intra-prediction is performed for the coded target block of the original image acquired in step 301. The intra prediction mode is as described above. Prediction is performed for a plurality of modes according to each intra prediction mode.

Next, in step 303, inter-prediction is performed for the coded target block of the original image acquired in step 301. The inter-prediction mode is as described above. Prediction is performed for a plurality of modes according to each inter-prediction mode.

Next, in step 320, synthesis prediction is performed for the coded target block of the original image acquired in step 301. The details of the composite prediction mode will be described later.

Next, in step 304, for each mode, the residual components are separated from the pixels of the coded target block predicted by intra-prediction, inter-prediction, and synthetic prediction, and the residual component conversion processing, quantization processing, and entropy code are performed. The coding process is performed to calculate the coded data.

Next, in step 305, a decoded image is created by performing inverse quantization and inverse transformation processing for each mode and synthesizing the residual component with the predicted image. The decoded image is managed together with the prediction data and various coding data in the intra prediction and the inter prediction, and is used for the prediction of other coding target blocks.

Next, in step 306, each mode is compared to determine the mode that can be most efficiently encoded. The modes include an intra prediction mode, an inter prediction mode, a composite prediction mode, and the like, and these are collectively called a coding mode. The mode selection method is as described above.

In step 307, the coded data of the coded block is output according to the determined coding mode. The coding process of each of the above coding target blocks is repeated for the entire image to encode the image.

Next, the synthetic prediction mode according to this embodiment will be described. The prediction image generation process in the composite prediction mode is the same on the coding side and the decoding side. Therefore, in the following description regarding the synthesis prediction mode according to this embodiment, the same processing is performed on the coding side and the decoding side unless otherwise specified.

The composite prediction mode according to the present embodiment generates a prediction image by synthesis prediction by weighting and averaging the intra prediction image by intra prediction and the inter prediction image by inter prediction. Specifically, as the inter prediction pixel Pinter and the intra prediction pixel Pintra, each prediction pixel Pcom (hereinafter referred to as a composite prediction pixel) of the prediction image is calculated by the following formula 1.

Pcom = (w * Pintra + (4-w) * Pinter) / 4… Equation 1
That is, the process of Equation 1 is a weighted averaging process of the inter-predicted pixel and the intra-predicted pixel using the weighting parameter w. The process may be expressed as weighted addition.

As the inter-prediction image used in the composite prediction mode according to the present embodiment, all the inter-prediction prediction images that can be used in the above-mentioned inter-prediction mode may be used as candidates. Further, among the above-mentioned inter-prediction modes, only the predicted image by the merge mode may be targeted.

As the intra-prediction image used in the synthetic prediction mode according to the present embodiment, all the prediction images by intra-prediction that can be used in the above-mentioned intra-prediction mode may be used as candidates. Further, among the above-mentioned intra prediction modes, only the predicted images by angler prediction, DC prediction, and planner prediction may be targeted.

Here, in the composite prediction mode according to the present embodiment, the weighting parameter w used for calculating the composite prediction pixel is determined according to the type of intra prediction for generating the intra prediction image used in the composite prediction mode. That is, a more suitable prediction image is generated by changing the weight of the intra prediction image in the weighted average of the intra prediction image and the inter prediction image according to each feature of the different intra prediction.

Hereinafter, a specific example of the determination process of the weighting parameter w in the synthetic prediction mode according to this embodiment will be described.

First, angler prediction, DC prediction, and planner prediction, which are intra predictions included in the candidates used for generating the intra prediction image used in the synthetic prediction mode according to the present embodiment, will be described with reference to FIGS. 6 and 7.

FIG. 6 shows the prediction direction of the angler prediction. Angler prediction is a prediction method in which each of the prediction target pixels is predicted by referring to the adjacent pixels pointed out by each prediction direction shown in the figure. The prediction is a one-way prediction, and can be said to be a prediction in which adjacent pixel values are copied in the opposite directions of each prediction direction. Each prediction direction can be identified by a number, and is set as, for example, the prediction directions 2 to 66 shown in the figure. In this figure, typical prediction directions 2 (lower left direction), 18 (left horizontal direction (horizontal direction)), 34 (upper left direction), 50 (upward direction (vertical direction)), 66 (upper right direction) are shown. Shown. In this figure, the notation of other prediction directions is omitted for the sake of simplicity, but the prediction direction is defined in the clockwise rotation direction from the prediction direction 2 to the prediction direction 66 as the prediction direction number increases. There is. Angler prediction is a one-way prediction based on adjacent pixel values.

Therefore, there is a tendency that the predicted residual can be reduced for an image in which the pattern is stretched in a predetermined direction such as a striped pattern. A smaller predicted residual means a higher compression ratio. When angler prediction is selected on the coding side, it is when encoding an image in which the prediction residual is small in angler prediction, but this is likely to be an image in which the pattern extends in a predetermined direction. Then, when the coding side selects the angler prediction for the coding target block, the angler prediction using the same prediction direction as the coding target block or the prediction direction close to the prediction direction of the coding target block even in the adjacent block. Is more likely to be adopted. In an image in which the pattern is stretched in a predetermined direction, as long as the pattern is stretched continuously, the prediction residual may be smaller in the angler prediction in the same prediction direction as the coded block or in the prediction direction close to the predicted direction of the coded block. Is high. This can also be expressed as having a high probability of high prediction accuracy.

In this case, it is highly possible that the angler prediction in the same prediction direction or the angler prediction in the prediction direction close to the prediction direction is repeatedly selected in a plurality of adjacent blocks. That is, it can be said that the angler prediction is an intra prediction with a relatively high approximation of the prediction method with the adjacent block. The "prediction direction close to the prediction direction of the coded block" may be specifically defined as a prediction direction in which the number of the prediction direction is ± 1 of the number of the prediction direction of the coded block. Alternatively, specifically, the number in the prediction direction may be defined as ± 2 of the number in the prediction direction of the coded block.

Further, in FIG. 6, the DC prediction number is further shown as 1. The DC prediction is a prediction method that generates a prediction image corresponding only to the DC component in the frequency domain. That is, all the pixel values in the predicted image of DC prediction are constant values. DC prediction is a prediction method that is not based on adjacent pixel values. However, an image in which the prediction residual tends to be reduced by DC prediction is a flat pattern in which constant brightness continues widely. Then, when DC prediction is selected for the coding target block on the coding side, there is a high possibility that the same DC prediction as the coding target block is adopted even in the adjacent block. This is because as long as a flat pattern having a constant brightness is connected in the peripheral direction, there is a high possibility that the predicted residual will be smaller with the same DC prediction.

This can also be expressed as having a high probability of high prediction accuracy. In this case, it is highly likely that the same DC prediction is repeatedly selected in a plurality of adjacent blocks. That is, although the DC prediction is a prediction method that is not based on the adjacent pixel value, it can be said that the DC prediction is an intra-prediction with a relatively high approximation of the prediction method with the adjacent block.

Further, in FIG. 6, the number of the planner prediction is further shown as 0. A specific prediction method for planner prediction is shown in FIG. In the planner prediction, the reference pixel is generated at the right boundary of the coded block by using the upper right adjacent pixel of the coded block (process 710) (process 711). Next, each predicted pixel is generated by linear interpolation using the right-side reference pixel and the left-side adjacent pixel facing the right-side reference pixel to generate a first predicted image (FIG. 7 (A)) (process). 712).

Further, the reference pixel is generated at the lower boundary of the coded block by using the lower left adjacent pixel of the coded block (process 720) (process 721). Next, each predicted pixel is generated by linear interpolation using the lower reference pixel and the upper adjacent pixel facing the lower reference pixel to generate a second predicted image (FIG. 7B). (Processing 722). Then, the first predicted image (FIG. 7 (A)) and the second predicted image (FIG. 7 (B)) generated in this way are averaged to generate the final predicted image. In the planer prediction, a prediction image of a nonlinear curved surface is generated by averaging the first prediction image (FIG. 7 (A)) and the second prediction image (FIG. 7 (B)) generated by linear interpolation in this way. It is an intra-prediction that is characteristic in that it does.

Planar prediction is a prediction based on adjacent pixel values. However, since the predicted image generated by the planner prediction has a characteristic non-linear curved surface, the approximation of the prediction method with the adjacent block is not high at all. As described above, when the prediction residual becomes small in the angler prediction or the DC prediction, there is a high possibility that the same prediction method as that of the adjacent block is repeated. On the other hand, there are not many images of a pattern in which the characteristic nonlinear curved surface of the predicted image of the planner prediction needs to be repeated.

Therefore, it can be said that the planner prediction is an intra prediction with a relatively low approximation of the prediction method with the adjacent block. Further, when the planner prediction is selected for the coded target block on the coding side, there is a high possibility that the prediction residual is not sufficiently small in the angler prediction and the DC prediction. An image whose prediction residual is not sufficiently small by angler prediction or DC prediction is likely not a simple pattern, so the prediction residual by planner prediction for an image whose prediction residual is not small by angler prediction or DC prediction. Is more likely to be larger than the predicted residual when the predicted residual becomes smaller in the angler prediction or the DC prediction. This can also be expressed as a high possibility that the prediction accuracy is low.

As described with reference to FIGS. 6 and 7, the angler prediction, DC prediction, and planner prediction that can be used in the synthetic prediction mode according to the present embodiment are intra predictions having different characteristics. Therefore, in the synthetic prediction mode according to the present embodiment, attention is paid to the characteristics of each type of intra-prediction, and control is performed to determine the value of the weighting parameter w of Equation 1 according to the type of intra-prediction. The w determination process in the synthetic prediction mode according to this embodiment will be described with reference to FIG.

FIG. 8 shows an example of w determination in the synthetic prediction mode according to this embodiment. FIG. 8 shows three examples of w determination example 1, w determination example 2, and w determination example 3. These are examples of different determination methods. In each determination method, the value of w is determined from one of 1, 2, and 3 for each condition. In Equation 1, the larger the value of w, the greater the weighting of the intra prediction in the synthetic prediction mode. On the contrary, in Equation 1, the larger and smaller the value of w, the smaller the weight of the intra prediction in the synthetic prediction mode. Examples of these determinations will be described below.

In the w determination example 1, when the intra prediction used in the synthetic prediction mode is the planner prediction, w = 1 is determined. When the intra prediction used in the composite prediction mode is an angler prediction and the prediction direction is the horizontal direction (18) or the vertical direction (50), w = 3 is determined. When the intra prediction used in the synthetic prediction mode is an angler prediction and the prediction direction is 2 to 17 or 51 to 66, w = 3 is determined. When the intra prediction used in the synthetic prediction mode is an angler prediction and the prediction direction is 19 to 49, w = 3 is determined. When the intra prediction used in the synthetic prediction mode is DC prediction, it is determined that w = 3.

That is, in w determination example 1, the planner prediction, which is an intra-prediction with a relatively low approximation of the prediction method with the adjacent block, is a determination method in which a smaller w value can be taken as compared with the angler prediction and the DC prediction. is there. Since the predicted image of the planner prediction is a characteristic non-linear curved surface and the prediction accuracy is likely to be low, it can be expressed that it is preferable to allow the smallest weighting in the synthetic prediction mode.

On the other hand, in w determination example 1, the angler prediction, which is an intra-prediction with a relatively high approximation of the prediction method with the adjacent block, has a w value larger than the w value in the planner prediction in any prediction direction. Select a certain 3. The angler prediction, which is an intra-prediction with a relatively high approximation of the prediction method with the adjacent block, is a determination method that allows a larger value of w than the planner prediction. Since the prediction image of the angler prediction is a simple one-way prediction surface and the prediction accuracy is likely to be high, it can be expressed that it is preferable to be able to take a relatively large weight in the composite prediction mode. ..

Further, in the w determination example 1, the DC prediction, which is an intra-prediction with a relatively high approximation of the prediction method with the adjacent block, selects 3 which is a w value larger than the w value in the planner prediction. The DC prediction, which is an intra-prediction with a relatively high approximation of the prediction method with the adjacent block, is a determination method that can take a larger value of w than the planner prediction. Since the prediction image of DC prediction is a simple constant value plane and the prediction accuracy is likely to be high, it can be expressed that it is preferable to be able to take a relatively large weight in the composite prediction mode.

In this way, the w determination example 1 determines the value of w according to the type of intra prediction. At this time, the planner prediction is set so as to be the smallest value. Further, in the angler prediction, the value of w is set to be constant in any prediction direction. That is, in w determination example 1, the value of w does not change according to the prediction direction of the angler prediction.

Next, in w determination example 2, when the intra prediction used in the synthetic prediction mode is the planner prediction, w = 1 is determined. When the intra prediction used in the composite prediction mode is an angler prediction and the prediction direction is the horizontal direction (18) or the vertical direction (50), w = 2 is determined. When the intra prediction used in the synthetic prediction mode is an angler prediction and the prediction direction is 2 to 17 or 51 to 66, w = 3 is determined. When the intra prediction used in the synthetic prediction mode is an angler prediction and the prediction direction is 19 to 49, w = 3 is determined. When the intra prediction used in the synthetic prediction mode is DC prediction, it is determined that w = 3.

That is, in w determination example 2, as in w determination example 1, the planner prediction, which is an intra prediction with a relatively low approximation of the prediction method with the adjacent block, has a smaller w value than the angler prediction and the DC prediction. This is a decision method that can be taken. The reason is the same as that of w determination example 1. In the w determination example 2, the angler prediction, which is an intra prediction with a relatively high approximation of the prediction method to the adjacent block, is a w value larger than the w value in the planner prediction in any prediction direction, 2 or 3. Select. The angler prediction, which is an intra-prediction with a relatively high approximation of the prediction method with the adjacent block, is a determination method that allows a larger value of w than the planner prediction.

Since the prediction image of the angler prediction is a simple one-way prediction surface and the prediction accuracy is likely to be high, it can be expressed that it is preferable to be able to take a relatively large weight in the composite prediction mode. .. However, in the w determination example 2, unlike the w determination example 1, the value of w is changed according to the prediction direction even if the angler prediction is the same.

Specifically, when the prediction direction is the horizontal direction (18) or the vertical direction (50), w = 2 is set, and the weight is set to be lower than w = 3 in the other prediction directions in the angler prediction. This is because the prediction direction in the horizontal direction (18) is that the reference pixel is only the pixel of the left adjacent block, and the approximation of the prediction method with the adjacent block is likely to be limited to the left adjacent block, and the overall adjacency is high. This is because the closeness of the prediction method to the block is considered to be lower than that of other prediction directions.

Similarly, in the vertical direction (50), the reference pixel is only the pixel of the upper adjacent block, and the approximation of the prediction method with the adjacent block is likely to be limited to the upper adjacent block. This is because the closeness of the prediction method with the comprehensive adjacent block is considered to be lower than that of other prediction directions. Further, in the w determination example 2, as in the w determination example 1, the DC prediction, which is an intra-prediction with a relatively high approximation of the prediction method with the adjacent block, has a w value larger than the w value in the planner prediction. Select a certain 3. The reason is the same as that of w determination example 1.

In this way, the w determination example 2 determines the value of w according to the type of intra prediction. When the intra prediction is an angler prediction, the value of w is determined in consideration of the prediction direction. In addition, the planner prediction is set so that it can be the smallest value.

The w determination example 3 differs from the w determination example 2 in that w = 2 is determined when the intra prediction used in the synthetic prediction mode is the angler prediction and the prediction directions are 2 to 17 or 51 to 66. The points are common. In the case of angler prediction in the prediction direction 2 to 17 or 51 to 66, the angle of the prediction direction is not a perfect horizontal direction or a perfect vertical direction, so the prediction method is approximate to both the upper adjacent block and the left adjacent block. Sex is recognized.

However, the pixel included in the reference pixel is either the pixel of the upper adjacent block or the pixel of the left adjacent block. In the case of the prediction directions 2 to 17 or 51 to 66, the prediction is a comprehensive adjacent block as compared with the case of the prediction directions 19 to 49 in which both the pixels of the upper adjacent block and the pixels of the left adjacent block are included in the reference pixels. The method may be less close. Therefore, in the w determination example 3, w = 2 is determined in the case of the prediction directions 2 to 17 or 51 to 66, and the weighting is smaller than that in the case of the prediction directions 19 to 49.

According to the determination example of w in the composite prediction mode of FIG. 8 described above, the weight of the intra prediction image in the synthesis of the intra prediction image and the inter prediction image is weighted according to the characteristics of each type of intra prediction used in the composite prediction mode. Can be determined more preferably.

As described above, the coding process according to the embodiment of the present application is performed.

According to the image coding apparatus and the image coding method according to the first embodiment described above, a more suitable synthesis prediction mode can be realized.

Further, the image coding device and the image coding method according to the first embodiment can be applied to a recording device, a mobile phone, a digital camera, or the like using them.

According to the image coding apparatus and the image coding method according to the first embodiment of the present invention described above, the amount of coded data is reduced and the image quality of the decoded image is deteriorated when the coded data is decoded. It becomes possible to prevent. That is, a high compression rate and a better image quality can be realized.

That is, according to the image coding apparatus and the image coding method according to the first embodiment of the present invention, a more suitable image coding technique can be provided.

(Example 2)
Next, FIG. 2 shows an example of a block diagram of the image decoding apparatus according to the second embodiment of the present invention.

The image decoding device includes, for example, a stream analysis unit 201, a block management unit 202, a mode determination unit 203, an intra prediction unit 204, an inter prediction unit 205, a synthesis prediction unit 220, a coefficient analysis unit 206, an inverse quantization / inverse conversion unit 207, and the like. It includes an image composition / filter unit 208, a decoded image management unit 209, and an image output unit 210.

The operation of each component of the image decoding device will be described in detail below.

The operation of each component of the image decoding device may be, for example, an autonomous operation of each component as described below. Further, for example, it may be realized by cooperating with software stored in the control unit or the storage unit.

First, the stream analysis unit 201 analyzes the input coded stream. Here, the stream analysis unit 201 also performs data extraction processing from packets and information acquisition processing of various headers and flags.

Further, at this time, the coded stream input to the stream analysis unit 201 is, for example, a coded stream generated by the image coding device and the image coding method according to the first embodiment. Since the generation method is as shown in the first embodiment, the description thereof will be omitted. It may be a coded stream read from the data recording medium shown in Example 3. The recording method will be described later.

Next, the block management unit 202 manages the block processing according to the block division information analyzed by the stream analysis unit 201. Generally, the coded image is divided into blocks, and each coded block is managed by a tree structure or the like. The block processing order is often the raster scan order, but it may be processed in an arbitrary predetermined order such as a zigzag scan. The block division method is as described in the first embodiment.

Next, the mode determination unit 203 determines the coding mode specified by the flag or the like for each coding target block. The following decoding process is performed according to the coding mode of the determination result. The processing for each coding mode will be described below.

First, when the coding mode is intra coding, the intra prediction unit 204 generates a prediction image by intra prediction. The intra prediction mode is as described in the first embodiment. In principle, the process of generating a predicted image by prediction is the same on the coding side and the decoding side.

When the coding mode is coding by inter-prediction, the inter-prediction unit 205 generates a prediction image by inter-prediction. The inter-prediction mode is as described in the first embodiment. In principle, the process of generating a predicted image by prediction is the same on the coding side and the decoding side.

When the coding mode is coding by synthetic prediction, the synthetic prediction unit 220 generates a predicted image by synthetic prediction. The synthetic prediction mode is as described in Example 1. In principle, the process of generating a predicted image by prediction is the same on the coding side and the decoding side.

On the other hand, the coefficient analysis unit 206 analyzes the coded data of each coded block included in the input coded stream, decodes the entropy coded data, and encodes the coded data including the coefficient sequence of the residual component. Is output. At this time, the process corresponding to the coding mode of the determination result of the mode determination unit 203 is performed.

The inverse quantization / inverse transformation unit 207 performs an inverse quantization process and an inverse transformation on the coded data including the coefficient sequence of the residual component, and restores the residual component. The methods of inverse quantization and inverse transformation are as described above. Inverse quantization and inverse transformation may be skipped by specifying the mode.

The residual component restored as described above is combined with the prediction image output from the intra prediction unit 204, the inter prediction unit 205, or the composition prediction unit 220 by the image composition / filter unit 208, and further, a loop filter or the like is used. It is processed and output as a decoded image.

The decoded image management unit 209 holds the decoded image and manages the image, the mode information, and the like referred to for the intra prediction, the inter prediction, or the composite prediction.

The last decoded image is output by the image output unit 210, and the image is decoded.

Next, the flow of the image decoding method in the image decoding apparatus according to the second embodiment of the present invention will be described with reference to FIG.

First, in step 401, the coded stream to be decoded is acquired and the data is analyzed. It also manages block processing according to the analyzed block division information. The block division method is as described in the first embodiment.

Next, in step 402, the coding mode for one coding unit (block unit, pixel unit, etc.) included in the coded data is determined by using the coding mode information analyzed in step 401. Here, in the case of the intra-coding mode, the process proceeds to step 403, in the case of the inter-coding mode, the process proceeds to step 404, and in the case of the synthetic coding mode, the process proceeds to step 420.

In step 403, the predicted image is generated by intra-prediction according to the method specified by the coding mode. The intra prediction mode is as described in the first embodiment.

In step 404, the predicted image is generated by inter-prediction according to the method specified by the coding mode. The inter-prediction mode is as described in the first embodiment.

In step 420, a predicted image is generated by synthetic prediction according to the method specified by the coding mode. The synthetic prediction mode is as described in Example 1.

In step 405, the coded data of each coded block is analyzed according to the method specified by the coding mode, the entropy-coded data is decoded, and the coded data including the coefficient sequence of the residual component is output. To do. Further, the coded data including the coefficient sequence of the residual component is subjected to the inverse quantization processing and the inverse transformation to restore the residual component. The methods of inverse quantization and inverse transformation are as described above. Inverse quantization and inverse transformation may be skipped by specifying the mode.

In step 406, the predicted image generated by intra-prediction, inter-prediction, composite prediction, or the like is combined with the restored residual component for each coded block, and further processing such as a loop filter is performed. Create a decoded image by. A decoded image is created by performing the decoding process performed for each coded block for the entire image.

In step 407, the generated decoded image is output and displayed.

Since the processing in the synthesis prediction mode on the decoding side according to this embodiment is substantially the same as the processing in the synthesis prediction mode described with reference to Equation 1, FIG. 6, FIG. 7, and FIG. 8 in Example 1. , The repeated description is omitted. The expression "encoded block" in the description of the synthetic prediction mode of the first embodiment may be read as "decoding target block" in the processing of the synthetic prediction mode on the decoding side.

In the synthesis prediction mode on the decoding side according to the present embodiment, as described above, the weighted averaging processing of the inter prediction pixel and the intra prediction pixel using the equation 1 is performed. Further, in the synthesis prediction mode on the decoding side according to the present embodiment, as described above, the weighting parameter w is determined by the determination process as shown in FIG. This makes it possible to more preferably determine the weighting of the intra-prediction image in the composition of the intra-prediction image and the inter-prediction image according to the characteristics of each type of intra-prediction used in the composite prediction mode.

In this embodiment as well, in addition to the examples, a coded stream in which each coded mode is subdivided and defined with the size of the block used in the coded mode as a parameter may be used as the decoding target stream.

As described above, the decoding process according to the embodiment of the present application is performed.

Further, the image decoding device and the image decoding method according to the second embodiment can be applied to a reproduction device, a mobile phone, a digital camera, or the like using them.

According to the image decoding apparatus and the image decoding method according to the second embodiment of the present invention described above, it is possible to decode coded data having a small amount of code with higher image quality.

That is, according to the image decoding apparatus and the image decoding method according to the second embodiment of the present invention, a more suitable image decoding technique can be provided.

(Example 3)
Next, FIG. 5 shows an example of the data recording medium according to the third embodiment of the present invention.

The coded stream according to the present embodiment of the present invention is a coded stream generated by the image coding apparatus or the image coding method according to the first embodiment according to the first embodiment. Since the generation method is as shown in the first embodiment, the description thereof will be omitted.

Here, the coded stream according to this embodiment is recorded as a data string 502 on the data recording medium 501, for example. The data string 502 is recorded, for example, as a coded stream according to a predetermined grammar.

First, the coded stream is taken out as a bit string divided into units of a certain size called a NAL (Network Abstraction Layer) unit 503. The bit string of the NAL unit is read according to a certain rule such as a variable length code and converted as an RBSP (Raw Byte Sequence Payload). The RBSP data is composed of information such as a sequence parameter set 504, a picture parameter set 505, a decoding parameter set, a video parameter set, and slice data 506.

Inside each slice, for example, information 507 about each block is contained. Inside the information about the block, for example, there is an area for recording each coding mode for each block, and this is set as a coding mode flag 508.

According to the data recording medium according to the third embodiment of the present invention described above, the code amount can be reduced and the deterioration of the image quality can be prevented. That is, it is possible to realize a data recording medium that has a high compression rate and records a coded stream having better image quality.

The coded stream generated by the image coding apparatus or the image coding method in each of the examples described below can also be recorded on the data recording medium according to the present embodiment by the configuration described above in the present embodiment. ..

(Example 4)
Next, Example 4 of the present invention will be described with reference to the drawings.

In the fourth embodiment of the present invention, in the image coding apparatus and the image coding method according to the first embodiment, the processing for determining the weighting parameter w in the processing of the synthesis prediction is changed.

Specifically, the determination process of the weighting parameter w is changed from the determination process of the weighting parameter w shown in FIG. 8 to the determination process of the weighting parameter w shown in FIG. In the image coding apparatus and the image coding method according to the fourth embodiment, the image coding apparatus and the image according to the first embodiment all have other configurations, operations, and processes other than the determination processing of the weighting parameter w shown in FIG. It is the same as the coding method. Therefore, the same points as the image coding apparatus and the image coding method according to the first embodiment will not be repeated.

FIG. 9 shows an example of a weighting parameter w determination process in the synthetic prediction process in the image coding apparatus and the image coding method according to the fourth embodiment of the present invention. Since the same weighting parameter w is determined on the decoding side, the prediction target block will be described by the expression “encoding (decoding) target block”.

In the example of FIG. 9, unlike the example of FIG. 8, when performing the process of determining the weighting parameter w, not only the intra prediction mode of the coded (decoded) target block but also the combination of the prediction modes of a plurality of adjacent blocks is used. Consider. Here, in this embodiment, as an example of a plurality of adjacent blocks, two adjacent blocks, a block immediately above the coding (decoding) target block and a block immediately adjacent to the left side of the coding (decoding) target block, are used. Consider the combination of prediction modes.

That is, in the example of FIG. 8, the weighting parameter w is determined only according to the intra prediction mode of the coded (decoded) target block, but in the example of FIG. 9, the intra of the coded (decoded) target block is determined. Even if the prediction modes are the same, the weighting parameter w is changed if the combination of prediction modes of a plurality of adjacent blocks is different.

In the first embodiment, it has already been described that the approximation of the prediction method with the adjacent block differs depending on the type of the intra prediction mode of the coded (decoded) target block. Specifically, in the case of planner prediction and angler prediction in the prediction direction 18 or 50, in the case of angler prediction in the prediction direction 2 to 17 or 51 to 66, and in the case of angler prediction in the prediction direction 2 to 17 or 51 to 66, the angler When the prediction direction is 19 to 49 in the prediction, the characteristics of the closeness of the prediction method with the adjacent block are explained for each of the DC predictions.

In FIG. 9, the weighting parameter w is determined according to the combination of the prediction modes of the plurality of adjacent blocks for each feature of these intra prediction mode types.

Specific conditions for combining the prediction modes of a plurality of adjacent blocks are (1) a block immediately above the coding (decoding) target block and a block immediately adjacent to the coding (decoding) target block. If neither of the adjacent blocks is available, (2) one of the two adjacent blocks, the block immediately above the coded (decoded) target block and the block immediately to the left of the coded (decoded) target block, If it is not available and the other is an inter-prediction, (3) both the two adjacent blocks, the block immediately above the coded (decoded) target block and the block immediately to the left of the coded (decoded) target block, are inter. In the case of prediction, (4) one of the two adjacent blocks, the block directly above the coded (decoded) target block and the block directly to the left of the coded (decoded) target block, is unavailable and the other is intra-predicted. In the case of (5), one of the two adjacent blocks, the block immediately above the coded (decoded) target block and the block immediately to the left of the coded (decoded) target block, is the inter-prediction and the other is the intra-prediction. In the case of (6), both the block immediately above the coded (decoded) target block and the two adjacent blocks immediately to the left of the coded (decoded) target block are the two adjacent blocks in the case of intra-prediction (1) to Consider the condition (6).

Specifically, it is determined as follows for each of the above conditions (1) to (6).

First, when the intra prediction mode of the coded (decoded) target block is planner prediction, w may be determined as shown in FIG. Specifically, w = 1 under the conditions (1) to (3), and w = 2 under the conditions (4) to (6). As described in the first embodiment, the planner prediction is an intra prediction with a relatively low approximation of the prediction method with the adjacent block.

However, if there is an intra-prediction block that can be used in the adjacent block, there is a possibility of planer prediction similar to that of the coded (decoded) target block. Improves. Therefore, it is preferable that the weighting of the conditions (4) to (6) is larger than that of the conditions (1) to (3). As a modification, w = 1 may be set in the conditions (1) to (5), and w = 2 may be set only in the condition (6). Since the planner prediction uses both the upper adjacent pixel and the left adjacent pixel of the target block in the processing, the idea that the approximation of the prediction method is improved only when both of the prediction modes of a plurality of adjacent blocks are intra predictions. This is a modified example used.

Next, when the intra prediction mode of the coded (decoded) target block is angler prediction and the prediction direction is 18 or 50, w may be determined as shown in FIG. Specifically, w = 1 under the conditions (1) to (3), and w = 2 under the conditions (4) to (6). When the intra prediction mode is angler prediction and the prediction direction is 18 or 50, the intra prediction has an approximation of the prediction method to the adjacent block in either the horizontal direction or the vertical direction.

Therefore, it is preferable that the conditions (4) to (6) in which the available intra prediction block exists are weighted more than the conditions (1) to (3). However, the effect of approximation between adjacent blocks and the prediction method is limited to either the horizontal or vertical direction. Therefore, even if the condition (6) is an intra prediction that both of the plurality of adjacent blocks can be used, the condition (4) (5) that only one of the plurality of adjacent blocks can be used is the intra prediction. Even if compared, the effect of the closeness between the adjacent block and the prediction method does not change. Therefore, in the example of this figure, w = 2 was set even in the condition (6).

Next, when the intra prediction mode of the coded (decoded) target block is the prediction direction 2 to 17 or 51 to 66 in the angler prediction and the prediction direction 2 to 17 or 51 to 66 in the angler prediction, it is shown in FIG. W may be determined as follows. Specifically, w = 1 under the conditions (1) to (3), w = 2 under the conditions (4) and (5), and w = 3 under the conditions (6). When the intra prediction mode is the prediction direction 2 to 17 or 51 to 66 in the angler prediction and the prediction direction 2 to 17 or 51 to 66 in the angler prediction, the prediction direction is an oblique direction, so that the block adjacent to the upper side is used. It is an intra-prediction that has an approximation of the prediction method to both blocks adjacent to the left side. Therefore, as the number of intra-prediction blocks available for the adjacent block increases, the approximation between the adjacent block and the prediction method tends to increase. Therefore, as described above, it is preferable to set the value of the weighting parameter w larger as the number of intra-prediction blocks available for adjacent blocks increases.

Next, when the intra prediction mode of the coded (decoded) target block is DC prediction, w may be determined as shown in FIG. Specifically, w = 2 under the conditions (1) to (3), and w = 3 under the conditions (4) to (6). As described in the first embodiment, when DC prediction is selected in the processing on the coding side, it is highly possible that the image to be coded is a flat pattern in which constant brightness continues widely. This is especially likely when the adjacent block contains an available intra-prediction block, so w = 3 is preferred. However, in the example of this figure, if the adjacent block does not include the available intra-prediction block, the possibility that the image to be encoded is a flat pattern with a wide constant brightness is slightly reduced, so w = It is set to 2.

According to the determination process of the weighting parameter w of FIG. 9 described above, in the determination of the weighting parameter w of the equation 1 in the synthetic prediction process, not only the intra prediction mode of the coded (decoding) target block but also a plurality of devices are determined. The weighting parameter w can be determined more preferably by considering the combination of the prediction modes of the adjacent blocks.

Further, by setting a combination of prediction modes of a plurality of adjacent blocks as a condition, it is possible to set a suitable condition without transmitting a new flag.

In the example of FIG. 9, the weighting w is made larger in (6) than in the condition (1) regardless of which prediction mode the coding (decoding) target block is. Regardless of the prediction mode of the coded (decoding) target block, if the number of intra prediction blocks that can be used for the adjacent block increases, the approximation between the adjacent block and the prediction method tends to increase. is there.

Further, in the example of FIG. 9, when neither the planner prediction nor the DC prediction is specified in the intra prediction used in the composite prediction, the multiple adjacent blocks are set in a plurality of adjacent blocks as compared with the case where there is no intra prediction block available in the plurality of adjacent blocks. If there are intra-prediction blocks available, increase the weighting w.

As a modification of FIG. 9, as shown in FIG. 10, the DC prediction conditions (4) and (5) may be w = 2. In the example of FIG. 10, only the condition (6) is that w = 3 in the DC prediction. Only in the condition (6), which is an intra-prediction in which both of the plurality of adjacent blocks can be used, it is an example in which it is highly likely that the image to be encoded is a flat pattern in which a constant brightness continues widely. Since the example of FIG. 10 is the same as that of FIG. 9 except for the conditions (4) and (5) for DC prediction, the repeated description will be omitted.

According to the image coding apparatus and the image coding method according to the fourth embodiment of the present invention described above, the image can be more preferably encoded.

(Example 5)
Next, Example 5 of the present invention will be described.

In the fifth embodiment of the present invention, in the image decoding apparatus and the image decoding method according to the second embodiment, the processing for determining the weighting parameter w in the processing of the synthesis prediction is changed.

Specifically, the determination process of the weighting parameter w is changed from the determination process of the weighting parameter w shown in FIG. 8 to the determination process of the weighting parameter w shown in FIG. 9 or 10. The image decoding apparatus and the image decoding method according to the fifth embodiment use the determination process of the weighting parameter w shown in FIG. 9 or 10, but other than the determination process of the weighting parameter w shown in FIG. 9 or 10. The other configurations, operations, and processes of the above are all the same as those of the image decoding apparatus and the image decoding method according to the second embodiment. Therefore, the same points as the image decoding apparatus and the image decoding method according to the second embodiment will not be repeated.

Further, since the details of the determination process of the weighting parameter w of FIG. 9 or FIG. 10 are as described in the fourth embodiment, the repeated description will be omitted.

In the image decoding apparatus and image decoding method according to the fifth embodiment of the present invention described above, coding (decoding) is performed in determining the weighting parameter w of the mathematical formula 1 in the synthetic prediction processing as shown in FIG. 9 or FIG. ) The weighting parameter w can be more preferably determined by considering not only the intra prediction mode of the target block but also the combination of the prediction modes of a plurality of adjacent blocks.

According to the image decoding apparatus and the image decoding method according to the fifth embodiment of the present invention described above, the image can be more preferably decoded.

(Example 6)
Next, Example 6 of the present invention will be described with reference to the drawings.

In the sixth embodiment of the present invention, in the image coding apparatus and the image coding method according to the first or fourth embodiment, a flag for designating the prediction mode of the prediction image to be combined with the intra prediction image in the processing of the synthesis prediction is newly added. Is set to, and the setting control of the flag and the determination control of the prediction mode using the flag are performed.

In the image coding apparatus and the image coding method according to the first or fourth embodiment, the merge mode has been described as an example of the prediction mode of the prediction image to be combined with the intra prediction image in the processing of the synthesis prediction. In addition, it was explained that inter-prediction in a prediction mode other than the merge mode may be used as an example of the prediction mode of the prediction image to be combined with the intra-prediction image in the processing of the synthesis prediction.

On the other hand, the sixth embodiment of the present invention proposes a more specific process of the method for determining the prediction mode.

Specifically, the inter-prediction designation flag of the composite prediction mode shown in FIG. 11 is set, and the flag is set on the coding side. Further, on the decoding side, the flag is used to control the determination of the prediction mode of the prediction image to be combined with the intra prediction image in the synthesis prediction processing.

In the image coding apparatus and the image coding method according to the sixth embodiment, all of the other configurations, operations, and processes other than the processing related to the inter-prediction designation flag of the composite prediction mode shown in FIG. 11 are the first embodiment or the embodiment. It is the same as the image coding apparatus and the image coding method according to 4. Therefore, the same points as the image coding apparatus and the image coding method according to the first or fourth embodiment will not be repeated.

FIG. 11 shows the prediction mode of the prediction image to be combined with the intra prediction image in the setting control of the inter-prediction designation flag of the synthesis prediction mode and the processing of the synthesis prediction in the image coding apparatus and the image coding method according to the sixth embodiment of the present invention. An example of decision control is shown.

In the example of FIG. 11, 1-bit information is set as the inter-prediction designation flag of the composite prediction mode. On the coding side, (1) 1 bit of the inter-prediction specification flag in the composite prediction mode is not transmitted, (2) the 1 bit is set to 0 for transmission, and (3) the 1 bit is set to 1. It is possible to execute three types of processing, that is, transmission.

On the other hand, on the decoding side, (1) when 1 bit of the inter-prediction designation flag of the composite prediction mode is not transmitted on the coding side, the prediction mode of the prediction image to be combined with the intra prediction image in the synthesis prediction processing. Is judged to be in merge mode. In this way, it is possible to specify the merge mode without adding 1-bit information. Since the merge mode itself is a prediction mode in which the prediction method is very close to the adjacent block, the merge mode can be a suitable option in many cases.

On the other hand, (2) when 1 bit of the inter-prediction specification flag of the composite prediction mode is set to 0 on the coding side and transmission is performed, the decoding side of the prediction image to be combined with the intra prediction image in the synthesis prediction processing. Regarding the prediction mode, if the immediately left adjacent block of the decoding target block is available and the inter prediction is performed, the same inter prediction mode as the immediately left adjacent block is selected. If the block immediately to the left of the block to be decoded does not satisfy this condition, the merge mode is selected as the prediction mode of the prediction image to be combined with the intra prediction image in the synthesis prediction process.

In this way, it becomes possible to select the prediction mode of the block immediately to the left of the decoding target block as the prediction mode other than the merge mode as the prediction mode of the prediction image to be combined with the intra prediction image in the synthesis prediction processing. .. It is preferable to use the prediction mode of the block immediately to the left of the block to be decoded when the coding efficiency is higher than that of the merge mode.

Further, (3) when 1 bit of the inter-prediction designation flag of the composite prediction mode is set to 1 on the coding side and transmission is performed, the decoding side is in the prediction mode of the prediction image to be combined with the intra prediction image in the synthesis prediction processing. For, if the block immediately above the block to be decoded is available and inter-prediction, the same inter-prediction mode as the immediately above adjacent block is selected.

If the block immediately above the block to be decoded does not satisfy this condition, the merge mode is selected as the prediction mode of the prediction image to be combined with the intra prediction image in the synthesis prediction process. In this way, it becomes possible to select the prediction mode of the block immediately above the decoding target block as the prediction mode other than the merge mode as the prediction mode of the prediction image to be combined with the intra prediction image in the synthesis prediction processing. It is preferable to use the prediction mode of the block immediately above the block to be decoded when the coding efficiency is higher than that of the merge mode.

According to the setting control of the inter-prediction designation flag of the composite prediction mode and the determination control of the prediction mode of the prediction image to be combined with the intra-prediction image in the processing of the synthesis prediction described above, the intra-prediction image in the processing of the synthesis prediction The merge mode and other prediction modes can be more preferably set and determined as the prediction mode of the prediction image to be combined with. In particular, it is preferable to introduce a prediction mode other than the merge mode as a prediction mode of the prediction image to be combined with the intra prediction image in the processing of the synthesis prediction only by setting the 1-bit flag. In addition, the merge mode, which is very close to the prediction method with the adjacent block, can be set and determined without the transmission of a special (or dedicated) flag that specifies the inter-prediction in the composite prediction mode, and the transmission information. It is suitable for suppressing the increase of.

According to the image coding apparatus and the image coding method according to the sixth embodiment of the present invention described above, the image can be more preferably encoded.

(Example 7)
Next, Example 7 of the present invention will be described.

In the seventh embodiment of the present invention, in the image decoding apparatus and the image decoding method according to the second or fifth embodiment, a flag for designating the prediction mode of the predicted image to be combined with the intra prediction image in the synthetic prediction process is newly set. Then, the setting control of the flag and the determination control of the prediction mode using the flag are performed.

Specifically, the prediction mode of the predicted image to be combined with the intra prediction image in the synthetic prediction process is determined by the method shown in FIG. In the composite prediction process shown in FIG. 11, any other configuration, operation, and process other than the prediction mode determination process using the flag that specifies the prediction mode of the intra-prediction image and the predicted image to be combined is Example 2 or Example. It is the same as the image decoding apparatus and the image decoding method according to 5. Therefore, the same points as the image decoding apparatus and the image decoding method according to the second or fifth embodiment will not be repeated.

Further, the details of the prediction mode determination process using the flag for specifying the prediction mode of the intra prediction image and the prediction image to be combined in the synthesis prediction process shown in FIG. 11 are as described in the sixth embodiment. The repetitive description will be omitted.

In the image decoding apparatus and image decoding method according to the seventh embodiment of the present invention described above, the merge mode and other prediction modes are more preferably determined as the prediction modes of the prediction image to be combined with the intra prediction image in the synthesis prediction processing. can do. In particular, it is preferable to introduce a prediction mode other than the merge mode as a prediction mode of the prediction image to be combined with the intra prediction image in the processing of the synthesis prediction only by setting the 1-bit flag. In addition, the merge mode, which is very close to the prediction method with the adjacent block, can be determined without the transmission of a special (or dedicated) flag that specifies the inter-prediction in the composite prediction mode, and the coding side. It is suitable for suppressing an increase in transmission information from.

According to the image decoding apparatus and the image decoding method according to the seventh embodiment of the present invention described above, the image can be more preferably decoded.

(Example 8)
Next, Example 8 of the present invention will be described with reference to the drawings.

In the eighth embodiment of the present invention, in the image coding apparatus and the image coding method according to the first embodiment, as the intra-prediction image used for the processing of the composite prediction, the prediction image by the matrix weighted intra-prediction is also supported. .. The matrix weighted intra-prediction in this embodiment is an example of the prediction process described as “matrix prediction” in the first embodiment.

Specifically, the matrix-weighted intra-prediction described with reference to FIG. 12 is used as the intra-prediction image for the synthetic prediction process. Further, the determination process of the weighting parameter w is changed from the determination process of the weighting parameter w shown in FIG. 8 to the determination process of the weighting parameter w shown in FIG. The image coding apparatus and the image coding method according to the eighth embodiment have all other configurations, operations, and processes other than the matrix weighted intra prediction described with reference to FIG. 12 and the determination process of the weighting parameter w shown in FIG. It is the same as the image coding apparatus and the image coding method according to the first embodiment. Therefore, the same points as the image coding apparatus and the image coding method according to the first embodiment will not be repeated.

First, the matrix weighted intra-prediction will be described with reference to FIG. Since the same matrix-weighted intra-prediction is performed on the decoding side, the prediction target block will be described by the expression “coding (decoding) target block”.
Matrix weighted intra-prediction predicts pixels by vector calculation using the boundary pixels of adjacent blocks of the block to be coded (decoded) and the matrix specified in the mode. The matrix weighted intra-prediction process generates a predicted image through (1) downsampling process, (2) vector operation (weighted matrix operation) process, and (3) upsampling process after boundary pixel preparation process.

First, as the boundary pixel preparation process, one line of boundary pixels is acquired from the adjacent block on the immediate left with respect to the block to be encoded (decoded), and the boundary pixel is formed into the immediate left boundary pixel string. Further, the boundary pixels of one line are acquired from the adjacent blocks directly above, and the boundary pixels are obtained as the boundary pixels directly above. Here, when the boundary pixel is not available, it is created by the same method as the existing intra prediction. If there is no decoding pixel in the boundary pixel directly to the left or directly above, the median value is set to 1 << (Bitdepth-1). Scan from the lower left pixel to the upper right in the order of upward and right, and copy the pixel that existed for the first time to the lower left pixel. Scan from the lower left to the upper again, and if there are no pixels, copy the pixels below it. Similarly, scan from the upper left to the right, and if there is no pixel, copy the pixel to the left.

Next, as (1) downsampling processing, the immediate left boundary pixel string and the immediate upper boundary pixel string are averaged every two pixels, respectively, and the reduced direct left boundary pixel string and the reduced direct upper boundary pixel string are obtained.

Next, as (2) vector calculation (weighting matrix calculation), the reduced direct left boundary pixel string and the reduced direct upper boundary pixel string are combined by a method determined by the size of the block to be encoded (decoded) and the specified mode. To generate a reduced boundary vector. Next, the vector operation by the matrix specified by the matrix weighted intra prediction mode is performed on the above-mentioned reduction boundary vector to generate the reduction prediction vector. The reduction prediction vector is used to generate the prediction pixel of the subsample position of the original decoding target block.

Next, as (3) upsampling processing, linear interpolation is performed from the boundary pixel string, the reduced boundary pixel string, and the predicted pixel of the subsample position to generate the entire predicted pixel of the coded (decoded) target block.

Next, FIG. 13 shows an example of a weighting parameter w determination process in the synthetic prediction process in the image coding apparatus and the image coding method according to the eighth embodiment of the present invention. Since the same weighting parameter w is determined on the decoding side, the prediction target block will be described by the expression “encoding (decoding) target block”.

In the process of determining the weighting parameter w in the processing of the composite prediction in FIG. 13, the weighting parameter when the intra prediction used in the processing of the composite prediction is the matrix weighted intra prediction in the process of determining the weighting parameter w in FIG. The determination process of w is added. Specifically, the rightmost column of the table in FIG. 13 has been added. Here, in the matrix weighted intra prediction, the predicted image changes depending on the weighted matrix specified by the flag for the coded (decoded) target block in the matrix weighted intra prediction mode. The flag is specified on the coding side and transmitted to the decoding side. As a result, the weighting matrix specified on the coding side can be discriminated on the decoding side as well. Therefore, the matrix weighted intra-prediction is an intra-prediction in which the approximation of the prediction method with the adjacent block is not high. However, since both the boundary pixel array of the adjacent block directly above and the boundary pixel array of the adjacent block directly above are used, the approximation of the predicted image of the matrix weighted intra prediction and the predicted image of the adjacent block is relatively high. Further, the matrix weighted intra prediction has a different tendency from the planner prediction in which it is difficult to improve the prediction accuracy because the prediction accuracy can be increased according to the image depending on the weighting matrix specified by the flag. Therefore, when matrix weighted intra-prediction is used as the intra-prediction used for the composite prediction, the weighting parameter w may be determined as w = 3 as shown in FIG. That is, the value of w that is larger than the value of w in the planner prediction may be selected.

According to the determination example of w in the synthetic prediction mode of FIG. 13 described above, even when matrix weighted intra prediction is used as an option of the type of intra prediction used in the synthetic prediction mode, the intra prediction image and the inter prediction image are combined. It becomes possible to more preferably determine the weighting of the intra-predicted image in.

According to the image coding apparatus and the image coding method according to the eighth embodiment described above, a more suitable synthesis prediction mode can be realized, and an image can be more preferably encoded.

(Example 9)
Next, Example 9 of the present invention will be described.

In the image decoding apparatus and the image decoding method according to the second embodiment, the ninth embodiment of the present invention also corresponds to the predicted image by the matrix weighted intra-prediction as the intra-predicted image used for the processing of the composite prediction. The matrix weighted intra prediction in this embodiment is an example of the prediction process described as “matrix prediction” in the second embodiment. Since the details of the matrix weighted intra-prediction processing are as described in the eighth embodiment, the repeated description will be omitted.

Specifically, the determination process of the weighting parameter w is changed from the determination process of the weighting parameter w shown in FIG. 8 to the determination process of the weighting parameter w shown in FIG. The image decoding apparatus and the image decoding method according to the ninth embodiment use the determination process of the weighting parameter w shown in FIG. 13, but other configurations and operations other than the determination process of the weighting parameter w shown in FIG. 13 are used. , The processing is the same as the image decoding apparatus and the image decoding method according to the second embodiment. Therefore, the same points as the image decoding apparatus and the image decoding method according to the second embodiment will not be repeated.

Further, since the details of the determination process of the weighting parameter w in FIG. 13 are as described in the eighth embodiment, the repeated description will be omitted.

In the image decoding apparatus and image decoding method according to the ninth embodiment of the present invention described above, as shown in FIG. 13, even when matrix-weighted intra-prediction is used as an option of the type of intra-prediction used in the composite prediction mode. , It becomes possible to more preferably determine the weighting of the intra-prediction image in the synthesis of the intra-prediction image and the inter-prediction image.

According to the image decoding apparatus and the image decoding method according to the ninth embodiment of the present invention described above, a more suitable synthesis prediction mode can be realized, and an image can be more preferably decoded.

It should be noted that any combination of the above-described drawings and examples of each method and the like can be an embodiment of the present invention.

According to each embodiment of the present invention described above, the amount of code can be reduced and deterioration of image quality can be prevented. That is, a high compression rate and a better image quality can be realized.

101 ... Image input unit, 102 ... Block division unit, 103 ... Mode management unit, 104 ... Intra prediction unit, 105 ... Inter prediction unit, 106 ... Block processing unit, 107 ... Conversion / quantization unit, 108 ... Inverse quantization / Inverse conversion unit, 109 ... image composition / filter unit, 110 ... decoded image management unit, 111 ... entropy coding unit, 112 ... data output unit, 201 ... stream analysis unit, 202 ... block management unit, 203 ... mode determination unit, 204 ... Intra prediction unit, 205 ... Inter prediction unit, 206 ... Coefficient analysis unit, 207 ... Inverse quantization / inverse conversion unit, 208 ... Image composition / filter unit, 209 ... Decoded image management unit, 210 ... Image output unit, 120 ... Synthetic prediction unit, 220 ... Synthetic prediction unit.

Claims (6)

An image coding method that encodes an image.
A prediction image generation step of generating a prediction image of the composite prediction by performing a synthesis process for synthesizing the prediction image of the inter-prediction and the prediction image of the intra-prediction for the coded target block.
A coding step for encoding the difference between the predicted image generated in the predicted image generation step and the pixel value of the image of the coded target block is provided.
A plurality of types of intra-prediction can be used for the intra-prediction used in the synthesis process, and the plurality of types of intra-prediction include a matrix-weighted intra-prediction. The prediction image of the inter prediction and the prediction image of the intra prediction are weighted, and the weighting parameters of the prediction image of the intra prediction in the weighting process are determined according to the type of the intra prediction. Image coding method. An image coding method that encodes an image.
A prediction image generation step of generating a prediction image of the composite prediction by performing a synthesis process for synthesizing the prediction image of the inter-prediction and the prediction image of the intra-prediction for the coded target block.
A coding step for encoding the difference between the predicted image generated in the predicted image generation step and the pixel value of the image of the coded target block is provided.
The types of intra-prediction that can be used in the prediction image generation step include planner prediction and matrix-weighted intra-prediction.
In the synthesis processing, the prediction image of the inter-prediction and the prediction image of the intra-prediction are weighted, and in the processing state of the weighting processing, the weighting of the prediction image of the intra-prediction is set. An image coding method in which, among the types of intra-prediction, the planner prediction is set to be weighted smaller than the matrix-weighted intra-prediction. An image coding method that encodes an image.
A prediction image generation step of generating a prediction image of the composite prediction by performing a synthesis process for synthesizing the prediction image of the inter-prediction and the prediction image of the intra-prediction for the coded target block.
A coding step for encoding the difference between the predicted image generated in the predicted image generation step and the pixel value of the image of the coded target block is provided.
The types of intra-prediction that can be used in the prediction image generation step include planner prediction and matrix-weighted intra-prediction.
In the synthesis processing, the prediction image of the inter-prediction and the prediction image of the intra-prediction are weighted, and in the processing state of the weighting processing, the type of the intra-prediction is planner prediction. , An image coding method in which the weighting of the predicted image of the intra prediction is set to the minimum. An image decoding method that decodes a coded stream that encodes an image.
A predictive image generation step of generating a predicted image of the composite prediction by performing a composite process of synthesizing the predicted image of the inter-prediction and the predicted image of the intra-prediction for the block to be decoded.
A decoded image generation step of generating a decoded image based on the predicted image generated in the predicted image generation step and the difference image of the decoding target block is provided.
A plurality of types of intra-prediction can be used for the intra-prediction used in the synthesis process, and the plurality of types of intra-prediction includes a matrix-weighted intra-prediction.
In the synthesis process, the prediction image of the inter-prediction and the prediction image of the intra-prediction are weighted, and the weighting parameter of the prediction image of the intra-prediction in the weighting process is the type of the intra-prediction. Image decoding method determined according to. An image decoding method that decodes a coded stream that encodes an image.
A predictive image generation step of generating a predicted image of the composite prediction by performing a composite process of synthesizing the predicted image of the inter-prediction and the predicted image of the intra-prediction for the block to be decoded.
A decoded image generation step of generating a decoded image based on the predicted image generated in the predicted image generation step and the difference image of the decoding target block is provided.
The types of intra-prediction that can be used in the prediction image generation step include planner prediction and matrix-weighted intra-prediction.
In the synthesis processing, the prediction image of the inter-prediction and the prediction image of the intra-prediction are weighted, and in the processing state of the weighting processing, the weighting of the prediction image of the intra-prediction is set. An image decoding method in which, among the types of intra-prediction, the planner prediction is set to be weighted smaller than the matrix-weighted intra-prediction. An image decoding method that decodes a coded stream that encodes an image.
A predictive image generation step of generating a predicted image of the composite prediction by performing a composite process of synthesizing the predicted image of the inter-prediction and the predicted image of the intra-prediction for the block to be decoded.
A decoded image generation step of generating a decoded image based on the predicted image generated in the predicted image generation step and the difference image of the decoding target block is provided.
The types of intra-prediction that can be used in the prediction image generation step include planner prediction and matrix-weighted intra-prediction.
In the synthesis processing, the prediction image of the inter prediction and the prediction image of the intra prediction are weighted, and in the processing state of the weighting processing, the type of the intra prediction is planner prediction. , An image decoding method in which the weighting of the predicted image of the intra prediction is set to the minimum.

Images