JP2022125265A - Decoding device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve prediction accuracy and encoding efficiency without increasing an amount of information to be transmitted by an encoding device and without increasing a computation time on the encoding device side.

SOLUTION: An encoding device 1 comprises: a composite region determination unit 12 that on the basis of an intra-prediction mode, determines a composite region X, in which a prediction image is generated from adjacent pixels that are not decoded, in an encoding target block CU/TU; and an intra-prediction unit 13 for changing a method for generating a prediction image for each region on the basis of whether or not the region is included in the composite region X.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a decoding device and program.

H. 265/HEVC (High Efficiency Video Coding), two types of prediction are used: inter prediction using temporal correlation between frames and intra prediction using spatial correlation within a frame. After performing prediction while switching and generating a residual signal, it is configured to perform orthogonal transform processing, loop filter processing, and entropy coding processing and output the obtained stream.

Intra prediction in HEVC has a total of 35 modes of planar prediction, DC prediction, and directional prediction, and is configured to perform intra prediction using adjacent decoded reference pixels according to the mode determined by the encoder. It is

Here, in intra-prediction, a CU (Coding Unit) located at the upper leftmost position in a frame, such as a CU having no adjacent decoded reference pixels, has a specified value ("512" for a 10-bit video). is configured to generate reference pixels used when generating a predicted image.

In addition, in conventional HEVC, encoding processing is performed in raster scan order from the upper left, so there are cases where reference pixels have not been decoded. In such a case, a prediction image is generated using a value obtained by extrapolating the nearest decoded reference pixel to the 0th order.

A CU is divided into a plurality of blocks (TU: Transform Unit) and subjected to orthogonal transform processing. In HEVC, an intra-prediction mode indicating the type of intra-prediction to be applied is common within a CU, and prediction is performed using a common intra-prediction mode for a plurality of TUs.

In intra prediction, as shown in FIG. 7, there are many cases where the reference pixel N located at the lower left or upper right of CU#X has not been decoded due to the encoding process in the raster scan order. There is a problem that the directional prediction from the direction in which the reference pixels are present lowers the prediction accuracy and the coding efficiency.

These problems will be specifically described below with reference to FIGS. 8(a) to 8(d). FIG. 8 shows an example of intra prediction in conventional HEVC.

In such an example, as shown in FIG. 8(a), as in the example of FIG. 7, all reference pixels of CU#A1 (the upper left CU in the frame) have already been decoded. Similarly, as shown in FIG. 8C, all the reference pixels of CU#A3 (the lower left CU in the frame) have been decoded.

On the other hand, as shown in FIG. 8B, reference pixels W1 to W3 located in CU#A1 have been decoded, but reference pixels B1 to B4 located in CU#A3 have not been decoded. Therefore, it cannot be directly used as a reference pixel when generating a prediction image for CU#A2 (the upper right CU in the frame).

For this reason, in conventional HEVC, as shown in FIG. are specified to be copied to undecoded reference pixels B1 to B4 located in the same column of .

Similarly, in conventional HEVC, as shown in FIG. It is defined to be copied to the undecoded reference pixel P3 located in the same column in the underlying CU.

Therefore, when directional prediction is performed from the lower left to the upper right, as in the example of FIG. However, there is a problem that the coding efficiency is reduced.

In order to solve this problem, in intra prediction in which TU partitioning is performed, the order of encoding processing for a plurality of TUs present in a CU may be raster scan order (for example, Z type), U type or X type. There is known a technique for improving prediction accuracy by giving flexibility to the order of encoding such as (see Non-Patent Document 1).

Mochizuki et al., “Applied Intra Prediction Method Based on Mean Value Coordinates”, Research Report of Information Processing Society of Japan, vol, 2012-AVM-77, No. 12

However, with the technology specified in Non-Patent Document 1, it is necessary to transmit a flag indicating what kind of encoding processing order is to be used in units of CUs. In order to select the best encoding process order from among all the encoding process orders, it is necessary for the encoding device to try all combinations, which increases the calculation time on the encoding device side. There was a problem.

SUMMARY OF THE INVENTION Accordingly, the present invention has been made to solve the above-described problems. An object of the present invention is to provide an encoding device, a decoding device, and a program capable of improving accuracy and encoding efficiency.

A first feature of the present invention is an encoding device that is configured to divide and encode an original image in units of frames that constitute a moving image into encoding target blocks, the encoding device comprising: a synthesis region determination unit configured to determine a synthesis region in which a predicted image is generated from adjacent pixels that have not been decoded in the encoding target block; and an intra prediction unit configured to change a method of generating a predicted image for each region.

A second feature of the present invention is a decoding device configured to divide an original image in units of frames constituting a moving image into encoding target blocks and decode them, based on the intra prediction mode, a synthesis region determination unit configured to determine a synthesis region in which a predicted image is generated from adjacent pixels that have not been decoded in the encoding target block; and each region based on whether it is included in the synthesis region and an intra prediction unit configured to change a method of generating a predicted image of the.

A gist of a third aspect of the present invention is a program for causing a computer to function as the encoding device according to the first or second aspect.

A gist of a fourth aspect of the present invention is a program for causing a computer to function as the decoding device according to the first or second aspect.

INDUSTRIAL APPLICABILITY According to the present invention, an encoding device capable of improving prediction accuracy and encoding efficiency without increasing the amount of information transmitted by the encoding device and without increasing the calculation time of the encoding device. , a decoding device and a program can be provided.

FIG. 1 is a functional block diagram of an encoding device 1 according to the first embodiment. FIG. 2 is a diagram showing an example of how a predicted image is generated in the first embodiment. FIG. 3 is a diagram showing an example of how a predicted image is generated in the first embodiment. FIG. 4 is a functional block diagram of the decoding device 3 according to the first embodiment. FIG. 5 is a flow chart showing operations of the encoding device 1 and the decoding device 3 according to the first embodiment. FIG. 6 is a diagram showing an example of how a predicted image is generated in the second embodiment. FIG. 7 is a diagram showing an example of how a predicted image is generated in conventional HEVC. FIG. 8 is a diagram showing an example of how a predicted image is generated in conventional HEVC.

(First embodiment)
An encoding device 1 and a decoding device 3 according to the first embodiment of the present invention will be described below with reference to FIGS. 1 to 5. FIG. Here, the encoding device 1 and the decoding device 3 according to this embodiment are configured to support intra prediction in HEVC.

The encoding device 1 according to the present embodiment is configured to divide an original image of each frame constituting a moving image into blocks to be encoded and encode the blocks. In this embodiment, it is assumed that all the pixels adjacent to the left side and upper side of the CU to be encoded have already been decoded.

Also, the encoding device 1 according to the present embodiment may be configured to be able to divide a CU into a plurality of TUs. Therefore, in this embodiment, the encoding target block may be a CU or a TU. In the present embodiment, a case in which a CU is used as an encoding target block will be described below as an example.

Further, in this embodiment, as an encoding order and a decoding order of CUs, a case is taken as an example where the raster scan order (Z type as shown in FIGS. 7 and 8) used in conventional HEVC is used. However, as the encoding order and decoding order, other encoding order and decoding order such as U-type and X-type may be used.

As shown in FIG. 1, the coding apparatus 1 according to the present embodiment includes an intra prediction mode determination unit 11, a combining region determination unit 12, an intra prediction unit 13, a residual signal generation unit 14, an orthogonal transform/ It comprises a quantization unit 15 , an inverse orthogonal transform/inverse quantization unit 16 , a locally decoded image generation unit 17 , a memory unit 18 , and an entropy coding unit 19 .

The intra-prediction mode determination unit 11 is configured to determine the optimum intra-prediction mode to apply to the CU.

The synthesis region determination unit 12 is configured to determine a synthesis region X in which a predicted image is generated from adjacent pixels that have not been decoded in the CU, based on the intra prediction mode determined by the intra prediction mode determination unit 11. there is

Specifically, as shown in FIG. 2, the synthesis region determination unit 12 determines that the direction of the intra prediction mode determined by the intra prediction mode determination unit 11 is from the lower left to the upper right direction (that is, from the lower left to the upper right direction). direction prediction is performed), the composite region X in which the prediction image is generated from the undecoded adjacent pixels B1 to B4 in the CU#A2 may be determined.

The present invention is also applicable when the direction of the intra prediction mode determined by the intra prediction mode determination unit 11 is from upper right to lower left (that is, when directional prediction is performed from upper right to lower left). It is possible, but hereinafter, the case where the direction of the intra prediction mode determined by the intra prediction mode determination unit 11 is the direction from the lower left to the upper right (that is, the case where the direction prediction is performed from the lower left to the upper right) is taken as an example. will be described.

In the drawings of this specification, the arrow indicating the direction of the intra prediction mode (prediction direction) is directed from the target pixel for intra prediction to the reference pixel, as described in the HEVC standard (the same applies hereinafter). .

The intra prediction unit 13 is configured to change the method of generating a predicted image for each region based on whether or not it is included in the synthesis region X determined by the synthesis region determination unit 12 .

Specifically, as shown in FIG. 3(a), the intra prediction unit 13 predicts an area (that is, an area in which reference pixels W1 to W3 have been decoded) in CU#A2 that is not included in the synthesis area X. Y is configured to use the intra prediction mode determined by the intra prediction mode determining unit 11 to generate a predicted image based on the decoded reference pixels W1 to W3.

On the other hand, as shown in FIG. 3B, in CU#A2, the intra prediction unit 13 defines in advance It is configured to generate a predicted image using the predicted method.

Here, the predetermined prediction method includes, for example, intra prediction such as DC prediction and planar prediction, as well as a prediction method that generates a prediction image by averaging values of adjacent decoded reference pixels.

Note that if the prediction method is determined in advance in the encoding device 1 and the decoding device 3, the prediction method is sent from the encoding device 1 to the decoding device 3 regardless of which prediction method is used. There is no need to transmit a new flag to indicate.

In addition, since the synthesis region determined by the synthesis region determination unit 12 can be uniquely determined based on the direction of the intra prediction mode and whether or not the adjacent pixels have been decoded, the encoding device 1 to the decoding device 3 , there is no need to newly transmit a flag indicating which area is the synthesis area.

For example, as shown in FIG. 3B , the intra prediction unit 13 uses planar prediction to predict pixel W3 and pixel D1 at the position of pixel X1 in the region included in synthesis region X in CU#A2. , the pixel D2 and the pixel D4 may be combined to generate the predicted image. Here, the value of pixel B4 is copied to pixel D4, and the value of pixel W1 is copied to pixel B4.

The residual signal generation unit 14 is configured to generate a residual signal from the difference between the predicted image generated by the intra prediction unit 13 and the original image.

The orthogonal transformation/quantization unit 15 is configured to perform orthogonal transformation processing and quantization processing on the residual signal generated by the residual signal generation unit 14 and generate quantized transform coefficients. .

The inverse orthogonal transform/inverse quantization unit 16 performs inverse quantization processing and inverse orthogonal transform processing again on the quantized transform coefficients generated by the orthogonal transform/quantization unit 15, and obtains a quantized residue. configured to generate a difference signal;

The local decoded image generation unit 17 adds the predicted image generated by the intra prediction unit 13 to the quantized residual signal generated by the inverse orthogonal transform/inverse quantization unit 16 to generate a local decoded image. is configured to

The memory unit 18 is configured to hold the local decoded image generated by the local decoded image generation unit 17 so that it can be used as a reference image.

The entropy coding unit 19 is configured to perform entropy coding processing on the flag information including the intra prediction mode determined by the intra prediction mode determination unit 11 and the quantized transform coefficients, and output a stream. there is

Further, the decoding device 3 according to the present embodiment is configured to divide an original image of each frame constituting a moving image into CUs and decode the CUs. Also, the decoding device 3 according to this embodiment is configured to be able to divide a CU into a plurality of TUs, like the encoding device 1 according to this embodiment.

As shown in FIG. 4, the decoding device 3 according to the present embodiment includes an entropy decoding unit 31, a synthesis region determination unit 32, an intra prediction unit 33, an inverse quantization/inverse transform unit 34, and a local decoded image generation unit. It comprises a section 35 and a memory section 36 .

The entropy decoding unit 31 is configured to decode transform coefficients, flag information, and the like from the stream output from the encoding device 1 . Here, the transform coefficients are quantized transform coefficients obtained by the encoding apparatus 1 as signals encoded by dividing the original image in units of frames into CUs. Also, the flag information includes accompanying information such as prediction mode.

The synthesis region determination unit 32 is configured to determine a synthesis region X in which a predicted image is generated from adjacent pixels that have not been decoded in the CU, based on the intra prediction mode output by the entropy decoding unit 31 .

Specifically, as shown in FIG. 2, the synthesis region determination unit 32 determines that the direction of the intra-prediction mode output by the entropy decoding unit 31 is from the lower left to the upper right (that is, from the lower left to the upper right). (when directional prediction is performed in CU#A2), a synthesis region X in which a prediction image is generated from adjacent pixels B1 to B4 that have not been decoded in CU#A2 may be determined.

The intra prediction unit 33 may be configured to generate a predicted image using the synthesis region X determined by the synthesis region determination unit 32 and the intra prediction mode output by the entropy decoding unit 31 .

Specifically, similar to the intra prediction unit 13, the intra prediction unit 33 determines a predicted image generation method for each region based on whether or not it is included in the synthesis region X determined by the synthesis region determination unit 12. configured to change.

For example, as shown in FIG. 3(a), the intra prediction unit 33 performs the The intra prediction mode output by the entropy decoding unit 31 is used to generate a prediction image based on the decoded reference pixels W1 to W3.

On the other hand, as shown in FIG. 3B, the intra prediction unit 33 performs the above-described It is configured to generate a predicted image using a predefined prediction method.

The inverse quantization/inverse transform unit 34 performs inverse quantization processing and inverse transform processing (for example, inverse orthogonal transform processing) on the quantized transform coefficients output from the entropy decoding unit 31 to obtain a residual configured to generate a signal;

The local decoded image generation unit 35 is configured to generate a local decoded image by adding the predicted image generated by the intra prediction unit 33 and the residual signal generated by the inverse quantization/inverse transform unit 34. there is

The memory unit 36 is configured to hold the locally decoded image generated by the locally decoded image generation unit 35 so that it can be used as a reference image for intra prediction and inter prediction.

FIG. 5 shows a flowchart for explaining an example of the operation of generating a predicted image by the encoding device 1 and the decoding device 3 according to this embodiment.

First, with reference to FIG. 5, an example of the operation of generating a predicted image by the encoding device 1 according to this embodiment will be described.

As shown in FIG. 5, in step S101, the encoding device 1 determines the optimum intra prediction mode to apply to the CU.

In step S102, the encoding device 1 determines a synthesis region X in which a predicted image is generated from adjacent pixels that have not been decoded in the CU, based on the intra prediction mode determined in step S101.

In step S<b>103 , the encoding device 1 determines whether or not the area for generating the predicted image is included in the synthesis area within the CU. If "No", the operation proceeds to step S104, and if "Yes", the operation proceeds to step S105.

In step S104, the encoding device 1 uses the intra prediction mode determined in step S101 to generate a predicted image based on the decoded reference pixels.

In step S105, the encoding device 1 generates a predicted image using the above-described predetermined prediction method.

Secondly, with reference to FIG. 5, an example of the operation of generating a predicted image by the decoding device 3 according to this embodiment will be described.

As shown in FIG. 5, in step S101, the decoding device 3 determines the intra prediction mode based on the information obtained by the entropy decoding process.

In step S102, the decoding device 3 determines a synthesis region X in which a predicted image is generated from adjacent pixels that have not been decoded in the CU, based on the intra prediction mode determined in step S101.

In step S103, the decoding device 3 determines whether or not the area for generating the predicted image is included in the synthesis area in the CU. If "No", the operation proceeds to step S104, and if "Yes", the operation proceeds to step S105.

In step S104, the decoding device 3 uses the intra prediction mode determined in step S101 to generate a predicted image based on the decoded reference pixels.

In step S105, the decoding device 3 generates a predicted image using the above-described predetermined prediction method.

According to the encoding device 1 and the decoding device 3 according to the present embodiment, when an undecoded pixel is included in the reference destination in the intra prediction mode direction (prediction direction), adjacent decoded reference pixels are copied. By generating a new predicted image by synthesizing the predicted image generated using the reference pixels interpolated by and the predicted image generated using the predetermined prediction method, it is possible to suppress the deterioration of prediction accuracy. .

(Second embodiment)
Hereinafter, with reference to FIG. 6, regarding the encoding device 1 and the decoding device 3 according to the second embodiment of the present invention, differences from the encoding device 1 and the decoding device 3 according to the above-described first embodiment will be described.

In the encoding device 1 according to the present embodiment, the intra prediction unit 13, when a predetermined condition is satisfied in the boundary area between the area included in the synthesis area X and the area Y not included in the synthesis area X, It is configured to apply a smoothing filter to the predicted image of the boundary area.

Similarly, in the decoding device 3 according to the present embodiment, the intra prediction unit 33 performs First, it is configured to apply a smoothing filter to the predicted image of the boundary area.

For example, in the example of FIG. 6, the boundary area includes pixels X1, X2, X3, and X4 in the area included in the composite area X and pixels Y1, Y2, and Y3 in the area Y not included in the composite area X. The area where it is located corresponds.

Further, the predetermined condition is that the discontinuity in the boundary area is higher than a predetermined threshold. Here, as the evaluation of discontinuity, for example, there is a method of expressing by linear combination of horizontal difference, vertical difference, oblique difference, and the like.

That is, the intra prediction unit 13 and the intra prediction unit 33 apply a smoothing filter to the predicted image of the boundary region when the evaluation value of discontinuity in the boundary region is higher than a predetermined threshold. It is configured.

For example, in the example of FIG. 6 , the intra prediction unit 13 and the intra prediction unit 33, when the discontinuity evaluation value in the boundary region is higher than a predetermined threshold value, the pixels in the region included in the synthesis region X It is configured to apply a smoothing filter to the predicted image of the region where the pixels Y1, Y2, and Y3 in the region Y not included in the composite region X are located.

Note that, in the example of FIG. 6, the intra prediction unit 13 and the intra prediction unit 33 also apply the smoothing filter to the prediction image of the region in which the pixels X5, X6, and X7 in the region included in the synthesis region X are located. is configured as

Here, by sharing the above-described predetermined threshold value between the encoding device 1 and the decoding device 3, there is no need to transmit a new flag from the encoding device 1 to the decoding device 3. FIG.

According to the encoding device 1 and the decoding device 3 according to the present embodiment, the discontinuity of the boundary region caused by changing the prediction image generation method for each region based on whether or not it is included in the synthesis region X can be reduced and the coding performance can be improved.

(Third Embodiment)
Hereinafter, with reference to FIG. 6, the encoding device 1 and the decoding device 3 according to the third embodiment of the present invention are similar to the encoding device 1 and the decoding device 3 according to the above-described first and second embodiments. Description will be made with a focus on the differences.

In the encoding device 1 and the decoding device 3 according to the present embodiment, the intra prediction unit 13 and the intra prediction unit 33 generate predicted images ( Predicted by a weighted average of the predicted image generated based on the undecoded reference pixels filled by the copy) and the predicted image generated in the synthesis region X (predicted image generated using the above-mentioned predefined prediction method) configured to generate an image;

Note that, in the encoding device 1 and the decoding device 3 according to the present embodiment, the intra prediction unit 13 and the intra prediction unit 33 perform the conventional A predicted image generated using the prediction method in HEVC (predicted image generated based on undecoded reference pixels embedded by copying) and a predicted image generated in the synthesis region X (using the above-mentioned predefined prediction method The predicted image may be generated by a weighted average of the predicted images generated using the

In the encoding device 1 and the decoding device 3 according to this embodiment, the intra prediction unit 13 and the intra prediction unit 33 may be configured to calculate weighting factors used when calculating such weighted averages.

The intra prediction unit 13 and the intra prediction unit 33 may be configured to use a predetermined value as the weighting factor, or the weighting factor may be calculated using the reliability of the decoded reference pixels when generating the predicted image. may be configured to calculate

In the example of FIG. 3B, the reliability of the decoded reference pixels W1 to W4 is high, and the reliability of the reference pixels B1 to B4 generated by copying is low. Further, when the reliability of the reference pixel B1 and the reliability of the reference pixel B4 are compared, the reliability of the reference pixel B4 closer to the copy source is higher.

Also, the intra prediction unit 13 and the intra prediction unit 33 may calculate the weighting factor according to the distance from the boundary region between the region included in the composite region X and the region Y not included in the composite region X. may be configured to

For example, the intra-prediction unit 13 and the intra-prediction unit 33 generate a prediction image generated using a prediction method in conventional HEVC (prediction generated based on undecoded reference pixels embedded by copying) in the vicinity of such a boundary region. image) is calculated so as to increase the weighting factor, and in an area away from such a boundary area, the weight of the predicted image generated in the synthesis area X (predicted image generated using the above-described predetermined prediction method) is weighted. You may calculate so that a coefficient may become strong.

(Other embodiments)
As noted above, the present invention has been described through the above-described embodiments, but the statements and drawings forming part of the disclosure in such embodiments should not be understood to limit the present invention. From such disclosure, various alternative embodiments, implementations and operational techniques will become apparent to those skilled in the art.

Moreover, although not particularly mentioned in the above-described embodiment, a program may be provided that causes a computer to execute each process performed by the above-described encoding device 1 and decoding device 3 . Also, such a program may be recorded on a computer-readable medium. Such programs can be installed on a computer using a computer readable medium. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be, for example, a recording medium such as CD-ROM or DVD-ROM.

Alternatively, a chip configured by a memory storing a program for realizing at least part of the functions in the encoding device 1 and the decoding device 3 described above and a processor executing the program stored in the memory may be provided. good.

REFERENCE SIGNS LIST 1 coding device 11 intra prediction mode determination unit 12 synthetic region determination unit 13 intra prediction unit 14 residual signal generation unit 15 orthogonal transform/quantization unit 16 inverse quantization/inverse orthogonal transformation unit 17 Locally decoded image generating unit 18 Memory unit 19 Entropy encoding unit 3 Decoding device 31 Entropy decoding unit 32 Combined region determination unit 33 Intra prediction unit 34 Inverse quantization/inverse transform unit 35 Local decoded image Generation unit 36: memory unit

Claims (2)

A decoding device configured to decode an encoding target block obtained by dividing an original image in units of frames constituting a moving image,
a first prediction unit that generates a first prediction image by a first prediction process that is determined based on a prediction mode specified by flag information transmitted from the encoding side;
a synthesis area determination unit configured to determine a synthesis area in the encoding target block;
A second prediction process that is different from the first prediction process, and is determined without transmitting new flag information in addition to the flag information transmitted from the encoding side. a second predictor that generates;
a predicted image synthesizing unit that generates a predicted image included in the synthesized region by weighted averaging of the first predicted image and the second predicted image,
The decoding device, wherein the predicted image synthesizing unit determines the weight used for the weighted average for each pixel according to the position of the pixel included in the synthesizing area. A program for causing a computer to function as the decoding device according to claim 1.

Images