JP7476279B2 - Encoding device, decoding device, and program - Google Patents

Description

The present invention relates to an encoding device, a decoding device, and a program.

Video (image) coding methods such as H.265/HEVC (High Efficiency Video Coding) are configured to generate residual signals by switching between two types of prediction: inter-prediction, which uses temporal correlation between frames, and intra-prediction, which uses spatial correlation within a frame, and then output the resulting stream by performing orthogonal transform processing, loop filter processing, and entropy coding processing.

HEVC provides 35 types of intra prediction modes, including planar prediction, DC prediction, and directional prediction, and is configured to perform intra prediction using adjacent decoded reference pixels according to the intra prediction mode determined by the encoder. Hereinafter, unless otherwise specified, the term "reference pixels" refers to decoded reference pixels.

Here, in intra prediction in HEVC, in a CU that has no adjacent reference pixels, such as the coding unit (CU) located at the top left corner of a frame, the reference pixels used when generating a predicted image are created by filling in a specified value (for a 10-bit video image, this is "512").

In addition, in conventional HEVC, the encoding process is performed in raster scan order from the top left, so there are cases where the reference pixels have not been decoded. In such cases, the system is configured to generate a predicted image using a value obtained by zero-order extrapolation of the closest decoded reference pixel.

In particular, with conventional HEVC intra prediction, the coding process uses raster scan order, so reference pixels located on the lower left or upper right side of a CU are often not yet decoded. In such cases, performing directional prediction from a direction where undecoded reference pixels exist reduces prediction accuracy, resulting in reduced coding efficiency.

To solve this problem, a technique is known for improving prediction accuracy in intra prediction by allowing flexibility in the coding order of multiple transform blocks (hereinafter referred to as "TU: Transform Unit") present in a CU, such as raster scan order (e.g., Z-type), as well as U-type and X-type coding orders (see Non-Patent Document 1).

Furthermore, the intra prediction used in HEVC is a prediction that uses spatially adjacent reference pixels above or to the left, and the accuracy of predicted pixels closer to the reference pixels tends to be high, while the accuracy of predicted pixels farther from the reference pixels tends to be low (see Figure 10).

Note that in the figures in this specification, the arrow indicating the direction of the intra prediction mode (prediction direction) points from the pixel targeted for intra prediction to the reference pixel, as described in the HEVC standard (same below).

Conventional HEVC takes advantage of this property by applying orthogonal transform processing such as discrete sine transform (DST) or discrete cosine transform (DCT) in the horizontal and vertical directions from the left and top directions where the reference pixels are located, thereby reducing the entropy of the residual signal.

In particular, as shown in Figure 11, the shape of the DST impulse response is asymmetric, with one end point closed and the other end point expanding, so that entropy can be effectively reduced by applying DST in accordance with the signal strength of the residual signal, as shown in Figure 12.

Incidentally, Non-Patent Document 2 discloses a technology for efficiently reducing entropy by applying a secondary orthogonal transform process (hereinafter, secondary orthogonal transform process) to residual signals in which entropy is difficult to reduce in the orthogonal transform processes (DCT and DST) used in conventional HEVC.

Specifically, the technology described in Non-Patent Document 2 is configured to apply conventional orthogonal transform processing to the residual signal obtained by intra prediction, and then divide the obtained orthogonal transform coefficients into small blocks and apply a quadratic orthogonal transform processing to each block.

The encoding device is configured to select an optimal quadratic orthogonal transform process from a group of quadratic orthogonal transform processes (a set of quadratic orthogonal transform processes) for which multiple types of bases are provided, and to output flag information indicating the selected quadratic orthogonal transform process as a stream (if it is optimal not to apply the quadratic orthogonal transform process, flag information indicating that the quadratic orthogonal transform process will not be applied as a stream).

In addition, the technology described in Non-Patent Document 2 is configured to switch between selectable quadratic orthogonal transform processing groups depending on the intra prediction mode, since the characteristics of the residual signal and the characteristics of the orthogonal transform coefficients differ depending on the direction of the intra prediction mode.

As a result, it is possible to select the optimal quadratic orthogonal transform process from among the group of quadratic orthogonal transform processes defined for each intra prediction mode, making it possible to reduce the amount of information required for the flag indicating which quadratic orthogonal transform process was used.

Mochizuki et al., "An adaptive intra prediction method based on mean coordinates," Information Processing Society of Japan Research Report, vol. 2012-AVM-77, no. X. Zhao, J. Chen, M. Karczewicz, "Mode-dependent non-separable secondary transform," ITU-T SG16/Q6 Doc. COM16-C1044, October 2015.

However, the technology described in Non-Patent Document 2 is configured to determine the optimal quadratic orthogonal transform process from a group of quadratic orthogonal transform processes according to the intra prediction mode, without taking into account the positions of the reference pixels used in the intra prediction mode.

Here, the characteristics of the residual signal obtained by intra prediction using the reference pixels on the right or bottom are different from the characteristics of the residual signal obtained by intra prediction using the reference signals on the top or left.

Therefore, in the technology described in Non-Patent Document 2, when intra prediction is performed using reference pixels on the right or bottom, if the quadratic orthogonal transform processing group is switched only in the intra prediction mode, there is a problem that the entropy may increase and the coding performance may decrease.

The present invention has been made to solve the above-mentioned problems, and aims to provide an encoding device, a decoding device, and a program that can efficiently reduce entropy and improve encoding performance in intra prediction.

The first feature of the present invention is an encoding device configured to divide an original image of each frame constituting a moving image into blocks to be encoded and encode the divided blocks, the encoding device comprising: an intra prediction unit configured to generate a predicted image using an intra prediction mode; a residual signal generation unit configured to generate a residual signal based on the difference between the predicted image generated by the intra prediction unit and the original image; an orthogonal transform unit configured to invert the residual signal generated by the residual signal generation unit in at least one of the horizontal and vertical directions and then perform an orthogonal transform process when the position of the reference pixel used to generate the predicted image includes at least one of the right and lower sides; and a quadratic orthogonal transform unit configured to select a quadratic orthogonal transform process to be applied from a group of quadratic orthogonal transforms defined in advance according to the intra prediction mode and the position of the reference pixel, and to perform the selected quadratic orthogonal transform process on the signal output from the orthogonal transform unit.

The second feature of the present invention is a decoding device configured to divide original images of frames constituting a moving image into blocks to be coded and decode the original images, and the decoding device includes an intra prediction unit configured to generate a predicted image using an intra prediction mode, an inverse quantization unit configured to perform an inverse quantization process on the quantized transform coefficients, a secondary inverse orthogonal transform unit configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms defined in advance according to the intra prediction mode and the positions of reference pixels used to generate the predicted image, and to perform the selected secondary inverse orthogonal transform process on the signal output from the inverse quantization unit, and an inverse orthogonal transform unit configured to invert the signal output from the secondary inverse orthogonal transform unit in at least one of the horizontal direction and the vertical direction and then perform an inverse orthogonal transform process when the positions of the reference pixels include at least one of the right side and the lower side.

The third feature of the present invention is an encoding device configured to divide an original image of each frame constituting a moving image into blocks to be encoded and encode the divided blocks, the encoding device comprising: an intra prediction unit configured to generate a predicted image using an intra prediction mode; a residual signal generation unit configured to generate a residual signal based on the difference between the predicted image generated by the intra prediction unit and the original image; an orthogonal transform unit configured to invert at least one of the horizontal and vertical bases and then perform orthogonal transform processing on the residual signal generated by the residual signal generation unit when the position of the reference pixel used to generate the predicted image includes at least one of the right and lower sides; and a quadratic orthogonal transform unit configured to select a quadratic orthogonal transform processing to be applied from a group of quadratic orthogonal transforms defined in advance according to the intra prediction mode and the position of the reference pixel, and to perform the selected quadratic orthogonal transform processing on the signal output from the orthogonal transform unit.

The fourth feature of the present invention is a decoding device configured to divide original images of frames constituting a moving image into blocks to be coded and decode the original images, and the decoding device includes an intra prediction unit configured to generate a predicted image using an intra prediction mode, an inverse quantization unit configured to perform an inverse quantization process on the quantized transform coefficients, a secondary inverse orthogonal transform unit configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms defined in advance according to the intra prediction mode and the position of the reference pixel used to generate the predicted image, and to perform the selected secondary inverse orthogonal transform process on the signal output from the inverse quantization unit, and an inverse orthogonal transform unit configured to invert at least one of the horizontal and vertical bases and then perform an inverse orthogonal transform process on the signal output from the secondary inverse orthogonal transform unit when the position of the reference pixel includes at least one of the right side and the lower side.

The fifth feature of the present invention is a program for causing a computer to function as the encoding device described in the first and third features above.

The sixth feature of the present invention is a program for causing a computer to function as the decoding device described in the second and fourth features above.

The present invention provides an encoding device, a decoding device, and a program that can efficiently reduce entropy and improve encoding performance in intra prediction.

FIG. 1 is a diagram showing an example of functional blocks of an encoding device 1 according to the first embodiment. FIG. 2 is a diagram showing an example of functional blocks of the orthogonal transform and quantization unit 14c of the encoding device 1 according to the first embodiment. FIG. 3 is a diagram showing an example of the directions of intra prediction modes used in the first embodiment. FIG. 4 is a diagram showing an example of a method for generating a predicted image in the first embodiment. FIG. 5 is a diagram showing an example of a method for generating a predicted image in the first embodiment. FIG. 6 is a flowchart showing an example of the operation of the encoding device 1 according to the first embodiment. FIG. 7 is a diagram illustrating an example of functional blocks of the decoding device 3 according to the first embodiment. FIG. 8 is a diagram showing an example of functional blocks of the inverse quantization and inverse transform unit 33b of the decoding device 3 according to the first embodiment. FIG. 9 is a flowchart showing an example of the operation of the decoding device 3 according to the first embodiment. FIG. 10 is a diagram for explaining the conventional technology. FIG. 11 is a diagram for explaining the conventional technology. FIG. 12 is a diagram for explaining the conventional technology.

First Embodiment
Hereinafter, an encoding device 1 and a decoding device 3 according to a first embodiment of the present invention will be described with reference to FIGS.

The encoding device 1 and decoding device 3 according to this embodiment are configured to support intra prediction in a video encoding method such as HEVC. Note that the encoding device 1 and decoding device 3 according to this embodiment are configured to support any video encoding method as long as the video encoding method performs intra prediction.

The encoding device 1 according to this embodiment is configured to divide original images of frames constituting a moving image into CUs and encode them. The encoding device 1 according to this embodiment may also be configured to divide a CU into multiple TUs. In the following, this embodiment will be described using an example of a case in which a CU is divided into multiple TUs, but the present invention can also be applied to cases in which a CU is not divided into multiple TUs and the position of the reference pixel of the CU includes the lower side and the right side.

In this embodiment, for a CU to be coded that has no adjacent decoded reference pixels, such as the CU located at the top left corner of a frame, the reference pixels used when generating a predicted image are created by filling in a specified value ("512" for a 10-bit video image), so that all adjacent pixels to the left of the CU to be coded can be used as reference pixels.

As shown in FIG. 1, the encoding device 1 according to this embodiment includes an intra-prediction mode determination unit 11, a TU partition determination unit 12, an encoding order control unit 13, a sequential local decoded image generation unit 14, a memory 15, and an entropy encoding unit 16.

The intra-prediction mode determination unit 11 is configured to determine the optimal intra-prediction mode to apply to the CU.

The TU partitioning determination unit 12 is configured to determine whether or not to partition a CU into multiple TUs. Note that in this embodiment, a four-part partition is used as an example of a method for partitioning a CU into multiple TUs, but the number of partitions and the partition shape when partitioning a CU into multiple TUs are not limited to this case.

The coding order control unit 13 is configured to determine the coding order of the TUs in a CU based on the intra prediction mode (e.g., the direction of the intra prediction mode).

For example, when the TU partition determination unit 12 determines to partition a CU into multiple TUs, and the direction of the intra prediction mode determined by the intra prediction mode determination unit 11 is from the bottom left to the top right (i.e., when directional prediction is performed from the bottom left to the top right), the coding order control unit 13 may be configured to adopt a predefined coding order for the coding of the TUs in the CU, such as the coding order of the bottom left TU in the CU → the bottom right TU in the CU → the top left TU in the CU → the top right TU in the CU, or the coding order of the bottom left TU in the CU → the top left TU in the CU → the bottom right TU in the CU → the top right TU in the CU, instead of the conventional raster scan order.

The sequential local decoded image generation unit 14 is configured to generate local decoded images (decoded images for each TU) based on the coding order and the division method of the CU into TUs determined by the coding order control unit 13.

Specifically, when the TU partition determination unit 12 determines that a CU is to be partitioned into multiple TUs, the sequential local decoded image generation unit 14 is configured to sequentially generate local decoded images in accordance with the coding order determined by the coding order control unit 13.

As shown in FIG. 1, the sequential local decoded image generation unit 14 includes an intra prediction unit 14a, a residual signal generation unit 14b, an orthogonal transform and quantization unit 14c, an inverse quantization unit and an inverse orthogonal transform unit 14d, and a local decoded image generation unit 14e.

The intra prediction unit 14a is configured to generate a predicted image using the intra prediction mode determined by the intra prediction mode determination unit 11. That is, the intra prediction unit 14a is configured to determine the positions of reference pixels to be used when generating a predicted image according to the intra prediction mode, and to generate a predicted image using the reference pixels.

Furthermore, the intra prediction unit 14a may be configured to generate a predicted image in the coding order determined by the coding order control unit 13.

The residual signal generating unit 14b is configured to generate a residual signal based on the difference between the predicted image generated by the intra prediction unit 14a and the original image.

The orthogonal transform and quantization unit 14c is configured to perform orthogonal transform processing and quantization processing on the residual signal generated by the residual signal generation unit 14b, and generate quantized transform coefficients.

As shown in FIG. 2, the orthogonal transform and quantization unit 14c includes an orthogonal transform unit 14c1, a quadratic orthogonal transform unit 14c2, and a quantization unit 14c3.

The orthogonal transform unit 14c1 is configured to perform orthogonal transform processing on the residual signal generated by the residual signal generating unit 14b.

Specifically, when the positions of the reference pixels used to generate the predicted image include at least one of the right and the lower side (when the predicted image is generated using the reference pixels adjacent to at least one of the right and the lower side), the orthogonal transform unit 14c1 is configured to invert the residual signal generated by the residual signal generation unit 14b in at least one of the horizontal and vertical directions and then perform orthogonal transform processing to obtain orthogonal transform coefficients.

For example, the orthogonal transform unit 14c1 may be configured to vertically invert the residual signal before performing the orthogonal transform process when the reference pixel position includes the lower side (when a predicted image is generated using adjacent reference pixels on the lower side).

Alternatively, the orthogonal transform unit 14c1 may be configured to horizontally invert the residual signal and then perform orthogonal transform processing when the position of the reference pixel includes the right side (when a predicted image is generated using adjacent reference pixels on the right side).

The quadratic orthogonal transform unit 14c2 is configured to select a quadratic orthogonal transform process to be applied from a predefined group of quadratic orthogonal transforms according to the intra prediction mode and the position of the reference pixel, and to apply the selected quadratic orthogonal transform process to the signal (orthogonal transform coefficient) output from the orthogonal transform unit 14c1.

Figure 3 shows an example of an intra prediction mode used in this embodiment. As shown in Figure 3, in this embodiment, intra prediction modes 2 to 9 are classified into category A, intra prediction modes 10 to 26 are classified into category B, and intra prediction modes 27 to 34 are classified into category C.

In this embodiment, an example using the intra prediction mode in HEVC shown in FIG. 3 will be described, but the present invention is also applicable to examples using other intra prediction modes.

Here, the secondary orthogonal transform process is a transform process applied to the orthogonal transform coefficients obtained by applying the orthogonal transform process to the residual signal in order to further reduce the entropy.

The energy distribution of the residual signal is statistically proportional to the distance from the reference pixels used in intra prediction. For this reason, the energy distribution of the residual signal differs between an intra prediction mode that uses only reference pixels located on the left side and an intra prediction mode that uses reference pixels located on the left side and above. In addition, the energy distribution of the orthogonal transform coefficients obtained by applying orthogonal transform processing to these residual signals also differs depending on the intra prediction mode.

Therefore, the technology described in Non-Patent Document 2 is configured to switch between selectable secondary orthogonal transform processing groups depending on the direction of the intra prediction mode by utilizing the correlation between the energy bias of the orthogonal transform coefficients and the direction of the intra prediction mode.

Figures 4(a) and 4(b) show, as examples of residual signal energy distribution, the difference in residual signal energy distribution between intra prediction mode 2 (an intra prediction mode that uses only reference pixels located on the left side) and intra prediction mode 18 (an intra prediction mode that uses reference pixels located on the left side and above) in HEVC.

However, when directional prediction of intra prediction mode 2 is performed using reference pixels located on the left and bottom, as in Figure 5, the energy distribution of the residual signal is different compared to when directional prediction of intra prediction mode 2 is performed using only reference pixels located on the left, due to the different positions of the reference pixels.

The energy distribution of the residual signal by directional prediction of intra prediction mode 2 using reference pixels located on the left and bottom (see FIG. 5) is the same as the energy distribution of the residual signal by directional prediction of intra prediction mode 18 using reference pixels located on the left and top shown in FIG. 4(b) flipped vertically.

In other words, when performing directional prediction in intra prediction mode 2, the energy distribution of the orthogonal transform coefficients obtained when the reference pixel positions include the left and bottom sides and the residual signal is flipped vertically and then orthogonal transform processing is applied will be similar to the energy distribution of the orthogonal transform coefficients obtained when performing directional prediction in intra prediction mode 18, the reference pixel positions include the left and top sides and the residual signal is not flipped horizontally or vertically and then orthogonal transform processing is applied (see Figures 4(b) and 5).

The energy distribution of the orthogonal transform coefficients obtained by orthogonal transform processing on the residual signal by directional prediction of intra prediction mode 2 differs depending on the position of the reference pixel used for intra prediction. Therefore, determining the applicable quadratic orthogonal transform processing group based only on the intra prediction mode may increase entropy and degrade coding performance.

The secondary orthogonal transform unit 14c2 is therefore configured to use a group of secondary orthogonal transform processes that are predefined according to a direction obtained by inverting the direction of the intra prediction mode in at least one of the vertical and horizontal directions depending on the position of the reference pixel, rather than the direction of the intra prediction mode.

In other words, when the intra prediction mode belongs to category B, the secondary orthogonal transform unit 14c2 may be configured to select a secondary orthogonal transform process to be applied from a group of secondary orthogonal transforms that are predefined according to the direction of the intra prediction mode.

For example, when the intra prediction mode is 18, the secondary orthogonal transform unit 14c2 may be configured to select a secondary orthogonal transform process to apply from a group of secondary orthogonal transforms that are predefined according to the direction of the intra prediction mode 18.

Alternatively, the secondary orthogonal transform unit 14c2 may be configured to select a secondary orthogonal transform process to be applied from a group of secondary orthogonal transforms that are predefined according to the direction of the intra prediction mode when the intra prediction mode belongs to category A and the position of the reference pixel does not include the lower side.

For example, when the intra prediction mode is 2 and the positions of the reference pixels do not include the lower side, the secondary orthogonal transform unit 14c2 may be configured to select a secondary orthogonal transform process to apply from a group of secondary orthogonal transforms that are predefined according to the direction of the intra prediction mode 2.

Alternatively, the quadratic orthogonal transform unit 14c2 may be configured to select a quadratic orthogonal transform process to be applied from a group of quadratic orthogonal transforms that are predefined according to the direction of the intra prediction mode when the intra prediction mode belongs to category C and the position of the reference pixel does not include the right side.

For example, when the intra prediction mode is 34 and the position of the reference pixel does not include the right side, the quadratic orthogonal transform unit 14c2 may be configured to select a quadratic orthogonal transform process to apply from a group of quadratic orthogonal transforms that are predefined according to the direction of the intra prediction mode 34.

The quadratic orthogonal transform unit 14c2 may also be configured to select a quadratic orthogonal transform process to be applied from a group of quadratic orthogonal transforms that are predefined according to the direction of the intra prediction mode that is vertically inverted when the position of the reference pixel includes the lower side.

Here, "vertically flipping the direction of the intra prediction mode" means, in the example of Figure 3, converting between the direction of intra prediction modes 2 to 9 and the direction of intra prediction modes 18 to 11, respectively. In other words, it means converting the direction of each intra prediction mode to the direction of an intra prediction mode that is in a line-symmetric positional relationship with the direction of intra prediction mode 10 as a reference.

In other words, when the intra prediction mode belongs to category A and the position of the reference pixel includes the lower side, the secondary orthogonal transform unit 14c2 may be configured to select a secondary orthogonal transform process to be applied from a group of secondary orthogonal transforms that are predefined according to the direction obtained by vertically flipping the direction of the intra prediction mode.

For example, when the intra prediction mode is 2 and the reference pixel positions include the lower side, the secondary orthogonal transform unit 14c2 may be configured to select a secondary orthogonal transform process to be applied from a group of secondary orthogonal transforms that are predefined according to the direction obtained by vertically inverting the direction of intra prediction mode 2 (the direction of intra prediction mode 18).

Alternatively, when the position of the reference pixel includes the right side, the secondary orthogonal transform unit 14c2 may be configured to select a secondary orthogonal transform process to be applied from a group of secondary orthogonal transforms that are predefined according to the direction obtained by horizontally inverting the direction of the intra prediction mode.

Here, "horizontally flipping the direction of the intra prediction mode" means, in the example of Figure 3, converting between the direction of intra prediction modes 18 to 25 and the direction of intra prediction modes 34 to 27, respectively. In other words, it means converting the direction of each intra prediction mode to the direction of an intra prediction mode that is in a line-symmetric positional relationship with respect to the direction of intra prediction mode 26.

In other words, when the intra prediction mode belongs to category C and the position of the reference pixel includes the right side, the secondary orthogonal transform unit 14c2 may be configured to select a secondary orthogonal transform process to be applied from a group of secondary orthogonal transforms that are predefined according to the direction obtained by horizontally flipping the direction of the intra prediction mode.

For example, when the intra prediction mode is 34 and the position of the reference pixel includes the right side, the quadratic orthogonal transform unit 14c2 may be configured to select a quadratic orthogonal transform process to be applied from a group of quadratic orthogonal transforms that are predefined according to the direction obtained by horizontally inverting the direction of the intra prediction mode 34 (the direction of the intra prediction mode 18).

The quantization unit 14c3 is configured to perform a quantization process on the signal output from the quadratic orthogonal transformation unit 14c2 and generate quantized transformation coefficients.

The inverse quantization unit and inverse orthogonal transform unit 14d is configured to again perform inverse quantization processing, secondary inverse orthogonal transform processing, and inverse orthogonal transform processing on the quantized transform coefficients generated by the orthogonal transform and quantization unit 14c to generate a residual signal.

The local decoded image generating unit 14e is configured to generate a local decoded image by adding the predicted image generated by the intra prediction unit 14a to the residual signal generated by the inverse quantization unit/inverse orthogonal transformation unit 14d.

The memory 15 is configured to hold the local decoded image generated by the sequential local decoded image generating unit 14 so that it can be used as a reference image.

The entropy coding unit 16 is configured to perform entropy coding processing on flag information including the intra prediction mode determined by the intra prediction mode determination unit 11 and quantized transform coefficients, and output the result as a stream.

Figure 6 shows a flowchart for explaining an example of the operation of the encoding device 1 according to this embodiment.

As shown in FIG. 6, in step S101, the encoding device 1 determines the positions of reference pixels to be used when generating a predicted image according to the determined intra prediction mode, and generates a predicted image using the reference pixels.

In step S102, the encoding device 1 generates a residual signal based on the difference between the predicted image and the original image.

In step S103, if the positions of the reference pixels used to generate the predicted image include at least one of the right and bottom sides, the encoding device 1 inverts the residual signal in at least one of the horizontal and vertical directions and then performs orthogonal transform processing.

In step S104, the encoding device 1 selects a quadratic orthogonal transform process to be applied from a predefined group of quadratic orthogonal transforms according to the intra prediction mode and the position of the reference pixel, and applies the selected quadratic orthogonal transform process to the orthogonal transform coefficients.

In step S105, the encoding device 1 performs a quantization process on the signal that has been subjected to the quadratic orthogonal transform process, and generates quantized transform coefficients.

In step S106, the encoding device 1 performs entropy encoding processing on flag information including intra-prediction mode and the quantized transform coefficients, and outputs the result as a stream.

The decoding device 3 according to this embodiment is configured to divide original images in units of frames that make up a moving image into CUs and decode them. Similarly to the encoding device 1 according to this embodiment, the decoding device 3 according to this embodiment is configured to be able to divide a CU into multiple TUs.

As shown in FIG. 7, the decoding device 3 according to this embodiment includes an entropy decoding unit 31, a decoding order control unit 32, a sequentially decoded image generating unit 33, and a memory 34.

The entropy decoding unit 31 is configured to decode the transform coefficients, flag information, etc. from the stream output from the encoding device 1 by performing an entropy decoding process on the stream output from the encoding device 1. Here, the transform coefficients are quantized transform coefficients obtained as a signal encoded by the encoding device 1 by dividing the original image in frame units into CUs.

The decoding order control unit 32 is configured to determine the decoding order of the TUs within the CU based on the intra prediction mode.

Specifically, the decoding order control unit 32 is configured to determine the decoding order of the TUs in the CU according to a flag indicating whether or not TU division has been performed (whether or not the CU has been divided into multiple TUs) output by the entropy decoding unit 31 and the direction of the intra prediction mode.

For example, like the coding order control unit 13, when a CU is divided into multiple TUs and the direction of the intra prediction mode is from the bottom left to the top right, the decoding order control unit 32 may be configured to perform the decoding process in a predefined decoding order, such as the decoding order of the bottom left TU in the CU → the bottom right TU in the CU → the top left TU in the CU → the top right TU in the CU, or the decoding order of the bottom left TU in the CU → the top left TU in the CU → the bottom right TU in the CU → the top right TU in the CU.

The sequential decoded image generation unit 33 is configured to generate a decoded image (a decoded image for each TU) based on the decoding order and the division method of the CU into TUs determined by the decoding order control unit 32.

Specifically, when a CU is divided into multiple TUs, the sequential decoded image generation unit 33 is configured to generate a decoded image by sequentially performing inverse quantization processing, inverse orthogonal transform processing, and intra prediction on the quantized transform coefficients output by the entropy decoding unit 31 in accordance with the decoding order determined by the decoding order control unit 32.

As shown in FIG. 7, the sequential decoded image generation unit 33 includes an intra prediction unit 33a, an inverse quantization and inverse transform unit 33b, and a decoded image generation unit 33c.

The intra prediction unit 33a may be configured to generate a predicted image using the intra prediction mode output by the entropy decoding unit 31 in accordance with the decoding order determined by the decoding order control unit 32.

The inverse quantization and inverse transform unit 33b is configured to generate a residual signal by performing inverse quantization processing and inverse transform processing (e.g., inverse orthogonal transform processing) on the quantized transform coefficients output by the entropy decoding unit 31.

As shown in FIG. 8, the inverse quantization and inverse transform unit 33b includes an inverse quantization unit 33b1, a secondary inverse orthogonal transform unit 33b2, and an inverse orthogonal transform unit 33b3.

The inverse quantization unit 33b1 is configured to perform inverse quantization processing on the quantized transform coefficients output by the entropy decoding unit 31.

The secondary inverse orthogonal transform unit 33b2 is configured to perform secondary inverse orthogonal transform processing on the signal (transformation coefficient) output from the inverse quantization unit 33b1.

Specifically, the secondary inverse orthogonal transform unit 33b2 is configured to select a secondary inverse orthogonal transform process to be applied from a predefined group of secondary inverse orthogonal transforms according to the intra prediction mode and the positions of the reference pixels used to generate the predicted image, similar to the secondary inverse orthogonal transform unit 14c2, and to perform the selected secondary inverse orthogonal transform process on the signal output from the inverse quantization unit 33b1.

In other words, when the intra prediction mode belongs to category B, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction of the intra prediction mode.

For example, when the intra prediction mode is 18, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction of the intra prediction mode 18.

Alternatively, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction of the intra prediction mode when the intra prediction mode belongs to category A and the positions of the reference pixels do not include the lower side.

For example, when the intra prediction mode is 2 and the positions of the reference pixels do not include the lower side, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction of the intra prediction mode 2.

Alternatively, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction of the intra prediction mode when the intra prediction mode belongs to category C and the position of the reference pixel does not include the right side.

For example, when the intra prediction mode is 34 and the position of the reference pixel does not include the right side, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to apply from a group of secondary inverse orthogonal transforms that are predefined according to the direction of the intra prediction mode 34.

The secondary inverse orthogonal transform unit 33b2 may also be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction of the intra prediction mode that is vertically inverted when the position of the reference pixel includes the lower side.

In other words, when the intra prediction mode belongs to category A and the position of the reference pixel includes the lower side, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction obtained by vertically inverting the direction of the intra prediction mode.

For example, when the intra prediction mode is 2 and the reference pixel positions include the lower side, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction obtained by vertically inverting the direction of intra prediction mode 2 (the direction of intra prediction mode 18).

Alternatively, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction of the intra prediction mode that is horizontally inverted when the position of the reference pixel includes the right side.

In other words, when the intra prediction mode belongs to category C and the position of the reference pixel includes the right side, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction obtained by horizontally flipping the direction of the intra prediction mode.

For example, when the intra prediction mode is 34 and the position of the reference pixel includes the right side, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction obtained by horizontally inverting the direction of the intra prediction mode 34 (the direction of the intra prediction mode 18).

The inverse orthogonal transform unit 33b3 is configured to perform inverse orthogonal transform processing on the signal output from the secondary inverse orthogonal transform unit 33b2.

Specifically, when the position of the reference pixel includes at least one of the right side and the lower side, the inverse orthogonal transform unit 33b3 is configured to invert the signal output from the secondary inverse orthogonal transform unit 33b2 in at least one of the horizontal and vertical directions and then perform an inverse orthogonal transform process.

For example, when the position of the reference pixel includes the lower side, the inverse orthogonal transform unit 33b3 may be configured to vertically invert the signal output from the secondary inverse orthogonal transform unit 33b2 and then perform the inverse orthogonal transform process.

Alternatively, the inverse orthogonal transform unit 33b3 may be configured to horizontally invert the signal output from the secondary inverse orthogonal transform unit 33b2 and then perform inverse orthogonal transform processing when the position of the reference pixel includes the right side.

The decoded image generating unit 33c is configured to generate a decoded image by adding the predicted image generated by the intra prediction unit 33a and the residual signal generated by the inverse quantization and inverse transform unit 33b.

The memory 34 is configured to hold the decoded image generated by the sequential decoded image generating unit 33 so that it can be used as a reference image for intra prediction and inter prediction.

Figure 9 shows a flowchart for explaining an example of the operation of the decoding device 3 according to this embodiment.

As shown in FIG. 9, in step S201, the decoding device 3 generates a predicted image using an intra prediction mode.

In step S202, the decoding device 3 performs inverse quantization processing on the quantized transform coefficients.

In step S203, the decoding device 3 selects a secondary inverse orthogonal transform process to be applied from a predefined group of secondary inverse orthogonal transforms according to the intra prediction mode and the positions of the reference pixels used to generate the predicted image, and applies the selected secondary inverse orthogonal transform process to the signal that has been subjected to the inverse quantization process.

In step S204, if the position of the reference pixel includes at least one of the right side and the lower side, the decoding device 3 inverts the signal that has been subjected to the secondary inverse orthogonal transform processing in at least one of the horizontal and vertical directions, and then performs an inverse orthogonal transform processing.

The encoding device 1 and decoding device 3 according to this embodiment make it possible to switch between different quadratic orthogonal transform processing groups depending on the position of the reference pixel for the orthogonal transform coefficients obtained by performing orthogonal transform processing on the residual signal obtained by intra prediction, thereby reducing entropy and improving encoding efficiency.

Second Embodiment
The encoding device 1 and the decoding device 3 according to the second embodiment of the present invention will be described below, focusing on the differences from the encoding device 1 and the decoding device 3 according to the above-mentioned first embodiment.

In the encoding device 1 according to this embodiment, when the positions of the reference pixels used to generate the predicted image include at least one of the right and lower sides, instead of inverting the residual signal generated by the residual signal generating unit 14b in at least one of the horizontal and vertical directions, the orthogonal transform unit 14c1 is configured to invert at least one of the horizontal and vertical bases of the residual signal generated by the residual signal generating unit 14b and then perform orthogonal transform processing on the residual signal.

For example, when the position of the reference pixel includes the lower side, the orthogonal transform unit 14c1 may be configured to invert the vertical basis of the residual signal and then perform orthogonal transform processing.

Alternatively, when the position of the reference pixel includes the right side, the orthogonal transform unit 14c1 may be configured to invert the horizontal basis of the residual signal and then perform orthogonal transform processing.

In the encoding device 1 according to this embodiment, the quadratic orthogonal transform unit 14c2 is configured to select a quadratic orthogonal transform process to be applied from a predefined group of quadratic orthogonal transforms according to the intra prediction mode and the position of the reference pixel, and to apply the selected quadratic orthogonal transform process to the signal (orthogonal transform coefficient) output from the orthogonal transform unit 14c1.

For example, when the position of the reference pixel includes the lower side, the secondary orthogonal transform unit 14c2 may be configured to select a secondary orthogonal transform process to be applied from a group of secondary orthogonal transforms that are predefined according to the direction of the intra prediction mode that is vertically inverted.

Alternatively, when the position of the reference pixel includes the right side, the quadratic orthogonal transform unit 14c2 may be configured to select a quadratic orthogonal transform process to be applied from a group of quadratic orthogonal transforms that are predefined according to the direction obtained by horizontally inverting the direction of the intra prediction mode.

In addition, in the decoding device 3 according to this embodiment, the secondary inverse orthogonal transform unit 33b2 is configured to select a secondary inverse orthogonal transform process to be applied from a predefined group of secondary inverse orthogonal transforms according to the intra prediction mode and the positions of the reference pixels used to generate the predicted image, and to apply the selected secondary inverse orthogonal transform process to the signal output from the inverse quantization unit.

For example, when the position of the reference pixel includes the lower side, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction of the intra prediction mode that is vertically inverted.

Alternatively, the secondary inverse orthogonal transform unit 33b2 may be configured to select a secondary inverse orthogonal transform process to be applied from a group of secondary inverse orthogonal transforms that are predefined according to the direction of the intra prediction mode that is horizontally inverted when the position of the reference pixel includes the right side.

In the decoding device 3 according to this embodiment, when the position of the reference pixel includes at least one of the right side and the lower side, instead of inverting the signal output from the secondary inverse orthogonal transform unit 33b2 in at least one of the horizontal and vertical directions, the inverse orthogonal transform unit 33b3 is configured to invert at least one of the horizontal and vertical bases of the signal output from the secondary inverse orthogonal transform unit 33b2 and then perform an inverse orthogonal transform process on the signal output from the secondary inverse orthogonal transform unit 33b2.

For example, when the position of the reference pixel includes the lower side, the inverse orthogonal transform unit 33b3 may be configured to invert the vertical base of the signal output from the secondary inverse orthogonal transform unit 33b2 and then perform an inverse orthogonal transform process.

Alternatively, the inverse orthogonal transform unit 33b3 may be configured to invert the horizontal base of the signal output from the secondary inverse orthogonal transform unit 33b2 and then perform an inverse orthogonal transform process when the position of the reference pixel includes the right side.

Other Embodiments
As described above, the present invention has been described by the above-mentioned embodiment, but the description and drawings forming a part of the disclosure of such embodiment should not be understood as limiting the present invention. From such disclosure, various alternative embodiments, examples and operating techniques will become apparent to those skilled in the art.

Although not specifically mentioned in the above embodiment, a program may be provided that causes a computer to execute each process performed by the encoding device 1 and the decoding device 3 described above. Such a program may be recorded on a computer-readable medium. By using a computer-readable medium, such a program can be installed on a computer. Here, the computer-readable medium on which such a program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

Alternatively, a chip may be provided that is configured with a memory that stores a program for implementing at least some of the functions of the encoding device 1 and the decoding device 3 described above, and a processor that executes the program stored in the memory.

1...Encoding device 11...Intra prediction mode determination unit 12...TU partition determination unit 13...Encoding order control unit 14...Sequential local decoded image generation unit 14a...Intra prediction unit 14b...Residual signal generation unit 14c...Orthogonal transformation/quantization unit 14c1...Orthogonal transformation unit 14c2...Secondary orthogonal transformation unit 14c3...Quantization unit 14d...Inverse quantization unit/inverse orthogonal transformation unit 14e...Local decoded image generation unit 15...Memory 16...Entropy encoding unit 3...Decoding device 31...Entropy decoding unit 32...Decoding order control unit 33...Sequential local decoded image generation unit 33a...Intra prediction unit 33b...Inverse quantization/inverse transformation unit 33b1...Inverse quantization unit 33b2...Secondary inverse orthogonal transformation unit 33b3...Inverse orthogonal transformation unit 33c...Decoded image generation unit 34...Memory

Claims (4)

An encoding device that divides an original image in units of frames constituting a moving image into blocks and encodes the divided original images,
an intra prediction unit that generates a predicted image using an intra prediction mode that indicates a type of intra prediction processing;
a residual signal generation unit that generates a residual signal based on a difference between the predicted image generated by the intra prediction unit and the original image;
a conversion unit that performs a conversion process on the residual signal generated by the residual signal generation unit;
a secondary conversion unit that switches between different secondary conversion processes for the signal output from the conversion unit according to a position of a reference pixel used for generating the predicted image and the intra prediction mode;
An encoding device comprising: A decoding device that decodes block units obtained by dividing an original image in units of frames constituting a moving image, comprising:
an intra prediction unit that generates a predicted image using an intra prediction mode that indicates a type of intra prediction processing;
an inverse quantization unit that performs an inverse quantization process on the quantized transform coefficients;
a secondary inverse transform unit that switches between different secondary inverse transform processes for the signal output from the inverse quantization unit according to the intra prediction mode and the positions of reference pixels used to generate the predicted image;
A decoding device comprising: A program for causing a computer to function as the encoding device described in claim 1. A program for causing a computer to function as the decoding device described in claim 2.

Images