JP7184984B2 - Image decoding device, image decoding method and program - Google Patents

Description

The present invention relates to an image decoding device, an image decoding method and a program.

Conventionally, techniques called RST (Reduced Secondary Transform) (see Non-Patent Document 2) and LFNST (Low-Frequency Non-Separable Transform) (see Non-Patent Document 1) for performing secondary transform on coefficients after primary transform are known. It is

Versatile Video Coding (Draft 5), JVET-N1001 CE6: Reduced Secondary Transform (RST) (CE6-3.1), JVET-N0193 Algorithm description for Versatile Video Coding and Test Model 5 (VTM 5), JVET-N1002

However, the conventional technique described above includes a plurality of types of counters that count the number of non-zero coefficients when decoding coefficients, and determines whether or not secondary transformation is applied based on the counter values. Therefore, it is necessary to add a coefficient counting process just to determine whether or not the secondary transformation is applied. The coefficient decoding process is a process that requires a high throughput, but there is a problem that the additional process increases the processing load.

Therefore, the present invention has been made in view of the above problems, and an image decoding apparatus, an image decoding method, and an image decoding apparatus that can omit additional processing for determining whether or not secondary transformation is applied and that can be expected to speed up processing. The purpose is to provide a program.

A first feature of the present invention is an image decoding device comprising an inverse transform unit configured to generate a prediction residual signal by inverse transform, wherein the inverse transform unit performs secondary transform on a target block is applied or not, and the transform coefficient decoding method is controlled according to the determination result.

A second feature of the present invention is an image decoding device, comprising an inverse transform unit configured to generate a prediction residual signal by inverse transform, the inverse transform unit, depending on the size of a target block, The gist is that the decoding method of the secondary transform index of the target block is controlled.

A third feature of the present invention is an image decoding device comprising an inverse transform unit configured to generate a prediction residual signal by inverse transform, wherein the inverse transform unit includes non-zero coefficients of a target block The gist of the invention is that the decoding method of the secondary transform index of the target block is controlled based on the information indicating the position of occurrence of the .

In the third feature of the present invention, the information indicating the position where the non-zero coefficient of the target block occurs is a coefficient that the non-zero coefficient cannot occur when the secondary transform is applied to the target block. When pointing to a position, the inverse transform unit may be configured not to decode the secondary transform index of the current block.

In the third feature of the present invention, when the information indicating the occurrence position of the non-zero coefficient of the target block indicates the coefficient position of the DC component of the target block, the inverse transform unit It may be configured not to decode the secondary transform index.

A fourth feature of the present invention is an image decoding device, comprising an inverse transform unit configured to generate a prediction residual signal by inverse transform, wherein the inverse transform unit generates nonzero The gist of the invention is that a decoding method of the secondary transform index of the target block is controlled by a flag indicating whether or not a coefficient is generated.

A fifth feature of the present invention is an image decoding device comprising an inverse transform unit configured to generate a prediction residual signal by inverse transform, wherein the inverse transform unit is a sub-block in a target block The gist of the invention is that a decoding method of the secondary transform index of the target block is controlled by a flag indicating whether or not a non-zero coefficient is generated.

In the fifth feature of the present invention, the flag indicating whether or not the non-zero coefficient in the sub-block is generated indicates that the non-zero coefficient is generated when the secondary transform is applied to the target block. The inverse transform unit may be configured not to decode the secondary transform index of the current block if it indicates that the non-zero coefficient occurs in a non-zero sub-block.

A sixth feature of the present invention is an image decoding method, comprising the step of generating a prediction residual signal by inverse transform, in the step, determining whether or not secondary transform is applied to a target block, and The gist is to control the decoding method of the transform coefficients according to the result.

A seventh feature of the present invention is a program for causing a computer to function as an image decoding device, the image decoding device comprising an inverse transform unit configured to generate a prediction residual signal by inverse transform. The inverse transform unit determines whether or not a secondary transform is applied to a target block, and controls a transform coefficient decoding method according to the determination result.

Advantageous Effects of Invention According to the present invention, it is possible to provide an image decoding device, an image decoding method, and a program that can omit additional processing for determining whether or not secondary transformation is applied, and that can be expected to speed up processing.

It is a figure showing an example of composition of image processing system 1 concerning one embodiment. 1 is a diagram showing an example of functional blocks of an image encoding device 100 according to an embodiment; FIG. 3 is a diagram showing an example of functional blocks of a transform/quantization unit 131 of the image encoding device 100 according to one embodiment; FIG. 2 is a diagram showing an example of functional blocks of an image decoding device 200 according to one embodiment; FIG. 3 is a diagram showing an example of functional blocks of an inverse transform/inverse quantization unit 220 of the image decoding device 200 according to one embodiment; FIG. FIG. 4 is a diagram for explaining an example of functions of an inverse transform/inverse quantization unit 220 of the image decoding device 200 according to one embodiment; FIG. 4 is a diagram for explaining an example of functions of an inverse transform/inverse quantization unit 220 of the image decoding device 200 according to one embodiment; 4 is a flowchart showing an example of a method for determining whether or not to apply a secondary transform and a method for decoding coefficients in the image decoding device 200 according to the first embodiment. 4 is a flowchart showing an example of a method for determining whether or not to apply a secondary transform and a method for decoding coefficients in the image decoding device 200 according to the first embodiment. 9 is a flowchart showing an example of a method for determining whether or not to apply secondary transform and a method for decoding coefficients in the image decoding device 200 according to Modification 1 of the first embodiment. It is a figure for demonstrating 2nd Embodiment. 14 is a flow chart showing an example of a method for determining whether or not to apply secondary transform and a method for decoding coefficients in the image decoding device 200 according to the third embodiment. 14 is a flow chart showing an example of a method for determining whether or not to apply secondary transform and a method for decoding coefficients in the image decoding device 200 according to the third embodiment.

<First embodiment>
An image processing system 10 according to a first embodiment of the present invention will be described below with reference to FIGS. 1 to 9. FIG. FIG. 1 is a diagram showing an image processing system 10 according to this embodiment.

As shown in FIG. 1, an image processing system 10 according to this embodiment has an image encoding device 100 and an image decoding device 200 .

The image encoding device 100 is configured to generate encoded data by encoding an input image signal. The image decoding device 200 is configured to generate an output image signal by decoding encoded data.

The encoded data may be transmitted from the image encoding device 100 to the image decoding device 200 via a transmission line. The encoded data may be stored in a storage medium and then provided from the image encoding device 100 to the image decoding device 200 .

(Image encoding device 100)
The image coding apparatus 100 according to this embodiment will be described below with reference to FIG. FIG. 2 is a diagram showing an example of functional blocks of the image encoding device 100 according to this embodiment.

As shown in FIG. 2 , the image coding apparatus 100 includes an inter prediction unit 111, an intra prediction unit 112, a subtractor 121, an adder 122, a transform/quantization unit 131, an inverse transform/inverse quantization It has a section 132 , an encoding section 140 , an in-loop filtering section 150 and a frame buffer 160 .

The inter prediction unit 111 is configured to generate a prediction signal by inter prediction (inter-frame prediction).

Specifically, the inter prediction unit 111 identifies a reference block included in the reference frame by comparing a frame to be encoded (hereinafter referred to as a target frame) and a reference frame stored in the frame buffer 160, and identifies a reference block included in the reference frame. It is configured to determine a predicted motion vector for the reference block.

Also, the inter prediction unit 111 is configured to generate, for each prediction block, a prediction signal included in the prediction block based on the reference block and motion vector. The inter prediction section 111 is configured to output the prediction signal to the subtractor 121 and the adder 122 . Here, the reference frame is a frame different from the target frame.

The intra prediction unit 112 is configured to generate a prediction signal by intra prediction (intra-frame prediction).

Specifically, the intra prediction unit 112 is configured to identify a reference block included in the target frame and generate a prediction signal for each prediction block based on the identified reference block. Also, the intra prediction unit 112 is configured to output the prediction signal to the subtractor 121 and the adder 122 .

Here, the reference block is a block that is referred to for a prediction target block (hereinafter referred to as target block). For example, the reference block is a block adjacent to the target block.

The subtractor 121 is configured to subtract the prediction signal from the input image signal and output the prediction residual signal to the transform/quantization section 131 . Here, the subtractor 121 is configured to generate a prediction residual signal that is a difference between a prediction signal generated by intra prediction or inter prediction and an input image signal.

The adder 122 adds the prediction signal to the prediction residual signal output from the inverse transform/inverse quantization unit 132 to generate a pre-filtering decoded signal. It is configured to output to the loop filter processing unit 150 .

Here, the unfiltered decoded signal constitutes a reference block used in intra prediction section 112 .

The transform/quantization unit 131 is configured to perform transform processing on the prediction residual signal and to obtain coefficient level values. Further, the transform/quantization unit 131 may be configured to quantize the coefficient level values.

Here, the transformation/quantization unit 131 is configured to output a quantization index when performing quantization of coefficient level values. Hereinafter, the output of the transform/quantization unit 131 will be described as a coefficient level value regardless of whether or not quantization is applied.

Here, transform processing is processing for transforming a prediction residual signal into a frequency component signal. In such transformation processing, a basis pattern (transformation matrix) corresponding to discrete cosine transform (DCT) may be used, and a basis pattern (transformation matrix) corresponding to discrete sine transform (DST) may be used. may be used.

Note that transform processing may be performed multiple times before quantization is performed. As an example, secondary conversion for performing the second conversion process will be described later.

The inverse transform/inverse quantization unit 132 is configured to perform inverse transform processing on the coefficient level values output from the transform/quantization unit 131 . Here, the inverse transform/inverse quantization unit 132 may be configured to perform inverse quantization of the coefficient level values prior to the inverse transform processing.

Here, the inverse transform processing and inverse quantization are performed in the reverse order of the transform processing and quantization performed by the transform/quantization unit 131 .

The encoding unit 140 is configured to encode the coefficient level values output from the transform/quantization unit 131 and output encoded data.

Here, for example, the encoding is entropy encoding that counts the coefficients of a target block (encoding block or transform block) and assigns codes of different lengths based on the probability of occurrence of coefficient level values. A method of counting coefficients will be described later.

In addition to the coefficient level values, the encoding unit 140 is also configured to encode control data used in the decoding process.

Here, the control data may include size data such as a coding block (CU: Coding Unit) size, a prediction block (PU: Prediction Unit) size, a transform block (TU: Transform Unit) size, and the like.

The in-loop filtering section 150 is configured to perform filtering on the unfiltered decoded signal output from the adder 122 and output the filtered decoded signal to the frame buffer 160 .

Here, for example, the filter processing is deblocking filter processing that reduces distortion that occurs at boundaries of blocks (encoding blocks, prediction blocks, or transform blocks).

The frame buffer 160 is configured to accumulate reference frames used by the inter prediction section 111 .

Here, the filtered decoded signal constitutes a reference frame used in inter prediction section 111 .

(Transformation/quantization unit 131)
The transform/quantization unit 131 of the image coding apparatus 110 according to this embodiment will be described below with reference to FIG. FIG. 3 is a diagram showing an example of functional blocks of the transform/quantization unit 131 of the image encoding device 110 according to this embodiment.

As shown in FIG. 13, the transform/quantization unit 131 has a primary transform unit 131A, a secondary transform unit 131B, and a quantization unit 131C.

The transform/quantization unit 131 is an example of a transform/quantization unit configured to generate coefficient level values from the prediction residual signal by transform/quantization.

The primary transform unit 131A is configured to receive the prediction residual signal as input and generate primary transform coefficients of the target block.

Here, the base pattern (transformation matrix) used for the primary conversion process may be selected from a plurality of base patterns. For example, in Non-Patent Document 1, base patterns corresponding to DCT2, DCT8, and DST7 are used, respectively. As for the method of selecting the base pattern, for example, the image coding device 100 selects the base pattern with the lowest coding cost, and the selected base pattern is transmitted to the image decoding device 200 as side information.

The secondary transform unit 131B is configured to receive the primary transform coefficients from the primary transform unit 131A as input and determine whether or not the secondary transform is applied to the target block.

Here, if the secondary transform unit 131B determines to apply the secondary transform, the secondary transform unit 131B performs transform processing on the primary transform coefficients to generate the secondary transform coefficients, and converts the secondary transform coefficients to the quantization unit 131C. configured to output to

On the other hand, the secondary transform unit 131B is configured to directly output the primary transform coefficients to the quantization unit 131C when it is determined not to apply the secondary transform.

In addition, the secondary transform unit 131B is configured to output information on whether or not secondary transform is applied (hereinafter referred to as a secondary transform index) to the encoding unit 140 .

The quantization unit 131C is configured to receive transform coefficients (primary transform coefficients or secondary transform coefficients) as input and determine whether or not quantization is applied to the transform coefficients of the target block.

Here, the quantization unit 131C is configured to generate a quantization index when it is determined to apply quantization. Note that a method for determining whether or not quantization is applied and a method for generating a quantization index can use known methods, so details thereof will be omitted.

(Encoder 140)
The encoding unit 140 may be configured to encode the coefficient level values using a context optimized for the application probability of the secondary transform when the secondary transform is applied.

(Inverse transform/inverse quantization unit 132)
The inverse transform/inverse quantization unit 132 operates in the same manner as the inverse transform/inverse quantization unit 220 in the image decoding device 200 described later. are treated in the same manner as the inverse transform/inverse quantization unit 220 .

(Image decoding device 200)
The image decoding device 200 according to this embodiment will be described below with reference to FIG. FIG. 4 is a diagram showing an example of functional blocks of the image decoding device 200 according to this embodiment.

As shown in FIG. 4, the image decoding device 200 includes a decoding unit 210, an inverse transform/inverse quantization unit 220, an adder 230, an inter prediction unit 241, an intra prediction unit 242, and an in-loop filtering unit. 250 and a frame buffer 260 .

The decoding unit 210 is configured to decode the encoded data generated by the image encoding device 100 and decode the coefficient level values.

Here, decoding is decoding of entropy-encoded data assigning different length codes based on, for example, the probability of occurrence of the coefficients.
Decoding is entropy decoding in a procedure opposite to the entropy encoding performed by the encoding unit 140 .

Further, the decoding unit 210 may be configured to acquire the control data by decoding the encoded data.

Note that, as described above, the control data may include size data such as the encoding block size, prediction block size, transform block size, and the like.

The inverse transform/inverse quantization unit 220 performs inverse quantization of the quantization index output from the decoding unit 210 and inverse transform processing of the coefficient level value output from the decoding unit 210 to generate a prediction residual signal. is configured to

The adder 230 adds the prediction signal to the prediction residual signal output from the inverse transform/inverse quantization unit 220 to generate a pre-filtering decoded signal. It is configured to output to the filter processing unit 250 .

Here, the unfiltered decoded signal constitutes a reference block used in intra prediction section 242 .

Like the inter prediction section 111, the inter prediction section 241 is configured to generate a prediction signal by inter prediction (inter-frame prediction).

Specifically, the inter prediction unit 241 is configured to generate a prediction signal for each prediction block based on a motion vector decoded from encoded data and a reference signal included in a reference frame. The inter prediction section 241 is configured to output a prediction signal to the adder 230 .

The intra prediction section 242, like the intra prediction section 112, is configured to generate a prediction signal by intra prediction (intra-frame prediction).

Specifically, the intra prediction unit 242 is configured to identify a reference block included in the target frame and generate a prediction signal for each prediction block based on the identified reference block. The intra prediction section 242 is configured to output the prediction signal to the adder 230 .

Similar to in-loop filtering section 150 , in-loop filtering section 250 performs filtering on the unfiltered decoded signal output from adder 230 and outputs the filtered decoded signal to frame buffer 260 . is configured to

Here, for example, the filter processing is deblocking filter processing that reduces distortion occurring at boundaries of blocks (encoding blocks, prediction blocks, transform blocks, or sub-blocks obtained by dividing them).

The frame buffer 260, like the frame buffer 160, is configured to accumulate reference frames used by the inter prediction section 241. FIG.

Here, the decoded signal after filtering constitutes a reference frame used in the inter prediction section 241 .

(Inverse transform/inverse quantization unit 220)
The inverse transform/inverse quantization unit 220 of the image decoding device 200 according to the present embodiment will be described below with reference to FIG. FIG. 5 is a diagram showing an example of functional blocks of the inverse transform/inverse quantization unit 220 of the image decoding device 200 according to this embodiment.

As shown in FIG. 5, the inverse transform/inverse quantization unit 220 has an inverse quantization unit 242A, an inverse secondary transform unit 242B, and an inverse primary transform unit 242C.

The inverse quantization unit 220A is configured to generate the transform coefficients of the target block using the quantization index as an input signal. A well-known method can be used as a method for generating a coefficient level value from a quantization index, so the details are omitted.

The inverse secondary transform unit 220B is configured to receive the coefficient level value as input and determine whether or not to apply the inverse secondary transform to the target block.

Here, when the inverse secondary transform unit 220B determines to apply the inverse secondary transform, the inverse transform processing is performed on the input coefficient level values to generate the primary transform coefficients, and the primary transform coefficients are generated. It is configured to output the coefficients to the primary transform coefficient section 220C.

On the other hand, the inverse secondary transform unit 220B is configured to directly output the input coefficient level value to the primary transform coefficient unit 220C as the primary transform coefficient when it is determined not to apply the secondary transform.

Hereinafter, in order to simplify the description, the secondary transformation and the inverse secondary transformation are generally referred to as the secondary transformation, and the secondary transformation in the inverse secondary transformation unit is appropriately read as the inverse secondary transformation.

A method of determining whether or not to apply the secondary conversion by the inverse secondary conversion unit 220B will be described later.

The inverse primary transform unit 220C is configured to receive a primary transform coefficient as input and generate a prediction residual signal of the target block.

Here, the base pattern (transformation matrix) used for inverse primary transform processing may be selected from a plurality of prescribed patterns, as in the case of the image encoding device 100 . Selection of the base pattern may be performed using, for example, side information transmitted from the image coding device 100 (base pattern information selected by the image coding device 100).

(Description of secondary conversion)
An example of the secondary transform performed by the image encoding device 100 and the image decoding device 200 according to this embodiment will be described below with reference to FIG.

Secondary transform is a technique for transforming the primary transform coefficients of the target block again before quantization, as described above. By transforming the primary transform coefficients again before quantization, coefficients with non-zero coefficient level values (hereinafter referred to as non-zero coefficients) tend to be more densely packed in the low-frequency region. Since coefficients with a value of zero (hereinafter referred to as zero coefficients) are continuous, further improvement of coding effect by entropy coding can be expected.

Here, the secondary transformation may use a plurality of base patterns (transformation matrices) as in the case of the primary transformation. For example, as in Non-Patent Document 2, base patterns that can be used may be set in advance according to the target block size, intra-prediction mode, and secondary transform index. A value of 0 for the secondary transform index indicates that the secondary transform is disabled. On the other hand, if the value is greater than or equal to 1, it indicates that the secondary transformation is valid. In addition, two types of base patterns may be adaptively selected, for example, as in Non-Patent Document 2, using such a secondary conversion index.

Also, the application range of the secondary transform may be adaptively set according to the block size of the target block. For example, FIG. 6 shows an example of the application range of secondary conversion in Non-Patent Document 2. In FIG.

As shown in FIG. 6A, when the block size of the target block is 8×8 pixels, the application range R1 of the secondary transform covers all 64 primary transform coefficients. The later non-zero coefficient generation region R2 is limited to the low-frequency region (4×2 pixels) of the secondary transform coefficients, and all other regions are assumed to be zero coefficients (hereinafter referred to as zeroing).

On the other hand, as shown in FIG. 6B, when the block size of the target block is M×N (however, M>8, N>8) pixels, the application range R1 of the secondary transform is 48 pixels. These are primary transform coefficients in the frequency domain. That is, secondary transform coefficients are not applied to the 48 primary transform coefficients other than the low-frequency region, and the primary transform coefficients are treated as they are. Also, the area where non-zero coefficients are generated after the secondary transform is limited to the low-frequency area (4×4 pixels) of the secondary transform coefficients, and the other areas are zeroed.

(Coefficient encoding method)
A method of encoding coefficient level values will be described below with reference to FIG. FIG. 7 is a diagram showing a scanning method of coefficient level values of a target block.

As a coefficient scanning method, for example, as described in Non-Patent Document 1, there is a method of scanning in the upper left direction from the coefficient C1 of the highest frequency component of the target block to the coefficient Cx of the lowest frequency component (hereinafter referred to as diagonal scanning). may be used (see FIG. 7).

In natural images, non-zero coefficients are less likely to occur in high-frequency regions. Therefore, by diagonally scanning from high frequency components to low frequency components, zero coefficients can be arranged continuously, and the effect of improving the encoding performance by entropy encoding can be expected.

Further, the oblique scanning may be performed in units of sub-blocks obtained by dividing the target block into 4×4 pixels, as in Non-Patent Document 1. In addition, the image coding apparatus 100 transmits a flag (coded_sub_block_flag) indicating the presence or absence of nonzero coefficients to the image decoding apparatus 200 according to the number of nonzero coefficients generated in units of subblocks. You may

Here, the reason for using such a flag (coded_sub_block_flag) is that, for example, a 1-bit flag (coded_sub_block_flag) is used to encode and The point is that it can be decrypted.

In addition, before encoding the flag (coded_sub_block_flag), the image coding apparatus 100 determines whether non-zero coefficients are generated from the sub-block in which non-zero coefficients are generated first in the scan order. may be performed.

Here, the image encoding device 100 may transmit the position information (last_coeff_pos) where the first non-zero coefficient occurs to the image decoding device 200 . As a result, since it is certain that the sub-block before the occurrence of the first non-zero coefficient has no non-zero coefficients, the transmission of the above flag (coded_sub_block_flag) becomes unnecessary, and the code amount can save

Note that the image coding apparatus 100 divides the x-coordinate value and y-coordinate value of the position information at which the non-zero coefficient first occurs in the target block into a prefix part and a suffix part, for example, as in Non-Patent Document 1. may be encoded.

(Method for Determining Whether to Apply Secondary Transform and Decoding Method for Coefficients)
8 and 9 are flowcharts showing an example of a method for determining whether or not to apply secondary transform and a method for decoding coefficients in the image decoding device 200. FIG. Hereinafter, an example of a method for determining whether or not to apply secondary transform will be described with reference to FIG. 8, and an example of a coefficient decoding method will be described with reference to FIG.

As shown in FIG. 8, in step S61, the image decoding device 200 decodes a flag (cbf) indicating the presence or absence of non-zero coefficients in the target block. Here, the value of cbf is either 0 or 1.

In step S62, the image decoding device 200 determines whether or not there is a non-zero coefficient based on cbf.

If it is determined that there are no nonzero coefficients (the number of nonzero coefficients is 0) (that is, if it is determined that cbf=0), this process ends.

On the other hand, if it is determined that there is a nonzero coefficient (one or more nonzero coefficients) (that is, if it is determined that cbf=1), the process proceeds to step S63.

Here, cbf may be decoded as separate flags for each of the luminance signal (Y signal) and color difference signals (Cb signal and Cr signal). In this case, the image decoding device 200 may make a determination as follows, depending on whether the block division tree structures of the luminance signal and the color difference signals of the target block are different (Dual Tree) or when they are the same (Single Tree). .

The image decoding apparatus 200 may decide to proceed to step S63 if cbf=1 of the Y signal in the Dual Tree mode, and terminate this process otherwise.

If cbf of at least one of the Cb/Cr signals in Dual Tree is 1, the image decoding apparatus 200 proceeds to step S63, otherwise, determines to end this process. You may

If the cbf of at least one of the Y signal, the Cb signal, and the Cr signal in Single Tree is 1, the image decoding apparatus 200 proceeds to step S63, and if not, determines to end this process. good too.

In step S63, the image decoding device 200 decodes the secondary transform index (lfnst_idx).

In step S64, the image decoding device 200 decodes information (last_coeff_pos) indicating the position where the first non-zero coefficient occurs in the target block in scan order.

In step S65, the image decoding device 200 performs coefficient decoding processing (see FIG. 9). Note that the image decoding device 200 performs decoding processing of the coefficients in units of sub-blocks obtained by dividing the above-described target block.

As shown in FIG. 9, in step S641, the image decoding device 200 determines whether the sub-block in which the first non-zero coefficient occurs is the first or last sub-block in the target block in the scanning order. do.

If it is determined to be either the first or the last sub-block, the process proceeds to step S645, since it is obvious that such sub-block has non-zero coefficients.

If it is determined that the sub-block is neither the first nor the last sub-block, in step S642, the image decoding device 200 determines whether the secondary transform is applied to the sub-block based on the secondary transform index (st_idx). determine whether or not there is

If it is determined that the secondary transform is applied to the sub-block (the secondary transform is effective), and it is obvious that all the coefficients are zero due to the secondary transform in the sub-block. If (zeroed), decoding of coded_sub_block_flag is unnecessary, so the process proceeds to step S646.

On the other hand, if it is determined that the secondary transform is not applied to the sub-block (the secondary transform is not valid), or even if the sub-block is subjected to the secondary transform, non-zero coefficients are generated. If the possibility remains, the process proceeds to step S643.

Note that the order of steps S641 and S642 may be changed.

In step S643, the image decoding device 200 decodes the flag (coded_sub_block_flag) indicating the presence or absence of non-zero coefficients.

In step S644, the image decoding apparatus 200 determines whether or not there is a non-zero coefficient in the sub-block based on coded_sub_block_flag.

When the value of coded_sub_block_flag is 0, the image decoding device 200 determines that there is no non-zero coefficient, and the process proceeds to step S646.

On the other hand, when the value of coded_sub_block_flag is 1, the image decoding device 200 determines that there is a non-zero coefficient, and the process proceeds to step S645.

In step S645, the image decoding device 200 decodes the level value of each coefficient in the sub-block.

In step S646, the image decoding device 200 determines whether or not the sub-block to be processed is the final block.

If it is determined that the block is not the final block, the process advances to step S641 to move to the next sub-block in the scanning order and restart the process. If the block is determined to be the final block, the process proceeds to finish.

The following two effects can be expected by using the above-described inverse secondary transform application/non-application determining method and coefficient decoding method.

The first effect is that the secondary transform index is obtained before the decoding process of the coefficients of the target block, thereby omitting the decoding process of the coefficients, which is necessary for determining whether or not to apply the secondary transform in the conventional technique. It is possible.

Specifically, in the prior art, two counters (a counter for measuring the number of non-zero coefficients generated in the target block and a counter for measuring the number of non-zero coefficients generated in the target block and a However, by introducing the technology according to the present embodiment, both of them can be omitted.

The second effect is that when the secondary transform is effective, the conventional technology redundantly decodes or encodes coded_sub_block_flag for a sub-block in which all coefficients are obviously zero in the secondary transform target block. I had to make it. On the other hand, if the technique according to the present embodiment is used, such processing can be omitted, so that improvement in coding performance can be expected.

<Modification 1>
Modification 1 of the first embodiment described above will be described below with reference to FIG. 10, focusing on differences from the first embodiment described above.

In the flowchart shown in FIG. 10, step S66, which was not present in the flowchart shown in FIG. 8, is added before step S63.

In FIG. 10, such step S66 is added because the transmission of the secondary conversion index becomes redundant when the secondary conversion is clearly not valid.

For example, if the application target of the secondary transform is limited to an intra picture or an intra block, a determination as to whether the target block is an intra picture or an intra block may be added to the predetermined condition.

In addition, when one coding block is divided into a plurality of prediction blocks or transform blocks, the number of transform blocks to which secondary transform is applied increases. Increased processing and decoding processing delays are possible.

Therefore, for example, when intra prediction (intra subdivision prediction) is enabled by dividing the coding block described in Non-Patent Document 1 into a plurality of predetermined conditions, by treating the secondary transformation as disabled, unintended processing An increase in delay can be suppressed. Therefore, a determination as to whether or not intra-subdivision prediction is applied to the target block may be added to the predetermined condition.

<Modification 2>
Modification 2 of the above-described first embodiment will be described below, focusing on differences from the above-described first embodiment and modification 1. FIG.

In the first embodiment described above, as shown in FIG. 8, the example in which the decoding of the secondary transform index is performed for each encoded block has been described.

In Modification 2, when an encoded block is divided into a plurality of transform blocks, the secondary transform index may be decoded for each transform block.

According to such a configuration, the secondary transform index can be decoded in different transform block units, so that the number of blocks to which the secondary transform is applied is increased, and an improvement in encoding performance can be expected.

<Second embodiment>
The second embodiment of the present invention will be described below with reference to FIG. 11, focusing on the differences from the first embodiment described above.

In this embodiment, as shown in FIG. 11, the decoding method of the position information (last_coeff_pos) where the first non-zero coefficient occurs in the target block is controlled depending on whether or not the secondary transform is applied. It is

This embodiment differs from the above-described first embodiment 1 with respect to the last_coeff_pos decoding method (see FIG. 8). Since this embodiment has the same configuration as the above-described first embodiment for other parts, the description is omitted.

For example, in Non-Patent Document 2, when secondary transform is valid for a block and the target block size is 4x4 or 8x8, the position and number of non-zero coefficients for the target area of secondary transform are set to the uppermost left subblock, respectively. And limited to 8 pieces.

At this time, the positions where the non-zero coefficients are generated can be identified as shown in FIG. 11 by considering the scan order of the sub-blocks.

In other words, last_coeff_pos cannot point to scan order 8-15 shown in FIG. 11 if quadratic transformation is enabled.

Therefore, last_coeff_pos is usually decoded by the prefix and suffix of the x-coordinate value and y-coordinate value. If so, the decoding method for last_coeff_pos may be modified as follows.

For example, last_coeff_pos can be transmitted with a 3-bit index in the scan order of the coefficients, ranging from 0 to 7 where non-zero coefficients can occur.

In the prior art, the x-coordinate value and the y-coordinate value are each represented by 2 bits for each subblock, so the code amount can be saved accordingly, and as a result, an effect of improving the coding performance can be expected.

<Third Embodiment>
Hereinafter, the third embodiment of the present invention will be described with reference to FIGS. 12 and 13, focusing on differences from the above-described first and second embodiments.

This embodiment differs from the first embodiment in terms of the method of determining whether or not to apply secondary transform and the method of decoding coefficients (see FIG. 8). Since this embodiment has the same configuration as the above-described first embodiment for other parts, the description is omitted.

12 and 13 are flowcharts showing an example of a method for determining whether or not to apply secondary transform and a method for decoding coefficients in the image decoding device 200 according to the present embodiment. An example of the operation of the image decoding device 200 according to this embodiment will be described below using such a flowchart.

As shown in FIG. 12, in step S61, the image decoding device 200 decodes the flag (cbf) indicating the presence/absence of non-zero coefficients in the target block, similarly to step S61 shown in FIG.

In step S62, the image decoding device 200 determines whether or not there is a non-zero coefficient based on cbf, as in step S62 shown in FIG.

If it is determined that there are no nonzero coefficients (the number of nonzero coefficients is 0) (that is, if it is determined that cbf=0), the process proceeds to step S91.

On the other hand, if it is determined that there are nonzero coefficients (one or more nonzero coefficients) (that is, if it is determined that cbf=1), the process proceeds to step S64.

In step S64, the image decoding device 200 decodes the information (last_coeff_pos) indicating the position of the first non-zero coefficient in the target block in scan order, as in step S64 shown in FIG.

In step S93, the image decoding device 200 performs decoding processing of the coefficients shown in FIG. The coefficient decoding process shown in FIG. 13 is the same as the coefficient decoding process shown in FIG. 9 except that step S642 shown in FIG. 9 is not included.

In step S91, the image decoding device 200 determines whether or not a predetermined condition, which will be described later, is satisfied. If it is determined that the predetermined condition is satisfied, the operation proceeds to step S92, and if it is determined that the predetermined condition is not satisfied, the operation ends.

Here, the predetermined condition may be the presence or absence of decoding of the secondary transform index according to the target block size.

For example, when the image decoding device 200 determines in step S91 that at least one of the width and height of the target encoding block is greater than 64, it determines in step S92 not to decode the secondary transform index. good too. The effects of such determination are as follows.

Non-Patent Document 3 employs a tool for maintaining decoder pipeline processing units called Virtual Pipeline Data Units (VPDU). Here, in Non-Patent Document 3, the VPDU size is set to 64×64 pixels, and the maximum width and height of the transform block is set to 64 pixels.

Therefore, if either the width or height of the target encoding block is greater than 64 pixels, the transform block is divided so that the width and height of the transform block are 64 pixels or less.

For example, when the encoding block size is 128×128 pixels, four transformation blocks of 64×64 pixels are included in the encoding block. Since the maximum size of the transform block and the size of the VPDU are defined to be the same size, the pipeline processing defined by the VPDU can be maintained by performing processing for each transform block.

When a plurality of transform blocks are included in the encoded block, the image decoding device 200 performs step S91 after performing steps S61, S62, S64, and S93 shown in FIG. 12 for all transform blocks. .

Therefore, when at least one of the width and height of the coded block is greater than 64 pixels, it is necessary to wait until the processing of all VPDUs that make up the target block is completed in order to control the decoding of the secondary transform index. This delays the pipeline processing of the decoder.

Therefore, if the decoding of the secondary transform index is restricted as described above, the effect of avoiding this processing delay can be expected.

In addition, the predetermined condition is determined using non-zero coefficient generation position information (last_coeff_pos) or a flag (coded_sub_block_flag) indicating whether or not a non-zero coefficient is generated in the target sub-block. The condition may be the presence or absence of decoding of the next transformation index.

For example, the predetermined condition is that when the non-zero coefficient generation area is limited when the secondary transform is applied, such as when the target block is 4×4 pixels or 8×8 pixels as shown in Non-Patent Document 2, For regions where zero coefficients cannot occur, it may include the condition that last_coeff_pos or coded_sub_block_flag do not indicate the presence of non-zero coefficients.

For example, the predetermined condition may include a condition that cbf of the target block is not zero or that last_coeff_pos of the target block does not point to a DC component (direct current component).

Note that determination by last_coeff_pos and cbf is performed when the luminance signal (Y signal) and color difference signals (Cb signal and Cr signal) of the target block have different block division tree structures (Dual Tree) and when both are the same ( Single Tree) may be determined as follows.

When the Y signal is a Dual Tree and last_coeff_pos is a DC component or cbf is 0, the image decoding device 200 does not decode the secondary transform index.

If last_coeff_pos is a DC component or cbf is 0 for both the Cb/Cr signal in Dual Tree and the Cb/Cr signal, the image decoding apparatus 200 does not decode the secondary transform index.

If last_coeff_pos is a DC component or cbf is 0 for all of the Y signal, Cb signal, and Cr signal in Single Tree, the image decoding device 200 does not decode the secondary transform index.

Alternatively, during Single Tree, if last_coeff_pos of the Y signal has a DC component or cbf is 0, the image decoding device 200 may not decode the secondary transform index.

With such a configuration, it is possible to determine whether or not to decode the secondary transform index without depending on the values of cbf and last_coeff_pos of the Cb signal and Cr signal.

In the above determination, if the secondary transformation index is not decoded, the secondary transformation index is implicitly treated as 0 (that is, it is determined that the secondary transformation is not applied).

As described above, when the region where nonzero coefficients occur when the secondary transform is applied is limited, for the region where nonzero coefficients cannot occur, last_coeff_pos or coded_sub_block Clearly, if the #flag indicates the presence of non-zero coefficients, no secondary transform is applied to the target block.

Therefore, for example, in the determination of the predetermined condition, when the target block is 4×4 pixels or 8×8 pixels, last_coeff_pos or coded_sub# If the block_flag indicates the presence of non-zero coefficients, the secondary transform index is not decoded, thereby saving the code amount required for transmission of the secondary transform index, and as a result, improving the coding performance. can be expected.

Also, if last_coeff_pos points to the DC component of the target block, the only non-zero coefficient in the target block is the DC component. On the other hand, the quadratic transform is effective when the nonzero primary transform coefficients are also present in the high-frequency component due to the nature of the quadratic transform to crowd the nonzero primary transform coefficients. If the primary transform coefficients of are only DC components, the secondary transform may not be applied.

Therefore, for example, when last_coeff_pos indicates a DC component in the target block, it is determined not to decode the secondary transform index, thereby saving the code amount required for transmission of the secondary transform index. can be expected to improve the coding performance.

Next, when the cbf of the target block is 0, as described above, there are no non-zero coefficients in the target block. By determining not to decode the secondary transform index, the code amount required for transmission of the secondary transform index can be saved, and as a result, an effect of improving the encoding performance can be expected.

In step S92, the image decoding device 200 decodes the secondary transform index and ends this operation.

<Modification 3>
Modification 3 will be described below, focusing on differences from the above-described third embodiment. Modification 3 is configured to use last_coeff_pos and coded_sub_block_flag to determine the presence or absence of secondary transform index decoding.

In the third embodiment described above, regarding the presence or absence of decoding of the secondary transform index of the target block, last_coeff_pos and coded_sub_block_flag indicate whether non-zero coefficients are generated when the secondary transform is applied. It was determined not to decode for the secondary transform index if it indicated the presence of non-zero coefficients for the restricted region.

On the other hand, in Modified Example 3, for example, when the secondary transform is applied and all the high frequency components of the target block are zeroed (zeroed) for simplification of processing, coded_sub_block# Determination by the flag is obviously unnecessary, so whether or not secondary transform decoding may be determined from only last_coeff_pos.

<Modification 4>
Modification 4 will be described below, focusing on differences from the above-described third embodiment.

In the above-described third embodiment, the example in which the decoding determination of the secondary transform index is performed for each encoded block has been described.

On the other hand, in Modification 4, when an encoding block is divided into a plurality of transform blocks, such a determination may be made for each transform block. According to such a configuration, the decoding of the secondary transform index is determined for each transform block, thereby increasing the number of target blocks to which the secondary transform may be applied, and as a result, it is expected that the encoding performance will be improved. be able to.

Note that the image encoding device 100 and the image decoding device 200 described above may be implemented as a program that causes a computer to execute each function (each process).

In the above-described embodiment, the present invention is applied to the image encoding device 100 and the image decoding device 200 as examples, but the present invention is not limited to such an example, and can be applied to the image encoding device. 100 and an image encoding/decoding system having each function of the image decoding apparatus 200.

According to the present invention, the conventional method requires additional processing for counting the number of non-zero coefficients in order to determine whether or not the secondary transform is applied when decoding the coefficients. This eliminates the need for additional processing, and can be expected to speed up processing or reduce the load.

Furthermore, according to the present invention, it is possible to omit the decoding of the secondary transform index for a block to which the secondary transform has not been applied or it can be determined that the effect is low, so an improvement in coding performance can be expected.

DESCRIPTION OF SYMBOLS 10... Image processing system 100... Image encoding apparatus 111, 241... Inter prediction part 112, 242... Intra prediction part 121... Subtractor 122, 230... Adder 131... Transform/quantization part 131A... Primary transform part 131B... Two Next transformation unit 131C Quantization units 132, 220 Inverse transformation/inverse quantization unit 220A Inverse quantization unit 220B Inverse secondary transformation unit 220C Inverse primary transformation unit 140 Encoding units 150, 250 In-loop filter Processing units 160, 260... Frame buffer 200... Image decoding device 210... Decoding unit

Claims (3)

An image decoding device,
an inverse transform unit configured to generate a prediction residual signal by inverse transform;
The inverse transform unit is configured to determine whether or not to decode the secondary transform index of the encoded block based on a flag indicating whether or not a non-zero coefficient is generated in the transform block. Image decoding device. generating a prediction residual signal by inverse transform;
An image decoding method, wherein, in said step , whether or not to decode a secondary transform index of an encoding block is determined based on a flag indicating whether or not a non-zero coefficient is generated in the transform block. A program that causes a computer to function as an image decoding device,
The image decoding device comprises an inverse transform unit configured to generate a prediction residual signal by inverse transform,
The inverse transform unit is configured to determine whether or not to decode the secondary transform index of the encoded block based on a flag indicating whether or not a non-zero coefficient is generated in the transform block. program.

Images