JP2022178335A - Image decoding device and image encoding device - Google Patents

Abstract

To provide an image decoding device and an image encoding device that extend the range of a transform coefficient while reducing complexity.SOLUTION: An image decoding device includes: a flag indicating whether to extend the range of a transform coefficient; a header decoding unit that decodes a syntax element indicating the maximum size of transform to derive the maximum size of the transform; a scaling unit that derives the range of transform coefficients according to a bit depth when extending the range of transform coefficients, and uses a first shift value derived according to the range of transform coefficient to right-shift the value obtained by multiplying the transform coefficient by a value that depends on a quantization parameter. The header decoding unit limits the maximum transform size to a predetermined value or less when the flag indicating whether to extend the range of transform coefficients is valid.SELECTED DRAWING: Figure 23

Description

TECHNICAL FIELD Embodiments of the present invention relate to an image decoding device and an image encoding device.

In order to efficiently transmit or record images, an image encoding device that generates encoded data by encoding an image and an image decoding device that generates a decoded image by decoding the encoded data are used. used.

Examples of specific image coding methods include H.264/AVC and HEVC (High-Efficiency Video
Coding) method, etc.

In such an image coding method, the images (pictures) that make up a moving image are slices obtained by dividing an image, and coding tree units (CTU: Coding Tree Unit) obtained by dividing a slice. , a coding unit obtained by dividing a coding tree unit (sometimes called a coding unit (CU)), and
Managed by a hierarchical structure composed of transform units (TU: Transform Units) obtained by dividing a coding unit, and encoded/decoded for each CU.

Further, in such an image coding method, a predicted image is normally generated based on a locally decoded image obtained by encoding/decoding an input image, and the predicted image is subtracted from the input image (original image). The prediction error (also called "difference image" or "residual image") obtained by Inter-prediction and intra-prediction are examples of predictive image generation methods.

In addition, Non-Patent Document 1 (VVC) can be cited as a recent image encoding and decoding technology. Non-Patent Document 1 discloses a technique for improving precision by increasing the range of transform coefficients and shift values according to bit depth in accordance with the extended_precision_processing_flag flag.

Non-Patent Document 2 discloses limiting the transform size to 32×32 when the quantization parameter is small. Non-Patent Document 3 discloses changing the shift value of the transform coefficient according to the magnitude of the residual.

VVC operation range extensions (Draft 3), JVET-V2005-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29 AHG8: A constraint of max transform size for high bit depth and high bit rate coding, JVET-0067-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29 AHG8: Content Adaptive Transform Precision for High Bit Depth Coding, JVET-0131-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29

In the technique for extending the range of transform coefficients in Non-Patent Document 1, the number of bits required for the transform product is 16.
In order to exceed the number of bits, there are problems such as a decrease in parallelism of SIMD processing in software and an increase in circuit scale in hardware. In Non-Patent Document 2, by prohibiting 64x64 conversion according to the quantization parameter, the circuit scale for conversion when the quantization parameter is small can be reduced. However, when the quantization parameter is large, there is a problem that the effect of reducing the circuit size cannot be obtained. In Non-Patent Document 3, by variably setting the shift value of the transform coefficient according to the value range of the transform coefficient, it is possible to improve the accuracy without increasing the range of the transform coefficient. However, there is a problem of increased complexity and latency due to analysis of the range of transform coefficients.

To solve the above problem, an image decoding device according to an aspect of the present invention decodes a flag indicating whether to expand the range of transform coefficients and a syntax element indicating the maximum size of transform, and decodes the maximum size of transform. and a header decoding unit that derives a transform coefficient range according to the bit depth when extending the transform coefficient range, and using the first shift value derived according to the transform coefficient range, A scaling unit that right-shifts a value obtained by multiplying a transform coefficient by a value that depends on a quantization parameter, and the header decoding unit, when a flag indicating whether to extend the range of the transform coefficient is enabled, the maximum size of the transform. is limited to a predetermined value or less.

The video decoding device is characterized by further comprising a TU decoding section that recursively divides the transform tree so that the size of the transform unit is equal to or smaller than the maximum size of the transform.

The header decoding unit decodes a flag indicating whether or not to use a predetermined maximum transform size TX as a syntax element indicating the maximum transform size, and the header decoding unit instructs to use the predetermined maximum transform size TX. and the maximum transform size TX is used as the maximum transform size when the flag indicating whether to extend the range of transform coefficients is not valid.

When the flag indicating that the header decoding unit uses a predetermined maximum transform size TX and indicating whether to extend the range of transform coefficients is not valid, the header decoding unit sets the maximum transform size to the above The maximum transform size TX is set, and in other cases, the maximum transform size TX/2 is set as the maximum transform size.

a header encoding unit that encodes a flag indicating whether to extend the range of transform coefficients and a syntax element indicating the maximum size of the transform to derive the maximum size of the transform;
when extending the range of transform coefficients, derive a range of transform coefficients according to the bit depth, and rely on a quantization parameter for transform coefficients using a first shift value derived according to the range of transform coefficients. a scaling unit that right-shifts the value multiplied by the value of
The header encoding unit is characterized by limiting the maximum size of transform to a predetermined value or less when a flag indicating whether to extend the range of transform coefficients is valid.

a header decoding unit that decodes a flag indicating whether to extend the range of transform coefficients;
Scaling for right-shifting a value obtained by multiplying a transform coefficient by a value dependent on a quantization parameter using a first shift value derived according to the range of the transform coefficient having a department,
The TU decoding unit is characterized by deriving the range of transform coefficients according to the flag and the block size of the transform block.

The TU decoding unit is characterized by extending the range of transform coefficients when the flag indicates valid and the block size of the transform block is smaller than a predetermined maximum transform size.

The TU decoding unit is characterized in that, when the flag indicates valid and the block size of the transform coefficients is smaller than 64, the range of the transform coefficients is extended.

a header encoding unit that encodes a flag indicating whether to extend the range of transform coefficients;
A TU encoding unit for deriving a range of transform coefficients is provided, and a first shift value derived according to the range of transform coefficients is used to right-shift a value obtained by multiplying the transform coefficients by a value dependent on a quantization parameter. Equipped with a scaling section,
The TU decoding unit is characterized by deriving the range of transform coefficients according to the flag and the block size of the transform block.

a header decoding unit that decodes a flag indicating whether to extend the range of transform coefficients;
a TU decoder for deriving a range of transform coefficients,
The TU decoding unit expands the range of transform coefficients when the flag is valid and the block size of the transform block is equal to or less than a predetermined size,
When the flag is valid and the block size of the transform block is larger than a predetermined size, the TU decoding unit converts the shift value without expanding the range of transform coefficients when the flag is valid. , depending on the maximum magnitude of the transform coefficients, and
The TU decoding unit derives the shift value without depending on the maximum value of the magnitude of the transform coefficient without extending the range of the transform coefficient unless the flag is valid,
A scaling unit is provided for right-shifting a value obtained by multiplying a transform coefficient by a value dependent on a quantization parameter using a first shift value derived according to the range of the transform coefficient.

a header encoding unit that encodes a flag indicating whether to extend the range of transform coefficients;
a TU decoder for deriving a range of transform coefficients,
The TU encoding unit expands the range of transform coefficients when the flag is valid and the block size of the transform block is equal to or less than a predetermined size,
When the flag is valid and the block size of the transform block is larger than a predetermined size, the TU encoding unit, when the flag is valid, does not extend the range of transform coefficients, and shifts the shift value , depending on the maximum magnitude of the transform coefficients,
The TU encoding unit derives the shift value without depending on the maximum value of the magnitude of the transform coefficient without extending the range of the transform coefficient unless the flag is valid,
A scaling unit is provided for right-shifting a value obtained by multiplying a transform coefficient by a value dependent on a quantization parameter using a first shift value derived according to the range of the transform coefficient.

According to the above configuration, any one of the above problems can be solved.

1 is a schematic diagram showing the configuration of an image transmission system according to this embodiment; FIG. 1 is a diagram showing the configuration of a transmitting device equipped with a moving image encoding device and a receiving device equipped with a moving image decoding device according to an embodiment; FIG. PROD_A indicates a transmitting device equipped with a video encoding device, and PROD_B indicates a receiving device equipped with a video decoding device. 1 is a diagram showing the configuration of a recording device equipped with a moving image encoding device and a reproducing device equipped with a moving image decoding device according to an embodiment; FIG. PROD_C indicates a recording device equipped with a moving image encoding device, and PROD_D indicates a reproducing device equipped with a moving image decoding device. FIG. 3 is a diagram showing the hierarchical structure of data in an encoded stream; FIG. 4 is a diagram showing an example of CTU division; 1 is a schematic diagram showing the configuration of a video decoding device; FIG. 4 is a flowchart for explaining schematic operations of a video decoding device; FIG. 4 is a schematic diagram showing the configuration of a TU decoding section; FIG. 11 shows syntax elements for quantized transform coefficients; FIG. 11 shows syntax elements for quantized transform coefficients; It is a syntax table about RRC. It is a syntax table about RRC. It is a syntax table about RRC. It is a syntax table about TSRC. It is a syntax table about TSRC. 1 is a block diagram showing the configuration of a video encoding device; FIG. It is a figure regarding Rice parameter derivation. FIG. 10 is a diagram showing processes of residual decoding, inverse quantization, and inverse transform; FIG. 4 is a diagram for explaining bit lengths of binarization of residuals using TR codes and EG(k) codes; FIG. 10 is a flowchart showing processing for limiting the transform size according to extended_precision_processing_flag in the present embodiment; FIG. 10 is a flowchart showing processing for limiting the transform size according to extended_precision_processing_flag in this embodiment. 10 is a syntax table showing decoding processing of a TU decoding unit 3024 in this embodiment. 4 is a flowchart showing processing for extending a transform coefficient range according to extended_precision_processing_flag and transform block sizes nTbW and nTbH in the present embodiment. 10 is a flowchart showing a process of switching between extending the transform coefficient range according to the extended_precision_processing_flag and the transform block sizes nTbW and nTbH and making the shift value variable according to the transform coefficient in the present embodiment. 10 is a flowchart showing a process of switching between extending the transform coefficient range according to the extended_precision_processing_flag and the transform block sizes nTbW and nTbH and making the shift value variable according to the transform coefficient in the present embodiment. FIG. 4 is a block diagram showing the configuration of a TU decoding unit 3024 and an inverse quantization/inverse transform unit 311 according to the embodiment of the present invention;

[Embodiment 1]
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a schematic diagram showing the configuration of an image transmission system 1 according to this embodiment.

The image transmission system 1 is a system that transmits an encoded stream obtained by encoding an encoding target image, decodes the transmitted encoded stream, and displays the image. The image transmission system 1 includes a moving image coding device (image coding device) 11 , a network 21 , a moving image decoding device (image decoding device) 31 and an image display device (image display device) 41 .

An image T is input to the moving image encoding device 11 .

The network 21 transmits the encoded stream Te generated by the video encoding device 11 to the video decoding device 31 . Network 21 includes the Internet (Internet), wide area network (
Wide Area Network (WAN), Local Area Network (LAN), or a combination thereof. The network 21 is not necessarily a two-way communication network, and may be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. Also, the network 21 may be replaced by a storage medium recording the encoded stream Te such as a DVD (Digital Versatile Disc: registered trademark) or a BD (Blue-ray Disc: registered trademark).

The video decoding device 31 decodes each of the encoded streams Te transmitted by the network 21, and generates one or a plurality of decoded decoded images Td.

The image display device 41 displays all or part of one or more decoded images Td generated by the moving image decoding device 31 . The image display device 41 includes a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. The form of the display includes stationary, mobile, HMD, and the like. In addition, when the moving image decoding device 31 has high processing power, it displays an image with high image quality, and when it has only lower processing power, it displays an image that does not require high processing power and display power. .

<operator>
The operators used in this specification are described below.

>> is right bit shift, << is left bit shift, & is bitwise AND, | is bitwise OR
, |= is the OR assignment operator, and || indicates the disjunction.

x?y:z is a ternary operator that takes y if x is true (not 0) and z if x is false (0).

Clip3(a,b,c) is a function that clips c to a value greater than or equal to a and less than or equal to b, if c<a returns a, if c>b returns b, otherwise is a function that returns c (where a<=b).

abs(a) is a function that returns the absolute value of a.

Int(a) is a function that returns the integer value of a.

floor(a) is a function that returns the smallest integer less than or equal to a.

ceil(a) is a function that returns the largest integer greater than or equal to a.

a/d represents division of a by d (truncated after the decimal point).

<Structure of encoded stream Te>
Prior to a detailed description of the video encoding device 11 and the video decoding device 31 according to the present embodiment, data of the encoded stream Te generated by the video encoding device 11 and decoded by the video decoding device 31 I will explain the structure.

FIG. 4 is a diagram showing the hierarchical structure of data in the encoded stream Te. The encoded stream Te illustratively includes a sequence and a plurality of pictures forming the sequence. FIG. 4 shows a coded video sequence that defines the sequence SEQ, a coded picture that defines the picture PICT, a coded slice that defines the slice S, coded slice data that defines the slice data, and coded slice data that defines the slice data. A diagram showing the included coding tree unit and the coding units included in the coding tree unit is shown.

(encoded video sequence)
The encoded video sequence defines a set of data that the video decoding device 31 refers to in order to decode the sequence SEQ to be processed. A sequence SEQ consists of a video parameter set, a sequence parameter set SPS, a picture parameter set PPS, and a picture header, as shown in the encoded video sequence of FIG. ), picture PICT, and Supplemental Enhancement Information (SEI).

A video parameter set VPS specifies a set of coding parameters common to multiple images, a set of coding parameters related to multiple layers included in an image, and individual layers in an image composed of multiple layers. It is

The sequence parameter set SPS defines a set of coding parameters that the video decoding device 31 refers to in order to decode the target sequence. For example, the width and height of the picture are defined. A plurality of SPSs may exist. In that case, one of a plurality of SPSs is selected from the PPS.

The picture parameter set PPS defines a set of coding parameters that the video decoding device 31 refers to in order to decode each picture in the target sequence. For example, it includes a quantization width reference value (pic_init_qp_minus26) used for picture decoding, a flag indicating the application of weighted prediction (weighted_pred_flag), and a scaling list (quantization matrix). A plurality of PPSs may exist. In that case, one of a plurality of PPSs is selected from each picture in the target sequence.

The picture header defines coding parameters common to all slices included in one coded picture. For example, POC (Picture Order Count) and coding parameters related to division are included.

(coded picture)
The encoded picture defines a set of data that the video decoding device 31 refers to in order to decode the picture PICT to be processed. A picture PICT includes slice 0 to slice NS-1 (NS is the total number of slices included in the picture PICT), as shown in the coded picture in FIG.
.

In the following description, if there is no need to distinguish between slices 0 to NS-1, the suffixes may be omitted. The same applies to other data with subscripts that are included in the encoded stream Te described below.

(coded slice)
The encoded slice defines a set of data that the video decoding device 31 refers to in order to decode the slice S to be processed. A slice is represented by the coded slice in Figure 4.
It contains a slice header and slice data.

The slice header includes a coding parameter group that the video decoding device 31 refers to in order to determine the decoding method for the target slice. Slice type designation information (slice_type) that designates a slice type is an example of a coding parameter included in a slice header.

Slice types that can be designated by the slice type designation information include (1) I slices that use only intra prediction during encoding, (2) P slices that use unidirectional prediction or intra prediction during encoding, (3) B slices using uni-prediction, bi-prediction, or intra-prediction in encoding. Note that inter prediction is not limited to uni-prediction and bi-prediction, and a predicted image may be generated using more reference pictures. Hereinafter, when referred to as P and B slices, they refer to slices containing blocks for which inter prediction can be used.

Note that the slice header may include a reference (pic_parameter_set_id) to the picture parameter set PPS.

(encoded slice data)
The encoded slice data defines a set of data that the video decoding device 31 refers to in order to decode slice data to be processed. The slice data contains CTU, as shown in the encoded slice header in FIG. A CTU is a fixed-size (for example, 64x64) block that forms a slice, and is also called a largest coding unit (LCU).

(encoding tree unit)
The coding tree unit in FIG. 4 defines a set of data that the video decoding device 31 refers to in order to decode the CTU to be processed. CTU is a recursive quadtree decomposition (QT)
split), binary tree split (BT (Binary Tree) split) or ternary tree split (TT (Ternary Tree)
division) into coding units CU, which are basic units for coding processing. BT partitioning and TT partitioning are collectively called multi-tree partitioning (MT (Multi Tree) partitioning). A tree-structured node obtained by recursive quadtree partitioning is called a coding node. Intermediate nodes of quadtrees, binary trees, and ternary trees are coding nodes, and the CTU itself is defined as the top-level coding node.

CT includes, as CT information, a segmentation flag indicating whether to perform QT segmentation. Fig. 5 shows an example of division.

A CU is a terminal node of an encoding node and is not split any further. A CU is a basic unit of encoding processing.

(encoding unit)
As shown in the encoding unit in FIG. 4, a set of data that the video decoding device 31 refers to in order to decode the encoding unit to be processed is defined. Specifically, a CU is composed of a CU header CUH, prediction parameters, transform parameters, quantized transform coefficients, and the like. A prediction mode and the like are defined in the CU header.

Prediction processing may be performed in units of CUs or may be performed in units of sub-CUs obtained by further dividing a CU. If the CU and sub-CU sizes are equal, there is one sub-CU in the CU. CU is sub CU
, the CU is split into sub-CUs. For example, if the CU is 8x8 and the sub-CU is 4x4, the CU is divided into 4 sub-CUs consisting of 2 horizontal divisions and 2 vertical divisions.

The type of prediction (prediction mode CuPredMode) includes at least two of intra prediction (MODE_INTRA) and inter prediction (MODE_INTER). In addition intra block copy prediction (MODE_IBC)
may be provided. Intra prediction and intra block copy prediction are predictions within the same picture, and inter prediction refers to prediction processing performed between different pictures (for example, between display times or between layer images).

Transformation/quantization processing is performed in units of CUs, but the quantized transform coefficients may be entropy-encoded in units of subblocks such as 4x4.

A predicted image is derived from the prediction parameters associated with the block. The prediction parameters include prediction parameters for intra prediction and inter prediction.

(Configuration of video decoding device)
The configuration of the video decoding device 31 (FIG. 6) according to this embodiment will be described.

The video decoding device 31 includes an entropy decoding unit 301, a parameter decoding unit (prediction image decoding device) 302, a loop filter 305, a reference picture memory 306, a prediction parameter memory 307, a prediction image generation unit (prediction image generation device) 308, an inverse It includes a quantization/inverse transformation unit 311 (scaling unit) and an addition unit 312 . Note that the moving image decoding device 31 may have a configuration in which the loop filter 305 is not included in accordance with the moving image encoding device 11 described later.

Parameter decoding section 302 further includes header decoding section 3020 , CT information decoding section 3021 , and CU decoding section 3022 (prediction mode decoding section), and CU decoding section 3022 includes TU decoding section 3024 . These may be collectively called a decoding module. Header decoding section 3020 decodes parameter set information such as VPS, SPS, and PPS, and slice headers (slice information) from encoded data. CT information decoding section 3021 decodes CT from encoded data. A CU decoding unit 3022 decodes a CU from encoded data. The TU decoding unit 3024 decodes the transform skip flag transform_skip_flag, QP update information (quantization correction value), and quantized transform coefficient (residual_coding) from the encoded data.

TU decoding section 3024 decodes transform_skip_flag[x0][y0][cIdx] when the transform unit size (tbWidth and tbHeight) is equal to or smaller than a predetermined maximum size (tbWidth<=MaxTsSize &&tbHeight<=MaxTsSize).

The TU decoding unit 3024 decodes the QP update information and the quantized transform coefficients from the encoded data when the TU contains a prediction error (for example, when tu_cbf_luma[x0][y0] is 1). Derivation of quantized transform coefficients may comprise multiple modes (eg, RRC mode and TSRC mode). Specifically, different processes are used for normal prediction error derivation using transform (RRC: Regular Residual Coding) and prediction error derivation in transform skip mode without transform (TSRC: Transform Skip Residual Coding). may be implemented. The QP update information is a difference value from the quantization parameter prediction value qPpred, which is the prediction value of the quantization parameter QP.

Further, although an example using CTU and CU as units of processing will be described below, the processing is not limited to this example, and processing may be performed in units of sub-CUs. Alternatively, CTU and CU may be read as blocks, and sub-CU may be read as sub-blocks, and processing may be performed in units of blocks or sub-blocks.

The entropy decoding unit 301 performs entropy decoding on an externally input encoded stream Te to parse individual codes (syntax elements). For entropy coding, a method of variable-length coding syntax elements using a context (probability model) adaptively selected according to the type of syntax elements and surrounding circumstances, a predetermined table, or There is a method of variable-length coding syntax elements using a formula. An example of the former is CABAC (Context Adaptive Binary Arithmetic Coding). The parsed code includes prediction information for generating a prediction image, prediction error for generating a difference image, and the like.

Entropy decoding section 301 outputs the separated code to parameter decoding section 302 . The separated code is, for example, the prediction mode CuPredMode. Control of which code is to be decoded is performed based on an instruction from parameter decoding section 302 .

(Basic flow)
FIG. 7 is a flowchart for explaining a schematic operation of the video decoding device 31. FIG.

(S1100: Parameter Set Information Decoding) Header decoding section 3020 decodes parameter set information such as VPS, SPS, PPS, SEI, and PH from the encoded data.

(S1200: Slice Information Decoding) Header decoding section 3020 decodes the slice header (slice information) from the encoded data.

Below, the video decoding device 31 performs steps S1300 to S5000 for each CTU included in the target picture.
The decoded image of each CTU is derived by repeating the process of .

(S1300: CTU Information Decoding) CT information decoding section 3021 decodes CTU from encoded data.

(S1400: CT Information Decoding) CT information decoding section 3021 decodes CT from encoded data.

(S1500: CU Decoding) CU decoding section 3022 executes S1510 and S1520 to decode CU from encoded data. Also, the CU decoding unit 3022 decodes the quantization parameter difference CuQpDeltaVal from the encoded data in units of CUs, and derives the quantization parameter.

(S1510: CU information decoding) CU decoding section 3022 decodes CU information, prediction information, TU split flag split_transform_flag, CU residual flag cbf_cb, cbf_cr, cbf_luma, etc. from encoded data.

(S1520: TU information decoding) TU decoding section 3024 decodes the QP update information and the quantized transform coefficients from the encoded data when the TU contains a prediction error.

(S2000: Generate Predicted Image) The predicted image generation unit 308 generates a predicted image for each block included in the target CU based on the prediction information.

(S3000: Inverse Quantization/Inverse Transform) The inverse quantization/inverse transform unit 311 executes inverse quantization/inverse transform processing for each TU included in the target CU.

(S4000: Decoded image generation) The addition unit 312 adds the predicted image supplied from the predicted image generation unit 308 and the prediction error supplied from the inverse quantization/inverse transform unit 311, thereby decoding the target CU. Generate an image.

(S5000: Loop Filter) Loop filter 305 applies loop filters such as a deblocking filter, SAO (Sample Adaptive Filter), and ALF (Adaptive Loop Filter) to the decoded image to generate a decoded image.

(Derivation of quantized transform coefficients, residual coding)
In the case of lossless encoding or when the correlation between pixels in the original image is small, the encoding efficiency may be higher if no transform is performed. A technique that does not perform any transformation is called Transform Skip. Transform skipping, also called Identical Transform, only scales the transform coefficients according to the quantization parameter. The transform skip is notified using the syntax element transform_skip_flag. transform_skip_flag may be signaled for each color component (cIdx) of Y, Cb, and Cr.

Normal prediction error derivation using transform (RRC: Regular Residual Coding) and prediction error derivation in transform skip mode (TSRC: Transform Skip Residual Coding) differ in prediction error encoding and decoding methods.

FIG. 8 is a block diagram of TU decoding section 3024, which includes RRC section 30241 and TSRC section 30242. FIG. The RRC unit 30241 is a processing unit that derives a normal prediction error using transform, and the TSRC unit 30242 is a processing unit that derives a prediction error in transform skip mode.

FIG. 9(a) sps_transform_skip_enabled_flag is a flag indicating whether transform_skip_flag is notified in each TU. sps_transform_skip_enabled_flag=1 indicates that transform_skip_flag is notified in each TU. sps_transform_skip_enabled_flag=0 indicates that transform_skip_flag is not signaled in each TU. Assumes 0 if sps_transform_skip_enabled_flag is not signaled.

transform_skip_flag[x0][y0][cIdx] in FIG. 10(b) has upper left coordinates (x0, y0) and color component cIdx
, indicates whether the transform is applied. If transform_skip_flag=1 (transform skip mode), no transform is applied to this block. transform_skip_flag=0
, whether a transform is applied to this block depends on other parameters.

FIGS. 11 to 13 and FIGS. 14 to 15 respectively show the structures (encoding methods) of encoded data of transform coefficients (prediction errors) in RRC (Regular Residual Coding) and TSRC (Transform Skip Residual Coding). It is a syntax table.

(RRC part, RRC mode)
In the normal prediction error encoding method without transform skip (FIGS. 11 to 13), the RRC unit 30241 decodes the syntax element (not shown) indicating the LAST position and derives the LAST position (LastSignificantCoeffX, LastSignificantCoeffY). The LAST position is the position of the last non-zero coefficient when scanning the transform coefficients of the TU from low frequency components to high frequency components. When the transform coefficients (coefficients) are encoded and decoded in order from the high frequency component, the LAST position indicates the position of the quantized transform coefficient to be decoded first. Next, the RRC unit 30241 references the LAST position and decodes sb_coded_flag. sb_coded_flag is a flag indicating whether or not the sub-block contains non-zero coefficients. A subblock is an area obtained by dividing a TU into 4x4 units. If sb_coded_flag=1 (the sub-block contains non-zero coefficients), the RRC unit 30241 decodes sig_coeff_flag. sig_coeff_flag is a flag indicating whether the coefficient value is non-zero. If sig_coeff_flag=1 (coefficient value is non-zero), the RRC unit 30241 decodes abs_level_gtx_flag, par_level_flag, abs_remainder, and dec_abs_level. These are syntax elements that indicate the absolute value of the coefficient. abs_level_gtx_flag[n][j] is a flag indicating whether the absolute value of the coefficient at scan position n is greater than (j<<1)+1. If abs_level_gtx_flag[n][j] is not signaled, it is assumed to be 0. par_level_flag[n] is the parity of the coefficients at scan position n. If par_level_flag[n][j] is not signaled, it is assumed to be 0. abs_remainder[n] represents the remainder of the absolute value of the coefficient at scan position n, decoded by Golomb-Rice code. If abs_remainder[n] is not signaled, it is assumed to be 0. dec_abs_level[n] is the absolute value of the residual to derive the absolute value of the coefficient at scan position n. If dec_abs_level[n] is not signaled, it is assumed to be 0. The RRC unit 30241 derives the absolute values of the coefficients from these syntax elements.

As described above, the RRC unit 30241 is characterized by decoding the LAST position in the TU subblock.

(TSRC section, TSRC mode)
In the predictive error coding method (FIGS. 14 and 15) when no transform is performed (transform skip mode), the TSRC unit 30242 decodes sb_coded_flag of each subblock. If sb_coded_flag=1 (the sub-block contains non-zero coefficients), the TSRC unit 30242 determines the position (xC,
Decode the sig_coeff_flag[xC][yC] of the transform coefficients of xC). If sig_coeff_flag=1 (coefficient value is non-zero), the TSRC unit 30242 decodes coeff_sign_flag, abs_level_gtx_flag, par_level_flag and abs_remainder. These are syntax elements that indicate the absolute value of the coefficients and are defined above. The TSRC unit 30242 derives the absolute values of the coefficients from these syntax elements.

As described above, the TSRC unit 30242 is characterized by not decoding the LAST position in the subblocks of the TU.

(decoding of abs_remainder and dec_abs_level)
The TU decoding unit 3024 decodes the residual absolute value syntax value (abs_remainder or dec_abs_level, hereinafter referred to as residual) from the encoded data using the Rice parameter cRiceParam. A code consisting of a prefix and a suffix is used for the binarization of the syntax. As prefixes and suffixes, Golomb-Rice alpha codes and fixed-length binarizations may be used. The suffix may use an EG(k) (Exponential-Golomb code) code. The “absolute value of residual of transform coefficient (absolute residual value)” encoded as a syntax element is simply expressed as “residual”.

In this embodiment, Truncated Rice (TR) code is used as the Golomb Rice code, and the prefix is divided into prefixValTR and suffixValTR for encoding. An example of encoding will be described. The quotients of dividing the residual (abs_remainder or dec_abs_level) by 1<<cRiceParam, (((1<<expVal))-1)<<cRiceParam are encoded using prefixValTR and expVal, respectively. The remainders of dividing the residual (abs_remainder or dec_abs_level) by 1<<cRiceParam, (1<<expVal<<cRiceParam) are encoded using suffixValTR and escapeVal, respectively. That is, the residual is encoded as follows.

((prefixValTR + ((1<<expVal-1))) << cRiceParam) + suffixValTR + escapeVal
If the residual is less than or equal to a predetermined value cMax, only prefixValTR and suffixValTR are encoded.

prefixVal = (prefixValTR<<cRiceParam) + suffixValTR
If the residual is greater than a predetermined value (if suffixVal is present) then prefixValTR indicating prefixVal equal to cMax plus expVal and escapeVal are encoded.

prefixVal = (prefixValTR<<cRiceParam) = cMax
suffixVal = (((1<<expVal)-1)<<cRiceParam) + escapeVal
Here the residual can be expressed as The residual when it can be expressed only by the prefix is the following formula.

Residual = ((prefixValTR) << cRiceParam) + suffixValTR
Otherwise (if it cannot be represented by prefix only, or if prefixValTR=6), the residual is given by the following equation.

residual = ((prefixValTR + (1<<(expVal)-1 )) << cRiceParam) + escapeVal
You can also define prefixVal and suffixVal as follows.

prefixVal = (prefixValTR << cRiceParam) + suffixValTR
suffixVal = (((2<<expVal)-2) << cRiceParam) + escapeVal
(flow)
TU decoding section 3024 first derives the maximum value cMax of prefix prefixVal from cRiceParam.

cMax = 6 << cRiceParam
Here 6 is the maximum value of prefixValTR. Also called maxPrefixValTR.

The TU decoding unit 3024 decodes the prefix value (prefixVal) from the encoded data based on binarization of Truncated Rice (TR) code, which will be described later. The relationship between prefixes and residuals is as follows.

prefixVal = Min(cMax,abs_remainder[n])
Or it can be:

prefixVal = Min(cMax,dec_abs_level)
Subsequently, if the residual is larger than cMax (if the residual cannot be represented by prefixVal alone), TU decoding section 3024 further decodes suffixVal that satisfies the following relational expression.

suffixVal = abs_remainder[n] - cMax
Or it can be:

suffixVal = dec_abs_level - cMax
Specifically, when suffixVal exists, TU decoding section 3024 decodes suffixVal from the encoded data based on the binarization of Limited k-th order Exp-Golomb (EG(k)). A case where suffixVal exists is a case where the bit string of prefixVal of the TR code has a length of 6 (=maxPrefixValTR), that is, "111111". Here, the order k is cRiceParam, the variable maxPreExtLen is 11, and the variable truncSuffixLen is 15. maxPreExtLen may not be a fixed value (11), but may be derived according to the range log2TransformRange of transform coefficients.

In this embodiment, for example, maxPreExtLen may be derived by the following formula.

maxPreExtLen = (32 - 6) - log2TransformRange = 26 - log2TransformRange
For example, when the log2TransformRange is (Equation R-1) described later, it may be derived as follows.

maxPreExtLen = extended_precision_processing_flag ? 26 - Max(15, BitDepth + BDOFFSET) : 11
Here, maxPreExtLen and escapeLength respectively indicate the maximum lengths of the exp part and the escape part in the EG(k) code. log2TransformRange is a variable that indicates the range of coefficients in the transform processed by the inverse quantization/inverse transform unit 311 .

FIG. 19 is a diagram explaining the sign relationship of prefixValTR, maxPreExtLen and truncSuffixLen. The residual code consists of a TR code prefix (prefixValTR) and an EG(k) code suffix, and the suffix consists of an exp part and an escape part. The lengths of prefixValTR and escape are maxPrefixValTR and truncSuffixLen, respectively, and the maximum length of exp is maxPreExtLen. In the following explanation, maxPrefixValTR=6.

(binarization of Truncated Rice (TR) code)
The input symbolVal for TR binarization is prefixVal, and in TR binarization, the quotient of symbolVal (==prefixVal) by (1<<cRiceParam) is encoded as prefixValTR and the remainder as suffixValTR. The relationship between the input symbolVal of the TR code and prefixValTR is as follows.

prefixValTR = symbolVal>>cRiceParam
prefixValTR is converted to a bit string with the following definition. prefixValTR<(cMax>>cRiceParam)
, is a bit string indexed by binIdx of length prefixValTR+1. If binIdx<prefixValTR, the bins are "1", and if binIdx==prefixValTR, the bins are "0". Otherwise (prefixValTR==(cMax>>cRiceParam), it is a string of "1"s of length cMax>>cRiceParam. A bin is a bit string consisting of "0" or "1". binIdx is a value that indicates the position from the beginning of the bit string, and binIdx=0 corresponds to the beginning (the left end of the binary string).An example of the bit string (bins) when prefixValTR is 0 to 5 is shown below.

prefixValTR==0 bins=0
prefixValTR==1 bins=10
prefixValTR==2 bins=110
prefixValTR==3 bins=1110
prefixValTR==4 bins=11110
prefixValTR==5 bins=111110
…
However, if symbolVal>=cMax (that is, prefixVal==(6<<cRiceParam)), the following formula is used.

prefixValTR==6 bins=111111
The bins of prefixValTR of the TR code are truncated alpha codes (unary codes). Here truncated means omitting the trailing 0 if it matches the maximum value.

If cMax is greater than symbolVal (symbolVal<cMax) and cRiceParam>0, then there is a TR bin string suffixTR, derived as follows: suffixValTR is the suffix of the TR bit string.

suffixValTR = symbolVal - (prefixValTR<<cRiceParam)
Also, for the binarization of the suffix of the TR bit string, fixed-length binarization (described later) is used with the maximum value cMaxFL set to (1<<cRiceParam)-1.

Note that when symbolVal==cMax, the binarization of the EG(k) code is used for the residual suffix instead of the suffixValTR of the above TR binarization.

(Limited k-th order Exp-Golomb/EG(k) code binarization)
The residual suffixVal may use binarization of the EG(k) code. The EG(k) code is a binarization obtained with the order k (=cRiceParam) and maxPreExtLen as input, and is shown in the pseudocode below. An EG(k) code consists of an exp part of length maxPreExtLen or less and an escape part of fixed length binarization of escapeLength length. preExtLen is a variable that counts the number of put(1)s in the exp part and is equal to the value espVal encoded in the exp part. That is, expVal=preExtLen. The corresponding residual values are expVal<<cRiceParam. escapeVal is the value encoded in the escape part.
// exp part
codeValue = symbolVal >> k
preExtLen = 0
while((preExtLen<maxPreExtLen) && (codeValue >((2<<preExtLen) - 2)))) {
preExtLen++
put(1)
}
if(preExtLen == maxPreExtLen)
escapeLength = truncSuffixLen
else {
escapeLength = preExtLen + k
put(0)
}
// escape part
symbolVal = symbolVal - (((1<<preExtLen)-1)<<k)
while((escapeLength--)>0)
put((symbolVal>>escapeLength) & 1)
where symbolVal is the input value and the residual minus prefixVal(==cMax). put(X) indicates an operation to create a bit string by adding X ("0" or "1") to the end of the bit string. The initial value of the bit string is empty ({}). escapeLength is a variable indicating the bit position output from the escape section in symbolVal.
Note that the escape section may be configured as follows.

symbolVal = symbolVal - (((2<<preExtLen)-2)<<k)
while((escapeLength--)>0)
put((symbolVal>>escapeLength) & 1)
(fixed length binarization)
Fixed-length binarization is performed using an unsigned integer sequence (binary sequence) of length fixedLength of Ceil(Log2(cMaxFL+1)) for maximum value cMaxFL and value symbolVal. Note that 0 in position binIdx corresponds to the most significant bit (MSB), and as binIdx increases, it approaches the least significant bit (LSB).

(Summary of residual decoding)
TU decoding section 3024 decodes the residual consisting of prefixVal and suffixVal.

TU decoding section 3024 derives cMax from cRiceParam, decodes prefixVal from encoded data with cMax as the upper limit, and decodes suffixVal if suffixVal exists.

Specifically, the TU decoding unit 3024 decodes the prefixValTR of the TR binarization, and if symbolVal<cMax, cRiceParam>0 (that is, prefixVal<cMax, prefixVal<6), the fixed-length binary Decode the suffixValTR of the suffixValTR. In this case suffixVal does not exist.

TU decoding section 3024 derives abs_remainder[n] and dec_abs_level by the following formula when the above suffixVal does not exist.

abs_remainder[n] = (prefixValTR<<cRiceParam) + suffixValTR
dec_abs_level[n]= (prefixValTR<<cRiceParam) + suffixValTR
Otherwise (if prefixVal==cMax, prefixValTR==6), the TU decoding unit 3024 decodes suffixVal using EG(k) code, and derives abs_remainder[n] and dec_abs_level[n] using the following formula. .

abs_remainder[n] = cMax + suffixVal
dec_abs_level[n] = cMax + suffixVal
Here, when suffixVal consists of preExtLen of the exp part of the EG(k) code and escapeVal of the escape part, the following holds.

suffixVal = (preExtLen<<cRiceParam) + escapeVal
In summary, the TU decoding unit 3024 decodes the TR binarization prefix prefixVal and the suffixVal consisting of preExtLen of the exp part of the EG(k) code and escapeVal of the escape part from the encoded data to obtain the residual (abs_remainder[n ] or dec_abs_level[n]).

abs_remainder[n] = prefixVal + suffixVal = (prefixValTR<<cRiceParam) + (prefix
ValTR < 6) ? suffixValTR : (1<<expVal<<cRiceParam) + escapeVal
Here, if (prefixValTR < 6) and preExtLen=escapeVal=0,
abs_remainder[n] = (prefixValTR+preExtLen)<<cRiceParam + (prefixValTR < 6) ? suffixValTR : escapeVal
Derivation of dec_abs_level[n] is similar.
Note that the code that concatenates the bins of prefixValTR of the truncated alpha code and the bins of predExtLen of the alpha code is also an alpha code. good. In this case, an alpha code of maximum length maxPreExtLen+6 is read from the encoded data, and the length prefixtmp (=prefixValTR+expVal) of the alpha code is decoded. Here, maxPreExtLen+6 is the sum of the maximum length 6 of prefixValTR and the maximum length maxPreExtLen of the exp part of the EG(k) code.

The TU decoding unit 3024 may divide the quotient of 1<<cRiceParam and the remainder to derive the residual as follows.

residual = (prefixRes << cRiceParam) + suffixRes
Assuming that the length of the decoded alpha code is prefixtmp, the residual exponent prefixRes is expressed by the following equation.

prefixRes = (prefixVal<6) ? prefixValTR : prefixValTR + (1<<expVal) - 1
If prefixtmp<6, prefixValTR=prefixtmp, and if prefixtmp>=6, prefixValTR=6 and expVal=prefixtmp-6, so they can be derived as follows.

prefixRes
= (prefixtmp<6) ? prefixtmp : 6 + (1<<(prefixtmp-6)) - 1
= (prefixtmp<6) ? prefixtmp : (1<<(prefixtmp-6)) + 5
again,
prefixRes = (prefixVal<6) ? prefixValTR : prefixValTR + (2<<expVal) - 2
In the case of a configuration using , the following may be used.

prefixRes
= (prefixtmp<6) ? prefixtmp : 6 + (2<<(prefixtmp-6)) - 2
= (prefixtmp<6) ? prefixtmp : (2<<(prefixtmp-6)) + 4
= (prefixtmp<6) ? prefixtmp : (1<<(prefixtmp-5)) + 4
Here prefixtmp<6 can also be treated as prefixtmp<5. Also, since both suffixValTR and escapeVal are fixed-length binarizations, the former can be fixedLength of cMaxFL, the latter can be fixedLength of escapeLength length, and suffixRes can be decoded in the same way.

(quantized transform coefficients)
The TU decoding unit 3024 (residual decoding unit 30241) sequentially decodes abs_remainder from the encoded data for the position n, and derives the quantized transform coefficient AbsLevel from AbsLevelPass1 and abs_remainder by the following formula. n is any position below the last coefficient position in the scan order in the sub-block until a predetermined number of bits are decoded.

AbsLevel[xC][yC] = AbsLevelPass1[xC][yC] + 2 * abs_remainder[n]
where AbsLevelPass1[xC][yC] = sig_coeff_flag[xC][yC] + par_level_flag[n] + abs_level_gtx_flag[n][0] + 2 * abs_level_gtx_flag[n][1].

In cases other than the above (n when decoding a predetermined number of bits), TU decoding section 3024 decodes dec_abs_level from the encoded data and derives AbsLevel using the following equation.

if (dec_abs_level[n] == ZeroPos[n])
AbsLevel[xC][yC] = 0
else if (dec_abs_level[n] < ZeroPos)
AbsLevel[xC][yC] = dec_abs_level[n] + 1
else /*dec_abs_level[n] > ZeroPos[n]*/
AbsLevel[xC][yC] = dec_abs_level[n]
Here, ZeroPos is derived by the following formula.

ZeroPos[n] = ( QState < 2 ? 1 : 2 ) << cRiceParam
where QState indicates the state of dependent quantization.

(Rice parameter derivation)
Of the syntax elements representing the absolute values of coefficients, abs_remainder and dec_abs_level are values binarized by Golomb-Rice code (or Rice code) and decoded by bypass code (equal probability code of CABAC). abs_remainder is the difference of the absolute value of the transform coefficient with respect to the offset, and dec_abs_level is the absolute value of the transform coefficient.

In the Golomb-Rice code, syntax elements are encoded by dividing them into two parts, the first half (prefix) and the second half (reminder, suffix). An alpha code (unary code) capable of encoding a value close to 0 with a short code amount is used for the prefix. Fixed-length binarization or EG(k) binarization using Rice parameter cRiceParam is used for the suffix. The larger the Rice parameter, the longer the bit length of small values and the shorter the bit length of large values. It is possible to reduce the code amount by adjusting the Rice parameter according to the occurrence probability of the value of the syntax element.

In the video decoding device and the video encoding device of this embodiment, values of syntax elements to be decoded are predicted from the absolute values of already derived coefficients. Then, an appropriate Rice parameter is derived from the predicted value. Details are shown below with reference to FIG.

Let (x0,y0) be the upper left coordinate of the current TU, (xC,yC) be the current scan position, and log2TbWidth and log2TbHeight be the log2 values of the width and height of the TU, respectively. Also, the array that stores the absolute values of the coefficients is
It is represented by AbsLevel[x][y] using the position (x,y). FIG. 17(a) shows the positions of decoded coefficients around coefficients used when decoding abs_remainder or dec_abs_level. Letting locSumAbs be a variable representing the sum of absolute values of surrounding coefficients, TU decoding section 3024 derives locSumAbs in RRC as follows.

locSumAbs = 0
if(xC <(1<<log2TbWidth)-1) {
locSumAbs += AbsLevel[xC+1][yC]
if(xC <(1<<log2TbWidth)-2)
locSumAbs += AbsLevel[xC+2][yC]
if(yC <(1<<log2TbHeight)-1)
locSumAbs += AbsLevel[xC+1][yC+1]
}
if(yC <(1<<log2TbHeight)-1) {
locSumAbs += AbsLevel[xC][yC+1]
if(yC <(1<<log2TbHeight)-2)
locSumAbs += AbsLevel[xC][yC+2]
}
locSumAbs0 = locSumAbs
locSumAbs = Clip3(0, 31, locSumAbs-baseLevel*5)
1) Add the value of t1(AbsLevel[xC+1][yC]) to locSumAbs when the coefficient position (xC+1,yC) is within the block (xC<(1<<log2TbWidth)-1).
2) In case of 1), furthermore, when the position (xC+2,yC) is within the block (xC<(1<<log2TbWidth)-2), locSumAbs has t2(AbsLevel[xC+2][yC]) Add value.
3) If 1) and position (xC+1, yC+1) is within the block (yC<(1<<log2TbHeigh)-1), locSumAbs has t3(AbsLevel[xC+1][yC+1 ]) value.
4) Add the value of t4(AbsLevel[xC][yC+1]) to locSumAbs when position (xC,yC+1) is within the block (yC<(1<<log2TbHeight)-1).
5) If 4) then position (xC,yC+2) is also within the block (yC<(1<<log2TbHeight)-2)
then add the value of t5(AbsLevel[xC][yC+2]) to locSumAbs.
The locSumAbs calculated from the above processing are updated as follows by the variable baseLevel and clip processing.

locSumAbs = Clip3(0, 31, locSumAbs-baseLevel*5)
Here, baseLevel may be 4 when decoding abs_remainder (tcoeff==0) and 0 when decoding dec_abs_level (tcoeff==1). tcoeff is a variable indicating whether the decoding target is abs_remainder or dec_abs_level.

Using the table shown in FIG. 17(b), Rice parameters used for decoding abs_remainder and dec_abs_level are derived from the derived locSumAbs.

Although not shown, the TU decoding unit 3024 may derive the Rice parameter cRiceParam using a formula instead of using a table as follows.

if (baseLevel != 0)
cRiceParam = Max(0, floor(log2(15 * locSumAbs)) - 7)
else
cRiceParam = Max(0, floor(log2(9 * locSumAbs) + 20) - 5)
As the formula, the sum of products with the sum locSumAbs of surrounding transform coefficients and the logarithm with base 2 can be used as described above. Here, baseLevel!=0 indicates the case of using for decoding abs_remainder, and baseLevel==0 below else indicates the case of using for decoding of dec_abs_level.

The TU decoding unit 3024 always sets the Rice parameter cRiceParam to 1 without using the surrounding decoded coefficients in the TSRC.

(Derivation of quantized transform coefficients, dependent quantization)
The inverse quantization/inverse transform unit 311 inversely quantizes the transform coefficients. As quantization methods, two methods of scalar quantization and dependent quantization are provided. In the case of dependent quantization, the RRC section 30241 may also perform part of the inverse quantization processing.

The inverse quantization/inverse transform unit 311 derives the linear scale value ls[x][y] according to the values of the quantization parameter qP, rectNonTsFlag, and quantization matrix m[][] as follows. The inverse quantization/inverse transform unit 311 switches the derivation method of ls[][] depending on whether the dependent quantization is valid and the transform skip is invalid or otherwise.

if (sh_dep_quant_used_flag && !transform_skip_flag)
ls[x][y] = (m[x][y]*levelScale[rectNonTsFlag][(qP+1)%6]) << ((qP+1)/6)
else
ls[x][y] = (m[x][y]*levelScale[rectNonTsFlag][qP%6]) << (qP/6)
rectNonTsFlag = (transform_skip_flag==0 && (((Log2(nTbW)+Log2(nTbH)) & 1)==1))
? Ten
bdShift1 = transform_skip_flag==1 ? 10 : BitDepth+rectNonTsFlag+((Log2(nTbW)+Log2(nTbH))/2)-5+sh_dep_quant_used_flag
Note that when expanding the range of coefficients as described later, the scaling shift value bdShift1 may be derived by the following equation.
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - log2TransformRange + sh_dep_quant_used_flag
(if not dependent quantization)
In the case of non-dependent quantization, the inverse quantization unit 3111 (scaling unit 31111, transform coefficient clipping unit 31112) uniquely derives transform coefficients from the quantized transform coefficients and quantization parameters. For example, a transform coefficient value d is derived.

TransCoeffLevel[x0][y0][cIdx][xC][yC] = AbsLevel[xC][yC] * (1-2*coeff_sign_flag[n]) <expression TC_REGULAR>
dz[xC][yC] = TransCoeffLevel[x0][y0][cIdx][xC][yC]
ls[x][y] = (m[x][y] * levelScale[rectNonTsFlag][qP%6]) << (qP / 6)
dnc[xC][yC] = (dz[xC][yC]*ls[xC][yC]+((1<<bdShift1)>>1)) >> bdShift1
d[xC][yC] = Clip3(CoeffMin, CoeffMax, dnc[xC][yC])
Here, AbsLevel is the quantized transform coefficient value, and ls and bdShift1 are variables derived from the quantization parameter qP. CoeffMin and CoeffMax are the clipping minimum and maximum values, which are derived from the following equations.

CoeffMin = -(1 << log2TransformRange)
CoeffMax = (1 << log2TransformRange) - 1
log2TransformRange indicates the range of transform coefficients. If log2TransformRange=15 then CoeffMin = -(1<<15), CoeffMax = (1<<15)-1.

The range of transform coefficients log2TransformRange may be set depending on BitDepth by:

log2TransformRange = extended_precision_processing_flag ? Max(15, BitDepth +
BDOFFSET) : 15 (equation R-1)
You may also limit the maximum length of log2TransformRange as follows:
log2TransformRange = extended_precision_processing_flag ? Min(Max(15, BitDepth +
BDOFFSET), 20) : 15 (equation R-1A)
It may be further restricted when bitDepth>10.
log2TransformRange = extended_precision_processing_flag ? Min(Max(15, BitDepth +
BDOFFSET), 20) : 15 (equation R-1B)
Here, BDOFFSET is a fixed value for calculating the range of transform coefficients, and 4, 5, 6, etc. are appropriate. extended_precision_processing_flag is a flag indicating whether to use an extended range as the range of transform coefficients.

(for dependent quantization)
On the other hand, dependent quantization comprises two quantizers with different levels, as shown in FIG. The TU decoding unit 3024 (residual decoding unit 30242) switches between four states QState using the parity of the intermediate values (AbsLevelPass1, AbsLevel) of the quantized transform coefficients. Then, quantization/inverse quantization is performed according to the QState.

QState = QStateTransTable[QState][AbsLevelPass1[xC][yC] & 1]
TransCoeffLevel[x0][y0][cIdx][xC][yC] = (2*AbsLevel[xC][yC]-(QState>1 ? 1 : 0)) * (1-2*coeff_sign_flag[n]) < Formula TC_DQ>
dz[xC][yC] = TransCoeffLevel[x0][y0][cIdx][xC][yC]
The inverse quantization unit 3111 (scaling unit 31111) performs scaling using the variable ls derived using the quantization parameter.

ls[x][y] = (m[x][y] * levelScale[rectNonTsFlag][(qP+1) % 6]) << ((qP+1) / 6
dnc[xC][yC] = (dz[xC][yC]*ls[xC][yC]+((1<<bdShift1)>>1)) >> bdShift1
The inverse quantization unit 3111 (transform coefficient clipping unit 31112) clips transform coefficients between CoeffMin and CoeffMax.

d[xC][yC] = Clip3(CoeffMin, CoeffMax, dnc[xC][yC])
Here, QState is the state, QStateTransTable[][] is the table used for state transition, for example QStateTransTable[][]={{0,2},{2,0},{1,3},{3,1 }}.

QState may be derived by the following formula without using QStateTransTable[][].

QState = (32040 >> ((QState << 2) + ((AbsLevelPass1[xC][yC] & 1) << 1))) &
3
Depending on the value of QState, different TransCoeffLevel (or d) can be derived even if the AbsLevel is the same. Since QState is derived by referring to the quantized transform coefficient value decoded one before, compared to general scalar (inverse) quantization, (inverse) quantization with better coding efficiency using correlation between coefficients can be made

(inverse transformation)
The inverse quantization/inverse transform unit 311 is composed of an inverse quantization unit 3111 and an inverse transform unit 3112 . The inverse transforming unit 3112 includes an inverse core transforming unit 31123 and a non-separating transforming unit (LFNST unit, secondary transforming unit).

The inverse core transform unit 31123 (vertical transform unit 311231) performs vertical transform on the obtained residual d[][].

e[x][y] = Σ(transMatrix[y][j]×d[x][j]) (j = 0..nTbS - 1)
The inverse core transform unit 31123 (intermediate value clipping unit 311232) shifts and clips the first intermediate value e[x][y] to derive the second intermediate value g[x][y].

g[x][y] = Clip3(CoeffMin, CoeffMax, (e[x][y] + 64) >> 7)
CoeffMin and CoeffMax are the minimum and maximum clipping values.

The inverse core transform unit 31123 (horizontal transform unit 311233) uses transform coefficients d[ ][ ] or modified transform coefficients d[
][ ] is transformed to derive the prediction error r[][] (S5).

r[x][y] = Σ(TransMatrix[x][j]×g[j][y]) (j = 0..nTbS - 1)
Then, r[][] is shifted according to the bit depth (BitDepth) to derive an error resSamples[][] with the same accuracy as the predicted image derived by the predicted image generation unit 308. For example, the shift is expressed as:

resSamples[x][y] = (r[x][y] + (1 << (bdShift2 - 1))) >> bdShift2 (expression BD-1)
Here, the shift value after transformation bdShift2=5+log2TransformRange-BitDepth is derived. In the case of (Formula R-1),
bdShift2 = extended_precision_processing_flag ? (5 + BDOFFSET) : Max(20 - BitDepth, 0)
The above shift value bdShift2 is used when both of the transform sizes nTbW and nTH are greater than 1; You can increase the value by 1.

bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + log2TransformRange - BitDepth : 6 + log2TransformRange - BitDepth
FIG. 18 is a diagram showing residual decoding, inverse quantization, and inverse transform processing. As described above, the residual dz[ ] is decoded by the TU decoding unit 3024 as a residual with a width of 1<<(log2CoeffRange+1). Inverse quantization section 3111 performs inverse quantization to obtain dnc[][]. The inverse quantization unit 3111 further shifts to the right by bdShift1 and limits the variable range to the width of 1<<(log2TransformRange+1) by clipping to obtain d[][]. The inverse transform unit 3112 vertically transforms d[][], right-shifts it by 7, and limits the range of the variable to a width of 1<<(log2TransformRange+1) by clipping to obtain g[][]. The inverse transforming unit 3112 horizontally transforms the obtained g[][], and obtains a difference image r[][] by right-shifting bdShift2. In the following embodiment, when the depth is high, the conversion accuracy is improved by reducing the shift number of bdShift1 and increasing the shift number of bdShift2, but the clip range needs to be widened accordingly. be. We also need to increase the width of the residuals before entering the transform. The following embodiments are characterized by how the range and shift values are set.

(Embodiment 1: Limitation of transform size according to range of transform coefficients)
The video encoding device 11 and the video decoding device 10 of the present embodiment limit the maximum size of the transform block when the flag extended_precision_processing_flag indicating that the range of transform coefficients is extended indicates validity (for example, 1). good too.

FIG. 20 is a flowchart showing processing for limiting the maximum size of transform blocks according to extended_precision_processing_flag. In S3021, if the extended_precision_processing_flag is not valid, transition to S3022 and do not limit the maximum size of the transform block. Conversely, if extended_precision_processing_flag is valid, the process proceeds to S3023 and limits the maximum size of transform blocks.

For example, when extended_precision_processing_flag is enabled, the parameter decoding unit 302 may clip the maximum transform block size MaxTbSizeY to 32 or less (the logarithm of MaxTbSizeY MaxTbLog2SizeY is 5 or less).
MaxTbSizeY = extended_precision_processing_flag ? Min(32, MaxTbSizeY) : MaxTbSizeY
MaxTbLog2SizeY = extended_precision_processing_flag ? Min(5, MaxTbLog2SizeY) : MaxTbLog2SizeY
FIG. 21 is a flowchart showing processing for limiting transform size according to extended_precision_processing_flag. In S3021A1, if extended_precision_processing_flag is not valid, transition to S3021A2. Then, it is determined whether sps_max_luma_transform_size_TX_flag is valid. sps_max_luma_transform_size_TX_flag is a flag indicating whether or not to use the maximum size TX (TX is 64, 128, for example) of a predetermined transform block. If the determination is valid, the process proceeds to S3022A and sets the maximum transform block size to TX. If extended_precision_processing_flag is valid in S3021A or sps_max_luma_transform_size_TX_flag is not valid in S3021A2, transition to S3023A and set the maximum transform block size to TX/2.

More specifically, the parameter decoding unit 302 sets the variable MaxTbSizeY (
A logarithmic value (MaxTbLog2SizeY) may be set to use a predetermined maximum size (eg 64, logarithmic value is 6). That is, parameter decoding section 302 may use maximum size TX only when sps_max_luma_transform_size_TX_flag decoded from encoded data is valid and extended_precision_processing_flag is not valid. The case of TX=64 is shown below.
MaxTbLog2SizeY = sps_max_luma_transform_size_64_flag && extended_precision_processing_flag==0 ? 6 : 5
MaxTbSizeY = 1 << MaxTbLog2SizeY
As described above, when sps_max_luma_transform_size_TX_flag is valid and extended_precision_processing_flag is invalid, parameter decoding section 302 sets a value TX indicating the maximum size of a predetermined transform block (for example, 64, the corresponding value is 6) as the maximum size MaxTbSizeY. good.
Otherwise, MaxTbSizeY may be set to a value less than a predetermined maximum size (eg, 32, logarithmic value of 5).

Also, the logarithmic value of the predetermined maximum size is not limited to 6 (64x64), but may be 7 (128x128) or 8 (256x256). For example, in a configuration with a predetermined maximum size of 128x128 transform, the parameter decoding unit 302 may decode sps_max_luma_transform_size_128_flag from the encoded data and derive the maximum size as follows. sps_max_luma_transform_size_128_flag is a flag indicating whether to use 128 as the maximum size of a given transform block.
MaxTbLog2SizeY = sps_max_luma_transform_size_128_flag && extended_precision_processing_flag==0 ? 7 : 6
MaxTbSizeY = 1 << MaxTbLog2SizeY
For example, in a configuration with a predetermined maximum size of 256x256 transform, the parameter decoding unit 302 may decode sps_max_luma_transform_size_256_flag from the encoded data and derive the maximum size as follows. sps_max_luma_transform_size_256_flag is a flag indicating whether to use 256 as the maximum size of a given transform block.
MaxTbLog2SizeY = sps_max_luma_transform_size_256_flag && extended_precision_processing_flag==0 ? 8 : 7
MaxTbSizeY = 1 << MaxTbLog2SizeY
TU decoding section 3024 derives maximum transform sizes maxTbWidth and maxTbHeight according to the value of cIdx, SubWidthC and SubHeightC as follows. cIdx is an index indicating a color component. SubWidthC and SubHeightC are variables indicating color subsampling of color difference.
maxTbWidth = (cIdx==0) ? MaxTbSizeY : MaxTbSizeY / SubWidthC
maxTbHeight = (cIdx==0) ? MaxTbSizeY : MaxTbSizeY / SubHeightC
FIG. 22 is a syntax table showing the decoding process of TU decoding section 3024. FIG. The TU decoding unit 3024 recursively decodes the transform tree (transform_tree( ) in the figure) when decoding the residual of the CU. Specifically, if any of the transformation tree sizes tbWidth and tbHeight exceeds the maximum size MaxTbSizeY and MaxTbSizeY (tbWidth>MaxTbSizeY || tbHeight>MaxTbSizeY), set the width or height of the transformation tree to half, Decode the transform tree again. Conversely, when both tbWidth and tbHeight do not exceed MaxTbSizeY and MaxTbSizeY, the TU decoding unit 3024 decodes a transform unit (transform_unit in the drawing) of size tbWidth and tbHeight.

transform_unit(x0, y0, tbWidth, tbHeight, treeType, 0, chType)
transform_unit is the encoded data representing the residual. Note that the TU decoding unit 3024 derives a flag verSplitFirst that indicates whether or not to split the vertical direction first, using the following formula. can be 1/2.

verSplitFirst = (tbWidth>MaxTbSizeY &&tbWidth>tbHeight) ? 1 : 0
trafoWidth = verSplitFirst ? (tbWidth/2) : tbWidth
trafoHeight = !verSplitFirst ? (tbHeight/2) : tbHeight
The TU decoding unit 3024 decodes the first transform_tree obtained by dividing the transform_tree into two using the derived transform sizes trafoWidth, trafoHeight and upper left coordinates (x0, y0).

transform_tree(x0, y0, trafoWidth, trafoHeight, treeType, chType)
Furthermore, the TU decoding unit 3024 uses the derived transform sizes trafoWidth and trafoHeight and the upper left coordinates (x0+tbWidth, y0) or (x0, y0+tbHeight) to divide the transform_tree into two Decode the second transform_tree. When verSplitFirst is 1, the upper left coordinate is (x0+tbWidth, y0), otherwise (x0, y0+tbHeight). In this way, when decoding the transform_unit, which is the coded data of the residual of the CU, the CU is recursively divided using the nodes of the transform_tree. size) can be limited to the maximum size TX or less derived from sps_max_luma_transform_size_TX_flag and extended_precision_processing_flag. The entropy decoding unit 301 decodes a quantized transform block of size tbWidth and tbHeight indicated by transform_unit.

An inverse quantization/inverse transform unit 311 inversely quantizes a quantized transform block of size tbWidth and tbHeight. Then, transform is performed using the transform basis specified by the transform size.

According to the above configuration, when the range of transform coefficients is large (extended_precision_processing_flag==1), the transform size is limited to a predetermined size or less, so that the size of the hardware circuit is reduced when the range of transform coefficients is large. be able to. This effect can always be obtained regardless of the quantization parameter. Also, when the range of transform coefficients does not increase (extended_precision_processing_flag==0), the transform size is not limited to a predetermined size or less, and a large size can be used. Therefore, if the range of transform coefficients does not increase, there is an effect that the coding efficiency does not decrease.

(Embodiment 2: When the size of the transform is equal to or larger than the predetermined size, the range of the transform is not extended)
The video encoding device 11 and the video decoding device 10 of this embodiment may extend the range of transform coefficients according to the size of the transform block.

FIG. 23 is a flowchart showing processing for extending the transform coefficient range according to extended_precision_processing_flag and transform block sizes nTbW and nTbH. In S3021B1, if extended_precision_processing_flag is valid, transition to S3021B2 and determine whether the transform block size is less than a predetermined maximum size TX. If the determination is YES, the process transitions to S3022B to extend the transform coefficient range. For example, set log2TransformRange=bitDepth+BDOFFSET depending on the bit depth. log2TransformRange is a parameter indicating the range of transform coefficients. If the extended_precision_processing_flag is not valid in S3021B1, or if the transform block size is not less than the predetermined maximum size TX in S3021B2, the process proceeds to S3023B and does not extend the transform coefficient range. For example, set log2TransformRange=15.

TU decoding section 302, when transform_unit or residual block size tbWidth and tbHeight are equal to or greater than a predetermined maximum size TX (or equal to a predetermined maximum size), even if extended_precision_processing_flag is enabled (==1), the range of transform coefficients is Do not increase the value of the indicated variable. extended_precision_processing_flag is a flag indicating extending the range of transform coefficients.

As described below, TU decoding section 302 determines residual block sizes nTbW, nTbHextended_precision_
Depending on processing_flag, derive log2TransformRange as follows.

log2TransformRange = extended_precision_processing_flag &&nTbW<TX&&nTbH<TX ? Max(15, BitDepth+BDOFFSET) : 15 (Formula R-1TXA)
Also, TU decoding section 302 may be derived by the following equation.

log2TransformRange = extended_precision_processing_flag &&nTbW<=TX/2&&nTbH<=TX/2 ? Max(15, BitDepth+BDOFFSET) : 15 (Formula R-1TXB)
That is, only when nTbW and nTbH are less than a predetermined size TX, the range of actual transform coefficients is expanded depending on the bit depth.
Here TX may be 64, 128, 256, and so on.

Also, when TX=64, the parameter decoding unit 302 may decode sps_max_luma_transform_size_64_flag from the encoded data and derive the maximum value of the transform size as follows. sps_max_luma_transform_size_64_flag is a flag indicating whether to use 64 as the maximum size of a given transform block.
MaxTbLog2SizeY = sps_max_luma_transform_size_64_flag ? 6 : 5
That is, TU decoding section 302 may be derived by the following equation.

log2TransformRange = extended_precision_processing_flag &&nTbW<64&&nTbH<64 ? Max(15, BitDepth+BDOFFSET) : 15 (Formula R-1T64A)
Also, TU decoding section 302 may be derived by the following equation.

log2TransformRange = extended_precision_processing_flag &&nTbW<=32&&nTbH<=32 ? Max(15, BitDepth+BDOFFSET) : 15 (Formula R-1T64B)
As another example, if TX=MaxTbLog2SizeY, the parameter decoding unit 302 may decode sps_max_luma_transform_size_TX_flag from the encoded data to derive the maximum transform size as follows. sps_max_luma_transform_size_TX_flag is a flag indicating whether to use TX as the maximum size of a given transform block.
MaxTbLog2SizeY = sps_max_luma_transform_size_TX_flag ? log2(TX) : log2(TX)-1
Also, TU decoding section 302 may derive the maximum value of the transform size using the following equation.

log2TransformRange = extended_precision_processing_flag &&nTbW<(1<<MaxTbLog2SizeY)&&nTbH<(1<<MaxTbLog2SizeY) ? Max(15, BitDepth+BDOFFSET) : 15 (Formula R-1TV)
As already explained, the transform coefficient clip ranges CoeffMin, CoeffMax, bdShift1, bdShift2, maxPreExtLen may also be derived according to log2TransformRange.

As described above, even when the range of transform coefficients is extended, the maximum value of the range of transform coefficients may be clipped to a predetermined value (here, 20) or less. That is, Max(15, BitDepth+BDOFFSET):15 described in the embodiment may be replaced with Min(Max(15, BitDepth+BDOFFSET), 20):15.

Also, derive the variable ExtendedPrecisionFlag according to nTbW, nTbH and extended_precision_processing_flag as follows, and use ExtendedPrecisionFlag to log2TransformRange
, CoeffMin, CoeffMax, bdShift1, bdShift2maxPreExtLen may be derived. ExtendedPrecisionFlag is a variable that indicates whether or not to actually extend the range of transform coefficients.

ExtendedPrecisionFlag = extended_precision_processing_flag &&nTbW<TX&&nTbH<TX ? l : 0
Also, the following may be used.

ExtendedPrecisionFlag = extended_precision_processing_flag &&nTbW<=TX/2&&nTbH<=TX/2 ? l : 0
Also, the following may be used.

ExtendedPrecisionFlag = extended_precision_processing_flag &&nTbW<64&&nTbH<64 ? l : 0
ExtendedPrecisionFlag = extended_precision_processing_flag &&nTbW<=32&&nTbH<=32 ? l : 0
In this case, TU decoding section 302 derives log2TransformRange using ExtendedPrecisionFlag derived according to nTbW and nTbH.

log2TransformRange = ExtendedPrecisionFlag ? Max(15, BitDepth+BDOFFSET) : 15
(Formula R-1TV)
As already explained, when extending the range of transform coefficients, the maximum value of the range of transform coefficients may be clipped below a predetermined value (here, 20).
log2TransformRange = ExtendedPrecisionFlag? Min(Max(15, BitDepth+BDOFFSET), 20) : 15
According to the above configuration, even if the transform size is large, if the syntax element indicating the range extension of transform coefficients indicates that it is valid (extended_precision_processing_flag==1), the transform size that is actually used is reduced to a predetermined size or less. Restrict. Therefore, the scale of the hardware circuit can be reduced. This effect can always be obtained regardless of the quantization parameter. In addition, when the range of transform coefficients does not increase (extended_precision_processing_flag==0), the transform size is not limited to a predetermined size or less, and large sizes can be used, so there is no reduction in coding efficiency. Play.

(Embodiment 3: Bitstream restriction)
The encoded data encoded by the video encoding device 11 of the present embodiment and the encoded data decoded by the video decoding device 10 limit the maximum size of the transform block when extended_precision_processing_flag indicates valid (for example, 1). may be provided as a conformance constraint. extended_precision_processing_flag is a flag indicating extending the range of transform coefficients.

Specifically, the video decoding device 10 may decode encoded data having the following bitstream restrictions as conformance restrictions.

If extended_precision_processing_flag is 1, then transform blocks with nTbW=64 or nTbH=64 shall not be used.
Conformance restrictions may be:

If extended_precision_processing_flag is 1, both transform block sizes nTbW and nTbH must be 32 or less.
Alternatively, one of the following conformance restrictions:

If extended_precision_processing_flag is 1, sps_max_luma_transform_size_TX_flag MUST be 0.

extended_precision_processing_flag must be 0 if sps_max_luma_transform_size_TX_flag is 1.

Disallows both sps_max_luma_transform_size_TX_flag and extended_precision_processing_flag to be 1.

Either sps_max_luma_transform_size_TX_flag or extended_precision_processing_flag must be 0.

According to the moving image decoding device 10 and the moving image encoding device 10 configured as described above, when the transform size is equal to or less than a predetermined size, only the encoded data with the range of transform coefficients extended can be decoded and encoded. , the scale of the hardware circuit can be reduced.

(Embodiment 4: Configuration using transform coefficient dependent shift only when block size is large)
FIG. 24 is a flowchart showing a process of switching between extending the transform coefficient range and variably setting the shift value according to the transform coefficients according to the extended_precision_processing_flag and the transform block sizes nTbW and nTbH. In S3021C1, if the extended_precision_processing_flag is not valid, transition to S3024C and do not extend the transform coefficient range. For example, in S3024C, TU decoding section 3024 or TU encoding section 1114 sets variable log2TransformRange indicating the range of transform coefficients to a normal value of 15. In this case, the shift values bdShift1 and bdShift2 are derived without depending on the range of transform coefficients of the residual to be decoded (for example, Log2ResidualRange described later). Log2ResidualRange is a variable that indicates the magnitude of residuals (transform coefficients) obtained by decoding encoded data, that is, a variable that indicates the actual range of transform coefficients before scaling.

log2TransformRange = 15
CoeffMin = -(1 << log2TransformRange)
CoeffMax = (1 << log2TransformRange) - 1
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - log2TransformRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + log2TransformRange - BitDepth : 6 + log2TransformRange - BitDepth
In S3021C1, if extended_precision_processing_flag is valid, transition to S3021C2 to determine whether the transform block size is less than a predetermined maximum size TX. If the determination in S3021C2 is YES, the process transitions to S3022C to extend the transform coefficient range according to the bit depth. For example, in S3022C, TU decoding section 3024 or TU encoding section 1114 sets log2TransformRange=bitDepth+BDOFFSET. Not limited to (Formula R-1), using variations such as (Formula R-1A), (Formula R-1B)... (Formula R-1TXA)... (Formula R-1TV) good too. Then, the shift values bdShift1 and bdShift2 are derived independently of the transform coefficients.

log2TransformRange = bitDepth + BDOFFSET
CoeffMin = -(1 << log2TransformRange)
CoeffMax = (1 << log2TransformRange) - 1
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - log2TransformRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + log2TransformRange - BitDepth : 6 + log2TransformRange - BitDepth
If the determination in S3021C2 is NO, that is, if the size of the transform block is not less than the predetermined maximum size TX, transition to S3023C, do not extend the transform coefficient range log2TransformRange, and change the shift values bdShift1 and bdShift2 according to the transform coefficients. derive In S3023C, TU decoding section 3024 or TU encoding section 1114 uses the variable Log2ResidualRange derived according to the transform coefficients before scaling to derive shift values as follows, for example.

log2TransformRange = 15
CoeffMin = -(1 << log2TransformRange)
CoeffMax = (1 << log2TransformRange) - 1
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - Log2ResidualRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + Log2ResidualRange - BitDepth : 6 + Log2ResidualRange - BitDepth
In summary, TU decoding section 3024 or TU encoding section 1114 may derive parameters using the following equations. maxRange is a variable derived from the magnitude of residuals (transform coefficients), which will be described later.

log2TransformRange = extended_precision_processing_flag &&(nTbW<TX&&nTbH<TX)&&bitDepth>10 ? bitDepth+6 : 15
CoeffMin = -(1 << log2TransformRange)
CoeffMax = (1 << log2TransformRange) - 1
Log2ResidualRange = extended_precision_processing_flag &&(nTbW>=TX || nTbH>=TX)&&bitDepth>10 ? Ceil(maxRange) : log2TransformRange
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - Log2ResidualRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + Log2ResidualRange - BitDepth : 6 + Log2ResidualRange - BitDepth
According to the configuration of FIG. 24 above, only when the block size is large, the complexity and latency can be reduced by analyzing the value range of the transform coefficient and setting the shift value of the transform coefficient according to the actual size of the transform coefficient. There is an effect that it can be reduced. In general, the larger the block size, the more room there is in processing. If the block size is too small for processing, the range of transform coefficients is not analyzed. Also, when the block size is large, the range of transform coefficients is not expanded, so it is possible to prevent complexity associated with latency from increasing.

(Embodiment 5: Configuration using transform coefficient dependent shift only when block size is small)
The fifth embodiment has a configuration similar to that of the fourth embodiment, but differs in the conditions when transform coefficient dependent shift (shift according to the range of the magnitude of the actual transform coefficient) is used.

FIG. 25 is another flowchart showing the process of switching between extending the transform coefficient range and variably setting the shift value according to the transform coefficients according to the extended_precision_processing_flag and the transform block sizes nTbW and nTbH. In S3021D1, if extended_precision_processing_flag is not valid, TU decoding section 3024 or TU encoding section 1114 transitions to S3024D and does not extend the transform coefficient range. For example, set log2TransformRange=15.

log2TransformRange = 15
CoeffMin = -(1 << log2TransformRange)
CoeffMax = (1 << log2TransformRange) - 1
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - log2TransformRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + log2TransformRange - BitDepth : 6 + log2TransformRange - BitDepth
In this case, TU decoding section 3024 or TU encoding section 1114 derives shift values independently of transform coefficients.

In S3021D1, if extended_precision_processing_flag is valid, transition to S3021D2 to determine whether the transform block size is less than a predetermined maximum size TX. If the determination in S3021D2 is NO, the process proceeds to S3022D to extend the conversion coefficient range. In S3022D, TU decoding section 3024 or TU encoding section 1114 sets log2TransformRange=bitDepth+BDOFFSET according to the bit depth, for example, and derives a shift value without depending on the transform coefficient.

log2TransformRange=bitDepth + BDOFFSET
CoeffMin = -(1 << log2TransformRange)
CoeffMax = (1 << log2TransformRange) - 1
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - Log2TransformRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + log2TransformRange - BitDepth : 6 + log2TransformRange - BitDepth
If the determination in S3021D2 is YES, that is, if the transform block size is less than the predetermined maximum size TX, the TU decoding unit 3024 or TU encoding unit 1114 does not extend the transform coefficient range log2TransformRange, and the shift value Derive bdShift1 and bdShift2. Specifically, in S3023D, TU decoding section 3024 or TU encoding section 1114, according to Log2ResidualRange, for example,
It is derived as follows. Log2ResidualRange is a variable derived with reference to transform coefficients decoded from encoded data.

log2TransformRange = 15
CoeffMin = -(1 << log2TransformRange)
CoeffMax = (1 << log2TransformRange) - 1
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - Log2ResidualRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + Log2ResidualRange - BitDepth : 6 + Log2ResidualRange - BitDepth
However, when the bit depth is 16 bits, it is difficult to limit to 16 bits, so the range log2TransformRange of transform coefficients may be set as follows.

log2TransformRange = BitDepth == 16 ? 16: 15
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - Log2ResidualRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + Log2ResidualRange - BitDepth : 6 + Log2ResidualRange - BitDepth
In summary, TU decoding section 3024 or TU encoding section 1114 may derive the shift value in the following equation. Here, maxRange is a variable derived from the magnitude of residuals (transform coefficients), which will be described later.

log2TransformRange = extended_precision_processing_flag &&(nTbW>=TX&&nTbH>=TX)&&bitDepth>10 ? bitDepth + 6 : 15
CoeffMin = -(1 << log2TransformRange)
CoeffMax = (1 << log2TransformRange) - 1
Log2ResidualRange = extended_precision_processing_flag &&(nTbW<TX&&nTbH<TX)&&bitDepth>10 ? Ceil(maxRange) : log2TransformRange
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - Log2ResidualRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + Log2ResidualRange - BitDepth : 6 + Log2ResidualRange - BitDepth
According to the configuration of FIG. 25, the size of the temporary memory required for analysis can be reduced by variably setting the shift value of the transform coefficient from the analysis of the value range of the transform coefficient only when the block size is small. produce the effect possible. In general, the smaller the block size, the smaller the effect of memory increase. Also, when the block size is small, the range of transform coefficients is expanded, so that the coding efficiency can be kept high.

The configuration of FIG. 24 and the configuration of FIG. 25 are preferably selected depending on whether the design emphasizes latency or the design emphasizes memory size. In terms of latency, the configuration in FIG. 24 is fast, and in terms of memory size, the configuration in FIG. 25 is a small memory.

(More specific configuration example of Embodiment 4 and Embodiment 5)
FIG. 26 is a block diagram showing the configuration of the TU decoding section 3024 and the inverse quantization/inverse transform section 311 of the embodiment of the present invention. The RRC section 30241 of the TU decoding section 3024 is composed of a residual decoding section 30242 , a residual range derivation section 30243 and a transform coefficient range derivation section 30244 . The inverse quantization/inverse transform unit 311 is composed of an inverse quantization unit 3111 (scaling unit 31111, coefficient clip unit 31112) and an inverse core transform unit 31123 (vertical transform unit 31123, intermediate clip unit 31124, horizontal transform unit 31125). be. Since the residual decoding unit 30242 and the inverse quantization/inverse transform unit 311 have already been explained, the operations of the residual range derivation unit 30243 and the transform coefficient range derivation unit 30244 will be explained here.

The residual range derivation unit 30243 derives the range of transform coefficients before scaling in order to derive the scaling shift value. Specifically, the maximum value maxLevel of the absolute values AbsLevel[xC][yC] of the transform coefficients of the residual block of width nTbW and height nTbH may be derived.

maxLevel = 0
for (yC in 0..nTbH-1) {
for (xC in 0..nTbW-1) {
if (maxLevel < AbsLevel[xC][yC]) {
maxLevel = AbsLevel
}
}
}
Alternatively, the residual range deriving unit 30243 derives the maximum absolute value maxLevel of the transform coefficients TransCoeffLevel[x0][y0][cIdx][xC][yC] of the residual block of width nTbW and height nTbH. good too.

maxLevel = 0
for (yC in 0..nTbH-1) {
for (xC in 0..nTbW-1) {
if (maxLevel < Abs(TransCoeffLevel[x0][y0][cIdx][xC][yC]) {
maxLevel = Abs(TransCoeffLevel[x0][y0][cIdx][xC][yC])
}
}
}
Note that when the range is expressed by an exponent of 2, it usually takes a range from -N to N-1. may

maxLevel = 0
for (yC in 0..nTbH-1) {
for (xC in 0..nTbW-1) {
if (TransCoeffLevel[x0][y0][cIdx][xC][yC] > 0) {
if maxLevel < TransCoeffLevel[x0][y0][cIdx][xC][yC])
maxLevel = TransCoeffLevel[x0][y0][cIdx][xC][yC]
}
}
if (TransCoeffLevel[x0][y0][cIdx][xC][yC] < 0) {
if maxLevel < -TransCoeffLevel[x0][y0][cIdx][xC][yC]-1)
maxLevel = -TransCoeffLevel[x0][y0][cIdx][xC][yC]-1
}
}
}
}
Also, the residual range derivation unit 30243 may derive maxLevel using the following.

maxLevel = 0
for (yC in 0..nTbH-1) {
for (xC in 0..nTbW-1) {
if (maxLevel < TransCoeffLevel[x0][y0][cIdx][xC][yC]) {
maxLevel = TransCoeffLevel[x0][y0][cIdx][xC][yC]
}
if (maxLevel < -TransCoeffLevel[x0][y0][cIdx][xC][yC]-1) {
maxLevel = -TransCoeffLevel[x0][y0][cIdx][xC][yC]-1
}
}
}
may be used. In this case, since maxLevel=N-1 in the range of -N to N-1, maxLevel=maxLevel+1 may be set at the end.
Also, the residual range derivation unit 30243 may derive maxLevel using the following.

maxLevel = 0
for (yC in 0..nTbH-1) {
for (xC in 0..nTbW-1) {
if (maxLevel < TransCoeffLevel[x0][y0][cIdx][xC][yC]+1) {
maxLevel = TransCoeffLevel[x0][y0][cIdx][xC][yC]+1
}
if (maxLevel < -TransCoeffLevel[x0][y0][cIdx][xC][yC]) {
maxLevel = -TransCoeffLevel[x0][y0][cIdx][xC][yC]
}
}
}
Also, in the above, maxLevel may be obtained for each color component. In this case, an array of maxLevel[cIdx] is used instead of the variable maxLevel. It may also be derived depending on the upper left position (x0, y0) of the block. In this case, use maxLevel[cIdx][x0][y0].

The transform coefficient range derivation unit 30244 derives the logarithmic value Log2ResidualRange of the maximum value of transform coefficients using the following formula.

Log2ResidualRange=Ceil(maxLevel)
Further, the Log2ResidualRange may be kept within a predetermined range by clipping.

Log2ResidualRange = Clip(0, 15, Log2ResidualRange)
Also, when the bit depth is 16 bits, it is difficult to limit to 16 bits, so if the bit depth is 16 bits, it may be limited to 16 bits.

Log2ResidualRange = Clip(0, (BitDepth == 16 ? 16 : 15), Log2ResidualRange)
Transform coefficient range derivation unit 30244 uses Log2ResidualRange to obtain shift values bdShift1, bdSh
ift2 may be derived by the following equation.

bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - Log2ResidualRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + Log2ResidualRange - BitDepth : 6 + Log2ResidualRange - BitDepth
The scaling unit 31111 of the inverse quantization unit 3111 scales the residual using the shift by bdShift1. Similarly, the horizontal transform unit 31123 of the inverse core transform unit 31123 also performs a shift by bdShift2 to derive a residual image with BitDepth accuracy.

Note that adaptive processing will be described with reference to FIG.

When extended_precision_processing_flag==0, the transform coefficient range deriving unit 30244 derives the transform coefficient range log2TransformRange as a fixed value independent of BitDepth using, for example, the following formula (S3024C).

log2TransformRange = 15
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - log2TransformRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + log2TransformRange - BitDepth : 6 + log2TransformRange - BitDepth
When extended_precision_processing_flag==1 and the transform block size is less than the predetermined size, the transform coefficient range deriving unit 30244 extends log2TransformRange to a value dependent on BitDepth,
For example, it is derived using the following formula (S3022C).

log2TransformRange = BitDepth + 6
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - log2TransformRange + sh_dep_quant_used_flag = rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 4 + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + log2TransformRange - BitDepth : 6 + log2TransformRange - BitDepth
When extended_precision_processing_flag==1 and the transform block size is equal to or greater than a predetermined size, the transform coefficient range deriving unit 30244 does not extend log2TransformRange to a BitDepth-dependent value and sets it to a fixed value of 15. Then, the shift value is derived according to the residual range (for example, the logarithm of the maximum residual value Log2ResidualRange) (S3023C). For example, it is derived using the following formula.

log2TransformRange = 15
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - Log2ResidualRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + Log2ResidualRange - BitDepth : 6 + Log2ResidualRange - BitDepth
However, as described above, when BitDepth==16, the range may be extended as follows.

log2TransformRange = BitDepth == 16 ? 16 : 15
Adaptive processing will be described with reference to FIG.

When extended_precision_processing_flag==0, the transform coefficient range derivation unit 30244 derives the transform coefficient range log2TransformRange as a fixed value without extending it to a BitDepth-dependent value, for example, using the following formula (S3024D).

log2TransformRange = 15
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - log2TransformRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + log2TransformRange - BitDepth : 6 + log2TransformRange - BitDepth
When extended_precision_processing_flag==1 and the transform block size is equal to or greater than a predetermined size, the transform coefficient range derivation unit 30244 extends the transform coefficient range log2TransformRange to a BitDepth-dependent value using the following formula, for example, using the following formula: (S3022D).

log2TransformRange = BitDepth + 6
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - log2TransformRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + log2TransformRange - BitDepth : 6 + log2TransformRange - BitDepth
When extended_precision_processing_flag==1 and the transform block size is less than the predetermined size, the transform coefficient range deriving unit 30244 uses the following formula to set the transform coefficient range log2TransformRange to a fixed value without extending it to a BitDepth-dependent value, while shifting A value is derived according to the range of residuals Log2ResidualRange (S3023D). For example, it is derived using the following formula.

log2TransformRange = 15
bdShift1 = BitDepth + rectNonTsFlag + (((Log2(nTbW)+Log2(nTbH))/2) + 10 - Log2ResidualRange + sh_dep_quant_used_flag
bdShift2 = (nTbH>1 &&nTbW>1) ? 5 + Log2ResidualRange - BitDepth : 6 + Log2ResidualRange - BitDepth
However, as described above, when BitDepth==16, the range may be extended as follows.

log2TransformRange = BitDepth == 16 ? 16 : 15
The inverse quantization/inverse transform unit 311 scales (inverse quantizes) the quantized transform coefficients input from the entropy decoding unit 301 to obtain transform coefficients d[][]. These quantized transform coefficients are coefficients that are obtained by quantizing prediction errors by transforming them using DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), etc. in the encoding process. be. When transform_skip_flag is 0, the inverse quantization/inverse transform unit 311 performs inverse frequency transform such as inverse DCT or inverse DST on the scaled transform coefficient d[][], and converts the prediction error res[][] to calculate. When transform_skip_flag is 1, the inverse quantization/inverse transform unit 311 sets res[x][y]=d[x][y]. Inverse quantization/inverse transform section 311 outputs the prediction error to addition section 312 .

Note that the inverse transform and the transform are paired processes, so the transform and the inverse transform may be interpreted by replacing each other. Alternatively, if the inverse transform is called a transform, the transform may be called a forward transform. For example, if the inverse non-separable transform is called the non-separable transform, the non-separable transform may be called the forward non-separable transform. In addition, separate transformations are simply referred to as transformations.

The addition unit 312 adds the predicted image of the block input from the predicted image generation unit 308 and the prediction error input from the inverse quantization/inverse transform unit 311 for each pixel to generate a decoded image of the block.
The adder 312 stores the decoded image of the block in the reference picture memory 306 and also outputs it to the loop filter 305 .

(Configuration of video encoding device)
Next, the configuration of the video encoding device 11 according to this embodiment will be described. FIG. 16 is a block diagram showing the configuration of the video encoding device 11 according to this embodiment. The video encoding device 11 includes a predicted image generation unit 101, a subtraction unit 102, a transformation/quantization unit 103, an inverse quantization/inverse transformation unit 105, an addition unit 106, a loop filter 107, a prediction parameter memory (prediction parameter storage unit , frame memory) 108 , reference picture memory (reference image storage unit, frame memory) 109 , coding parameter determination unit 110 , parameter coding unit 111 and entropy coding unit 104 .

The predicted image generation unit 101 generates a predicted image for each CU, which is an area obtained by dividing each picture of the image T. FIG. The operation of the predicted image generation unit 101 is the same as that of the predicted image generation unit 308 already described, and the description thereof is omitted.

The subtraction unit 102 subtracts the pixel values of the predicted image of the block input from the predicted image generation unit 101 from the pixel values of the image T to generate a prediction error. The subtraction unit 102 transforms and quantizes the prediction error
output to

Transform/quantization section 103 calculates transform coefficients by frequency transforming the prediction error input from subtraction section 102, and derives quantized transform coefficients by quantization. The transform/quantization unit 103 is
The quantized transform coefficients are output to entropy coding section 104 and inverse quantization/inverse transform section 105 .

The inverse quantization/inverse transform unit 105 is the same as the inverse quantization/inverse transform unit 311 (FIG. 10) in the moving image decoding device 31, and the description thereof is omitted. The calculated prediction error is output to addition section 106 .

Entropy coding section 104 receives the quantized transform coefficients from transform/quantization section 103 and the coding parameters from parameter coding section 111 . An encoding parameter is predMode which shows prediction mode, for example. predMode may be either MODE_INTRA indicating intra prediction, MODE_INTER indicating inter prediction, or MODE_INTRA, MODE_INTER, or MODE_IBC indicating intra block copy prediction in which blocks within a screen are copied into a predicted image.

The entropy encoding unit 104 entropy-encodes the division information, prediction parameters, quantized transform coefficients, and the like to generate and output an encoded stream Te.

Parameter coding section 111 includes header coding section 1110, CT information coding section 1111, CU
An encoding unit 1112 (prediction mode encoding unit), an inter prediction parameter encoding unit 112 and an intra prediction parameter encoding unit 113 are provided. CU encoding section 1112 further converts TU encoding section 1114
It has

The general operation of each module will be described below. A parameter encoding unit 111 performs encoding processing of parameters such as header information, division information, prediction information, and quantized transform coefficients.

CT information encoding section 1111 encodes QT, MT (BT, TT) division information and the like from encoded data.

CU encoding section 1112 encodes CU information, prediction information, TU division flags, CU residual flags, and the like.

TU encoding section 1114 encodes the QP update information and the quantized transform coefficients when the TU contains a prediction error. TU encoding section 1114 encodes the residual using TR code and EG(k) code, as described for TU decoding section 3024 . That is, the syntax elements (abs_remainder, dec_abs_level) indicating the absolute value of the residual are encoded using TR code and EG(k) code. The derivation method of the Rice parameter used for encoding, the maximum value of the prefix of the TR code, the maximum length maxPreExtLen of the exp part, and the length truncSuffixLen of the escape part use one of the methods described in the image decoding device. The transformation/quantization unit 103 and the inverse quantization/inverse transformation unit 105 can also use the values of CoeffMin, CoeffMax, bdShift1, and bdShift2 as described in the image decoding device.

That is, as shown in FIG. 26, TU encoding section 1114 calculates the actual range Log2ResidualRange of the values of the transform coefficients in the transform block before inverse quantization/scaling in the decoding device (after quantization/scaling in the encoding device). bdShift1 and bdShift2 may be derived adaptively according to . Furthermore, as shown in FIGS. 24 and 25, depending on the extended_precision_flag and the transform block sizes nTbW and nTbH, the transform coefficient ranges log2TransformRange, bdShift1 and bdShift2 may be derived.

The CT information encoding unit 1111 and the CU encoding unit 1112 supply the entropy encoding unit 104 with syntax elements such as inter prediction parameters, intra prediction parameters, and quantized transform coefficients.

The addition unit 106 adds pixel values of the predicted image of the block input from the predicted image generation unit 101 and prediction errors input from the inverse quantization/inverse transform unit 105 for each pixel to generate a decoded image. The addition unit 106 stores the generated decoded image in the reference picture memory 109 .

A loop filter 107 applies a deblocking filter, SAO, and ALF to the decoded image generated by the adder 106 . Note that the loop filter 107 does not necessarily include the three types of filters described above, and may be configured with only a deblocking filter, for example.

SAO is a filter that adds an offset according to the classification result in sample units, and ALF is a filter that uses the sum of products of transmitted filter coefficients and a reference image (or the difference between a reference image and a target pixel).

The prediction parameter memory 108 stores the prediction parameters generated by the coding parameter determination unit 110 in predetermined positions for each current picture and CU.

The reference picture memory 109 stores the decoded image generated by the loop filter 107 in a predetermined position for each target picture and CU.

Coding parameter determination section 110 selects one set from a plurality of sets of coding parameters. The coding parameter is the above-described QT, BT or TT division information, prediction parameters, or parameters to be coded generated in relation to these. The predicted image generating unit 101 generates predicted images using these coding parameters.

Coding parameter determination section 110 calculates an RD cost value indicating the amount of information and the coding error for each of the plurality of sets. Coding parameter determination section 110 selects a set of coding parameters that minimizes the calculated cost value. As a result, entropy encoding section 104 outputs the selected set of encoding parameters as encoded stream Te. Coding parameter determination section 110 stores the determined coding parameters in prediction parameter memory 108 .

Note that a part of the video encoding device 11 and the video decoding device 31 in the above-described embodiment, for example, the entropy decoding unit 301, the parameter decoding unit 302, the loop filter 305, the prediction image generation unit 308, the inverse quantization/inverse Transformation unit 311, addition unit 312, prediction image generation unit 101, subtraction unit 102, transformation/quantization unit 103, entropy coding unit 104, inverse quantization/inverse transformation unit 105, loop filter 107, coding parameter determination unit 110 , the parameter encoding unit 111 may be realized by a computer. In that case, a program for realizing this control function may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read into a computer system and executed. The “computer system” here is a computer system built in either the moving image encoding device 11 or the moving image decoding device 31, and includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible discs, magneto-optical discs, ROMs and CD-ROMs, and storage devices such as hard disks built into computer systems. Furthermore, "computer-readable recording medium" means a medium that dynamically stores a program for a short period of time, such as a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include a volatile memory inside a computer system that serves as a server or client in that case, which holds the program for a certain period of time. Further, the program may be for realizing a part of the functions described above, or may be capable of realizing the functions described above in combination with a program already recorded in the computer system.

Also, part or all of the moving image encoding device 11 and the moving image decoding device 31 in the above-described embodiments may be realized as an integrated circuit such as LSI (Large Scale Integration). Each functional block of the moving image encoding device 11 and the moving image decoding device 31 may be individually processorized, or may be partially or entirely integrated and processorized. Also, the method of circuit integration is not limited to LSI, but may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integration circuit technology that replaces LSI appears due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

Although one embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc., can be made without departing from the gist of the present invention. It is possible to

[Application example]
The moving image encoding device 11 and the moving image decoding device 31 described above can be used by being installed in various devices for transmitting, receiving, recording, and reproducing moving images. The moving image may be a natural moving image captured by a camera or the like, or may be an artificial moving image (including CG and GUI) generated by a computer or the like.

First, it will be described with reference to FIG. 2 that the moving image encoding device 11 and the moving image decoding device 31 described above can be used for transmitting and receiving moving images.

FIG. 2 shows a block diagram showing the configuration of the transmission device PROD_A equipped with the video encoding device 11. As shown in FIG. As shown in the figure, the transmission device PROD_A includes an encoding unit PROD_A1 that obtains encoded data by encoding a moving image, and a modulated signal by modulating a carrier wave with the encoded data obtained by the encoding unit PROD_A1. and a transmitter PROD_A3 for transmitting the modulated signal obtained by the modulator PROD_A2. The video encoding device 11 described above is used as this encoding unit PROD_A1.

The transmission device PROD_A uses a camera PROD_A4 for capturing a moving image, a recording medium PROD_A5 for recording the moving image, an input terminal PROD_A6 for externally inputting the moving image, and , and an image processing unit A7 for generating or processing an image. In the drawing, the configuration in which the transmitter PROD_A has all of these is illustrated, but some of them may be omitted.

The recording medium PROD_A5 may record an unencoded moving image, or record a moving image encoded by an encoding scheme for recording different from the encoding scheme for transmission. can be anything. In the latter case, a decoding unit (
(not shown) may be interposed.

Also, FIG. 2 shows a block diagram showing the configuration of the receiving device PROD_B equipped with the video decoding device 31. As shown in FIG. As shown in the figure, the receiver PROD_B includes a receiver PROD_B1 that receives a modulated signal, a demodulator PROD_B2 that obtains encoded data by demodulating the modulated signal received by the receiver PROD_B1, and a demodulator PROD_B2 that obtains encoded data. and a decoding unit PROD_B3 that obtains a moving image by decoding the encoded data. The video decoding device 31 described above is used as this decoding unit PROD_B3.

The receiving device PROD_B supplies the moving image output from the decoding unit PROD_B3 to a display PROD_B4 for displaying the moving image, a recording medium PROD_B5 for recording the moving image, and an output terminal for outputting the moving image to the outside. PROD_B6 may also be provided. In the drawing, the configuration in which the receiving device PROD_B has all of these is illustrated, but some of them may be omitted.

The recording medium PROD_B5 may be used for recording unencoded moving images, or may be encoded by an encoding scheme for recording that is different from the encoding scheme for transmission. may In the latter case, an encoding unit (not shown) may be interposed between the decoding unit PROD_B3 and the recording medium PROD_B5 to encode the moving image acquired from the decoding unit PROD_B3 according to the recording encoding method.

A transmission medium for transmitting the modulated signal may be wireless or wired. Further, the transmission mode for transmitting the modulated signal may be broadcasting (here, transmission mode in which the destination is not specified in advance), or communication (here, transmission mode in which the destination is specified in advance). aspect) may be used. That is, transmission of the modulated signal may be realized by any of wireless broadcasting, wired broadcasting, wireless communication, and wired communication.

For example, a broadcasting station (broadcasting equipment, etc.)/receiving station (television receiver, etc.) for terrestrial digital broadcasting is an example of a transmitting device PROD_A/receiving device PROD_B that transmits and receives modulated signals by radio broadcasting. A broadcasting station (broadcasting equipment, etc.)/receiving station (television receiver, etc.) of cable television broadcasting is an example of a transmitting device PROD_A/receiving device PROD_B that transmits/receives a modulated signal by cable broadcasting.

In addition, servers (workstations, etc.)/clients (television receivers, personal computers, smartphones, etc.) for VOD (Video On Demand) services and video sharing services using the Internet are transmission devices that transmit and receive modulated signals through communication. This is an example of PROD_A/receiving device PROD_B (usually, in LAN, either wireless or wired transmission medium is used, and in WAN, wired transmission medium is used). Here, personal computers include desktop PCs, laptop PCs, and tablet PCs. Smartphones also include multifunctional mobile phone terminals.

A client of the video sharing service has a function of decoding encoded data downloaded from a server and displaying the data on a display, as well as a function of encoding a video captured by a camera and uploading it to the server. That is, the client of the video sharing service functions as both the transmitting device PROD_A and the receiving device PROD_B.

Next, it will be described with reference to FIG. 3 that the moving image encoding device 11 and the moving image decoding device 31 described above can be used for recording and reproducing moving images.

FIG. 3 shows a block diagram showing the configuration of a recording device PROD_C equipped with the moving image encoding device 11 described above. As shown in the figure, the recording device PROD_C includes an encoding unit PROD_C1 that obtains encoded data by encoding a moving image, and a writing unit PROD_C2 that writes the encoded data obtained by the encoding unit PROD_C1 to the recording medium PROD_M. and have. Video encoding device 11 described above
is used as this encoder PROD_C1.

The recording medium PROD_M may be (1) a type built into the recording device PROD_C, such as a HDD (Hard Disk Drive) or SSD (Solid State Drive), or (2) an SD memory. It may be of a type connected to the recording device PROD_C, such as a card or USB (Universal Serial Bus) flash memory. Disc: registered trademark) may be loaded in a drive device (not shown) incorporated in the recording device PROD_C.

In addition, the recording device PROD_C includes a camera PROD_C3 for capturing the moving image, an input terminal PROD_C4 for inputting the moving image from the outside, and a receiving terminal for receiving the moving image as a supply source of the moving image to be input to the encoding unit PROD_C1. It may further include a unit PROD_C5 and an image processing unit PROD_C6 that generates or processes an image. In the drawing, the configuration in which the recording device PROD_C includes all of these is exemplified, but some of them may be omitted.

Note that the receiving unit PROD_C5 may receive an unencoded moving image, or receive encoded data encoded by an encoding scheme for transmission that is different from the encoding scheme for recording. It may be something to do. In the latter case, it is preferable to interpose a decoding unit for transmission (not shown) that decodes encoded data encoded by an encoding method for transmission between the receiving unit PROD_C5 and the encoding unit PROD_C1.

Examples of such a recording device PROD_C include a DVD recorder, a BD recorder, an HDD (Hard Disk Drive) recorder, etc. (In this case, the input terminal PROD_C4 or the receiver PROD_C5 is the main supply source of moving images.) . In addition, camcorders (in this case, the camera PROD_C3 is the main source of moving images), personal computers (in this case, the receiver PROD_C5 or the image processing unit C6 is the main source of moving images), smartphones (in this case, the camera PROD_C3 is the main source of moving images) In this case, the camera PROD_C3 or the receiving unit PROD_C5 serves as the main supply source of moving images) is also an example of such a recording device PROD_C.

Further, FIG. 3 shows a block diagram showing the configuration of a playback device PROD_D equipped with the moving image decoding device 31 described above. As shown in the figure, the playback device PROD_D includes a reading unit PROD_D1 that reads encoded data written to the recording medium PROD_M, and a decoding unit PROD_D2 that obtains a moving image by decoding the encoded data read by the reading unit PROD_D1. , is equipped with The video decoding device 31 described above is used as this decoding unit PROD_D2.

The recording medium PROD_M may be (1) a type built into the playback device PROD_D, such as an HDD or SSD, or (2) an SD memory card, USB flash memory, or the like. It may be of the type connected to the playback device PROD_D, or (3) like a DVD or BD, it may be loaded into a drive device (not shown) built into the playback device PROD_D. good.

Further, the playback device PROD_D includes a display PROD_D3 for displaying the moving image, an output terminal PROD_D4 for outputting the moving image to the outside, and a transmitting unit for transmitting the moving image, as destinations to which the moving image output by the decoding unit PROD_D2 is supplied. PROD_D5 may also be provided. In the drawing, the configuration in which the playback device PROD_D includes all of these is illustrated, but some of them may be omitted.

The transmission unit PROD_D5 may transmit an unencoded moving image, or transmit encoded data encoded by an encoding scheme for transmission different from the encoding scheme for recording. It may be something to do. In the latter case, it is preferable to interpose an encoding unit (not shown) that encodes a moving image using an encoding method for transmission between the decoding unit PROD_D2 and the transmitting unit PROD_D5.

Such a playback device PROD_D includes, for example, a DVD player, a BD player, an HDD player, and the like (in this case, the output terminal PROD_D4 to which a television receiver or the like is connected is the main supply destination of moving images). . Also, the TV receiver (in this case the display PROD_D3
is the main destination of moving images), digital signage (also called electronic billboards, electronic bulletin boards, etc., and the display PROD_D3 or transmission unit PROD_D5 is the main destination of moving images), desktop PCs (in this case, Output terminal PROD_D4 or transmitter PROD_D5 is the main destination of video), laptop or tablet PC (in this case, display PROD_D3 or transmitter PROD_D5 is the main video), smartphone ( In this case, the display PROD_D3 or the transmission unit PROD_D5 is the main supply destination of the moving image) is also an example of such a playback device PROD_D.

(Hardware realization and software realization)
Further, each block of the moving image decoding device 31 and the moving image encoding device 11 described above may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or a CPU (Central Processing Unit) may be used for software implementation.

In the latter case, each of the above devices has a CPU that executes the instructions of the program that implements each function, a ROM (Read Only Memory) that stores the above program, and a RAM (Random Memory) that expands the above program.
Access Memory), and a storage device (recording medium) such as a memory for storing the above program and various data. An object of the embodiments of the present invention is a computer-readable record of the program code (executable program, intermediate code program, source program) of the control program for each of the above devices, which is software for realizing the above functions. It can also be achieved by supplying a medium to each of the devices described above and causing the computer (or CPU or MPU) to read and execute the program code recorded on the recording medium.

Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks/hard disks, CD-ROMs (Compact Disc Read-Only Memory)/MO disks (Magneto-Optical discs). ) / MD (Mini Disc) / DVD (Digital Versatile Disc: registered trademark) / CD-R (CD Recordable) / Blu-ray Disc (Blu-ray
Disc: Discs including optical discs such as registered trademark), IC cards (including memory cards)
/ cards such as optical cards, mask ROM / EPROM (Erasable Programmable Read-Only Memory) / EEPROM (Electrically Erasable and Programmable Read-Only Memory: registered trademark)
/Semiconductor memories such as flash ROMs, or logic circuits such as PLDs (Programmable Logic Devices) and FPGAs (Field Programmable Gate Arrays) can be used.

Further, each device may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. This communication network is not particularly limited as long as it can transmit the program code. For example, Internet, intranet, extranet, LAN (Local Area Network), ISDN (Integrated Services Digital Network), VAN (Value-Added Network), CATV (Community Antenna television/Cable Television) communication network, Virtual Private Network Network), telephone line network, mobile communication network, satellite communication network, etc. can be used. Also, the transmission medium constituting this communication network is not limited to a specific configuration or type as long as it can transmit the program code. For example, even wired such as IEEE (Institute of Electrical and Electronic Engineers) 1394, USB, power line carrier, cable TV line, telephone line, ADSL (Asymmetric Digital Subscriber Line) line, infrared rays such as IrDA (Infrared Data Association) and remote control , BlueTooth (registered trademark), IEEE802.11 wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA (Digital Living Network Alliance: registered trademark), mobile phone networks, satellite circuits, terrestrial digital broadcasting networks, etc. Also available wirelessly. Embodiments of the invention may also be implemented in the form of a computer data signal embedded in a carrier wave, with the program code embodied in electronic transmission.

The embodiments of the present invention are not limited to the embodiments described above, and various modifications are possible within the scope of the claims. That is, the technical scope of the present invention also includes embodiments obtained by combining technical means appropriately modified within the scope of the claims.

INDUSTRIAL APPLICABILITY Embodiments of the present invention are preferably applied to a moving image decoding device that decodes encoded image data and a moving image encoding device that generates encoded image data. be able to. Also, the present invention can be preferably applied to the data structure of encoded data generated by a video encoding device and referenced by a video decoding device.

31 Video decoder
301 Entropy Decoder
302 Parameter decoder
3020 Header decoder
308 Predictive image generator
311 Inverse Quantization/Inverse Transform Unit
312 adder
11 Video encoder
101 Predictive image generator
102 Subtractor
103 Transform/Quantization Unit
104 Entropy Encoder
105 Inverse quantization/inverse transform unit (scaling unit)
107 loop filter
110 Encoding parameter determination unit
111 Parameter encoder
1110 Header encoder
1111 CT information encoder
1112 CU encoder (prediction mode encoder)
1114 TU encoder
311 Inverse quantization/inverse transform unit (scaling unit)
3111 Inverse quantization section
3112 Inverse Transformer
31123 Inverse Core Transformer

Claims (11)

a header decoding unit that decodes a flag indicating whether to extend the range of transform coefficients and a syntax element indicating the maximum size of the transform to derive the maximum size of the transform;
when extending the range of transform coefficients, derive a range of transform coefficients according to the bit depth, and rely on a quantization parameter for transform coefficients using a first shift value derived according to the range of transform coefficients. a scaling unit that right-shifts the value multiplied by the value of
The moving image decoding device, wherein the header decoding unit limits the maximum size of transform to a predetermined value or less when a flag indicating whether to expand the range of transform coefficients is valid. 2. The moving image according to claim 1, wherein the moving image decoding device further comprises a TU decoding unit that recursively divides the transform tree so that the size of the transform unit is equal to or less than the maximum size of the transform. Image decoder. The header decoding unit decodes a flag indicating whether or not to use a predetermined maximum transform size TX as a syntax element indicating the maximum transform size, and the header decoding unit instructs to use the predetermined maximum transform size TX. 2. The moving image decoding device according to claim 1, wherein the maximum transform size TX is used as the maximum transform size when the flag indicating whether to extend the range of transform coefficients is not valid. When the flag indicating that the header decoding unit uses a predetermined maximum transform size TX and indicating whether to extend the range of transform coefficients is not valid, the header decoding unit sets the maximum transform size to the above 4. The moving image decoding device according to claim 3, wherein a maximum transform size TX is set, and in other cases, the maximum transform size TX/2 is set as the maximum transform size. a header encoding unit that encodes a flag indicating whether to extend the range of transform coefficients and a syntax element indicating the maximum size of the transform to derive the maximum size of the transform;
when extending the range of transform coefficients, derive a range of transform coefficients according to the bit depth, and rely on a quantization parameter for transform coefficients using a first shift value derived according to the range of transform coefficients. a scaling unit that right-shifts the value multiplied by the value of
The moving image coding apparatus, wherein the header coding unit limits the maximum size of transform to a predetermined value or less when a flag indicating whether to extend the range of transform coefficients is valid. a header decoding unit that decodes a flag indicating whether to extend the range of transform coefficients;
Scaling for right-shifting a value obtained by multiplying a transform coefficient by a value dependent on a quantization parameter using a first shift value derived according to the range of the transform coefficient having a department,
The moving image decoding device, wherein the TU decoding unit derives the range of transform coefficients according to the flag and the block size of the transform block. 7. The TU decoding unit according to claim 6, wherein, when the flag indicates valid and the block size of the transform block is smaller than a predetermined maximum transform size, the range of transform coefficients is extended. Video decoding device. 8. The moving picture decoding device according to claim 7, wherein the TU decoding unit extends the range of transform coefficients when the flag indicates valid and the block size of the transform coefficients is smaller than 64. a header decoding unit that decodes a flag indicating whether to extend the range of transform coefficients;
A TU encoding unit for deriving a range of transform coefficients is provided, and a first shift value derived according to the range of transform coefficients is used to right-shift a value obtained by multiplying the transform coefficients by a value dependent on a quantization parameter. Equipped with a scaling section,
The moving image coding apparatus, wherein the TU coding unit derives the range of transform coefficients according to the flag and the block size of the transform block. a header decoding unit that decodes a flag indicating whether to extend the range of transform coefficients;
a TU decoder for deriving a range of transform coefficients,
The TU decoding unit expands the range of transform coefficients when the flag is valid and the block size of the transform block is equal to or less than a predetermined size,
When the flag is valid and the block size of the transform block is larger than a predetermined size, the TU decoding unit converts the shift value without expanding the range of transform coefficients when the flag is valid. , depending on the maximum magnitude of the transform coefficients, and
The TU decoding unit derives the shift value without depending on the maximum value of the magnitude of the transform coefficient without extending the range of the transform coefficient unless the flag is valid,
A video decoding device comprising a scaling unit that right-shifts a value obtained by multiplying a transform coefficient by a value dependent on a quantization parameter using a first shift value derived according to the range of the transform coefficient. . a header encoding unit that decodes a flag indicating whether to extend the range of transform coefficients;
a TU encoder for deriving a range of transform coefficients,
The TU encoding unit expands the range of transform coefficients when the flag is valid and the block size of the transform block is equal to or less than a predetermined size,
When the flag is valid and the block size of the transform block is larger than a predetermined size, the TU encoding unit, when the flag is valid, does not extend the range of transform coefficients, and shifts the shift value , depending on the maximum magnitude of the transform coefficients,
The TU encoding unit derives the shift value without depending on the maximum value of the magnitude of the transform coefficient without extending the range of the transform coefficient unless the flag is valid,
A moving image encoding comprising a scaling unit that right-shifts a value obtained by multiplying a transform coefficient by a value dependent on a quantization parameter using a first shift value derived according to the range of the transform coefficient. Device.

Images