WO2021251400A1

WO2021251400A1 - Decoding device and program

Info

Publication number: WO2021251400A1
Application number: PCT/JP2021/021816
Authority: WO
Inventors: 俊輔岩村; 慎平根本; 雄一近藤; 和久井口; 敦郎市ヶ谷
Original assignee: 日本放送協会
Priority date: 2020-06-10
Filing date: 2021-06-08
Publication date: 2021-12-16
Also published as: JPWO2021251400A1

Abstract

Provided is a decoding device for decoding a block obtained by dividing an original image, the decoding device comprising: an entropy decoding unit for outputting a conversion coefficient corresponding to the block by decoding an encoded stream; an inverse quantization/inverse conversion unit for recovering a prediction residual corresponding to the block by performing an inverse quantization process and an inverse conversion process on the conversion coefficient; a synthesizing unit for recovering the block by synthesizing the recovered prediction residual and a predicted block obtained by predicting the block; a deblocking filter for performing a filter process with respect to a boundary of a pair of blocks comprising the recovered block and a previously recovered block adjacent to the block; and a filter control unit for controlling the boundary filtering strength of the deblocking filter on the basis of whether at least one of the pair of blocks is encoded using joint coding of chroma residual (JCCR) where one combined prediction residual is generated from the prediction residual of each of a Cb color difference component and a Cr color difference component.

Description

Decoding device and program

The present invention relates to a decoding device and a program.

In HEVC (High Efficiency Video Coding) and VVC (Versatile Video Coding), which is a next-generation coding method, as an in-loop filter, in order to suppress distortion at the boundary of blocks when coding processing is performed on a block-by-block basis. Deblocking filter is adopted. In the control of the deblocking filter, the boundary filter strength (boundary filtering strength) of the deblocking filter is controlled according to whether or not a non-zero conversion coefficient exists in at least one of the two adjacent blocks. This is because the energy of the predicted residual is distributed over the entire block due to the inverse transformation of the non-zero transformation coefficient, so that there is a high possibility that a discontinuity will occur at the boundary between the two blocks.

The decoding device according to the first aspect is a decoding device that decodes a block obtained by dividing an original image, and is an entropy decoding unit that outputs a conversion coefficient corresponding to the block by decoding a coded stream. And the inverse quantization / inverse conversion unit that restores the predicted residual corresponding to the block by performing the inverse quantization processing and the inverse conversion processing on the conversion coefficient, and the restored predicted residual and the block. Filtering is performed on the boundary of a pair of blocks consisting of a synthesis unit that restores the block by synthesizing the predicted block obtained by predicting the above, the restored block, and the restored block adjacent to the block. Using a deblocking filter and JCCR (Joint coding of chroma resin) in which at least one block of the pair of blocks generates one joint predicted residual from the predicted residuals of the Cb color difference component and the Cr color difference component, respectively. A filter control unit for controlling the boundary filter strength of the deblocking filter based on whether or not it is encoded is provided.

The program according to the second aspect causes the computer to function as the decoding device according to the first aspect.

It is a figure which shows the structure of the coding apparatus which concerns on embodiment. It is a figure for demonstrating the operation of the deblocking filter which concerns on embodiment. It is a figure which shows the structure of the decoding apparatus which concerns on embodiment. It is a figure which shows the operation example of the filter control part which concerns on embodiment. It is a figure which shows the operation example of the filter control part which concerns on change example 2. FIG. It is a figure which shows the structure of the coding apparatus which concerns on modification 3. It is a figure which shows the structure of the decoding apparatus which concerns on modification 3.

In the draft VVC standard, when the chroma format of the input video is 4: 4: 4, the color space (RGB space) of the predicted residual is converted to the YCgCo space, and the predicted residual after the color space conversion is converted. -A technique called adaptive color conversion (ACT: Adaptive Color Transfer) that performs coding processing such as entropy coding processing is adopted (see Non-Patent Document 1). The coding device can control whether or not to apply the ACT for each coded block, and outputs the ACT application flag for each coded block as a stream. The decoding device restores the predicted residual by performing entropy decoding / inverse conversion processing, etc. on the block encoded using ACT, and reverses the restored color space (YCgCo space) of the predicted residual to the RGB space. Convert.

In the conventional technique described above, when the decoding device has a non-zero conversion coefficient in at least one of two adjacent blocks, the decoding device applies a deblocking filter to the boundary between the two blocks. On the other hand, if there is no non-zero conversion coefficient in both blocks of two adjacent blocks, it is possible not to apply the deblocking filter to the boundary between the two blocks.

However, for the block to which the ACT is applied, the predicted residual is restored from the conversion coefficient by the inverse conversion process, and then the color space of the predicted residual is inversely converted from the YCgCo space to the RGB space by the color space inverse conversion. Therefore, when a non-zero conversion coefficient exists in a block of a certain color component, the non-zero conversion coefficient affects the block of another color component at the time of color space inverse transformation.

Therefore, if deblocking filter control based on the presence or absence of a non-zero conversion coefficient is applied to a block encoded using ACT, the boundary filter strength of the deblocking filter cannot be appropriately controlled and the image quality deteriorates. There is a concern that it will cause.

The same problem as described above may occur when another coding tool, predicted residual joint coding (Joint coding of chroma error: JCCR), is applied.

Therefore, an object of the present invention is to provide a decoding device and a program that suppresses deterioration of image quality.

The coding device and the decoding device according to the embodiment will be described with reference to the drawings. The coding device and the decoding device according to the embodiment encode and decode a moving image represented by MPEG, respectively. In the description of the drawings below, the same or similar parts are designated by the same or similar reference numerals.

[First Embodiment]
<Configuration of coding device>
First, the configuration of the coding apparatus according to the present embodiment will be described. FIG. 1 is a diagram showing a configuration of a coding device 1 according to the present embodiment.

As shown in FIG. 1, the coding apparatus 1 includes a block dividing unit 100, a residual generation unit 110, a switching unit 111, a color space conversion unit 112, a conversion / quantization unit 120, and an entropy coding unit. It has 130, an inverse quantization / inverse conversion unit 140, a switching unit 143, a color space inverse conversion unit 144, a synthesis unit 150, a deblocking filter 160, a memory 170, and a prediction unit 180.

The block division unit 100 divides the original image, which is an input image for each frame (or picture) constituting the moving image, into a plurality of image blocks, and outputs the image block obtained by the division to the residual generation unit 110. The size of the image block is, for example, 32 × 32 pixels, 16 × 16 pixels, 8 × 8 pixels, 4 × 4 pixels, or the like. The shape of the image block is not limited to a square and may be a rectangle (non-square). The image block is a unit in which the coding device 1 performs a coding process (that is, a block to be coded) and a unit in which the decoding device performs a decoding process (that is, a block to be decoded). Such an image block may be called a CU (Coding Unit).

The input image may be an RGB signal and the chroma format may be 4: 4: 4. The RGB space is an example of the first color space. The "R" component corresponds to the first component, the "G" component corresponds to the second component, and the "B" component corresponds to the third component. The block division unit 100 outputs a block by performing block division for each of the R component, the G component, and the B component constituting the image. In the following description of the coding device, when each color component is not distinguished, it is simply referred to as a coded block.

The residual generation unit 110 calculates a predicted residual representing a difference (error) between the coded block output by the block dividing unit 100 and the predicted block obtained by predicting the coded block by the prediction unit 180. .. Specifically, the residual generation unit 110 calculates the predicted residual by subtracting each pixel value of the predicted block from each pixel value of the coded block, and outputs the calculated predicted residual to the switching unit 111. do. In the present embodiment, the residual generation unit 110 generates a predicted residual of each color component by the difference between the coded block of each color component and the predicted block of each color component.

The switching unit 111 outputs the predicted residual of each color component output by the residual generation unit 110 to either the conversion / quantization unit 120 or the color space conversion unit 112. The switching unit 111 outputs the predicted residual to the conversion / quantization unit 120 when the color space conversion processing (ACT processing) is not performed, and outputs the predicted residual to the color space conversion unit 112 when the ACT processing is performed. ..

The color space conversion unit 112 performs ACT processing on the predicted residuals of each color component, and outputs the predicted residuals after the ACT processing to the conversion / quantization unit 120. The color space conversion unit 112 generates a new predicted residual by performing the following conversion calculation on the R component, the G component, and the B component of the predicted residual of the coded block.

Co = RB
t = B + (Co >> 1)
Cg = Gt
Y = t + (Cg >> 1)

However, ">>" represents a right shift operation. Further, the "Y" component corresponds to the first component, the "Cg" component corresponds to the second component, and the "Co" component corresponds to the third component. Such a YCgCo space is an example of a second color space.

The switching unit 111 and the color space conversion unit 112 can control whether or not to perform color conversion processing for each coded block. The entropy coding unit 130 signals in the bit stream a flag (ACT application flag) indicating whether or not the color conversion process has been performed on the coded block.

The ACT process in the color space conversion unit 112 may generate a predicted residual composed of new color components by addition / subtraction / multiplication / division / shift processing for each color component. Also, the ACT process does not have to be a transformation that affects all color components. For example, the color space conversion unit 112 maintains the first component unchanged, sets the average value of the second component and the third component as the new second component, and sets the difference between the second component and the third component. An ACT process of using a new third component may be applied.

The conversion / quantization unit 120 performs conversion processing and quantization processing in block units. The conversion / quantization unit 120 includes a conversion unit 121 and a quantization unit 122.

The conversion unit 121 performs conversion processing on the predicted residual (referred to as predicted residual regardless of whether or not the ACT processing is applied) output by the switching unit 111 or the color space conversion unit 112, and calculates the conversion coefficient. The calculated conversion coefficient is output to the quantization unit 122. Specifically, the conversion unit 121 generates a conversion coefficient for each color component by performing a conversion process on the predicted residual of each color component in block units. The conversion process may be frequency conversion such as DCT, DST, and discrete wavelet conversion. Further, the conversion unit 121 outputs information regarding the conversion process to the entropy coding unit 130.

The conversion process includes conversion skip without conversion process, which is adopted in HEVC and VVC standard draft. In the conversion skip mode of HEVC, the conversion coefficient is set by scaling the predicted residual without performing the horizontal and vertical conversion processing, but the conversion skip according to the present embodiment is a conversion to which the conversion processing is applied only to the horizontal. Also includes conversions that apply conversion processing only to vertical and vertical. Further, the conversion unit 121 may perform a secondary conversion process in which the conversion process is further applied to the conversion coefficient obtained by the conversion process. Further, the secondary conversion process may be applied only to a part of the area of the conversion coefficient.

The quantization unit 122 quantizes the conversion coefficient output from the conversion unit 121 using the quantization parameter and the scaling list, and outputs the quantized conversion coefficient to the entropy coding unit 130 and the inverse quantization / inverse conversion unit 140. do. Further, the quantization unit 122 outputs information regarding the quantization process (specifically, information on the quantization parameters and the scaling list used in the quantization process) to the entropy coding unit 130 and the inverse quantization unit 141. ..

The entropy coding unit 130 performs entropy coding on the quantization conversion coefficient output by the quantization unit 122, performs data compression to generate a bit stream (encoded data), and outputs the bit stream to the decoding side. do. For the entropy coding, a Huffman code, CABAC (Context-based Adaptive Binary Arithmetic Coding) or the like can be used. Further, the entropy coding unit 130 includes information on the conversion process input from the conversion unit 121 in the bitstream and signals it to the decoding side, and includes information on the prediction process input from the prediction unit 180 in the bitstream. Signaling to the decryption side.

Further, the entropy coding unit 130 includes a color space conversion flag indicating whether or not ACT is applied in the bit stream for each coded block and signals it to the decoding side. Such a color space conversion flag is also called an ACT application flag. When the ACT application flag is on (“1”), it indicates that the ACT is applied to the corresponding coded block. When the ACT application flag is off (“0”), it indicates that ACT is not applied to the corresponding coded block. The ACT non-applicable flag may be used instead of the ACT application flag. In that case, when the ACT non-application flag is on (“1”), it indicates that ACT is not applied to the corresponding coded block. When the ACT non-application flag is off (“0”), it indicates that ACT is applied to the corresponding coded block.

The inverse quantization / inverse transformation unit 140 performs inverse quantization processing and inverse transformation processing in block units. The inverse quantization / inverse transformation unit 140 has an inverse quantization unit 141 and an inverse transformation unit 142.

The inverse quantization unit 141 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores and restores the conversion coefficient by inversely quantizing the quantization conversion coefficient output by the quantization unit 122 using the quantization parameter (Qp) and the scaling list. The converted conversion coefficient is output to the inverse conversion unit 142.

The reverse conversion unit 142 performs the reverse conversion process corresponding to the conversion process performed by the conversion unit 121. For example, when the conversion unit 121 performs the discrete cosine transform, the inverse transform unit 142 performs the inverse discrete cosine transform. The inverse transformation unit 142 performs an inverse transformation process on the conversion coefficient output by the inverse quantization unit 141 to restore the predicted residual, and outputs the restored predicted residual, which is the restored predicted residual, to the switching unit 143. ..

The switching unit 143 outputs the restoration prediction residual of each color component output by the inverse conversion unit 142 to either the synthesis unit 150 or the color space inverse conversion unit 144. The switching unit 143 outputs the restored predicted residual to the synthesis unit 150 for the block to which the ACT is applied, and outputs the restored predicted residual to the color space inverse conversion unit 144 for the block to which the ACT is not applied.

The color space reverse conversion unit 144 performs color space reverse conversion processing (reverse ACT processing), which is the reverse processing of the ACT processing performed by the color space conversion unit 112, and outputs the predicted residual after the reverse ACT processing to the synthesis unit 150. .. Specifically, the inverse transformation from the YCgCo space to the RGB space is performed by performing the following inverse transformation calculation using the Y component, the Cg component, and the Co component of the predicted residual after restoration.

t = Y- (Cg >> 1)
G = Cg + t
B = t- (Co >> 1)
R = Co + B

The synthesizing unit 150 synthesizes the restored prediction residual output by the inverse transformation unit 142 or the color space inverse transformation unit 144 with the prediction block output by the prediction unit 180 in pixel units. The synthesizing unit 150 adds each pixel value of the restoration prediction residual and each pixel value of the prediction block to restore (reconstruct) the coded block, and outputs the restored block to the deblocking filter 160. The restored block may be called a reconstructed block.

The deblocking filter 160 performs a filter process on the restored block output by the synthesis unit 150, and outputs the restored block after the filter process to the memory 170. The filter control unit 161 controls the deblocking filter 160. Details of the deblocking filter 160 and the filter control unit 161 will be described later.

The memory 170 stores the restored blocks after the filtering process output by the deblocking filter 160, and stores the restored blocks as restored images in frame units. The memory 170 outputs the stored restored block or restored image to the prediction unit 180.

The prediction unit 180 performs prediction processing in block units. The prediction unit 180 generates a prediction block for each color component by performing prediction processing such as intra prediction and inter prediction for the coded block. The prediction unit 180 includes an inter prediction unit 181, an intra prediction unit 182, and a switching unit 183.

The inter-prediction unit 181 performs inter-prediction using the correlation between frames. Specifically, the inter-prediction unit 181 uses the restored image stored in the memory 170 as a reference image, calculates a motion vector by a method such as block matching, predicts the block to be encoded, and predicts the inter-prediction block. Is generated, and the generated inter-prediction block is output to the switching unit 183. Here, the inter-prediction unit 181 selects the optimum inter-prediction method from among inter-prediction using a plurality of reference images (typically bi-prediction) and inter-prediction using one reference image (one-way prediction). Select and perform inter-prediction using the selected inter-prediction method. The inter-prediction unit 181 outputs information (motion vector and the like) related to the inter-prediction to the entropy coding unit 130.

The intra prediction unit 182 performs intra prediction using the spatial correlation in the frame. Specifically, the intra prediction unit 182 generates an intra prediction block by referring to the restored pixels in the vicinity of the coded target block among the restored images stored in the memory 170, and the generated intra prediction block. Is output to the switching unit 183. The intra prediction unit 182 selects an intra prediction mode to be applied to the coded target block from the plurality of intra prediction modes, and predicts the coded target block using the selected intra prediction mode.

The switching unit 183 switches between the inter prediction block output by the inter prediction unit 181 and the intra prediction block output by the intra prediction unit 182, and outputs one of the prediction blocks to the residual generation unit 110 and the synthesis unit 150.

Next, the deblocking filter 160 and the filter control unit 161 according to the present embodiment will be described.

The deblocking filter 160 performs filtering processing on the block boundary of two blocks consisting of a restored block (first block) and a restored block (second block) adjacent to the restored block, and each after filtering processing is performed. The restored block is output to the memory 170. The filter processing is a processing for reducing signal deterioration caused by processing in block units, and is a filtering processing for smoothing a signal gap at a block boundary between two adjacent blocks.

The filter control unit 161 controls the deblocking filter 160. Specifically, the filter control unit 161 controls the boundary filter strength (Bs: Boundary strength) indicating whether or not to perform the filter processing on the block boundary of the block pair, and the filter strength of the deblocking filter 160. The boundary filter strength Bs refers to a parameter for determining whether or not to apply the filtering process and the type of the filtering process. The control of whether or not the filter processing is performed can be regarded as the control of whether the boundary filter strength Bs is 1 or more or zero.

FIG. 2 is a diagram for explaining the operation of the deblocking filter 160 according to the present embodiment. In the example shown in FIG. 2, the deblocking filter 160 performs filter processing on the block boundary of each block of 8 × 8 pixels. Further, the deblocking filter 160 performs filter processing in units of 4 rows or 4 columns. The blocks P and Q shown in FIG. 2 show an example in which the deblocking filter 160 is one unit of the filter processing and the block size is 4 × 4 pixels. Each of the blocks P and Q may be referred to as a subblock. The block Q is a restored block corresponding to the coded block, and the block P is a restored block adjacent to the block Q.

The filter control unit 161 determines the boundary filter strength Bs based on Table 1 below. In the present embodiment, the value of the boundary filter strength Bs is either 0, 1, or 2.

As shown in FIG. 2 and Table 1, the filter control unit 161 sets the Bs value to 2 when the intra prediction is applied to at least one of the blocks P and Q.

When the motion compensation prediction (inter prediction) is applied to both the blocks P and Q, and at least one of the following conditions (a) to (d) is satisfied, the filter control unit 161 Bs. The value is 1, and in other cases, the Bs value is 0.

(A) The absolute value of the difference between the motion vectors of blocks P and Q is equal to or greater than the threshold value (for example, 1 pixel).

(B) The number of motion vectors of blocks P and Q or the reference image is different.

(C) At least one of blocks P and Q contains a significant conversion factor (ie, a non-zero conversion factor).

(D) ACT is applied to at least one of blocks P and Q.

The filter control unit 161 controls the deblocking filter 160 so that the deblocking filter processing is not performed when the value of the boundary filter strength Bs is 0. Hereinafter, the vertical block boundary shown in FIG. 2 will be described as an example.

When the value of the boundary filter strength Bs is 1 or 2, the filter control unit 161 may control the deblocking filter 160 so as to perform the deblocking filter processing when the following equation (1) is satisfied.

When the deblocking filter process is performed, the filter control unit 161 may apply a strong filter when all of the following conditional expressions (2) to (7) are satisfied, and may apply a weak filter in other cases. good.

However, the values of the threshold values β and t _C change according to _{the average value Q av} of the quantization parameters of the adjacent blocks P and Q.

<Configuration of decryption device>
Next, the decoding device according to the present embodiment will be mainly described as being different from the coding device 1. FIG. 3 is a diagram showing a configuration of a decoding device 2 according to the present embodiment.

As shown in FIG. 3, the decoding device 2 includes an entropy decoding unit 200, an inverse quantization / inverse conversion unit 210, a switching unit 215, a color space inverse conversion unit 216, a synthesis unit 220, and a deblocking filter 230. And a memory 240 and a prediction unit 250.

The entropy decoding unit 200 decodes the coded data (bitstream), acquires the quantization conversion coefficient corresponding to the decoding target block, and outputs the acquired quantization conversion coefficient to the inverse quantization / inverse conversion unit 210. Further, the entropy decoding unit 200 acquires information on the conversion process and the quantization process, and outputs the information on the conversion process and the quantization process to the inverse quantization / inverse conversion unit 210. Further, the entropy decoding unit 200 acquires information on the prediction process and outputs the information on the prediction process to the prediction unit 250. The entropy decoding unit 200 acquires the color space conversion flag for each decoding target block, and outputs the acquired color space conversion flag to the switching unit 215 and the filter control unit 231.

The inverse quantization / inverse transformation unit 210 performs inverse quantization processing and inverse transformation processing in block units. The inverse quantization / inverse conversion unit 210 has an inverse quantization unit 211 and an inverse conversion unit 212.

The inverse quantization unit 211 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122 of the coding apparatus 1. The dequantization unit 211 restores and restores the conversion coefficient of the block to be decoded by dequantizing the quantization conversion coefficient output by the entropy decoding unit 200 using the quantization parameter (Qp) and the scaling list. The converted conversion coefficient is output to the inverse conversion unit 212.

The reverse conversion unit 212 performs reverse conversion processing corresponding to the conversion processing performed by the conversion unit 121 of the coding device 1. The inverse transformation unit 212 performs an inverse transformation process on the conversion coefficient output by the inverse quantization unit 211 to restore the predicted residual, and outputs the restored predicted residual to the switching unit 215.

The switching unit 215 outputs the predicted residual of each color component output by the inverse conversion unit 212 to either the synthesis unit 220 or the color space inverse conversion unit 216 based on the color space conversion flag. The switching unit 111 outputs the predicted residual to the conversion / quantization unit 120 for the block to which the color space inverse transformation processing (ACT) is applied, and the color space inverse transformation of the predicted residual for the block to which the ACT is applied. Output to unit 216.

The color space reverse conversion unit 216 performs color space reverse conversion processing (reverse ACT processing), which is the reverse processing of the ACT processing performed by the color space conversion unit 112 of the coding apparatus 1, and synthesizes the predicted residual after the reverse ACT processing. Output to unit 220. Specifically, the color space inverse transformation unit 216 performs the following inverse transformation calculation using the Y component, Cg component, and Co component of the predicted residual after restoration.

t = Y- (Cg >> 1)
G = Cg + t
B = t- (Co >> 1)
R = Co + B

The synthesizing unit 220 decodes (reconstructs) the original block by synthesizing the prediction residual output by the switching unit 215 or the color space inverse conversion unit 216 and the prediction block output by the prediction unit 250 on a pixel-by-pixel basis. Then, the restored block is output to the deblocking filter 230.

The deblocking filter 230 performs a filter process on the restored block output by the synthesis unit 220, and outputs the restored block after the filter process to the memory 240. Specifically, the deblocking filter 230 filters the block boundary of two blocks including the restored block (first block) and the restored block (second block) adjacent to the restored block. Each restored block after filtering is output to the memory 240. The function of the deblocking filter 230 is the same as the function of the deblocking filter 160 of the coding device 1.

The filter control unit 231 controls the deblocking filter 230. The filter control unit 231 controls the deblocking filter 230. Specifically, the filter control unit 231 controls the boundary filter strength (Bs: Boundary strength) indicating whether or not to perform the filter processing on the block boundary of the block pair, and the filter strength of the deblocking filter 230. The function of the filter control unit 231 is the same as the function of the filter control unit 161 of the coding device 1. The function of the filter control unit 231 determines the boundary filter strength Bs based on Table 1 above.

That is, the filter control unit 231 according to the present embodiment is a boundary of the deblocking filter 230 based on whether or not at least one of the adjacent blocks P and Q is encoded by using adaptive color conversion (ACT). The filter strength Bs is controlled.

As described above, the entropy decoding unit 200 acquires a flag (color space conversion flag) indicating whether or not the code is encoded by using the adaptive color conversion for each of the block P and the block Q. The filter control unit 231 controls the boundary filter intensity Bs of the deblocking filter 230 based on the color space conversion flags for each of the block P and the block Q.

When at least one of the block P and the block Q is encoded by using the adaptive color conversion, the filter control unit 231 controls the boundary filter intensity Bs so as to perform the filtering process by the deblocking filter 230 (that is, the boundary). Filter strength Bs = 1 is set). Specifically, the filter control unit 231 encodes at least one of the block P and the block Q using the adaptive color conversion even when the non-zero conversion coefficient does not exist in both the block P and the block Q. If so, the boundary filter strength Bs is controlled so as to perform the filtering process by the deblocking filter 230.

The memory 240 stores the restored blocks output by the compositing unit 220, and stores the restored blocks as restored images in frame units. The memory 240 outputs the restored block or the restored image to the prediction unit 250. Further, the memory 240 outputs the restored image in frame units to the outside of the decoding device 2.

The prediction unit 250 makes predictions for each color component in block units. The prediction unit 250 includes an inter-prediction unit 251, an intra-prediction unit 252, and a switching unit 253.

The inter-prediction unit 251 performs inter-prediction using the correlation between frames. Specifically, the inter-prediction unit 251 encodes the restored image stored in the memory 240 as a reference image based on the information related to the inter-prediction (for example, motion vector information) output by the entropy decoding unit 200. The target block is predicted to generate an inter-prediction block, and the generated inter-prediction block is output to the switching unit 253.

The intra prediction unit 252 performs intra prediction using the spatial correlation in the frame. Specifically, the intra prediction unit 252 uses an intra prediction mode corresponding to information related to the intra prediction output by the entropy decoding unit 200 (for example, intra prediction mode information), and the restored image stored in the memory 240. An intra prediction block is generated with reference to the restored pixels around the coded block, and the generated intra prediction block is output to the switching unit 253.

The switching unit 253 switches between the inter prediction block output by the inter prediction unit 251 and the intra prediction block output by the intra prediction unit 252, and outputs one of the prediction blocks to the synthesis unit 220.

As described above, the decoding device 2 according to the present embodiment has the entropy decoding unit 200 that outputs the conversion coefficient corresponding to the block P by decoding the coded stream, and the conversion coefficient output by the entropy decoding unit 200. The inverse quantization / inverse transformation unit 210 that restores the predicted residual corresponding to the block P (first block) by performing the inverse quantization processing and the inverse transformation processing, and the restored predicted residual and the block P are predicted. Filtering is performed on the boundary between the synthesizing unit 220 that restores the block P by synthesizing the predicted blocks obtained in the above process, the restored block P, and the restored block Q (second block) adjacent to the block P. Filter control that controls the boundary filter strength Bs of the deblocking filter 230 based on whether at least one of the deblocking filter 230 and the block P and the block Q is encoded by the adaptive color transformation (ACT). It has a portion 231 and.

For the block to which ACT is applied, the predicted residual is restored from the conversion coefficient by the inverse conversion process, and then the color space of the predicted residual is inversely converted from the YCgCo space to the RGB space by the color space inverse conversion (inverse ACT). To. Therefore, when a non-zero conversion coefficient exists in a block of a certain color component, the non-zero conversion coefficient affects the block of another color component at the time of color space inverse transformation.

Therefore, the filter control unit 231 according to the present embodiment not only controls the boundary filter strength Bs of the deblocking filter 230 based on the presence or absence of a non-zero conversion coefficient, but also considers whether or not the ACT is applied to the deblocking filter 230. Boundary filter strength Bs is controlled. As a result, the boundary filter strength Bs of the deblocking filter 230 can be appropriately controlled, so that deterioration of image quality can be suppressed even when ACT is applied.

<Operation of filter control unit>
Next, the operations of the filter control unit 161 and the filter control unit 231 according to the present embodiment will be described. Since the filter control unit 161 on the coding side and the filter control unit 231 on the decoding side perform the same operation, the filter control unit 231 on the decoding side will be described here as an example. FIG. 4 is a diagram showing an operation example of the filter control unit 231 according to the present embodiment. The order of determination shown in FIG. 4 is an example, and the order of determination may be changed.

As shown in FIG. 4, in step S1, the filter control unit 231 determines whether or not the intra prediction is applied to at least one of the block pairs consisting of the blocks P and Q. When the intra prediction is applied to at least one of the block pairs (step S1: YES), in step S2, the filter control unit 231 controls the deblocking filter 230 to perform the deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 2.

When the intra prediction is not applied to any of the block pairs (step S1: NO), in step S3, the filter control unit 231 determines whether or not the difference between the motion vectors of the target block pairs is equal to or greater than the threshold value. When the difference between the motion vectors of the target block pair is equal to or greater than the threshold value (step S3: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 so as to perform the deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1.

When the difference between the motion vectors of the block pairs is not equal to or greater than the threshold value (step S3: NO), in step S5, the filter control unit 231 determines whether the number of motion vectors of the block pairs or the reference image is different. When the number of motion vectors of the block pair or the reference image is different (step S5: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 so as to perform the deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1.

If the number of motion vectors of the block pair or the reference image is the same (step S5: NO), in step S6, the filter control unit 231 determines whether or not at least one block of the block pair contains a non-zero conversion coefficient. Is determined. When at least one of the block pairs contains a non-zero conversion coefficient (step S6: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform the deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1.

When none of the blocks of the block pair contains a non-zero conversion coefficient (step S6: NO), in step S7, the filter control unit 231 outputs the color space conversion flag of each block output by the entropy decoding unit 200. Based on, it is determined whether at least one block of the block pair is encoded using ACT. When at least one block of the block pair is encoded by using ACT (step S7: YES), in step S4, the filter control unit 231 controls the deblocking filter 230 to perform the deblocking filter processing. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1. On the other hand, when none of the blocks of the block pair is encoded by using ACT (step S7: NO), in step S8, the filter control unit 231 controls the deblocking filter 230 so as not to perform the deblocking filter processing. do. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 0.

<Change example 1>
The filter control unit 231 on the decoding side determines whether or not at least one block of the block pair (block P and block Q) contains a non-zero conversion coefficient based on the flag signaled from the coding device 1. You may.

Specifically, the entropy coding unit 130 of the coding device 1 includes a flag (tu_coded_flag) indicating whether or not a non-zero conversion coefficient is included in the coding stream for each block. For example, the entropy coding unit 130 sets the flag (tu_coded_flag) to "1" for a block containing a non-zero conversion coefficient, and sets the flag (tu_coded_flag) to "0" for a block containing no non-zero conversion coefficient. ..

The entropy decoding unit 200 of the decoding device 2 acquires a flag (tu_coded_flag) for each block, and outputs the acquired flag (tu_coded_flag) to the filter control unit 231. The filter control unit 231 interprets that the block whose flag (tu_coded_flag) is "1" does not include the non-zero conversion coefficient. Then, the filter control unit 231 sets the boundary filter strength Bs of the deblocking filter 230 as shown in Table 2 below.

As shown in Table 2, the filter control unit 231 controls the deblocking filter 230 to perform the deblocking filter processing when the tu_coded_flag of at least one block of the block pair is "1". Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1.

<Change example 2>
In the above-mentioned modification 1, the filter control unit 231 sets the boundary filter strength Bs = 1 when the tu_coded_flag of at least one block of the block pair (block P and block Q) is “1”. The deblocking filter 230 is controlled so as to perform the blocking filter processing.

Further, the filter control unit 231 performs the deblocking filter processing by setting the boundary filter strength Bs = 1 even when the ACT is applied to at least one block of the block pair (block P and block Q). The deblocking filter 230 is controlled to do so.

In this modified example, it is possible to combine such two-step judgments into one. Specifically, the filter control unit 231 sets tu_coded_flag = "1" when ACT is applied to at least one block of the block pair. As a result, the boundary filter strength Bs = 1 is set, and the deblocking filter 230 is controlled so as to perform the deblocking filter processing (see Table 3).

For example, the filter control unit 231 changes the tu_coded_flag corresponding to the block based on whether or not the block is encoded by using ACT, and controls the boundary filter strength Bs of the deblocking filter 230 based on the tu_coded_flag. do. Such a change process of tu_coded_flag may be executed by the color space inverse conversion unit 216. In this case, at least a part of the filter control unit 231 may be incorporated in the color space inverse conversion unit 216.

Specifically, the filter control unit 231 changes tu_coded_flag to indicate that the block contains a non-zero conversion factor when the block is coded using ACT. That is, when the block is encoded by using the ACT, the filter control unit 231 has a tu_y_coded_flag corresponding to the block of the first color component, a tu_cb_coded_flag corresponding to the block of the second color component, and a third color. "1" is set for each of the tu_cr_coded_flag corresponding to the block of the component. Here, even if the original tu_y_coded_flag, tu_cb_coded_flag, and tu_cr_coded_flag are "0", the block to which the ACT is applied is changed to "tu_y_coded_flag, tu_cb_coded_flag", and both are changed to "code_flag".

FIG. 5 is a diagram showing the operation of the filter control unit 231 according to this modified example.

As shown in FIG. 5, in step S11, the filter control unit 231 determines whether or not the block corresponding to the ACT application flag is encoded by using the ACT based on the ACT application flag output by the entropy decoding unit 200. Is determined. If the block is not coded using ACT (step S11: NO), the process ends without changing the tu_coded_flag (tu_y_coded_flag, tu_kb_coded_flag, and tu_cr_coded_flag).

When the block is coded using ACT (step S11: YES), in step S12, the filter control unit 231 sets “1” for tu_coded_flag (tu_y_coded_flag, tu_cb_coded_flag, and tu_cr_coded_flag). That is, the filter control unit 231 changes tu_coded_flag so that the block includes a non-zero conversion coefficient.

After that, the filter control unit 231 sets the boundary filter strength Bs of the deblocking filter 230 according to Table 3.

In this modification example, an example of rewriting tu_coded_flag itself has been described. However, the filter control unit 231 changes the value of tu_coded_flag while maintaining the original value (“1” or “0”) of tu_coded_flag, and then writes it back to the original value in the subsequent stage of the deblocking filter process. May be good.

<Change example 3>
As described above, for the block to which the ACT is applied, the predicted residual is restored from the conversion coefficient by the inverse transformation process, and then the color space of the predicted residual is inversely converted from the YCgCo space to the RGB space by the inverse ACT. .. Therefore, when a non-zero conversion coefficient exists in a block of a certain color component, the non-zero conversion coefficient affects the block of another color component at the time of inverse ACT.

Therefore, when ACT is applied, the control of the boundary filter strength Bs of the deblocking filter 230 based on the presence or absence of the non-zero conversion coefficient does not function properly. Therefore, in this modification, the boundary filter strength Bs of the deblocking filter 230 is controlled based on the predicted residual after the inverse transformation processing, not on the conversion coefficient. As a result, even if the non-zero conversion coefficient affects the blocks of other color components during inverse ACT, the boundary filter intensity Bs of the deblocking filter 230 is set based on the predicted residual after the effect. Can be controlled appropriately.

That is, the filter control unit 161 and the filter control unit 231 according to this modification set the boundary filter strength Bs of the deblocking filter 230 based on Table 4 below.

FIG. 6 is a diagram showing a configuration of a coding device 1 according to this modified example.

As shown in FIG. 6, in the coding apparatus 1, the filter control unit 161 is input with the same predicted residual as the predicted residual (restored predicted residual) input to the synthesis unit 150. For the block to which the ACT is applied, the predicted residual after the inverse ACT is input to the filter control unit 161. The filter control unit 161 controls the deblocking filter 160 to perform the deblocking filter processing when the restored predicted residual of at least one block of the block pair (block P and block Q) contains a non-zero value. Specifically, the filter control unit 161 sets the boundary filter strength Bs = 1.

FIG. 7 is a diagram showing the configuration of the decoding device 2 according to this modified example.

As shown in FIG. 7, in the decoding device 2, the same predicted residual as the predicted residual (restored predicted residual) input to the synthesis unit 220 is input to the filter control unit 231. For the block to which the ACT is applied, the predicted residual after the inverse ACT is input to the filter control unit 231. The filter control unit 231 controls the deblocking filter 230 to perform the deblocking filter processing when the restored predicted residual of at least one block of the block pair (block P and block Q) contains a non-zero value. Specifically, the filter control unit 231 sets the boundary filter strength Bs = 1.

[Second Embodiment]
In the above-described embodiment and its modification, an example in which ACT, which is one of the coding tools, is applied has been described. However, when another coding tool, Joint coding of chroma error (JCCR), is applied, the same problem as described above may occur. Therefore, the above-described embodiment and its modification may be applied to JCCR, and the ACT in the above-mentioned embodiment and its modification may be appropriately read as JCCR. For example, the boundary filter strength Bs can be controlled as shown in Table 5 below.

JCCR is a coefficient coding mode for color difference components (see Non-Patent Document 1). In JCCR, the coding apparatus 1 utilizes the correlation of the predicted residuals of the first color difference component (Cb component) and the second color difference component (Cr component) to obtain the predicted residual of the first color difference component and the second. One joint predicted residual is generated from the predicted residual of the color difference component. For example, the coding apparatus 1 generates a joint predicted residual by synthesizing the predicted residual of the second color difference component with the predicted residual of the first color difference component. Then, the coding apparatus 1 performs conversion processing, quantization processing, and entropy coding processing on the generated joint predicted residuals and transmits the generated joint prediction residuals.

The decoding device 2 reconstructs the predicted residual of the first color difference component and the predicted residual of the second color difference component from the transmitted joint predicted residual. In this way, the coding efficiency is improved by transmitting only one congruent predicted residual for the two color difference components.

As described above, the deblocking filter device according to the present embodiment is a deblocking filter (160, 230) that performs a filtering process on the boundary between the first reconstructed block (block P) and the second reconstructed block (block Q). And JCCR in which at least one of the first reconstructed block (block P) and the second reconstructed block (block Q) generates one congruent predicted residual from the respective predicted residuals of the Cb color difference component and the Cr color difference component. It is provided with a filter control unit (161, 231) for controlling the boundary filter strength Bs of the deblocking filter (160, 230) based on whether or not it is encoded using (Joint coding of chroma residual).

The filter control unit (161, 231) is a deblocking filter (160,) when at least one of the first reconstruction block (block P) and the second reconstruction block (block Q) is encoded using JCCR. The boundary filter strength Bs may be controlled so as to perform the filtering process according to 230).

The filter control unit (161, 231) performs the first reconstruction even when the non-zero conversion coefficient does not exist in both the first reconstruction block (block P) and the second reconstruction block (block Q). When at least one of the block (block P) and the second reconstruction block (block Q) is encoded using JCCR, the boundary filter strength is to be filtered by the deblocking filter (160, 230). Bs may be controlled.

In the present embodiment, the entropy decoding unit 200 of the decoding device 2 sets a flag indicating whether or not it is encoded by using JCCR in the first reconstruction block (block P) and the second reconstruction block (block Q). You may get each one. The filter control unit 231 may control the boundary filter strength Bs of the deblocking filter 230 based on the flags for each of the first reconstruction block (block P) and the second reconstruction block (block Q). For example, the filter control unit 231 filters the deblocking filter 230 when the flag of at least one of the first reconstruction block (block P) and the second reconstruction block (block Q) is “1”. The boundary filter strength Bs may be controlled so as to perform the above.

In the present embodiment, the entropy decoding unit 200 of the decoding device 2 has tu_coded_flag indicating whether or not it includes a non-zero conversion coefficient in the first reconstruction block (block P) and the second reconstruction block (block Q), respectively. You may get more about. The filter control unit 231 may further control the boundary filter strength Bs of the deblocking filter 230 based on the tu_coded_flag for each of the first reconstruction block (block P) and the second reconstruction block (block Q). .. For example, when the tu_coded_flag of at least one of the first reconstructed block (block P) and the second reconstructed block (block Q) is "1", the filter control unit 231 performs the filter processing by the deblocking filter 230. The boundary filter strength Bs may be controlled as such.

For each block of the first color difference component (Cb component) and the second color difference component (Cr component) to which JCCR is applied, although the above-mentioned tu_coded_flag is set to "1", it is actually converted to one color component. There will be no coefficient. Therefore, as in the above-mentioned modification 2, whether or not the reconstructed predicted residual includes a non-zero value instead of determining whether or not a non-zero conversion coefficient exists for a certain color component. It is preferable to use the determination.

[Other embodiments]
In the above-described embodiment, an example of controlling the boundary filter strength Bs as the control of the deblocking filter 230 has been mainly described. However, the control is not limited to the control of the boundary filter strength Bs, and the filter length may be controlled or a plurality of filters may be switched.

A program for causing a computer to execute each process performed by the above-mentioned coding device 1 may be provided. Further, a program for causing the computer to execute each process performed by the decoding device 2 may be provided. The program may be recorded on a computer-readable medium. Computer-readable media can be used to install programs on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, but may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

A circuit that executes each process performed by the coding device 1 may be integrated, and the coding device 1 may be configured by a semiconductor integrated circuit (chipset, SoC). A circuit that executes each process performed by the decoding device 2 may be integrated, and the decoding device 2 may be configured by a semiconductor integrated circuit (chipset, SoC).

Although the embodiments have been described in detail with reference to the drawings above, the specific configuration is not limited to the above, and various design changes and the like can be made within a range that does not deviate from the gist.

This application claims the priority of Japanese Patent Application No. 2020-101293 (filed on June 10, 2020) and Japanese Patent Application No. 2020-101913 (filed on June 11, 2020). All are incorporated herein by reference.

Claims

It is a decoding device that decodes the block obtained by dividing the original image.
An entropy decoding unit that outputs the conversion coefficient corresponding to the block by decoding the coded stream, and
An inverse quantization / inverse transformation unit that restores the predicted residual corresponding to the block by performing inverse quantization processing and inverse transformation processing on the conversion coefficient.
A compositing unit that restores the block by synthesizing the restored predicted residual and the predicted block obtained by predicting the block.
A deblocking filter that filters the boundary of a pair of blocks consisting of the restored block and the restored block adjacent to the block.
Is at least one of the pair of blocks encoded using a JCCR (Joint coding of chroma manual) that produces one joint predicted residual from the respective predicted residuals of the Cb color difference component and the Cr color difference component? A decoding device including a filter control unit that controls the boundary filter strength of the deblocking filter based on whether or not the deblocking filter is used.
The decoding device according to claim 1, wherein the filter control unit controls the boundary filter strength so as to perform the filter processing when at least one block of the pair of blocks is encoded by using the JCCR. ..
The filter control unit may use the JCCR to encode at least one block of the pair of blocks, even if both of the pair of blocks do not have a non-zero conversion factor. The decoding device according to claim 2, wherein the boundary filter strength is controlled so as to perform the filtering process.
The entropy decoding unit acquires a flag indicating whether or not it is encoded using the JCCR for each block of the pair of blocks.
The decoding device according to any one of claims 1 to 3, wherein the filter control unit controls the boundary filter strength of the deblocking filter based on the flag for each block of the pair of blocks.
The decoding device according to claim 4, wherein the filter control unit controls the boundary filter strength so as to perform the filter processing when the flag of at least one block of the pair of blocks is "1".
The entropy decoding unit further acquires tu_coded_flag indicating whether or not it contains a non-zero conversion coefficient for each block of the pair of blocks.
The decoding device according to claim 4 or 5, wherein the filter control unit controls the boundary filter strength of the deblocking filter based on the tu_coded_flag for each block of the pair of blocks.
The decoding device according to claim 6, wherein the filter control unit controls the boundary filter strength so as to perform the filter processing when the tu_coded_flag of at least one of the pair of blocks is "1".
A program characterized in that a computer functions as the decryption device according to any one of claims 1 to 7.