WO2021040037A1 - 符号化装置、復号装置、及びプログラム - Google Patents

符号化装置、復号装置、及びプログラム Download PDF

Info

Publication number
WO2021040037A1
WO2021040037A1 PCT/JP2020/032783 JP2020032783W WO2021040037A1 WO 2021040037 A1 WO2021040037 A1 WO 2021040037A1 JP 2020032783 W JP2020032783 W JP 2020032783W WO 2021040037 A1 WO2021040037 A1 WO 2021040037A1
Authority
WO
WIPO (PCT)
Prior art keywords
block
unit
prediction
filter
boundary
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/032783
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
俊輔 岩村
市ヶ谷　敦郎
慎平 根本
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Japan Broadcasting Corp
Original Assignee
Nippon Hoso Kyokai NHK
Japan Broadcasting Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nippon Hoso Kyokai NHK, Japan Broadcasting Corp filed Critical Nippon Hoso Kyokai NHK
Priority to CN202080060836.4A priority Critical patent/CN114365487B/zh
Priority to JP2021543093A priority patent/JP7474772B2/ja
Publication of WO2021040037A1 publication Critical patent/WO2021040037A1/ja
Priority to US17/681,296 priority patent/US12015789B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167—Position within a video image, e.g. region of interest [ROI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures

Definitions

  • the present invention relates to a coding device, a decoding device, and a program.
  • a deblocking filter is adopted as a filter in a coding loop (see, for example, Non-Patent Document 1).
  • the deblocking filter is an in-loop filter for suppressing distortion of the block boundary of a block when encoding processing is performed in block units.
  • the filter control unit that controls the deblocking filter applies the deblocking filter to the block boundary of the two blocks.
  • the filter control unit performs the 2 according to the value of the motion vector of each of the two blocks and the degree of coincidence of the reference destination (reference image). The continuity of one block is determined, and whether or not to apply a deblocking filter to the block boundary of the two blocks is determined.
  • VVC plans to adopt triangle merge mode as a kind of inter-prediction.
  • the coding device divides the coded block into triangle regions, predicts two triangle regions using two motion vectors, and generates two triangle prediction regions. Then, the prediction block of the coded target block is generated by synthesizing the boundary region of the generated plurality of triangle prediction regions by the weighted average.
  • the coding device uses the information about the motion vector used for the triangle synthesis prediction, that is, the information indicating the reference direction, the reference.
  • Information indicating the previous frame, the number of motion vectors (indicating whether it is single prediction or double prediction, and the value of the motion vector are stored in the motion vector storage.
  • the coding apparatus divides the coded block into a region of 4 ⁇ 4 pixels (hereinafter referred to as “4 ⁇ 4 unit”) and belongs to a composite region synthesized by a weighted average of 4 ⁇ 4.
  • 4 ⁇ 4 unit a region of 4 ⁇ 4 pixels
  • the dual prediction is applied, and the information about the two motion vectors is stored in the motion vector storage.
  • the simple prediction is applied, and the information regarding one of the two motion vectors defined in advance is stored in the motion vector storage.
  • 4 ⁇ 4 units belonging to a region other than the composite region assuming that simple prediction is applied, information on one motion vector used for generating a predicted image in the region is stored in the motion vector storage.
  • the continuity between the blocks is determined based on the information about the motion vector of the two blocks. ing.
  • the weighted average is applied to the composition area by the triangle composition prediction. Since the conventional continuity judgment does not consider such a weighted average, the deblocking filter is not applied even when a discontinuity occurs at the block boundary of two blocks, resulting in visual deterioration. There is a problem that it ends up.
  • the present invention provides a coding device, a decoding device, and a program that improve image quality and coding efficiency by appropriately controlling the deblocking filter.
  • the coding device is a coding device that performs coding in block units obtained by dividing an image, and operates for each of a plurality of divided regions obtained by dividing the coded block.
  • a composite prediction unit that generates a region prediction image using a vector and synthesizes the boundary regions of the generated plurality of region prediction images by a weighted average to generate a prediction block of the coded target block, and the coded target.
  • a conversion / quantization unit that performs conversion processing and quantization processing on the predicted residual that represents the difference between the block and the prediction block, and an inverse quantization processing and dequantization processing on the conversion coefficient obtained by the conversion / quantization unit.
  • An inverse quantization / inverse conversion unit that performs inverse conversion processing to restore the predicted residual, a synthesis unit that synthesizes the restored predicted residual and the predicted block to restore the coded target block, and the above.
  • the deblocking filter that filters the block boundary of two blocks consisting of the restored coded block and the adjacent block, and the deblocking filter based on the position of the synthetic region synthesized by the synthetic predictor by the weighted average. It includes a filter control unit that controls the blocking filter.
  • the decoding device is a decoding device that decodes in block units obtained by dividing an image, and outputs a conversion coefficient corresponding to the block to be decoded by decoding the coded stream.
  • the decoding unit, the inverse quantization / inverse conversion unit that restores the predicted residual by performing inverse quantization processing and inverse conversion processing on the conversion coefficient output by the entropy decoding unit, and the decoding target block are divided.
  • a region prediction image is generated using the motion vector for each of the plurality of divided regions obtained, and the region of the boundary of the generated plurality of region prediction images is combined by a weighted average to generate a prediction block of the decoding target block.
  • the block boundary of two blocks including the composite prediction unit for restoring the decryption target block by synthesizing the restored prediction residual and the prediction block, and the restored decoding target block and the adjacent block. It is provided with a deblocking filter that performs a filtering process on the deblocking filter, and a filter control unit that controls the deblocking filter based on the position of the composite region synthesized by the composite prediction unit by the weighted average.
  • the gist of the program according to the third aspect is to make the computer function as a coding device according to the first aspect.
  • the gist of the program according to the fourth aspect is to make the computer function as a decoding device according to the second aspect.
  • the present invention it is possible to provide a coding device, a decoding device, and a program that improve image quality and coding efficiency by appropriately controlling the deblocking filter.
  • the coding device and the decoding device according to the embodiment will be described with reference to the drawings.
  • the coding device and the decoding device according to the embodiment encode and decode moving images represented by MPEG, respectively.
  • MPEG moving images represented by MPEG
  • the same or similar parts are designated by the same or similar reference numerals.
  • FIG. 1 is a diagram showing a configuration of a coding device 1 according to the present embodiment.
  • the coding device 1 is a device that performs coding in block units obtained by dividing an image.
  • the coding apparatus 1 synthesizes a block dividing unit 100, a subtracting unit 110, a conversion / quantization unit 120, an entropy coding unit 130, and an inverse quantization / inverse conversion unit 140. It has a unit 150, a deblocking filter 160, a filter control unit 161, a memory 170, and a prediction unit 180.
  • the block division unit 100 divides the input image in units of frames (or pictures) constituting the moving image into a plurality of image blocks, and outputs the image block obtained by the division to the subtraction unit 110.
  • the size of the image block is, for example, 32 ⁇ 32 pixels, 16 ⁇ 16 pixels, 8 ⁇ 8 pixels, 4 ⁇ 4 pixels, or the like.
  • the shape of the image block is not limited to a square and may be a rectangle (non-square).
  • the image block is a unit for coding by the coding device 1 (encoding target block) and a unit for decoding by the decoding device (decoding target block). Such an image block is sometimes called a CU (Coding Unit).
  • the block division unit 100 divides the luminance signal and the color difference signal into blocks.
  • the case where the shape of the block division is the same for the luminance signal and the color difference signal will be mainly described, but the division may be controlled independently for the luminance signal and the color difference signal.
  • the luminance block and the color difference block are not particularly distinguished, they are simply called a coded block.
  • the subtraction unit 110 calculates a prediction residual representing a difference (error) between the coding target block output from the block division unit 100 and the prediction block obtained by predicting the coding target block by the prediction unit 180.
  • the subtraction unit 110 calculates the prediction residual by subtracting each pixel value of the prediction block from each pixel value of the block, and outputs the calculated prediction residual to the conversion / quantization unit 120.
  • the conversion / quantization unit 120 performs conversion processing and quantization processing in block units.
  • the conversion / quantization unit 120 includes a conversion unit 121 and a quantization unit 122.
  • the conversion unit 121 performs conversion processing on the predicted residual output from the subtraction unit 110, calculates a conversion coefficient for each frequency component, and outputs the calculated conversion coefficient to the quantization unit 122.
  • the conversion process (conversion) is a process of converting a signal in the pixel region into a signal in the frequency domain, for example, discrete cosine transform (DCT), discrete sine transform (DST), carunen-reve transform (KLT), and the like. Is an integer transform of.
  • the quantization unit 122 quantizes the conversion coefficient output from the conversion unit 121 using the quantization parameter (Qp) and the quantization matrix, and the quantized conversion coefficient is the entropy coding unit 130 and the inverse quantization / inverse conversion. Output to unit 140.
  • the quantization parameter (Qp) is a parameter commonly applied to each conversion coefficient in the block, and is a parameter that determines the roughness of quantization.
  • the quantization matrix is a matrix having a quantization value as an element when quantizing each conversion coefficient.
  • the entropy coding unit 130 performs entropy coding on the conversion coefficient output from the quantization unit 122, performs data compression to generate a coded stream (bit stream), and encodes the coded stream into the coding device 1. Output to the outside of.
  • a Huffman code Context-based Adaptive Binary Arithmetic Coding; context-adaptive binary arithmetic coding
  • CABAC Context-based Adaptive Binary Arithmetic Coding; context-adaptive binary arithmetic coding
  • the entropy encoding unit 130 acquires information such as the size and shape of each coded block from the block dividing unit 100, and acquires information related to prediction (for example, information on the prediction mode and motion vector) from the prediction unit 180. It also encodes this information.
  • Inverse quantization / inverse conversion unit 140 performs inverse quantization processing and inverse conversion processing in block units.
  • the inverse quantization / inverse conversion unit 140 includes an inverse quantization unit 141 and an inverse conversion unit 142.
  • the inverse quantization unit 141 performs the inverse quantization process corresponding to the quantization process performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores and restores the conversion coefficient by inversely quantizing the conversion coefficient output from the quantization unit 122 using the quantization parameter (Qp) and the quantization matrix. The converted conversion coefficient is output to the inverse conversion unit 142.
  • the inverse conversion unit 142 performs an inverse conversion process corresponding to the conversion process performed by the conversion unit 121. For example, when the conversion unit 121 performs the DCT, the inverse conversion unit 142 performs the inverse DCT. The inverse conversion unit 142 performs an inverse conversion process on the conversion coefficient output from the inverse quantization unit 141 to restore the predicted residual, and outputs the restored predicted residual, which is the restored predicted residual, to the synthesis unit 150. To do.
  • the synthesizing unit 150 synthesizes the restoration prediction residual output from the inverse conversion unit 142 with the prediction block output from the prediction unit 180 in pixel units.
  • the synthesizing unit 150 adds each pixel value of the restoration prediction residual and each pixel value of the prediction block to restore (decode) the coded block, and deblocks the decoded image (restoration block) of the restored block unit. Output to 160.
  • the deblocking filter 160 performs filtering on the block boundary of two blocks (hereinafter referred to as "target block pair") consisting of a restoration block and an adjacent block adjacent to the restoration block, and the restored block after the filtering processing is performed. Output to memory 170.
  • the filter processing is a processing for reducing signal deterioration caused by processing in block units, and is a filtering processing for smoothing a signal gap at a block boundary of a target block pair.
  • the deblocking filter 160 is generally configured as a low-pass filter that moderates signal fluctuations.
  • the filter control unit 161 controls the deblocking filter 160. Specifically, the filter control unit 161 controls the boundary strength (Bs: Boundary strength) indicating whether or not the block boundary of the target block pair is filtered, and the filter strength of the deblocking filter 160.
  • the boundary strength Bs refers to a parameter for determining whether or not to apply the filtering process and the type of the filtering process.
  • the control of whether or not to perform the filtering process can be regarded as the control of whether the boundary strength Bs is 1 or more or zero.
  • the filter control unit 161 controls the deblocking filter 160 based on the fluctuation of the pixel value in the region near the boundary of the target block pair, the prediction mode, the quantization parameter, and the motion vector value used for the motion compensation prediction (inter-prediction). To do. The details of the operation of the filter control unit 161 will be described later.
  • the memory 170 stores the restoration block output from the deblocking filter 160 as a decoded image in frame units.
  • the memory 170 outputs the stored decoded image to the prediction unit 180.
  • the prediction unit 180 generates a prediction block corresponding to the coded target block by performing prediction processing in block units, and outputs the generated prediction block to the subtraction unit 110 and the synthesis unit 150.
  • the prediction unit 180 includes an inter-prediction unit 181, an intra-prediction unit 182, and a switching unit 183.
  • the inter-prediction unit 181 uses the decoded image stored in the memory 170 as a reference image, calculates a motion vector by a method such as block matching, predicts a block to be encoded, and generates an inter-prediction block.
  • the inter-prediction block is output to the switching unit 183.
  • the inter-prediction unit 181 selects the optimum inter-prediction method from inter-prediction using a plurality of reference images (typically bi-prediction) and inter-prediction using one reference image (one-way prediction). Inter-prediction is performed using the selected inter-prediction method.
  • the inter-prediction unit 181 outputs information (motion vector, etc.) related to the inter-prediction to the entropy coding unit 130 and the filter control unit 161.
  • the inter prediction unit 181 has a synthesis prediction unit 181a that performs triangle composition prediction.
  • the composite prediction unit 181a divides the coded block into triangle regions, predicts two triangle regions using two motion vectors, and generates two triangle prediction regions (two region prediction images).
  • the prediction block of the coded block is generated by synthesizing the region of the boundary between the two triangle prediction regions by the weighted average. The details of the triangle composition prediction will be described later.
  • the intra prediction unit 182 selects the optimum intra prediction mode to be applied to the coded target block from the plurality of intra prediction modes, and predicts the coded target block using the selected intra prediction mode.
  • the intra prediction unit 182 generates an intra prediction block by referring to the decoded pixel value adjacent to the coded target block among the decoded images stored in the memory 170, and outputs the generated intra prediction block to the switching unit 183. To do. Further, the intra prediction unit 182 outputs information regarding the selected intra prediction mode to the entropy encoding unit 130 and the filter control unit 161.
  • the switching unit 183 switches between the inter prediction block output from the inter prediction unit 181 and the intra prediction block output from the intra prediction unit 182, and outputs one of the prediction blocks to the subtraction unit 110 and the synthesis unit 150.
  • the coding apparatus 1 generates a region prediction image using motion vectors for each of the plurality of divided regions obtained by dividing the coded block, and generates a plurality of region predictions.
  • the synthetic prediction unit 181a that generates the prediction block of the coded target block by synthesizing the boundary area of the image by the weighted average, and the conversion process and the prediction residual that represents the difference between the coded target block and the predicted block.
  • a conversion / quantization unit 120 that performs quantization processing, and an inverse quantization / inverse conversion unit that restores the predicted residual by performing dequantization processing and deconversion processing on the conversion coefficient obtained by the conversion / quantization unit.
  • Filtering is performed on the block boundary of two blocks consisting of 140, a synthesis unit 150 that synthesizes the restored predicted residual and the predicted block to restore the coded block, and the restored coded block and adjacent blocks. It has a deblocking filter 160 to perform.
  • 2 to 5 are diagrams showing the operation of the synthesis prediction unit 181a according to the present embodiment.
  • the synthesis prediction unit 181a divides the coded block into two triangular regions (Partition 1 and Partition 2). There are two types of division methods, the method shown in FIG. 2 (a) and the method shown in FIG. 2 (b).
  • the synthesis prediction unit 181a divides the coded block by a dividing line passing through the upper left zenith position and the lower right zenith position.
  • the synthesis prediction unit 181a divides the coded block by a dividing line passing through the upper right zenith position and the lower left zenith position.
  • the composite prediction unit 181a sets the motion vector for each triangular region by using the motion vector reference source as a candidate.
  • the composite prediction unit 181a arranges the seven types of candidates shown in FIGS. 3 to 7 in a predetermined priority order, and selects one type of motion vector for each triangular region from the top five motion vectors.
  • the composite prediction unit 181a synthesizes a prediction image for each triangle region (hereinafter referred to as "region prediction image") after performing motion compensation prediction for each triangle region using the motion vector of each triangle region. By doing so, a predicted image (predicted block) of the coded block is generated.
  • region prediction image a prediction image for each triangle region
  • the boundary between the two region prediction images is smoothly combined by the weighted average (weighted average) to suppress the discontinuity of the boundary between the two region prediction images.
  • the predicted images in the triangle synthesis prediction are the predicted image 1 (“P 1 ” in FIG. 4) generated by the motion vector applied to Partition 1 in the coded block, and the motion vector applied to Partition 2.
  • the predicted image 2 (“P 2 ” in FIG. 4) generated by the above is synthesized by a weighted average using a weight map (Weight map) according to the block size and block shape of the block to be encoded.
  • the region composed of pixels in which any of “1” to “7” has a weighting coefficient is a composite region synthesized by the weighted average.
  • the coded block may be divided into 4 ⁇ 4 pixel areas (4 ⁇ 4 units), and it may be determined whether or not each 4 ⁇ 4 unit is a composite area. Whether or not each 4 ⁇ 4 unit is a composite area is determined to be a composite area if at least one of “1” to “7” is included in the weighting coefficient set for the pixels in the 4 ⁇ 4 unit. Alternatively, it may be determined as a composite region when the weighting coefficient set for the pixel of the 4 ⁇ 4 unit does not include “0” or “8”.
  • the composite prediction unit 181a stores the motion vector used for the triangle composite prediction in the motion vector storage.
  • the motion vector used for the coded block may be reused when the adjacent block is coded, and the motion vector is stored in the motion vector storage for the reuse of the motion vector.
  • the motion vector storage is provided in the synthesis prediction unit 181a or the memory 170.
  • the synthesis prediction unit 181a divides the coded block into a 4 ⁇ 4 pixel area and prepares a motion vector storage for each 4 ⁇ 4 unit.
  • the composite prediction unit 181a determines the information about the corresponding motion vector, that is, the reference direction of the motion vector, the information indicating the reference frame of the motion vector, and the number of motion vectors (single prediction or double prediction) with respect to the motion vector storage. Show), and the value of the motion vector, etc. are stored.
  • the boundary region between Partition 1 (region 1) and Partition 2 (region 2) of the coded block was synthesized by the weighted average. It is a synthetic area.
  • the composite prediction unit 181a used each 4 ⁇ 4 unit included in such a composite region in Partition 1 as a dual prediction when the reference directions of the motion vectors applied to Partition 1 and Partition 2 are different. Information about two motion vectors, the motion vector and the motion vector used in Partition 2, is stored in the motion vector storage. On the other hand, when the reference directions of the motion vectors applied to the partition 1 and the partition 2 are the same, the information about the motion vector used in the partition 2 is stored in the motion vector storage as a simple prediction.
  • the composite prediction unit 181a stores information about one motion vector used in Partition 1 as a simple prediction for each 4 ⁇ 4 unit included in the region to which the weighted average is not applied in Partition 1 in the motion vector storage. Store. In addition, the composite prediction unit 181a stores information about one motion vector used in Partition 2 as a simple prediction for each 4 ⁇ 4 unit included in the region to which the weighted average is not applied in Partition 2 in the motion vector storage. Store.
  • FIGS. 4 and 5 the case where the area is divided diagonally is described, and when the area is divided diagonally, the motion vector information is stored in the motion vector storage so as to be line-symmetrically inverted to the left and right. To store.
  • FIG. 6 is a diagram showing an operation example of the deblocking filter 160 according to the present embodiment.
  • the deblocking filter 160 performs filter processing on the block boundary of each block of 8 ⁇ 8 pixels. Further, the deblocking filter 160 performs filter processing in units of 4 rows or 4 columns.
  • the blocks P and Q shown in FIG. 6 are one unit of the filter processing of the deblocking filter 160, and show an example in which the block size is 4 ⁇ 4 pixels. Each of the blocks P and Q may be referred to as a subblock.
  • the filter control unit 161 obtains the boundary strength Bs based on, for example, Table 1 below.
  • the value of the boundary strength Bs is one of 0, 1, and 2.
  • the boundary strength of the luminance signal and the color difference signal for each block may be calculated, or the combination of the boundary intensity of the luminance signal and the color difference signal block may be determined as one boundary intensity.
  • the filter control unit 161 sets the value of Bs to 2 when the intra prediction is applied to at least one of the blocks P and Q.
  • the filter control unit 161 sets the Bs value to 1 when the inter-prediction is applied to both the blocks P and Q and at least one of the following (a) to (d) is satisfied. In other cases, the Bs value is set to 0.
  • At least one block of blocks P and Q belongs to the composite region of the triangle composite prediction. (In other words, the boundary between blocks P and Q is adjacent to the composite region of the triangle composite prediction.)
  • At least one of blocks P and Q contains a significant conversion factor (ie, a non-zero conversion factor).
  • the filter control unit 161 can change the priority of applying the conditions shown in Table 1 within a range in which the Bs values determined as a result do not differ.
  • the filter control unit 161 controls the deblocking filter 160 so that the filter processing is not performed when the value of the boundary strength Bs is 0.
  • the vertical block boundary shown in FIG. 6 will be described as an example.
  • the filter control unit 161 may control the deblocking filter 160 so as to perform the filter processing only when the following equation (1) is satisfied.
  • the filter control unit 161 may apply a strong filter when all of the following conditional expressions (2) to (7) are satisfied, and may apply a weak filter in other cases.
  • the values of the threshold values ⁇ and t C change according to the average value Qav of the quantization parameters of the adjacent blocks P and Q.
  • the filter control unit 161 controls the deblocking filter based on the position of the composite region synthesized by the composite prediction unit 181a by the weighted average as described in (a) above. Specifically, the filter control unit 161 controls the deblocking filter so that when the block boundaries of the blocks P and Q (two subblocks) are adjacent to the composite region, the block boundaries are filtered.
  • the discontinuity of the block boundary caused by the composite region can be suppressed by the deblocking filter processing, and the visual sense Deterioration can be suppressed.
  • FIG. 7 is a diagram for explaining a specific example of the operation of the filter control unit 161 according to the present embodiment.
  • the two motion vectors used by block P for bi-prediction are P0 and P1.
  • the two motion vectors that block Q uses for bi-prediction are Q0 and Q1.
  • the reference destinations of P0 and Q0 are the same frame, and the reference destinations of P1 and Q1 are the same frame. Further, the difference between the motion vector values of P0 and Q0 is equal to or less than the threshold value, and the difference between the motion vector values of P1 and Q1 is equal to or less than the threshold value.
  • condition (a) above is newly introduced. Therefore, when at least one of the blocks P and Q is the composite region of the triangle synthesis prediction, it is possible to control the filtering process for the boundary between the blocks P and Q, so that the discontinuity of the boundaries between the blocks P and Q can be suppressed. ..
  • FIG. 8 is a diagram showing a specific example of the operation of the deblocking filter 160 and the filter control unit 161 according to the present embodiment.
  • the filter control unit 161 sets two subblocks (blocks P and Q) straddling the block boundary between the coded block and the adjacent block adjacent to the coded block. Filter processing is performed as one unit.
  • the filter control unit 161 performs the deblocking filter 160 so as to perform the filter processing on the subblock boundary. To control.
  • the filter control unit 161 refers to the motion vector storage and positions the position corresponding to the subblock.
  • the motion vector information of the 4 ⁇ 4 unit is acquired, and it is determined whether or not the two motion vectors are stored (that is, double prediction).
  • this subblock belongs to the composite area of the triangle composite prediction.
  • the filter control unit 161 controls the deblocking filter 160 so as to filter the subblock boundaries of the two subblocks (blocks P and Q).
  • this sub Whether or not the block belongs to the composite region is determined based on the weighting coefficient set for each pixel in the subblock.
  • the weighting coefficient set for the pixel included in the subblock does not include "0" or "8"
  • the system can preliminarily synthesize the size and shape of the coded block and the divided shape of the triangular area.
  • the area determination table may be prepared and the determination may be made by referring to the area determination table.
  • the filter control unit 161 will perform two subblocks (block P). And Q), the deblocking filter 160 is controlled so as to perform filtering on the subblock boundary.
  • the filter control unit 161 has a non-zero conversion coefficient in each of the two subblocks. Controls whether or not to filter the subblock boundaries based on the presence or absence of and at least one of the information about each motion vector of the two subblocks.
  • the filter control unit 161 determines the presence or absence of a non-zero conversion coefficient in each of the two subblocks. And based on at least one of the information about the motion vector of each of the two subblocks, it is controlled whether or not the subblock boundary is filtered.
  • FIG. 9 is a diagram showing a configuration of a decoding device 2 according to the present embodiment.
  • the decoding device 2 is a device that decodes the decoding target block from the coded stream.
  • the decoding device 2 predicts the entropy decoding unit 200, the inverse quantization / inverse conversion unit 210, the synthesis unit 220, the deblocking filter 230, the filter control unit 231 and the memory 240. It has a part 250.
  • the entropy decoding unit 200 decodes the coded stream generated by the coding device 1 and decodes various signaling information. Specifically, the entropy decoding unit 200 acquires information related to the prediction applied to the decoding target block (for example, prediction type information, motion vector information), and outputs the acquired information to the prediction unit 250 and the filter control unit 231. To do.
  • information related to the prediction applied to the decoding target block for example, prediction type information, motion vector information
  • the entropy decoding unit 200 decodes the coded stream, acquires the quantized conversion coefficient, and outputs the acquired conversion coefficient to the inverse quantization / inverse conversion unit 210 (inverse quantization unit 211).
  • Inverse quantization / inverse conversion unit 210 performs inverse quantization processing and inverse conversion processing in block units.
  • the inverse quantization / inverse conversion unit 210 includes an inverse quantization unit 211 and an inverse conversion unit 212.
  • the inverse quantization unit 211 performs the inverse quantization process corresponding to the quantization process performed by the quantization unit 122 of the coding device 1.
  • the inverse quantization unit 211 restores the conversion coefficient of the decoding target block by inversely quantizing the quantization conversion coefficient output from the entropy decoding unit 200 using the quantization parameter (Qp) and the quantization matrix. , The restored conversion coefficient is output to the inverse conversion unit 212.
  • the inverse conversion unit 212 performs an inverse conversion process corresponding to the conversion process performed by the conversion unit 121 of the coding device 1.
  • the inverse conversion unit 212 performs inverse conversion processing on the conversion coefficient output from the inverse quantization unit 211 to restore the predicted residual, and outputs the restored predicted residual (restored predicted residual) to the synthesis unit 220. To do.
  • the synthesizing unit 220 restores (decodes) the decoding target block by synthesizing the prediction residual output from the inverse conversion unit 212 and the prediction block output from the prediction unit 250 on a pixel-by-pixel basis, and restores (decodes) the restoration block. Output to the deblocking filter 230.
  • the deblocking filter 230 operates in the same manner as the deblocking filter 160 of the coding device 1.
  • the deblocking filter 230 performs filter processing on the boundary of the target block pair consisting of the restoration block output from the synthesis unit 220 and the block adjacent to the restoration block, and outputs the restored block after the filtering processing to the memory 240.
  • the filter control unit 231 performs the same operation as the filter control unit 161 of the coding device 1 based on the information output from the entropy decoding unit 200.
  • the filter control unit 231 selects the boundary strength Bs by the method shown in Table 1, for example, and controls the deblocking filter 230 according to the selected boundary strength Bs.
  • the memory 240 stores the restoration block output from the deblocking filter 230 as a decoded image in frame units.
  • the memory 240 outputs a frame-by-frame decoded image to the outside of the decoding device 2.
  • the prediction unit 250 makes a prediction in block units.
  • the prediction unit 250 includes an inter-prediction unit 251, an intra-prediction unit 252, and a switching unit 253.
  • the inter-prediction unit 251 uses the decoded image stored in the memory 240 as a reference image to predict the decoding target block by inter-prediction.
  • the inter-prediction unit 251 generates an inter-prediction block by performing inter-prediction using the motion vector information output from the entropy decoding unit 200, and outputs the generated inter-prediction block to the switching unit 253.
  • the inter prediction unit 251 has a synthesis prediction unit 251a that performs triangle composition prediction.
  • the composite prediction unit 251a divides the decoding target block into triangle regions, predicts two triangle regions using two motion vectors, generates two triangle prediction regions, and generates a boundary between the two triangle prediction regions. By synthesizing the regions of the above by the weighted average, the predicted block of the coded block is generated.
  • the operation of the synthesis prediction unit 251a is the same as the operation of the synthesis prediction unit 181a of the coding device 1. However, the flag indicating that the triangle composition prediction is applied, the flag indicating the division method of the triangle composition prediction (FIGS. 2A or 2B), and the like are transmitted from the coding device 1 to the decoding device 2 by the coding stream. Signaled. The synthesis prediction unit 251a performs triangle synthesis prediction based on the signaled information.
  • the intra prediction unit 252 refers to the reference pixel adjacent to the decoding target block among the decoded images stored in the memory 240, and predicts the decoding target block by intra prediction based on the information output from the entropy decoding unit 200. .. Then, the intra prediction unit 252 generates an intra prediction block, and outputs the generated intra prediction block to the switching unit 253.
  • the switching unit 253 switches between the inter prediction block output from the inter prediction unit 251 and the intra prediction block output from the intra prediction unit 252, and outputs one of the prediction blocks to the synthesis unit 220.
  • the decoding device 2 has the entropy decoding unit 200 that outputs the conversion coefficient corresponding to the decoding target block by decoding the coded stream, and the conversion coefficient output by the entropy decoding unit 200.
  • the inverse quantization / inverse conversion unit 210 that performs the inverse quantization processing and the inverse conversion processing to restore the predicted residual, and the motion vector for each of the plurality of divided regions obtained by dividing the block to be decoded are used.
  • a composite prediction unit 251a that generates a region prediction image and synthesizes the boundary regions of a plurality of generated region prediction images by a weighted average to generate a prediction block of a block to be decoded, and a restored prediction residual and a prediction block. It has a compositing unit 220 that synthesizes and restores the decoding target block, and a deblocking filter 230 that performs a filtering process on the block boundary of two blocks including the restored decoding target block and the adjacent block.
  • the deblocking filter 230 performs filter processing with two subblocks (blocks P and Q) straddling the block boundary as one unit.
  • the filter control unit 231 controls the deblocking filter 230 so as to perform a filter process on the subblock boundary when the subblock boundary of the two subblocks is adjacent to the synthesis region of the triangle composition prediction.
  • the filter control unit 231 determines the presence or absence of a non-zero conversion coefficient in each of the two subblocks. And based on at least one of the information about the motion vector of each of the two subblocks, it is controlled whether or not the subblock boundary is filtered.
  • FIG. 10 is a diagram showing an example of an operation flow of the filter control unit 231 according to the present embodiment.
  • step S1 the filter control unit 231 determines whether or not the intra prediction is applied to at least one of the blocks P and Q.
  • step S5 the filter control unit 231 includes a non-zero conversion coefficient in at least one of blocks P and Q. Judge whether or not.
  • step S7 the filter control unit 231 determines whether or not the difference between the motion vectors of blocks P and Q is equal to or greater than the threshold value. Is determined.
  • step S8 the filter control unit 231 filters.
  • step S3 may be performed after the determination in step S7.
  • a program may be provided that causes a computer to execute each process performed by the coding device 1.
  • a program may be provided that causes the computer to execute each process performed by the decoding device 2.
  • the program may be recorded on a computer-readable medium.
  • Computer-readable media can be used to install programs on a computer.
  • the computer-readable medium on which the program is recorded may be a non-transient recording medium.
  • the non-transient recording medium is not particularly limited, but may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.
  • a circuit that executes each process performed by the coding device 1 may be integrated, and the coding device 1 may be configured by a semiconductor integrated circuit (chipset, SoC).
  • a circuit that executes each process performed by the decoding device 2 may be integrated, and the decoding device 2 may be configured by a semiconductor integrated circuit (chipset, SoC).

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
PCT/JP2020/032783 2019-08-29 2020-08-28 符号化装置、復号装置、及びプログラム Ceased WO2021040037A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202080060836.4A CN114365487B (zh) 2019-08-29 2020-08-28 编码装置、解码装置及程序
JP2021543093A JP7474772B2 (ja) 2019-08-29 2020-08-28 符号化装置、復号装置、及びプログラム
US17/681,296 US12015789B2 (en) 2019-08-29 2022-02-25 Encoding device, decoding device and program for triangle merge protection on boundary between two blocks

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-157493 2019-08-29
JP2019157493 2019-08-29

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/681,296 Continuation US12015789B2 (en) 2019-08-29 2022-02-25 Encoding device, decoding device and program for triangle merge protection on boundary between two blocks

Publications (1)

Publication Number Publication Date
WO2021040037A1 true WO2021040037A1 (ja) 2021-03-04

Family

ID=74685106

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/032783 Ceased WO2021040037A1 (ja) 2019-08-29 2020-08-28 符号化装置、復号装置、及びプログラム

Country Status (4)

Country Link
US (1) US12015789B2 (https=)
JP (1) JP7474772B2 (https=)
CN (1) CN114365487B (https=)
WO (1) WO2021040037A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12368883B2 (en) * 2022-04-11 2025-07-22 Qualcomm Incorporated Block vector difference binarization and coding in video coding

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018123444A1 (ja) * 2016-12-28 2018-07-05 ソニー株式会社 画像処理装置及び画像処理方法
WO2019039322A1 (en) * 2017-08-22 2019-02-28 Panasonic Intellectual Property Corporation Of America IMAGE ENCODER, IMAGE DECODER, IMAGE ENCODING METHOD, AND IMAGE DECODING METHOD

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105282558B (zh) * 2014-07-18 2018-06-15 清华大学 帧内像素预测方法、编码方法、解码方法及其装置
WO2017030199A1 (ja) * 2015-08-20 2017-02-23 日本放送協会 画像符号化装置、画像復号化装置、及びこれらのプログラム
MY185713A (en) * 2015-09-25 2021-05-31 Huawei Tech Co Ltd Apparatus and method for video motion compensation
JP6985287B2 (ja) * 2016-11-28 2021-12-22 日本放送協会 符号化装置、復号装置、符号化方法、及び復号方法
WO2020094051A1 (en) * 2018-11-06 2020-05-14 Beijing Bytedance Network Technology Co., Ltd. Signaling of side information for inter prediction with geometric partitioning
CN116506598B (zh) * 2018-11-08 2025-08-12 Oppo广东移动通信有限公司 视频信号编码/解码方法以及用于所述方法的设备
CN119450038A (zh) * 2018-12-11 2025-02-14 交互数字Vc控股公司 用于对图像进行解块的方法和设备
CN118612430A (zh) * 2019-02-08 2024-09-06 松下电器（美国）知识产权公司 编码装置、解码装置以及非暂时性计算机可读介质

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018123444A1 (ja) * 2016-12-28 2018-07-05 ソニー株式会社 画像処理装置及び画像処理方法
WO2019039322A1 (en) * 2017-08-22 2019-02-28 Panasonic Intellectual Property Corporation Of America IMAGE ENCODER, IMAGE DECODER, IMAGE ENCODING METHOD, AND IMAGE DECODING METHOD

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
SHUNSUKE IWAMURA ET AL.: "[AHG18][non-CE5] Boundary strength derivation for CUs with TPM", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, 16TH MEETING, no. JVET-P0602 v1, October 2019 (2019-10-01), Geneva, CH, pages 1 - 4, XP030217825 *
SHUNSUKE IWAMURA ET AL.: "Non-CE5: Boundary strength derivation of deblocking filter for transform-skipped block", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11,15TH MEETING, no. JVET-O0656, 26 June 2019 (2019-06-26), Gothenburg, SE, pages 1 - 6, XP030220181 *
XUEWEI MENG ET AL.: "AHG18/Non-CE5: Deblocking for TPM and BCW", 16TH MEETING JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, no. JVET- P0269-v2, 3 October 2019 (2019-10-03), Geneva, CH, pages 1 - 7, XP030216761 *

Also Published As

Publication number Publication date
CN114365487A (zh) 2022-04-15
JP7474772B2 (ja) 2024-04-25
CN114365487B (zh) 2024-08-13
US20220182646A1 (en) 2022-06-09
US12015789B2 (en) 2024-06-18
JPWO2021040037A1 (https=) 2021-03-04

Similar Documents

Publication Publication Date Title
CN105409221B (zh) 用于样本自适应偏移滤波的编码器侧决策
JP2023143946A (ja) クロマデブロックフィルタリングのための量子化パラメータオフセット
US12137254B2 (en) Encoding device, decoding device and program
JP7749052B2 (ja) 予測装置、符号化装置、復号装置、及びプログラム
US20250097419A1 (en) Encoding device, decoding device and program
CN114830641A (zh) 图像编码方法和图像解码方法
JP7474772B2 (ja) 符号化装置、復号装置、及びプログラム
CN114830646A (zh) 图像编码方法和图像解码方法
JP7357721B2 (ja) デブロッキングフィルタ制御装置及びプログラム
WO2021251398A1 (ja) デブロッキングフィルタ装置、復号装置、及びプログラム
CN114830644A (zh) 图像编码方法和图像解码方法
CN114830647A (zh) 图像编码方法和图像解码方法
CN114830642A (zh) 图像编码方法和图像解码方法
JP2021052249A (ja) 符号化装置、復号装置、及びプログラム
CN114830650A (zh) 图像编码方法和图像解码方法
CN114788270A (zh) 图像编码方法和图像解码方法
CN114830645A (zh) 图像编码方法和图像解码方法
WO2021045059A1 (ja) 符号化装置、復号装置、及びプログラム
JP2020195063A (ja) イントラ予測装置、画像符号化装置、画像復号装置、及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20857106

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021543093

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20857106

Country of ref document: EP

Kind code of ref document: A1