WO2023055220A1 - Procédé de traitement de signal vidéo pour déterminer un mode de prédiction intra sur la base d'une image de référence, et dispositif associé - Google Patents

Procédé de traitement de signal vidéo pour déterminer un mode de prédiction intra sur la base d'une image de référence, et dispositif associé Download PDF

Info

Publication number
WO2023055220A1
WO2023055220A1 PCT/KR2022/014893 KR2022014893W WO2023055220A1 WO 2023055220 A1 WO2023055220 A1 WO 2023055220A1 KR 2022014893 W KR2022014893 W KR 2022014893W WO 2023055220 A1 WO2023055220 A1 WO 2023055220A1
Authority
WO
WIPO (PCT)
Prior art keywords
block
intra prediction
current block
prediction mode
decoder
Prior art date
Application number
PCT/KR2022/014893
Other languages
English (en)
Korean (ko)
Inventor
김경용
김동철
손주형
곽진삼
Original Assignee
주식회사 윌러스표준기술연구소
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 주식회사 윌러스표준기술연구소 filed Critical 주식회사 윌러스표준기술연구소
Publication of WO2023055220A1 publication Critical patent/WO2023055220A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/43—Hardware specially adapted for motion estimation or compensation
    • H04N19/433—Hardware specially adapted for motion estimation or compensation characterised by techniques for memory access
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/96—Tree coding, e.g. quad-tree coding

Definitions

  • the present invention relates to a method and apparatus for processing a video signal, and more particularly, to a method and apparatus for processing a video signal for encoding or decoding a video signal.
  • Compression coding refers to a series of signal processing techniques for transmitting digitized information through a communication line or storing it in a form suitable for a storage medium.
  • Targets of compression coding include voice, video, text, and the like, and in particular, a technique of performing compression coding for video is called video image compression.
  • Compression encoding of a video signal is performed by removing redundant information in consideration of spatial correlation, temporal correlation, and stochastic correlation.
  • a more highly efficient video signal processing method and apparatus are required.
  • An object of the present specification is to increase coding efficiency of a video signal by providing a video signal processing method and an apparatus therefor.
  • a video signal decoding apparatus includes a processor.
  • the processor When reconstructing a current block using intra prediction, the processor derives an intra prediction mode corresponding to a neighboring block based on motion information of a neighboring block of the current block and a reference picture to which the neighboring block refers, and The current block is reconstructed using intra prediction modes corresponding to neighboring blocks.
  • the neighboring blocks are reconstructed using inter prediction.
  • the processor determines an intra prediction mode of a pixel corresponding to any one pixel of neighboring blocks of the current block in the reference picture according to motion information of the neighboring blocks.
  • An intra prediction mode corresponding to a neighboring block may be induced.
  • the processor When reconstructing the current block using the intra prediction, the processor assigns an intra prediction mode of a pixel corresponding to any one pixel of the current block in the reference picture to the neighboring block according to the motion information of the neighboring block.
  • a corresponding intra prediction mode may be induced.
  • the processor may initialize an intra prediction mode corresponding to the neighboring block based on a predefined condition.
  • the predetermined condition may include a condition regarding a difference between a picture order count (POC) of a current picture including the current block and a POC of the reference picture.
  • POC picture order count
  • the predefined condition may include a condition regarding a difference between a decoding order of the current picture including the current block and a decoding order of the reference picture.
  • the processor may set a value of the intra prediction mode corresponding to the neighboring block to a predefined mode.
  • the processor may divide the value of the intra prediction mode by a predetermined value and store the values.
  • the processor may store consecutive values of a predetermined range of values of the intra prediction mode as one value.
  • the processor may generate a most probale mode (MPM) list using an intra prediction mode corresponding to the neighboring block.
  • MPM most probale mode
  • the current block may be a block to which geometric partitioning mode (GPM) is applied.
  • the processor may infer an angle of an oblique line dividing the current block based on an intra prediction mode derived from the reference block, and reconstruct the current block using the inferred angle of the oblique line.
  • a video signal encoding apparatus includes a processor, and the processor obtains a bitstream decoded by a decoding method.
  • the decoding method derives an intra prediction mode corresponding to a neighboring block based on motion information of a neighboring block of the current block and a reference picture to which the neighboring block refers, and restoring the current block using an intra prediction mode corresponding to the neighboring block.
  • the neighboring blocks are reconstructed using inter prediction.
  • the reconstructing of the current block may include changing an intra prediction mode of a pixel corresponding to any one pixel of neighboring blocks of the current block in the reference picture to an intra prediction mode corresponding to the neighboring block according to motion information of the neighboring block. It may include an induction step.
  • the reconstructing of the current block may include inducing an intra prediction mode of a pixel corresponding to any one pixel of the current block in the reference picture to an intra prediction mode corresponding to the neighboring block according to motion information of the neighboring block.
  • the decoding method may further include initializing an intra prediction mode corresponding to the neighboring block based on a predefined condition.
  • the predetermined condition may include a condition regarding a difference between a picture order count (POC) of a current picture including the current block and a POC of the reference picture.
  • POC picture order count
  • the predefined condition may include a condition regarding a difference between a decoding order of the current picture including the current block and a decoding order of the reference picture.
  • the initializing of the intra prediction mode corresponding to the neighboring block based on the predefined condition may include initializing a value of the intra prediction mode corresponding to the neighboring block to the power of the predefined mode.
  • the restoring of the current block may further include generating a most probale mode (MPM) list using an intra prediction mode corresponding to the neighboring block.
  • MPM most probale mode
  • the bitstream is decoded by a decoding method.
  • the decoding method derives an intra prediction mode corresponding to a neighboring block based on motion information of a neighboring block of the current block and a reference picture to which the neighboring block refers, and restoring the current block using an intra prediction mode corresponding to the neighboring block.
  • the neighboring blocks are reconstructed using inter prediction.
  • the present specification provides a method for efficiently processing a video signal.
  • the effects obtainable in the present specification are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.
  • FIG. 1 is a schematic block diagram of a video signal encoding apparatus according to an embodiment of the present invention.
  • FIG. 2 is a schematic block diagram of a video signal decoding apparatus according to an embodiment of the present invention.
  • FIG. 3 shows an embodiment in which a coding tree unit within a picture is divided into coding units.
  • FIG. 4 illustrates one embodiment of a method for signaling splitting of quad trees and multi-type trees.
  • 5 and 6 show an intra prediction method according to an embodiment of the present invention in more detail.
  • FIG. 7 is a diagram illustrating positions of neighboring blocks used to construct a motion candidate list in inter prediction.
  • FIG. 8 shows that a video signal processing apparatus according to an embodiment of the present invention stores intra- and inter-prediction modes of a current block.
  • FIG. 9 shows prediction of a current block using information on neighboring blocks of the current block described with reference to FIG. 8 .
  • FIG. 10 is an intra prediction corresponding to a block reconstructed by using an intra prediction mode of a block referred to in intra prediction using motion information of a block reconstructed by using inter prediction by a decoder according to an embodiment of the present invention. shows what leads to the mode.
  • FIG. 11 shows derivation of an intra prediction mode of a neighboring block of a reconstructed current block using inter prediction when a decoder according to an embodiment of the present invention reconstructs a current block using intra prediction.
  • FIG. 12 shows a process in which an intra prediction mode corresponding to a neighboring block is derived by a decoder according to an embodiment of the present invention by applying motion information of a neighboring block reconstructed using inter prediction to a pixel of a current block.
  • FIG. 13 shows a process in which an intra prediction mode corresponding to a neighboring block is derived by a decoder according to an embodiment of the present invention by applying motion information of a neighboring block reconstructed using inter prediction to a pixel of a subblock of a current block.
  • FIG. 14 shows a method of storing motion information for a current block when the decoder reconstructs the current block using intra prediction according to an embodiment of the present invention.
  • a decoder according to an embodiment of the present invention stores an intra prediction mode determined in an I picture as an intra prediction mode corresponding to a block reconstructed using inter prediction in a next picture.
  • 16 shows that a bitstream of a specific picture is dropped due to a network condition when a decoder according to an embodiment of the present invention performs decoding.
  • FIG. 17 shows a method of storing an intra prediction mode corresponding to a block reconstructed through inter prediction by a decoder according to an embodiment of the present invention.
  • FIG. 18 shows that a decoder according to an embodiment of the present invention derives an intra prediction mode from a block encoded in CIIP mode.
  • FIG. 19 shows that a decoder according to an embodiment of the present invention reconstructs a current block using GPM.
  • FIG. 22 shows that an encoder and a decoder perform transform on a residual block to obtain transform coefficients and perform inverse quantization on a quantized transform coefficient to reconstruct a residual block according to an embodiment of the present invention.
  • FIG. 23 illustrates a process in which an encoder and a decoder derive a transform type using at least one of information on a current block and neighboring blocks, an intra prediction mode, an encoding mode, and parsed transform type index information according to an embodiment of the present invention.
  • FIG 24 shows an index of a transform type set mapped to an intra-screen orientation mode (0 to 34 and MIP) of a current block and a size index (0 to 15) of the current block according to an embodiment of the present invention.
  • 25 shows a conversion type set corresponding to nTrSet according to an embodiment of the present invention.
  • 26 shows a conversion type combination table according to an embodiment of the present invention.
  • FIG. 27 shows a threshold table for IDT conversion types according to an embodiment of the present invention.
  • 29 shows a method of reconstructing a block predicted by intra prediction by a decoder according to an embodiment of the present invention.
  • FIG. 30 shows a transform kernel usable by an encoder and a decoder according to an embodiment of the present invention.
  • 'A and/or B' may be interpreted as meaning 'including at least one of A or B'.
  • Coding can be interpreted as either encoding or decoding, as the case may be.
  • a device that performs encoding (encoding) of a video signal to generate a video signal bitstream is referred to as an encoding device or an encoder
  • a device that performs decoding (decoding) of a video signal bitstream to restore a video signal is referred to as a decoding device.
  • a device or decoder a video signal processing apparatus is used as a conceptual term including both an encoder and a decoder.
  • a 'unit' is used to indicate a basic unit of image processing or a specific location of a picture, and refers to an image area including at least one of a luma component and a chroma component.
  • a 'block' refers to an image area including a specific component among luminance components and chrominance components (ie, Cb and Cr).
  • terms such as 'unit', 'block', 'partition', 'signal' and 'region' may be used interchangeably depending on embodiments.
  • a 'current block' means a block currently scheduled to be encoded
  • a 'reference block' means a block that has already been coded or decoded and is used as a reference in the current block.
  • terms such as 'luma', 'luma', 'luminance', and 'Y' may be used interchangeably.
  • terms such as 'chroma', 'chroma', 'color difference', and 'Cb or Cr' may be used interchangeably.
  • a unit may be used as a concept including all of a coding unit, a prediction unit, and a transform unit.
  • a picture refers to a field or a frame, and the terms may be used interchangeably depending on embodiments. Specifically, when a photographed image is an interlace image, one frame is divided into an odd (or odd, top) field and an even (or even, bottom) field, and each field is composed of one picture unit. and can be encoded or decoded. If the photographed image is a progressive image, one frame may be configured as a picture and encoded or decoded. Also, in this specification, terms such as 'error signal', 'residual signal', 'residual signal', 'residual signal', and 'difference signal' may be used interchangeably.
  • POC Picture Order Count
  • the encoding apparatus 100 of the present invention includes a transform unit 110, a quantization unit 115, an inverse quantization unit 120, an inverse transform unit 125, a filtering unit 130, and a prediction unit 150. ) and an entropy coding unit 160.
  • the transform unit 110 transforms the residual signal, which is the difference between the received video signal and the prediction signal generated by the predictor 150, to obtain a transform coefficient value.
  • a discrete cosine transform DCT
  • DST discrete sine transform
  • Discrete cosine transform and discrete sine transform perform conversion by dividing an input picture signal into blocks.
  • coding efficiency may vary according to the distribution and characteristics of values within a transformation domain.
  • a transform kernel used for transforming a residual block may be a transform kernel having separable characteristics of vertical transform and horizontal transform. In this case, transformation of the residual block may be performed by dividing the vertical transformation and the horizontal transformation.
  • the encoder may perform vertical transform by applying a transform kernel in the vertical direction of the residual block.
  • the encoder may perform horizontal transformation by applying a transformation kernel in the horizontal direction of the residual block.
  • a transform kernel may be used as a term referring to a set of parameters used for transforming a residual signal, such as a transform matrix, a transform array, a transform function, and a transform.
  • the conversion kernel may be any one of a plurality of available kernels.
  • transform kernels based on different transform types may be used for each of the vertical transform and the horizontal transform.
  • an error signal may exist only in a partial region in a coding block.
  • the conversion process may be performed only on an arbitrary partial area.
  • an error signal may exist only in the first 2NxN block in a block having a size of 2Nx2N, and a conversion process is performed only in the first 2NxN block, but the conversion process is not performed on the second 2NxN block and may not be encoded or decoded.
  • N can be any positive integer.
  • the encoder may perform additional transforms before the transform coefficients are quantized.
  • the transform method described above is referred to as a primary transform, and an additional transform may be referred to as a secondary transform.
  • Secondary transformation may be selective for each residual block.
  • the encoder may improve coding efficiency by performing secondary transform on a region in which it is difficult to concentrate energy in a low frequency region with only the primary transform.
  • secondary transformation may be additionally performed on a block having large residual values in a direction other than the horizontal or vertical direction of the residual block. Unlike the first conversion, the secondary conversion may not be performed separately into vertical conversion and horizontal conversion.
  • This secondary transform may be referred to as a Low Frequency Non-Separable Transform (LFNST).
  • LFNST Low Frequency Non-Separable Transform
  • the quantization unit 115 quantizes the transform coefficient value output from the transform unit 110 .
  • a picture signal is not coded as it is, but a picture is predicted using an area already coded through the prediction unit 150, and a residual value between the original picture and the predicted picture is added to the predicted picture to obtain a reconstructed picture.
  • a method for obtaining is used.
  • the decoder when the encoder performs prediction, the decoder must also use available information. To this end, the encoder performs a process of restoring the encoded current block again.
  • the inverse quantization unit 120 inversely quantizes the transform coefficient value, and the inverse transform unit 125 restores the residual value using the inverse quantized transform coefficient value.
  • the filtering unit 130 performs a filtering operation to improve quality and coding efficiency of a reconstructed picture.
  • a deblocking filter For example, a deblocking filter, a Sample Adaptive Offset (SAO), and an adaptive loop filter may be included.
  • a picture that has undergone filtering is stored in a decoded picture buffer (DPB, 156) to be output or used as a reference picture.
  • DPB decoded picture buffer
  • a deblocking filter is a filter for removing distortion within a block generated at a boundary between blocks in a reconstructed picture.
  • the encoder may determine whether to apply a deblocking filter to a corresponding edge through a distribution of pixels included in several columns or rows based on an arbitrary edge in a block.
  • the encoder may apply a long filter, a strong filter, or a weak filter according to the strength of the deblocking filtering.
  • horizontal direction filtering and vertical direction filtering can be processed in parallel.
  • the sample adaptive offset (SAO) may be used to correct an offset from an original image in units of pixels for a residual block to which a deblocking filter is applied.
  • the encoder In order to correct the offset for a specific picture, the encoder divides the pixels included in the image into a certain number of areas, determines the area to perform offset correction, and uses a method (Band Offset) to apply the offset to the area. can Alternatively, the encoder may use a method (Edge Offset) of applying an offset in consideration of edge information of each pixel.
  • An adaptive loop filter is a method of dividing pixels included in an image into predetermined groups, determining one filter to be applied to the group, and performing filtering differentially for each group. Information related to whether to apply ALF may be signaled in units of coding units, and the shape and filter coefficients of an ALF filter to be applied may vary according to each block. In addition, the ALF filter of the same form (fixed form) may be applied regardless of the characteristics of the target block to be applied.
  • the prediction unit 150 includes an intra prediction unit 152 and an inter prediction unit 154.
  • the intra prediction unit 152 performs intra prediction within the current picture, and the inter prediction unit 154 predicts the current picture using the reference picture stored in the decoded picture buffer 156. Do it.
  • the intra prediction unit 152 performs intra prediction on reconstructed regions in the current picture and transfers intra-encoding information to the entropy coding unit 160 .
  • the intra encoding information may include at least one of an intra prediction mode, a most probable mode (MPM) flag, an MPM index, and information about a reference sample.
  • the inter prediction unit 154 may again include a motion estimation unit 154a and a motion compensation unit 154b.
  • the motion estimation unit 154a refers to a specific region of the reconstructed reference picture to find a part most similar to the current region and obtains a motion vector value that is a distance between the regions.
  • Motion information reference direction indication information (L0 prediction, L1 prediction, bi-directional prediction), reference picture index, motion vector information, etc.) for the reference region acquired by the motion estimation unit 154a is transferred to the entropy coding unit 160. so that it can be included in the bitstream.
  • the motion compensation unit 154b performs inter-motion compensation using the motion information transmitted from the motion estimation unit 154a to generate a prediction block for the current block.
  • the inter prediction unit 154 transfers inter encoding information including motion information on the reference region to the entropy coding unit 160 .
  • the predictor 150 may include an intra block copy (IBC) predictor (not shown).
  • the IBC prediction unit performs IBC prediction from reconstructed samples in the current picture and transfers IBC encoding information to the entropy coding unit 160 .
  • the IBC prediction unit refers to a specific region in the current picture and obtains a block vector value indicating a reference region used for prediction of the current region.
  • the IBC prediction unit may perform IBC prediction using the obtained block vector value.
  • the IBC prediction unit transfers the IBC encoding information to the entropy coding unit 160 .
  • the IBC encoding information may include at least one of size information of a reference region and block vector information (index information for predicting a block vector of a current block in a motion candidate list and block vector difference information).
  • the transform unit 110 obtains a transform coefficient value by transforming a residual value between an original picture and a predicted picture.
  • transformation may be performed in units of a specific block within a picture, and the size of a specific block may vary within a preset range.
  • the quantization unit 115 quantizes the transform coefficient values generated by the transform unit 110 and transfers the quantized transform coefficients to the entropy coding unit 160 .
  • the quantized transform coefficients in the form of a two-dimensional array may be rearranged into a form of a one-dimensional array for entropy coding.
  • a scanning method for quantized transform coefficients may be determined according to a size of a transform block and an intra-prediction mode. As an embodiment, diagonal, vertical, and horizontal scans may be applied. Such scan information may be signaled in units of blocks and may be derived according to pre-determined rules.
  • the entropy coding unit 160 generates a video signal bitstream by entropy coding information representing quantized transform coefficients, intra-encoding information, and inter-encoding information.
  • a variable length coding (VLC) method and an arithmetic coding method may be used.
  • VLC variable length coding
  • a variable length coding (VLC) method converts input symbols into continuous codewords, the length of which can be variable. For example, frequently occurring symbols are represented by short codewords, and infrequently occurring symbols are represented by long codewords.
  • a context-based adaptive variable length coding (CAVLC) scheme may be used as a variable length coding scheme.
  • Arithmetic coding converts successive data symbols into a single prime number using a probability distribution of each data symbol. Arithmetic coding can obtain an optimal number of decimal bits required to represent each symbol.
  • As arithmetic coding context-based adaptive binary arithmetic code (CABAC) may be used.
  • CABAC context-based adaptive binary arithmetic code
  • CABAC is a method of encoding binary arithmetic through several context models generated based on probabilities obtained through experiments.
  • a context model can also be referred to as a context model.
  • the encoder binarizes each symbol using exp-Golomb or the like.
  • a binarized 0 or 1 can be described as a bin.
  • the CABAC initialization process is divided into context initialization and arithmetic coding initialization.
  • Context initialization is a process of initializing the occurrence probability of each symbol, and is determined according to the symbol type, quantization parameter (QP), and slice type (whether I, P, or B).
  • QP quantization parameter
  • slice type whether I, P, or B
  • the context model provides information (valMPS) about the probability of occurrence of a least probable symbol (LPS) or most probable symbol (MPS) for a symbol to be currently coded and which bin value among 0 and 1 corresponds to the MPS.
  • valMPS information about the probability of occurrence of a least probable symbol (LPS) or most probable symbol (MPS) for a symbol to be currently coded and which bin value among 0 and 1 corresponds to the MPS.
  • LPS least probable symbol
  • MPS most probable symbol
  • One of several context models is selected through a context index (ctxIdx), and the context index can be derived through information of a block to be currently encoded or information of neighboring blocks.
  • Initialization for binary arithmetic coding is performed based on the probability model selected in the context model.
  • Binary arithmetic encoding is performed by dividing into probability intervals through the occurrence probabilities of 0 and 1, and then the probability interval corresponding to the bin to be processed becomes the entire probability interval for
  • Position information within the probability interval where the last bin was processed is output.
  • a renormalization process is performed to widen the probability interval and corresponding location information is output.
  • a probability update process may be performed in which a probability of a next bin to be processed is newly set based on information of the processed bin.
  • the generated bitstream is encapsulated in a network abstraction layer (NAL) unit as a basic unit.
  • the NAL unit is divided into a VCL (Video Coding Layer) NAL unit including video data and a non-VCL NAL unit including parameter information for decoding video data.
  • VCL Video Coding Layer
  • non-VCL NAL unit including parameter information for decoding video data.
  • the NAL unit is composed of NAL header information and data, RBSP (Raw Byte Sequence Payload), and the NAL header information includes summary information about the RBSP.
  • the RBSP of the VCL NAL unit includes a coded integer number of coding tree units.
  • the bitstream In order to decode a bitstream in a video decoder, the bitstream must first be divided into NAL unit units and then each separated NAL unit must be decoded. Meanwhile, information necessary for decoding a video signal bitstream is included in a Picture Parameter Set (PPS), a Sequence Parameter Set (SPS), a Video Parameter Set (VPS), etc. and transmitted.
  • PPS Picture Parameter Set
  • SPS Sequence Parameter Set
  • VPS Video Parameter Set
  • FIG. 1 shows the encoding apparatus 100 according to an embodiment of the present invention, and the separately displayed blocks logically distinguish elements of the encoding apparatus 100. Accordingly, the elements of the encoding apparatus 100 described above may be mounted as one chip or as a plurality of chips according to the design of the device. According to one embodiment, the operation of each element of the above-described encoding device 100 may be performed by a processor (not shown).
  • the decoding apparatus 200 of the present invention includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 225, a filtering unit 230, and a prediction unit 250.
  • the entropy decoding unit 210 entropy-decodes the video signal bitstream and extracts transform coefficient information, intra-encoding information, and inter-encoding information for each region. For example, the entropy decoding unit 210 may obtain a binarization code for transform coefficient information of a specific region from a video signal bitstream. Also, the entropy decoding unit 210 inversely binarizes the binary code to obtain quantized transform coefficients. The inverse quantization unit 220 inversely quantizes the quantized transform coefficient, and the inverse transform unit 225 restores a residual value using the inverse quantized transform coefficient. The video signal processing apparatus 200 restores an original pixel value by adding the residual value obtained from the inverse transform unit 225 to the prediction value obtained from the predictor 250.
  • the filtering unit 230 improves picture quality by performing filtering on pictures. This may include a deblocking filter to reduce block distortion and/or an adaptive loop filter to remove distortion of the entire picture.
  • the filtered picture is output or stored in the decoded picture buffer (DPB) 256 to be used as a reference picture for the next picture.
  • DPB decoded picture buffer
  • the prediction unit 250 includes an intra prediction unit 252 and an inter prediction unit 254 .
  • the prediction unit 250 generates a predicted picture by utilizing the coding type decoded through the above-described entropy decoding unit 210, transform coefficients for each region, intra/inter coding information, and the like.
  • a current picture including the current block or a decoded area of other pictures may be used.
  • a picture (or tile/slice) that can be performed is called an inter picture (or tile/slice).
  • a picture (or tile/slice) using up to one motion vector and reference picture index to predict sample values of each block among inter-pictures (or tiles/slices) is called a predictive picture or a P picture (or , tile/slice), and a picture (or tile/slice) using up to two motion vectors and a reference picture index is called a bi-predictive picture or B picture (or tile/slice).
  • a P picture (or tile/slice) uses at most one set of motion information to predict each block
  • a B picture (or tile/slice) uses at most two sets of motion information to predict each block.
  • the motion information set includes one or more motion vectors and one reference picture index.
  • the intra prediction unit 252 generates a prediction block using intra encoding information and reconstructed samples in a current picture.
  • the intra encoding information may include at least one of an intra prediction mode, a most probable mode (MPM) flag, and an MPM index.
  • the intra predictor 252 predicts sample values of the current block by using reconstructed samples located on the left side and/or above the current block as reference samples.
  • reconstructed samples, reference samples, and samples of a current block may represent pixels. Also, sample values may represent pixel values.
  • reference samples may be samples included in neighboring blocks of the current block.
  • the reference samples may be samples adjacent to the left boundary of the current block and/or samples adjacent to the upper boundary of the current block.
  • the reference samples are samples located on a line within a preset distance from the left boundary of the current block among samples of neighboring blocks of the current block and/or located on a line within a preset distance from the upper boundary of the current block.
  • the neighboring blocks of the current block may be a left (L) block, an upper (A) block, a below left (BL) block, an above right (AR) block, or an above left (Above Left) block adjacent to the current block.
  • AL may include at least one of the blocks.
  • the inter prediction unit 254 generates a prediction block using a reference picture stored in the decoded picture buffer 256 and inter encoding information.
  • the inter-encoding information may include a motion information set (reference picture index, motion vector information, etc.) of a current block with respect to a reference block.
  • Inter prediction may include L0 prediction, L1 prediction, and bi-prediction.
  • L0 prediction refers to prediction using one reference picture included in the L0 picture list
  • L1 prediction refers to prediction using one reference picture included in the L1 picture list.
  • one set of motion information eg, a motion vector and a reference picture index
  • up to two reference regions can be used, and these two reference regions may exist in the same reference picture or in different pictures.
  • the bi-prediction method up to two sets of motion information (eg, a motion vector and a reference picture index) can be used, and the two motion vectors may correspond to the same reference picture index or to different reference picture indices. may correspond.
  • the reference pictures are pictures positioned before or after the current picture in terms of time, and may be pictures that have already been reconstructed.
  • two reference regions used in the bi-prediction method may be regions selected from each of the L0 picture list and the L1 picture list.
  • the inter prediction unit 254 may obtain a reference block of the current block by using the motion vector and the reference picture index.
  • the reference block exists in a reference picture corresponding to a reference picture index.
  • a sample value of a block specified by a motion vector or an interpolated value thereof may be used as a predictor of a current block.
  • an 8-tap interpolation filter for a luminance signal and a 4-tap interpolation filter for a chrominance signal may be used.
  • an interpolation filter for motion prediction in units of subpels is not limited thereto.
  • the inter prediction unit 254 performs motion compensation for predicting the texture of the current unit from the previously reconstructed picture.
  • the inter prediction unit may use the motion information set.
  • the prediction unit 250 may include an IBC prediction unit (not shown).
  • the IBC prediction unit may reconstruct the current region by referring to a specific region including reconstructed samples in the current picture.
  • the IBC prediction unit may perform IBC prediction using the IBC encoding information obtained from the entropy decoding unit 210 .
  • IBC encoding information may include block vector information.
  • a reconstructed video picture is generated by adding the prediction value output from the intra prediction unit 252 or the inter prediction unit 254 and the residual value output from the inverse transform unit 225. That is, the video signal decoding apparatus 200 reconstructs the current block by using the prediction block generated by the prediction unit 250 and the residual obtained from the inverse transform unit 225.
  • FIG. 2 shows the decoding apparatus 200 according to an embodiment of the present invention, and the separately displayed blocks logically distinguish elements of the decoding apparatus 200. Accordingly, elements of the decoding apparatus 200 described above may be mounted as one chip or as a plurality of chips according to the design of the device. According to one embodiment, the operation of each element of the decoding apparatus 200 described above may be performed by a processor (not shown).
  • the technology proposed in this specification is a technology applicable to both encoder and decoder methods and devices, and parts described as signaling and parsing may be described for convenience of explanation.
  • signaling is for encoding each syntax from an encoder point of view
  • parsing is for interpreting each syntax from a decoder point of view. That is, each syntax may be included in a bitstream from the encoder and signaled, and the decoder may parse the syntax and use it in the restoration process.
  • a sequence of bits for each syntax arranged according to a defined hierarchical configuration may be referred to as a bitstream.
  • One picture may be coded after being divided into sub-pictures, slices, tiles, and the like.
  • a subpicture may contain one or more slices or tiles. When one picture is divided into several slices or tiles and encoded, all slices or tiles in the picture must be decoded before being displayed on the screen. On the other hand, when one picture is coded with several subpictures, only a certain subpicture can be decoded and displayed on the screen.
  • a slice may contain multiple tiles or subpictures. Alternatively, a tile may include multiple subpictures or slices. Since subpictures, slices, and tiles can be encoded or decoded independently of each other, it is effective in improving parallel processing and processing speed. However, since coded information of other adjacent subpictures, other slices, and other tiles cannot be used, the amount of bits increases.
  • Subpictures, slices, and tiles may be coded after being divided into several Coding Tree Units (CTUs).
  • CTUs Coding Tree Units
  • a coding tree unit may include a luma coding tree block (CTB), two chroma coding tree blocks, and encoded syntax information thereof.
  • CB luma coding tree block
  • One coding tree unit may be composed of one coding unit, or one coding tree unit may be divided into several coding units.
  • One coding unit may include a luminance coding block (CB), two color difference coding blocks, and their encoded syntax information.
  • One coding block may be divided into several sub coding blocks.
  • One coding unit may be composed of one transform unit (TU), or one coding unit may be divided into several transform units.
  • One transform unit may include a luminance transform block (TB), two color difference transform blocks, and encoded syntax information thereof.
  • a coding tree unit may be divided into a plurality of coding units.
  • a coding tree unit may be a leaf node without being split. In this case, the coding tree unit itself may be a coding unit.
  • a coding unit refers to a basic unit for processing a picture in the process of processing a video signal described above, that is, intra/inter prediction, transformation, quantization, and/or entropy coding.
  • the size and shape of a coding unit within one picture may not be constant.
  • a coding unit may have a square or rectangular shape.
  • a rectangular coding unit (or rectangular block) includes a vertical coding unit (or vertical block) and a horizontal coding unit (or horizontal block).
  • a vertical block is a block whose height is greater than its width
  • a horizontal block is a block whose width is greater than its height.
  • a non-square block may refer to a rectangular block, but the present invention is not limited thereto.
  • the coding tree unit is first divided into a quad tree (QT) structure. That is, in the quad tree structure, one node having a size of 2NX2N may be divided into four nodes having a size of NXN.
  • a quad tree may also be referred to as a quaternary tree. Quad tree splitting can be done recursively, and not all nodes need to be split to the same depth.
  • the leaf node of the aforementioned quad tree may be further divided into a multi-type tree (MTT) structure.
  • MTT multi-type tree
  • one node in a multi-type tree structure, one node may be split into a binary (binary) or ternary (ternary) tree structure of horizontal or vertical split. That is, there are four partition structures of vertical binary partitioning, horizontal binary partitioning, vertical ternary partitioning, and horizontal ternary partitioning in the multi-type tree structure.
  • both the width and height of a node in each tree structure may have a power of 2 value.
  • a node having a size of 2NX2N is divided into two NX2N nodes by vertical binary partitioning and divided into two 2NXN nodes by horizontal binary partitioning.
  • a node of size 2NX2N is divided into nodes of (N/2)X2N, NX2N and (N/2)X2N by vertical ternary division, and horizontal ternary division It can be divided into 2NX(N/2), 2NXN and 2NX(N/2) nodes by partitioning.
  • This multi-type tree partitioning can be performed recursively.
  • a leaf node of a multi-type tree can be a coding unit. If the coding unit is not large compared to the maximum transform length, the coding unit may be used as a unit of prediction and/or transformation without further division. As an embodiment, when the width or height of the current coding unit is greater than the maximum transform length, the current coding unit may be divided into a plurality of transform units without explicit signaling regarding division. Meanwhile, in the aforementioned quad tree and multi-type tree, at least one of the following parameters may be defined in advance or may be transmitted through a higher level set of RBSPs such as PPS, SPS, and VPS.
  • RBSPs such as PPS, SPS, and VPS.
  • Preset flags may be used to signal splitting of the aforementioned quad tree and multi-type tree.
  • a flag 'split_cu_flag' indicating whether a node is split
  • a flag 'split_qt_flag' indicating whether a quad tree node is split
  • a flag 'mtt_split_cu_vertical_flag' indicating a split direction of a multi-type tree node
  • At least one of flags 'mtt_split_cu_binary_flag' indicating a split shape of a type tree node may be used.
  • 'split_cu_flag' which is a flag indicating whether to split a current node, may be signaled first. If the value of 'split_cu_flag' is 0, it indicates that the current node is not split, and the current node becomes a coding unit.
  • the coding tree unit includes one undivided coding unit.
  • the current node is a quad tree node 'QT node'
  • the current node is a leaf node 'QT leaf node' of the quad tree and becomes a coding unit.
  • the current node is a multi-type tree node 'MTT node'
  • the current node is a leaf node 'MTT leaf node' of the multi-type tree and becomes a coding unit.
  • the current node may be split into quad tree or multi-type tree nodes according to the value of 'split_qt_flag'.
  • a coding tree unit is a root node of a quad tree, and can be first partitioned into a quad tree structure. In the quad tree structure, 'split_qt_flag' is signaled for each node 'QT node'.
  • quad tree partitioning may be limited according to the type of current node. Quad tree splitting may be allowed if the current node is a coding tree unit (root node of a quad tree) or a quad tree node, and quad tree splitting may not be allowed if the current node is a multi-type tree node.
  • Each quad tree leaf node 'QT leaf node' can be further partitioned into a multi-type tree structure. As described above, when 'split_qt_flag' is 0, the current node may be split into multi-type nodes. In order to indicate the split direction and split shape, 'mtt_split_cu_vertical_flag' and 'mtt_split_cu_binary_flag' may be signaled.
  • a luminance block and a chrominance block may be equally divided. That is, the chrominance block may be divided by referring to the division form of the luminance block. If the size of the current chrominance block is smaller than a predetermined size, the chrominance block may not be divided even if the luminance block is divided.
  • the luminance block and the chrominance block may have different shapes.
  • partition information for the luminance block and partition information for the chrominance block may be signaled respectively.
  • encoding information of the luminance block and the chrominance block as well as partition information may be different.
  • at least one intra encoding mode of the luminance block and the chrominance block, encoding information about motion information, and the like may be different.
  • Nodes to be divided into the smallest units can be processed as one coding block.
  • the coding block may be divided into several sub-blocks (sub-coding blocks), and the prediction information of each sub-block may be the same or different.
  • the intra prediction modes of each sub-block may be the same or different.
  • motion information of each sub-block may be identical to or different from each other.
  • each sub-block may be independently encoded or decoded.
  • Each sub-block may be identified through a sub-block index (sbIdx).
  • a coding unit when a coding unit is divided into sub-blocks, it may be divided in a horizontal or vertical direction or diagonally.
  • ISP Intra Sub Partitions
  • a mode in which the current coding block is divided into oblique lines in the inter mode is called a geometric partitioning mode (GPM).
  • GPM geometric partitioning mode
  • the position and direction of the oblique line are derived using a predetermined angle table, and index information of the angle table is signaled.
  • Picture prediction (motion compensation) for coding is performed for a coding unit (that is, a leaf node of a coding tree unit) that is not further divided.
  • a basic unit that performs such prediction is hereinafter referred to as a prediction unit or a prediction block.
  • the term unit used in this specification may be used as a substitute for the prediction unit, which is a basic unit for performing prediction.
  • the present invention is not limited thereto, and may be understood as a concept including the coding unit in a more broad sense.
  • the intra prediction unit predicts sample values of the current block by using reconstructed samples located on the left side and/or above the current block as reference samples.
  • FIG. 5 shows an example of reference samples used for prediction of a current block in intra prediction mode.
  • the reference samples may be samples adjacent to a left boundary and/or an upper boundary of the current block.
  • the size of the current block is WXH and samples of a single reference line adjacent to the current block are used for intra prediction, up to 2W+2H+1 located on the left and/or upper side of the current block Reference samples may be set using the number of neighboring samples.
  • pixels of multiple reference lines may be used for intra prediction of the current block.
  • Multiple reference lines may be composed of n lines located within a predetermined range from the current block.
  • separate index information indicating lines to be set as reference pixels may be signaled, and this may be referred to as a reference line index.
  • the intra prediction unit may obtain reference samples by performing a reference sample padding process. Also, the intra prediction unit may perform a reference sample filtering process to reduce intra prediction errors. That is, filtered reference samples may be obtained by filtering the neighboring samples and/or the reference samples obtained through the reference sample padding process. The intra predictor predicts samples of the current block using the reference samples obtained in this way. The intra predictor predicts samples of the current block using unfiltered reference samples or filtered reference samples.
  • neighboring samples may include samples on at least one reference line.
  • the neighboring samples may include neighboring samples on a line adjacent to the boundary of the current block.
  • FIG. 6 shows an embodiment of prediction modes used for intra prediction.
  • intra prediction mode information indicating an intra prediction direction may be signaled.
  • the intra prediction mode information indicates one of a plurality of intra prediction modes constituting an intra prediction mode set. If the current block is an intra prediction block, the decoder receives intra prediction mode information of the current block from the bitstream. The intra prediction unit of the decoder performs intra prediction on the current block based on the extracted intra prediction mode information.
  • the intra prediction mode set may include all intra prediction modes used for intra prediction (eg, a total of 67 intra prediction modes). More specifically, the intra prediction mode set may include a planar mode, a DC mode, and multiple (eg, 65) angular modes (ie, directional modes). Each intra prediction mode may be indicated through a preset index (ie, an intra prediction mode index). For example, as shown in FIG. 6 , an intra prediction mode index 0 indicates a planar mode, and an intra prediction mode index 1 indicates a DC mode.
  • intra prediction mode indices 2 to 66 may indicate different angular modes, respectively. The angle modes each indicate different angles within a preset angle range.
  • the angle mode may indicate an angle within an angle range between 45 degrees and -135 degrees in a clockwise direction (ie, the first angle range).
  • the angle mode may be defined based on the 12 o'clock direction.
  • the intra prediction mode index 2 indicates a horizontal diagonal (HDIA) mode
  • the intra prediction mode index 18 indicates a horizontal (HOR) mode
  • the intra prediction mode index 34 indicates a diagonal (DIA) mode.
  • an intra prediction mode index of 50 indicates a vertical (VER) mode
  • an intra prediction mode index of 66 indicates a vertical diagonal (VDIA) mode.
  • the preset angle range may be set differently according to the shape of the current block. For example, when the current block is a rectangular block, a wide-angle mode indicating an angle exceeding 45 degrees or less than -135 degrees in a clockwise direction may be additionally used. If the current block is a horizontal block, the angle mode may indicate an angle within an angular range (ie, a second angle range) between (45+offset1) degrees and (-135+offset1) degrees clockwise. At this time, angle modes 67 to 76 outside the first angle range may be additionally used.
  • the angle mode may indicate an angle within an angular range (ie, a third angle range) between (45-offset2) and (-135-offset2) degrees clockwise.
  • angle modes -10 to -1 outside the first angle range may be additionally used.
  • the values of offset1 and offset2 may be determined differently according to the ratio between the width and height of the rectangular block. Also, offset1 and offset2 may be positive numbers.
  • the plurality of angular modes constituting the intra prediction mode set may include a basic angular mode and an extended angular mode.
  • the extended angle mode may be determined based on the basic angle mode.
  • the basic angle mode is a mode corresponding to an angle used in intra prediction of an existing High Efficiency Video Coding (HEVC) standard
  • the extended angle mode corresponds to an angle newly added in intra prediction of a next-generation video codec standard. It may be a mode that More specifically, the default angular mode is the intra prediction mode ⁇ 2, 4, 6, ... , 66 ⁇ , and the extended angle mode is an intra prediction mode ⁇ 3, 5, 7, . . . , 65 ⁇ . That is, the extended angular mode may be an angular mode between basic angular modes within the first angular range. Accordingly, an angle indicated by the extended angle mode may be determined based on an angle indicated by the basic angle mode.
  • HEVC High Efficiency Video Coding
  • the basic angle mode may be a mode corresponding to an angle within a preset first angle range
  • the extended angle mode may be a wide angle mode outside the first angle range. That is, the default angle mode is the intra prediction mode ⁇ 2, 3, 4, ... , 66 ⁇ , and the extended angle mode is an intra prediction mode ⁇ -14, -13, -12, . . . , -1 ⁇ and ⁇ 67, 68, ... , 80 ⁇ .
  • An angle indicated by the extended angle mode may be determined as an angle opposite to an angle indicated by the corresponding basic angle mode. Accordingly, an angle indicated by the extended angle mode may be determined based on an angle indicated by the basic angle mode.
  • the number of expansion angle modes is not limited thereto, and additional expansion angles may be defined according to the size and/or shape of the current block.
  • the total number of intra prediction modes included in the intra prediction mode set may vary according to the configuration of the basic angular mode and the extended angular mode.
  • the interval between the extended angle modes may be set based on the interval between the corresponding basic angle modes.
  • extended angle modes ⁇ 3, 5, 7, ... , 65 ⁇ corresponds to the corresponding basic angle modes ⁇ 2, 4, 6, ... , 66 ⁇ .
  • the extended angle modes ⁇ -14, -13, . . . , -1 ⁇ the corresponding opposite fundamental angle modes ⁇ 53, 53, ... , 66 ⁇ , and the expansion angle modes ⁇ 67, 68, . . . , 80 ⁇ corresponds to the opposite fundamental angle modes ⁇ 2, 3, 4, ... , 15 ⁇ .
  • An angular interval between extended angular modes may be set to be the same as an angular interval between corresponding basic angular modes.
  • the number of extended angular modes in the intra prediction mode set may be set to be less than or equal to the number of basic angular modes.
  • the extended angle mode may be signaled based on the basic angle mode.
  • the wide-angle mode ie, the extended angle mode
  • the wide-angle mode may replace at least one angle mode (ie, the basic angle mode) within the first angle range.
  • the default angular mode that is replaced may be an angular mode that corresponds to the opposite side of the wide-angle mode. That is, the replaced basic angle mode is an angle mode corresponding to an angle in a direction opposite to the angle indicated by the wide angle mode or an angle different from the angle in the opposite direction by a predetermined offset index.
  • the preset offset index is 1.
  • the intra prediction mode index corresponding to the replaced basic angle mode may be mapped back to the wide-angle mode to signal the corresponding wide-angle mode.
  • wide-angle mode ⁇ -14, -13, ... , -1 ⁇ is the intra prediction mode index ⁇ 52, 53, ... , 66 ⁇
  • the wide-angle mode ⁇ 67, 68, . . . , 80 ⁇ is the intra prediction mode index ⁇ 2, 3, ... , 15 ⁇ , respectively.
  • the intra prediction mode index for the basic angular mode signals the extended angular mode, so even if the configurations of the angular modes used for intra prediction of each block are different, the same set of intra prediction mode indexes are used for intra prediction mode signaling. can be used Accordingly, signaling overhead according to a change in intra prediction mode configuration can be minimized.
  • whether to use the extended angle mode may be determined based on at least one of the shape and size of the current block. According to an embodiment, if the size of the current block is larger than a preset size, the extended angle mode is used for intra prediction of the current block, otherwise only the basic angle mode is used for intra prediction of the current block. According to another embodiment, when the current block is a non-square block, the extended angle mode is used for intra prediction of the current block, and when the current block is a square block, only the basic angle mode is used for intra prediction of the current block.
  • the intra prediction unit determines reference samples to be used for intra prediction of the current block and/or interpolated reference samples based on intra prediction mode information of the current block.
  • the intra prediction mode index indicates a specific angle mode
  • a reference sample corresponding to the specific angle from the current sample of the current block or an interpolated reference sample is used to predict the current pixel. Accordingly, different sets of reference samples and/or interpolated reference samples may be used for intra prediction according to the intra prediction mode.
  • the decoder restores sample values of the current block by adding the residual signal of the current block obtained from the inverse transform unit to the intra prediction value of the current block. .
  • Motion (motion) information used for inter prediction may include reference direction indication information (inter_pred_idc), reference picture indices (ref_idx_l0, ref_idx_l1), and motion (motion) vectors (mvL0, mvL1).
  • Reference picture list utilization information predFlagL0, predFlagL1 may be set according to the reference direction indication information.
  • the coding unit may be divided into several sub-blocks, and prediction information of each sub-block may be the same or different.
  • the intra prediction modes of each sub-block may be the same or different.
  • motion information of each sub-block may be identical to or different from each other.
  • each sub-block may be independently encoded or decoded.
  • Each sub-block may be identified through a sub-block index (sbIdx).
  • the motion vector of the current block is highly likely to be similar to the motion vectors of neighboring blocks. Accordingly, motion vectors of neighboring blocks may be used as motion vector predictors (mvp), and motion vectors of the current block may be derived using motion vectors of neighboring blocks.
  • mvp motion vector predictors
  • a motion vector difference (mvd) between an optimum motion vector of the current block found as an original image by the encoder and a predicted value of motion information may be signaled.
  • the motion vector may have various resolutions, and the resolution of the motion vector may vary on a block-by-block basis.
  • the motion vector resolution may be expressed in integer units, half-pixel units, 1/4 pixel units, 1/16 pixel units, 4 integer pixel units, and the like. Since an image such as screen content is in the form of a simple graphic such as text, an interpolation filter does not need to be applied, and thus an integer unit and an integer pixel unit of 4 may be selectively applied in block units.
  • Blocks encoded in affine mode capable of expressing rotation and scale vary greatly in shape, so integer units, 1/4 pixel units, and 1/16 pixel units can be selectively applied on a block basis.
  • Information on whether to selectively apply motion vector resolution in block units is signaled as amvr_flag. If applied, which motion vector resolution to apply to the current block is signaled by amvr_precision_idx.
  • weights between two prediction blocks may be the same or different when weight average is applied, and information about weights is signaled through bcw_idx.
  • a merge or advanced motion vector prediction (AMVP) method may be selectively used in units of blocks.
  • the merge method is a method of configuring the motion information of the current block to be the same as the motion information of neighboring blocks adjacent to the current block, and has the advantage of increasing the encoding efficiency of motion information by propagating motion information spatially without change in a homogeneous motion domain.
  • the AMVP method is a method of predicting motion information in L0 and L1 prediction directions, respectively, and signaling the most optimal motion information in order to express accurate motion information.
  • the decoder uses a reference block located in motion information derived from a reference picture as a prediction block for the current block.
  • a method of deriving motion information in Merge or AMVP may be a method in which a motion candidate list is constructed using prediction values of motion information derived from neighboring blocks of the current block, and then index information on an optimal motion candidate is signaled.
  • AMVP since motion candidate lists for L0 and L1 are derived, optimal motion candidate indices (mvp_l0_flag and mvp_l1_flag) for L0 and L1 are signaled.
  • merge since one motion candidate list is derived, one merge index (merge_idx) is signaled.
  • Motion candidate lists derived from one coding unit may vary, and a motion candidate index or merge index may be signaled for each motion candidate list. In this case, a mode in which there is no information about a residual block in a block encoded in the Merge mode may be referred to as a MergeSkip mode.
  • Bidirectional motion information for the current block may be derived by mixing AMVP and Merge modes.
  • motion information in the L0 direction may be derived using the AMVP method
  • motion information in the L1 direction may be derived using the Merge method.
  • Merge can be applied to L0
  • AMVP can be applied to L1.
  • Such an encoding mode may be referred to as an AMVP-merge mode.
  • Symmetric MVD is a method of reducing the amount of bits of transmitted motion information by making Motion Vector Difference (MVD) values of L0 and L1 directions symmetrical in the case of bi-directional prediction.
  • MVD information in the L1 direction that is symmetrical with the L0 direction is not transmitted, and reference picture information in the L0 and L1 directions is not transmitted and is derived in the decoding process.
  • OBMC Overlapped Block Motion Compensation
  • merge motion candidates have low motion accuracy.
  • a Merge mode with MVD (MMVD) method may be used.
  • the MMVD method is a method of correcting motion information using one candidate selected from several motion difference value candidates.
  • Information on a compensation value of motion information obtained through the MMVD method (eg, an index indicating one selected from among motion differential value candidates) may be included in a bitstream and transmitted to a decoder.
  • the amount of bits can be saved by including the information on the compensation value of the motion information in the bitstream.
  • the TM (Template Matching) method is a method of compensating motion information by constructing a template using neighboring pixels of a current block and finding a matching area having the highest similarity with the template.
  • Template matching is a method of performing motion prediction in a decoder without including motion information in a bitstream in order to reduce the size of an encoded bitstream. In this case, the decoder may roughly derive motion information for the current block using the already reconstructed neighboring blocks since there is no original image.
  • the DMVR (Decoder-side Motion Vector Refinement) method is a method of correcting motion information through correlation of previously reconstructed reference images to find more accurate motion information. This is a method of using, as a new bi-directional motion, a point where the reference blocks in a reference picture are best matched within a predetermined area.
  • the encoder corrects motion information by performing DMVR in one block unit, then divides the block into sub-blocks and performs DMVR in each sub-block unit to correct motion information of the sub-block again.
  • MP-DMVR Multi-pass DMVR
  • the Local Illumination Compensation (LIC) method is a method of compensating for a luminance change between blocks. After deriving a linear model using neighboring pixels adjacent to the current block, the luminance information of the current block is compensated for through the linear model.
  • BDOF Bi-Directional Optical Flow
  • the motion of the current block may be corrected using the motion information derived from the BDOF of the VVC.
  • PROF Prediction refinement with optical flow
  • PROF is a technique for improving the accuracy of affine motion prediction in sub-block units to be similar to that of pixel-unit motion prediction. Similar to BDOF, PROF is a technique for obtaining a final prediction signal by calculating correction values in units of pixels for pixel values affine motion compensated in units of sub-blocks based on optical-flow.
  • CIIP Combined Inter-/Intra-picture Prediction
  • An intra block copy (IBC) method is a method in which a part most similar to a current block is found in an already reconstructed region within a current picture, and a corresponding reference block is used as a prediction block for the current block.
  • information related to a block vector which is a distance between the current block and the reference block, may be included in the bitstream.
  • the decoder may calculate or set a block vector for the current block by parsing information related to the block vector included in the bitstream.
  • BCW Bi-prediction with CU-level Weights
  • a multi-hypothesis prediction (MHP) method is a method of performing weight prediction through various prediction signals by transmitting additional motion information to unidirectional and bidirectional motion information during inter-screen prediction.
  • Cross-component linear model is a method of constructing a linear model using a high correlation between a luminance signal and a chrominance signal located at the same position as the luminance signal, and then predicting the chrominance signal through the linear model.
  • parameters for the linear model are derived through the template.
  • the current luminance block reconstructed according to the size of the chrominance block selectively according to the image format is downsampled.
  • the chrominance block of the current block is predicted using the downsampled luminance block and the corresponding linear model.
  • MMLM multi-model linear mode
  • a reconstructed coefficient t' k for an input coefficient t k depends only on a related quantization index q k . That is, a quantization index for a certain reconstructed coefficient has a different value from quantization indices for other reconstructed coefficients.
  • t' k may be a value including a quantization error in t k , and may be different or the same according to quantization parameters.
  • t' k may be referred to as a reconstructed transform coefficient or an inverse quantized transform coefficient, and a quantization index may be referred to as a quantized transform coefficient.
  • reconstructed coefficients have a characteristic of being equally spaced.
  • the distance between two adjacent restoration values may be referred to as a quantization step size.
  • 0 may be included, and the entire set of usable reconstructed values may be uniquely defined according to the size of the quantization step.
  • the quantization step size may vary depending on the quantization parameter.
  • a simple vector quantization method used in video encoding includes sign data hiding. This is a method in which the encoder does not encode the sign of one non-zero coefficient, and the decoder determines the sign of the corresponding coefficient according to whether the sum of the absolute values of all coefficients is an even number or an odd number.
  • at least one coefficient may be increased or decreased by '1', which is selected so that at least one coefficient is optimal in terms of cost for rate-distortion, and the value is can be adjusted As an example, a coefficient having a value close to the boundary of the quantization interval may be selected.
  • Another vector quantization method includes trellis-coded quantization, and in video encoding, it is used as an optimal path search technique for obtaining an optimized quantization value in dependent quantization.
  • quantization candidates for all coefficients in the block are placed in the Trellis graph, and the optimal Trellis path between the optimized quantization candidates is considered at the cost of rate-distortion.
  • dependent quantization applied to video encoding may be designed such that a set of allowable reconstructed transform coefficients for a transform coefficient depends on the value of a transform coefficient that precedes the current transform coefficient in the reconstruction order. In this case, by selectively using a plurality of quantizers according to transform coefficients, an average error between an original image and a reconstructed image is minimized, thereby increasing coding efficiency.
  • the MIP (Matrix Intra Prediction) method is a matrix-based intra prediction method. Unlike prediction methods that have directionality from pixels of neighboring blocks adjacent to the current block, the MIP (Matrix Intra Prediction) method is a matrix-based matrix in which pixels on the left and top of neighboring blocks are predefined. This is a method of obtaining a prediction signal using the offset value and .
  • the decoder may generate a prediction template for a template using neighboring pixels (references) adjacent to the template, and may use an intra prediction mode in which a prediction template most similar to a previously reconstructed template is generated to reconstruct a current block. This method may be referred to as template intra mode derivation (TIMD).
  • TMD template intra mode derivation
  • an encoder may determine a prediction mode for generating a prediction block and generate a bitstream including information about the determined prediction mode.
  • the decoder may set the intra prediction mode by parsing the received bitstream.
  • the amount of bits of information about the prediction mode may be about 10% of the size of the entire bitstream.
  • the encoder may not include information about the intra prediction mode in the bitstream. Accordingly, the decoder may derive (determine) an intra prediction mode for reconstruction of the current block using characteristics of neighboring blocks, and may reconstruct the current block using the derived intra prediction mode.
  • the decoder infers directional information by applying Sobel filters in horizontal and vertical directions to neighboring pixels (pixels) adjacent to the current block, and converts the directional information into the intra prediction mode.
  • a mapping method can be used.
  • a method in which a decoder derives an intra prediction mode using neighboring blocks may be described as decoder side intra mode derivation (DIMD).
  • FIG. 7 is a diagram illustrating positions of neighboring blocks used to construct a motion candidate list in inter prediction.
  • Neighboring blocks may be spatially positioned blocks or temporally positioned blocks. Neighboring blocks that are spatially adjacent to the current block are Left (A1) blocks, Left Below (A0) blocks, Above (B1) blocks, Above Right (B0) blocks, or Above Left (Above Left) blocks. , B2) may be at least one block.
  • a neighboring block temporally adjacent to the current block may be a block including a position of an upper left pixel of a bottom right (BR) block of the current block in a collocated picture.
  • TMVP Temporal Motion Vector Predictor
  • sbTMVP sub-block temporal motion vector predictor
  • slice type information eg, I slice, P slice, or B slice
  • slice type information eg, I slice, P slice, or B slice
  • whether it is a tile, whether it is a sub picture the size of the current block, the depth of the coding unit, and the current block. It may be determined based on at least one of information about whether the luminance block is a chrominance block, whether it is a reference frame or a non-reference frame, a temporal layer according to a reference order, and a layer.
  • Information used to determine whether or not the methods described in this specification will be applied may be information previously agreed upon between the decoder and the encoder. Also, these pieces of information may be determined according to profiles and levels.
  • Such information may be expressed as a variable value, and information on the variable value may be included in a bitstream. That is, the decoder may determine whether the above-described methods are applied by parsing information on variable values included in the bitstream. For example, whether the above-described methods are to be applied may be determined based on a horizontal length or a vertical length of a coding unit. If the horizontal length or the vertical length is 32 or more (eg, 32, 64, 128, etc.), the above methods can be applied. In addition, the above-described methods may be applied when the horizontal or vertical length is less than 32 (eg, 2, 4, 8, or 16). In addition, when the horizontal length or the vertical length is 4 or 8, the above-described methods can be applied.
  • FIG. 8 shows that a video signal processing apparatus according to an embodiment of the present invention stores intra- and inter-prediction modes of a current block.
  • Methods of predicting the current block can be largely divided into methods of performing intra prediction using spatial correlation and methods of performing inter prediction using temporal correlation. If the current block is a block reconstructed through intra prediction, the decoder stores information related to intra prediction of the current block in memory and does not store information related to inter prediction of the current block. If the current block is a block reconstructed through inter prediction, the decoder stores information related to inter prediction of the current block in memory and does not store information related to intra prediction of the current block.
  • encoding information of the current block may be predicted through encoded information of neighboring blocks.
  • the decoder can predict the current block through intra prediction information of neighboring blocks of the current block.
  • neighboring blocks of the current block are blocks reconstructed through inter prediction, the efficiency of intra prediction of the current block may be lowered.
  • the decoder may store information about intra prediction of a block reconstructed through inter prediction. It is possible to increase the efficiency of intra prediction of a block to be processed next to a block reconstructed through inter prediction through name. 8(a) shows that information about intra prediction is stored in a memory when this method is applied. In this case, information about intra prediction of a block reconstructed through inter prediction may be information about intra prediction of a reference block used in inter prediction. This is to use the fact that the image characteristics of the current block and the image characteristics of the reference block are highly likely to be similar.
  • the decoder can predict the current block through inter prediction information of neighboring blocks of the current block.
  • neighboring blocks of the current block are blocks reconstructed through intra prediction, inter prediction efficiency of the current block may be reduced.
  • the decoder may store information about inter prediction of a block reconstructed through intra prediction.
  • the efficiency of inter prediction of a block to be processed next to a block reconstructed through intra prediction through name can be increased.
  • 8(b) shows that information about inter prediction is stored in a memory when this method is applied.
  • information on inter prediction of a block reconstructed through intra prediction may be information about inter prediction of a neighboring block used as a reference pixel in intra prediction. This is to use the fact that the image characteristics of the current block and the image characteristics of the reference block are highly likely to be similar.
  • FIG. 9 shows prediction of a current block using information on neighboring blocks of the current block described with reference to FIG. 8 .
  • the decoder may generate an MPM list using information about intra prediction of neighboring blocks of the current block, and derive an intra prediction mode of the current block based on the generated MPM list.
  • the decoder may generate an MPM list using information about intra prediction of the neighboring blocks.
  • information about intra prediction may be an intra prediction mode.
  • the decoder When the current block is reconstructed through inter prediction, the decoder generates a motion candidate list using information about inter prediction of neighboring blocks of the current block, and based on the generated motion candidate list, the current block is reconstructed.
  • Motion information can be derived.
  • motion information may represent a motion vector.
  • the decoder may generate a motion candidate list using information on inter prediction of the neighboring blocks.
  • information on inter prediction may be motion information.
  • the decoder may store prediction information in at least one memory among the current picture memory and the reference picture memory.
  • the 10 is an intra prediction corresponding to a block reconstructed by using an intra prediction mode of a block referred to in inter prediction using motion information of a block reconstructed by using inter prediction by a decoder according to an embodiment of the present invention. shows what leads to the mode.
  • the restored block may be the current block.
  • the decoder may derive an intra prediction mode of a reference block using motion information of a block reconstructed using inter prediction.
  • the intra prediction mode of the derived reference block may be stored as the intra prediction mode of the reconstructed block.
  • the decoder may store the intra prediction mode of the reference block as the intra prediction mode of the reconstructed block.
  • the decoder determines, as a reference block, a block at a location where the reconstructed block is moved to a reference picture according to motion information used to reconstruct the reconstructed block.
  • the decoder may store an intra prediction mode per pixel.
  • the decoder may store the intra prediction mode in units of a predetermined size.
  • a decoder may store two or more intra prediction modes per block.
  • the predetermined size may be 4x4 pixels or 8x8 pixels.
  • the current block is reconstructed by inter prediction.
  • the decoder obtains a reference block corresponding to the current block according to motion information of the current block in a reference picture referred to by the current block.
  • the decoder stores an intra prediction mode of a pixel corresponding to a specific pixel of the current block as an intra prediction mode of the current block.
  • the intra-prediction mode of a pixel is an intra-prediction mode stored in horizontal and vertical positions of a pixel, and is referred to as an intra-prediction mode of a pixel for convenience of explanation.
  • the specific pixel of the current block is the upper left pixel.
  • a specific pixel of the current block may be a center pixel of the current block.
  • the decoder determines that the intra prediction mode of the current block is the K8 intra prediction mode of the reference picture.
  • the intra prediction mode of the current block is the K8 intra prediction mode of the reference picture.
  • a pixel of a reference picture from which an intra prediction mode is extracted is referred to as an extracted pixel.
  • FIG. 11 shows derivation of an intra prediction mode of a neighboring block of a reconstructed current block using inter prediction when a decoder according to an embodiment of the present invention reconstructs a current block using intra prediction.
  • the decoder when the current block is reconstructed through intra prediction, the decoder generates an MPM list using information about intra prediction of neighboring blocks of the current block, and an intra prediction mode of the current block based on the generated MPM list. can induce In this case, when a neighboring block is reconstructed by inter prediction, information on intra prediction of the neighboring block may be an intra prediction mode derived from a reference block of the neighboring block. As described with reference to FIG. 10 , the decoder derives the intra prediction mode of the neighboring block from the reference block of the neighboring block.
  • the decoder derives the intra prediction mode from the reference block of the Ne-A0 block, which is a neighboring block of the current block. At this time, the decoder determines, as a reference block, a block at a position where the Ne-A0 block is moved as a reference picture according to motion information used to reconstruct the Ne-A0 block. As described with reference to FIG. 10 , the decoder determines the intra prediction mode corresponding to the upper left (H4) pixel of the reference block as the intra prediction mode of the Ne-A0 block.
  • H4 upper left
  • the intra prediction mode of the current block is highly likely to be similar to the intra prediction mode of neighboring blocks.
  • the closer the location used to derive the intra prediction mode of the neighboring block to the current block the higher the possibility that the accuracy of the intra prediction mode becomes higher. Therefore, the decoder can determine the position of the extracted pixel based on the distance between the block corresponding to the current block in the reference picture and the extracted pixel.
  • the decoder may determine the intra prediction mode corresponding to H5 or I5 of the reference block of the Ne-A0 block as the intra prediction mode of the Ne-A0 block.
  • the decoder may determine the intra prediction mode corresponding to the I11 pixel, the I13 pixel, or the I15 pixel of the reference block of the Ne-L1/L2 block as the intra prediction mode of the Ne-L1/L2 block.
  • decoder determines the location of a pixel from which an intra prediction mode of a reference picture is extracted will be described with reference to FIG. 12 .
  • FIG. 12 shows a process in which an intra prediction mode corresponding to a neighboring block is derived by a decoder according to an embodiment of the present invention by applying motion information of a neighboring block reconstructed using inter prediction to a pixel of a current block.
  • the decoder determines a pixel corresponding to a specific pixel in a reference picture as an extracted pixel according to motion information of a neighboring block, and determines the intra prediction mode of the extracted pixel as the intra prediction mode corresponding to the neighboring block. did The decoder may determine an extracted pixel as a pixel corresponding to a specific pixel of the current block when moving the specific pixel of the current block to a reference picture according to motion information of neighboring blocks of the current block.
  • the decoder may determine a pixel corresponding to the extracted pixel.
  • the corresponding pixel M10 is determined as an extracted pixel.
  • the decoder may use the intra prediction mode of the M10 pixel as the intra prediction mode of the Ne-A2/A3 block.
  • the decoder may use the derived intra prediction mode of neighboring blocks to generate the MPM list of the current block.
  • the decoder may determine a corresponding pixel as an extraction pixel when an arbitrary pixel of the current block is moved to a reference picture using motion information of a neighboring block.
  • the arbitrary pixels are the upper left pixel of the current block, the upper center pixel of the current block, the upper right pixel of the current block, the left center pixel of the current block, the lower left pixel of the current block, the lower center pixel of the current block, and the right side of the current block. It can be one of the bottom pixel, the right center pixel of the current block, and the middle pixel of the current block.
  • FIG. 13 shows a process in which an intra prediction mode corresponding to a neighboring block is derived by a decoder according to an embodiment of the present invention by applying motion information of a neighboring block reconstructed using inter prediction to a pixel of a subblock of a current block.
  • the decoder can divide the current block into a plurality of sub-blocks and reconstruct them using inter prediction for each sub-block. Specifically, the decoder may derive an intra prediction mode for each sub-block of the current block.
  • the decoder reconstructs the subblocks adjacent to the block boundary, the A0/L0 block, the A1 block, the A2 block, the A3 block, the L1 block, the L2 block, and the L3 block in FIG. 13, using the intra prediction mode of the neighboring block of the current block. can do.
  • the decoder uses subblocks that are not adjacent to the boundary of the current block, A1/L1 block, A2/L1 block, A3/L2 block, A1/L2 block, A2/L2 block, A3/L2 block, A1/L2 block, A1/L2 block in FIG.
  • Intra-prediction modes of blocks in which the L3 block, the A2/L3 block, and the A3/L3 block are moved to the reference picture according to the motion information of the neighboring blocks are derived, and the derived intra-prediction mode is used to determine whether a block is not adjacent to the boundary of the current block. Subblocks can be restored.
  • the decoder may derive a plurality of intra prediction modes for subblocks not adjacent to the boundary of the current block, and use any one of the plurality of derived intra prediction modes to reconstruct the subblocks not adjacent to the boundary of the current block. can do. For example, in the embodiment of FIG.
  • the decoder may determine the pixel N11 as the first extracted pixel according to the motion information of the block Ne-A2/A3, which is a neighboring block for the block A2/L2. Also, the decoder determines N14 as the second extracted pixel according to the motion information of the block Ne-L1/L2, which is a neighboring block. Also, the decoder determines O15 as the third extracted pixel according to the motion information of the block Ne-L3, which is a neighboring block. In this case, the decoder may reconstruct the A2/L2 block using one of intra prediction mode A stored in the first extracted pixel, intra prediction mode B stored in the second extracted pixel, and intra prediction mode C stored in the third extracted pixel. .
  • the decoder can reconstruct a sub-block that is not adjacent to the boundary of the current block by using an optimal intra-prediction mode among a plurality of derived intra-prediction modes.
  • the decoder may determine a median value among index values of a plurality of acquired intra prediction modes as an optimal intra prediction mode.
  • the decoder may determine an average value among index values of a plurality of acquired intra prediction modes as an optimal intra prediction mode.
  • the decoder may determine a minimum value among the acquired index values of a plurality of intra prediction modes as an optimal intra prediction mode.
  • the decoder may determine an optimal intra prediction mode by performing a weighted average of the obtained index values of a plurality of intra prediction modes.
  • a neighboring block may be a neighboring block spatially adjacent to the current block or a block temporally corresponding to the current block in a reference picture.
  • FIG. 14 shows a method of storing motion information for a current block when the decoder reconstructs the current block using intra prediction according to an embodiment of the present invention.
  • the decoder may select a reconstructed neighboring block using inter prediction based on an intra prediction mode of the current block and store motion information of the current block based on motion information of the selected neighboring block.
  • the decoder may store motion information of the current block as motion information of any one of pixels included in neighboring blocks located above the current block.
  • the intra prediction mode of the current block is B-direction
  • the decoder may store motion information of the current block as motion information of any one of pixels included in neighboring blocks located to the left of the current block.
  • the decoder may store the intra prediction mode used in the I picture as an intra prediction mode corresponding to a block reconstructed using inter prediction in the next picture.
  • the decoder may store the stored intra prediction mode as an intra prediction mode corresponding to a block reconstructed using inter prediction of another picture. This will be described in detail with reference to FIG. 15 .
  • a decoder according to an embodiment of the present invention stores an intra prediction mode determined in an I picture as an intra prediction mode corresponding to a block reconstructed using inter prediction in a next picture.
  • each picture is displayed in a time order direction (POC; Picture Order Count).
  • POC is an output number for each picture and is incremented in chronological order.
  • the decoder may store the intra prediction mode A used in the first I picture as an intra prediction mode corresponding to the inter prediction block of picture 1, which is the next B picture.
  • the decoder may store the corresponding intra prediction mode A as an intra prediction mode corresponding to a block reconstructed using inter prediction in picture 2, which is the next B picture.
  • a correlation between the first I picture and a picture to be decoded later in time may be lowered. Accordingly, a correlation between an intra prediction mode used in the first I picture and a block reconstructed by using inter prediction of a picture temporally distant from the I picture may be lowered.
  • the decoder replaces the intra prediction mode used in the I picture with the intra prediction mode of the block recently reconstructed using intra prediction, and stores the replaced intra prediction mode as the intra prediction mode corresponding to the block reconstructed using inter prediction.
  • two pieces of motion information are used to reconstruct a specific block of picture 3, which is a B picture.
  • the decoder may store B, which is an intra prediction mode of a reference picture temporally closer to the current picture among two reference blocks of the specific block, as an intra prediction mode of the specific block.
  • the decoder may store at least one of a POC of a picture in which the intra prediction mode is used and the number of propagation times of the intra prediction mode together with the intra prediction mode.
  • the decoder may determine the priority of the intra prediction mode to be inserted into the MPM list based on at least one of the POC stored with the intra prediction mode and the propagation count. In this case, the decoder may increase the priority of the intra prediction mode to be inserted into the MPM list as the POC is closer to the current picture.
  • the decoder can increase the priority of the intra prediction mode to be inserted into the MPM list as the number of propagation decreases.
  • 16 shows that a bitstream of a specific picture is dropped due to a network condition when a decoder according to an embodiment of the present invention performs decoding.
  • a bitstream of one picture may be dropped.
  • picture 1 which is a B picture
  • the decoder cannot check the information of picture 1 and thus cannot derive an intra prediction mode for picture 2. Accordingly, the decoder may fail to decode picture 2.
  • the decoder needs to delete and initialize an intra prediction mode stored as an intra prediction mode corresponding to a picture reconstructed by inter prediction according to a predefined condition.
  • the predefined condition may include a condition regarding a difference between a POC of a picture from which the intra prediction mode is derived and a POC of a picture from which the intra prediction mode is initially derived.
  • the decoder may delete and initialize the stored intra prediction mode when the difference between the POC of the picture from which the intra prediction mode is derived and the POC of the picture from which the intra prediction mode is first derived is equal to or greater than a predetermined number.
  • the predetermined number may be any one of 2, 3 and 4.
  • predefined conditions may include conditions regarding the decoding order of pictures from which the intra-prediction mode is derived and the decoding order of pictures from which the intra-prediction mode is first derived.
  • the decoder may delete and initialize the stored intra prediction mode when the difference between the decoding order of the picture from which the intra prediction mode is derived and the decoding order of the picture from which the intra prediction mode is first derived is equal to or greater than a predetermined number.
  • the predetermined number may be any one of 2, 3 and 4.
  • the decoder may set the intra prediction mode to one of a predefined value among a planar mode, a DC mode, and an angular mode.
  • the decoder may store syntax indicating that the intra prediction mode stored in the PPS, picture header, and slice header is an initialized intra prediction mode value. If there are multiple intra prediction modes derived by the decoder to use intra prediction for the current block and the multiple intra prediction modes correspond to the values of the initialized intra prediction modes, the decoder selects an uninitialized intra prediction mode among the multiple intra prediction modes. Intra prediction may be performed on the current block by preferentially using the mode value. Specifically, the decoder may insert an uninitialized intra prediction mode value among a plurality of intra prediction modes into the MPM list with a higher priority than the initialized intra prediction mode value.
  • the value of the intra prediction mode may correspond to any one of values from -14 to 80.
  • the value of the intra prediction mode is stored as a value range of 0 to 66 representing intra prediction modes other than the extended angle mode.
  • the decoder may obtain an intra prediction mode value corresponding to the extended angle mode by adding a predetermined offset value according to the aspect ratio of the current block to the intra prediction mode value.
  • the aspect ratio of the block from which the intra prediction mode is derived may be different from the aspect ratio of the current block. Therefore, a method of storing an index of an intra prediction mode in consideration of this is required.
  • the decoder when the intra prediction mode to be stored by the decoder corresponds to the extended angle mode, the decoder converts the value of the intra prediction mode to the extended angle mode as an index before an offset value is added according to the aspect ratio of the block. value can be stored. Therefore, the range of values of the intra prediction mode stored by the decoder is 0 to 66. Also, when the decoder applies the stored intra prediction mode to the current block, the decoder may use the stored intra prediction mode value as it is regardless of the aspect ratio of the current block.
  • the decoder when the intra prediction mode to be stored by the decoder corresponds to the extended angle mode, the decoder converts the value of the intra prediction mode to the extended angle mode as an index before an offset value is added according to the aspect ratio of the block. value can be stored. Therefore, the range of values of the intra prediction mode stored by the decoder is 0 to 66. Also, when the decoder applies the stored intra prediction mode to the current block, the decoder may add a preset offset according to the aspect ratio of the current block to the stored intra prediction mode value.
  • the decoder when the intra prediction mode to be stored by the decoder corresponds to the extended angle mode, the decoder converts the value of the intra prediction mode to the extended angle mode, an index value obtained by adding an offset value according to the aspect ratio of the block, That is, it can be stored as an index value indicating the extended angle mode. Therefore, the range of values of the intra prediction mode stored by the decoder is -14 to 80. When the decoder applies the stored intra prediction mode to the current block, the decoder may use the stored intra prediction mode value without conversion.
  • the decoder when the intra prediction mode to be stored corresponds to the extended angular mode, the decoder converts the value of the intra prediction mode into an index value obtained by adding an offset value according to the aspect ratio of the block to the extended angular mode, that is, extended It can be stored as an index value indicating the angle mode. Therefore, the range of values of the intra prediction mode stored by the decoder is -14 to 80.
  • the decoder may add a predetermined offset according to the aspect ratio of the current block to the value of the stored intra prediction mode.
  • the decoder may store the value of the intra prediction mode as it is. At this time, the range of values of the intra prediction mode stored by the decoder is 0 to 66. When the decoder applies the stored intra prediction mode to the current block, the decoder may use the stored intra prediction mode value as it is.
  • the decoder may store the value of the intra prediction mode as it is. At this time, the range of values of the intra prediction mode stored by the decoder is 0 to 66.
  • the decoder may add a predetermined offset according to the aspect ratio of the current block to the value of the stored intra prediction mode.
  • the decoder may store the intra prediction mode based on whether the intra prediction mode to be stored corresponds to the extended angle mode and whether the extended angle mode can be applied to the current block.
  • the decoder may not store the intra prediction mode.
  • the decoder may quantize the value of the intra prediction mode.
  • the decoder explicitly signals the intra prediction mode of a block to be reconstructed using the intra prediction mode, the decoder can store the value of the intra prediction mode in the range of 0 to 66.
  • the decoder uses 7 bits to store the intra prediction mode.
  • FIG. 17 shows a method of storing an intra prediction mode corresponding to a block reconstructed through inter prediction by a decoder according to an embodiment of the present invention.
  • the decoder may divide the value of the intra prediction mode by a pre-specified value and store it.
  • the predefined value may be 2.
  • the decoder may store the value of the intra prediction mode as 16.
  • the decoder when storing an intra prediction mode corresponding to a block reconstructed through inter prediction, the decoder may store a value of the intra prediction mode in the range of 0 to 33. In this embodiment the decoder uses 6 bits to store the intra prediction mode.
  • the decoder retrieves and uses the stored intra prediction mode value, the decoder may use a value obtained by multiplying the intra prediction mode value by a predetermined value. For example, when the value of the stored intra prediction mode is 16 and the pre-specified value is 2, the decoder may use the value of the intra prediction mode as 32.
  • the decoder when the decoder stores an intra prediction mode corresponding to a block reconstructed through inter prediction, the decoder performs clipping to store consecutive values within a pre-specified range of the intra prediction mode as one intra prediction mode value. can do. For example, when the value designated in advance by the decoder is 2 in the above-described embodiment, the decoder can store all 62 to 66 as 31. When the stored intra prediction mode value is 31 and the predefined value is 2, the decoder may use the intra prediction mode value as 62.
  • the decoder may not perform quantization and clipping for the planar mode and the DC mode.
  • the decoder may store intra prediction mode values 0 (plane mode) and 1 (DC mode) as 0 and 1, respectively. Also, the decoder may store values 2 to 4 of the intra prediction mode as 2.
  • 17(a) shows that the decoder stores the value of the intra prediction mode in the memory using the quantization and clipping described above.
  • 17(b) shows that the decoder restores the value of the intra prediction mode stored in the memory.
  • the decoder may store at least one of POC information and propagation count together with an intra prediction mode. Specifically, the decoder may store at least one of POC information and propagation number as much as the number of bits saved according to the above-described embodiments. In another specific embodiment, the decoder may store the priority or accuracy of the intra prediction mode together with the intra prediction mode. Specifically, the decoder may store the priority of the intra prediction mode as much as the number of bits saved according to the above-described embodiments. For example, in the case of 6 bits indicating the value of the intra prediction mode, the decoder may store the priority of the intra prediction mode as 2 bits.
  • FIG. 18 shows that a decoder according to an embodiment of the present invention derives an intra prediction mode from a block encoded in CIIP mode.
  • the decoder When a decoder reconstructs a current block using intra prediction to use CIIP, when a specific pixel of the current block is moved to a reference picture according to motion information of the current block, the decoder can use the intra prediction mode of the corresponding pixel.
  • the specific pixel of the current block may be any one of an upper left pixel, an upper right pixel, a center pixel, a lower left pixel, a lower right pixel, and a middle pixel of the current block.
  • the decoder when a decoder reconstructs a current block using intra prediction to use CIIP, the decoder divides the current block into a plurality of subblocks and moves a specific pixel of each of the plurality of subblocks to a reference picture.
  • the intra prediction mode of the corresponding pixel may be used.
  • the specific pixel of the sub-block may be any one of an upper-left pixel, an upper-right pixel, a center pixel, a lower-left pixel, a lower-right pixel, and a middle pixel of the sub-block.
  • the decoder generates a prediction block by performing a weighted average of a reconstructed block by performing inter prediction and a reconstructed block by performing intra prediction.
  • a decoder uses a flat mode as an intra prediction mode.
  • an intra prediction mode of a block having a high correlation with the current block may be used.
  • FIG. 19 shows that a decoder according to an embodiment of the present invention reconstructs a current block using GPM.
  • the decoder uses GPM to reconstruct a block
  • the decoder divides the CU into two blocks with an oblique line with angles and distances corresponding to the individual modes of GPM, and at least one of inter-prediction and intra-prediction is applied to the two divided blocks. do either one 19(a) shows the angle of the oblique line applied according to each mode of GPM, and FIG. 19(b) shows the distance of the oblique line applied according to each mode of GPM.
  • the decoder can reconstruct the GPM block using the intra prediction mode of the corresponding reference block.
  • the intra prediction mode of the reference block is highly likely to reflect the image characteristics of the current block, and thus is highly likely to have a high correlation with the oblique line dividing the GPM block.
  • the decoder may infer the angle of the oblique line dividing the GPM block based on the intra prediction mode derived from the reference block of the GPM block for each GPM block.
  • the encoder may store in a bitstream a difference between an angle indicated by an intra prediction mode derived from a reference block of a GPM block and an angle of a diagonal line dividing the GPM block.
  • this angular difference is referred to as a difference angle.
  • the decoder may infer the angle of the oblique line dividing the GPM block using the intra prediction mode derived from the reference block of each GPM block and the difference angle stored in the bitstream.
  • the decoder when there are two intra prediction modes derived from a reference block of a GPM block, the decoder can select one of the two intra prediction modes based on whether the derived intra prediction mode is an angular mode.
  • the decoder may select one of the two intra prediction modes based on an angle indicated by the derived intra prediction modes.
  • information indicating the distance of the oblique line may be stored in the bit stream.
  • merge_gpm_idx0[x0][y0] and merge_gpm_idx1[x0][y0] are index information used to derive motion information of a GPM block.
  • the decoder uses the motion information indicated by the index information in the motion candidate list configured for the GPM block as motion information of two divided blocks.
  • merge_gpm_delta_angle[x0][y0] indicate the difference angle stored in the bitstream.
  • merge_gpm_distance_idx[x0][y0] indicates the distance of the oblique line applied to the GPM block.
  • the decoder can derive the intra prediction mode from the reference block of the GPM block and add a difference angle (merge_gpm_delta_angle) to the derived intra prediction mode to infer the angle of the oblique line applied to the GPM block. Also, the decoder may obtain the distance of the oblique line (merge_gpm_distance_idx) from the bitstream. The decoder may determine the shape of the GPM block using the inferred oblique angle and the obtained oblique distance.
  • Entropy coding using context adaptive binary arithmetic coding may be applied to values indicated by merge_gpm_delta_angle and merge_gpm_distance_idx.
  • Context models for merge_gpm_delta_angle and merge_gpm_distance_idx can be defined as values obtained through experiments.
  • FIGS. 20 and 21 may have the same values.
  • initValue represents context models for merge_gpm_delta_angle and 'merge_gpm_distance_idx. shiftIdx is used when updating the probabilities for merge_gpm_delta_angle and merge_gpm_distance_idx.
  • initValue is determined according to whether the current slice type is I slice, P slice, or B slice.
  • 20(b) and 21(b) show context models that can be used according to each slice type.
  • initType when the current slice type is P slice, initType may have a value of 0 to 4.
  • initType can have a value of any one of 5 to 9.
  • initValue There may be at least one initValue used for each slice type.
  • the value of initValue can be defined one by one according to the slice type. In this case, when the current slice type is P slice, initType is 0, and initValue corresponding to 0 is 41. Also, when the current slice type is B slice, initType is 5 and initValue corresponding to 5 is 42.
  • initValue according to the slice type may be selectively applied to each slice.
  • the value of initType mapped according to the slice type may vary according to the value of sh_cabac_init_flag defined in the slice header.
  • the value of sh_cabac_init_flag may be 1. At this time, when the current slice type is P slice, initType is 5. Also, when the current slice type is B slice, initType is 0.
  • the sh_cabac_init_flag value may be 0. At this time, when the current slice type is P slice, initType is 0. If the current slice type is B slice, initType is 5.
  • the encoder selects one of several context models for merge_gpm_delta_angle and merge_gpm_distance_idx symbols is described.
  • the encoder may apply the following embodiments individually or together.
  • the encoder may select a context model for merge_gpm_delta_angle and merge_gpm_distance_idx of the current block based on merge_gpm_delta_angle values and merge_gpm_distance_idx values of neighboring blocks of the current block. Specifically, the encoder may select a context model for merge_gpm_delta_angle and merge_gpm_distance_idx of the current block based on the sum of merge_gpm_delta_angle values and merge_gpm_distance_idx values of adjacent blocks of the current block.
  • the value of the context index indicating the context model of the current block is any one of 0 to 2. If the current block is located at a position where neighboring blocks cannot be used, the sum of the value of merge_gpm_delta_angle and the value of merge_gpm_distance_idx is zero.
  • the encoder may select context models for merge_gpm_delta_angle and merge_gpm_distance_idx of the current block according to whether neighboring blocks of the current block use template matching.
  • the encoder may determine the value of the context index as 2.
  • the encoder may determine the context index as 1.
  • the encoder may determine the value of the context index as 0.
  • the encoder may select a context model for merge_gpm_delta_angle and merge_gpm_distance_idx of the current block according to the size of the current block.
  • the encoder may determine the value of the context index as 2. If the size of the current block is smaller than a predetermined second value, the encoder may determine the value of the context index as 0.
  • the encoder may determine the value of the context index as 1. In this case, the first value may be 32x32. Also, the second value may be 16x16. Also, the first value and the second value may be determined based on the sum of the width and height of the current block.
  • the encoder may apply bypass-type binary arithmetic encoding using a fixed probability interval to merge_gpm_delta_angle and merge_gpm_distance_idx without applying binary arithmetic encoding using a context model.
  • binary arithmetic encoding in the form of a bypass may be selectively applied to the luminance block and the chrominance block. Specifically, binary arithmetic encoding may be performed on the luminance block through a context model, and binary arithmetic encoding of a bypass type may be applied to the chrominance block. In another specific embodiment, binary arithmetic encoding may be performed on a chrominance block through a context model, and binary arithmetic encoding of a bypass type may be applied to a luminance block.
  • the encoder may perform binary arithmetic encoding of merge_gpm_delta_angle and merge_gpm_distance_idx using only one context model. Since each slice type is mapped to one context model, the encoder can use a fixed context model for all blocks of the slice without deriving a context model index.
  • FIG. 22 shows that an encoder and a decoder perform transform on a residual block to obtain transform coefficients and perform inverse quantization on a quantized transform coefficient to reconstruct a residual block according to an embodiment of the present invention.
  • the encoder may quantize a transform coefficient value obtained by transforming the residual signal described above, and code the quantized transform coefficient.
  • the transform unit may obtain a transform coefficient value by transforming the residual signal.
  • the residual signal of the specific block may be distributed over the entire area of the current block. Accordingly, the encoder may concentrate energy in a low frequency domain by applying frequency domain transform to the residual signal. Through this, the encoder can improve coding efficiency.
  • the encoder may obtain at least one residual block including a residual signal for the current block.
  • the residual block may be either a current block or a block divided from the current block.
  • a residual block may be referred to as a residual array or residual matrix including residual samples of the current block.
  • a residual block may indicate a block having the same size as a transform unit or transform block.
  • the encoder can transform the residual block using a transform kernel.
  • a transform kernel used for transforming the residual block may be a transform kernel having separable characteristics of vertical transform and horizontal transform.
  • the transformation of the residual block may be performed by dividing it into vertical transformation and horizontal transformation.
  • the encoder may perform vertical transform by applying a transform kernel in the vertical direction of the residual block.
  • the encoder may perform horizontal transform by applying a transform kernel in the horizontal direction of the residual block.
  • a transform kernel may be used as a term referring to a parameter set used for transforming a residual signal, such as a transform matrix, a transform array, a transform function, and a transform.
  • the conversion kernel may be any one of a plurality of available kernels.
  • transform kernels based on different transform types may be used for each of the vertical transform and the horizontal transform.
  • the encoder before performing the primary transformation, derives transformation methods for vertical and horizontal directions by using at least one of the intra prediction mode, encoding mode, and size information of the current block. can do.
  • the encoder may process the high frequency region as 0 while leaving only the low frequency region. This process is called high-frequency zeroing.
  • the encoder may set the size of the low-frequency region to a predetermined size. For example, the encoder may select one of 4, 8, 16, and 32 horizontal and vertical sizes, respectively.
  • the encoder may transfer the transform block converted from the residual block to the quantization unit for quantization.
  • the transform block may include a plurality of transform coefficients.
  • the transform block may include a plurality of transform coefficients in a two-dimensional array.
  • the size of the transform block may be the same as either the current block or a block divided from the current block, similar to the residual block.
  • Transform coefficients transmitted to the quantization unit may be expressed as quantized values.
  • the encoder can perform additional transforms before the transform coefficients are quantized.
  • the transform method described above is referred to as a primary transform, and an additional transform may be referred to as a secondary transform.
  • Secondary transformation may be selective for each residual block.
  • the encoder may improve coding efficiency by performing secondary transformation on a region in which it is difficult to concentrate energy in a low-frequency region with only primary transformation.
  • the encoder may additionally perform a secondary transform on a block in which residual values appear large in a direction other than the horizontal or vertical direction of the residual block.
  • a residual value of a block reconstructed using intra prediction has a higher probability of changing in a direction other than a horizontal or vertical direction compared to a residual value of a block reconstructed using inter prediction.
  • the encoder may further perform a secondary transform on the residual signal of the reconstructed block using intra prediction. Also, the encoder can omit the secondary transform for the residual signal of the reconstructed block using inter prediction. The encoder can perform high-frequency zeroing during the second transform that can be used in the first transform.
  • the encoder may determine whether to perform the secondary transform according to the size of the current block or the residual block. Also, the encoder may use transform kernels having different sizes according to the size of the current block or the residual block. For example, the encoder may apply an 8X8 secondary transform to a block in which the shorter side of the width or height is greater than or equal to the first predetermined length. In addition, the encoder may apply a 4X4 secondary transform to a block in which the shorter side of the width or height is greater than or equal to the second predetermined length and smaller than the first predetermined length. In this case, the first predetermined length may be greater than the second predetermined length. Also, unlike the first transformation, the secondary transformation may not be performed separately into vertical transformation and horizontal transformation. This secondary transform may be referred to as a Low Frequency Non-Separable Transform (LFNST).
  • LFNST Low Frequency Non-Separable Transform
  • the encoder can omit transforming the residual signal of a specific region.
  • the encoder may determine whether or not to perform transform on the residual signal of a specific region based on a syntax element related to transform of the specific region.
  • the syntax element may include transform skip information.
  • Transform skip information may be a transform skip flag.
  • the decoder may parse the syntax elements related to the transformation described above from the bitstream of the video signal.
  • the decoder may entropy-decode the video signal bitstream to obtain transform-related syntax elements.
  • the encoder may entropy-code the transform-related syntax elements and include them in the video signal bitstream.
  • the decoder may obtain information related to a conversion process required for decoding by parsing the bitstream.
  • Information related to the transform process may include index information for primary and secondary transform types and quantized transform coefficients.
  • 22(b) shows an inverse transform process in an encoder or decoder.
  • the inverse transform unit may obtain a residual signal by inverse transforming the inverse quantized transform coefficient.
  • the inverse transformation unit may detect whether inverse transformation of a corresponding area is performed from syntax elements related to transformation of a specific area. When a transform related syntax element for a specific transform block indicates transform skip, the transform for the corresponding transform block may be omitted. In this case, both the first-order inverse transform and the second-order inverse transform can be omitted for the transform block.
  • the inverse quantized transform coefficient may be used as a residual signal.
  • the decoder may reconstruct the current block by using the inverse quantized transform coefficient as a residual signal.
  • the second-order inverse transform may be performed and the first-order inverse transform may be omitted.
  • the secondary inverse transformed value may be used as the residual signal.
  • the first-order inverse transform described above represents an inverse transform for the first-order transform, and may be referred to as an inverse primary transform.
  • the second-order inverse transform represents an inverse transform for the second-order transform, and may be referred to as an inverse secondary transform or inverse LFNST.
  • a first (inverse) transformation may be referred to as a first (inverse) transformation
  • a secondary (inverse) transformation may be referred to as a second (inverse) transformation.
  • FIG. 23 illustrates a process in which an encoder and a decoder derive a transform type using at least one of information on a current block and neighboring blocks, an intra prediction mode, an encoding mode, and parsed transform type index information according to an embodiment of the present invention.
  • a method of deriving a transform type is selected through a controller of an encoder or a controller of a decoder, and the following embodiments may be applied individually or together.
  • the encoder when the encoder encodes the current block using ISP and performs secondary transform, the encoder may set the primary transform type to DCT2, which is a basic transform type.
  • the encoder when the encoder does not use a multiple transform set (MTS) for the current block, the encoder may set the primary transform type of the current block to DCT2, which is a basic transform type.
  • MTS multiple transform set
  • the encoder may set the transform type of the current block based on at least one of whether the current block is coded using MTS, ISP, or template matching mode and the size of the current block.
  • the encoder codes the current block using MTS, ISP, and template matching modes, and the horizontal or vertical size of the current block is a predefined size, the encoder may set the transform type of the current block to DST7.
  • the pre-specified size can be greater than 4 and less than 16.
  • the encoder may set the transform type of the current block to DCT2.
  • the encoder may determine the transform type of the current block based on whether sub-block transform is applied to the current block.
  • the sub-block transform indicates that the current block is divided into a plurality of sub-blocks and a transform for each of the plurality of sub-blocks is applied.
  • the encoder may apply a sub-block transform mode to the current block and determine the transform type of each sub-block based on the size of the current block, the split direction of the sub-block, the split type, and the sub-block.
  • the division type may indicate whether the division is symmetrical or asymmetrical.
  • the division direction may be divided into vertical and horizontal.
  • the encoder may determine DCT2 as the transform type in the horizontal and vertical directions.
  • the sub-block transformation mode is applied, the sub-block division type is vertical division and symmetric division, and the sub-block is located on the left side, the encoder moves the sub-block in the horizontal direction.
  • DCT8 may be applied to transform for the sub-block and DCT7 may be applied to transform for the vertical direction of the sub-block.
  • the encoder determines a set of transform types to be used for the current block based on the direction indicated by the intra prediction mode of the current block and the size of the current block.
  • the encoder may perform transformation on the current block by selecting the most efficient transformation type from among the set of determined transformation types.
  • the decoder may obtain the transform type applied to the current block by parsing information on the transform type selected by the encoder from the bitstream.
  • the encoder may perform transformation on the current block by selecting a transform type with high efficiency from among a set of predefined transform types.
  • the predefined transform type set may be specified regardless of the direction indicated by the intra prediction mode of the current block and the size of the current block.
  • the decoder may obtain the transform type applied to the current block by parsing information on the transform type selected by the encoder from the bitstream.
  • the encoder may perform transformation on the current block by selecting a transform type with high efficiency from among a predefined transform type set.
  • the predefined transform type set may be specified regardless of the direction indicated by the intra prediction mode of the current block and the size of the current block.
  • the decoder may obtain the transform type applied to the current block by parsing information on the transform type selected by the encoder from the bitstream. Specifically, the decoder may parse the value of mts_idx corresponding to the current block from the bitstream and determine the transform type applied to the current block according to the value of mts_idx.
  • the encoder selects a set of transform types usable in the current block based on the direction indicated by the intra prediction mode of the current block, the size of the current block, and the horizontal to vertical ratio of the current block. It can be determined according to the following examples.
  • the decoder may set values of nSzIdxW and nSzIdxH based on the size of the current block.
  • the decoder can compute the log of 2 for the width of the current block, discard the decimal places, and set 2 as the difference value and the minimum of 3 as the value of nSzIdxW.
  • the decoder calculates the log value of 2 at the height of the current block, discards the decimal point, and sets the difference value of 2 and the minimum value of 3 to the value of nSzIdxH.
  • the encoder may reduce the precision by setting the value of the intra prediction mode to 67.
  • the encoder may set the value of ucMode, the value of nMdIDX, and the value of isTrTranposde according to the following embodiment.
  • the encoder may set the value of ucMode to 0, the value of mMdIdx to 35, and the value of isTrTransposed to a value derived from MIP.
  • the encoder may set values of ucMode, mMdIdx, and isTrTransposed according to the following embodiments.
  • the encoder may set the value of ucMode to the value of the intra prediction mode of the current block.
  • the encoder may convert the value of predMode into an extended angle mode according to the ratio of the horizontal and vertical sizes of the current block and set it as the value of predMode.
  • the encoder may clip predMode to a value between 2 and 66. If the value of preMode is greater than 34 indicating diagonal mode, the encoder may set the value of isTrTransposed to 1. If the value of preMode is not greater than 34 indicating diagonal mode, the value of isTrTranspose can be set to 0. When the value of predMode is greater than 34, the encoder may set the value of preMode as a difference between the value of preMode and the value obtained by adding 1 to the maximum value of 67 of preMode. Through this, the value of preMode is distributed based on the value of preMode indicating the diagonal mode, 34, and the encoder can reduce the size of the conversion mapping table, for example, FIG. 24 by half.
  • FIG 24 shows an index of a transform type set mapped to an intra-screen orientation mode (0 to 34 and MIP) of a current block and a size index (0 to 15) of the current block according to an embodiment of the present invention.
  • the encoder derives the value of nSzIdx based on the values of nSzIdxW, nSzIdxH, and isTrTransposed.
  • the encoder can set a value obtained by multiplying nSzIdxH by 4 and adding nSzIdxW as the value of nSzIdx.
  • the encoder may derive nTrSet indicating an index of a set of transform types usable for the current block based on the value of nSzIdx indicating the size of the current block and the value of nMdIdx indicating the direction of the intra prediction mode of the current block.
  • the encoder may derive nTrSet based on the values of nSzIdx and nMdIdx using the conversion table of FIG. 24 .
  • a value of nTrSet may indicate 80 conversion types.
  • the value of nTrset is 7.
  • 25 shows a conversion type set corresponding to nTrSet according to an embodiment of the present invention.
  • the encoder may obtain a transform type set corresponding to nTrSet by substituting the parsed value of mts_idx into the table of FIG. 25 . Also, the encoder may set the vertical and horizontal transform types differently depending on whether the value of preMode is greater than 34. When the value of nTrSet is 7 and the value of mts_idx is 3, the encoder can determine 22 among 2, 17, 18, and 22 as the index of the transform type set corresponding to nTrSet.
  • 26 shows a conversion type combination table according to an embodiment of the present invention.
  • the encoder selects 22 as the index of the transform type set corresponding to nTrSet, and the encoder can select DST1 and DCT5 as the transform type set corresponding to nTrSet according to the table of FIG. 26 .
  • the encoder may determine the vertical transform type of the block as DST1 and the horizontal transform type as DCT5.
  • the transform type in the vertical direction and the transform type in the horizontal direction described above may be exchanged.
  • the encoder may reset the IDT transform type of the vertical transform type or the horizontal transform type using the following embodiments.
  • FIG. 27 shows a threshold table for IDT conversion types according to an embodiment of the present invention.
  • the encoder may reset the vertical transformation type to the IDT transformation type. . If the absolute value of the difference between the intra prediction mode value of the current block and the intra prediction mode value 50 indicating the horizontal direction is smaller than a pre-specified value, the encoder may reset the horizontal direction transform type to the IDT transform type. .
  • the predetermined value may be an integer.
  • a predetermined value may be determined according to the width and height of the current block. In a specific embodiment, the encoder may determine a predetermined value using the table of FIG. 27 . The table in Fig.
  • FIG. 27 (a) shows a case where the threshold value is set differently whenever the horizontal or vertical size differs by '4', and the table in Fig. 27 (b) sets the threshold value whenever the horizontal or vertical size differs by two times. if set differently. If the size of the current block is 16x16, the IDT conversion type is not reset, and the existing conversion type is maintained.
  • the distribution of residual signals of the current block varies according to a position of a sample to be predicted and a distance between a reference pixel and an intra prediction mode. Specifically, the smaller the distance between the reference pixel and the sample to be predicted, the smaller the value of the residual signal. Also, as the distance between the reference pixel and the sample to be predicted increases, the value of the residual signal increases. In addition, the directionality of the residual signal may vary according to the direction indicated by the intra prediction mode. An optimal transform type for the current block may vary according to the distribution of the residual signal.
  • the encoder selects the optimum for the current block based on at least one of the intra prediction mode of the current block, the distance between the position of the sample to be predicted and the reference pixel, the width or height of the current block, and whether the current block is a luminance block or a chrominance block.
  • the conversion type can be determined. Specifically, the encoder determines the current block for the current block based on at least one of the intra prediction mode of the current block, the distance between the position of the sample to be predicted and the reference pixel, the width or height of the current block, and whether the current block is a luminance block or a chrominance block.
  • a set of optimal transformation types can be determined. This will be described with reference to FIG. 28 .
  • the encoder may use the extended angle mode according to the ratio of the width and height of the current block.
  • the encoder needs to remap the values of the extended angle mode to 0 to 63.
  • the encoder may remap values of -14 to -1 of the intra prediction mode having a value smaller than 0 to No. 2 of the transformation mapping table of FIG. 24.
  • the encoder reads -8 as 10 indicating a direction symmetrical to the direction indicated by -8 based on the direction indicated by 2. can be mapped.
  • the encoder when the value of the extended angle mode is -8, the encoder remaps -8 to 58 indicating the direction opposite to the direction indicated by -8 based on the center as shown in FIG. 28(b). can do. In another specific embodiment, when the extended angle mode of the current block is -8, the encoder may remap -8 to 42 indicating a direction symmetrical to the direction indicated by -8 based on the horizontal direction indicated by 18. there is. When the value of the extended angle mode is remapped according to the above-described embodiments, the encoder may set the value of isTrTransposed to 1. The encoder may equally apply the above-described embodiments to values of the extended angle mode greater than 66 and less than 80.
  • the encoder may remap 72 to 60 that is symmetrical to the direction indicated by 72 based on the direction indicated by the value 66 of the extended angle mode.
  • the encoder may remap 72 to 8 indicating a direction symmetrical with the direction indicated by 72 based on the center.
  • the encoder rewrites 72 as 28 indicating a direction symmetrical to the direction indicated by 72 based on the direction indicated by the value 50 of the extended angle mode. can be mapped.
  • 29 shows a method of reconstructing a block predicted by intra prediction by a decoder according to an embodiment of the present invention.
  • the intra prediction mode of the current block is 0 or 1, or if the current block is coded using MIP, the intra prediction mode indicates a non-directional mode.
  • the encoder may not use the transform type determination method using the tables of FIGS. 24 to 27 described above.
  • the encoder may designate a predetermined transform type and include syntax indicating the designated transform type in the bitstream.
  • the transform type designated by the encoder may be an optimal transform type among a plurality of transform types.
  • the decoder may parse the optimal transform type and set it as the vertical and horizontal transform types of the current block.
  • the decoder may apply position dependent intra prediction combination (PDPC) filtering to the block predicted using intra prediction.
  • PDPC position dependent intra prediction combination
  • the uniformity of the residual signal may be lower than before applying the PDPC filtering, and the ratio of the high frequency component of the error signal may be higher than the low frequency component of the error signal.
  • the decoder may not apply PDPC filtering to the block predicted using intra prediction. At this time, the decoder may perform transformation and quantization on the residual signal. 29 shows this process.
  • the decoder may reconstruct an image by adding a prediction block to which PDPC filtering is not performed. Thereafter, the decoder may perform PDPC filtering using an intra prediction mode between the reconstructed current block and neighboring blocks.
  • the decoder can derive the transform type using an intra prediction directional mode derived from a reference picture.
  • the decoder determines the width or height of the current block and the ratio between the width and height, whether AMVR is used, the difference between the POC of the reference picture and the POC of the current picture, the motion vector difference (MVD) value, and the amount of error signals. , it is possible to determine whether to derive a transform type using at least one of positional information of the last transform coefficient or to use the transform type derivation scheme.
  • the decoder may use the transform type derivation method.
  • FIG. 30 shows a transform kernel usable by an encoder and a decoder according to an embodiment of the present invention.
  • FIG. 30 shows DCT-II, discrete cosine transform type-V (DCT-V), discrete cosine transform type-VIII (DCT-VIII), discrete sine transform type-I (DST-I), and Shows the formula of the DST-VII kernel.
  • DCT and DST can be expressed as functions of cosine and sine, respectively.
  • index i represents the index in the frequency domain
  • index j represents an index within the basis function. As i becomes smaller, a low-frequency basis function is indicated, and as i increases, a high-frequency basis function is indicated.
  • the basis function T i (j) can represent the j-th element of the i-th row, and all of the transform kernels shown in FIG. 30 have separable characteristics. Accordingly, the residual signal X may be transformed in the horizontal direction and the vertical direction, respectively. That is, when the residual signal block is X and the transformation kernel matrix is T, the transformation of the residual signal X can be expressed as TXT'. In this case, T' means a transpose of the transform kernel matrix T.
  • DCT and DST are decimal forms, not integers. Therefore, processing the DCT and DST in decimal form imposes a processing burden on the hardware of the decoder and encoder.
  • the encoder and the decoder can approximate the decimal-type transform kernel to an integer-type transform kernel through scaling and rounding.
  • the integer precision of the conversion kernel may be determined to be 8 bits or 10 bits, but if the precision is low, encoding efficiency may decrease.
  • the orthonormal properties of DCT and DST may not be maintained, but since the resulting loss of encoding efficiency is not large, approximating the transform kernel in integer form is advantageous in terms of hardware encoder and decoder implementation.
  • IDTR Identity Transform
  • the identity transformation configures the transformation matrix by setting 1 to the position where the row and column have the same value.
  • the identity transformation is used to equally increase or decrease the value of the input residual signal by using an arbitrary fixed value other than 1.
  • the methods described above in this specification may be performed through a processor of a decoder or encoder.
  • the encoder may generate a bitstream that is decoded by a video signal processing method.
  • the bitstream generated by the encoder may be stored in a computer-readable non-transitory storage medium (recording medium).
  • parsing in this specification has been described focusing on the process of obtaining information from a bitstream, but from the encoder side, it can be interpreted as constructing corresponding information in a bitstream. Therefore, the term parsing is not limited to a decoder operation, but can also be interpreted as an act of constructing a bitstream in an encoder. In addition, such a bitstream may be configured by being stored in a computer readable recording medium.
  • embodiments of the present invention may be implemented through various means.
  • embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.
  • the method according to the embodiments of the present invention includes one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices) , Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.
  • ASICs Application Specific Integrated Circuits
  • DSPs Digital Signal Processors
  • DSPDs Digital Signal Processing Devices
  • PLDs Programmable Logic Devices
  • FPGAs Field Programmable Gate Arrays
  • processors controllers, microcontrollers, microprocessors, etc.
  • the method according to the embodiments of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above.
  • the software code can be stored in memory and run by a processor.
  • the memory may be located inside or outside the processor, and may exchange data with the processor by various means known in the art.
  • Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures or other data in a modulated data signal, such as program modules, or other transport mechanism, and includes any information delivery media.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne un dispositif de décodage d'un signal vidéo, le dispositif comprenant un processeur. Lors de la récupération du bloc courant au moyen d'une prédiction intra, le processeur : dérive un mode de prédiction intra correspondant aux blocs environnants du bloc courant sur la base d'informations de mouvement concernant les blocs environnants et d'une image de référence désignée par les blocs environnants ; et récupère le bloc courant au moyen du mode de prédiction intra correspondant aux blocs environnants. Les blocs environnants représentent des blocs qui ont été récupérés au moyen d'une prédiction intra.
PCT/KR2022/014893 2021-10-01 2022-10-04 Procédé de traitement de signal vidéo pour déterminer un mode de prédiction intra sur la base d'une image de référence, et dispositif associé WO2023055220A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR10-2021-0131062 2021-10-01
KR20210131062 2021-10-01
KR10-2021-0152596 2021-11-08
KR20210152596 2021-11-08

Publications (1)

Publication Number Publication Date
WO2023055220A1 true WO2023055220A1 (fr) 2023-04-06

Family

ID=85783306

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2022/014893 WO2023055220A1 (fr) 2021-10-01 2022-10-04 Procédé de traitement de signal vidéo pour déterminer un mode de prédiction intra sur la base d'une image de référence, et dispositif associé

Country Status (1)

Country Link
WO (1) WO2023055220A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20190034172A (ko) * 2011-10-28 2019-04-01 삼성전자주식회사 비디오 부호화 방법 및 장치, 비디오 복호화 방법 및 장치, 및 비트스트림을 포함하는 기록매체
KR20190107581A (ko) * 2018-03-12 2019-09-20 한국전자통신연구원 인트라 예측 모드의 유도를 위한 방법 및 장치
US20210112252A1 (en) * 2017-03-21 2021-04-15 Lg Electronics Inc. Image decoding method and device according to intra prediction in image coding system
KR20210079363A (ko) * 2019-02-06 2021-06-29 텐센트 아메리카 엘엘씨 비디오 코딩에서 이웃 블록 가용성을 위한 방법 및 장치
KR20210094046A (ko) * 2018-11-27 2021-07-28 오피 솔루션즈, 엘엘씨 명시적 및 암시적 시그널링을 사용한 이용 불가능한 참조 프레임들의 적응형 블록 업데이트

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20190034172A (ko) * 2011-10-28 2019-04-01 삼성전자주식회사 비디오 부호화 방법 및 장치, 비디오 복호화 방법 및 장치, 및 비트스트림을 포함하는 기록매체
US20210112252A1 (en) * 2017-03-21 2021-04-15 Lg Electronics Inc. Image decoding method and device according to intra prediction in image coding system
KR20190107581A (ko) * 2018-03-12 2019-09-20 한국전자통신연구원 인트라 예측 모드의 유도를 위한 방법 및 장치
KR20210094046A (ko) * 2018-11-27 2021-07-28 오피 솔루션즈, 엘엘씨 명시적 및 암시적 시그널링을 사용한 이용 불가능한 참조 프레임들의 적응형 블록 업데이트
KR20210079363A (ko) * 2019-02-06 2021-06-29 텐센트 아메리카 엘엘씨 비디오 코딩에서 이웃 블록 가용성을 위한 방법 및 장치

Similar Documents

Publication Publication Date Title
WO2021015523A1 (fr) Procédé et dispositif de traitement de signal vidéo
WO2019125093A1 (fr) Procédé et dispositif de traitement du signal vidéo
WO2016200100A1 (fr) Procédé et appareil de codage ou de décodage d'image au moyen d'une syntaxe de signalisation pour une prédiction de poids adaptatif
WO2018062788A1 (fr) Procédé de traitement d'image basé sur un mode de prédiction intra et appareil associé
WO2018070742A1 (fr) Dispositif et procédé de codage et de décodage d'image, et support d'enregistrement dans lequel le flux binaire est stocké
WO2017043949A1 (fr) Procédé et dispositif pour traiter un signal vidéo
WO2019190211A1 (fr) Procédé de traitement de signal vidéo et dispositif faisant intervenir une compensation de mouvement
WO2019117634A1 (fr) Procédé de codage d'image fondé sur une transformation secondaire et dispositif associé
WO2018070713A1 (fr) Procédé et appareil pour dériver un mode de prédiction intra pour un composant de chrominance
WO2019107998A1 (fr) Procédé et dispositif de traitement de signal vidéo
WO2018066958A1 (fr) Procédé et appareil de traitement de signal vidéo
WO2019050115A1 (fr) Procédé de traitement d'image fondé sur un mode de prédiction inter et appareil correspondant
WO2018105759A1 (fr) Procédé de codage/décodage d'image et appareil associé
WO2018062950A1 (fr) Procédé de traitement d'image et appareil associé
WO2017069505A1 (fr) Procédé de codage/décodage d'image et dispositif correspondant
WO2020076142A1 (fr) Dispositif et procédé de traitement de signal vidéo à l'aide d'un modèle linéaire inter-composants
WO2017150823A1 (fr) Procédé d'encodage/décodage de signal vidéo, et appareil associé
WO2017065422A1 (fr) Procédé permettant de traiter une image et appareil s'y rapportant
WO2016200235A1 (fr) Procédé de traitement d'image basé sur un mode de prédiction intra et appareil associé
WO2017030270A1 (fr) Procédé de traitement d'image basé sur un mode d'intraprédiction, et appareil associé
WO2021162494A1 (fr) Procédé et dispositif de codage/décodage d'images permettant de signaler de manière sélective des informations de disponibilité de filtre, et procédé de transmission de flux binaire
WO2023055220A1 (fr) Procédé de traitement de signal vidéo pour déterminer un mode de prédiction intra sur la base d'une image de référence, et dispositif associé
WO2023140635A1 (fr) Procédé de traitement de signal vidéo faisant appel à un bloc hors limite, et appareil associé
WO2024010430A1 (fr) Procédé de traitement de signal vidéo à l'aide d'un modèle linéaire, et dispositif associé
WO2023033617A1 (fr) Procédé de traitement de signal vidéo au moyen d'un mode de compensation d'éclairage local (lic), et appareil associé

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22876991

Country of ref document: EP

Kind code of ref document: A1