WO2024053987A1

WO2024053987A1 - Video signal processing method using geometric division, and device therefor

Info

Publication number: WO2024053987A1
Application number: PCT/KR2023/013228
Authority: WO
Inventors: 김경용; 김동철; 손주형; 곽진삼
Original assignee: 주식회사 윌러스표준기술연구소
Priority date: 2022-09-06
Filing date: 2023-09-05
Publication date: 2024-03-14

Abstract

A video signal decoding device is disclosed. The decoding device comprises a processor. The processor divides a current block into multiple areas according to a reference line, and performs prediction for each divided area. At least one of the multiple areas is predicted through intra prediction.

Description

Video signal processing method using geometric division and device therefor

The present invention relates to a method and device for processing video signals, and more particularly, to a method and device for processing video signals for encoding or decoding video signals.

Compression encoding refers to a series of signal processing technologies for transmitting digitized information through communication lines or storing it in a form suitable for storage media. Targets of compression coding include audio, video, and text. In particular, the technology for performing compression coding on video is called video image compression. Compressive coding for video signals is accomplished by removing redundant information by considering spatial correlation, temporal correlation, and probabilistic correlation. However, due to recent developments in various media and data transmission media, more highly efficient video signal processing methods and devices are required.

The purpose of this specification is to increase the coding efficiency of video signals by providing a video signal processing method and apparatus for the same.

A video signal decoding device according to an embodiment of the present invention includes a processor,

The processor divides the current block into a plurality of regions according to the baseline and performs prediction for each divided region. At this time, at least one of the plurality of regions is predicted through intra prediction.

When generating a candidate list including candidates for intra prediction directional mode information used for prediction of at least one region predicted through intra prediction, the processor selects candidates for each combination of the type of segmentation and the intra prediction directional mode. can be created.

The processor may exclude from the candidate list a candidate whose difference between the angle corresponding to the direction corresponding to the division type and the angle indicated by the intra prediction direction mode is less than or equal to a predetermined value.

The processor may exclude from the candidate list a candidate whose difference between the index of the intra prediction directional mode and the index of the intra prediction directional mode corresponding to the direction corresponding to the division type is less than or equal to a predetermined value.

If the angle corresponding to the direction corresponding to the type of division of the first candidate in the candidate list is different from the angle indicated by the intra prediction directional mode of the first candidate, the processor determines the division of the first candidate. The angle corresponding to the direction corresponding to the shape can be set to the angle indicated by the intra prediction directionality mode.

The processor determines that the difference between the index of the intra prediction direction mode of the neighboring block adjacent to the current block and the index of the intra prediction direction mode used for the first candidate in the candidate list is within a predetermined value, and the neighboring block adjacent to the current block If the difference between the index of the intra prediction direction mode and the index of the intra prediction direction mode used for the second candidate in the candidate list is equal to or greater than the predetermined value, the priority of the first candidate is higher than the second candidate. It can be specified as a high priority.

The processor may determine a weight used when calculating the cost of the candidate based on the difference between the index of the intra prediction direction mode of a neighboring block adjacent to the current block and the index of the intra prediction direction mode used for the candidate in the candidate list. there is.

If the difference between the intra prediction direction mode of the adjacent neighboring block and the index of the intra prediction direction mode used for the candidate is greater than a predetermined value, the processor adds a cost of the candidate calculated by SAD or MR-SAD greater than 1. The cost can be recalculated by multiplying the weight.

The bitstream including the video signal may include syntax indicating whether it is encoded in either GPM mode or ISP mode. At this time, the processor can obtain the syntax from the bitstream and determine whether the current block is encoded in either GPM mode or ISP mode.

The processor may determine the strength of a deblocking filter used to restore the current block based on which one of the plurality of regions was encoded using an intra prediction mode.

An encoding device that encodes a video signal according to an embodiment of the present invention includes a processor. The processor divides the current block into a plurality of regions according to the baseline and performs prediction for each divided region. At this time, at least one of the plurality of regions is predicted through intra prediction.

The processor may include syntax indicating whether the bitstream including the video signal is encoded in either GPM mode or ISP mode.

In a computer-readable non-transitory storage medium storing a bitstream according to an embodiment of the present invention, the bitstream is decoded using a decoding method. The decoding method includes dividing the current block into a plurality of regions according to a reference line; and

It includes the step of performing prediction for each divided region. At this time, at least one of the plurality of regions is predicted through intra prediction.

This specification provides a method for efficiently processing video signals. The effects that can be obtained in this specification are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 is a schematic block diagram of a video signal encoding device according to an embodiment of the present invention.

Figure 2 is a schematic block diagram of a video signal decoding device according to an embodiment of the present invention.

Figure 3 shows an embodiment in which a coding tree unit is divided into coding units within a picture.

Figure 4 shows one embodiment of a method for signaling splitting of quad trees and multi-type trees.

Figures 5 and 6 show the intra prediction method according to an embodiment of the present invention in more detail.

Figure 7 is a diagram showing the positions of neighboring blocks used to generate a motion candidate list in inter prediction.

Figure 8 shows a syntax for signaling by integrating the angle of the baseline and the distance from the baseline and the center position in GPM mode according to an embodiment of the present invention, and specific information signaled by the syntax.

Figure 9 shows a baseline according to the value of the merge GPM split index in an embodiment of the present invention and shows syntax indicating information about the GPM mode.

Figure 10 shows a method of deriving a GPM segmentation type and intra prediction mode based on a template according to an embodiment of the present invention.

Figure 11 shows a method of deriving GPM and intra prediction directional mode based on a template according to an embodiment of the present invention.

Figure 12 shows the angle of the dividing reference line used in GPM mode according to an embodiment of the present invention.

Figure 13 shows the syntax structure of a method for signaling by integrating GPM mode and ISP mode according to an embodiment of the present invention.

Figure 14 shows the division form of GPM mode in which the current block is divided into two or more areas according to an embodiment of the present invention.

Figure 15 shows the division form of the current block to which GPM mode is applied according to an embodiment of the present invention.

Figure 16 shows the types of transform kernels that can be used for video coding according to an embodiment of the present invention.

Figure 17 shows how MTS and LFNST are applied to the error signal of the current block to which GPM mode is applied according to an embodiment of the present invention.

Figure 18 is a table showing the relationship between the directional mode value of the intra prediction mode corresponding to the area of the current block divided in GPM mode according to an embodiment of the present invention and the index of the MTS set determined according to the size of the current block. .

Figure 19 is a table showing the relationship between the values of nTrSet and mts_idx according to an embodiment of the present invention.

Figure 20 is a table showing the relationship between an index indicating the conversion type and the kernel used for conversion according to an embodiment of the present invention.

Figure 21 shows a table showing the relationship between the horizontal and vertical sizes of the current block and a pre-designated standard for resetting the transformation type according to an embodiment of the present invention.

Figure 22 shows a method of performing intra prediction by determining a reference pixel line based on a template according to an embodiment of the present invention.

Figure 23 shows a method of performing prediction using a block vector of a block encoded in IBC mode according to an embodiment of the present invention.

Figure 24 shows RRIBC being performed according to an embodiment of the present invention.

Figure 25 shows RRIBC being performed according to an embodiment of the present invention.

Figure 26 shows an example of a block vector of a block encoded in Intra TMP mode according to an embodiment of the present invention.

Figure 27 shows that the current block is divided into two regions in GPM mode and the divided regions are encoded in IBC mode according to an embodiment of the present invention.

Figure 28 shows that the current block is encoded in IBC-CIIP mode according to an embodiment of the present invention.

Figure 29 shows an example of a block boundary and samples around the boundary during a deblocking filtering process according to an embodiment of the present invention.

The terms used in this specification are general terms that are currently widely used as much as possible while considering the function in the present invention, but this may vary depending on the intention of a person skilled in the art, custom, or the emergence of new technology. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in the description of the relevant invention. Therefore, we would like to clarify that the terms used in this specification should be interpreted based on the actual meaning of the term and the overall content of this specification, not just the name of the term.

In this specification, ‘A and/or B’ may be interpreted as meaning ‘including at least one of A or B.’

Some terms in this specification may be interpreted as follows. Coding can be interpreted as encoding or decoding depending on the case. In this specification, a device that performs encoding (encoding) of a video signal to generate a video signal bitstream is referred to as an encoding device or encoder, and a device that performs decoding (decoding) of a video signal bitstream to restore a video signal is referred to as a decoder. Referred to as a device or decoder. Additionally, in this specification, a video signal processing device is used as a term that includes both an encoder and a decoder. Information is a term that includes values, parameters, coefficients, elements, etc., and the meaning may be interpreted differently depending on the case, so the present invention is not limited thereto. 'Unit' is used to refer to a basic unit of image processing or a specific location of a picture, and refers to an image area containing at least one of a luminance (luma) component and a chrominance (chroma) component. Additionally, 'block' refers to an image area containing specific components among the luminance component and chrominance component (i.e., Cb and Cr). However, depending on the embodiment, terms such as 'unit', 'block', 'partition', 'signal', and 'area' may be used interchangeably. Additionally, in this specification, 'current block' refers to a block currently scheduled to be encoded, and 'reference block' refers to a block for which encoding or decoding has already been completed and is used as a reference in the current block. Additionally, in this specification, terms such as 'luma', 'luma', 'luminance', and 'Y' may be used interchangeably. In addition, in this specification, terms such as 'chroma', 'chroma', 'color difference', and 'Cb or Cr' may be used interchangeably, and since the color difference component is divided into two types, Cb and Cr, each color difference component will be used separately. You can. Additionally, in this specification, a unit may be used as a concept that includes all coding units, prediction units, and transformation units. A picture refers to a field or frame, and depending on the embodiment, the above terms may be used interchangeably. Specifically, when the captured image is an interlaced image, one frame is divided into an odd (or odd, top) field and an even (or even, bottom) field, and each field is created as a picture unit. and can be encoded or decoded. If the captured image is a progressive image, one frame may be created as a picture and encoded or decoded. Additionally, in this specification, terms such as 'error signal', 'residual signal', 'residual signal', 'residual signal', and 'difference signal' may be used interchangeably. Additionally, in this specification, terms such as 'intra prediction mode', 'intra prediction directional mode', 'intra-screen prediction mode', and 'intra-screen prediction directional mode' may be used interchangeably. Additionally, in this specification, terms such as 'motion' and 'movement' may be used interchangeably. In addition, in this specification, 'left', 'upper left', 'upper', 'upper right', 'right', 'lower right', 'bottom', and 'lower left' mean 'left', 'upper left', ' It can be used interchangeably with 'top', 'top right', 'bottom right', 'bottom right', 'bottom', and 'bottom left'. Additionally, element and member can be used interchangeably. POC (Picture Order Count) represents temporal location information of a picture (or frame), can be the playback order displayed on the screen, and each picture can have a unique POC.

Figure 1 is a schematic block diagram of a video signal encoding device 100 according to an embodiment of the present invention. Referring to Figure 1, the encoding device 100 of the present invention includes a transform unit 110, a quantization unit 115, an inverse quantization unit 120, an inverse transform unit 125, a filtering unit 130, and a prediction unit 150. ) and an entropy coding unit 160.

The conversion unit 110 obtains a conversion coefficient value by converting the residual signal, which is the difference between the input video signal and the prediction signal generated by the prediction unit 150. For example, Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), or Wavelet Transform may be used. Discrete cosine transform and discrete sine transform perform transformation by dividing the input picture signal into blocks. In transformation, coding efficiency may vary depending on the distribution and characteristics of values within the transformation area. The transformation kernel used for transformation of the residual block may be a transformation kernel with separable characteristics of vertical transformation and horizontal transformation. In this case, transformation for the residual block can be performed separately into vertical transformation and horizontal transformation. For example, the encoder can perform vertical transformation by applying a transformation kernel in the vertical direction of the residual block. Additionally, the encoder can perform horizontal transformation by applying a transformation kernel in the horizontal direction of the residual block. In this disclosure, a transform kernel may be used as a term to refer to a set of parameters used for transforming a residual signal, such as a transform matrix, transform array, transform function, or transform. For example, the transformation kernel may be any one of a plurality of available kernels. Additionally, transformation kernels based on different transformation types may be used for each of vertical transformation and horizontal transformation.

Higher conversion coefficients are distributed toward the top left of the block, and coefficients closer to '0' are distributed toward the bottom right of the block. As the size of the current block increases, there is a possibility that there will be more coefficients of '0' in the lower right area. In order to reduce the conversion complexity of large blocks, only the upper left area can be left and the remaining areas can be reset to '0'.

Additionally, error signals may exist only in some areas of the coding block. In this case, the conversion process may be performed only for some arbitrary areas. As an example, in a block of size 2Nx2N, an error signal may exist only in the first 2NxN block, and a conversion process is performed only on the first 2NxN block, but the conversion process is not performed on the second 2NxN block and may not be encoded or decoded. Here N can be any positive integer.

The encoder may perform additional transformations before the transform coefficients are quantized. The above-described transformation method may be referred to as a primary transform, and additional transformation may be referred to as a secondary transform. Secondary transformation may be optional for each residual block. According to one embodiment, the encoder may improve coding efficiency by performing secondary transformation on a region where it is difficult to concentrate energy in the low-frequency region only through primary transformation. For example, secondary transformation may be additionally performed on a block whose residual values appear large in directions other than the horizontal or vertical direction of the residual block. Unlike primary transformation, secondary transformation may not be performed separately into vertical transformation and horizontal transformation. This secondary transform may be referred to as Low Frequency Non-Separable Transform (LFNST).

The quantization unit 115 quantizes the transform coefficient value output from the transform unit 110.

In order to increase coding efficiency, rather than coding the picture signal as is, the picture is predicted using the already coded area through the prediction unit 150, and the residual value between the original picture and the predicted picture is added to the predicted picture to create a reconstructed picture. A method of obtaining is used. To prevent mismatches between the encoder and decoder, information available in the decoder must be used when performing prediction in the encoder. For this purpose, the encoder performs a process of restoring the current encoded block. The inverse quantization unit 120 inversely quantizes the transform coefficient value, and the inverse transform unit 125 restores the residual value using the inverse quantized transform coefficient value. Meanwhile, the filtering unit 130 performs a filtering operation to improve the quality of the reconstructed picture and improve coding efficiency. For example, deblocking filters, sample adaptive offset (SAO), and adaptive loop filters may be included. The filtered picture is output or stored in a decoded picture buffer (DPB, 156) to be used as a reference picture.

A deblocking filter is a filter for removing distortion within blocks created at the boundaries between blocks in a restored picture. The encoder can determine whether to apply a deblocking filter to the edge based on the distribution of pixels included in several columns or rows based on an arbitrary edge within the block. When a deblocking filter is applied to a block, the encoder can apply a long filter, strong filter, or weak filter depending on the deblocking filtering strength. Additionally, horizontal filtering and vertical filtering can be processed in parallel. Sample adaptive offset (SAO) can be used to correct the offset from the original image on a pixel basis for a residual block to which a deblocking filter has been applied. In order to correct the offset for a specific picture, the encoder divides the pixels included in the image into a certain number of areas, determines the area to perform offset correction, and uses a method (Band Offset) to apply the offset to the area. You can. Alternatively, the encoder can use a method of applying an offset (Edge Offset) by considering the edge information of each pixel. Adaptive Loop Filter (ALF) is a method of dividing pixels included in an image into predetermined groups, then determining one filter to be applied to the group, and performing differential filtering for each group. Information related to whether to apply ALF may be signaled in units of coding units, and the shape and filter coefficients of the ALF filter to be applied may vary for each block. Additionally, an ALF filter of the same type (fixed type) may be applied regardless of the characteristics of the target block to be applied.

The prediction unit 150 includes an intra prediction unit 152 and an inter prediction unit 154. The intra prediction unit 152 performs intra prediction within the current picture, and the inter prediction unit 154 performs inter prediction using the reference picture stored in the decoded picture buffer 156. Perform. The intra prediction unit 152 performs intra prediction from the reconstructed areas in the current picture and transmits intra encoding information to the entropy coding unit 160. Intra encoding information may include at least one of an intra prediction mode, a Most Probable Mode (MPM) flag, an MPM index, and information about a reference sample. The inter prediction unit 154 may be generated including a motion estimation unit 154a and a motion compensation unit 154b. The motion estimation unit 154a refers to a specific region of the reconstructed reference picture, finds the part most similar to the current region, and obtains a motion vector value that is the distance between regions. Motion information (reference direction indication information (L0 prediction, L1 prediction, bi-directional prediction), reference picture index, motion vector information, etc.) about the reference area obtained from the motion estimation unit 154a is transmitted to the entropy coding unit 160. so that it can be included in the bitstream. Using the motion information transmitted from the motion estimation unit 154a, the motion compensation unit 154b performs inter-motion compensation to generate a prediction block for the current block. The inter prediction unit 154 transmits inter encoding information including motion information about the reference region to the entropy coding unit 160.

According to an additional embodiment, the prediction unit 150 may include an intra block copy (IBC) prediction unit (not shown). The IBC prediction unit performs IBC prediction from the reconstructed samples in the current picture and transmits IBC encoding information to the entropy coding unit 160. The IBC prediction unit refers to a specific region in the current picture and obtains a block vector value indicating a reference region used for prediction of the current region. The IBC prediction unit may perform IBC prediction using the obtained block vector value. The IBC prediction unit transmits IBC encoding information to the entropy coding unit 160. IBC encoding information may include at least one of reference area size information and block vector information (index information for block vector prediction of the current block within the motion candidate list, block vector difference information).

When the above picture prediction is performed, the transform unit 110 obtains a transform coefficient value by transforming the residual value between the original picture and the predicted picture. At this time, transformation may be performed on a specific block basis within the picture, and the size of the specific block may vary within a preset range. The quantization unit 115 quantizes the transform coefficient value generated by the transform unit 110 and transmits the quantized transform coefficient to the entropy coding unit 160.

The quantized transform coefficients in the form of a two-dimensional array can be rearranged into a one-dimensional array for entropy coding. The scanning method for the quantized transform coefficient may be determined depending on the size of the transform block and the intra-screen prediction mode. As an example, diagonal, vertical, and horizontal scans may be applied. This scan information can be signaled in block units and can be derived according to already established rules.

The entropy coding unit 160 generates a video signal bitstream by entropy coding information representing quantized transform coefficients, intra encoding information, and inter encoding information. The entropy coding unit 160 may use a variable length coding (VLC) method or an arithmetic coding method. The variable length coding (VLC) method converts input symbols into continuous codewords, and the length of the codewords may be variable. For example, frequently occurring symbols are expressed as short codewords, and infrequently occurring symbols are expressed as long codewords. As a variable length coding method, Context-based Adaptive Variable Length Coding (CAVLC) can be used. Arithmetic coding converts consecutive data symbols into a single decimal number using the probability distribution of each data symbol. Arithmetic coding can obtain the optimal decimal bits needed to express each symbol. As arithmetic coding, context-based adaptive binary arithmetic code (CABAC) can be used.

CABAC is a method of binary arithmetic encoding using multiple context models created based on probabilities obtained through experiments. The context model can also be called a context model. First, if the symbols are not in binary form, the encoder binarizes each symbol using exp-Golomb, etc.

Binarized

0 or 1 can be described as a bin. The CABAC initialization process is divided into context initialization and arithmetic coding initialization. Context initialization is a process of initializing the probability of occurrence of each symbol, and is determined depending on the type of symbol, quantization parameter (QP), and slice type (whether I, P, or B). The context model with this initialization information can use probability-based values obtained through experimentation. The context model provides the probability of occurrence of LPS (Least Probable Symbol) or MPS (Most Probable Symbol) for the symbol currently being coded and information (valMPS) about which empty value among 0 and 1 corresponds to the MPS. One of several context models is selected through a context index (ctxIdx), and the context index can be derived through information on the current block to be encoded or information on surrounding blocks. Initialization for binary arithmetic coding is performed based on the probability model selected from the context model. Binary arithmetic coding is divided into probability intervals using the probability of occurrence of 0 and 1, and then coding is carried out through the process where the probability interval corresponding to the bin to be processed becomes the entire probability interval for the next bin to be processed. Location information within the probability interval where the last bin was processed is output. However, since the probability interval cannot be divided indefinitely, when it is reduced to within a certain size, a renormalization process is performed to widen the probability interval and the corresponding location information is output. Additionally, after each bin is processed, a probability update process may be performed in which the probability of the next bin to be processed is newly set through information on the processed bin.

The generated bitstream is encapsulated in a NAL (Network Abstraction Layer) unit as a basic unit. NAL units are divided into VCL (Video Coding Layer) NAL units containing video data and non-VCL NAL units containing parameter information for decoding video data. There are various types of VCL or non-VCL NAL units. . The NAL unit is created with NAL header information and data, RBSP (Raw Byte Sequence Payload), and the NAL header information includes summary information about the RBSP. The RBSP of the VCL NAL unit includes an encoded integer number of coding tree units. In order to decode a bitstream in a video decoder, the bitstream must first be separated into NAL units, and then each separated NAL unit must be decoded. Meanwhile, the information required for decoding the video signal bitstream will be transmitted in a picture parameter set (PPS), sequence parameter set (SPS), video parameter set (VPS), etc. You can.

Meanwhile, the block diagram of FIG. 1 shows the encoding device 100 according to an embodiment of the present invention, and the separately displayed blocks show elements of the encoding device 100 logically distinguished. Accordingly, the elements of the above-described encoding device 100 may be mounted as one chip or as a plurality of chips depending on the design of the device. According to one embodiment, the operation of each element of the above-described encoding device 100 may be performed by a processor (not shown).

Figure 2 is a schematic block diagram of a video signal decoding device 200 according to an embodiment of the present invention. Referring to FIG. 2, the decoding device 200 of the present invention includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 225, a filtering unit 230, and a prediction unit 250.

The entropy decoding unit 210 entropy decodes the video signal bitstream and extracts transform coefficient information, intra encoding information, and inter encoding information for each region. For example, the entropy decoder 210 may obtain a binarization code for transform coefficient information of a specific area from a video signal bitstream. Additionally, the entropy decoding unit 210 inversely binarizes the binarization code to obtain a quantized transform coefficient. The inverse quantization unit 220 inversely quantizes the quantized transform coefficient, and the inverse transform unit 225 restores the residual value using the inverse quantized transform coefficient. The video signal processing device 200 restores the original pixel value by summing the residual value obtained from the inverse transform unit 225 with the predicted value obtained from the prediction unit 250.

Meanwhile, the filtering unit 230 improves image quality by performing filtering on the picture. This may include a deblocking filter to reduce block distortion and/or an adaptive loop filter to remove distortion of the entire picture. The filtered picture is output or stored in the decoded picture buffer (DPB, 256) to be used as a reference picture for the next picture.

The prediction unit 250 includes an intra prediction unit 252 and an inter prediction unit 254. The prediction unit 250 generates a prediction picture using the coding type decoded through the entropy decoding unit 210, transform coefficients for each region, intra/inter coding information, etc. To restore the current block on which decoding is performed, the current picture including the current block or the decoded area of other pictures can be used. Only the current picture is used for reconstruction, that is, a picture (or tile/slice) that performs intra prediction or intra BC prediction is used as an intra picture or I picture (or tile/slice), intra prediction, both inter prediction and intra BC prediction. A picture (or tile/slice) that can be performed is called an inter picture (or tile/slice). To predict sample values of each block among inter pictures (or tiles/slices), a picture (or tile/slice) that uses up to one motion vector and reference picture index is called a predictive picture or P picture (or , tile/slice), and a picture (or tile/slice) using up to two motion vectors and a reference picture index is called a bi-predictive picture or B picture (or tile/slice). In other words, a P picture (or tile/slice) uses at most one set of motion information to predict each block, and a B picture (or tile/slice) uses at most two sets of motion information to predict each block. Use a set. Here, the motion information set includes one or more motion vectors and one reference picture index.

The intra prediction unit 252 generates a prediction block using intra encoding information and reconstructed samples in the current picture. As described above, intra encoding information may include at least one of an intra prediction mode, a Most Probable Mode (MPM) flag, and an MPM index. The intra prediction unit 252 predicts sample values of the current block using reconstructed samples located to the left and/or above the current block as reference samples. In this disclosure, reconstructed samples, reference samples, and samples of the current block may represent pixels. Additionally, sample values may represent pixel values.

According to one embodiment, the reference samples may be samples included in neighboring blocks of the current block. For example, the reference samples may be samples adjacent to the left border and/or samples adjacent to the upper boundary of the current block. In addition, the reference samples are samples of neighboring blocks of the current block, which are located on a line within a preset distance from the left border of the current block and/or are located on a line within a preset distance from the upper border of the current block. These may be samples that do. At this time, the surrounding blocks of the current block are the left (L) block, upper (A) block, Below Left (BL) block, Above Right (AR) block, or Above Left block adjacent to the current block. AL) may include at least one block.

The inter prediction unit 254 generates a prediction block using the reference picture and inter encoding information stored in the decoded picture buffer 256. Inter-encoding information may include a set of motion information (reference picture index, motion vector information, etc.) of the current block with respect to the reference block. Inter prediction may include L0 prediction, L1 prediction, and bi-prediction. L0 prediction refers to prediction using one reference picture included in the L0 picture list, and L1 prediction refers to prediction using one reference picture included in the L1 picture list. This may require one set of motion information (eg, motion vector and reference picture index). In the pair prediction method, a maximum of two reference regions can be used, and these two reference regions may exist in the same reference picture or in different pictures. That is, in the pair prediction method, up to two sets of motion information (e.g., a motion vector and a reference picture index) can be used, and the two motion vectors may correspond to the same reference picture index or may correspond to different reference picture indices. It may be possible to respond. At this time, the reference pictures are pictures located temporally before or after the current picture, and may be pictures that have already been reconstructed. According to one embodiment, the two reference regions used in the bi-prediction method may be regions selected from the L0 picture list and the L1 picture list, respectively.

The inter prediction unit 254 may obtain a reference block of the current block using a motion vector and a reference picture index. The reference block exists in a reference picture corresponding to a reference picture index. Additionally, the sample value of the block specified by the motion vector or its interpolated value may be used as a predictor of the current block. For motion prediction with sub-pel unit pixel accuracy, for example, an 8-tap interpolation filter can be used for the luminance signal and a 4-tap interpolation filter can be used for the chrominance signal. However, the interpolation filter for motion prediction in subpel units is not limited to this. In this way, the inter prediction unit 254 performs motion compensation to predict the texture of the current unit from the previously restored picture. At this time, the inter prediction unit can use a motion information set.

According to an additional embodiment, the prediction unit 250 may include an IBC prediction unit (not shown). The IBC prediction unit can reconstruct the current region by referring to a specific region containing reconstructed samples in the current picture. The IBC prediction unit may perform IBC prediction using the IBC encoding information obtained from the entropy decoding unit 210. IBC encoding information may include block vector information.

The predicted value output from the intra prediction unit 252 or the inter prediction unit 254 and the residual value output from the inverse transform unit 225 are added to generate a restored video picture. That is, the video signal decoding apparatus 200 restores the current block using the prediction block generated by the prediction unit 250 and the residual obtained from the inverse transform unit 225.

Meanwhile, the block diagram of FIG. 2 shows a decoding device 200 according to an embodiment of the present invention, and the separately displayed blocks show elements of the decoding device 200 logically distinguished. Accordingly, the elements of the above-described decoding device 200 may be mounted as one chip or as a plurality of chips depending on the design of the device. According to one embodiment, the operation of each element of the above-described decoding device 200 may be performed by a processor (not shown).

Meanwhile, the technology proposed in this specification is applicable to both encoder and decoder methods and devices, and parts described as signaling and parsing may be described for convenience of explanation. In general, signaling can be described as encoding each syntax from an encoder's perspective, and parsing can be described as interpreting each syntax from a decoder's perspective. That is, each syntax can be signaled by being included in the bitstream from the encoder, and the decoder can parse the syntax and use it in the restoration process. At this time, the sequence of bits for each syntax listed according to the prescribed hierarchical generation can be referred to as a bitstream.

One picture may be divided into sub-pictures, slices, tiles, etc. and encoded. A subpicture may include one or more slices or tiles. If one picture is divided into multiple slices or tiles and encoded, it can be displayed on the screen only when all slices or tiles in the picture have been decoded. On the other hand, when one picture is encoded with several subpictures, only arbitrary subpictures can be decoded and displayed on the screen. A slice may contain multiple tiles or subpictures. Alternatively, a tile may include multiple subpictures or slices. Subpictures, slices, and tiles can be encoded or decoded independently of each other, which is effective in improving parallel processing and processing speed. However, there is a disadvantage in that the bit amount increases because encoded information of other adjacent subpictures, other slices, and other tiles cannot be used. Subpictures, slices, and tiles can be divided into multiple Coding Tree Units (CTUs) and encoded.

Figure 3 shows an embodiment in which a Coding Tree Unit (CTU) is divided into Coding Units (CUs) within a picture. In the process of coding a video signal, a picture can be divided into a sequence of coding tree units (CTUs). A coding tree unit can be created with a luma coding tree block (CTB), two chroma coding tree blocks, and its encoded syntax information. One coding tree unit can be created as one coding unit, or one coding tree unit can be divided into multiple coding units. One coding unit can be created with a luminance coding block (CB), two chrominance coding blocks, and its encoded syntax information. One coding block can be divided into several sub-coding blocks. One coding unit can be created as one transform unit (TU), or one coding unit can be divided into several transform units. One transform unit can be created with a luminance transform block (Transform Block, TB), two chrominance transform blocks, and its encoded syntax information. A coding tree unit may be divided into a plurality of coding units. A coding tree unit may be a leaf node without being split. In this case, the coding tree unit itself may be a coding unit.

A coding unit refers to a basic unit for processing a picture in the video signal processing process described above, that is, intra/inter prediction, transformation, quantization, and/or entropy coding. The size and shape of a coding unit within one picture may not be constant. The coding unit may have a square or rectangular shape. A rectangular coding unit (or rectangular block) includes a vertical coding unit (or vertical block) and a horizontal coding unit (or horizontal block). In this specification, a vertical block is a block whose height is greater than its width, and a horizontal block is a block whose width is greater than its height. Additionally, in this specification, a non-square block may refer to a rectangular block, but the present invention is not limited thereto.

Referring to Figure 3, the coding tree unit is first divided into a quad tree (Quad Tree, QT) structure. That is, in a quad tree structure, one node with a size of 2NX2N can be divided into four nodes with a size of NXN. In this specification, a quad tree may also be referred to as a quaternary tree. Quad-tree partitioning can be performed recursively, and not all nodes need to be partitioned to the same depth.

Meanwhile, the leaf nodes of the aforementioned quad tree can be further divided into a multi-type tree (MTT) structure. According to an embodiment of the present invention, in a multi-type tree structure, one node may be divided into a binary or ternary tree structure with horizontal or vertical division. That is, there are four division structures in the multi-type tree structure: vertical binary division, horizontal binary division, vertical ternary division, and horizontal ternary division. According to an embodiment of the present invention, the width and height of the nodes in each tree structure may both have values that are powers of 2. For example, in a Binary Tree (BT) structure, a node of size 2NX2N may be divided into two NX2N nodes by vertical binary division and into two 2NXN nodes by horizontal binary division. Additionally, in the Ternary Tree (TT) structure, a node of size 2NX2N is divided into nodes of (N/2)X2N, NX2N and (N/2)X2N by vertical ternary division, and horizontal ternary division By division, it can be divided into nodes of 2NX(N/2), 2NXN, and 2NX(N/2). This multi-type tree partitioning can be performed recursively.

Leaf nodes of a multi-type tree can be coding units. If the coding unit is not larger than the maximum transformation length, the coding unit can be used as a unit of prediction and/or transformation without further division. As an example, if the width or height of the current coding unit is greater than the maximum transform length, the current coding unit may be split into a plurality of transform units without explicit signaling regarding splitting. Meanwhile, in the above-described quad tree and multi-type tree, at least one of the following parameters may be defined in advance or transmitted through an RBSP of a higher level set such as PPS, SPS, VPS, etc. 1) CTU size: the root node size of the quad tree, 2) minimum QT size (MinQtSize): minimum allowed QT leaf node size, 3) maximum BT size (MaxBtSize): maximum allowed BT root node size, 4) Maximum TT Size (MaxTtSize): Maximum TT root node size allowed, 5) Maximum MTT Depth (MaxMttDepth): Maximum allowed depth of MTT split from leaf nodes of QT, 6) Minimum BT Size (MinBtSize): Allowed Minimum BT leaf node size, 7) Minimum TT size (MinTtSize): Minimum TT leaf node size allowed.

Figure 4 shows one embodiment of a method for signaling splitting of quad trees and multi-type trees. Preset flags can be used to signal division of the above-described quad tree and multi-type tree. Referring to Figure 4, a flag 'split_cu_flag' indicating whether to split a node, a flag 'split_qt_flag' indicating whether to split a quad tree node, a flag 'mtt_split_cu_vertical_flag' indicating the splitting direction of a multi-type tree node, or a multi-type tree node. At least one of the flags 'mtt_split_cu_binary_flag' that indicates the split shape of the type tree node can be used.

According to an embodiment of the present invention, 'split_cu_flag', a flag indicating whether to split the current node, may be signaled first. If the value of 'split_cu_flag' is 0, it indicates that the current node is not split, and the current node becomes a coding unit. If the current node is a coating tree unit, the coding tree unit includes one undivided coding unit. If the current node is a quad tree node 'QT node', the current node is a leaf node 'QT leaf node' of the quad tree and becomes a coding unit. If the current node is a multi-type tree node 'MTT node', the current node is a leaf node 'MTT leaf node' of the multi-type tree and becomes a coding unit.

If the value of 'split_cu_flag' is 1, the current node can be divided into nodes of a quad tree or multi-type tree according to the value of 'split_qt_flag'. The coding tree unit is the root node of the quad tree and can be first divided into a quad tree structure. In the quad tree structure, 'split_qt_flag' is signaled for each node 'QT node'. If the value of 'split_qt_flag' is 1, the node is split into 4 square nodes, and if the value of 'split_qt_flag' is 0, the node becomes a leaf node 'QT leaf node' of the quad tree, and the node becomes a multi-square node. -Divided into type nodes. According to an embodiment of the present invention, quad tree division may be limited depending on the type of the current node. Quad tree splitting may be allowed if the current node is a coding tree unit (root node of the quot tree) or a quot tree node, and quot tree splitting may not be allowed if the current node is a multi-type tree node. Each quad tree leaf node 'QT leaf node' can be further divided into a multi-type tree structure. As described above, if 'split_qt_flag' is 0, the current node can be split into multi-type nodes. To indicate the split direction and split shape, 'mtt_split_cu_vertical_flag' and 'mtt_split_cu_binary_flag' may be signaled. If the value of 'mtt_split_cu_vertical_flag' is 1, vertical splitting of the node 'MTT node' is indicated, and if the value of 'mtt_split_cu_vertical_flag' is 0, horizontal splitting of the node 'MTT node' is indicated. Additionally, if the value of 'mtt_split_cu_binary_flag' is 1, the node 'MTT node' is divided into two rectangular nodes, and if the value of 'mtt_split_cu_binary_flag' is 0, the node 'MTT node' is divided into three rectangular nodes.

In the tree division structure, the luminance block and the chrominance block can be divided into the same form. That is, the chrominance block can be divided by referring to the division type of the luminance block. If the current chrominance block is smaller than a certain size, the chrominance block may not be divided even if the luminance block is divided.

In the tree division structure, the luminance block and the chrominance block may have different forms. At this time, division information for the luminance block and division information for the chrominance block may be signaled, respectively. Additionally, not only the division information but also the encoding information of the luminance block and the chrominance block may be different. As an example of an embodiment, at least one intra coding mode of a luminance block and a chrominance block, encoding information for motion information, etc. may be different.

A node to be divided into the smallest unit can be processed as one coding block. When the current block is a coding block, the coding block may be divided into several sub-blocks (sub-coding blocks), and the prediction information of each sub-block may be the same or different. As an example embodiment, when the coding unit is an intra mode, the intra prediction mode of each subblock may be the same or different from each other. Additionally, when the coding unit is in inter mode, the motion information of each sub-block may be the same or different. Additionally, each sub-block may be encoded or decoded independently from each other. Each sub-block can be distinguished through a sub-block index (sbIdx). Additionally, when a coding unit is divided into sub-blocks, it may be divided horizontally or vertically or diagonally. In intra mode, the mode in which the current coding unit is divided into 2 or 4 sub-blocks horizontally or vertically is called ISP (Intra Sub Partitions). In inter mode, the mode in which the current coding block is divided diagonally is called GPM (geometric partitioning mode). In GPM mode, the position and direction of the diagonal line are derived using a pre-designated angle table, and the index information of the angle table is signaled.

Picture prediction (motion compensation) for coding is performed on coding units that are no longer divided (i.e., leaf nodes of coding tree units). The basic unit that performs such prediction is hereinafter referred to as a prediction unit or prediction block.

Hereinafter, the term unit used in this specification may be used as a replacement for the prediction unit, which is a basic unit for performing prediction. However, the present invention is not limited to this, and can be understood more broadly as a concept including the coding unit.

Figures 5 and 6 show the intra prediction method according to an embodiment of the present invention in more detail. As described above, the intra prediction unit predicts sample values of the current block using reconstructed samples located to the left and/or above the current block as reference samples.

First, Figure 5 shows an example of reference samples used for prediction of the current block in intra prediction mode. According to one embodiment, the reference samples may be samples adjacent to the left boundary and/or samples adjacent to the upper boundary of the current block. As shown in Figure 5, when the size of the current block is WXH and samples of a single reference line adjacent to the current block are used for intra prediction, a maximum of 2W+2H+1 located to the left and/or above the current block Reference samples can be set using the surrounding samples.

Meanwhile, pixels of multiple reference lines may be used for intra prediction of the current block. Multiple reference lines can be created with n lines located within a preset range from the current block. According to one embodiment, when pixels of multiple reference lines are used for intra prediction, separate index information indicating lines to be set as reference pixels may be signaled, and may be called a reference line index.

Additionally, when at least some samples to be used as reference samples have not yet been reconstructed, the intra prediction unit may obtain reference samples by performing a reference sample padding process. Additionally, the intra prediction unit may perform a reference sample filtering process to reduce the error of intra prediction. That is, filtered reference samples can be obtained by performing filtering on surrounding samples and/or reference samples obtained through a reference sample padding process. The intra prediction unit predicts samples of the current block using the reference samples obtained in this way. The intra prediction unit predicts samples of the current block using unfiltered or filtered reference samples. In this disclosure, peripheral samples may include samples on at least one reference line. For example, neighboring samples may include adjacent samples on a line adjacent to the boundary of the current block.

Next, Figure 6 shows an example of prediction modes used for intra prediction. For intra prediction, intra prediction mode information indicating the intra prediction direction may be signaled. Intra prediction mode information indicates one of a plurality of intra prediction modes that generate an intra prediction mode set. If the current block is an intra prediction block, the decoder receives intra prediction mode information of the current block from the bitstream. The intra prediction unit of the decoder performs intra prediction on the current block based on the extracted intra prediction mode information.

According to an embodiment of the present invention, the intra prediction mode set may include all intra prediction modes used for intra prediction (eg, a total of 67 intra prediction modes). More specifically, the intra prediction mode set may include a planar mode, a DC mode, and multiple (e.g., 65) angular modes (i.e., directional modes). Each intra prediction mode may be indicated through a preset index (i.e., intra prediction mode index). For example, as shown in FIG. 6, intra prediction mode index 0 indicates planar mode, and intra prediction mode index 1 indicates DC mode. Additionally, intra prediction mode indices 2 to 66 may respectively indicate different angle modes. The angle modes each indicate different angles within a preset angle range. For example, the angle mode may indicate an angle within an angle range between 45 degrees and -135 degrees clockwise (i.e., a first angle range). The angle mode can be defined based on the 12 o'clock direction. At this time, intra prediction mode index 2 indicates horizontal diagonal (HDIA) mode, intra prediction mode index 18 indicates horizontal (HORizontal, HOR) mode, and intra prediction mode index 34 indicates diagonal (DIA) mode. The mode is indicated, and intra prediction mode index 50 indicates vertical (VER) mode, and intra prediction mode index 66 indicates vertical diagonal (VDIA) mode.

Meanwhile, the preset angle range may be set differently depending on the shape of the current block. For example, if the current block is a rectangular block, a wide-angle mode indicating an angle exceeding 45 degrees or less than -135 degrees clockwise may be additionally used. If the current block is a horizontal block, the angle mode may indicate an angle within an angle range (i.e., a second angle range) between (45+offset1) degrees and (-135+offset1) degrees in a clockwise direction. At this time, angle modes 67 to 76 outside the first angle range may be additionally used. Additionally, if the current block is a vertical block, the angle mode may indicate an angle within an angle range between (45-offset2) degrees and (-135-offset2) degrees clockwise (i.e., a third angle range). . At this time, angle modes -10 to -1 outside the first angle range may be additionally used. According to an embodiment of the present invention, the values of offset1 and offset2 may be determined differently depending on the ratio between the width and height of the rectangular block. Additionally, offset1 and offset2 can be positive numbers.

According to a further embodiment of the present invention, the plurality of angle modes generating the intra prediction mode set may include a basic angle mode and an extended angle mode. At this time, the extended angle mode may be determined based on the basic angle mode.

According to one embodiment, the basic angle mode corresponds to the angle used in intra prediction of the existing HEVC (High Efficiency Video Coding) standard, and the extended angle mode corresponds to the angle newly added in intra prediction of the next-generation video codec standard. It may be a mode that does this. More specifically, the default angle mode is the intra prediction mode {2, 4, 6,... , 66}, and the extended angle mode is the intra prediction mode {3, 5, 7,... , 65} may be an angle mode corresponding to one of the following. That is, the extended angle mode may be an angle mode between basic angle modes within the first angle range. Accordingly, the angle indicated by the extended angle mode can be determined based on the angle indicated by the basic angle mode.

According to another embodiment, the basic angle mode may be a mode corresponding to an angle within a preset first angle range, and the extended angle mode may be a wide angle mode outside the first angle range. That is, the default angle mode is the intra prediction mode {2, 3, 4, … , 66}, and the extended angle mode is the intra prediction mode {-14, -13, -12,... , -1} and {67, 68, … , 80} may be an angle mode corresponding to one of the following. The angle indicated by the extended angle mode may be determined as the angle opposite to the angle indicated by the corresponding basic angle mode. Accordingly, the angle indicated by the extended angle mode can be determined based on the angle indicated by the basic angle mode. Meanwhile, the number of expansion angle modes is not limited to this, and additional expansion angles may be defined depending on the size and/or shape of the current block. Meanwhile, the total number of intra prediction modes included in the intra prediction mode set may vary depending on the generation of the basic angle mode and extended angle mode described above.

In the above embodiment, the spacing between extended angle modes may be set based on the spacing between corresponding basic angle modes. For example, the extended angle modes {3, 5, 7, … , 65} are the corresponding fundamental angular modes {2, 4, 6, … , 66} can be determined based on the interval between them. Additionally, the extended angle modes {-14, -13,... , -1} are the corresponding opposite fundamental angular modes {53, 53,... , 66} is determined based on the spacing between the extended angle modes {67, 68,... , 80} are the corresponding opposite fundamental angular modes {2, 3, 4, … , 15} can be determined based on the interval between them. The angular spacing between the extended angle modes may be set to be equal to the angular spacing between the corresponding basic angle modes. Additionally, the number of extended angle modes in the intra prediction mode set may be set to less than the number of basic angle modes.

According to an embodiment of the present invention, the extended angle mode may be signaled based on the basic angle mode. For example, a wide angle mode (i.e., extended angle mode) may replace at least one angle mode (i.e., basic angle mode) within the first angle range. The basic angle mode that is replaced may be an angle mode corresponding to the opposite side of the wide angle mode. That is, the basic angle mode that is replaced is an angle mode that corresponds to an angle in the opposite direction of the angle indicated by the wide-angle mode or to an angle that differs from the angle in the opposite direction by a preset offset index. According to an embodiment of the present invention, the preset offset index is 1. The intra-prediction mode index corresponding to the replaced basic angle mode may be remapped to the wide-angle mode to signal the corresponding wide-angle mode. For example, wide angle mode {-14, -13, … , -1} is the intra prediction mode index {52, 53, … , 66}, respectively, and the wide-angle mode {67, 68, … , 80} is the intra prediction mode index {2, 3, … , 15} can be signaled respectively. In this way, the intra prediction mode index for the basic angle mode signals the extended angle mode, so that even if the angle modes used for intra prediction of each block are generated differently, the same set of intra prediction mode indexes are used for signaling of the intra prediction mode. can be used Accordingly, signaling overhead due to changes in intra prediction mode generation can be minimized.

Meanwhile, whether to use the extended angle mode may be determined based on at least one of the shape and size of the current block. According to one embodiment, if the size of the current block is larger than the preset size, the extended angle mode may be used for intra prediction of the current block, otherwise, only the basic angle mode may be used for intra prediction of the current block. According to another embodiment, if the current block is a non-square block, the extended angle mode may be used for intra prediction of the current block, and if the current block is a square block, only the basic angle mode may be used for intra prediction of the current block.

The intra prediction unit determines reference samples and/or interpolated reference samples to be used for intra prediction of the current block, based on intra prediction mode information of the current block. When the intra prediction mode index indicates a specific angle mode, a reference sample or an interpolated reference sample corresponding to the specific angle from the current sample of the current block is used for prediction of the current pixel. Accordingly, different sets of reference samples and/or interpolated reference samples may be used for intra prediction depending on the intra prediction mode. After intra prediction of the current block is performed using reference samples and intra prediction mode information, the decoder restores the sample values of the current block by adding the residual signal of the current block obtained from the inverse transformer to the intra prediction value of the current block. .

Movement (motion) information used for inter prediction may include reference direction indication information (inter_pred_idc), reference picture indices (ref_idx_l0, ref_idx_l1), and motion (motion) vectors (mvL0, mvL1). Reference picture list utilization information (predFlagL0, predFlagL1) may be set according to the reference direction indication information. As an example of an embodiment, in the case of unidirectional prediction using an L0 reference picture, predFlagL0=1 and predFlagL1=0 may be set. In the case of unidirectional prediction using an L1 reference picture, predFlagL0=0 and predFlagL1=1 can be set. In the case of bidirectional prediction using both L0 and L1 reference pictures, predFlagL0=1 and predFlagL1=1 can be set.

When the current block is a coding unit, the coding unit may be divided into several sub-blocks, and the prediction information of each sub-block may be the same or different. As an example embodiment, when the coding unit is an intra mode, the intra prediction mode of each subblock may be the same or different from each other. Additionally, when the coding unit is in inter mode, the motion information of each sub-block may be the same or different. Additionally, each sub-block may be encoded or decoded independently from each other. Each sub-block can be distinguished through a sub-block index (sbIdx).

The motion vector of the current block is likely to be similar to the motion vector of neighboring blocks. Therefore, the motion vector of the neighboring block can be used as a motion vector predictor (mvp), and the motion vector of the current block can be derived using the motion vector of the neighboring block. Additionally, in order to increase the accuracy of the motion vector, the motion vector difference (mvd) between the optimal motion vector of the current block found in the original image by the encoder and the predicted value of the motion information may be signaled.

The motion vector may have various resolutions, and the resolution of the motion vector may vary on a block basis. Motion vector resolution can be expressed in integer units, half-pixel units, 1/4 pixel units, 1/16 pixel units, integer pixel units of 4, etc. Since images such as screen content are in the form of simple graphics such as text, there is no need to apply an interpolation filter, so integer units and integer pixel units of 4 can be selectively applied on a block basis. Blocks encoded in affine mode, which can express rotation and scale, have significant changes in shape, so integer units, 1/4 pixel units, and 1/16 pixel units can be selectively applied on a block basis. Information on whether to selectively apply motion vector resolution on a block basis is signaled with amvr_flag. If applied, which motion vector resolution to apply to the current block is signaled with amvr_precision_idx.

For blocks to which bidirectional prediction is applied, when applying weight average, the weights between two prediction blocks may be the same or different, and information about the weights is signaled through bcw_idx.

To increase the accuracy of the prediction value of motion information, merge or advanced motion vector prediction (AMVP) methods can be selectively used on a block basis. The Merge method is a method that generates the motion information of the current block to be the same as the motion information of neighboring blocks adjacent to the current block. The Merge method has the advantage of increasing the coding efficiency of motion information by spatially propagating motion information without change in a motion region with homogeneity. There is. On the other hand, the AMVP method is a method that predicts motion information in the L0 and L1 prediction directions respectively and signals the most optimal motion information in order to express accurate motion information. The decoder derives motion information for the current block through the AMVP or Merge method and then uses the reference block located in the motion information derived from the reference picture as a prediction block for the current block.

A method of deriving motion information in Merge or AMVP may be to generate a motion candidate list using the predicted value of motion information derived from neighboring blocks of the current block, and then index information for the optimal motion candidate is signaled. . In the case of AMVP, since motion candidate lists are derived for each of L0 and L1, the optimal motion candidate indices (mvp_l0_flag, mvp_l1_flag) for each of L0 and L1 are signaled. In the case of Merge, since one motion candidate list is derived, one merge index (merge_idx) is signaled. The motion candidate list derived from one coding unit may vary, and a motion candidate index or merge index may be signaled for each motion candidate list. At this time, a mode in which there is no information about the remaining blocks in blocks encoded in Merge mode can be called Merge Skip mode.

Bidirectional movement information for the current block can be derived by mixing AMVP and Merge modes. For example, motion information in the L0 direction can be derived using the AMVP method, and motion information in the L1 direction can be derived using the Merge method. Conversely, Merge can be applied to L0 and AMVP to L1. This encoding mode can be called AMVP-merge mode.

SMVD (Symmetric MVD) is a method of reducing the amount of bits of transmitted motion information by ensuring that the MVD (Motion Vector Difference) values in the L0 and L1 directions are symmetrical in the case of bi-directional prediction. MVD information in the L1 direction, which is symmetrical to the L0 direction, is not transmitted, and in addition, reference picture information in the L0 and L1 directions is not transmitted and is derived during the decoding process.

OBMC (Overlapped Block Motion Compensation) generates prediction blocks for the current block using the motion information of neighboring blocks when the motion information between blocks is different, and then weight averages the prediction blocks to create the final prediction block for the current block. How to create it. This has the effect of reducing the blocking phenomenon that occurs at the block boundaries of motion compensated images.

In general, merge movement candidates have low movement accuracy. To increase the accuracy of these merge motion candidates, the MMVD (Merge mode with MVD) method can be used. The MMVD method is a method of correcting motion information using one candidate selected from among several motion difference value candidates. Information about the correction value of motion information obtained through the MMVD method (for example, an index indicating one candidate selected from motion difference value candidates, etc.) may be included in the bitstream and transmitted to the decoder. Compared to including existing motion information difference values in the bitstream, the amount of bits can be saved by including information about the correction value of motion information in the bitstream.

The TM (Template Matching) method is a method of correcting motion information by creating a template using surrounding pixels of the current block and finding a matching area with the highest similarity to the template. Template matching (TM) is a method of performing motion prediction in a decoder without including motion information in the bitstream in order to reduce the size of the encoded bitstream. At this time, since the decoder does not have the original image, it can roughly derive motion information for the current block using already restored neighboring blocks.

The DMVR (Decoder-side Motion Vector Refinement) method is a method of correcting motion information through the correlation of already restored reference images in order to find more accurate motion information. It uses the bidirectional motion information of the current block to compare the two reference pictures. This is a method of using the point with the best matching between reference blocks in a reference picture within a certain area as a new bidirectional movement. When such DMVR is performed, the encoder corrects the motion information by performing DMVR in one block unit, then divides the block into sub-blocks and performs DMVR in each sub-block unit to correct the motion information of the sub-block again. This can be done, and this can be called MP-DMVR (Multi-pass DMVR).

The LIC (Local Illumination Compensation) method is a method of compensating for luminance changes between blocks. It derives a linear model using surrounding pixels adjacent to the current block, and then compensates for the luminance information of the current block through the linear model.

Existing video coding methods perform motion compensation considering only horizontal, vertical, and horizontal movements, so coding efficiency deteriorates when coding videos that include movements such as enlargement, reduction, and rotation that are commonly encountered in reality. To express such movements for enlargement, reduction, and rotation, an Affine model-based motion prediction technology that uses a 4 (rotation) or 6 (enlargement, reduction, rotation) parameter model can be applied.

BDOF (Bi-Directional Optical Flow) is used to correct the prediction block by estimating the amount of pixel change based on optical-flow from the reference block of the block generated by bidirectional movement. The motion of the current block can be corrected using motion information derived from the BDOF of this VVC.

PROF (Prediction refinement with optical flow) is a technology to improve the accuracy of sub-block-level affine motion prediction to be similar to the accuracy of pixel-level motion prediction. PROF, similar to BDOF, is a technology that obtains the final prediction signal by calculating correction values in pixel units for affine motion compensated pixel values in sub-block units based on optical flow.

When CIIP (Combined Inter-/Intra-picture Prediction) generates a prediction block for the current block, the final prediction block is created by weighting the prediction blocks generated by the intra-picture prediction method and the prediction blocks generated by the inter-picture prediction method. How to create it.

The IBC (Intra Block Copy) method is a method that finds the part most similar to the current block in an already reconstructed area in the current picture and uses the corresponding reference block as a prediction block for the current block. At this time, information related to the block vector, which is the distance between the current block and the reference block, may be included in the bitstream. The decoder can calculate or set the block vector for the current block by parsing information related to the block vector contained in Beaststream.

The BCW (Bi-prediction with CU-level Weights) method does not generate a prediction block by averaging two prediction blocks that have been motion-compensated from different reference pictures, but applies weights adaptively on a block-by-block basis to compensate for motion. This is a method of performing a weighted average on two prediction blocks.

The MHP (Multi-hypothesis prediction) method is a method of performing weight prediction using various prediction signals by transmitting additional motion information to unidirectional and bidirectional motion information when predicting between screens.

CCLM (Cross-component linear model) is a method of creating a linear model using the high correlation between a luminance signal and a chrominance signal located at the same location as the luminance signal, and then predicting the chrominance signal through the linear model. After creating a template using a restored block among neighboring blocks adjacent to the current block, parameters for the linear model are derived through the template. Next, depending on the video format, the restored current luminance block is selectively down-sampled to fit the size of the chrominance block. Finally, the chrominance block of the current block is predicted using the down-sampled luminance block and the corresponding linear model. At this time, the method of using two or more linear models is called MMLM (Multi-model Linear mode).

In independent scalar quantization, the restored coefficients t' _k for input coefficients t _k depend only on the associated quantization index q _k . That is, the quantization index for any restored coefficient has a different value from the quantization indexes for other restored coefficients. At this time, t' _k may be a value including the quantization error at t _k and may be different or the same depending on the quantization parameter. Here, _t'k may be named a restored transform coefficient or a dequantized transform coefficient, and the quantization index may be named a quantized transform coefficient.

In Uniform Reconstruction Quantizers (URQ), the reconstructed coefficients have the characteristic of being arranged at equal intervals. At this time, the distance between two adjacent restored values can be referred to as the quantization step size. The restored values may include 0, and the entire set of available restored values may be uniquely defined depending on the quantization step size. The quantization step size may vary depending on the quantization parameter.

In the existing method, the set of allowable restored transform coefficients decreases due to quantization, and the number of elements of this set may be finite. Because of this, there is a limit to minimizing the average error between the original image and the restored image. Vector quantization can be used as a method to minimize this average error.

A simple form of vector quantization method used in video encoding is sign data hiding. This is a method in which the encoder does not encode the sign of one non-zero coefficient, and the decoder determines the sign for the corresponding coefficient depending on whether the sum of the absolute values of all coefficients is even or odd. To this end, at least one coefficient may be increased or decreased by '1' in the encoder, and at least one coefficient is selected to be optimal in terms of cost for rate-distortion, so that the value is It can be adjusted. As an example embodiment, a coefficient having a value close to the boundary of the quantization interval may be selected.

Another vector quantization method is Trellis-Coded Quantization, and in video coding, it is used as an optimal path search technique to obtain an optimized quantization value in dependent quantization. On a block basis, quantization candidates for all coefficients within a block are placed in a trellis graph, and the optimal trellis path between optimized quantization candidates is determined considering the cost of rate-distortion. and explore. Specifically, dependent quantization applied to video encoding may be designed such that the set of allowable restored transform coefficients for a transform coefficient depends on the value of the transform coefficient that precedes the current transform coefficient in reconstruction order. At this time, by selectively using multiple quantizers according to the transformation coefficient, the average error between the original image and the restored image is minimized, thereby increasing coding efficiency.

Among intra prediction coding technologies, the MIP (Matrix Intra Prediction) method is a matrix-based intra prediction method. Unlike prediction methods that have directionality from pixels of neighboring blocks adjacent to the current block, pixels on the left and top of neighboring blocks are used as a predefined matrix. This is a method of obtaining a prediction signal using the and offset values.

In order to derive the intra prediction mode of the current block, based on a template, which is a random area restored while adjacent to the current block, the intra prediction mode for the template derived through the surrounding pixels of the template is used to restore the current block. It can be used for. First, the decoder can generate a prediction template for the template using surrounding pixels (references) adjacent to the template, and use the intra prediction mode, which generates a prediction template most similar to the already restored template, to restore the current block. This method can be called TIMD (Template intra mode derivation).

In general, an encoder can determine a prediction mode for generating a prediction block and generate a bitstream containing information about the determined prediction mode. The decoder can set the intra prediction mode by parsing the received bitstream. At this time, the bit amount of information about the prediction mode may be about 10% of the total bitstream size. In order to reduce the bit amount of information about the prediction mode, the encoder may not include information about the intra prediction mode in the bitstream. Accordingly, the decoder can derive (determine) an intra prediction mode for restoration of the current block using the characteristics of the surrounding blocks, and can restore the current block using the derived intra prediction mode. At this time, the decoder infers directionality information by applying a Sobel filter horizontally and vertically to each surrounding pixel adjacent to the current block to derive the intra prediction mode, and then converts the directionality information into the intra prediction mode. A mapping method can be used. The method by which the decoder derives an intra prediction mode using neighboring blocks can be described as DIMD (Decoder side intra mode derivation).

A block predicted using the intra prediction directional mode may have discontinuous edges at the top and left boundaries of the block. For example, if the current block is predicted using the vertical direction mode, a discontinuous edge may appear at the left border of the block. To alleviate this discontinuity, the encoder and decoder can apply filtering to samples at the boundaries within the prediction block. Filtering is performed on restored samples adjacent to the current prediction block, location information of restored samples adjacent to the current prediction block, samples at the boundary portion inside the current prediction block, location information of samples at the boundary portion inside the current prediction block, and location information of the samples at the boundary portion inside the current prediction block. Whether to apply filtering and/or filtering weights may be determined using at least one of the intra prediction directional mode, the horizontal size, and the vertical size of the current prediction block. The filtering weight refers to the weight for the sample in the boundary part inside the current prediction block and the weight for the restored sample adjacent to the current prediction block. This filtering method can be called PDPC (Position dependent intra prediction combination).

Surrounding blocks may be blocks in a spatial location or blocks in a temporal location. Surrounding blocks that are spatially adjacent to the current block are Left (A1) block, Left Below (A0) block, Above (B1) block, Above Right (B0) block, or Above Left. , B2) It can be at least one of the blocks. The neighboring block temporally adjacent to the current block may be a block containing the upper left pixel position of the bottom right (BR) block of the current block in the corresponding picture (Collocated picture). If a neighboring block temporally adjacent to the current block is encoded in intra mode or a neighboring block temporally adjacent to the current block is located in an unusable position, the picture corresponding to the current picture (collocated picture) is displayed. A block containing the horizontal and vertical center (Center, Ctr) pixel position of the block can be used as a temporal peripheral block. Motion candidate information derived from the corresponding picture may be referred to as TMVP (Temporal Motion Vector Predictor). Only one TMVP can be derived from one block, and after dividing one block into several sub-blocks, each TMVP candidate can be derived for each sub-block. The TMVP derivation method on a sub-block basis may be referred to as sbTMVP (sub-block Temporal Motion Vector Predictor).

Whether the methods described herein will be applied depends on slice type information (e.g., whether it is an I slice, a P slice, or a B slice), whether it is a tile, whether it is a subpicture, the size of the current block, the depth of the coding unit, and the current block. It may be determined based on at least one of information about whether it is a luminance block or a chrominance block, whether it is a reference frame or a non-reference frame, reference order, and temporal hierarchy according to the hierarchy. Information used to determine whether the methods described in this specification will be applied may be information previously agreed upon between the decoder and the encoder. Additionally, this information may be determined according to profile and level. This information can be expressed as variable values, and the bitstream can include information about variable values. That is, the decoder can determine whether the above-described methods are applied by parsing information about variable values included in the bitstream. For example, it may be determined whether the above-described methods will be applied based on the horizontal or vertical length of the coding unit. If the horizontal or vertical length is 32 or more (e.g., 32, 64, 128, etc.), the above-described methods can be applied. Additionally, the above-described methods can be applied when the horizontal or vertical length is less than 32 (e.g., 2, 4, 8, 16). Additionally, the above-described methods can be applied when the horizontal or vertical length is 4 or 8.

As previously described, the GPM mode is a prediction mode in which the current block is divided into a plurality of regions, for example, two regions, according to a baseline, and prediction is performed for each region. At this time, each divided area may be one of triangles, squares, or pentagons. The encoder may include in the bitstream a syntax element (merge_gpm_partition_idx) that indicates which angle the baseline used in GPM mode corresponds to and how far the baseline is from the center of the current block. For convenience of explanation, this syntax element is referred to as the merge GPM partition index. The decoder can obtain a merge GPM partition index from the bitstream and determine the type of area into which the current block is divided in GMP mode according to the merge GPM partition index. Specifically, the decoder can determine the angle of the baseline dividing the current block into two regions and the distance from the baseline to the center of the current block based on the merge GPM division index. The merge GPM split index may indicate the angle of the baseline and the distance from the baseline to the center of the current block according to a pre-specified table. In a specific embodiment, the merge GPM split index may have a value of 0 to 63. The pre-designated table may be as shown in FIG. 8.

Figures 9 (a) and (b) show a baseline according to the value of the merge GPM division index described in Figure 8 and show syntax indicating information about the GPM mode. When a region divided in GPM mode is encoded in inter-prediction mode, the bitstream contains not only the merge GPM division index but also motion information corresponding to the region encoded in inter-prediction mode, for example, indicating one motion vector in the motion candidate list. May contain information. In Figure 9(c), merge_gpm_idx0 is an index indicating a motion vector of the first area in the motion candidate list, and merge_gpm_idx1 is an index indicating a motion vector of the second area in the motion candidate list.

Each region divided in GPM mode may be encoded in intra prediction mode or inter prediction mode. The encoder may include information related to the prediction directionality mode of the region encoded in intra prediction mode in the bitstream. Information related to the prediction directionality mode includes information about whether the region is encoded in DIMD mode, information about whether the region is coded in TIMD mode, information about whether the region is coded in MIP mode, and which reference pixel line the region is associated with. Information on whether the area was encoded using ISP mode, information on whether the area was encoded using ISP mode, information on what type of partition ISP mode was used in the area, information on the MPM list used for encoding of the area. It may include at least one of information and information about which intra prediction directional mode the corresponding area used. The decoder may obtain information related to the prediction directionality mode of the region encoded in intra prediction mode from the bitstream and generate a prediction block based on the information related to the prediction directionality mode.

The encoder may include motion information of a region encoded in inter prediction mode, for example, information related to a motion vector, in the bitstream. Information related to motion information includes information about whether the region was coded using TM, information about whether the region was coded using AMVP or Merge mode, information about whether the region was coded using IBC mode, information about whether the region was coded using IBC mode, Information about whether the region was coded using Intra TMP mode, information about whether the region was coded using affine mode, information about whether the region was coded using SMVD mode, encoding of the region Reference direction indication information used, information about which reference picture the corresponding area was encoded using, information related to the difference value of the motion information used in encoding the corresponding area, information related to the AMVR used in encoding the corresponding area , at least one of MMVD-related information used in encoding of the corresponding area, index information indicating which motion candidate was used among the motion candidate list used in encoding of the corresponding area, and information related to BCW used in encoding of the corresponding area. More may be included. The decoder may obtain information related to motion information of a region encoded in inter prediction mode from the bitstream and generate a prediction block based on the information related to the motion information.

In GPM mode, individual areas can be divided into 2 or 4 areas and encoded by applying ISP mode. The bitstream may include information indicating in which direction the area using the ISP mode was divided and encoded. The decoder can determine which direction it is divided in ISP mode based on the information. In the previously described embodiments, when the corresponding area is divided into a square shape in GMP mode, ISP mode can be used to encode the corresponding area.

If the current block is in GPM mode that is not divided into squares, the encoder may not include information related to the ISP mode in the bitstream. The decoder may not parse information related to ISP mode in GPM mode that is not segmented into squares. Additionally, for an area in GPM mode that is not divided into squares, the encoder can infer that the area has not been encoded using ISP mode.

When the encoder encodes both regions divided in GPM mode in intra-prediction mode, information related to the intra-prediction directionality mode and information related to the segmentation shape for each region are signaled respectively or encoded through a template-based derivation method. You can. Figure 10(a) shows a method of signaling encoding information for each region. The encoder may include information related to the division type (partition_mode) and information related to the intra prediction directionality mode of each region (intra_pred_mode0, intra_pred_mode1) in the bitstream. The decoder can parse information related to the split type and information related to the intra prediction directionality mode from the bitstream and then use the information to generate a prediction block for the current block. FIG. 10(b) shows a method of deriving information related to the intra prediction directional mode and information related to the segmentation type for each region based on a template, and details are explained in FIG. 11.

In order to reduce the amount of bits required for signaling, a template-based prediction method can be used in the encoder and decoder, as shown in (b) of FIG. 10. The template-based prediction method performs prediction using information related to the segmentation type and intra prediction direction mode for each region of the current block using already restored surrounding pixels of the current block. First, the encoder and decoder construct a reference template using already restored surrounding pixels of the current block. The standard template may consist of the left template of the current block and the top template of the current block. The size of the left template may be M x the height of the current block, and the size of the top template may be M x the width of the current block. At this time, M may be an integer of 1 or more, for example, 4. Next, the encoder and decoder construct a GPM split prediction candidate list using the 64 split types available for the current block and intra prediction direction information for each region. At this time, each candidate in the GPM partition prediction candidate list contains information about the partition type of the current block (e.g., merge_gpm_partition_idx in FIG. 8) and information related to the intra prediction directional mode for the two regions (e.g., in FIG. 10) It may include intra_pred_mode0 and intra_pred_mode1) in (a). Therefore, when the encoder and decoder generate a GPM segmentation prediction candidate list for prediction of a region predicted through intra prediction, they can generate candidates for each combination of the type of segmentation and the intra prediction directionality mode. Next, the encoder and decoder construct a prediction template by predicting the reference template using information on all candidates in the GPM segmentation prediction candidate list and using surrounding pixels of the reference template. The similarity between the reference template and the predicted template is calculated and the cost value is output. At this time, the cost value can be calculated using SAD (Sum of Absolute Differences) or MR-SAD (Mean-Removed SAD). The decoder and encoder may calculate cost values for all candidates in the GPM split prediction candidate list and then rearrange the GPM split prediction candidate list in ascending order based on the cost value. The encoder may include the index of the candidate most suitable for the current block among the reordered GPM segmentation prediction candidate list in the bitstream. Alternatively, the candidate with the lowest cost value in the reordered GPM segmentation prediction candidate list may be used for encoding. Through this, information indicating any one candidate from the GPM split prediction candidate list may not be indicated.

The decoder parses the optimal candidate index information for the current block from the bitstream, and predicts the candidate's prediction information indicated by the optimal candidate index from the reordered GPM segmentation prediction candidate list generated in the same way as the encoder, such as the division form of the current block. The current block can be predicted using information related to merge_gpm_partition_idx of FIG. 8 and information related to the intra prediction directionality mode for the two regions, for example, intra_pred_mode0 and intra_pred_mode1 of (a) of FIG. 10. In another specific embodiment, the decoder may construct a reordered GPM split prediction candidate list and then use the candidate with the lowest cost value as the optimal candidate to predict the current block.

The number of candidates that can be included in the GPM split prediction candidate list is a combination of 64 GPM split types and 67 intra prediction directionality modes for each region, so the total number of candidates is 64x67x66 (assuming that the intra prediction direction modes are different between regions), or about 280,000. Candidates may exist. To reduce complexity, costs may be calculated for only some candidates. The encoder and decoder determine the size of the current block, the horizontal or vertical size of the current block, the number of pixels in the current block, whether the current block is a luminance signal or a chrominance signal, the intra prediction directionality mode of neighboring blocks adjacent to the current block, Some candidates may be selected based on at least one of the MPM list for the current block, the intra prediction direction mode derived from DIMD or TIMD, and the GPM division method of neighboring blocks adjacent to the current block. In a specific embodiment, the encoder and decoder do not construct the GPM segmentation prediction candidate list using all intra-prediction directionality modes, but only use the intra-prediction directionality mode in the MPM list for the current block to construct the GPM segmentation prediction candidate list. . In another specific embodiment, the encoder and decoder may construct a GPM segmentation prediction candidate list using at least one of the intra prediction directional modes derived from DIMD or TIMD. When the encoder and decoder derive an intra prediction mode from a neighboring block, if the coding mode of the neighboring block is encoded in TIMD mode or TMRL mode, the range of intra prediction modes of the neighboring block is expanded from the existing 67 to 131. mode is used. At this time, the encoder and decoder can reduce the precision of the intra prediction mode to the existing 67 modes and then construct a GPM split prediction candidate list using the changed intra prediction mode. In another specific embodiment, the encoder and decoder may construct a GPM segmentation prediction candidate list by using the extended range of intra prediction modes without change. Details about TMRL will be described later.

In another specific embodiment, the encoder and decoder may not construct the GPM segmentation prediction candidate list using all segmentation types, but may construct the GPM segmentation prediction candidate list using only some segmentation types. Specifically, the encoder and decoder may not include a GPM segmentation in which the number of pixels in any segmented area is within a predetermined number in the GPM segmentation prediction candidate list. At this time, the predetermined number may be an integer of 1 or more. Specifically, it may be 10. In another specific embodiment, the encoder and decoder provide an absolute value of the x-axis difference value (ex-sx) and a y-axis difference value between the starting position (sx, sy) and the ending position (ex, ey) of the baseline used for segmentation. If the sum of the absolute values of (ey-sy) is within a pre-specified value or a GPM split whose baseline length is within a pre-specified value may not be included in the GPM split prediction candidate list. At this time, the pre-designated value may be an integer of 1 or more. Specifically, the pre-designated value may be 3.

When increasing the number of candidates in the GPM segmentation prediction candidate list, coding efficiency is high, but complexity may also increase. In order to reduce the increase in complexity, the encoder can categorize the characteristics of the GPM split prediction candidate list for the current block into several characteristics and then include information related to the classification in the bitstream. After parsing classification-related information from the bitstream, the decoder may construct a GPM split prediction candidate list for the current block based on the classification-related information. At this time, the criteria for classifying the characteristics of the GPM segmented prediction candidate list may include at least one of the complexity between pixels of two regions divided by GPM or the intra prediction directionality mode. At this time, complexity can be calculated based on the absolute sum between pixels in the area. If the absolute sum is greater than a pre-specified value, the encoder and decoder may determine that the complexity of the corresponding area is large. Additionally, if the absolute sum is equal to or smaller than a pre-specified value, the encoder and decoder may determine that the complexity of the corresponding area is large. The pre-specified value may be determined based on the number of pixels in the current block. Additionally, the pre-designated value may be an integer of 1 or more. In another specific embodiment, complexity may be calculated based on whether the predicted directional mode is an angular mode, an undirectional mode, or a MIP mode. At this time, the non-directional mode may include at least one of DC mode and planar mode.

Through the embodiments described above, the allowable intra prediction directionality mode may vary for each group classified. Additionally, the division type allowed in GPM mode may vary for each classified group. Specifically, regions included in low complexity groups may not be allowed to be encoded using the intra prediction directional mode. At this time, the intra prediction directional mode may include at least one of DIMD mode, TIMD mode, and angle mode. Additionally, the decoder may exclude the intra prediction directional mode from the GPM segmentation prediction candidate list for the region included in the low complexity group. Additionally, areas included in low complexity groups may not be allowed to be encoded using intra MIP mode. Additionally, the decoder may exclude the MIP mode from the GPM segmentation prediction candidate list for the region included in the low complexity group. For example, two areas of the current block to which GPM mode is applied may be determined to have low complexity, and a syntax indicating this determination may be included in the bitstream. The decoder can parse the syntax from the non-stream indicating that the two regions of the current block are judged to have low complexity, and configure the GPM split prediction candidate list for the two regions of the block only in a pre-specified mode. At this time, the pre-designated mode may be an intra prediction mode in which there is not much change in each direction within the region. Specifically, the predefined mode may include one or more of DC mode, planar mode, vertical mode, horizontal mode, diagonal mode, and anti-diagonal mode. Through these embodiments, the number of candidates in the GPM split prediction candidate list can be reduced, thereby reducing complexity. Alternatively, when constructing a GPM segmentation prediction candidate list, if an intra prediction directional mode is used in which one of the two regions has gentle characteristics and the other region has complex characteristics, the encoder and decoder have smooth characteristics. Complexity can be reduced by restricting the intra prediction directional mode of the region to be used only in Planar mode (or DC).

The following is a description of how to derive a reconstructed GPM segmentation prediction candidate list by reconstructing the GPM segmentation prediction candidate list constructed using the above method using various methods.

In addition, in the GPM split prediction candidate list, the difference between the index of the intra prediction directional mode corresponding to the first area of the first candidate and the index of the intra prediction directional mode corresponding to the first area of the second candidate is within a pre-specified value, and If the difference between the index of the intra prediction directional mode corresponding to the second area of candidate 1 and the index of the intra prediction directional mode corresponding to the second area of the second candidate is within a pre-specified value, the encoder and decoder Any one of the 2 candidates can be excluded from the GPM split prediction candidate list. The pre-specified value may be an integer greater than or equal to 1. Specifically, the pre-designated value may be 3. For example, the index of the intra prediction directionality mode for the first area of the first candidate in the GPM division candidate list may be 35, and the index of the intra prediction directionality mode for the second area may be 5. At this time, the index of the intra prediction directionality mode for the first area of the second candidate in the GPM division candidate list may be 34, and the intra prediction directionality mode for the second area may be 6. The difference between the index of the intra prediction directional mode corresponding to the first region of the first candidate and the intra prediction directional mode corresponding to the first region of the second candidate is within 3, and the intra prediction corresponding to the second region of the first candidate The difference between the index of the directional mode and the intra prediction directional mode corresponding to the second region of the second candidate is within 3. Therefore, the encoder and decoder can exclude either the first candidate or the second candidate.

In the above-described embodiments, in the GPM split prediction candidate list, the difference between the index of the intra prediction directional mode corresponding to the first region of the first candidate and the index of the intra prediction directional mode corresponding to the first region of the second candidate is a predetermined value. or, if the difference between the index of the intra prediction directional mode corresponding to the second area of the first candidate and the index of the intra prediction directional mode corresponding to the second area of the second candidate is within a predetermined value, the encoder and decoder Any one of the first candidate and the second candidate can be excluded from the GPM split prediction candidate list. Additionally, in the previously described embodiment, the encoder and decoder may exclude candidates inserted late into the GPM candidate list among the first and second candidates. In another specific embodiment, the encoder and decoder may exclude the candidate that requires more cost for restoration among the first candidate and the second candidate.

In another specific embodiment, the encoder and decoder may exclude a candidate from the GPM segmentation prediction candidate list based on the similarity between the intra prediction directional mode of the candidate and the segmentation form of the GPM mode. The encoder and decoder may exclude the corresponding candidate from the GPM segmentation prediction candidate list if the similarity between the intra prediction directional mode of the candidate and the segmentation form of the GPM mode is less than a predetermined value. Similarity can be calculated based on the difference between the angle of the intra prediction directional mode and the direction of the division reference line of the GPM mode converted into an angle.

The encoder and decoder can adjust the angle of the segmentation reference line used in GPM mode based on the angle of the intra prediction directional mode used in intra prediction. This is because the angle precision of the division reference line used in GPM mode is lower than the angle precision of the intra prediction directional mode used in intra prediction. Specifically, if the angle of the segmentation baseline used in GPM mode and the angle of the intra prediction directional mode used in intra prediction are different, the encoder and decoder change the angle of the segmentation baseline used in GPM mode to the angle of the intra prediction directional mode used in intra prediction. It can be modified to an angle of . Additionally, the encoder and decoder can reconstruct the GPM segmentation prediction candidate list based on the angle of the segmentation reference line used in the modified GPM mode. In the embodiment of Figure 12, the angle of the division reference line of one candidate in the GPM division prediction candidate list is 3, and the angle of the intra prediction directional mode used for intra prediction of the candidate is 37. At this time, the encoder and decoder change the angle of the segmentation reference line of the candidate to 3'.

The coding efficiency of the template-based block prediction method may vary depending on the similarity between the template and the current block. Embodiments that increase the efficiency of the template-based prediction method by increasing the similarity between the template and the current block will be described.

The encoder and decoder use the GPM segmentation prediction candidate list generated through the above-described embodiments to determine the size of the current block, the horizontal or vertical size of the current block, the number of pixels of the current block, and the luminance signal of the current block. Perception or chrominance signal, intra prediction directionality mode of neighboring blocks adjacent to the current block, MPM list for the current block, intra prediction directionality mode derived from DIMD or TIMD, GPM division method of neighboring blocks adjacent to the current block, GPM division The GPM split prediction candidate list can be rearranged based on whether the intra prediction directional mode of the prediction candidate is planar mode or DC mode and at least one of the cost values of each candidate in the GPM split prediction candidate list. The encoder and decoder determine the priority of the candidate whose difference between the index of the intra prediction directionality mode of the neighboring block adjacent to the current block and the index of the intra prediction directionality mode used for the candidate in the GPM split prediction candidate list is within a pre-specified value adjacent to the current block. The absolute value of the difference between the index of the intra prediction direction mode of a neighboring block and the index of the intra prediction direction mode used for the candidate in the GPM split prediction candidate list can be given higher priority than the candidate whose absolute value is equal to or greater than a pre-specified value. At this time, the pre-designated value may be an integer of 1 or more. The pre-specified value may be 3. In another specific embodiment, the encoder and the decoder prioritize candidates whose intra prediction directionality mode is Planar mode or DC mode among the candidates in the GPM split prediction candidate list, and whose intra prediction directionality mode is Planar mode or DC mode. The list of GPM split prediction candidates can be rearranged to have a higher priority than the non-GPM candidates.

In another specific embodiment, the encoder and the decoder determine the number of candidates in the GPM split prediction candidate list based on the difference between the index of the intra prediction direction mode of the adjacent neighboring block and the index of the intra prediction direction mode used for the candidate in the GPM split prediction candidate list. You can determine the weight used when calculating cost. Specifically, if the absolute value of the difference between the index of the intra-prediction directionality mode of an adjacent neighboring block and the index of the intra-prediction directionality mode used for the candidate in the GPM split prediction candidate list is greater than a pre-specified value, the encoder and decoder use SAD or MR- The cost can be recalculated by multiplying the candidate's cost calculated with SAD by a weight greater than 1. The pre-specified value may be 0. In another specific embodiment, the pre-specified value may be an integer greater than 1. For example, the pre-designated value may be 3.

In another specific embodiment, when the intra prediction directional mode for a candidate in the GPM segmentation prediction candidate list is Planar mode or DC mode, the encoder and decoder add a cost of the candidate calculated by SAD or MR-SAD greater than 1. The cost can be recalculated by multiplying the weight.

Additionally, the weight value may increase as the difference between the intra prediction directionality mode of each candidate and the intra prediction directionality mode of the neighboring block increases. In another specific embodiment, the weight value greater than 1 may be an integer greater than or equal to 2. For example, the weight value may be 2.

Alternatively, if the intra prediction directional mode for a candidate in the GPM split prediction candidate list is not Planar or DC mode, the encoder and decoder add a weight greater than 1 to the cost value of the candidate calculated by SAD or MR-SAD, For example, the cost value can be recalculated by multiplying by 1.1 or 1.2.

If both regions of the current block encoded in GPM mode are encoded in intra prediction mode, ISP mode may not be allowed in those regions. That is, if the current block is encoded in GPM mode, the decoder does not parse the syntax related to the ISP and can infer that the value of the syntax related to the ISP is 0. Additionally, if the current block is encoded in ISP mode, the decoder may not parse the syntax related to the GPM mode and may infer that the value of the syntax related to the GPM mode is 0. At this time, 0 may mean that the mode is not applied to the current block.

If both regions of the current block encoded in GPM mode are encoded in intra prediction mode, only the reference pixel line in the corresponding region that is closest to the current block can be used. If the current block is encoded in GPM mode, the decoder may infer the syntax related to the reference pixel line as 0 without parsing the syntax related to the reference pixel line. Alternatively, if the reference pixel line used for intra prediction of the current block is not 0, the decoder may not parse the syntax related to the GPM mode and may infer that the value of the syntax related to the GPM mode is 0. At this time, 0 may mean that the mode is not applied to the current block.

GPM mode divides the current block into multiple areas, for example, two areas and encodes them. Therefore, GPM mode has similar characteristics to ISP mode, which divides vertically or horizontally. Additionally, the selection ratio of GPM mode is not large. Therefore, by integrating signaling of the GPM mode and the ISP mode, the number of bins to be encoded can be reduced and coding efficiency can be improved. Specifically, information about whether the current block is encoded in either GPM mode or ISP mode can be signaled using one syntax. For example, the decoder can parse information about whether the current block is encoded in GPM mode or ISP mode. For convenience of explanation, information about whether it is encoded in GPM mode or ISP mode is referred to as partition mode (partion_mode). If the value of the split mode is 0, this may mean that the current block is not encoded in GPM mode or ISP mode. If the value of the split mode is 1, the split mode may mean that the current block is encoded in ISP mode. At this time, the decoder may parse one or more syntax related to the ISP mode. If the value of the split mode is 2, the split mode may mean that the current block is encoded in GPM mode. At this time, the decoder may parse one or more syntax related to the GPM mode. At this time, the decoder can parse intra_gpm_index, which is a syntax related to the GPM mode. Intra_gpm_index may be index information indicating the optimal candidate in the GPM split prediction candidate list. The decoder can select the optimal GPM split prediction candidate from the GPM split prediction candidate list according to the index indicated by Intra_gpm_index. At this time, the split mode can be signaled in up to two bins. Additionally, in order for the partition mode to be encoded or decoded with CABAC, different context models may be applied to the two bins signaling the partition mode. In another specific embodiment, the context model may be applied to only the first bin of the two bins signaling the split mode, and the second bin may be encoded as a by-pass.

Figure 14 (a) shows the GPM mode divided into three areas based on the same angle. Figure 14 (b) shows the GPM mode divided into three areas based on different angles. In addition, GPM mode, which divides the current block into four or more areas, can be used. Each region divided from the current block may be encoded or decoded using intra prediction mode or inter prediction mode, respectively. In another specific embodiment, at least one region among the regions into which the current block is divided may be encoded or decoded in an intra prediction mode, and at least one region among the regions into which the current block is divided may be encoded or decoded in an inter prediction mode. For example, if two of three regions in which the current block is divided by GPM mode are encoded in intra-prediction mode, the remaining region may need to be encoded in inter-prediction mode. In another specific embodiment, all areas in which the current block is divided by GPM mode may be encoded in intra prediction mode. At this time, different intra prediction directional modes can be used in all divided areas. In another specific embodiment, the same intra prediction directional mode may be used in all divided regions. In another specific embodiment, the intra prediction mode for at least one region may be set differently from the intra prediction directional mode for another region, or a planar mode or DC mode may be used.

In GPM mode, the encoder and decoder may use template-based prediction in areas where there are more adjacent areas to the template among the areas where the current block is divided, and may not use template-based prediction in areas where there are fewer adjacent areas. For example, in FIG. 15, the first area (A) is adjacent to the template, and the second area (B) is not adjacent to the template. Therefore, the encoder and decoder may perform encoding and decoding using template-based prediction in the first area (A) and may not use template-based prediction in the second area (B). In this embodiment, MPM-based intra prediction can be used in areas where template-based prediction is not used. In another specific embodiment, a region where template-based prediction is not used may be predicted using any one of planar mode, DC mode, or MIP mode. Specifically, the current block may be encoded in GPM mode, and both regions of the current block divided into GPM mode may be encoded in intra prediction mode. At this time, the encoder and decoder can configure a template-based GPM segmentation prediction candidate list for one region. At this time, the decoder can decode the remaining area using any one of MPM-based intra prediction directional mode, planar mode, DC mode, and MIP mode.

An embodiment applied when both regions divided by GPM mode are divided into intra prediction mode or inter prediction mode will be described. At this time, when one region is encoded using the intra prediction mode, whether or not PDPC is applied to the region may vary depending on how adjacent the region is to the reconstructed sample adjacent to the current block. Specifically, if the boundary sample in one area is adjacent to a predetermined number of reconstructed neighboring block samples, PDPC may be applied to that area. Here, the pre-designated number may be an integer greater than or equal to 0. Specifically, the pre-designated number may be 4. As an example of an embodiment, area A in FIG. 15 is mostly adjacent to reconstructed samples adjacent to the current block, and PDPC may be applied to area A. However, in area B, PDPC may not be applied because there is no or only one sample adjacent to the reconstructed sample adjacent to the current block.

The encoder and decoder may determine the strength of the deblocking filter to be used when restoring the current block based on which one of the plurality of regions divided by the GPM mode is encoded using the intra prediction mode. If either of the two regions divided by GPM mode is encoded in intra prediction mode, the encoder and decoder can set the strength of deblocking filtering (bs) to a pre-specified value when restoring the current block. For example, the pre-designated value may be 2. This allows strong filtering to be applied to the boundaries of the current block.

If any of the two regions of the current block to which GPM mode is applied is encoded in inter-prediction mode, the motion information of the corresponding region, such as a motion vector, is corrected through at least one of TM, DMVR, Multi-pass DMVR, and MMVD. It can be. In another specific embodiment, when one of two regions of the current block to which GPM mode is applied is encoded in inter prediction mode, motion information, for example, a motion vector, of the corresponding region may not be corrected. At this time, the decoder may infer a pre-designated value without parsing the syntax related to motion information correction for the area encoded in inter prediction mode among the current area divided by GPM mode. The pre-specified value may be 0. The syntax related to motion information correction may include at least one of information indicating whether to apply TM, information indicating whether to apply DMVR, and information indicating whether to apply MMVD. In another specific embodiment, if one of the two regions of the current block to which the GPM mode is applied is encoded in inter prediction mode, and the boundary part sample in the region is adjacent to the restored neighboring block sample by a predetermined number or more, the corresponding OBMC can be applied to the area. Here, the pre-designated number may be an integer greater than or equal to 0. At this time, the pre-designated number may be 4. In another specific embodiment, if one of the two regions of the current block to which the GPM mode is applied is encoded in the inter prediction mode, and the boundary part sample in the region is adjacent to the restored neighboring block sample less than a predetermined number, OBMC may not apply to that area. At this time, the decoder may infer a pre-designated value without parsing the OBMC-related syntax for the inter prediction mode encoded area among the current areas divided by GPM mode. The pre-specified value may be 0. Additionally, a pre-specified value may mean that OBMC is not applied. In another specific embodiment, if one of two areas of the current block to which GPM mode is applied is encoded in inter prediction mode, MHP mode may be applied to that area. In another specific embodiment, if one of two areas of the current block to which GPM mode is applied is encoded in inter prediction mode, MHP mode may not be applied to that area. At this time, the decoder may infer a pre-specified value without parsing the syntax related to the MHP mode for the inter prediction mode encoded area among the current areas divided by the GPM mode. The pre-specified value may be 0. Additionally, a pre-specified value may mean that MHP is not applied.

The encoder can convert the residual signal to increase coding efficiency and quantize the transform coefficient value obtained through conversion. As described above, the converter may obtain a transform coefficient value by converting the residual signal. At this time, the residual signal of a specific block may be distributed throughout the entire area of the block. At this time, the encoder can focus energy in the low-frequency region by applying frequency domain transformation to the residual signal. Through this, the encoder can improve coding efficiency.

First, the encoder may obtain at least one residual block containing the residual signal for the current block. The remaining block may be either the current block or blocks divided from the current block. For convenience of explanation, the residual block may be referred to as a residual array or residual matrix including residual samples of the current block. Additionally, the remaining block may represent a block with the same size as the size of the transform unit or transform block.

Next, the encoder can transform the residual block using a transform kernel. The transformation kernel used for transformation of the remaining block may be a transformation kernel with separable characteristics of vertical transformation and horizontal transformation. In this case, transformation for the remaining block can be performed separately into vertical transformation and horizontal transformation. For example, the encoder can perform vertical transformation by applying a transformation kernel in the vertical direction of the residual block. Additionally, the encoder can perform horizontal transformation by applying a transformation kernel in the horizontal direction of the remaining block. For convenience of explanation, a transform kernel may be used as a term to refer to a set of parameters used for transforming a residual signal, such as a transform matrix, transform array, transform function, or transform. In a specific embodiment, the conversion kernel may be one selected from a plurality of kernels. Additionally, transformation kernels based on different transformation types may be used for each of vertical transformation and horizontal transformation. That is, before performing the first transformation, a transformation method for the vertical and horizontal directions is derived using at least one of the intra prediction mode of the current block, the encoding mode, the transformation method parsed from the bitstream, and the size information of the current block. It can be. Additionally, in order to reduce computational complexity in the conversion process for large blocks, only the low-frequency region may be left and the high-frequency region may be treated as 0. This is called high-frequency zeroing, and for this zeroing, the conversion size during the actual primary conversion can be set. In the high-frequency zeroing process, the low-frequency area can be set to an arbitrary, predetermined size. The horizontal size of the low-frequency area may be any of 4, 8, 16, and 32, and the vertical size may be any of 4, 8, 6, and 32.

The encoder can quantize the transform block converted from the residual block by transmitting it to the quantization unit. At this time, the transform block may include a plurality of transform coefficients. Specifically, a transform block may be composed of a plurality of transform coefficients in a two-dimensional array. The size of the transform block, like the remaining block, may be the same as either the current block or a block divided from the current block. The transformation coefficient transmitted to the quantization unit can be expressed as a quantized value.

Additionally, the encoder may perform additional transformation before the transform coefficients are quantized. The above-described transformation method may be referred to as a primary transform, and additional transformation may be referred to as a secondary transform. Secondary transformation can be selectively applied to each remaining block. In a specific embodiment, the encoder may perform secondary conversion for an area where it is difficult to concentrate energy in the low-frequency area only through primary conversion. For example, a secondary transformation may be added to a block whose residual value appears large in a direction other than the horizontal or vertical direction of the residual block. The residual value of an intra-predicted block may have a higher probability of changing in directions other than the horizontal or vertical direction compared to the residual value of an inter-predicted block. Therefore, the encoder can additionally perform secondary transformation on the residual signal of the intra-predicted block. Additionally, the encoder may omit secondary transformation for the residual signal of the inter-predicted block. Even in the secondary conversion process, high-frequency zeroing in the primary conversion can be performed.

In another specific embodiment, whether to perform secondary transformation may be determined depending on the size of the current block or remaining block. Additionally, transformation kernels of different sizes may be used depending on the size of the current block or remaining block. For example, 8X8 secondary transformation may be applied to a block where the length of the shorter side of width or height is greater than or equal to the first predetermined length. In addition, 4 At this time, the first predetermined length may be larger than the second predetermined length. Additionally, unlike the primary transformation, the secondary transformation may not be performed separately into vertical transformation and horizontal transformation. This secondary transform may be referred to as Low Frequency Non-Separable Transform (LFNST).

Additionally, in the case of video signals in a specific area, high-frequency band energy may not be reduced even if frequency conversion is performed due to rapid changes in brightness. Accordingly, compression performance by quantization may deteriorate. Additionally, when conversion is performed on an area where residual values rarely exist, encoding time and decoding time may unnecessarily increase. Accordingly, conversion of the residual signal in a specific area may be omitted. Whether or not to perform conversion on the residual signal of a specific area may be determined by syntax elements related to conversion of the specific area. Syntax elements related to transformation may include transform skip information. Transformation skip information may be a transform skip flag. If the transformation skip information for the remaining block indicates a transformation skip, transformation is not performed on the corresponding remaining block. At this time, the encoder can quantize the residual signal on which conversion has not been performed.

The above-described conversion-related syntax element may be information parsed from a video signal bitstream. The decoder can obtain conversion-related syntax elements by entropy decoding the bitstream. Additionally, the encoder may generate a video signal bitstream by entropy coding transformation-related syntax elements.

The decoder can obtain encoding information necessary for decoding by parsing the bitstream. Information related to the transformation process may include an index indicating the primary transformation type, an index indicating the secondary transformation type, and a quantized transformation coefficient. The inverse transform unit may obtain a residual signal by inversely transforming the inverse quantized transform coefficient. First, the inverse transformation unit can detect whether inverse transformation is performed for a specific region from the transformation-related syntax elements of the region. If a transform-related syntax element for a specific transform block indicates transform skipping, transform for the corresponding transform block may be omitted. At this time, both the first inverse transform and the second inverse transform may be omitted for the transform block. Additionally, the dequantized transform coefficient can be used as a residual signal. For example, the decoder can restore the current block using the dequantized transform coefficient as a residual signal. Alternatively, the second-order inverse transformation may be performed and the first-order inverse transformation may be omitted. The second-order inverse transformed value can be used as the residual signal. The above-described first-order inverse transform represents the inverse transformation of the first-order transform, and may be referred to as an inverse primary transform. The secondary inverse transform refers to the inverse transformation of the secondary transform, and may be referred to as an inverse secondary transform or inverse LFNST. In the present invention, the first (inverse) transformation may be referred to as the first (inverse) transformation, and the secondary (inverse) transformation may be referred to as the second (inverse) transformation.

Specifically, Figure 16 shows DCT-II, DCT-V (discrete cosine transform type-V), DCT-VIII (discrete cosine transform type-VIII), DST-I (discrete sine transform type-I), and DST applied to MTS. -VII Represents the kernel formula. DCT and DST can be expressed as functions of cosine and sine, respectively. When the basis function of the conversion kernel for the number of samples N is expressed as Ti(j), index i is the index in the frequency domain. Indicates that index j represents the index within the basis function. That is, as i becomes smaller, it represents a low-frequency basis function, and as i becomes larger, it represents a high-frequency basis function. When expressed as a two-dimensional matrix, the basis function Ti(j) represents the jth element of the ith row. The conversion kernels shown in Figure 16 all have separable characteristics. Therefore, the encoder can perform conversion on the residual signal X in the horizontal and vertical directions, respectively. Specifically, when the residual signal block is called X and the transformation kernel matrix is T, the transformation for the residual signal X can be expressed as TXT'. At this time, T' is the transpose of the transformation kernel matrix T. Since DCT and DST are in decimal form rather than integer, it is burdensome to implement them as is in a hardware encoder and decoder. Therefore, the decimal type conversion kernel must be approximated to an integer type conversion kernel through scaling and rounding. The integer precision of the conversion kernel can be determined to be 8 or 10 bits. If precision is low, coding efficiency may decrease. Depending on the approximation, the orthonormal properties of DCT and DST may not be maintained, but the resulting loss of coding efficiency is not significant. Therefore, approximating the conversion kernel to an integer form may be advantageous in terms of implementing a hardware encoder and decoder. IDTR (Identity Transform) is a transformation in which the result of transformation is the self before transformation, and is called an identity transformation. In general, identity transformation constructs a transformation matrix by setting 1 at positions where rows and columns have the same value. However, here, the identity transformation uses an arbitrary fixed value other than 1 to equally increase or decrease the value of the input residual signal.

Figure 17 shows how MTS and LFNST are applied to the error signal of the current block to which GPM mode is applied according to an embodiment of the present invention. Figure 17 (a) shows the operation of the encoder, and (b) shows the operation of the decoder.

When GPM mode is applied, the current block is divided into two areas, and intra prediction directional mode can be applied to the two areas. The decoder and encoder can determine the MTS and LFNST conversion method based on one of the intra prediction directional modes corresponding to each of the two regions. At this time, the MTS and LFNST transformation methods can be determined based on the angle mode among the two intra prediction directional modes corresponding to the two regions. Among the two intra prediction directional modes, the MTS and LFNST transformation methods can be determined based on the angular mode rather than the non-directional mode, such as planar or DC mode. If both of the two intra prediction directional modes corresponding to the two areas are angle modes, the MTS and LFNST conversion methods can be determined based on the intra prediction directional modes of the first area or the second area. At this time, the method of selecting the intra prediction directional mode used in the MTS and LFNST methods can be determined based on how few pixels (pixels) adjacent to the current block and adjacent portions of each region are. At this time, the first area may have a high probability of being adjacent to an adjacent pixel of the current block, so the error signal may be small. Therefore, the MTS and LFNST conversion methods can be determined based on the intra prediction directional mode in the second region, which is likely to have relatively many error signals.

If the current block was predicted using GPM mode, we describe how the encoder and decoder select an available multiple transform set (MTS) for the current block. Figure 18 is a table showing the relationship between the value of the directional mode of the intra prediction mode corresponding to the area of the current block divided in GPM mode according to an embodiment of the present invention and the index of the MTS set determined according to the size of the current block. . The row of the table corresponds to the size of the current block, and the column corresponds to an index indicating the directional mode of the intra prediction mode. At this time, the directional mode includes MIP.

1) In order to map the horizontal and vertical sizes of the current block to one variable, the encoder and decoder derive the nSzIdxW and nSzIdxH values based on the size of the current block as follows. The encoder and decoder calculate nSzIdxW as the logarithm of 2 to the width value of the current block, then discard decimal places from the obtained value and set 2 as the minimum value among the difference value and 3. The encoder and decoder set the value of nSzIdxH using the same method as setting nSzIdxW based on the height value of the current block.

2) The intra-directional mode (predMode) of the current block is derived, and when the TIMD mode is used for restoration of the current block, the intra prediction modes expanded from the existing 67 to 131 are used. The encoder and decoder reduce the number of intra-directional modes (preMode) to 67 without the extended intra-prediction mode.

3) The ucMode, nMdIdx and isTrTransposed values are derived as follows.

A. If the current block is encoded in MIP mode, the encoder and decoder set the value of ucMode to 0, the value of nMdIdx to 35, and the value of isTrTransposed to the value derived from MIP.

B. If the current block is not encoded in MIP mode, the encoder and decoder derive the ucMode, nMdIdx, and isTrTransposed values as follows.

i) The encoder and decoder set the value of ucMode to the intra-directional mode of the current block.

ii) The encoder and decoder convert the value of predMode to extended angle mode according to the width-to-height size ratio of the current block and set it as the value of predMode.

iii) The encoder and decoder clip the value of predMode to a value between 2 and 66.

iv) If the value of predMode is greater than 34, which is the diagonal mode, the encoder and decoder set the value of isTrTransposed to 1, otherwise, set the value of isTrTransposed to 0.

v) If the value of predMode is greater than 34, which is the diagonal mode, the encoder and decoder reset the value of preMode to the difference between the value of 67, which is the maximum value of preMode, plus 1. For example, if the value of predMode is 35, the encoder and decoder can reset angle mode 35 to angle mode 33 (67+1-35). If the value of predMode is 66, the video signal processing device can reset angle mode 66 to angle mode 2 (67+1-66(). Through this, the angle range of the intra directional mode indicated by preMode is diagonal. It can be reduced to within 34. In GPM mode, the number of combinations of the MTS set determined according to the size of the current block and the value of the directional mode of the intra prediction mode corresponding to the area of the divided current block can be reduced by half.

4) The encoder and decoder derive the nSzIdx value based on the nSzIdxW, nSzIdxH, and isTrTransposed values. If the value of isTrTransposed is 1, the encoder and decoder multiply the value of nSzIdxH by 4 and then set the value added by nSzIdxW as the value of nSzIdx. If the value of isTrTransposed is 0, the encoder and decoder multiply the value of nSzIdxW by 4 and then set the value added by nSzIdxH as the value of nSzIdx.

5) The encoder and decoder use a predefined table as shown in Figure 18 through the values of nSzIdx, which indicates the size of the current block, and nMdIdx, which indicates the intra directionality mode of the current block, and nTrSet, which is an index of the set of available transformation types. induces. There can be 80 types of values that nTrSet can have. In the embodiment of FIG. 18, if the size of the current block is 4x8 and the index of the directional mode within the screen of the current block is 13, the value of nTrSet is 7.

6) The encoder and decoder obtain the transformation type corresponding to the parsed mts_idx value from the transformation type set corresponding to nTrSet. Figure 19 is a table showing the relationship between the values of nTrSet and mts_idx according to an embodiment of the present invention. At this time, the row corresponds to the index of nTrSet and the column corresponds to the index of mts_idx. The conversion type in the vertical and horizontal directions is set differently based on whether the value of predMode is greater than 34, which is the diagonal mode. In the embodiment of Figure 19, if the value of nTrSet is 7 and the value of mts_idx is 3, 22 of

indices

2, 17, 18, and 22 indicating the transformation type is selected.

Figure 20 is a table showing the relationship between an index indicating the conversion type and the kernel used for conversion according to an embodiment of the present invention. At this time, the column corresponds to the index of the transformation type, and the row corresponds to the transformation direction. In the example of FIG. 20, when the value of the transformation type is 22, the vertical transformation kernel is DST1 and the horizontal transformation kernel is DCT5. If the intra-screen prediction directionality mode of the current block is greater than 34, which is the diagonal mode, the vertical and horizontal transformation types are replaced with each other.

7) Finally, when the mts_idx value is 3 and both the horizontal and vertical sizes of the current block are 16 or less, the vertical or horizontal transformation type can be reset to the IDT transformation type through the process below.

If the absolute value difference between the intra-screen prediction direction mode of the current block and the horizontal direction mode, 18, is smaller than a predetermined value, the encoder and decoder reset the vertical direction transformation type to the IDT transformation type. If the absolute value difference between the intra-screen prediction directionality mode of the current block and the horizontal direction mode, 50, is smaller than a pre-specified value, the encoder and decoder reset the horizontal direction transformation type to the IDT transformation type. At this time, it may be a pre-designated integer. Additionally, the pre-specified value may be determined based on the horizontal or vertical size of the current block. Figure 21 shows a table showing the relationship between the horizontal and vertical sizes of the current block and a pre-specified value that serves as a standard for resetting the conversion type according to an embodiment of the present invention. Figure 21 (a) shows a case where a pre-specified value is set differently every time the horizontal or vertical size differs by 4. Additionally, Figure 21 (b) shows a case where a pre-specified value is set differently every time the horizontal or vertical size differs by two times. In the embodiment of Figure 21, when the size of the current block is 16x16, the encoder and decoder do not reset the transformation type to the IDT transformation type and maintain the existing transformation type as is.

A template-based cost calculation method may be used to determine the reference pixel line used for intra prediction of the current block. The encoder and decoder can construct a template using already restored samples adjacent to the current block. A plurality of reference pixel lines are located at predetermined positions based on the current block. Specifically, the plurality of reference pixel lines and templates may be as shown in FIG. 22. Additionally, the plurality of reference pixel lines may be located any one of 1, 3, 5, 7, and 12 away from the boundary of the current block. The encoder and decoder generate prediction samples for the template position based on the reference pixel. The encoder and decoder calculate the cost based on the difference between the generated prediction sample and the template. At this time, the encoder and decoder can use SAD or MR-SAD. The encoder can encode the current block using the reference pixel line with the lowest cost. In this embodiment, the decoder performs intra prediction using the reference pixel line with the lowest cost in relation to the template among the plurality of reference pixel lines. In another specific embodiment, the encoder may configure a list of a plurality of reference pixel lines in ascending order based on cost, and include in the list an index indicating a reference pixel line used for intra prediction in the non-stream. In this embodiment, the decoder constructs a list in which a plurality of reference pixels are sorted based on cost, and performs intra prediction using the reference pixel line indicated by the signaled indicator in the constructed list. Intra prediction according to the above-described embodiments is referred to as a Template-based Multiple Reference Line (TMRL) intra prediction method.

IBC (Intra Block Copy) is a method of finding the block most similar to the current block among the restored areas in the current picture and performing intra prediction by referring to the found block. At this time, the relationship between the reference block used for intra prediction and the current block is expressed as a vector. This vector is referred to as a block vector. The encoder includes information indicating the block vector in the bitstream. The decoder can obtain a block vector for the current block by parsing information indicating a block vector from the bitstream. In IBC, one or multiple reference blocks can be used. At this time, when multiple reference blocks are used, information about multiple block vectors may be included in the bitstream. Additionally, the block vector used to predict the luminance block can be used to predict the chrominance block corresponding to the luminance block. Figure 23 shows predicting a luminance block using two reference blocks and predicting a chrominance block using two block vectors used to predict the luminance block.

Figures 24 and 25 show RRIBC being performed according to an embodiment of the present invention.

Reconstruction-Reordered IBC (RRIBC) coding mode can be used in IBC blocks. RRIBC flips blocks around a reference axis. At this time, a vertical flip or a horizontal flip may be performed. The encoder flips the current block vertically or horizontally to create a flipped current block. The encoder finds the reference block most similar to the flipped current block from the already restored reference region of the current picture. The encoder calculates the block vector, which is the distance to the reference block based on the upper left position of the current block. At this time, the block vector can be predicted through the block vectors of neighboring blocks adjacent to the current block. The prediction method for block vectors in RRIBC mode will be described later. The encoder may include the difference between the actual block vector and the predicted block vector in the bitstream. Additionally, if the current block is encoded in IBC mode, the encoder includes flip type information in the bitstream, whether a vertical flip is used, a horizontal flip is used, or RRIBC mode is not used. You can do it. The encoder may generate an error block that is the difference between the flipped current block and the reference block, transform and quantize the error block, and then include the quantized transform coefficient in the bitstream.

If the current block is encoded in IBC mode, the decoder can determine the RRIBC mode for the current block by parsing flip type information. The decoder can calculate the actual block vector by parsing the difference value of the block vector and adding it to the predicted block vector value. The decoder can generate an error block by parsing the quantized transform coefficient from the bitstream and then performing inverse quantization and inverse transformation. The decoder can restore the current block by adding the reference block indicated by the actual block vector and the error block. The decoder can flip the restored current block according to flip type information to generate the final restored current block.

If the current block is in RRIBC mode, the encoder and decoder can predict the block vector prediction value (BV ^C ) for the current block using the following method. In Figures 24 and 25, (Xn, Yn) is the neighboring block. represents the center position of , and (Xc, Yc) represents the center position of the current block. BV ⁿ represents the block vector of the neighboring block, and BV ^C represents the block vector of the current block. As shown in Figure 24, when RRIBC is performed on the current block in the horizontal direction, BV ^C is 2 * (Xn - Xc) + BV ⁿ . As shown in Figure 25, when RRIBC is performed on the current block in the vertical direction, BV ^C is 2 * (Yn - Yc) + BV ⁿ . At this time, since BV ⁿ and BV ^C use previously restored blocks, the sign of the block vector can only be negative.

In intra TMP (template matching prediction), the encoder and decoder construct a reference template using pixel values of neighboring blocks adjacent to the current block. At this time, the encoder and decoder find the area most similar to the reference template in the already reconstructed area in the current picture, and determine the block containing the similar area as the reference block. The encoder and decoder predict the current block based on the reference block. At this time, there may be multiple reference blocks. The prediction block for the luminance block is restored using the Intr TMP, and the encoder and decoder can predict the chrominance block using the block vector of the luminance block. At this time, the block vector used to predict the chrominance block may be the same as the block vector of the luminance block. Alternatively, the block vector of the luminance block may be changed according to the difference in resolution between the luminance block and the chrominance block and used as the block vector of the chrominance block. Figure 26 shows predicting a luminance block based on two reference luminance blocks and two block vectors in Intra-TMP. Additionally, the encoder and decoder derive two block vectors for the current chrominance block based on the two block vectors used for prediction of the luminance block. The encoder and decoder use the two derived block vectors to find two reference blocks, and use the two reference blocks to predict the current chrominance block.

The encoder and decoder can encode or decode the current block using IBC-GPM mode. In IBC-GPM mode, the current block is divided using GPM mode, and the divided region may be encoded in intra prediction mode or IBC prediction mode. The current block is divided into two regions, one of the divided regions may be encoded in intra prediction mode, and the remaining region may be encoded in IBC prediction mode. At this time, the divided region may be encoded using Intra TMP mode rather than IBC prediction mode. Figure 27 shows this. This can be referred to as Intra TMP-GPM mode.

The encoder and decoder can predict the current block using the weighted average between the intra predicted block and the IBC predicted block. This may be referred to as IBC-CIIP mode. At this time, it can be encoded or decoded using the Intra TMP prediction mode rather than the IBC predicted block, and this can be referred to as the IntraTMP-CIIP mode.

There may be a problem with how to perform deblocking filtering in the various prediction methods described above. First, general deblocking filtering will be explained, and then how to perform deblocking for each prediction mode.

In Figure 29(a), the dotted line between the P block and the Q block is a block boundary. Block boundaries exist at any given size. Figure 29(a) shows that block boundaries exist at every size of 4. Figure 29(b) shows a sample in which filtering is performed based on block boundaries. The encoder and decoder can perform deblocking filtering on the currently restored block to alleviate blocking phenomenon that occurs at block boundaries. In the deblocking filtering process, filtering may be performed after determining the transform block boundary, sub-block boundary, filter length, filtering strength (bS), filtering parameters, whether to perform filtering, and the type of filtering.

Neighboring blocks or the current block may be encoded in IBC-CIIP mode or IntraTMP-CIIP mode. At this time, how the encoder and decoder determine the strength of deblocking filtering applied to the boundary between the surrounding block and the current block will be described. The encoder and decoder can perform deblocking filtering with the same filtering strength as the filtering strength applied to the intra-predicted coding mode on the boundary between the neighboring block and the current block. In another specific embodiment, the encoder and decoder may perform deblocking filtering with a predetermined filtering strength on the boundary between the neighboring block and the current block. The filtering strength can be divided into 0, 1, and 2, with 2 indicating the strongest filtering strength, 1 indicating the second strongest filtering strength, and 0 indicating that no filtering is performed. At this time, the pre-designated intensity may be 2. This is because Intra prediction is used in IBC-CIIP mode and IntraTMP-CIIP mode. In another specific embodiment, the predetermined intensity may be 1. This is because not only Intra prediction but also Inter prediction is used in IBC-CIIP mode and IntraTMP-CIIP mode.

In another specific embodiment, the encoder and decoder may perform deblocking filtering with a filtering strength predetermined at the boundary between the neighboring block and the current block based on the difference in block vectors between the neighboring block and the current block. Specifically, if the difference in block vectors between the neighboring blocks and the current block is within a pre-specified value, the encoder and decoder can perform deblocking filtering with a pre-specified filtering strength on the boundary between the neighboring blocks and the current block. At this time, the pre-designated filtering strength may be 0. Additionally, when the difference in block vectors between neighboring blocks and the current block is greater than a pre-specified value, the encoder and decoder may perform deblocking filtering with a pre-specified filtering strength on the boundary between the neighboring blocks and the current block. At this time, the pre-designated filtering strength may be 1. Additionally, the pre-specified value may be determined according to the resolution of the block vector. Additionally, the pre-designated value may be an integer of 1 or more. Additionally, the resolution of the block vector may be any one of ¼ pixel unit, ½ pixel unit, 1 pixel unit, and 4 pixel unit.

Neighboring blocks or the current block may be encoded in IntraTMP mode. At this time, how the encoder and decoder determine the strength of deblocking filtering applied to the boundary between the surrounding block and the current block will be described. The encoder and decoder can perform deblocking filtering with the same filtering strength as the filtering strength applied to the inter-predicted coding mode on the boundary between the neighboring block and the current block. In another specific embodiment, the encoder and decoder may perform deblocking filtering with a predetermined filtering strength on the boundary between the neighboring block and the current block. The pre-specified value may be 1.

Neighboring blocks or the current block can be encoded in IBC-GPM or IntraTMP-GPM mode. At this time, how the encoder and decoder determine the strength of deblocking filtering applied to the boundary between the surrounding block and the current block will be described. The encoder and decoder can perform deblocking filtering with the same filtering strength as the filtering strength applied to the intra-predicted coding mode on the boundary between the neighboring block and the current block. Specifically, the encoder and decoder can perform deblocking filtering with a filtering strength of 2 on the boundary between neighboring blocks and the current block.

A neighboring block or the current block may be encoded in IBC-GPM or IntraTMP-GPM mode, and any region divided using GPM mode may be encoded using intra prediction mode. At this time, the encoder and decoder may perform deblocking filtering with a filtering strength predetermined at the boundary between the neighboring block and the current block based on the difference in block vectors between the neighboring block and the current block. The pre-specified filtering strength may be 2.

Neighboring blocks or the current block may be encoded in RRIBC mode. At this time, how the encoder and decoder determine the strength of deblocking filtering applied to the boundary between the surrounding block and the current block will be described. The encoder and decoder may perform deblocking filtering with a filtering strength of 0 on the boundary between the neighboring block and the current block based on the difference in block vectors between the neighboring block and the current block. RRIBC mode is a mode performed on screen content and is effective in images where clear shapes such as letters and shapes appear repeatedly. Therefore, if the surrounding block or the current block is encoded in RRIBC mode, the blocking phenomenon at the block boundary may not exist. In another specific embodiment, when the neighboring block or current block is IBC-GPM mode or IBC-CIIP, and the IBC mode is RRIBC mode, the encoder and decoder determine the neighboring block based on the difference in block vectors between the neighboring block and the current block. Deblocking filtering can be performed with a pre-specified filtering strength on the boundary between and the current block. The pre-specified filtering strength may be 1.

The method of determining the strength of deblocking filtering through the above-described embodiments can be equally applied not only to the boundary of the luminance block but also to the boundary of the chrominance block.

The methods described above in this specification may be performed through a processor of a decoder or encoder. Additionally, the encoder can generate a bitstream that is decoded by a video signal processing method. Additionally, the bitstream generated by the encoder may be stored in a computer-readable non-transitory storage medium (recording medium).

Although this specification is mainly described from the perspective of a decoder, it can be operated equally in an encoder. The term parsing in this specification has been described with a focus on the process of obtaining information from the bitstream, but from the encoder perspective, it can be interpreted as generating corresponding information in the bitstream. Therefore, the term parsing is not limited to the decoder operation, but can also be interpreted as the act of generating a bitstream in the encoder. Additionally, this bitstream can be generated and stored in a computer-readable recording medium.

Embodiments of the present invention described above can be implemented through various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

In the case of hardware implementation, the method according to embodiments of the present invention uses one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), and Programmable Logic Devices (PLDs). , can be implemented by FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, etc.

In the case of implementation by firmware or software, the method according to embodiments of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above. Software code can be stored in memory and run by a processor. The memory may be located inside or outside the processor, and may exchange data with the processor through various known means.

Some embodiments may also be implemented in the form of a recording medium containing instructions executable by a computer, such as program modules executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and non-volatile media, removable and non-removable media. Additionally, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures or other data of modulated data signals such as program modules, or other transmission mechanisms, and includes any information delivery medium.

The description of the present invention described above is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. will be. Therefore, the embodiments described above are illustrative in all respects and should be interpreted as limited. For example, each generation element described as single may be implemented in a distributed manner, and similarly, generation elements described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

Claims

In a decoding device that decodes a video signal,

Includes a processor,

The processor is

Divide the current block into multiple areas according to the baseline,

Predictions are made for each divided area,

At least one of the plurality of regions is predicted through intra prediction

Decoding device.
In paragraph 1:

The processor is

When generating a candidate list including candidates of intra prediction directionality mode information used for prediction of at least one region predicted through intra prediction, generating candidates for each combination of the type of division and the intra prediction directionality mode.

Decoding device.
In paragraph 2,

The processor is

Excluding from the candidate list candidates whose difference between the angle corresponding to the direction corresponding to the type of division and the angle indicated by the intra prediction direction mode is less than or equal to a predetermined value

Decoding device.
In part 2,

The processor is

Excluding from the candidate list a candidate whose difference between the index of the intra prediction directional mode and the index of the intra prediction directional mode corresponding to the direction corresponding to the division type is less than or equal to a predetermined value.

Decoding device.
In paragraph 2,

The processor is

If the angle corresponding to the direction corresponding to the type of division of the first candidate in the candidate list is different from the angle indicated by the intra prediction directional mode of the first candidate, it corresponds to the type of division of the first candidate Set the angle corresponding to the direction to the angle indicated by the intra prediction directionality mode.

Decoding device.
In paragraph 2,

The processor is

The difference between the index of the intra prediction direction mode of the neighboring block adjacent to the current block and the index of the intra prediction direction mode used for the first candidate in the candidate list is within a predetermined value, and the intra prediction of the neighboring block adjacent to the current block If the difference between the index of the directional mode and the index of the intra prediction directional mode used for the second candidate in the candidate list is equal to or greater than the predetermined value, the priority of the first candidate is set to a higher priority than the second candidate. specified as

Decoding device.
In paragraph 2,

The processor is

Determining a weight used when calculating the cost of the candidate based on the difference between the index of the intra prediction direction mode of a neighboring block adjacent to the current block and the index of the intra prediction direction mode used for the candidate in the candidate list.

Decoding device.
In paragraph 7:

The processor is

If the difference between the intra prediction direction mode of the adjacent neighboring block and the index of the intra prediction direction mode used for the candidate is greater than a predetermined value, the cost of the candidate calculated by SAD or MR-SAD is multiplied by a weight greater than 1. Recalculating the cost by value

Decoding device.
In paragraph 1:

The bitstream including the video signal includes syntax indicating whether it is encoded in either GPM mode or ISP mode,

The processor is

Obtaining the syntax from the bitstream to determine whether the current block is encoded in either GPM mode or ISP mode

Decoding device.
In paragraph 1:

The processor is

Determining the strength of a deblocking filter used for restoration of the current block based on which one of the plurality of regions is encoded using an intra prediction mode

Decoding device.
In an encoding device that encodes a video signal,

Includes a processor,

The processor is

Divide the current block into multiple areas according to the baseline,

Predictions are made for each divided area,

At least one of the plurality of regions is predicted through intra prediction

encoding device.
In paragraph 11:

The processor is

When generating a candidate list including candidates of intra prediction directional mode information used for prediction of at least one region predicted through intra prediction, generating candidates for each combination of the type of division and the intra prediction directional mode

encoding device.
In paragraph 12:

The processor is

Excluding from the candidate list candidates whose difference between the angle corresponding to the direction corresponding to the type of division and the angle indicated by the intra prediction direction mode is less than or equal to a predetermined value

encoding device.
In Lesson 12,

The processor is

Excluding from the candidate list a candidate whose difference between the index of the intra prediction directional mode and the index of the intra prediction directional mode corresponding to the direction corresponding to the division type is less than or equal to a predetermined value.

encoding device.
In paragraph 12:

The processor is

If the angle corresponding to the direction corresponding to the type of division of the first candidate in the candidate list is different from the angle indicated by the intra prediction direction mode of the first candidate, it corresponds to the type of division of the first candidate Set the angle corresponding to the direction to the angle indicated by the intra prediction directionality mode.

encoding device.
In paragraph 12:

The processor is

The difference between the index of the intra prediction direction mode of the neighboring block adjacent to the current block and the index of the intra prediction direction mode used for the first candidate in the candidate list is within a predetermined value, and the intra prediction of the neighboring block adjacent to the current block If the difference between the index of the directional mode and the index of the intra prediction directional mode used for the second candidate in the candidate list is equal to or greater than the predetermined value, the priority of the first candidate is set to a higher priority than the second candidate. specified as

encoding device.
In paragraph 12:

The processor is

Determining a weight used when calculating the cost of the candidate based on the difference between the index of the intra prediction direction mode of a neighboring block adjacent to the current block and the index of the intra prediction direction mode used for the candidate in the candidate list.

encoding device.
In paragraph 17:

The processor is

If the difference between the intra prediction direction mode of the adjacent neighboring block and the index of the intra prediction direction mode used for the candidate is greater than a predetermined value, the cost of the candidate calculated by SAD or MR-SAD is multiplied by a weight greater than 1. Recalculating the cost by value

encoding device.
In paragraph 11:

The processor is

Including a syntax indicating whether the bitstream including the video signal is encoded in either GPM mode or ISP mode

encoding device.
In paragraph 11:

The processor is

Determining the strength of a deblocking filter used for restoration of the current block based on which one of the plurality of regions is encoded using an intra prediction mode

Encoding device.
A computer-readable non-transitory storage medium storing a bitstream, wherein the bitstream is decoded by a decoding method,

The decoding method is

dividing the current block into a plurality of regions according to a baseline; and

Including the step of performing prediction for each divided area,

At least one of the plurality of regions is predicted through intra prediction

Non-transitory storage medium.