WO2024091094A1 - Video encoding/decoding method and recording medium for storing bitstream - Google Patents

Abstract

A video encoding/decoding method according to the present invention comprise the steps of: dividing a chroma block into a plurality of partitions; and performing intra-prediction for each of the partitions in the chroma block.

Description

Video encoding/decoding method and recording medium for storing bitstream

This disclosure relates to a video signal processing method and device.

Recently, demand for high-resolution, high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images is increasing in various application fields. As video data becomes higher resolution and higher quality, the amount of data increases relative to existing video data. Therefore, when video data is transmitted using media such as existing wired or wireless broadband lines or stored using existing storage media, transmission costs and Storage costs increase. High-efficiency video compression technologies can be used to solve these problems that arise as video data becomes higher resolution and higher quality.

Inter-screen prediction technology that predicts pixel values included in the current picture from pictures before or after the current picture using video compression technology, intra-screen prediction technology that predicts pixel values included in the current picture using pixel information in the current picture, There are various technologies, such as entropy coding technology, which assigns short codes to values with a high frequency of occurrence and long codes to values with a low frequency of occurrence. Using these video compression technologies, video data can be effectively compressed and transmitted or stored.

Meanwhile, as the demand for high-resolution video increases, the demand for three-dimensional video content as a new video service is also increasing. Discussions are underway regarding video compression technology to effectively provide high-resolution and ultra-high-resolution stereoscopic video content.

The purpose of the present disclosure is to provide a method and device for predicting a chroma block using a restored luma block when encoding/decoding a video signal.

The purpose of the present disclosure is to provide a method and device for predicting a chroma block based on the linearity of the luma component and the chroma component when encoding/decoding a video signal.

The purpose of the present disclosure is to provide a method and device for deriving encoding/decoding information for a chroma block from a co-located luma block when encoding/decoding a video signal.

The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

An image encoding/decoding method according to the present disclosure includes dividing a chroma block into a plurality of partitions; And it may include performing intra prediction on each of the partitions in the chroma block. At this time, the partition type of the chroma block may be determined based on at least one of the partition type or directionality of the luma block at the same location as the chroma block.

In the video decoding/encoding method according to the present disclosure, the partition type of the chroma block may be set to be the same as the partition type of the co-located luma block.

In the video decoding/encoding method according to the present disclosure, the plurality of partitions are generated by dividing the chroma block in a horizontal or vertical direction, and the division direction of the chroma block is the directionality of the co-located luma block. It can be decided based on .

In the image decoding/encoding method according to the present disclosure, the decoding order of the plurality of partitions in the chroma block may be determined based on the directionality of the co-located luma block.

The image decoding/encoding method according to the present disclosure may further include the step of determining whether to perform flipping on the chroma block. At this time, the flipping of the chroma block may be performed in at least one of the horizontal or vertical directions.

In the image decoding/encoding method according to the present disclosure, whether to perform the flipping and the direction in which the flipping is performed may be determined based on the directionality of the co-located luma block.

In the video decoding/coding method according to the present disclosure, the flipping may be allowed only when the co-located luma block is encoded by intra prediction.

In the image decoding/encoding method according to the present disclosure, the intra prediction mode of each of the plurality of partitions in the chroma block may be set to be the same as the intra prediction mode of each of the plurality of partitions included in the co-located luma block. You can.

In the image decoding/encoding method according to the present disclosure, a predefined intra prediction mode is applied to a partition corresponding to an area with a large prediction error in the co-located luma block among the plurality of partitions in the chroma block, and otherwise, For partitions that do not exist, the intra prediction mode of the co-located luma block may be applied.

In the image decoding/encoding method according to the present disclosure, whether the sub-region in the co-located luma block is an area with a large prediction error can be determined by comparing the average value of the absolute value of residual samples in the sub-region with a threshold value. .

In the image decoding/encoding method according to the present disclosure, the threshold value may be derived based on the average value of absolute values of residual samples within the co-located luma block.

In the image decoding/encoding method according to the present disclosure, the directionality of the co-located luma block is determined according to a result of intra prediction performed based on predefined intra prediction modes and reference samples around the co-located luma block. It can be determined based on the derived optimal intra prediction mode.

In the image decoding/encoding method according to the present disclosure, the directionality of the co-located luma block may be set to be the same as the directionality of the reference block including the co-located luma block.

A computer-readable recording medium that stores a bitstream encoded by the video encoding method according to the present disclosure may be provided.

According to the present disclosure, encoding/decoding efficiency can be improved by predicting a chroma block using a restored luma block.

According to the present disclosure, the accuracy of intra prediction can be improved by predicting a chroma block based on the linearity of the luma component and the chroma component.

According to the present disclosure, compression efficiency for the chroma block can be improved by deriving the encoding/decoding information of the chroma block with reference to the co-located luma block.

The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 is a block diagram showing a video encoding device according to an embodiment of the present disclosure.

Figure 2 is a block diagram showing a video decoding device according to an embodiment of the present disclosure.

FIG. 3 illustrates an image encoding/decoding method performed by the image encoding/decoding device according to the present disclosure.

Figures 4 and 5 show an example of a plurality of intra prediction modes according to the present disclosure.

Figure 6 illustrates a planar mode-based intra prediction method according to the present disclosure.

Figure 7 shows a DC mode-based intra prediction method according to the present disclosure.

Figure 8 illustrates a directional mode-based intra prediction method according to the present disclosure.

Figure 9 shows a method for deriving samples of fractional positions.

Figures 10 and 11 show that the tangent value for the angle is scaled by 32 times for each intra prediction mode.

Figure 12 is a diagram illustrating an intra prediction aspect when the directional mode is one of modes 34 to 49.

Figure 13 is a diagram for explaining an example of generating an upper reference sample by interpolating left reference samples.

Figure 14 shows an example in which intra prediction is performed using reference samples arranged in a 1D array.

Figure 15 is a diagram for explaining the encoding/decoding order when a single tree structure is used.

Figure 16 is a diagram for explaining the encoding/decoding order when a dual tree structure is used.

Figure 17 is a flowchart showing a method of predicting a chroma block using a restored luma block.

Figures 18 to 20 show examples of down-sampling a luma block.

FIG. 21 is a diagram for explaining an example of a location where down sampling is applied.

Figures 22 and 23 are diagrams for explaining an example of determining the division structure of a chroma block with reference to a co-located luma block in a dual tree structure.

Figure 24 shows an example in which encoding/decoding information for a chroma block is determined based on the directionality of luma samples.

Figure 25 shows an example of using restored samples in the same location luma block as reference samples.

Figure 26 shows an example in which intra prediction modes are classified into a plurality of groups.

Figures 27 and 28 show the scan order for each sub-block in the chroma block according to the direction of the co-located luma block.

Figure 29 shows an example in which flipping is performed on samples within a chroma block.

Figure 30 shows an example of determining whether or not an area has a large prediction error on a sub-block basis within the same location luma block.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention. While describing each drawing, similar reference numerals are used for similar components.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component without departing from the scope of the present invention, and similarly, the second component may also be named a first component. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. Hereinafter, the same reference numerals will be used for the same components in the drawings, and duplicate descriptions of the same components will be omitted.

Figure 1 is a block diagram showing a video encoding device according to an embodiment of the present invention.

Referring to FIG. 1, the image encoding device 100 includes a picture segmentation unit 110, prediction units 120 and 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, and an entropy encoding unit ( 165), an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

Each component shown in FIG. 1 is shown independently to represent different characteristic functions in the video encoding device, and does not mean that each component is comprised of separate hardware or one software component. That is, each component is listed and included as a separate component for convenience of explanation, and at least two of each component can be combined to form one component, or one component can be divided into a plurality of components to perform a function, and each of these components can perform a function. Integrated embodiments and separate embodiments of the constituent parts are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

Additionally, some components may not be essential components that perform essential functions in the present invention, but may simply be optional components to improve performance. The present invention can be implemented by including only essential components for implementing the essence of the present invention excluding components used only to improve performance, and a structure including only essential components excluding optional components used only to improve performance. is also included in the scope of rights of the present invention.

The picture division unit 110 may divide the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transformation unit (TU), or a coding unit (CU). The picture division unit 110 divides one picture into a combination of a plurality of coding units, prediction units, and transformation units, and combines one coding unit, prediction unit, and transformation unit based on a predetermined standard (for example, a cost function). You can encode the picture by selecting .

For example, one picture may be divided into a plurality of coding units. To split the coding unit in a picture, a recursive tree structure such as the Quad Tree Structure can be used. Coding is split into other coding units with one image or the largest coding unit as the root. A unit can be divided into child nodes equal to the number of divided coding units. A coding unit that is no longer divided according to certain restrictions becomes a leaf node. That is, assuming that only square division is possible for one coding unit, one coding unit can be divided into up to four different coding units.

Hereinafter, in the embodiments of the present invention, the coding unit may be used to mean a unit that performs encoding, or may be used to mean a unit that performs decoding.

A prediction unit may be divided into at least one square or rectangular shape of the same size within one coding unit, and any one of the prediction units divided within one coding unit may be a prediction unit of another prediction unit. It may be divided to have a different shape and/or size than the unit.

If the prediction unit for which intra prediction is performed based on the coding unit is not the minimum coding unit when generated, intra prediction can be performed without dividing the prediction unit into a plurality of prediction units NxN.

The prediction units 120 and 125 may include an inter prediction unit 120 that performs inter prediction and an intra prediction unit 125 that performs intra prediction. It is possible to determine whether to use inter prediction or intra prediction for a prediction unit, and determine specific information (eg, intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. At this time, the processing unit in which the prediction is performed and the processing unit in which the prediction method and specific contents are determined may be different. For example, the prediction method and prediction mode are determined in prediction units, and prediction may be performed in transformation units. The residual value (residual block) between the generated prediction block and the original block may be input to the conversion unit 130. Additionally, prediction mode information, motion vector information, etc. used for prediction may be encoded in the entropy encoder 165 together with the residual value and transmitted to the decoding device. When using a specific encoding mode, it is possible to encode the original block as is and transmit it to the decoder without generating a prediction block through the prediction units 120 and 125.

The inter prediction unit 120 may predict a prediction unit based on information on at least one picture among the pictures before or after the current picture, and in some cases, prediction based on information on a partially encoded region within the current picture. Units can also be predicted. The inter prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

The reference picture interpolation unit may receive reference picture information from the memory 155 and generate pixel information of an integer number of pixels or less from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information of an integer pixel or less in 1/4 pixel units. In the case of color difference signals, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information of an integer pixel or less in 1/8 pixel units.

The motion prediction unit may perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods, such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm), can be used to calculate the motion vector. The motion vector may have a motion vector value in units of 1/2 or 1/4 pixels based on the interpolated pixels. The motion prediction unit can predict the current prediction unit by using a different motion prediction method. As a motion prediction method, various methods such as the skip method, the merge method, the Advanced Motion Vector Prediction (AMVP) method, and the intra block copy method can be used.

The intra prediction unit 125 may generate a prediction unit based on reference pixel information around the current block, which is pixel information in the current picture. If the neighboring block of the current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed is the reference pixel of the block on which intra prediction has been performed. It can be used in place of information. That is, when a reference pixel is not available, the unavailable reference pixel information can be replaced with at least one reference pixel among available reference pixels.

In intra prediction, the prediction mode can include a directional prediction mode that uses reference pixel information according to the prediction direction and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luminance information and the mode for predicting chrominance information may be different, and intra prediction mode information used to predict luminance information or predicted luminance signal information may be used to predict chrominance information.

When performing intra prediction, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is made based on the pixel on the left, the pixel on the top left, and the pixel on the top of the prediction unit. can be performed. However, when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction can be performed using a reference pixel based on the transformation unit. Additionally, intra prediction using NxN partitioning can be used only for the minimum coding unit.

The intra prediction method can generate a prediction block after applying an Adaptive Intra Smoothing (AIS) filter to the reference pixel according to the prediction mode. The type of AIS filter applied to the reference pixel may be different. To perform the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction mode of prediction units existing around the current prediction unit. When predicting the prediction mode of the current prediction unit using predicted mode information from neighboring prediction units, if the intra prediction mode of the current prediction unit and neighboring prediction units are the same, predetermined flag information is used to predict the current prediction unit and neighboring prediction units. Information that the prediction modes of are the same can be transmitted, and if the prediction modes of the current prediction unit and neighboring prediction units are different, entropy encoding can be performed to encode the prediction mode information of the current block.

In addition, a residual block may be generated that includes residual information that is the difference between the prediction unit on which prediction was performed based on the prediction unit generated by the prediction units 120 and 125 and the original block of the prediction unit. The generated residual block may be input to the conversion unit 130.

The transform unit 130 transforms the residual block, including the original block and the residual value information of the prediction unit generated through the prediction units 120 and 125, into DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT and It can be converted using the same conversion method. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on intra prediction mode information of the prediction unit used to generate the residual block.

The quantization unit 135 may quantize the values converted to the frequency domain by the conversion unit 130. The quantization coefficient may change depending on the block or the importance of the image. The value calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the realignment unit 160.

The rearrangement unit 160 may rearrange coefficient values for the quantized residual values.

The rearrangement unit 160 can change the coefficients in a two-dimensional block form into a one-dimensional vector form through a coefficient scanning method. For example, the realignment unit 160 can scan from DC coefficients to coefficients in the high frequency region using a zig-zag scan method and change it into a one-dimensional vector form. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional block-type coefficients in the column direction or a horizontal scan that scans the two-dimensional block-type coefficients in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, it can be determined which scan method among zig-zag scan, vertical scan, and horizontal scan will be used.

The entropy encoding unit 165 may perform entropy encoding based on the values calculated by the reordering unit 160. Entropy coding can use various coding methods, such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

The entropy encoding unit 165 receives coding unit residual value coefficient information, block type information, prediction mode information, division unit information, prediction unit information and transmission unit information, and motion information from the reordering unit 160 and prediction units 120 and 125. Various information such as vector information, reference frame information, block interpolation information, and filtering information can be encoded.

The entropy encoding unit 165 may entropy encode the coefficient value of the coding unit input from the reordering unit 160.

The inverse quantization unit 140 and the inverse transformation unit 145 inversely quantize the values quantized in the quantization unit 135 and inversely transform the values transformed in the transformation unit 130. The residual value generated in the inverse quantization unit 140 and the inverse transform unit 145 is restored by combining prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction units 120 and 125. You can create a block (Reconstructed Block).

The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

The deblocking filter can remove block distortion caused by boundaries between blocks in the restored picture. To determine whether to perform deblocking, it is possible to determine whether to apply a deblocking filter to the current block based on the pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. Additionally, when applying a deblocking filter, horizontal filtering and vertical filtering can be processed in parallel when vertical filtering and horizontal filtering are performed.

The offset correction unit may correct the offset of the deblocked image from the original image in pixel units. In order to perform offset correction for a specific picture, the pixels included in the image are divided into a certain number of areas, then the area to perform offset is determined and the offset is applied to that area, or the offset is performed by considering the edge information of each pixel. You can use the method of applying .

Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered restored image and the original image. After dividing the pixels included in the image into predetermined groups, filtering can be performed differentially for each group by determining one filter to be applied to that group. Information related to whether to apply ALF may be transmitted for each coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied may vary for each block. Additionally, an ALF filter of the same type (fixed type) may be applied regardless of the characteristics of the block to which it is applied.

The memory 155 may store a reconstructed block or picture calculated through the filter unit 150, and the stored reconstructed block or picture may be provided to the prediction units 120 and 125 when inter prediction is performed.

Figure 2 is a block diagram showing an image decoding device according to an embodiment of the present invention.

Referring to FIG. 2, the image decoding device 200 includes an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, 235, and a filter unit ( 240) and memory 245 may be included.

When a video bitstream is input from a video encoding device, the input bitstream can be decoded in a procedure opposite to that of the video encoding device.

The entropy decoding unit 210 may perform entropy decoding in a procedure opposite to that of performing entropy encoding in the entropy encoding unit of the video encoding device. For example, various methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) may be applied in response to the method performed in the image encoding device.

The entropy decoder 210 can decode information related to intra prediction and inter prediction performed by the encoding device.

The rearrangement unit 215 may rearrange the bitstream entropy-decoded by the entropy decoder 210 based on the method in which the encoder rearranges the bitstream. Coefficients expressed in the form of a one-dimensional vector can be restored and rearranged as coefficients in the form of a two-dimensional block. The reordering unit 215 may receive information related to coefficient scanning performed by the encoder and perform reordering by reverse scanning based on the scanning order performed by the encoder.

The inverse quantization unit 220 may perform inverse quantization based on the quantization parameters provided by the encoding device and the coefficient values of the rearranged blocks.

The inverse transform unit 225 may perform inverse transform, that is, inverse DCT, inverse DST, and inverse KLT, on the transform performed by the transformer, that is, DCT, DST, and KLT, on the quantization result performed by the image encoding device. Inverse transformation may be performed based on the transmission unit determined by the video encoding device. The inverse transform unit 225 of the video decoding device may selectively perform a transformation technique (eg, DCT, DST, KLT) according to a plurality of information such as a prediction method, the size of the current block, and the prediction direction.

The prediction units 230 and 235 may generate a prediction block based on prediction block generation-related information provided by the entropy decoder 210 and previously decoded block or picture information provided by the memory 245.

As described above, when performing intra prediction in the same manner as the operation in the video encoding device, when the size of the prediction unit and the size of the transformation unit are the same, the pixel existing on the left of the prediction unit, the pixel existing on the upper left, and the size of the transformation unit are the same. Intra prediction of the prediction unit is performed based on existing pixels, but when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction is performed using a reference pixel based on the transformation unit. can do. Additionally, intra prediction using NxN partitioning only for the minimum coding unit can be used.

The prediction units 230 and 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit discriminator receives various information such as prediction unit information input from the entropy decoder 210, prediction mode information of the intra prediction method, and motion prediction-related information of the inter prediction method, distinguishes the prediction unit from the current coding unit, and makes predictions. It is possible to determine whether a unit performs inter-prediction or intra-prediction. The inter prediction unit 230 uses the information required for inter prediction of the current prediction unit provided by the video encoding device to determine the current prediction unit based on the information included in at least one of the pictures before or after the current picture including the current prediction unit. Inter prediction can be performed on prediction units. Alternatively, inter prediction may be performed based on information on a pre-restored partial region within the current picture including the current prediction unit.

To perform inter prediction, based on the coding unit, the motion prediction method of the prediction unit included in the coding unit is Skip Mode, Merge Mode, AMVP Mode, and Intra Block Copy Mode. You can judge whether it is a certain method or not.

The intra prediction unit 235 may generate a prediction block based on pixel information in the current picture. If the prediction unit is a prediction unit that has performed intra prediction, intra prediction may be performed based on intra prediction mode information of the prediction unit provided by the video encoding device. The intra prediction unit 235 may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixels of the current block, and can be applied by determining whether or not to apply the filter according to the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixel of the current block using the prediction mode and AIS filter information of the prediction unit provided by the video encoding device. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

If the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on pixel values obtained by interpolating the reference pixel, the reference pixel interpolator may interpolate the reference pixel to generate a reference pixel in pixel units of an integer value or less. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is DC mode.

The restored block or picture may be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

Information on whether a deblocking filter has been applied to the corresponding block or picture can be provided from the video encoding device, and when a deblocking filter has been applied, information on whether a strong filter or a weak filter has been applied. The deblocking filter of the video decoding device receives information related to the deblocking filter provided by the video encoding device, and the video decoding device can perform deblocking filtering on the corresponding block.

The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding and offset value information.

ALF can be applied to the coding unit based on ALF application availability information, ALF coefficient information, etc. provided from the coding device. This ALF information may be included and provided in a specific parameter set.

The memory 245 can store the restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

As described above, hereinafter, in the embodiments of the present invention, the term coding unit is used as a coding unit for convenience of explanation, but it may also be a unit that performs not only encoding but also decoding.

In addition, the current block represents an encoding/decoding target block and, depending on the encoding/decoding stage, is a coding tree block (or coding tree unit), a coding block (or coding unit), a transform block (or transform unit), or a prediction block. (or prediction unit), etc. In this specification, 'unit' may represent a basic unit for performing a specific encoding/decoding process, and 'block' may represent a pixel array of a predetermined size. Unless otherwise specified, ‘block’ and ‘unit’ can be used with the same meaning. For example, in embodiments described later, a coding block (coding block) and a coding unit (coding unit) may be understood to have equivalent meanings.

FIG. 3 illustrates an image encoding/decoding method performed by the image encoding/decoding device according to the present disclosure.

Referring to FIG. 3, a reference line for intra prediction of the current block can be determined (S300).

The current block may use one or more of a plurality of reference line candidates pre-defined in the video encoding/decoding device as a reference line for intra prediction. Here, the plurality of pre-defined reference line candidates may include a neighboring reference line adjacent to the current block to be decoded and N non-neighboring reference lines that are 1-sample to N-sample away from the boundary of the current block. N may be an integer of 1, 2, 3, or more. Hereinafter, for convenience of explanation, it is assumed that the plurality of reference line candidates available for the current block consists of a neighboring reference line candidate and three non-neighboring reference line candidates, but is not limited thereto. That is, of course, the plurality of reference line candidates available for the current block may include four or more non-neighboring reference line candidates.

The video encoding device can determine an optimal reference line candidate among a plurality of reference line candidates and encode an index for specifying it. The video decoding device can determine the reference line of the current block based on the index signaled through the bitstream. The index may specify one of a plurality of reference line candidates. The reference line candidate specified by the index can be used as the reference line of the current block.

The number of indices signaled to determine the reference line of the current block may be 1, 2, or more. For example, when the number of signaled indices is 1, the current block may perform intra prediction using only a single reference line candidate specified by the signaled index among a plurality of reference line candidates. Alternatively, when the number of signaled indices is two or more, the current block may perform intra prediction using a plurality of reference line candidates specified by a plurality of indices among a plurality of reference line candidates.

Referring to FIG. 3, the intra prediction mode of the current block can be determined (S310).

The intra prediction mode of the current block may be determined from a plurality of intra prediction modes predefined in the video encoding/decoding device. The plurality of pre-defined intra prediction modes will be examined with reference to FIGS. 4 and 5.

Figure 4 shows an example of a plurality of intra prediction modes according to the present disclosure.

Referring to FIG. 4, a plurality of intra prediction modes pre-defined in the video encoding/decoding device may be comprised of a non-directional mode and a directional mode. The non-directional mode may include at least one of planar mode or DC mode. The directional mode may include directional modes numbered 2 to 66.

The directional mode may be expanded further than shown in FIG. 4. Figure 5 shows an example in which the directional mode is expanded.

In Figure 5, modes -1 to -14 and modes 67 to 80 are illustrated as being added. These directional modes may be referred to as wide angle intra prediction modes. Whether to use the wide angle intra prediction mode can be determined depending on the type of the current block. For example, if the current block is a non-square block with a width greater than the height, some directional modes (eg, 2 to 15) may be converted to wide angle intra prediction modes 67 to 80. On the other hand, if the current block is a non-square block with a height greater than the width, some directional modes (e.g., numbers 53 to 66) may be converted to wide angle intra prediction modes between -1 and -14. there is.

The range of available wide-angle intra prediction modes can be adaptively determined depending on the width-to-height ratio of the current block. Table 1 shows the range of available wide-angle intra prediction modes according to the width and height ratio of the current block.

width/height	Wide angle intra prediction mode range available
W/H = 16	67~80
W/H = 8	67~78
W/H = 4	67~76
W/H = 2	67~74
W/H = 1	doesn't exist
W/H = 1/2	-1~-8
W/H = 1/4	-1~-10
W/H = 1/8	-1~-12
W/H = 1/16	-1~-14

Among the plurality of intra prediction modes, K candidate modes (most probable mode, MPM) can be selected. A candidate list including the selected candidate mode may be created. An index indicating one of the candidate modes belonging to the candidate list may be signaled. The intra prediction mode of the current block may be determined based on the candidate mode indicated by the index. As an example, the candidate mode indicated by the index may be set as the intra prediction mode of the current block. Alternatively, the intra prediction mode of the current block may be determined based on the value of the candidate mode indicated by the index and a predetermined difference value. The difference value may be defined as the difference between the value of the intra prediction mode of the current block and the value of the candidate mode indicated by the index. The difference value may be signaled through a bitstream. Alternatively, the difference value may be a value pre-defined in the video encoding/decoding device. Alternatively, the intra prediction mode of the current block may be a flag indicating whether a mode identical to the intra prediction mode of the current block exists in the candidate list. It can be decided based on . For example, when the flag is the first value, the intra prediction mode of the current block may be determined from the candidate list. In this case, an index indicating one of a plurality of candidate modes belonging to the candidate list may be signaled. The candidate mode indicated by the index may be set as the intra prediction mode of the current block. On the other hand, when the flag is the second value, one of the remaining intra prediction modes may be set as the intra prediction mode of the current block. The remaining intra prediction modes may refer to modes excluding candidate modes belonging to the candidate list among a plurality of pre-defined intra prediction modes. When the flag is the second value, an index indicating one of the remaining intra prediction modes may be signaled. The intra prediction mode indicated by the signaled index may be set as the intra prediction mode of the current block. The intra prediction mode of the chroma block may be selected from among the intra prediction mode candidates of a plurality of chroma blocks. To this end, index information indicating one of the intra prediction mode candidates of the chroma block can be explicitly encoded and signaled through a bitstream. Table 2 illustrates intra prediction mode candidates for chroma block.

index	Intra prediction mode candidates for chroma blocks
	Luma Mode: 0	Luma mode: 50	Luma Mode: 18	Luma Mode: 1	etc
0	66	0	0	0	0
One	50	66	50	50	50
2	18	18	66	18	18
3	One	One	One	66	One
4	DM

In the example of Table 2, Direct Mode (DM) means setting the intra prediction mode of the luma block existing at the same location as the chroma block to the intra prediction mode of the chroma block. Meanwhile, a luma block that exists at the same location as a chroma block may be determined based on the location of the upper left sample or the center sample of the chroma block. For example, if the intra prediction mode (luma mode) of the luma block is number 0 (flat mode) and the index points to number 2, the intra prediction mode of the chroma block may be determined to be the horizontal mode (number 18). For example, if the intra prediction mode (luma mode) of the luma block is number 1 (DC mode) and the index indicates number 0, the intra prediction mode of the chroma block may be determined as planar mode (number 0).

As a result, the intra prediction mode of the chroma block can also be set to one of the intra prediction modes shown in FIG. 4 or FIG. 5. The intra prediction mode of the current block may be used to determine the reference line of the current block, in which case step S310 may be performed before step S300.

Referring to FIG. 3, intra prediction may be performed on the current block based on the reference line and intra prediction mode of the current block (S320).

Hereinafter, we will look at the intra prediction method for each intra prediction mode in detail with reference to FIGS. 6 to 8. However, for convenience of explanation, it is assumed that a single reference line is used for intra prediction of the current block. However, even when multiple reference lines are used, the intra prediction method described later may be applied in the same/similar manner.

Figure 6 illustrates a planar mode-based intra prediction method according to the present disclosure.

Referring to FIG. 6, T represents a reference sample located at the upper right corner of the current block, and L represents a reference sample located at the lower left corner of the current block. P1 can be generated through horizontal interpolation. As an example, P1 can be generated by interpolating T with a reference sample located on the same horizontal line as P1. P2 can be generated through interpolation in the vertical direction. As an example, P2 can be generated by interpolating L with a reference sample located on the same vertical line as P2. The current sample in the current block can be predicted through the weighted sum of P1 and P2, as shown in Equation 1 below.

In Equation 1, the weights α and β can be determined considering the width and height of the current block. Depending on the width and height of the current block, weights α and β may have the same value or different values. If the width and height of the current block are the same, the weights α and β can be set to be the same, and the prediction sample of the current sample can be set to the average value of P1 and P2. If the width and height of the current block are not the same, the weights α and β may have different values. For example, if the width is greater than the height, a smaller value can be set to the weight corresponding to the width of the current block, and a larger value can be set to the weight corresponding to the height of the current block. Conversely, if the width is greater than the height, a larger value can be set to the weight corresponding to the width of the current block, and a smaller value can be set to the weight corresponding to the height of the current block. Here, the weight corresponding to the width of the current block may mean β, and the weight corresponding to the height of the current block may mean α.

Figure 7 shows a DC mode-based intra prediction method according to the present disclosure.

Referring to FIG. 7, the average value of neighboring samples adjacent to the current block can be calculated, and the calculated average value can be set as the predicted value of all samples in the current block. Here, the surrounding samples may include the top reference sample and the left reference sample of the current block. However, depending on the type of the current block, the average value may be calculated using only the top reference sample or the left reference sample. For example, if the width of the current block is greater than the height, the average value can be calculated using only the top reference sample of the current block. Alternatively, if the ratio of the width and height of the current block is greater than or equal to a predetermined threshold, the average value can be calculated using only the top reference sample of the current block. Alternatively, if the ratio of the width and height of the current block is less than or equal to a predetermined threshold, the average value can be calculated using only the upper reference sample of the current block. On the other hand, if the width of the current block is smaller than the height, the average value can be calculated using only the left reference sample of the current block. Alternatively, if the ratio of the width and height of the current block is less than or equal to a predetermined threshold, the average value can be calculated using only the left reference sample of the current block. Alternatively, if the ratio of the width and height of the current block is greater than or equal to a predetermined threshold, the average value can be calculated using only the left reference sample of the current block.

Figure 8 illustrates a directional mode-based intra prediction method according to the present disclosure.

If the intra prediction mode of the current block is a directional mode, projection can be performed to a reference line according to the angle of the directional mode. If a reference sample exists at the projected position, the reference sample can be set as the prediction sample of the current sample. If a reference sample does not exist at the projected position, a sample corresponding to the projected position may be generated using one or more surrounding samples adjacent to the projected position. As an example, interpolation may be performed based on two or more neighboring samples in both directions based on the projected position, thereby generating a sample corresponding to the projected position. Alternatively, one surrounding sample adjacent to the projected position can be set as the sample corresponding to the projected position. At this time, among a plurality of neighboring samples adjacent to the projected position, the neighboring sample closest to the projected position may be used. The sample corresponding to the projected position can be set as the predicted sample of the current sample.

Referring to FIG. 8, in the case of the current sample B, when projection is performed from that position to the reference line according to the angle of the intra prediction mode, a reference sample exists at the projected position (i.e., a reference sample at an integer position, R3 ). In this case, the reference sample of the projected position can be set as the predicted sample of the current sample B. In the case of the current sample A, when projection is performed from that position to the reference line according to the angle of the intra prediction mode, there is no reference sample (i.e., reference sample at the integer position) at the projected position. In this case, interpolation may be performed based on surrounding samples (e.g., R2 and R3) neighboring the projected position to generate a sample (r) of the fractional position. The sample (r) at the generated fractional position can be set as the predicted sample of the current sample A.

Figure 9 shows a method for deriving samples of fractional positions.

In the example of Figure 9, the variable h refers to the vertical distance (i.e., vertical distance) from the position of the predicted sample A to the reference sample line, and the variable w refers to the horizontal distance from the position of the predicted sample A to the fractional position sample. (i.e., horizontal distance). Additionally, the variable θ refers to an angle predefined according to the directionality of the intra prediction mode, and the variable x refers to the fractional position.

The variable w can be derived as in Equation 2 below.

Then, by removing the integer positions from the variable w, finally, the fractional positions can be derived.

Fractional position samples can be generated by interpolating adjacent integer position reference samples. As an example, the integer position reference sample R2 and the integer position reference sample R3 may be interpolated to generate a fractional position reference sample at the x position.

In deriving fractional position samples, a scaling factor can be used to avoid real numbers. For example, when the scaling factor f is set to 32, the distance between neighboring integer reference samples may be set to 32 instead of 1, as in the example shown in (b) of FIG. 8.

Additionally, the tangent value for the angle θ determined according to the directionality of the intra prediction mode can also be scaled up using the same scaling factor (eg, 32).

Figures 10 and 11 show that the tangent value for the angle is scaled by 32 times for each intra prediction mode.

FIG. 10 shows the scaled results of tangent values for the non-wide angle intra prediction mode, and FIG. 11 shows the scaled results of the tangent values for the wide angle intra prediction mode.

If the tangent value (tanθ) to the angle value in intra prediction mode is positive, reference samples belonging to the top line of the current block (i.e., top reference samples) or reference samples belonging to the left line of the current block (i.e., left Intra prediction can be performed using only one of the reference samples. Meanwhile, when the tangent value for the angle value of the intra prediction mode is negative, both the reference samples located at the top and the reference samples located on the left are used.

At this time, to simplify implementation, the left reference samples are projected upward, or the upper reference samples are projected to the left, the reference samples are arranged in a 1D array, and intra prediction is performed using the reference samples in the 1D array. You may.

Figure 12 is a diagram illustrating an intra prediction aspect when the directional mode is one of modes 34 to 49.

When the intra prediction mode of the current block is one of modes 34 to 49, intra prediction is performed using not only the top reference samples of the current block but also the left reference samples. At this time, as in the example shown in FIG. 12, the reference sample located on the left of the current block can be copied to the position of the top line, or the reference samples located on the left can be interpolated to generate the reference sample of the top line.

For example, if you want to obtain a reference sample for the A position at the top of the current block, considering the directionality of the intra prediction mode of the current block, projection can be performed from the A position on the top line to the left line of the current block. . If the projected position is called a, the value corresponding to the position a can be copied, or a fractional position value corresponding to a can be created and set as the value of the A position. For example, if the position a is an integer position, the value of the position A can be generated by copying the integer position reference sample. On the other hand, when the a position is a fractional position, the reference sample located above the a position and the reference sample located below the a position can be interpolated, and the interpolated value can be set as the value of the A position. Meanwhile, at position A at the top of the current block, the direction projected to the left line of the current block may be parallel to and opposite to the direction of the intra prediction mode of the current block.

Figure 13 is a diagram for explaining an example of generating an upper reference sample by interpolating left reference samples.

In Figure 13, the variable h represents the horizontal distance between the position A on the top line and the position A on the left line. The variable w represents the vertical distance between the position A on the top line and the position A on the left line. Additionally, the variable θ refers to an angle predefined according to the directionality of the intra prediction mode, and the variable x refers to the fractional position.

The variable h can be derived as in Equation 3 below.

Then, by removing the integer positions from the variable h, finally, the fractional positions can be derived.

In deriving fractional position samples, a scaling factor can be used to avoid real numbers. As an example, the tangent value for variable θ can be scaled using the scaling factor f1. Here, since the direction projected to the left line is parallel and opposite to the directional prediction model, the scaled tangent value shown in FIGS. 10 and 11 may be used.

When the scaling factor f1 is applied, Equation 3 can be modified and used as shown in Equation 4 below.

In the same way as above, a 1D reference sample array can be constructed only with reference samples belonging to the top line. As a result, intra prediction for the current block can be performed using only the upper reference samples composed of a 1D array.

Figure 14 shows an example in which intra prediction is performed using reference samples arranged in a 1D array.

As in the example shown in FIG. 14, by projecting left reference samples to generate top reference samples, prediction samples of the current block can be obtained using only reference samples belonging to the top line.

Contrary to what is shown in FIGS. 12 and 14, the top reference sample may be projected onto the left line to form a 1D reference sample array using only reference samples belonging to the left line. Specifically, for modes 19 to 33 among the directional modes in which the tangent value (tanθ) for the angle of the directional mode is negative, reference samples belonging to the top line are projected to the left line to generate a left reference sample. You can.

As described above, pictures can be encoded/decoded in block units. As an example, a picture may be divided into blocks of a predetermined size. A block of a predetermined size may be called a coding tree block (Coding Tree Block, or coding tree unit) or a reference block. Information indicating the size of the reference block may be signaled through a bitstream. As an example, information indicating the size of a coding tree block may be encoded through a sequence parameter set or a picture header.

Thereafter, blocks of various sizes may be added to the standard block based on at least one of the tree structures predefined within the standard block. Then, encoding/decoding processing such as prediction, transformation, quantization, and/or entropy coding may be performed on each divided block. Each of the divided blocks may be a coding block, prediction block, or transform block.

Meanwhile, in the case of a color picture, the luma picture and the chroma picture are encoded/decoded respectively. At this time, chroma pictures generally have similar characteristics to luma pictures. That is, the characteristics of chroma samples within a chroma picture tend to be similar to the characteristics of luma samples at the same location within a luma picture. Using this property, the division structure for the standard block can be independently determined for only one standard component among the color components, and the tree structure determined in the existing component can be applied in the same way to other components. In this way, a division structure in which the tree division structure of the standard component is directly applied to other components may be called a single tree structure.

Meanwhile, in embodiments described later, it is assumed that the reference component is a luma component. That is, it is assumed that the tree division structure is independently applied to the luma component, while the tree division structure of the luma component is applied as is to the chroma component.

Information about the tree division structure in the reference component may be explicitly encoded and signaled. As an example, tree partition structure information about the luma reference block may be encoded and signaled. On the other hand, for the chroma reference block, the encoding/decoding of the tree division structure information is omitted, and the tree division structure information in the luma reference block can be used in the same way.

Figure 15 is a diagram for explaining the encoding/decoding order when a single tree structure is used.

As in the example shown in FIG. 15, when a single tree structure is applied, the tree division structure for the luma reference block can be directly applied to the chroma reference block. Accordingly, the division form for the chroma reference block is the same as the division form for the luma reference block.

Meanwhile, encoding/decoding may be performed in an alternate order of luma components and chroma components. As an example, in Figure 15, numbers written in each leaf node block indicate the encoding/decoding order.

Meanwhile, a division structure in which the tree structure for each color component is independently determined can be called a dual tree structure. When a dual tree structure is applied, tree partition structure information for the luma component and tree partition structure information for the chroma component may be independently encoded and signaled.

Figure 16 is a diagram for explaining the encoding/decoding order when a dual tree structure is used.

When a dual tree structure is applied, after encoding/decoding for one component is completed, encoding/decoding for the next component may begin.

As an example, after encoding/decoding of the luma component picture is completed, encoding/decoding of the chroma component picture may be performed.

Alternatively, the encoding/decoding order between components can be set on a reference block basis. As an example, after encoding/decoding for the luma component reference block is completed, encoding/decoding for the chroma component reference block is performed. can be performed.

In the example shown in FIG. 16, the numbers written within the block indicate the encoding/decoding order. In the example of FIG. 16, it is illustrated that the reference block of the luma component is encoded/decoded before the reference block of the chroma component.

Prediction of the chroma block can be performed using the restored luma block. The above prediction model using different color components may be called a Cross Component Linear Model (CCLM). When CCLM is applied, the process of deriving the intra prediction mode of the chroma block based on the intra prediction mode of the luma block can be omitted.

Figure 17 is a flowchart showing a method of predicting a chroma block using a restored luma block.

Referring to FIG. 17, first, for prediction of a chroma block, prediction parameters can be derived (S1710). At this time, prediction parameters may be derived in different ways depending on the video format of the picture. The video format indicates the chroma subsampling rate and can be determined as one of 4:4:4, 4:2:2, or 4:2:0.

If the video format is not 4:4:4, the luma block is downsampled and adjusted to match the size of the chroma block.

Figures 18 to 20 show examples of down-sampling a luma block.

For convenience of explanation, it is assumed that the video format is 4:2:0.

When the video format is 4:2:0, as in the example shown in FIG. 18, the size of the chroma block corresponding to the 4x4 luma block is 2x2. In this case, a 4x4 luma block can be reduced to a 2x2 size by applying a down-sampling filter to the luma block. The following equation 5 shows the application aspect of the down-sampling filter.

In Equation 5, Downsampled_Luma refers to the sample value within the down-sampled luma block, and Luma refers to the value of the luma sample before down-sampling. For example, Luma[0][0] may indicate the location of the upper left sample in the luma block before downsampling. Since the size of the down-sampled luma block is 2x2, the variables w and h representing the coordinates of the sample can each have values in the range from 0 to 1.

When applying the down-sampling filter according to Equation 5, the value of the down-sampled luma sample can be obtained by applying a cross-shaped down-sampling filter to the luma samples. As an example, the value of the down-sampled luma sample at the (0, 0) position is the luma sample at the (0, 0) position, the top luma sample at the (0, 0) position, and the left luma sample at the (0, 0) position. It can be obtained by applying a down-sampling filter to the luma sample, the bottom luma sample at the (0, 0) position, and the right luma sample at the (0, 0) position.

A down-sampling filter of a different type from that shown in FIG. 19 may be applied. For example, a down-sampled luma sample can be obtained by applying a 1D filter, a rectangular or square filter. 1D filters can be 1x3 or 3x1 in size, rectangular filters can be 2x3 or 3x2 in size, and square filters can be 2x2 or 3x2 in size.

The type of filter may be predefined in the encoder and decoder.

Alternatively, the shape of the filter may be adaptively determined based on at least one of the size/shape of the current block, the intra prediction mode applied to the luma block, whether the position of the chroma sample matches the position of the luma sample, or the image format. .

Alternatively, information indicating one of a plurality of filter candidates may be encoded and signaled.

Alternatively, depending on the down sampling location, the filter type may be different. For example, a 1D filter or a rectangular filter may be applied to luma samples located at the border of a luma block, while a cross-shaped filter may be applied to luma samples not located at the border of the luma block.

As shown in FIG. 19, a down-sampling filter can be applied to a location where both the x-axis coordinate and the y-axis coordinate are even numbers.

The application position of the down-sampling filter may be set differently from that shown in FIG. 19. Figure 20 shows various examples of application positions of the down-sampling filter.

After pre-defining a plurality of candidates related to the down sampling application location, one of the plurality of candidates may be selected. As an example, the examples in (a) to (d) of FIG. 20 may be defined as a plurality of candidates, and then index information indicating one of the plurality of examples may be encoded and signaled.

Alternatively, one of a plurality of candidates may be selected based on whether the location of the chroma sample matches the location of the luma sample.

A down-sampling filter can also be applied to reference samples around the luma block. Here, the reference sample may represent a previously restored sample. Specifically, a down-sampling filter may be applied to at least one of the top reference area adjacent to the top of the luma block or the left reference area adjacent to the left, to obtain a down-sampled luma reference sample.

The same number of down-sampled luma reference samples as the number of reference samples included in the reference area of the chroma block can be obtained.

Meanwhile, the reference area of the luma block may be referred to as a luma reference area, and the reference area of the chroma block may be referred to as a chroma reference area.

The inter-component prediction mode can be divided into a top inter-component prediction mode, a left inter-component prediction mode, and a top and left inter-component prediction mode, depending on the configuration of the reference area. When the top inter-component prediction mode is selected, the reference area of each luma block and chroma block consists of only the top reference area. When the left inter-component prediction mode is selected, the reference area of each luma block and chroma block consists of only the left reference area. When the top and left inter-component prediction mode is selected, the reference areas of each luma block and chroma block may be composed of a top reference area and a left reference area.

Information indicating which of the top inter-component prediction mode, left inter-component prediction mode, and top and left component prediction mode has been applied to the current block may be explicitly encoded and signaled. As an example, index information indicating the type of inter-component prediction mode may be encoded and signaled.

Alternatively, based on at least one of the size/shape of the current block, whether the current block borders a CTU or picture boundary, or an intra prediction mode applied to the luma block, a top inter-component prediction mode, a left inter-component prediction mode, and a top and left One of the inter-component prediction modes may be selected.

For convenience of explanation, in the embodiment described later, it is assumed that the reference areas of each luma block and chroma block include an upper reference area and a left reference area.

The type of down-sampling filter applied to the reference area of the luma block may be the same as the down-sampling filter applied to the luma block. Alternatively, the type of down-sampling filter applied to the reference area of the luma block may be different from the down-sampling filter applied to the luma block. Alternatively, the form of the down-sampling filter applied to the upper reference area of the luma block may be different from the form of the down-sampling filter applied to the left reference area of the luma block.

Meanwhile, the location where down sampling is applied within the reference area may be predefined in the encoder and decoder.

In other cases, the decoder may independently determine the location in the reference region where down sampling is applied in the same way as the encoder.

FIG. 21 is a diagram for explaining an example of a location where down sampling is applied.

When the video format is 4:2:0, a 1x1 chroma block corresponds to a 2x2 luma block. Accordingly, a down-sampling filter can be applied to one position among the four luma reference samples to derive a down-sampled luma reference sample corresponding to the chroma reference sample.

When four luma reference samples corresponding to one chroma reference sample are referred to as A to D, down sampling can be performed on each of the positions A to D in the reference area, and then the cost for each position can be calculated. Here, the cost for a specific location is the sum of the difference between the down-sampled luma reference sample obtained by applying a down-sampling filter centered on that location and the chroma reference sample corresponding to that location, or the sum of the absolute value of the difference. It can be derived based on In this way, the cost derived based on the sum of the absolute values of the differences may be called SAD (Sum of Difference).

Thereafter, the location with the lowest cost is determined as the optimal location, and a prediction parameter derivation process, which will be described later, can be performed using the down-sampled luma samples from the optimal location.

Alternatively, information indicating one of multiple positions to which a down-sampling filter can be applied may be encoded and signaled. For example, in the example shown in FIG. 21, an index indicating one of positions A to D may be encoded and signaled. To this end, the encoder obtains a prediction parameter for each of a plurality of positions to which a down-sampling filter can be applied, and encodes and signals an index indicating the position used to derive the optimal prediction parameter among the plurality of prediction parameters. You can. Here, the optimal prediction parameters can be derived by the cost of each prediction parameter or RDO (Rate Distortion Optimization).

Meanwhile, determining the optimal down-sampling application location within the upper reference area may be independent of determining the optimal down-sampling application location within the left reference area. In this case, the optimal down sampling application location within the upper reference area and the optimal down sampling application location within the left reference area may be different.

Using down-sampled luma reference samples and reference samples of the chroma block, prediction parameters for the chroma block can be derived. Prediction parameters may include weight α and offset β. Prediction parameters can be derived using the least square method or the like.

Alternatively, the weight α offset β can be derived based on the linearity of the maximum and minimum values of the down-sampled luma reference samples and the maximum and minimum values of the chroma reference samples.

At this time, prediction parameters may be derived using only chroma reference samples at predefined positions and down-sampled luma reference samples corresponding thereto. In this case, the process of deriving prediction parameters is simplified, and complexity in the encoder and decoder can be reduced. As an example, a prediction parameter can be derived using chroma reference samples at the positions illustrated in Equation 6 below.

In the above example, W and H represent the width and height of the chroma block, respectively. According to the above example, a prediction parameter can be derived using four chroma reference samples and four down-sampled luma reference samples corresponding thereto.

Prediction parameters can also be obtained using reference samples in positions different from the above example. As an example, the positions of reference samples may be determined as shown in Equation 7 and Equation 8 below.

After pre-defining a plurality of candidates for the positions of reference samples, one of the plurality of candidates can be selected. As an example, each of the examples of Equation 6 to Equation 8 listed above may be set as a location candidate, and then reference samples may be selected according to one of the plurality of location candidates.

Information for selecting one of a plurality of location candidates may be encoded and signaled. As an example, an index indicating one of a plurality of location candidates can be encoded and signaled.

Alternatively, one of a plurality of location candidates may be adaptively selected based on at least one of the size/shape of the current block, color format, or whether the location of the chroma sample matches the location of the luma sample.

For example, if the current block is in a square shape, the prediction parameter can be derived using the position candidate in Equation 6. On the other hand, if the current block is non-square, the prediction parameter can be derived using the position candidate of Equation 7 or Equation 8. As an example, if the current block is in a non-square form with a width greater than the height, the position candidate of Equation 7 can be used, and if the current block is in a non-square form with a height greater than the width, the position candidate of Equation 8 can be used. there is.

Once the prediction parameter is derived, a prediction sample of the chroma block can be obtained based on the down-sampled luma sample (S1720). As an example, a prediction sample of a chroma block can be obtained according to Equation 9 below.

In Equation 9, PredChroma represents a prediction sample of a chroma block, and Downsampled_Luma represents a down-sampled luma sample at a position corresponding to the chroma prediction sample.

Meanwhile, if the video format is 4:4:4, the above-described down-sampling process can be omitted. That is, when the video format is 4:4:4, the process of performing down-sampling on restored samples within the luma block and the process of performing down-sampling on reference samples of the luma block can be omitted.

As another example, regardless of the video format, a down-sampling filter may not be applied to the reference area of the luma block. That is, when deriving a prediction parameter, instead of using the minimum and maximum values among the down-sampled luma reference samples, the minimum and maximum values among the luma reference samples may be used.

Meanwhile, pictures can be encoded/decoded on a block basis. As an example, a picture may be divided into blocks of a predetermined size. A block of a predetermined size may be called a coding tree block (Coding Tree Block, or coding tree unit) or a reference block. Information indicating the size of the reference block may be signaled through a bitstream. As an example, information indicating the size of a coding tree block may be encoded through a sequence parameter set or a picture header.

Thereafter, blocks of various sizes may be added to the standard block based on at least one of the tree structures predefined within the standard block. Then, encoding/decoding processing such as prediction, transformation, quantization, and/or entropy coding may be performed on each divided block. Each of the divided blocks may be a coding block, prediction block, or transform block.

Meanwhile, in the case of a color picture, the luma picture and the chroma picture are encoded/decoded respectively. At this time, chroma pictures generally have similar characteristics to luma pictures. That is, the characteristics of chroma samples within a chroma picture tend to be similar to the characteristics of luma samples at the same location within a luma picture. Using this property, the division structure for the standard block can be independently determined for only one standard component among the color components, and the tree structure determined in the existing component can be applied in the same way to other components. In this way, a division structure in which the tree division structure of the standard component is directly applied to other components may be called a single tree structure.

Meanwhile, in embodiments described later, it is assumed that the reference component is a luma component. That is, it is assumed that the tree division structure is independently applied to the luma component, while the tree division structure of the luma component is applied as is to the chroma component.

As another example, encoding/decoding information for the chroma block may be derived with reference to the co-located luma block, and then encoding/decoding for the chroma block may be performed based on the derived encoding/decoding information. At this time, in the single tree structure, the division structure of the luma reference block and the division structure of the chroma reference block are the same, so the luma block at the same location as the chroma block can be clearly specified.

On the other hand, in the dual tree structure, the division structure of the luma reference block and the division structure of the chroma reference block are independent. Under the dual tree structure, the area within the luma picture corresponding to the area occupied by the chroma block, or the upper node luma block including the area within the luma picture, can be treated as a luma block at the same location as the chroma block.

Alternatively, a luma block including an area within the luma picture corresponding to the area occupied by the chroma block can be set as a co-located luma block. At this time, if the area within the luma picture is included in a plurality of luma blocks, a representative block among the plurality of luma blocks may be determined as the co-located luma block. Here, the representative block may be a luma block occupying the largest portion of the area, a block with the largest size among luma blocks, or a luma block including a reference position sample in an area within the luma picture. The reference position may be the upper left position, the central position, the upper right position, the lower left position, or the lower right position within the area.

Alternatively, if a plurality of luma blocks are included in an area within the luma picture corresponding to the area occupied by the chroma block, a representative block among the plurality of luma blocks may be determined as the co-located luma block. Here, the representative block may be a luma block occupying the largest portion of the area, a block with the largest size among luma blocks, or a luma block including a reference position sample in an area within the luma picture.

Alternatively, if a plurality of luma blocks are included in an area within the luma picture corresponding to the area occupied by the chroma block, the upper node luma block including the plurality of luma blocks may be determined as the co-located luma block.

Alternatively, a reference position for determining a chroma reference block and a co-located luma block may be defined, and the co-located luma block may be determined based on the reference position. The reference position may be the upper left position, center position, upper right position, lower left position, or lower right position of the chroma block.

For example, when the reference position is the central position of the chroma block, a luma block including a luma sample corresponding to the central position of the chroma block may be determined as the luma block at the same position as the chroma block.

In the dual tree structure, when determining the division structure of the chroma block, information can be referenced to the division structure of the co-located luma block. Whether to divide the chroma block into a plurality of partitions may be determined based on whether the co-located luma block is divided into a plurality of partitions.

Figures 22 and 23 are diagrams for explaining an example of determining the division structure of a chroma block with reference to a co-located luma block in a dual tree structure.

As explained with reference to FIG. 16, when the dual tree structure is applied, after encoding/decoding of the luma component is completed, encoding/decoding of the chroma component is performed. Accordingly, when encoding/decoding for the chroma block starts, encoding/decoding for the luma block at the same location is completed.

Accordingly, encoding/decoding information for the chroma block can be derived by referring to encoding/decoding information of the co-located luma block. As a specific example, when encoding/decoding a chroma block, the division information of the luma block can be directly applied to the chroma block. Here, the partition information may include at least one of whether the block is partitioned or the partition type of the block. Here, the partition type may relate to at least one of the number of partitions created by dividing the block, whether or not it is asymmetrically divided, or the direction in which the block is divided.

For example, in the example shown in FIG. 22, the partition type of the first chroma block may be set to be the same as the partition type of the luma block existing at the same location. Specifically, in the example shown in FIG. 22, the co-located luma block for the first chroma block is the upper node luma block divided into luma block B and luma block C. As in the illustrated example, the co-located luma block is shown to be divided into two in the horizontal direction. Accordingly, the first chroma block can also be divided into two horizontally, like the luma block.

Not only the division information of the luma block, but also the prediction information of the luma block can be referred to when encoding/decoding the chroma block. That is, prediction information for each partition created by dividing the chroma block may be set to be the same as the co-located luma block for each partition.

As an example, in the example shown in FIG. 22, the prediction mode of each of the two partitions (i.e., b and c) created by dividing the chroma block is the prediction mode of the two partitions (i.e., B and C) each prediction mode may be set identically. Here, the prediction mode may refer to at least one of an encoding mode, an intra prediction mode, or an inter prediction mode. The encoding mode indicates whether intra prediction or inter prediction was applied to the block. The intra prediction mode may be information indicating one of non-directional intra prediction modes and directional prediction modes used when performing intra prediction. The inter prediction mode may be information indicating at least one of a motion information merging mode, a motion vector prediction mode, and a template matching mode.

Likewise, the partition type for the second chroma block may be set to be the same as the partition type of the luma block existing at the same location. In the example shown in FIG. 22, quad type division is applied to the co-located luma block of the second chroma block. Accordingly, quad type division will be applied to the second chroma block as well.

In addition, the prediction mode used in the partitions created by dividing the chroma block (i.e., e, f, g, and h) is the prediction mode used in the partitions created by dividing the luma block (i.e., E, F, G, and H). It can be set the same as the prediction mode of .

Meanwhile, information indicating whether the encoding/decoding information of the chroma block is set to be the same as that of the co-located luma block may be encoded and signaled. The information may be a 1-bit flag.

As an example, a first flag indicating whether to set the partition type of the chroma block to be the same as that of the co-located luma block may be encoded and signaled. If the first flag is true, encoding/decoding of partition information for the chroma block is omitted, and the partition type of the chroma block may be set to be the same as that of the co-located luma block.

As an example, a second flag indicating whether to set the prediction mode of the chroma block to be the same as that of the co-located luma block may be encoded and signaled. If the second flag is true, encoding/decoding of the prediction mode for the chroma block may be omitted, and the prediction mode of the chroma block may be set to be the same as that of the co-located luma block.

Meanwhile, in the example shown in FIG. 22, the chroma block and the co-located luma block are illustrated as having the same shape and corresponding sizes. Here, having sizes that correspond to each other indicates that the size of the chroma block and the size of the co-located luma block correspond to each other when considering the chroma format. For example, when the chroma format is 4:4:4, the size of the co-located luma block corresponding to the size of the chroma block may be the same size as the chroma block. On the other hand, when the chroma format is 4:2:0, the size of the co-located luma block corresponding to the size of the chroma block may be twice the width and height of the chroma block, respectively.

Meanwhile, for a chroma block, it may be adaptively determined whether prediction and transformation are performed sequentially. As an example, when at least one of the partition information and/or prediction information of the co-located luma block is applied to the chroma block, after prediction and conversion for the partition that precedes the encoding/decoding order within the chroma block is completed, prediction for the next partition is performed. and conversion may be performed. For example, in the example shown in FIG. 22, the encoding/decoding order for each of the four partitions included in the second chroma block may be as follows.

Perform prediction on subblock E → Perform transformation and quantization (and restoration or entropy coding) on subblock E → Perform prediction on subblock F → Perform transformation and quantization (and restoration or entropy coding) on subblock F → Perform prediction on subblock G → Perform transformation and quantization (and restoration or entropy coding) on subblock G → Perform prediction on subblock H → Perform transformation and quantization (and restoration or entropy coding) on subblock H

Alternatively, prediction may be performed on each of the sub-blocks first, and then, after prediction on the sub-blocks is completed, transformation on the hub blocks may be performed.

Unlike the example shown in FIG. 22, there may be cases where the shape of the chroma block and the luma block existing at the same location are not the same, or the size of the chroma block does not correspond to the size of the luma block at the same location. Even in this case, the prediction information of the chroma block can be set to be the same as the prediction information of the co-located luma block.

As an example, the partition type of the chroma block can be determined by referring to the partition type of the co-located luma block. When the chroma block is divided into a plurality of partitions, the prediction mode of each partition may be set to be the same as the prediction mode of the partition in the luma block corresponding to each of the partitions.

For example, in FIG. 23, the luma block corresponding to the first chroma block is luma block B. In this case, the intra-screen prediction mode of luma block B may be applied as is to the first chroma block.

The second chroma block has a size corresponding to the sum of a plurality of luma blocks (ie, E, F, G, H, and J). In this case, chroma block 2 can be divided into a plurality of luma blocks in the same form. That is, after dividing the chroma block into two horizontally, quad division can be applied to the upper chroma block created by the two divisions. Then, the prediction mode of each of the partitions (i.e., e, f, g, h, j) created by dividing the chroma block is changed to the prediction mode of each of the corresponding luma blocks (i.e., E, F, G, H, J). It can be set the same as prediction mode.

Meanwhile, it is possible to adaptively determine whether to apply the encoding/decoding information of the luma block to the chroma block by comparing the sizes of the chroma block and the co-located luma block. As an example, when the size of the chroma block does not correspond to the size of the luma block at the same location, in deriving the encoding/decoding information (e.g., at least one of the partition type or prediction mode) of the chroma block, the encoding/decoding information of the luma block is You can set it so that it does not refer to decryption information.

For example, in the example shown in FIG. 22, the size of the luma block at the same location as the first chroma block corresponds to the size of the first chroma block. In this case, it may be allowed to set the encoding/decoding information of the luma block to the encoding/decoding information of the chroma block.

On the other hand, in the example shown in FIG. 23, the size of the luma block at the same location as the first chroma block does not correspond to the size of the first chroma block. In this case, setting the encoding/decoding information of the luma block to the encoding/decoding information of the chroma block may not be permitted.

As another example, when encoding/decoding a chroma block, whether to divide the chroma block and/or the encoding/decoding order may be determined based on the directionality of luma samples.

Figure 24 shows an example in which encoding/decoding information for a chroma block is determined based on the directionality of luma samples.

In Figure 24, the luma block at the same location as the chroma block is indicated as block A.

Once the co-located luma block is determined, the direction for the co-located luma block is determined based on luma samples included in the co-located luma block. Specifically, after applying a plurality of intra prediction modes to the co-located luma block, the intra prediction mode with the lowest cost among the plurality of intra prediction modes may be selected. At this time, intra prediction can be performed by assuming that neighboring samples adjacent to the co-located luma block (i.e., block A) are reference samples. The cost for each of the intra prediction modes is calculated by subtracting the predicted value generated through intra prediction from the restored (or original) sample included in the co-located luma block (i.e., block A), and then calculating the cost for each of the intra-prediction modes at all locations within the co-located luma block. It can be derived by summing the absolute difference values of . Afterwards, the intra prediction mode with the lowest cost is set to the directionality of the co-located luma block (i.e., block A).

Specifically, for block A, if the intra prediction mode with the optimal cost is one of modes 2 to 33 in FIG. 4, the directionality of block A may be set to the left. Alternatively, for block A, if the intra prediction mode with the optimal cost is one of modes 34 to 66 of FIG. 4, the directionality of block A may be determined to be upper.

Meanwhile, when encoding/decoding a chroma block, in the case of a co-located luma block (i.e., block A), previously restored samples may exist not only in the top and left areas, but also in the right and bottom areas. Accordingly, in addition to the predefined intra prediction modes (i.e., the intra prediction modes shown in FIG. 4), extended directional intra prediction modes using the right and bottom reference samples (i.e., the directional intra prediction modes shown in FIG. 4) The directionality of the co-located luma block may be determined by additionally checking prediction modes and intra prediction modes in the opposite direction.

Meanwhile, to simplify implementation, block A and surrounding samples adjacent to block A can be rotated clockwise by 180 degrees, and then the cost for each of the predefined intra prediction modes can be calculated. When block A and surrounding samples are rotated 180 degrees clockwise, lower reference samples may be set as upper reference samples, and right reference samples may be set as left reference samples. After selecting the mode with the lowest cost among the predefined intra prediction modes, the selected intra prediction mode is rotated 180 degrees in the opposite direction (i.e. counterclockwise) to determine the optimal intra prediction mode for the right/bottom direction. You can decide.

Then, by comparing the cost between the optimal intra prediction mode for the left/top direction selected when block A is not rotated and the optimal intra prediction mode for the right/bottom direction selected when block A is rotated, block A direction can be determined.

As an example, the optimal intra prediction mode for block A is derived when block A is rotated 180 degrees, and if the optimal intra prediction mode is one of modes 2 to 33 in FIG. 4, block A The direction of can be determined to the right. Alternatively, if the optimal intra prediction mode for block A is derived when block A is rotated 180 degrees, and the optimal intra prediction mode is one of modes 34 to 66 of FIG. 4, the The direction can be determined to be downward.

It is also possible to determine the optimal intra prediction mode for the co-located luma block by setting restored samples located at the boundary within the co-located luma block as reference samples.

Figure 25 shows an example of using restored samples in the same location luma block as reference samples.

In the example shown in FIG. 25, w0, w1, h0, and h1 respectively represent areas containing restored samples used as reference samples in the co-located luma block. By performing intra prediction on the remaining area excluding the area containing the reference samples, the optimal intra prediction mode for the co-located luma block can be derived.

As in the above example, the optimal intra prediction mode for the co-location luma block can be determined by calculating the cost for each of the predefined intra prediction modes and/or the extended direction intra prediction modes.

Alternatively, for simplicity, when determining the directionality of the co-located luma block, the intra prediction mode of the co-located luma block may be used. That is, the directionality of the co-located luma block can be determined based on the intra prediction mode used to encode/decode the co-located luma block.

Alternatively, the directionality of the co-location luma block may be determined by classifying intra prediction modes into a plurality of groups and calculating a cost for each group.

Figure 26 shows an example in which intra prediction modes are classified into a plurality of groups.

As in the example shown in FIG. 26, predefined intra prediction modes and extended intra prediction modes in the opposite direction to the directional intra prediction modes among the predefined intra prediction modes can be divided into a plurality of groups. there is.

Thereafter, for each group, the cost of each intra prediction mode belonging to the group can be calculated and the calculated costs can be added to derive the cost of the group.

Thereafter, the cost for the left direction can be calculated by adding the costs of the groups corresponding to the left direction among the plurality of groups.

For example, in the example shown in FIG. 26, the cost for the left direction can be calculated by adding the costs of group a and group h. Additionally, the cost for the upper direction can be derived by combining the costs of group b and group c, and the cost for the right direction can be derived by combining the costs of group d and group e. Additionally, the cost for the bottom direction can be derived by combining the costs of group f and group g.

By comparing the costs of each of the top, right, bottom, and left directions, the direction with the lowest cost can be selected as the direction of the co-located luma block.

Alternatively, in the example shown in FIG. 26, the sum of the costs of group a and group b can be set as the cost for the left direction, and the sum of the costs of group c and group d can be set as the cost for the top direction. Additionally, the sum of the costs of group e and group f can be set as the cost for the right direction, and the sum of the costs of group g and group h can be set as the cost for the bottom direction.

Meanwhile, for the luma component, the directionality may be determined for each block of a predefined unit, and the directionality for the luma block at the same location may be set to be the same as the directionality of the block of the predefined unit to which the luma block at the same location belongs.

For example, after determining the directionality in units of coding tree units, the directionality of the coding tree unit to which the co-located luma block belongs can be determined as the directionality of the co-located luma block. At this time, the method of deriving the optimal directionality for the coding tree unit may follow the above-described embodiments.

Meanwhile, depending on the directionality of the co-located luma block, it may be determined whether to divide the chroma block into a plurality of partitions. For example, if the optimal intra prediction mode for a co-located luma block is a non-directional prediction mode, the chroma block may not be divided into a plurality of partitions. On the other hand, if the optimal intra prediction mode for the co-located luma block is the directional prediction mode, it may be decided to divide the chroma block into a plurality of partitions.

Alternatively, it may be independently determined whether to split the chroma block without considering the co-located luma block.

Once the directionality of the co-located luma block (i.e., block A) is determined, the encoding/decoding order of the plurality of sub-blocks in the chroma block can be determined based on the directionality of the co-located luma block. For example, when a chroma block is divided into four and encoded/decoded, the encoding/decoding start point and scan order for the four sub-blocks may be determined depending on the direction of the co-located luma block.

Figures 27 and 28 show the scan order for each sub-block in the chroma block according to the direction of the co-located luma block.

In Figures 27 and 28, it is assumed that the chroma block is divided into four partitions and encoded/decoded. Here, each partition may be at least one of a coding unit (ie, CU), prediction unit (ie, PU), or transformation unit (TU).

Unlike the illustrated example, this embodiment can be applied even when the chroma block is divided into a larger number of partitions or the chroma block is divided into a smaller number of partitions.

In Figure 27, the chroma block is illustrated as being divided into 4 partitions whose height and width are each 1/2 that of the chroma block.

Figure 27 (a) shows the encoding/decoding order of sub-blocks in the chroma block when the directionality of the co-located luma block is upper (i.e., when the optimal intra prediction mode is one of boards 34 to 66). It represents. Arrows indicate the encoding/decoding order of divided blocks. The starting point of the arrow indicates that it is encoded/decoded first, and the ending point of the arrow indicates that it is encoded/decoded later.

Figure 27(b) shows the encoding/decoding order of sub-blocks in the chroma block when the directionality of the co-located luma block is left (i.e., when the optimal intra prediction mode is one of modes 2 to 33). It represents.

Figure 27(c) shows the sub-blocks in the chroma block when the directionality of the co-located luma block is right (i.e., when the optimal intra prediction mode is 180 degrees opposite to one of modes 2 to 33). This shows the encoding/decoding order.

(d) of FIG. 27 shows, when the directionality of the co-located luma block is at the bottom (i.e., when the optimal intra prediction mode is 180 degrees opposite to one of modes 34 to 66), the sub-block within the chroma block This shows the encoding/decoding order.

Unlike the example shown in FIG. 27, the chroma block may be divided into partitions whose height or width is 1/4 of that of the chroma block.

At this time, the division direction of the chroma block may be determined according to the direction of the co-located luma block. For example, when the direction of the co-located luma block is top or bottom, the chroma block may be divided in the horizontal direction, as in the example shown in (a) and (b) of FIG. 28. On the other hand, when the direction of the co-located luma block is left or right, the chroma block may be divided in the vertical direction, as in the examples shown in (c) and (d) of Figures 28.

Meanwhile, in (a) of FIG. 28, when the directionality of the co-located luma block is upper (i.e., when the optimal intra prediction mode is one of boards 34 to 66), the sub-blocks in the chroma block are shown in (a). This shows the decoding order.

In (b) of Figure 28, when the direction of the co-located luma block is at the bottom (i.e., when the optimal intra prediction mode is 180 degrees opposite to one of modes 34 to 66), the sub-block within the chroma block This shows the encoding/decoding order.

Figure 28 (c) shows the encoding/decoding order of sub-blocks in the chroma block when the directionality of the co-located luma block is left (i.e., when the optimal intra prediction mode is one of modes 2 to 33). It represents.

Figure 28 (d) shows the sub-blocks in the chroma block when the directionality of the co-located luma block is right (i.e., when the optimal intra prediction mode is 180 degrees opposite to one of modes 2 to 33). This shows the encoding/decoding order.

Meanwhile, when encoding/decoding a chroma block, the directionality of a luma block existing at the same location may be determined, and rotation and/or flipping of the chroma block may be performed depending on the directionality of the luma block. Here, rotation and/or flipping may be performed in units of reference blocks, coding blocks, prediction blocks, or transform blocks.

When rotation and/or flipping are performed, encoding/decoding may be performed on the rotated and/or flipped chroma block.

Figure 29 shows an example in which flipping is performed on samples within a chroma block.

In Figure 29, the size of the chroma block is shown to be 4x4.

In the example shown in FIG. 29, p1 to p16 refer to samples in the chroma block.

Figure 29(a) shows a chroma block in a state where flipping has not been performed.

Figure 29(b) shows a chroma block on which flipping was performed in the horizontal direction, and Figure 29(c) shows a chroma block on which flipping was performed on the vertical direction.

Figure 29(d) shows a chroma block on which 180 degree flipping (i.e., horizontal flipping and vertical flipping) was performed.

If the direction for the co-located luma block is left, flipping may not be applied, as in the example shown in (a) of FIG. 29. On the other hand, when the direction for the same location luma block is upper, horizontal flipping may be applied, as in the example shown in (b) of FIG. 29. Additionally, when the direction for the co-located luma block is right, horizontal and vertical flipping can be applied simultaneously, as in the example shown in (d) of FIG. 29. On the other hand, when the direction for the same location luma block is lower, vertical flipping may be applied, as in the example shown in (c) of FIG. 29.

Alternatively, at least one of information indicating whether flipping is applied to the chroma block or information indicating the flipping direction may be encoded and signaled, rather than based on the direction of the co-located luma block.

Meanwhile, flipping for a chroma block may be set to be applied only when the co-located luma block is encoded with intra-screen prediction. As an example, if the co-located luma block is encoded with inter-screen prediction rather than intra-screen prediction, encoding/decoding of flipping-related information for the chroma block is omitted, and flipping will not be performed for the chroma block. You can.

As another example, the intra prediction mode of the chroma block may be determined based on the method for determining the optimal intra prediction mode for the co-located luma block described above. That is, the optimal intra prediction mode for the co-located luma block can be set to the intra prediction mode of the chroma block.

Alternatively, intra prediction may be performed on the chroma block based on the predetermined intra prediction mode of the chroma block and the intra prediction mode of the co-located luma block. Here, the intra prediction mode of the predetermined chroma block may be at least one of DM (Direct Mode), Horizontal Mode, Vertical Mode, DC Mode, Planar Mode, or Inter-Component Cross Prediction Mode. there is. Here, the DM mode may set the intra prediction mode used to encode/decode the co-located luma block to the intra prediction mode of the chroma block.

As an example, a first prediction block for the chroma block is obtained based on a predetermined intra prediction mode of the chroma block, and a second prediction block is obtained for the chroma block based on the optimal intra prediction mode of the co-located luma block. It can be obtained. Afterwards, the final prediction block for the chroma block can be obtained based on the average or weighted sum operation of the first prediction block and the second prediction block.

As another example, the final prediction block for the chroma block may be obtained by combining the inter prediction mode and the intra prediction mode. As an example, the chroma block is calculated through an average or weighted sum operation between an intra prediction block (or first prediction block) obtained by intra prediction and an inter prediction block (or second prediction block) obtained by inter prediction. The final prediction block can be generated.

At this time, intra prediction may be performed based on the optimal intra prediction mode of the co-location luma block or may be performed based on CCLM mode.

Additionally, inter prediction may be performed by at least one of a motion information merging mode, a motion vector prediction mode, or a template matching mode.

Meanwhile, when obtaining the final prediction block based on a weighted sum operation between the first prediction block and the second prediction block, the weights applied to each of the first prediction block and the second prediction block may always be set to the same value. . Alternatively, the weight applied to each of the first prediction block and the second prediction block may be determined using one of a plurality of weight candidates in a preset weight table. In this case, an index indicating one of a plurality of weight candidates may be encoded and signaled.

Alternatively, the weight applied to each of the first prediction block and the second prediction block may be determined by referring to the encoding mode of at least one neighboring block. For example, when the number of blocks encoded/decoded through intra prediction is greater than the number encoded/decoded by inter prediction, the weight applied to the first prediction block is greater than the weight applied to the second prediction block. You can have it. On the other hand, if the number of the plurality of neighboring blocks encoded/decoded through inter prediction is greater than the number encoded/decoded by intra prediction, the weight applied to the second prediction block will have a larger value than the weight applied to the first prediction block. You can. If the number encoded/decoded by intra prediction and the number encoded/decoded by inter prediction among a plurality of neighboring blocks are the same, the weight applied to the first prediction block and the weight applied to the second prediction block may be set to the same value. there is.

As another example, when deriving the intra prediction mode of a chroma block, when the DM mode is used, post-processing can be performed on prediction samples derived by performing intra prediction on the chroma block. The post-processing may be to correct prediction samples.

For example, when the intra-prediction mode of the chroma block is DM mode, the intra-prediction mode of the co-located luma block is set to the intra-prediction mode of the chroma block.

Meanwhile, the co-located luma block can be restored by combining a prediction sample obtained through intra prediction and a residual sample representing the difference between the original sample and the prediction sample. At this time, the DM mode can be set not to be used in the area within the chroma block corresponding to the area with a large prediction error within the co-located luma block.

Here, whether a region has a large prediction error can be determined by comparing the values of residual samples with a threshold value. Additionally, determination of an area with a large prediction error may be performed on a sample basis or a sub-block unit of a predefined size. As an example, the determination may be performed in units of subblocks of 2x2 or 4x4 size.

Figure 30 shows an example of determining whether or not an area has a large prediction error on a sub-block basis within the same location luma block.

As in the example shown in FIG. 30, the chroma block and the luma block existing at the same location as the chroma block may be divided into sub-blocks of a preset size. At this time, the size and/or shape of the subblock may be predefined in the encoder and decoder. Alternatively, the size and/or shape of the sub-block may be adaptively determined depending on the size and/or shape of the chroma block to be encoded/decoded.

Afterwards, for each sub-block in the co-located luma block, it is determined whether it is an area with a large prediction error. The determination may be performed by comparing the average of the absolute values of residual samples within the sub-block with a threshold value. For example, if the average value is greater than the threshold, the corresponding sub-block may be determined to be an area with a large prediction error. Otherwise, the sub-block may be determined to be an area with a small prediction error.

Alternatively, the minimum, maximum, or median value among absolute values of residual samples within a subblock may be compared with a threshold value to determine whether the corresponding subblock is an area with a large prediction error.

Here, the threshold value may be a value predefined in the encoder and decoder. Alternatively, information representing the threshold may be explicitly encoded and signaled.

Alternatively, the threshold value can be derived based on the average of the absolute values of residual samples within the same location luma block. As an example, the threshold value may be an average of the absolute values of residual samples in the luma block, or may be a value obtained by adding or subtracting an offset to the average value.

Alternatively, the threshold value may be derived based on the minimum, maximum, or median value among the absolute values of residual samples in the same location luma block. As an example, the threshold value may be set equal to the minimum, maximum, or median value among the absolute values of residual samples in the luma block, or may be a value obtained by adding or subtracting an offset to the minimum, maximum, or median value.

In the example shown in FIG. 30, subblock A and subblock B within the co-located luma block are determined to be areas with large prediction errors. In this case, the DM mode may not be used in subblock a and subblock b in the chroma block corresponding to subblock A and subblock B.

In areas where DM mode is not used, intra prediction may be performed using a predefined intra prediction mode. As an example, the predefined intra prediction mode may be planar mode or DC mode.

Alternatively, the decoder may derive an intra prediction mode for an area where DM mode is not used. As an example, the optimal intra prediction mode for the area with large prediction error can be derived using restored samples that exist around the area with large prediction error in the luma block. For example, in the example shown in FIG. 30, the optimal intra prediction mode can be derived for each of subblock A and subblock B within the co-located luma block. When the optimal intra prediction mode is derived for each of subblock A and subblock B, the optimal intra prediction mode derived for subblock A is set to the intra prediction mode of subblock a in the chroma block, and subblock B The optimal intra prediction mode derived for can be set as the intra prediction mode of subblock b in the chroma block.

Meanwhile, for areas with large prediction errors within the co-located luma block, information indicating whether application of DM mode is restricted may be explicitly encoded and signaled. The information may be a 1-bit flag.

The names of syntaxes used in the above-described embodiments are merely named for convenience of explanation.

Applying the embodiments described focusing on the decoding process or encoding process to the encoding process or decoding process is included in the scope of the present disclosure. Modification of the embodiments described in the given order to an order different from that described is also included within the scope of the present disclosure.

Although the above-described disclosure is explained based on a series of steps or a flowchart, this does not limit the chronological order of the invention, and may be performed simultaneously or in a different order as needed. In addition, each of the components (e.g., units, modules, etc.) constituting the block diagram in the above-described disclosure may be implemented as a hardware device or software, and a plurality of components may be combined to form a single hardware device or software. It could be. As an example, the hardware device may include at least one of a processor for performing operations, a memory for storing data, a transmitter for transmitting data, and a receiver for receiving data.

The above-described disclosure may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., singly or in combination.

Additionally, according to the present disclosure, a computer-readable recording medium that stores a bitstream generated by the above-described encoding method can be provided. The bitstream may be transmitted by an encoding device, and the decoding device may receive the bitstream and decode the image.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specifically configured to store and perform program instructions, such as ROM, RAM, flash memory, etc. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present disclosure, and vice versa.

Embodiments through this disclosure can be applied to electronic devices that encode or decode images.

Claims (15)

1. dividing the chroma block into a plurality of partitions; and

   Including performing intra prediction on each of the partitions in the chroma block,

   An image decoding method, wherein the partition type of the chroma block is determined based on at least one of the partition type or the directionality of the co-located luma block of the chroma block.
2. According to claim 1,

   The partition type of the chroma block is set to be the same as the partition type of the co-located luma block.
3. According to claim 1,

   The plurality of partitions are created by dividing the chroma block in a horizontal or vertical direction,

   An image decoding method, wherein the division direction of the chroma block is determined based on the directionality of the co-located luma block.
4. According to claim 1,

   An image decoding method, wherein the decoding order of the plurality of partitions in the chroma block is determined based on the directionality of the co-located luma block.
5. According to claim 1,

   Further comprising determining whether to perform flipping on the chroma block,

   An image decoding method, wherein the flipping of the chroma block is performed in at least one of the horizontal direction and the vertical direction.
6. According to clause 5,

   An image decoding method, wherein whether to perform the flipping and a direction to perform the flipping are determined based on the directionality of the co-located luma block.
7. According to clause 5,

   The flipping is allowed only when the co-located luma block is encoded by intra prediction.
8. According to claim 1,

   An image decoding method, wherein the intra prediction mode of each of the plurality of partitions in the chroma block is set to be the same as the intra prediction mode of each of the plurality of partitions included in the co-located luma block.
9. According to claim 1,

   A predefined intra prediction mode is applied to a partition corresponding to an area with a large prediction error in the co-located luma block among the plurality of partitions in the chroma block,

   An image decoding method, characterized in that the intra prediction mode of the co-located luma block is applied to the partition otherwise.
10. According to clause 9,

    An image decoding method, characterized in that whether the sub-region in the co-located luma block is an area with a large prediction error is determined by comparing the average value of absolute values of residual samples in the sub-region with a threshold value.
11. According to claim 10,

    The threshold value is derived based on the average value of absolute values of residual samples within the co-located luma block.
12. According to claim 1,

    The directionality of the co-located luma block is determined based on predefined intra prediction modes and an optimal intra prediction mode derived according to an intra prediction result performed based on reference samples around the co-located luma block. A video decoding method, characterized in that.
13. According to claim 12,

    The directionality of the co-located luma block is:

    An image decoding method, characterized in that the direction of the reference block including the co-located luma block is set to be the same.
14. dividing the chroma block into a plurality of partitions; and

    Including performing intra prediction on each of the partitions in the chroma block,

    An image encoding method, wherein the partition type of the chroma block is determined based on at least one of the partition type or the directionality of the co-located luma block of the chroma block.
15. dividing the chroma block into a plurality of partitions; and

    Including performing intra prediction on each of the partitions in the chroma block,

    The partition type of the chroma block is determined based on at least one of the partition type or directionality of the co-located luma block of the chroma block. A computer-readable device for storing a bitstream generated by an image encoding method. Recording media.

Images