KR20210018137A - Method and apparatus for intra prediction coding of video data - Google Patents

Abstract

Disclosed are a method and an apparatus for intra prediction coding of video data. According to one aspect of the present invention, an apparatus for decoding video data comprises: a decoding unit to acquire information about a chroma prediction mode and information about a luma prediction mode of a current coding block from a bitstream; and an intra prediction unit to generate luma prediction samples and chroma prediction samples of the current coding block. The intra prediction unit derives a luma intra prediction type and a luma intra prediction mode of the current coding block based on the information about the luma prediction mode, and determines a chroma intra prediction mode of the current coding block based on the luma intra prediction type and the luma intra prediction mode of the current coding block and the information about the chroma prediction mode.

Description

Method and apparatus for intra prediction coding of video data TECHNICAL FIELD [Method and Apparatus for Intra Prediction CODING OF VIDEO DATA]

The present invention relates to encoding and decoding of moving picture data.

Since moving picture data has a large amount of data compared to audio data or still image data, it requires a lot of hardware resources including memory in order to store or transmit itself without processing for compression.

Accordingly, when moving or transmitting moving picture data, the moving picture data is compressed and stored or transmitted using an encoder, and the decoder receives the compressed moving picture data, decompresses and reproduces the compressed moving picture data. As such video compression technologies, there are H.264/AVC and HEVC (High Efficiency Video Coding), which improves coding efficiency by about 40% compared to H.264/AVC.

However, the size, resolution, and frame rate of pictures are gradually increasing, and accordingly, the amount of data to be encoded is also increasing. Accordingly, a new compression technique having higher encoding efficiency and higher quality improvement effect than the existing compression technique is required.

This disclosure mainly presents improved techniques for intra prediction coding a block of moving picture data.

According to an aspect of the present disclosure, a method of decoding video data includes obtaining information about a luma prediction mode and information about a chroma prediction mode of a current coding block from a bitstream; Deriving a luma intra prediction type and a luma intra prediction mode of the current coding block based on the information on the luma prediction mode-The luma intra prediction type is a matrix based intra prediction (MIP) and a normal intra Including regular intra prediction -; Determining a chroma intra prediction mode of the current coding block based on a luma intra prediction type and a luma intra prediction mode of the current coding block and information about the chroma prediction mode; And generating chroma prediction samples of the current coding block based on the chroma intra prediction mode of the current coding block.

In the determining of the chroma intra prediction mode of the current coding block, the information on the chroma prediction mode indicates a direct mode (DM), a luma intra prediction type of the current coding block is a matrix-based intra prediction, and the video When the data sampling format is 4:4:4, it is determined that the chroma intra prediction type of the current coding block is matrix-based intra prediction, and the chroma intra prediction mode corresponding to the chroma intra prediction type of the current coding block is the current And determining that it is the same as the matrix-based intra prediction mode derived from the luma intra prediction mode of the coding block.

In the determining of the chroma intra prediction mode of the current block, the information on the chroma prediction mode indicates a direct mode (DM), a luma intra prediction type of the current coding block is a matrix-based intra prediction type, and the video When the data sampling format is 4:2:0 or 4:2:2, determining a chroma intra prediction mode of the current coding block as a PLANAR mode.

In the determining of the chroma intra prediction mode of the current block, when the information on the chroma prediction mode indicates a direct mode (DM) and the luma intra prediction type of the current coding block is a normal intra prediction type, the current And determining that the chroma intra prediction mode of the coding block is the same as the normal intra prediction mode derived from the luma intra prediction mode of the current coding block.

According to another aspect of the present disclosure, an apparatus for decoding video data includes a decoder that obtains information about a luma prediction mode and information about a chroma prediction mode of a current coding block from a bitstream, and luma prediction samples of the current coding block. And an intra prediction unit that generates and chroma prediction samples. The intra prediction unit derives a luma intra prediction type and a luma intra prediction mode of the current coding block based on the information on the luma prediction mode, and the luma intra prediction type and luma intra prediction mode and the chroma prediction mode of the current coding block Based on the information on, a chroma intra prediction mode of the current coding block is determined.

The intra prediction unit, as part of determining a chroma intra prediction mode of the current coding block, the information on the chroma prediction mode indicates a direct mode (DM), and the luma intra prediction type of the current coding block is matrix-based. Intra prediction, and when the sampling format of the video data is 4:4:4, it is determined that the chroma intra prediction type of the current coding block is the matrix-based intra prediction, and corresponds to the chroma intra prediction type of the current coding block It is determined that the chroma intra prediction mode is the same as the matrix-based intra prediction mode derived from the luma intra prediction mode of the current coding block.

1 is an exemplary block diagram of a video encoding apparatus capable of implementing the techniques of this disclosure.
2 is a diagram for explaining a method of dividing a block using a QTBTTT structure.
3A is a diagram illustrating a plurality of intra prediction modes.
3B is a diagram illustrating a plurality of intra prediction modes including wide-angle intra prediction modes.
4 is an exemplary block diagram of a video decoding apparatus that can implement the techniques of this disclosure.
5 is a conceptual diagram illustrating the main process of MIP technology that may be used in the techniques of this disclosure.
6 is a flowchart illustrating a method of decoding video data according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding identification codes to elements of each drawing, it should be noted that the same elements have the same symbols as possible even if they are indicated on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

1 is an exemplary block diagram of a video encoding apparatus capable of implementing the techniques of this disclosure. Hereinafter, a video encoding apparatus and sub-elements of the apparatus will be described with reference to FIG. 1.

The video encoding apparatus includes a picture division unit 110, a prediction unit 120, a subtractor 130, a transform unit 140, a quantization unit 145, a rearrangement unit 150, an entropy encoding unit 155, an inverse quantization unit. (160), an inverse transform unit 165, an adder 170, a filter unit 180, and a memory 190 may be included.

Each component of the video encoding apparatus may be implemented by hardware or software, or by a combination of hardware and software. In addition, functions of each component may be implemented as software, and a microprocessor may be implemented to execute a function of software corresponding to each component.

One image (video) is composed of a plurality of pictures. Each picture is divided into a plurality of regions, and encoding is performed for each region. For example, one picture is divided into one or more tiles or/and slices. Here, one or more tiles may be defined as a tile group. Each tile or/slice is divided into one or more Coding Tree Units (CTUs). And each CTU is divided into one or more CUs (Coding Units) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in one CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks in one slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one picture is a picture parameter set (PPS) or picture. It is coded in the header. Further, information commonly referred to by a plurality of pictures is encoded in a sequence parameter set (SPS). In addition, information commonly referred to by one or more SPSs is encoded in a video parameter set (VPS). Also, information commonly applied to one tile or tile group may be encoded as syntax of a tile or tile group header.

The picture dividing unit 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as the syntax of the SPS or PPS and transmitted to the video decoding apparatus.

After dividing each picture constituting an image into a plurality of CTUs (Coding Tree Units) having a predetermined size, the picture dividing unit 110 repetitively divides the CTU using a tree structure. (recursively) split. A leaf node in the tree structure becomes a coding unit (CU), which is a basic unit of coding.

As a tree structure, a quad tree (QuadTree, QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, or a binary tree (BinaryTree) in which an upper node is divided into two lower nodes. , BT), or a ternary tree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure in which two or more of these QT structures, BT structures, and TT structures are mixed. have. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, by combining BTTT, it may be referred to as MTT (Multiple-Type Tree).

2 shows a QTBTTT split tree structure. As shown in FIG. 2, the CTU may be first divided into a QT structure. The quadtree division may be repeated until the size of a splitting block reaches the minimum block size (MinQTSize) of a leaf node allowed in QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is divided into four nodes of a lower layer is encoded by the entropy encoder 155 and signaled to the video decoding apparatus. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, it may be further divided into one or more of a BT structure or a TT structure. In the BT structure and/or the TT structure, a plurality of division directions may exist. For example, there may be two directions in which a block of a corresponding node is divided horizontally and a direction vertically divided. As shown in FIG. 2, when MTT splitting starts, a second flag (mtt_split_flag) indicating whether nodes are split, and if split, a flag indicating a split direction (vertical or horizontal) and/or a split type (Binary or Ternary). A flag indicating) is encoded by the entropy encoder 155 and signaled to the video decoding apparatus. Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is divided into four nodes of a lower layer, a CU split flag (split_cu_flag) indicating whether the node is divided is encoded. It could be. When it is indicated that the value of the CU split flag (split_cu_flag) is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a coding unit (CU), which is a basic unit of encoding. When indicating that the value of the CU split flag (split_cu_flag) is to be split, the video encoding apparatus starts encoding from the first flag in the above-described manner.

When QTBT is used as another example of the tree structure, two types of horizontally splitting a block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type splitting vertically (i.e., symmetric vertical splitting) Branches can exist. A split flag indicating whether each node of the BT structure is divided into blocks of a lower layer and split type information indicating a type to be divided are encoded by the entropy encoder 155 and transmitted to the video decoding apparatus. Meanwhile, a type of dividing the block of the corresponding node into two blocks having an asymmetric shape may further exist. The asymmetric form may include a form of dividing a block of a corresponding node into two rectangular blocks having a size ratio of 1:3, or a form of dividing a block of a corresponding node in a diagonal direction.

The CU can have various sizes according to the QTBT or QTBTTT split from the CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (ie, a leaf node of QTBTTT) is referred to as a'current block'. According to the adoption of the QTBTTT division, the shape of the current block may be not only square but also rectangular.

The prediction unit 120 predicts the current block and generates a prediction block. The prediction unit 120 includes an intra prediction unit 122 and an inter prediction unit 124.

In general, each of the current blocks in a picture can be predictively coded. In general, prediction of the current block is performed using an intra prediction technique (using data from a picture containing the current block) or an inter prediction technique (using data from a picture coded before a picture containing the current block). Can be done. Inter prediction includes both one-way prediction and two-way prediction.

The intra prediction unit 122 predicts pixels in the current block by using pixels (reference pixels) located around the current block in the current picture including the current block. There are a plurality of intra prediction modes according to the prediction direction. For example, as shown in FIG. 3A, the plurality of intra prediction modes may include two non-directional modes including a PLANAR mode and a DC mode, and 65 directional modes. Depending on each prediction mode, the surrounding pixels to be used and the calculation expression are defined differently. The table below lists intra prediction modes numbers and names.

For efficient directional prediction for the rectangular-shaped current block, directional modes (67 to 80, intra prediction modes -1 to -14) shown by dotted arrows in FIG. 3B may be additionally used. These may be referred to as "wide angle intra-prediction modes". Arrows in FIG. 3B indicate corresponding reference samples used for prediction, and do not indicate a prediction direction. The prediction direction is opposite to the direction indicated by the arrow. In the wide-angle intra prediction modes, when the current block is a rectangle, a specific directional mode is predicted in the opposite direction without additional bit transmission. In this case, among the wide-angle intra prediction modes, some wide-angle intra prediction modes available for the current block may be determined based on a ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes with an angle less than 45 degrees (intra prediction modes 67 to 80) can be used when the current block has a rectangular shape with a height smaller than the width, and wide-angle with an angle greater than -135 degrees. The intra prediction modes (intra prediction modes -1 to -14) can be used when the current block has a rectangular shape whose height is greater than the width.

The intra prediction unit 122 may determine an intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit 122 may encode the current block using several intra prediction modes and select an appropriate intra prediction mode to use from the tested modes. For example, the intra prediction unit 122 calculates rate distortion values using rate-distortion analysis for several tested intra prediction modes, and has the best rate distortion characteristics among the tested modes. It is also possible to select an intra prediction mode.

The intra prediction unit 122 selects one intra prediction mode from among a plurality of intra prediction modes, and predicts the current block using a neighboring pixel (reference pixel) determined according to the selected intra prediction mode and an equation. Information on the selected intra prediction mode is encoded by the entropy encoder 155 and transmitted to the video decoding apparatus.

In addition, the intra prediction unit 122 may generate a prediction block for the current block by using matrix-based intra prediction (MIP) to be described later. The intra prediction unit 122 generates a prediction block for the current block using a boundary vector derived from samples reconstructed on the left side of the current block and samples reconstructed above the current block, a predefined matrix, and an offset vector. You may.

The inter prediction unit 124 generates a prediction block for the current block through a motion compensation process. The inter prediction unit 124 searches for a block most similar to the current block in the coded and decoded reference picture prior to the current picture, and generates a prediction block for the current block using the searched block. Then, a motion vector corresponding to a displacement between the current block in the current picture and the prediction block in the reference picture is generated. In general, motion estimation is performed on a luma component, and a motion vector calculated based on the luma component is used for both the luma component and the chroma component. Motion information including information on a reference picture used to predict the current block and information on a motion vector is encoded by the entropy encoder 155 and transmitted to the video decoding apparatus.

The subtractor 130 generates a residual block by subtracting the prediction block generated by the intra prediction unit 122 or the inter prediction unit 124 from the current block.

The transform unit 140 divides the residual block into one or more transform blocks, applies the transform to one or more transform blocks, and transforms residual values of the transform blocks from the pixel domain to the frequency domain. In the frequency domain, transformed blocks are referred to as coefficient blocks comprising one or more transform coefficient values. A 2D transformation kernel may be used for transformation, and a 1D transformation kernel may be used for each of the horizontal and vertical directions. The transform kernel may be based on discrete cosine transform (DCT), discrete sine transform (DST), or the like.

The transform unit 140 may transform residual signals in the residual block by using the entire size of the residual block as a transform unit. In addition, the transform unit 140 may divide the residual block into two sub-blocks in a horizontal or vertical direction, and may perform transformation on only one of the two sub-blocks. Accordingly, the size of the transform block may be different from the size of the residual block (and thus the size of the prediction block). Non-zero residual sample values may not exist or may be very sparse in a subblock on which transformation is not performed. The residual samples of the subblock on which the transformation is not performed are not signaled, and may be regarded as "0" by the video decoding apparatus. There can be several partition types depending on the partitioning direction and the partitioning ratio. The transform unit 140 includes information on the coding mode (or transform mode) of the residual block (e.g., information indicating whether the residual block is transformed or the residual subblock is transformed, and a partition type selected to divide the residual block into subblocks) The entropy encoding unit 155 may be provided with information indicating information and information identifying a subblock on which transformation is performed. The entropy encoder 155 may encode information about a coding mode (or transform mode) of the residual block.

The quantization unit 145 quantizes the transform coefficients output from the transform unit 140 and outputs the quantized transform coefficients to the entropy encoding unit 155. The quantization unit 145 may immediately quantize a related residual block for a certain block or frame without transformation.

The rearrangement unit 150 may rearrange coefficient values on the quantized residual values. The rearrangement unit 150 may change a two-dimensional coefficient array into a one-dimensional coefficient sequence through coefficient scanning. For example, the rearrangement unit 150 may scan from a DC coefficient to a coefficient in a high frequency region using a zig-zag scan or a diagonal scan to output a one-dimensional coefficient sequence. . Depending on the size of the transform unit and the intra prediction mode, instead of zig-zag scan, a vertical scan that scans a two-dimensional coefficient array in a column direction or a horizontal scan that scans a two-dimensional block shape coefficient in a row direction may be used. That is, a scan method to be used may be determined from among zig-zag scan, diagonal scan, vertical scan, and horizontal scan according to the size of the transform unit and the intra prediction mode.

The entropy encoding unit 155 uses various encoding methods such as Context-based Adaptive Binary Arithmetic Code (CABAC), Exponential Golomb, and the like, and the quantized transform coefficients of 1D output from the reordering unit 150 are A bitstream is generated by encoding the sequence.

In addition, the entropy encoder 155 encodes information such as a CTU size related to block division, a CU division flag, a QT division flag, an MTT division type, and an MTT division direction, so that the video decoding apparatus performs the same block as the video encoding apparatus. Make it possible to divide. In addition, the entropy encoder 155 encodes information on a prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and intra prediction information (ie, intra prediction) according to the prediction type. Mode information) or inter prediction information (reference picture and motion vector information) is encoded.

The inverse quantization unit 160 inverse quantizes the quantized transform coefficients output from the quantization unit 145 to generate transform coefficients. The inverse transform unit 165 converts transform coefficients output from the inverse quantization unit 160 from the frequency domain to the spatial domain to restore the residual block.

The addition unit 170 restores the current block by adding the restored residual block and the prediction block generated by the prediction unit 120. The pixels in the reconstructed current block are used as reference pixels when intra-predicting the next block.

The filter unit 180 filters reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation/quantization. Perform. The filter unit 180 may include a deblocking filter 182 and a sample adaptive offset (SAO) filter 184.

The deblocking filter 180 filters the boundary between reconstructed blocks to remove blocking artifacts caused by block-based encoding/decoding, and the SAO filter 184 adds additional information to the deblocking-filtered image. Filtering is performed. The SAO filter 184 is a filter used to compensate for a difference between a reconstructed pixel and an original pixel caused by lossy coding.

The reconstructed block filtered through the deblocking filter 182 and the SAO filter 184 is stored in the memory 190. When all blocks in one picture are reconstructed, the reconstructed picture may be used as a reference picture for inter prediction of a block in a picture to be encoded later.

4 is an exemplary functional block diagram of a video decoding apparatus capable of implementing the techniques of this disclosure. Hereinafter, a video decoding apparatus and subordinate components of the apparatus will be described with reference to FIG. 4.

The video decoding apparatus includes an entropy decoding unit 410, a rearrangement unit 415, an inverse quantization unit 420, an inverse transform unit 430, a prediction unit 440, an adder 450, a filter unit 460, and a memory 470. ) Can be included.

Like the video encoding apparatus of FIG. 1, each component of the video decoding apparatus may be implemented as hardware or software, or a combination of hardware and software. In addition, functions of each component may be implemented as software, and a microprocessor may be implemented to execute a function of software corresponding to each component.

The entropy decoding unit 410 determines the current block to be decoded by decoding the bitstream generated by the video encoding apparatus and extracting information related to block division, and the prediction information and residual signal required to restore the current block. Extract information, etc.

The entropy decoding unit 410 determines the size of the CTU by extracting information on the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), and divides the picture into CTUs of the determined size. Then, the CTU is determined as the uppermost layer of the tree structure, that is, the root node, and the CTU is divided using the tree structure by extracting partition information for the CTU.

For example, in the case of splitting the CTU using the QTBTTT structure, first, a first flag (QT_split_flag) related to the splitting of the QT is extracted and each node is split into four nodes of a lower layer. And, for the node corresponding to the leaf node of QT, the second flag (MTT_split_flag) related to the splitting of the MTT and the splitting direction (vertical / horizontal) and/or split type (binary / ternary) information are extracted, and the corresponding leaf node is MTT. Divide into structure. Through this, each node below the leaf node of the QT is recursively divided into a BT or TT structure.

As another example, when splitting the CTU using the QTBTTT structure, first extract the CU split flag (split_cu_flag) indicating whether to split the CU, and if the corresponding block is split, the first flag (QT_split_flag) is extracted. May be. In the segmentation process, each node may have 0 or more repetitive MTT segmentation after 0 or more repetitive QT segmentation. For example, in the CTU, MTT division may occur immediately, or, conversely, only multiple QT divisions may occur.

As another example, when the CTU is divided using the QTBT structure, each node is divided into four nodes of a lower layer by extracting the first flag (QT_split_flag) related to the division of the QT. In addition, a split flag indicating whether or not the node corresponding to the leaf node of the QT is further split into BT and split direction information are extracted.

Meanwhile, when determining the current block to be decoded through the division of the tree structure, the entropy decoder 410 extracts information on a prediction type indicating whether the current block is intra prediction or inter prediction. When the prediction type information indicates intra prediction, the entropy decoder 410 extracts a syntax element for intra prediction information (intra prediction mode) of the current block. When the prediction type information indicates inter prediction, the entropy decoder 410 extracts a syntax element for the inter prediction information, that is, information indicating a motion vector and a reference picture referenced by the motion vector.

On the other hand, the entropy decoder 410 includes information on the coding mode of the residual block (e.g., information on whether the residual block is encoded or only the subblocks of the residual block are encoded, and is selected to divide the residual block into subblocks). Information indicating the partition type, information identifying the encoded residual subblock, quantization parameters, etc.) are extracted from the bitstream. In addition, the entropy decoder 410 extracts information on quantized transform coefficients of the current block as information on the residual signal.

The rearrangement unit 415, in the reverse order of the coefficient scanning order performed by the video encoding apparatus, reconverts the sequence of one-dimensional quantized transform coefficients entropy-decoded by the entropy decoder 410 to a two-dimensional coefficient array (i.e., Block).

The inverse quantization unit 420 inverse quantizes the quantized transform coefficients, and the inverse transform unit 430 inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain based on information on the coding mode of the residual block By reconstructing the signals, a reconstructed residual block for the current block is generated.

When the information on the coding mode of the residual block indicates that the residual block of the current block is coded in the video encoding apparatus, the inverse transform unit 430 determines the size of the current block with respect to the inverse quantized transformation coefficients. A reconstructed residual block for the current block is generated by performing inverse transformation using the residual block size) as a transformation unit.

In addition, when the information on the coding mode of the residual block indicates that only one subblock of the residual block is coded in the video encoding apparatus, the inverse transform unit 430 performs the transformed sub- By using the size of the block as a transformation unit, performing inverse transformation to restore residual signals for the transformed subblock, and filling the residual signals for untransformed subblocks with a value of "0", the reconstructed current block Create a residual block.

The prediction unit 440 may include an intra prediction unit 442 and an inter prediction unit 444. The intra prediction unit 442 is activated when the prediction type of the current block is intra prediction, and the inter prediction unit 444 is activated when the prediction type of the current block is inter prediction.

The intra prediction unit 442 determines an intra prediction mode of the current block among a plurality of intra prediction modes from the syntax element for the intra prediction mode extracted from the entropy decoding unit 410, and references around the current block according to the intra prediction mode. Predict the current block using pixels. In addition, the intra prediction unit 442 may generate a prediction block for the current block by using matrix-based intra prediction (MIP) to be described later. The intra prediction unit 422 generates a prediction block for the current block by using a boundary vector derived from samples reconstructed on the left side of the current block and samples reconstructed above the current block, and a predefined matrix and offset vector. You may.

The inter prediction unit 444 determines a motion vector of the current block and a reference picture referenced by the motion vector using the syntax element for the intra prediction mode extracted from the entropy decoding unit 410, and determines the motion vector and the reference picture. Is used to predict the current block.

The adder 450 restores the current block by adding the residual block output from the inverse transform unit and the prediction block output from the inter prediction unit or the intra prediction unit. The pixels in the reconstructed current block are used as reference pixels for intra prediction of a block to be decoded later.

The filter unit 460 may include a deblocking filter 462 and an SAO filter 464. The deblocking filter 462 performs deblocking filtering on the boundary between reconstructed blocks in order to remove blocking artifacts caused by decoding in units of blocks. The SAO filter 464 performs additional filtering on the reconstructed block after deblocking filtering in order to compensate for the difference between the reconstructed pixel and the original pixel caused by lossy coding. The reconstructed block filtered through the deblocking filter 462 and the SAO filter 464 is stored in the memory 470. When all blocks in one picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of a block in a picture to be encoded later.

The techniques of this disclosure generally relate to intra predictive coding. The following description is mainly focused on a decoding technique, that is, the operation of a video decoder, and descriptions of the encoding techniques are simplified because they are opposite to the comprehensively described decoding technique.

In the discussion of the next-generation video coding standard (VVC; Versatile Video Coding), several new coding tools have been introduced that enable better coding performance compared to the High Efficiency Video Coding (HEVC) standard.

Matrix-based Intra Prediction (MIP) is a new intra prediction technology introduced in VTM 5.0. The original idea is to use a neural network-based intra prediction technique, i.e., using a multilayer neural network to predict current PU pixel values based on adjacent reconstructed pixels. However, due to the high complexity of the prediction method using a neural network, an intra prediction technique based on affine linear transformation using pre-trained matrices was introduced.

To predict a rectangular block PU with width W and height H, MIP takes as inputs the reconstructed H samples on the left side of the block and the reconstructed W samples on the top side of the block. And the final predicted pixels are obtained by averaging, matrix-vector multiplication, and linear interpolation.

Block sizes to which MIP is applied are classified into three categories as follows.

Depending on idx(W,H), the number of MIP modes (numModes), boundary size (boundarySize), and prediction block sizes (predW, predH, predC) are defined as follows. In the table below, MipSizeId = idx(W,H).

5 is a conceptual diagram illustrating the main process of MIP technology that may be used in the techniques of this disclosure.

(1) Averaging

The main purpose of this step is to normalize the reference samples. Depending on the block size and shape (width and height) (i.e. according to MipSizeId), 4 or 8 samples are obtained. If both the width and height of the current block are 4 (W=H=4), a total of 4 samples including 2 from the left and 2 from the top are obtained (boundarySize = 2). In the remaining case, a total of 8 samples including 4 from the left and 4 from the top are obtained (boundarySize = 4).

로 표기되고, 좌측 이웃 샘플들은

로 표기된다.

와

에 대해 각각 평균 연산을 수행하여, 다운 샘플링된 샘플 세트

와

가 얻어진다. 평균 연산은 다음과 같은 다운 샘플링 프로세스이다.As shown in Figure 5, the upper neighboring samples are

And the left neighboring samples are

It is marked as.

Wow

Averaging each of the sets of samples down-sampled

Wow

Is obtained. The average operation is the downsampling process as follows.

로 저장되고, 상측 이웃의 경우

로 저장된다. In the above equation, bDwn is a downsampling scale value (nTbs / boundarySize), and refS indicates an original reference sample. The calculated redS is for the left neighbor

And in the case of the upper neighbor

Is saved as.

은 아래의 수학식과 같이 정의된다. 예를 들어, W=H=4이고 MIP 모드가 18 미만인 경우

와

의 순서로 접합하여 경계 벡터를 구성하며, W=H=4이고 MIP 모드가 18이상이면

와

의 순서로 접합된다. 아래 수학식에서, "mode" 는 MIP 모드를 의미한다.The down-sampled reference samples are stitched into vectors of length 4 or 8. Reduced boundary vector input to vector-matrix multiplication operation

Is defined as the following equation. For example, if W=H=4 and MIP mode is less than 18

Wow

The boundary vector is formed by joining in the order of, and if W=H=4 and MIP mode is 18 or higher,

Wow

Are joined in the order of. In the following equation, "mode" means a MIP mode.

(2) Matrix-Vector Multiplication

로부터 현재 블록의 다운 샘플링된 예측 신호

가 생성된다.

는 행렬-벡터의 곱과 오프셋의 합으로 다음과 같이 계산한다.At this stage, the reduced boundary vector

The downsampled prediction signal of the current block from

Is created.

Is the sum of the matrix-vector product and offset as follows.

의 크기는 W_red × H_red 이며, W_red 와 H_red 는 현재 블록의 크기와 형상에 따라 아래와 같이 정의된다. 행렬 A 는 W_red * H_red 만큼의 행(row)들을 가지고, W=H=4 인 경우는 4 개의 열(column)을 가지며, 그 외에는 8 개의 열들을 가진다. 오프셋 벡터 b 는 W_red * H_red 크기의 벡터이다.

The size of is W _red × H _red , and W _red and H _red are defined as follows according to the size and shape of the current block. Matrix A has as many rows as W _red * H _red , and has 4 columns when W=H=4, and 8 columns other than that. The offset vector b is a vector of size W _red * H _red .

Sets S ₀ , S ₁ , S ₂ of matrix A and offset vector b that can be used for the coding block are predefined for each category of the sizes of the coding block. The indices (0, 1, 2) of the set S are selected according to the aforementioned MipSizeId (i.e., idx(W,H)), and the matrix A and the offset vector b are one of the sets S ₀ , S ₁ , and S ₂ . Is extracted according to the MIP mode applied to the current block.

The set S ₀ consists of 18 matrices ( A ₀ ) with 16 rows and 4 columns, respectively, and 18 16-dimensional offset vectors ( b ₀ ), and is used for a 4×4 block. Set S ₁ consists of 10 matrices ( A ₁ ) and 10 16-dimensional offset vectors ( b ₁ ) each with 16 rows and 8 columns, and is used for blocks of size 4×8, 8×4 and 8×8. do. Finally, set S ₂ consists of 6 matrices ( A ₂ ) with 64 rows and 8 columns, respectively, and 6 64 dimensional offset vectors ( b ₂ ), which are used for all remaining block shapes.

(3) pixel interpolation

는 원래 블록의 다운 샘플링된 예측 신호이다. 이 때 크기 predW 과 predH 을 갖는 다운 샘플링된 예측 블록이 다음과 같이 정의된다. Interpolation is an up-sampling process. As mentioned above,

Is the down-sampled prediction signal of the original block. In this case, a down-sampled prediction block with sizes predW and predH is defined as follows.

pred _red [x][y], with x = 0..predW-1, y = 0..predH-1

A prediction block having an original block size (nTbW, nTbH) generated by linearly interpolating the prediction signal at the remaining positions in each direction is defined as follows.

predSamples[x][y], with x = 0..nTbW-1, y = 0..nTbH-1

로부터 predSamples의 일부 혹은 전부가 채워진다. Depending on the horizontal and vertical upsampling scale factors upHor(= nTbW / predW) and upVer (= nTbH / predH), as follows:

Some or all of the predSamples from are filled.

predSamples[(x+1) * upHor-1][(y+1) * upVer-1] = pred _red [x][y]

로부터 predSamples의 수평 방향의 모든 위치가 채워지며, upVer = 1이면

로부터 predSamples의 수직 방향의 모든 위치가 채워진다.if upHor = 1

All positions in the horizontal direction of predSamples are filled from, if upVer = 1

From, all positions in the vertical direction of the predSamples are filled.

이 predSamples[x][-1] 값들에 할당되며, 좌측의 원래 참조샘플들

이 predSamples[-1][y] 값들에 할당된다. 보간 순서는 현재블록의 크기에 따라 결정된다. 즉, 짧은 크기의 방향으로 먼저 보간이 수행되며 후속하여 긴 크기의 방향으로 보간이 수행된다.After that, the remaining empty samples of predSamples are filled through bi-linear interpolation. Interpolation in the horizontal direction and interpolation in the vertical direction are upsampling processes. For interpolation of left and upper samples in predSamples, down sampled samples

These predSamples[x][-1] values are assigned to the original reference samples on the left

These predSamples[-1][y] values are assigned. The interpolation order is determined according to the size of the current block. That is, interpolation is first performed in the direction of the short size and interpolation is performed in the direction of the long size subsequently.

(4) MIP intra prediction mode signaling

For each coding unit (CU) to be intra prediction coded, a flag indicating whether a matrix-based intra prediction mode (ie, MIP mode) is applied is transmitted. In the VVC 5 draft, for signaling in the MIP mode, an MPM list consisting of three MPMs is used, similar to the existing intra prediction mode (hereinafter referred to as'regular intra prediction mode') rather than matrix-based intra prediction. For example, intra_mip_mpm_flag, intra_mip_mpm_idx, and intra_mip_mpm_remainder are used for MIP mode signaling. intra_mip_mpm_idx is coded as a truncated binary code, and intra_mip_mpm_remainder is coded as a fixed length code.

Depending on the size of the coding block (CU), up to 35 MIP modes may be supported. For example, for a CU with max (W, H) <= 8 and W*H <32, 35 modes are available. In addition, 19 and 11 prediction modes are used for CUs with max(W, H) = 8 and max(W, H)> 8, respectively. In addition, a pair of modes (two modes) can share a matrix and an offset vector to reduce memory requirements. The specific sharing mode is calculated as follows. For example, for a 4×4 coding block, mode 19 uses a transposed matrix of the matrix assigned for mode 2.

MIP mode for MPM derivation of a regular block when there is a block to which MIP is applied around a block to which a regular intra prediction mode is applied (hereinafter referred to as a'normal block') other than matrix-based intra prediction (MIP) A mapping table defined between the and normal modes may be used. The mapping table is used to derive a normal mode of similar characteristics from the MIP mode of the neighboring block. The normal mode derived in this way is used for MPM derivation of the normal block. Similarly, even when inducing a chroma direct mode (DM), even when a MIP mode is applied to a luma block at the same location, a normal mode mapped to the MIP mode is derived using a mapping table to determine the intra prediction mode of the chroma block. . The equation below expresses the mapping between the regular mode and the MIP mode.

This disclosure proposes several modifications that can improve prediction performance without significantly increasing the implementation complexity of a matrix-based intra prediction (MIP) technique.

Derivation of matrices and offsets

According to an aspect of the present disclosure, an encoder and a decoder are provided with the remaining modes among the available MIP modes from explicitly defined sets of matrices and offsets for some of the available MIP modes for a given block size. You can create new matrices and sets of offsets for use.

The new set of matrices and offsets may be obtained by performing at least one of a transposition, an average operation, and a weighted average operation from two (or more) explicitly defined sets of matrices and offsets. The explicitly defined matrix and offset sets may be transmitted to the decoder by the encoder, or may be predefined in the memory of both the encoder and the decoder. Accordingly, it is possible to further reduce a memory requirement for storing a set of matrixes and offsets for each mode, without reducing the number of existing available modes.

For illustrative purposes, assume that M matrices are explicitly defined in the encoder and decoder for a given block size. The encoder can derive (generate) K new matrices from M matrices through a predefined method, so a total of M+K matrices can be used to perform matrix-based intra prediction for a given block. have. In the list of available matrices including M+K matrices, each matrices can be identified by an index.

When the MIP mode signaled from the encoder for the current block is defined to use an explicitly defined matrix, the decoder immediately uses the matrix stored in the memory to perform matrix-based intra prediction for the current block. I can. When the MIP mode signaled from the encoder for the current block is defined to use the derived matrix, the decoder may generate the derived matrix for the signaled MIP mode in the same manner as the encoder.

,

일 때, 가중평균 행렬 C는 다음과 같이 정의될 수 있다.In some embodiments, the derived matrix may be an average matrix or a weighted average matrix of two explicitly defined matrices. As an example, consider explicitly defined matrices A and B having a size of M×N.

,

When, the weighted average matrix C can be defined as follows.

Here, s1 and s2 are weights applied to matrices A and B, respectively. The weights s1 and s2 may be defined in advance or may be signaled in a higher level syntax by the encoder.

In the same way as the derived matrix, a new offset can also be derived.

In some other embodiments, the derived matrix may be a transposed matrix of a weighted average matrix of two explicitly defined matrices.

MIP prediction of chroma component

In the discussion of standardization of VVC, five basic prediction modes (Planar, DC, Horizontal, Vertical, DM (Direct Mode)) and three LM (Linear Model) modes were considered for intra prediction for a chroma block. The LM mode is also referred to as an LM_Chroma mode or a Cross Component Linear Mode (CCLM) mode.

The LM mode is a mode for predicting a chroma signal from a luma signal by using the correlation between channels between the luma signal and the chroma signal. In the LM mode, a linear model (LM) between the luma signal and the pixel values of the chroma signal is determined, and the predicted signal of the chroma block is calculated based on the restored luma signal of the corresponding luma block using the linear model. do. In the DM mode, the current chroma block uses the same prediction mode of the corresponding luma block as it is.

When the LM mode (ie, CCLM) is not applied to the chroma block, a syntax element (intra_chroma_pred_mode) specifying an intra prediction mode for the chroma block is coded. The intra prediction mode of the chroma block is determined as follows depending on the intra prediction mode (lumaIntraPredMode) of the luma block corresponding to the intra_chroma_pred_mode. For example, if intra_chroma_pred_mode = 4, a direct mode (DM) mode is applied as an intra prediction mode of a chroma block, and thus an intra prediction mode of a corresponding luma block is applied to the chroma block.

In the discussion of VVC standardization, the matrix-based intra prediction (MIP) technique is used only for the luma component, and its application is excluded for the chroma component, due to the coding complexity and memory bandwidth due to the required set of matrix and offset. Was introduced. Accordingly, in VVC draft 5, when determining an intra prediction mode of a chroma block, when the corresponding luma block is matrix-based intra prediction (MIP) coded, the encoder and decoder define the MIP mode applied to the luma block in the mapping table. Was configured to convert to the normal intra prediction mode.

In the following, several approaches for applying matrix-based intra prediction (MIP) to chroma components are presented.

According to an aspect of the present disclosure, a video encoder and a decoder are provided with blocks (chroma blocks) of chroma components at a corresponding position from the MIP mode of a block (luma block) of a luma component predicted by matrix-based intra prediction (MIP). It is possible to induce the MIP mode for. Accordingly, for matrix-based intra prediction (MIP) coded chroma blocks, signaling of a syntax element that explicitly specifies the MIP mode used by the encoder may be omitted.

According to an aspect of the present disclosure, instead of separately defining sets of a matrix and an offset used for matrix-based intra prediction (MIP) for a chroma block and a luma block, according to a video sampling format (or chroma format), The matrix and offset sets defined for the luma block may be commonly used for the chroma block, or the matrix and offset sets to be used for the chroma block may be derived from the matrix and offset sets defined for the luma block. Table 4 shows a sampling format of a video indicated by a syntax element (sps_chroma_format_idc; cIdx) signaled at the SPS level.

For example, when the video sampling format (or chroma format) is 4:4:4, the encoder and the decoder may use the matrix and offset sets defined for the luma block as they are for the chroma block.

As another example, when the resolution of the luma component and the chroma component are different (e.g., when the chroma format is 4:2:0), the video encoder and the decoder reduce the luma block matrix and offset by the luma/chroma block size ratio. By sampling, the MIP matrix and offset to be applied to the chroma block can be derived. When the chroma format of the video is 4:2:0, the chroma block has a size of 1/2 each of the length and width compared to the luma block. Therefore, the encoder and the decoder downsample the matrix and offset of the luma block to obtain a chroma block. You can create a 1/2 size matrix and offset to be applied to.

As a downsampling method, a method of subsampling a value at a specific position in a matrix of a luma block (for example, a method of generating a reduced matrix from values of even or odd rows and columns) can be used. A method of applying filtering to a matrix of blocks may be used.

In order to avoid the complexity that may be introduced due to the derivation process of downsampled matrix and offset, in some embodiments, matrix-based intra prediction is allowed for chroma blocks only when the sampling format of the video is 4:4:4. It could be. In addition, in some embodiments, matrix-based intra prediction may be allowed for blocks of chroma components (chroma blocks) only when the luma block at the same location is predicted by matrix-based intra prediction (MIP). In such a case, the MIP mode for blocks of chroma components (chroma blocks) may be considered the same as the MIP mode used for the co-located luma block. Further, the matrix and offset sets used for the MIP mode of the luma block may be used as it is for the chroma block.

In some embodiments, matrix-based intra prediction may be allowed for a chroma block only when the sampling format of the video is 4:4:4 and a single tree is used to divide the CTU.

According to some other embodiments, instead of allowing a matrix-based intra prediction for a chroma block, a CCLM for generating a prediction signal of a chroma block using a linear model from reconstructed values of a matrix-based intra prediction-coded luma block (Cross -Component Linear Model) technique may be allowed.

Temporal matrix and offset

The number of sets of matrices and offsets available for matrix-based intra prediction (MIP) may be preferably selected adaptively according to characteristics of the video data. According to an aspect of the present disclosure, the video encoder adaptively changes the number of sets of matrices and offsets available for matrix-based intra prediction (MIP) in units of sequences, pictures, or subgroups of pictures through RD-cost calculation. I can.

In one embodiment, in matrix-based intra prediction (MIP), N sets of matrices and offsets that are always allowed to be used irrespective of the characteristics of video data may be defined. In addition, L matrices and sets of offsets that are selectively allowed to be used in a higher-level unit may be defined. The video encoder may select M sets (M is less than or equal to L) to be used at the current higher level from among L sets to be used at the current higher level. Accordingly, the number of sets of matrices and offsets that can be determined to be usable for matrix-based intra prediction (MIP) in a higher level unit is at least N and at most N+L. Here, the higher level unit may be a sequence, a picture, a subgroup unit of a picture, or the like. A set of matrices and offsets allowed for selective use may be referred to as a temporary set, and a matrix for which selective use is allowed may be referred to as a temporal matrix. The set of matrices and offsets that are always allowed to be used may be referred to as a basic set.

The video encoder may signal a 1-bit flag indicating whether or not a temporary set is further used in addition to the basic set on a higher level unit. When a temporary set is used, the video encoder may signal the number of temporary sets to be used in a current higher-level unit and index information of the selected temporary sets as a bitstream. In an embodiment in which the same number of temporary sets are always used, signaling about the number of temporary sets to be used in a current higher level unit may be omitted.

When starting to decode a block of a new higher level (eg, a new sequence), the video decoder may parse a 1-bit flag indicating whether to use a temporary set in the higher level syntax. When the 1-bit flag indicates that the temporary set is used at the current higher level, the video decoder may parse a syntax element indicating the number of temporary sets used at the current higher level. In addition, the video decoder may parse syntax elements indicating the index of each temporary set used at the current higher level among predefined available temporary sets based on the number of temporary sets used at the current higher level. . In performing matrix-based intra prediction on blocks included in a higher level, the video decoder may use predefined basic sets and temporary sets selected at the higher level. The video decoder may construct a matrix consisting of predefined basic sets and selected temporary sets and a list of offsets. Each temporary set in the list of matrixes and offsets can be identified by a new index.

Signaling in MIP mode

For the coding unit (CU) coded in the intra prediction mode, a flag indicating whether the intra prediction type is matrix-based intra prediction (MIP) may be signaled. When matrix-based intra prediction (MIP) is applied, among a plurality of available MIP modes, one or more syntax elements indicating the MIP mode used in the current coding unit may be additionally signaled.

Similar to the existing intra prediction mode (hereinafter,'normal intra prediction mode'), an MPM list may be used to represent the MIP mode used in the current coding unit. Hereinafter, for matrix-based intra prediction (MIP), a method of deriving a history-based MIP_MPM list in which MIP modes of previously coded blocks for each block size are managed as an MPM list is disclosed. The MIP_MPM list is maintained and updated during the encoding/decoding process.

A plurality of MIP_MPM lists may be maintained and used according to information of at least one of a block size (width, height) and signal characteristics. As an example, the number of allowed MIP modes is different according to the size of the block to which MIP is applied, and thus, a separate candidate MIP mode list may be managed for each MipSizeId = {0, 1, 2}.

The MIP_MPM list consists of M unique MIP modes, where M is adaptively determined by at least one of the width, height, size, and signal characteristics of the block, or a fixed value regardless of these information (e.g., 3) may have.

Whenever a block is coded in MIP mode, the used MIP mode is added to the last entry of the MIP_MPM list corresponding to the size of the block, and if a duplicate mode is found in the MIP_MPM list, it is removed from the MIP_MPM list, otherwise the MIP_MPM list The first entry of may be removed from the MIP_MPM list.

When a new slice is found, the MIP_MPM list may be reset to predefined MIP modes. When starting encoding/decoding of the CTU, the MIP_MPM list may be inherited from the last block of the CTU to the left of the current CTU. When the current CTU is on the left boundary of the picture, the table can be inherited from the last block of the CTU above the current CTU.

When the MIP mode selected for the current block exists in the MIP_MPM list corresponding to the size of the current block, the encoder signals a 1-bit flag (eg, MIP_HMPM_flag) of a first value indicating that the MIP mode of the current block is MPM, A syntax element (eg, MIP_HMPM_index) specifying one MIP mode from the MIP_MPM list corresponding to the size of the current coding block may be signaled. MIP_HMPM_index may be coded as a truncated binary code.

When the MIP mode selected for the current block does not exist in the MIP_MPM list corresponding to the size of the current block, the encoder is a 1-bit flag with a second value indicating that the MIP mode of the current block is not MPM (eg, MIP_HMPM_flag) Is signaled, and a syntax element (eg, MIP_HMPM_remainder) indicating one of the remaining non-MPMs excluding MPMs may be signaled. MIP_HMPM_remainder may be coded using a fixed length code.

When the current block is coded in the MIP mode, the decoder parses a 1-bit flag (eg, MIP_HMPM_flag) indicating whether the MIP mode of the current block is MPM. When the 1-bit flag has a first value, the decoder parses a syntax element (eg, MIP_HMPM_index) specifying one MIP mode from the MIP_MPM list corresponding to the size of the current coding block. The decoder determines the MIP mode indicated by MIP_HMPM_index from the MIP_MPM list as the MIP mode for the current block.

If the 1-bit flag (e.g., MIP_HMPM_flag) has a second value, the decoder selects one of the remaining MIP modes (ie, non-MPMs) excluding the MPMs of the MIP_MPM list among the available MIP modes for the size of the current block. The indicated syntax element (eg, MIP_HMPM_remainder) is parsed. The decoder determines the non-MPM indicated by MIP_HMPM_remainder as the MIP mode for the current block.

In some other embodiments, different from the existing intra prediction mode (normal intra prediction mode), the MPM list may not be used for signaling of the MIP mode. For example, among a plurality of MIP modes, one syntax element (eg, intra_mip_mode) that can be coded with a truncated binary code indicating the MIP mode used in the current coding unit is used. I can.

A part of the example coding unit syntax proposed based on the VVC 5 draft is provided below. In the syntax below, the graying of the elements is used to aid understanding.

When intra_mip_flag [x0][y0] is 1, it indicates that the intra prediction type of the current block is matrix-based intra prediction (MIP). If intra_mip_flag [x0][y0] is 0, it indicates that the intra prediction type of the current block is normal intra prediction, not matrix-based intra prediction. If intra_mip_flag [x0][y0] does not exist, it can be inferred that it is equal to 0. intra_mip_mode [x0][y0] indicates the MIP mode used for the current block in matrix-based intra prediction (MIP), and is expressed as a truncated binary code.

Most Probable Mode (MPM)

In a conventional approach, intra prediction coding using Most Probable Mode (MPM) can be used. For example, in HEVC, a list of 3 MPMs is constructed from the intra prediction modes of the left and upper blocks. The drawback of this method is that more modes (intra mode, not MPM) belong to non-MPMs that have to be coded with more bits. Several methods have been proposed to extend the number of MPMs to 3 or more entries (eg 6 MPM modes). However, constructing such an MPM list with more entries may require more checks and conditions, which can result in a more complex implementation.

In order to keep the complexity of constructing the MPM list low, an MPM list including 6 MPM candidates may be constructed using intra prediction modes of the left block and the upper block adjacent to the current block. MPM candidates may be composed of a default intra prediction mode (eg, a PLANAR mode), an intra prediction mode of a neighboring block, and an intra prediction mode derived from an intra prediction mode of the neighboring block. When the intra prediction mode of the neighboring block is not used (for example, when the neighboring block is inter-predicted, the neighboring block is located in a different slice or another tile), the intra prediction mode of the intra prediction mode of the neighboring block is PLANAR. Can be set.

According to the type of intra prediction mode of the left block mode (Left) and the top block mode (Above), it is divided into 4 main branches. Left and Above are different, and when both modes are directional mode, the difference between Left and Above According to, the MPM list may be generated by further dividing into additional four branches. In the table below, Max refers to the larger mode among Left and Above, and MIN refers to the smaller mode among Left and Above.

The video encoder may signal a 1-bit flag (eg, mpm_flag) indicating whether the intra prediction mode of the current block corresponds to MPM. Typically, when the intra prediction mode of the current block corresponds to MPM, an MPM index indicating one of 6 MPMs (ie, PLANAR mode) will be additionally signaled.

Note that in Table 6, the PLANAR mode is always included in the MPM list. That is, 6 MPMs can be divided into PLANAR and 5 non-PLANAR MPMs. Therefore, when the intra prediction mode of the current block corresponds to MPM, the encoder first signals whether the intra prediction mode of the current block is a PLANAR mode (e.g., using a 1-bit flag), and then signals the intra prediction mode of the current block. As long as is equal to one of the remaining five non-PLANAR MPMs, it may be efficient to additionally signal an MPM index indicating one of the non-PLANAR MPMs. If the value of the bit flag (e.g., mpm_flag) indicates that the intra prediction mode of the current block corresponds to the MPM, before determining the list of five non-PLANAR MPMs, whether the intra prediction mode of the current block is the PLANAR mode. A 1-bit flag indicating whether or not can be parsed.

When the intra prediction mode of the current block does not correspond to MPM, a syntax element indicating one of the remaining 61 non-MPMs excluding 6 MPMs may be encoded using a truncated binary code. have.

Removal of mapping table between MIP mode and regular mode

As described above, in VVC draft 5, each MPM list is used for signaling of the MIP mode and the regular mode, and a mapping table between the MIP mode and the regular mode is required to configure the MIP list. For example, in deriving an MPM list for a block (ie, a normal block) coded in a normal intra prediction mode, when the left block or the upper block is matrix-based intra prediction (MIP) coded, the MIP mode of the left block Alternatively, the MIP mode of the upper block is converted to a normal intra prediction mode defined in the mapping table.

According to an aspect of the present disclosure, in deriving an MPM list for a regular block, when the left block and the upper block are matrix-based intra prediction (MIP) coded, the relationship with which MIP mode is applied to the neighboring block Without, the mode of the left block (Left) and the mode of the upper block (Above) may be regarded as a predefined regular mode. As a result, the need for the encoder and decoder to store a mapping table between MIP modes and regular modes in a memory can be eliminated.

In some embodiments, when the left block is matrix-based intra prediction (MIP) coded, the normal intra prediction mode of the left block may be regarded as the first mode (regardless of the MIP mode of the left block), and the upper block is MIP. When coded in the mode, the normal intra prediction mode of the upper block may be regarded as the second mode (regardless of the MIP mode of the upper block). The first mode and the second mode may be predefined identically or differently, and may be signaled in a higher level syntax.

In some other embodiments, in deriving the MPM list for a normal block, when the MIP mode is applied to the neighboring block, the normal intra prediction mode of the neighboring block (regardless of the MIP mode of the neighboring block) is the PLANAR mode ( Or DC mode). Due to the characteristics of the matrix-based intra prediction (MIP) technique including an average value operation and an interpolation operation, the residual signal of a block to which MIP is applied may have a low frequency component dominant in the transform domain. Note that the characteristics of such a residual signal may be similar to that of a block to which the PLANAR mode (or DC mode) is applied.

Similarly, even when inducing a chroma direct mode (DM), when matrix-based intra prediction (MIP) coding is applied to a luma block at the same location, instead of using a mapping table between the MIP mode and the normal mode, the luma block The intra prediction mode may be regarded as a PLANAR mode (or DC mode).

Accordingly, the decoder parses the syntax element specifying the intra prediction mode for the chroma block, and is indicated by the syntax element that the intra prediction mode of the chroma block uses the intra prediction mode of the luma block at the same position as it is. When MIP is applied to the luma block at the same location, the intra prediction mode of the luma block may be regarded as a PLANAR mode (or DC mode). That is, when MIP is applied to the luma block at the same location in the chroma direct mode (DM), the intra prediction mode of the chroma block may be determined as the PLANAR mode (or DC mode).

However, in the chroma direct mode (DM), when the video sampling format is 4:4:4 and the luma block at the same position is predicted by matrix-based intra prediction (MIP), matrix-based intra prediction is performed for the chroma block. It may be acceptable. In this case, the MIP mode for blocks of chroma components (chroma blocks) may be regarded as the same as the MIP mode used for the luma block at the same location.

6 is a flow chart illustrating a method of decoding video data, employing some of the above-described improvements, according to an embodiment of the present invention. The method of FIG. 6 may be performed by a video decoder such as the video decoding apparatus illustrated in FIG. 4. For example, the entropy decoding unit 410 and the intra prediction unit 442 may be involved in one or more steps described below.

The video decoder may obtain information about the luma prediction mode and information about the chroma prediction mode of the current coding block from the bitstream (S610). The video decoder may decode an encoded bitstream of video data and obtain information about a luma prediction mode and information about a chroma prediction mode.

The video decoder may derive a luma intra prediction type and a luma intra prediction mode of the current coding block based on information on the luma prediction mode (S620). The luma intra prediction type may include matrix based intra prediction (MIP) and regular intra prediction. The information on the luma prediction mode signals a syntax element indicating a luma intra prediction type of the current coding block, a syntax element indicating a matrix-based intra prediction mode selected for the current coding block, and a normal intra prediction mode selected for the current coding block. It may include one or more syntax elements for doing so.

As an example, the video decoder may parse a syntax element (eg, intra_mip_flag) indicating the luma intra prediction type of the current coding block from the bitstream. When the syntax element indicates that the luma intra prediction type of the current coding block is matrix-based intra prediction, the video decoder is a syntax element indicating a matrix-based intra prediction mode for the current coding block, and the width of the current coding block And a syntax element (eg, intra_mip_mode) represented by a truncated binary code specifying one of a plurality of matrix-based intra prediction modes allowed for height.

When the syntax element (e.g., intra_mip_flag) indicates that the luma intra prediction type of the current coding block is normal intra prediction, the video decoder is based on the intra prediction mode of neighboring blocks adjacent to the current coding block. Mode) Candidates are derived to construct an MPM list for the current coding block, and a luma intra prediction mode for the current coding block may be derived based on the MPM list. To this end, the video decoder may parse one or more syntax elements related to MPM. When the video decoder derives MPM candidates based on intra prediction modes of neighboring blocks adjacent to the current coding block, when the intra prediction type of the neighboring block is matrix-based intra prediction, the intra prediction mode of the neighboring block is PLANAR. It can be regarded (set) as a mode.

The video decoder may determine a chroma intra prediction mode of the current coding block based on a luma intra prediction type and a luma intra prediction mode of the current coding block, and information on the chroma prediction mode (S630). The information on the chroma prediction mode may include a syntax element (eg, intra_chroma_pred_mode), which may have one of 0 to 4, specifying the chroma intra prediction mode. In addition, the information on the chroma prediction mode may include a flag indicating whether CCLM is applied to the chroma block (eg, cclm_mode_flag) and index information (cclm_mode_idx) indicating one of three available CCLM modes. When CCLM is applied to a chroma block, intra_chroma_pred_mode may not exist.

As an example, in the video decoder, information about a chroma prediction mode indicates a direct mode (DM), a luma intra prediction type of a current coding block is a matrix-based intra prediction, and a sampling format of the video data is 4:4:4. When it is determined that the chroma intra prediction type of the current coding block is the matrix-based intra prediction, and the chroma intra prediction mode corresponding to the chroma intra prediction type of the current coding block is a matrix derived as the luma intra prediction mode of the current coding block It may be determined that it is the same as the base intra prediction mode.

As another example, in the video decoder, information on a chroma prediction mode indicates a direct mode (DM), a luma intra prediction type of the current coding block is a matrix-based intra prediction type, and a sampling format of the video data is 4:2. When :0 or 4:2:2, the chroma intra prediction mode of the current coding block may be determined as the PLANAR mode.

As another example, when information about a chroma prediction mode indicates a direct mode (DM) and a luma intra prediction type of the current coding block is a normal intra prediction type, the video decoder is a chroma intra prediction mode of the current coding block May be determined to be the same as the normal intra prediction mode derived as the luma intra prediction mode of the current coding block.

The video decoder may generate chroma prediction samples of the current coding block based on the chroma intra prediction mode of the current coding block (S640). The video decoder may generate chroma prediction samples of the current coding block by selectively performing matrix-based intra prediction or regular intra prediction.

As an example, in response to determining that the chroma intra prediction mode of the current coding block is the same as the matrix-based intra prediction mode derived as the luma intra prediction mode of the current coding block, the video decoder may respond to the derived matrix-based intra prediction mode. A corresponding matrix-based intra prediction may be performed to generate chroma prediction samples of the current coding block. The video decoder may derive an input boundary vector using adjacent chroma samples adjacent to the current coding block based on the width and height of the current coding block. Video decoder Chroma prediction samples for the current coding block may be generated based on a matrix-vector multiplication between a matrix predefined for the matrix-based intra prediction mode and the input boundary vector. The video decoder may derive a chroma prediction block for the current coding block based on the chroma prediction samples.

As another example, in response to determining the chroma intra prediction mode of the current coding block as the PLANAR mode, the video decoder may generate chroma prediction samples of the current coding block by performing regular intra prediction according to the PLANAR mode.

As another example, in response to determining that the chroma intra prediction mode of the current coding block is the same as the normal intra prediction mode derived as the luma intra prediction mode of the current coding block, the video decoder According to the normal intra prediction, chroma prediction samples of the current coding block may be generated.

In the following, some advanced techniques for inter prediction coding video data are disclosed. Some techniques described below relate to efficiently signaling motion information, and some other techniques relate to adaptively performing interpolation filtering according to motion information.

The motion vector may be composed of an n-dimensional vector, that is, n direction components. According to the technique of the present disclosure, it may be allowed that motion information is independently coded for each direction. The encoder may signal information indicating whether motion information of a coding block is independently coded for each direction.

The motion information may mean information explicitly signaled by the encoder, such as a motion set for each direction and a prediction mode of a motion vector. Also, the motion information may mean or include information derived from these information, that is, a value of a motion vector finally obtained by a decoder or a precision of a differential motion vector. Here, accuracy indicates how many decimal places each component of a vector is expressed, and may be referred to as a resolution of a motion vector. The motion set may consist of information such as vector components and accuracy. The vector component may be composed of an actual motion vector or a differential motion vector, and the accuracy information may be composed of a motion vector accuracy or an accuracy of a differential motion vector.

When motion information is independently coded for each direction, it may mean that a motion set is individually coded or decoded for each direction. For example, vector components in each direction may be expressed with separate accuracy and may be encoded/decoded as separate motion sets. The fact that the motion information defined for each direction is dependently coded means that, for at least two directions among n directions, the vector component of each direction is expressed with one accuracy and encoded/decoded as one motion set. can do.

Information indicating whether motion information is independently or dependently coded for each direction may be signaled by the encoding apparatus. As an example, the information may be transmitted as a higher-level syntax element such as a Supplemental Enhancement Information (SEI) message, APS, SPS, PPS, and slice/tile/tile group header. As another example, the information may be transmitted in a basic decoding processing unit and a divided block unit, or in both a higher level syntax and a block unit. Alternatively, the information may be derived from whether motion information of reference candidate blocks is independently coded and motion information of the reference candidate blocks.

In video data, which is a sequence of 2D pictures, a motion vector may be composed of components in two directions (eg, x direction and y direction). According to the technique of the present disclosure, motion sets for the x and y directions may be independently coded.

In coding motion information, a syntax element indicating whether independent coding of each element constituting motion information is allowed may be signaled at a higher level. For example, the value of the sps_MV_independent_enable flag signaled in the SPS indicates whether independent coding of elements constituting motion information is allowed. When sps_MV_independent_enable is 1, information indicating whether each element of motion information is independently encoded for each sub-unit such as picture/slice/tile may be additionally signaled. For example, the value of MV_independent_flag signaled in the slice header specifies whether each element of motion information has been independently decoded for the current decoding unit (eg, CU). When MV_independent_flag = 0, motion information in the x and y directions is coded as one motion set. When MV_independent_flag = 1, motion information is coded by being divided into a motion set for the x direction and a motion set for the y direction.

Hereinafter, a technique for adaptively performing interpolation filtering using motion information is disclosed in order to increase coding efficiency.

The interpolation filter refers to a filter used to change the resolution of a reference picture in coding techniques using motion information (e.g., inter prediction coding, intra block copy, CIIP technology that combines an inter prediction signal and an intra prediction signal, etc.) .

In some embodiments, configuration information for each of one or more interpolation filters to be used for interpolation may be directly transmitted from an encoder to a decoder. The configuration information of the interpolation filter may include information such as the number of taps of the interpolation filter, interpolation filter coefficients, the direction of the interpolation filter, and the shape of the interpolation filter.

In other embodiments, the encoder and the decoder may use one or more interpolation filter lists in which information on a plurality of interpolation filters is predefined. Each list includes configuration information of each of the interpolation filters included in the list. Information on the interpolation filter may be indirectly transmitted from the encoder to the decoder. For example, as for the information on the interpolation filter, an index indicating an available interpolation filter list among a plurality of interpolation filter lists may be indirectly transmitted as information on the interpolation filter.

When a plurality of interpolation filters are available, index information of an interpolation filter to be used among the plurality of interpolation filters may be explicitly signaled by the encoder. As an example, the index information may be transmitted as a higher level syntax element such as an SEI message, APS, SPS, PPS, slice/tile/tile group header. As another example, the index information may be transmitted in a basic decoding processing unit and a divided block unit, or in both a higher level syntax and a block unit. Alternatively, the index information of the interpolation filter to be used for the current block may be derived from at least one of a decoded neighboring block, a decoded reference block, and a neighboring block of the decoded reference block.

In one embodiment, the encoder and decoder may select the type of interpolation filter to use according to the position of the reference pixel among the available interpolation filters. In addition, information on additional interpolation filters in addition to predefined interpolation filters may be transmitted through high-level syntax such as SEI message, APS, SPS, PPS, and slice/tile/tile group header.

If there are several types of interpolation filters available for a given position x, the encoder may signal information (eg, filter index) specifying an interpolation filter used at that position. This filter index may be transmitted in a basic decoding processing unit and a divided block unit. Alternatively, an interpolation filter to be used for the decoding target block by using information that has already been decoded for at least one or more of the decoding target block, the decoded neighboring block, the decoded reference block, and the neighboring blocks of the decoded reference block. The index of may be derived. Here, the already decoded information may be an accuracy of a motion vector, an accuracy of a differential motion vector, and an accuracy of a final motion vector.

According to some possible embodiments, a syntax element indicating whether to selectively use an interpolation filter in a higher level syntax such as APS, PPS, Slice/tile header, etc. may be signaled at a higher level. When selective use of an interpolation filter is allowed, information specifying a filter set that can be used according to each interpolation position in a higher level syntax may be further signaled. The decoder may define a filter using a predefined filter and filter information derived from higher level syntax information. The encoder and decoder may derive an interpolation filter using information such as motion vector accuracy, differential motion vector accuracy, final motion vector accuracy, block size, and position. The encoder may select an appropriate filter through RD tests for several interpolation filters that can be used, and may signal the index of the selected filter to the decoder.

In addition, the interpolation filter to be used for the current block may be derived from referenced neighboring blocks to derive a motion prediction mode of the current block and/or a motion vector of the current block.

In an exemplary embodiment, when the current block is encoded in the merge mode, the decoder may perform interpolation of the current block using the same filter as the interpolation filter used in the merged block. The decoder may refer only to motion information of a block to be merged and may perform interpolation by decoding index information of an interpolation filter.

In an exemplary embodiment, when the current block is encoded in the affine mode, the decoder may use motion information of a neighboring reference block of the current block to derive a control point motion vector (CPMV) of the current block. . In this case, the decoder may derive the interpolation filter index of the current block by using the interpolation filter index information used in the neighboring reference block. Alternatively, the decoder may decode the interpolation filter index information for the explicitly signaled current block.

In an exemplary embodiment, when the current block is encoded with Pair-wise average Merge Candidates (PMC), interpolation of the current block using interpolation filter index information used in reference blocks used to obtain an average value of a motion vector Filter index information can be derived. Alternatively, the decoder may decode the interpolation filter index information for the explicitly signaled current block.

In an exemplary embodiment, when the current block is encoded by HMVP (History based MV Prediction), the interpolation filter index of the current block may be derived using interpolation filter index information used for the selected HMVP candidate. Alternatively, the decoder may decode the interpolation filter index information for the explicitly signaled current block.

In the above description, it should be understood that the exemplary embodiments may be implemented in many different ways. The functions or methods described in one or more examples may be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled "...unit" to particularly emphasize their implementation independence.

Meanwhile, various functions or methods described in the present disclosure may be implemented as instructions stored in a non-transitory recording medium that can be read and executed by one or more processors. The non-transitory recording medium includes, for example, all kinds of recording devices in which data is stored in a form readable by a computer system. For example, the non-transitory recording medium includes a storage medium such as an erasable programmable read only memory (EPROM), a flash drive, an optical drive, a magnetic hard drive, and a solid state drive (SSD).

The above description is merely illustrative of some embodiments according to the technical idea of the present invention, and those of ordinary skill in the technical field to which the present invention pertains, various modifications and variations without departing from the essential characteristics of the present invention. This will be possible. Accordingly, the illustrated embodiments are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

Claims (14)

As a method of decoding video data,
Acquiring information about a luma prediction mode and information about a chroma prediction mode of a current coding block from the bitstream;
Deriving a luma intra prediction type and a luma intra prediction mode of the current coding block based on the information on the luma prediction mode-The luma intra prediction type is a matrix based intra prediction (MIP) and a normal intra Including regular intra prediction -;
Determining a chroma intra prediction mode of the current coding block based on a luma intra prediction type and a luma intra prediction mode of the current coding block and information about the chroma prediction mode; And
Generating chroma prediction samples of the current coding block based on a chroma intra prediction mode of the current coding block,
Determining a chroma intra prediction mode of the current coding block,
When the information on the chroma prediction mode indicates a direct mode (DM), the luma intra prediction type of the current coding block is matrix-based intra prediction, and the sampling format of the video data is 4:4:4,
Matrix-based intra, which is determined that the chroma intra prediction type of the current coding block is matrix-based intra prediction, and the chroma intra prediction mode corresponding to the chroma intra prediction type of the current coding block is derived as the luma intra prediction mode of the current coding block Determining that it is the same as the prediction mode
Including a decoding method. The method of claim 1,
Generating chroma prediction samples of the current coding block based on the chroma intra prediction mode,
In response to determining that the chroma intra prediction mode of the current coding block is the same as the matrix-based intra prediction mode derived as the luma intra prediction mode of the current coding block,
Deriving an input boundary vector using neighboring chroma samples adjacent to the current coding block based on the width and height of the current coding block;
Generating chroma prediction samples for the current coding block based on a matrix-vector multiplication between the input boundary vector and a matrix predefined for the matrix-based intra prediction mode; And
Deriving a chroma prediction block for the current coding block based on the chroma prediction samples
It characterized in that it comprises a, decoding method. The method of claim 2,
A matrix defined in advance for the matrix-based intra prediction mode is commonly used to generate luma prediction samples and chroma prediction samples. The method of claim 1,
Determining a chroma intra prediction mode of the current block,
The information on the chroma prediction mode indicates a direct mode (DM), a luma intra prediction type of the current coding block is a matrix-based intra prediction type, and a sampling format of the video data is 4:2:0 or 4:2. When :2, further comprising the step of determining a chroma intra prediction mode of the current coding block as a PLANAR mode. The method of claim 1,
Determining a chroma intra prediction mode of the current block,
When the information on the chroma prediction mode indicates a direct mode (DM) and the luma intra prediction type of the current coding block is a normal intra prediction type, the chroma intra prediction mode of the current coding block is the luma of the current coding block. The method further comprising the step of determining that it is the same as the normal intra prediction mode derived by the intra prediction mode. The method of claim 1,
The step of deriving a luma intra prediction type and a luma intra prediction mode of the current coding block,
Parsing a syntax element indicating a luma intra prediction type of the current coding block; And
When the syntax element indicates that the luma intra prediction type of the current coding block is matrix-based intra prediction, as a syntax element indicating a matrix-based intra prediction mode for the current coding block, the width and height of the current coding block Parsing a syntax element represented by a truncated binary code specifying one of a plurality of matrix-based intra prediction modes allowed for
It characterized in that it comprises a, decoding method. The method of claim 1,
The step of deriving a luma intra prediction type and a luma intra prediction mode of the current coding block,
Parsing a syntax element indicating a luma intra prediction type of the current coding block; And
When the syntax element indicates that the luma intra prediction type of the current coding block is normal intra prediction, MPM (Most Probable Mode) candidates are derived based on intra prediction modes of neighboring blocks adjacent to the current coding block, and the current Constructing an MPM list for the coding block; And
Deriving a luma intra prediction mode for the current coding block based on the MPM list,
In deriving MPM candidates based on intra prediction modes of neighboring blocks adjacent to the current coding block, when the intra prediction type of the neighboring block is matrix-based intra prediction, the intra prediction mode of the neighboring block is regarded as a PLANAR mode. It characterized in that the decoding method. As an apparatus for decoding video data,
A decoding unit that obtains information about a luma prediction mode and information about a chroma prediction mode of a current coding block from the bitstream; And
A luma intra prediction type and a luma intra prediction mode of the current coding block are derived based on the information on the luma prediction mode, and information about a luma intra prediction type and a luma intra prediction mode and the chroma prediction mode of the current coding block Based on, an intra prediction unit that determines a chroma intra prediction mode of the current coding block and generates chroma prediction samples of the current coding block based on the chroma intra prediction mode of the current coding block
Including,
The luma intra prediction type includes matrix based intra prediction (MIP) and regular intra prediction,
The intra prediction unit,
As part of determining a chroma intra prediction mode of the current coding block,
When the information on the chroma prediction mode indicates a direct mode (DM), the luma intra prediction type of the current coding block is matrix-based intra prediction, and the sampling format of the video data is 4:4:4, the current Matrix-based intra prediction in which the chroma intra prediction type of the coding block is determined to be the matrix-based intra prediction, and the chroma intra prediction mode corresponding to the chroma intra prediction type of the current coding block is derived as the luma intra prediction mode of the current coding block A decoding device, characterized in that it is determined to be the same as the mode. The method of claim 8,
The intra prediction unit,
As part of generating chroma prediction samples of the current coding block based on the chroma intra prediction mode,
In response to determining that the chroma intra prediction mode is the same as a matrix-based intra prediction mode derived as a luma intra prediction mode of the current coding block,
Deriving an input boundary vector using neighboring chroma samples adjacent to the current coding block based on the width and height of the current coding block;
Generating chroma prediction samples for the current coding block based on a matrix-vector multiplication between the input boundary vector and a matrix predefined for the matrix-based intra prediction mode; And
Deriving a chroma prediction block for the current coding block based on the chroma prediction samples
A decoding device, characterized in that to perform. The method of claim 9,
A decoding apparatus, characterized in that a matrix predefined for the matrix-based intra prediction mode is commonly used to generate luma prediction samples and chroma prediction samples. The method of claim 8,
The intra prediction unit,
The information on the chroma prediction mode indicates a direct mode (DM), a luma intra prediction type of the current coding block is a matrix-based intra prediction type, and a sampling format of the video data is 4:2:0 or 4:2. When :2, the decoding apparatus, characterized in that the chroma intra prediction mode of the current coding block is determined as a PLANAR mode. The method of claim 8,
The intra prediction unit,
When the information on the chroma prediction mode indicates a direct mode (DM) and the luma intra prediction type of the current coding block is a normal intra prediction type, the chroma intra prediction mode of the current coding block is the luma of the current coding block. A decoding apparatus, characterized in that it is determined that it is the same as the normal intra prediction mode derived by the intra prediction mode. The method of claim 8,
The intra prediction unit,
As part of deriving a luma intra prediction type and a luma intra prediction mode of the current coding block,
Parsing a syntax element indicating a luma intra prediction type of the current coding block; And
When the syntax element indicates that the luma intra prediction type of the current coding block is matrix-based intra prediction, as a syntax element indicating a matrix-based intra prediction mode for the current coding block, the width and height of the current coding block Parsing a syntax element represented by a truncated binary code specifying one of a plurality of matrix-based intra prediction modes allowed for
A decoding device, characterized in that to perform. The method of claim 8,
The intra prediction unit,
As part of deriving a luma intra prediction type and a luma intra prediction mode of the current coding block,
Parsing a syntax element indicating a luma intra prediction type of the current coding block; And
When the syntax element indicates that the luma intra prediction type of the current coding block is normal intra prediction, MPM (Most Probable Mode) candidates are derived based on intra prediction modes of neighboring blocks adjacent to the current coding block, and the current Constructing an MPM list for the coding block; And
Deriving a luma intra prediction mode for the current coding block based on the MPM list
And
When the intra prediction unit derives MPM candidates based on intra prediction modes of neighboring blocks adjacent to the current coding block, when the intra prediction type of the neighboring block is matrix-based intra prediction, the intra prediction mode of the neighboring block is selected. A decoding device, characterized in that regarded as a PLANAR mode.

Images