KR20210000637A - Method and apparatus for deriving motion information - Google Patents

Abstract

Disclosed are a method for deriving motion information and an image decoding device. According to one embodiment of the present invention, in a method for deriving motion information of a triangular block, provided is the method for deriving motion information including: a step of decoding an index indicating any one of candidate blocks included in a candidate list from a bitstream; a step of determining whether or not a reference direction of a target candidate block, which is a candidate block indicated by the index, is bi-prediction; and a step of deriving one of the bidirectional motion information of the target candidate block as the motion information of the triangular block when the reference direction of the target candidate block is bidirectional. The present invention efficiently derives motion information for predicting the triangular block.

Description

Motion information derivation method and video decoding device {METHOD AND APPARATUS FOR DERIVING MOTION INFORMATION}

The present invention relates to encoding and decoding of an image, and more particularly, to a motion information derivation method and an image decoding apparatus in which unidirectional prediction information is derived from bidirectional prediction information to improve encoding and decoding efficiency.

Since moving picture data has a large amount of data compared to audio data or still image data, it requires a lot of hardware resources including memory in order to store or transmit itself without processing for compression.

Accordingly, when moving or transmitting moving picture data, the moving picture data is compressed and stored or transmitted using an encoder, and the decoder receives the compressed moving picture data, decompresses and reproduces the compressed moving picture data. As such video compression technologies, there are H.264/AVC and HEVC (High Efficiency Video Coding), which improves coding efficiency by about 40% compared to H.264/AVC.

However, the size, resolution, and frame rate of an image are gradually increasing, and accordingly, the amount of data to be encoded is also increasing. Accordingly, a new compression technique with higher encoding efficiency and higher quality improvement effect than the existing compression technique is required.

In order to meet these needs, the present invention aims to provide an improved video encoding and decoding technology. In particular, an aspect of the present invention is to efficiently derive motion information for prediction of a triangular block, thereby improving the efficiency of encoding and decoding. It is related to technology that improves.

An aspect of the present invention is a method of deriving motion information of a triangular block, the method comprising: decoding an index indicating any one of candidate blocks included in a candidate list from a bitstream; Determining whether a reference direction of a target candidate block, which is a candidate block indicated by the index, is bi-prediction; And when the reference direction of the target candidate block is bidirectional, deriving one of the two-way motion information of the target candidate block as the motion information of the triangular block.

Another aspect of the present invention is a decoder for decoding an index indicating any one of candidate blocks included in a candidate list from a bitstream; And determining whether a reference direction of a target candidate block, which is a candidate block indicated by the index, is bi-prediction, and when the reference direction of the target candidate block is bi-directional, among bidirectional motion information of the target candidate block It provides an image decoding apparatus including a prediction unit for inducing any one into motion information of a triangular block.

As described above, according to an embodiment of the present invention, motion information can be derived more accurately and efficiently, and thus accuracy and efficiency of encoding and decoding can be improved.

1 is an exemplary block diagram of an image encoding apparatus capable of implementing the techniques of the present disclosure.
2 is a diagram for explaining a method of dividing a block using a QTBTTT structure.
3 is a diagram for describing a plurality of intra prediction modes.
4 is an exemplary block diagram of an image decoding apparatus capable of implementing the techniques of the present disclosure.
5 is a flowchart illustrating a conventional method of classifying prediction modes.
6 is a diagram for describing locations of candidate blocks included in a candidate list.
7 is a diagram for explaining positions of candidate blocks used to construct a candidate list.
8 is a flowchart illustrating an embodiment of inducing motion information.
9 is a diagram for explaining examples of triangular blocks.
10 and 11 are flowcharts illustrating exemplary embodiments in which motion information is derived based on a weight.
12 is a flowchart for describing an embodiment of inducing motion information based on a distance difference between pictures.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding identification codes to elements of each drawing, it should be noted that the same elements have the same symbols as possible even if they are indicated on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

1 is an exemplary block diagram of an image encoding apparatus capable of implementing the techniques of the present disclosure. Hereinafter, an image encoding apparatus and sub-elements of the apparatus will be described with reference to FIG. 1.

The image encoding apparatus includes a block division unit 110, a prediction unit 120, a subtractor 130, a transform unit 140, a quantization unit 145, an encoding unit 150, an inverse quantization unit 160, and an inverse transform unit ( 165, an adder 170, a filter unit 180, and a memory 190 may be included.

Each component of the image encoding apparatus may be implemented by hardware or software, or by a combination of hardware and software. In addition, functions of each component may be implemented as software, and a microprocessor may be implemented to execute a function of software corresponding to each component.

One image (video) is composed of a plurality of pictures. Each picture is divided into a plurality of regions, and encoding is performed for each region. For example, one picture is divided into one or more tiles or/and slices. Here, one or more tiles may be defined as a tile group. Each tile or/slice is divided into one or more Coding Tree Units (CTUs). And each CTU is divided into one or more CUs (Coding Units) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in one CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks in one tile is encoded as the syntax of the tile or is encoded as the syntax of a tile group in which a plurality of tiles are collected, and information applied to all blocks constituting one picture is It is encoded in a picture parameter set (PPS) or a picture header. Further, information commonly referred to by a plurality of pictures is encoded in a sequence parameter set (SPS). In addition, information commonly referred to by one or more SPSs is encoded in a video parameter set (VPS).

The block dividing unit 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as the syntax of the SPS or PPS and transmitted to the video decoding apparatus.

After dividing each picture constituting the image into a plurality of coding tree units (CTUs) having a predetermined size, the block dividing unit 110 repetitively divides the CTU using a tree structure. (recursively) split. A leaf node in the tree structure becomes a coding unit (CU), which is a basic unit of coding.

As a tree structure, a quad tree (QuadTree, QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, or a binary tree (BinaryTree) in which an upper node is divided into two lower nodes. , BT), or a ternary tree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure in which two or more of these QT structures, BT structures, and TT structures are mixed. have. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT may be collectively referred to as MTT (Multiple-Type Tree).

2 shows a QTBTTT split tree structure. As shown in FIG. 2, the CTU may be first divided into a QT structure. The quadtree division may be repeated until the size of a splitting block reaches the minimum block size (MinQTSize) of a leaf node allowed in QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is divided into four nodes of a lower layer is encoded by the encoder 150 and signaled to the image decoding apparatus. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, it may be further divided into one or more of a BT structure or a TT structure. In the BT structure and/or the TT structure, a plurality of division directions may exist. For example, there may be two directions in which a block of a corresponding node is divided horizontally and a direction vertically divided. As shown in FIG. 2, when MTT splitting starts, a second flag (mtt_split_flag) indicating whether nodes are split, and if split, a flag indicating a split direction (vertical or horizontal) and/or a split type (Binary or Ternary). A flag indicating) is encoded by the encoder 150 and signaled to the image decoding apparatus.

As another example of a tree structure, when a block is divided using a QTBTTT structure, information about a CU split flag (split_cu_flag) indicating that the block has been split first and a QT split flag (split_qt_flag) indicating whether the split type is QT splitting is information from the encoding unit 150 ) And signaled to the video decoding apparatus. When it is indicated that the value of the CU split flag (split_cu_flag) is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a coding unit (CU), which is a basic unit of encoding. When indicating that the value of the CU split flag (split_cu_flag) is split, whether the split type is QT or MTT is identified through the value of the QT split flag (split_qt_flag). If the split type is QT, there is no additional information, and if the split type is MTT, a flag indicating additional MTT split direction (vertical or horizontal) (mtt_split_cu_vertical_flag) and/or a flag indicating MTT split type (Binary or Ternary) (mtt_split_cu_binary_flag) is encoded by the encoder 150 and signaled to the video decoding apparatus.

When QTBT is used as another example of the tree structure, two types of horizontally splitting a block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type splitting vertically (i.e., symmetric vertical splitting) Branches can exist. A split flag indicating whether each node of the BT structure is divided into blocks of a lower layer and split type information indicating a type to be divided are encoded by the encoder 150 and transmitted to the image decoding apparatus. Meanwhile, a type of dividing the block of the corresponding node into two blocks having an asymmetric shape may further exist. The asymmetric form may include a form of dividing a block of a corresponding node into two rectangular blocks having a size ratio of 1:3, or a form of dividing a block of a corresponding node in a diagonal direction.

The CU can have various sizes according to the QTBT or QTBTTT split from the CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (ie, a leaf node of QTBTTT) is referred to as a'current block'.

The prediction unit 120 predicts the current block and generates a prediction block. The prediction unit 120 includes an intra prediction unit 122 and an inter prediction unit 124.

In general, each of the current blocks in a picture can be predictively coded. In general, prediction of the current block is performed using an intra prediction technique (using data from a picture containing the current block) or an inter prediction technique (using data from a picture coded before a picture containing the current block). Can be done. Inter prediction includes both one-way prediction and two-way prediction.

The intra prediction unit 122 predicts pixels in the current block by using pixels (reference pixels) located around the current block in the current picture including the current block. There are a plurality of intra prediction modes according to the prediction direction. For example, as shown in FIG. 3, the plurality of intra prediction modes may include a non-directional mode including a planar mode and a DC mode, and 65 directional modes. Depending on each prediction mode, the surrounding pixels to be used and the equation are defined differently.

The intra prediction unit 122 may determine an intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit 122 may encode the current block using several intra prediction modes and select an appropriate intra prediction mode to use from the tested modes. For example, the intra prediction unit 122 calculates rate distortion values using rate-distortion analysis for several tested intra prediction modes, and has the best rate distortion characteristics among the tested modes. It is also possible to select an intra prediction mode.

The intra prediction unit 122 selects one intra prediction mode from among a plurality of intra prediction modes, and predicts the current block by using a neighboring pixel (reference pixel) and an equation determined according to the selected intra prediction mode. Information on the selected intra prediction mode is encoded by the encoder 150 and transmitted to the image decoding apparatus.

The inter prediction unit 124 generates a prediction block for the current block through a motion compensation process. A block that is most similar to a current block is searched for in a reference picture that is encoded and decoded before the current picture, and a prediction block for the current block is generated using the searched block. Then, a motion vector corresponding to a displacement between the current block in the current picture and the prediction block in the reference picture is generated. In general, motion estimation is performed on a luma component, and a motion vector calculated based on the luma component is used for both the luma component and the chroma component. Motion information including information on a reference picture used to predict a current block and information on a motion vector is encoded by the encoder 150 and transmitted to an image decoding apparatus.

The subtractor 130 generates a residual block by subtracting the prediction block generated by the intra prediction unit 122 or the inter prediction unit 124 from the current block.

The transform unit 140 converts the residual signal in the residual block having pixel values in the spatial domain into transform coefficients in the frequency domain. The transform unit 140 may transform residual signals in the residual block by using the total size of the residual block as a transform unit, or divide the residual block into two sub-blocks, which are transform regions and non-transform regions, Residual signals can be converted using only a block as a conversion unit. Here, the transform region subblock may be one of two rectangular blocks having a size ratio of 1:1 based on the horizontal axis (or vertical axis). In this case, a flag indicating that only the subblock has been transformed (cu_sbt_flag), directional (vertical/horizontal) information (cu_sbt_horizontal_flag), and/or location information (cu_sbt_pos_flag) are encoded by the encoder 150 and signaled to the video decoding apparatus. . In addition, the size of the transform region subblock may have a size ratio of 1:3 based on the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) that distinguishes the division is additionally encoded by the encoder 150 to decode the image. Signaled to the device.

The quantization unit 145 quantizes the transform coefficients output from the transform unit 140 and outputs the quantized transform coefficients to the encoding unit 150.

The encoder 150 generates a bitstream by encoding the quantized transform coefficients using a coding method such as a context-based adaptive binary arithmetic code (CABAC). The encoder 150 encodes information such as CTU size, CU split flag, QT split flag, MTT split direction, and MTT split type related to block splitting, so that the video decoding apparatus can split the block in the same way as the video encoding device. To be.

In addition, the encoder 150 encodes information on a prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and intra prediction information (ie, intra prediction mode) according to the prediction type. Information) or inter prediction information (information on a reference picture and a motion vector) is encoded.

The inverse quantization unit 160 inverse quantizes the quantized transform coefficients output from the quantization unit 145 to generate transform coefficients. The inverse transform unit 165 converts transform coefficients output from the inverse quantization unit 160 from the frequency domain to the spatial domain to restore the residual block.

The addition unit 170 restores the current block by adding the restored residual block and the prediction block generated by the prediction unit 120. The pixels in the reconstructed current block are used as reference pixels when intra-predicting the next block.

The filter unit 180 filters reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation/quantization. Perform. The filter unit 180 may include a deblocking filter 182 and a sample adaptive offset (SAO) filter 184.

The deblocking filter 180 filters the boundary between reconstructed blocks to remove blocking artifacts caused by block-based encoding/decoding, and the SAO filter 184 adds additional information to the deblocking-filtered image. Filtering is performed. The SAO filter 184 is a filter used to compensate for a difference between a reconstructed pixel and an original pixel caused by lossy coding.

The reconstructed block filtered through the deblocking filter 182 and the SAO filter 184 is stored in the memory 190. When all blocks in one picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of a block in a picture to be encoded later.

4 is an exemplary block diagram of an image decoding apparatus capable of implementing the techniques of the present disclosure. Hereinafter, an image decoding apparatus and sub-components of the apparatus will be described with reference to FIG. 4.

The image decoding apparatus may include a decoding unit 410, an inverse quantization unit 420, an inverse transform unit 430, a prediction unit 440, an adder 450, a filter unit 460, and a memory 470. have.

Like the image encoding apparatus of FIG. 1, each component of the image decoding apparatus may be implemented as hardware or software, or may be implemented as a combination of hardware and software. In addition, functions of each component may be implemented as software, and a microprocessor may be implemented to execute a function of software corresponding to each component.

The decoder 410 determines the current block to be decoded by decoding the bitstream received from the video encoding apparatus and extracting information related to block division, and information on prediction information and residual signals necessary to restore the current block, etc. Extract.

The decoding unit 410 determines the size of the CTU by extracting information on the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), and divides the picture into CTUs of the determined size. Then, the CTU is determined as the uppermost layer of the tree structure, that is, the root node, and the CTU is divided using the tree structure by extracting partition information for the CTU.

For example, in the case of splitting the CTU using the QTBTTT structure, first, a first flag (QT_split_flag) related to the splitting of the QT is extracted and each node is split into four nodes of a lower layer. And, for the node corresponding to the leaf node of QT, the second flag (MTT_split_flag) related to the splitting of the MTT and the splitting direction (vertical / horizontal) and/or split type (binary / ternary) information are extracted, and the corresponding leaf node is MTT. Divide into structure. Through this, each node below the leaf node of the QT is recursively divided into a BT or TT structure.

As another example, when a CTU is split using a QTBTTT structure, a CU split flag indicating whether to split a CU (split_cu_flag) is first extracted, and when the corresponding block is split, a QT split flag (split_qt_flag) is extracted. . When the split type is not QT but MTT, a flag (mtt_split_cu_vertical_flag) indicating the MTT split direction (vertical or horizontal) and/or a flag (mtt_split_cu_binary_flag) indicating the MTT split type (Binary or Ternary) is additionally extracted. In the segmentation process, each node may have 0 or more repetitive MTT segmentation after 0 or more repetitive QT segmentation. For example, in the CTU, MTT division may occur immediately, or, conversely, only multiple QT divisions may occur.

As another example, when the CTU is divided using the QTBT structure, each node is divided into four nodes of a lower layer by extracting the first flag (QT_split_flag) related to the division of the QT. In addition, a split flag indicating whether or not the node corresponding to the leaf node of the QT is further split into BT and split direction information are extracted.

Meanwhile, when determining the current block to be decoded through the division of the tree structure, the decoder 410 extracts information on a prediction type indicating whether the current block is intra-prediction or inter-prediction. When the prediction type information indicates intra prediction, the decoder 410 extracts a syntax element for intra prediction information (intra prediction mode) of the current block. When the prediction type information indicates inter prediction, the decoder 410 extracts a syntax element for the inter prediction information, that is, information indicating a motion vector and a reference picture referenced by the motion vector.

Meanwhile, the decoding unit 410 extracts information on quantized transform coefficients of the current block as information on the residual signal.

The inverse quantization unit 420 inverse quantizes the quantized transform coefficients, and the inverse transform unit 430 inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore residual signals to generate a residual block for the current block. .

In addition, when the inverse transform unit 430 performs inverse transform of only a partial region (subblock) of the transform block, a flag indicating that only the subblock of the transform block has been transformed (cu_sbt_flag), and the direction (vertical/horizontal) information of the subblock (cu_sbt_horizontal_flag) ) And/or sub-block location information (cu_sbt_pos_flag), and inversely transforming the transform coefficients of the sub-block from the frequency domain to the spatial domain to restore residual signals. Fill in to create the final residual block for the current block.

The prediction unit 440 may include an intra prediction unit 442 and an inter prediction unit 444. The intra prediction unit 442 is activated when the prediction type of the current block is intra prediction, and the inter prediction unit 444 is activated when the prediction type of the current block is inter prediction.

The intra prediction unit 442 determines an intra prediction mode of the current block among a plurality of intra prediction modes from a syntax element for the intra prediction mode extracted from the decoder 410, and a reference pixel around the current block according to the intra prediction mode. The current block is predicted using

The inter prediction unit 444 determines a motion vector of the current block and a reference picture referenced by the motion vector using the syntax element for the intra prediction mode extracted from the decoding unit 410, and uses the motion vector and the reference picture. To predict the current block.

The adder 450 restores the current block by adding the residual block output from the inverse transform unit and the prediction block output from the inter prediction unit or the intra prediction unit. The pixels in the reconstructed current block are used as reference pixels for intra prediction of a block to be decoded later.

The filter unit 460 may include a deblocking filter 462 and an SAO filter 464. The deblocking filter 462 performs deblocking filtering on the boundary between reconstructed blocks in order to remove blocking artifacts caused by decoding in units of blocks. The SAO filter 464 performs additional filtering on the reconstructed block after deblocking filtering in order to compensate for the difference between the reconstructed pixel and the original pixel caused by lossy coding. The reconstructed block filtered through the deblocking filter 462 and the SAO filter 464 is stored in the memory 470. When all blocks in one picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of a block in a picture to be encoded later.

The inter prediction method can be largely divided into a skip mode (skip mode), a merge mode (merge mode), and an adaptive (or advanced) motion vector predictor (AMVP) mode.

In the skip mode, motion information of any one of motion information candidates of neighboring blocks located around the current block is signaled from the video encoding apparatus to the video decoding apparatus. In the merge mode, motion information of any one of motion information candidates of a neighboring block and information obtained by encoding a residual after prediction are signaled. In the AMVP mode, motion information of a current block and information obtained by encoding a residual after prediction are signaled.

The motion information signaled in the skip mode and the merge mode is expressed as an index (merge index) value indicating any one of the motion information candidates (any one of the motion information candidates included in the candidate list). The motion information signaled in the AMVP mode is expressed as a difference value (mvd, motion vector difference) between motion information of a neighboring block and motion information of a current block.

Each of the flags cu_skip_flag and merge_flag that distinguishes the skip mode and the merge mode is signaled from the image encoding apparatus to the image decoding apparatus, and a merge index (merge_idx) is signaled to the image decoding apparatus according to the values of these flags. A method of constructing a candidate list or a method of signaling a merge index may be performed in the same manner in the skip mode and the merge mode.

In the case of the skip mode, the size of the current block is 2Nx2N, whereas in the case of the merge mode, the size of the current block may be 2NxN, Nx2N, or asymmetric partition as well as 2Nx2N. When the current block has a size of 2Nx2N and all zero transform coefficients, the current block is classified as a skip mode.

Therefore, in order for a current block having a size of 2Nx2N to be predicted in the merge mode, the current block must have at least one non zero transform coefficient. Whether the current block has one or more non zero transform coefficients may be indicated through rqt_root_cbf. In the case of skip mode, rqt_root_cbf is not signaled and its value is set to “0”. In merge mode and the current block size is 2Nx2N, rqt_root_cbf is not signaled and its value is set to “1”. In the merge mode and the current block size is not 2Nx2N, rqt_root_cbf is explicitly signaled.

In the skip mode, merge mode, and AMVP mode, information signaled from the video encoding device to the video decoding device is shown in Tables 1 to 3.

[Table 1]

[Table 2]

[Table 3]

The image decoding apparatus may classify a mode for inter prediction of a current block by using the information shown in Tables 1 to 3, and predict a current block based on the divided mode.

A conventional method of classifying prediction modes is shown in FIG. 13.

Information (cu_skip_flag) indicating whether the inter prediction mode of the current block corresponds to the skip mode is decoded from the bitstream (S510). When cu_skip_flag indicates the skip mode (S520), the merge index (merge_idx) is decoded (S522), and rqt_root_flag is not signaled and its value is set to “0” (S524).

When cu_skip_flag does not indicate the skip mode (S520), information indicating whether the current block is intra predicted (pred_mode_flag) is decoded (S530). When pred_mode_flag indicates inter prediction (S540), rqt_root_cbf is not signaled and its value is set to “1” (S570). On the contrary, when pred_mode_flag indicates that intra prediction is not performed (indicates that inter prediction is performed) (S540), information about a method in which the current block CU is divided into prediction blocks PU, and the current block Information (merge_flag) indicating whether the inter prediction mode of is corresponding to the merge mode is decoded (S542 and S544).

When the merge_flag does not indicate the merge mode (in the case of indicating the AMVP mode) (S546), the rqt_root_cbf is decoded (S560). In contrast, when merge_flag indicates a merge mode (S546), the merge index is decoded (S548). In the merge mode and when the size of the corresponding block is 2Nx2N (S550), rqt_root_cbf is not signaled and its value is set to “1” (S552). In the merge mode and if the size of the corresponding block is not 2Nx2N (S550), since rqt_root_cbf is explicitly signaled, it is decoded from the bitstream (S554).

On the other hand, a method of constructing a candidate list for skip mode and merge mode includes searching for spatial candidates and adding them to the candidate list, searching for temporal candidates and adding them to the candidate list, candidate list A step of adding the candidates added to the candidate list to a candidate list by combining (combined bi-directional candidates) and adding a zero motion vector candidate to the candidate list may be performed. Here, “adding candidates to the candidate list” may mean “setting the blocks themselves or motion information of the corresponding blocks as candidates”.

Locations of spatial candidates added to the candidate list are shown in FIG. 6 (a). The apparatus for encoding/decoding an image may search for candidate blocks in the order of A1→B1→B0→A0→B2 and include up to four candidates in the candidate list.

Locations of temporal candidates added to the candidate list are shown in FIG. 6(b). The apparatus for encoding/decoding an image may search for candidate blocks in the order of BR→CT and include at most one candidate in the candidate list. In the process of setting a candidate list, if the same motion information exists, the corresponding candidate is not added to the list. That is, overlapping motion information is not allowed.

Information on the number of candidates that may be included in the candidate list (five_minus_max_num_merge_cand) is recorded in the slice header, and up to five candidates may be designated or set through this information. A TU (truncated unary) method is used as a binarization method for indexing candidates, and Table 4 shows a codeword representing each candidate index when the maximum number of candidates is five.

[Table 4]

The present invention proposes a method of deriving unidirectional prediction information (motion information) from a candidate list used for inter prediction of a current block.

Since the candidate list used for inter prediction of the current block is composed of spatial or temporal neighboring blocks, this candidate list may include both unidirectional motion information and bidirectional motion information. In this case, unidirectional motion information may be derived from bidirectional motion information included in the candidate list through the methods proposed in the present invention.

The derivation method proposed in the present invention can be applied in all cases where unidirectional motion information is required. In addition, the derivation method proposed in the present invention can be applied to blocks in which only one-way prediction is allowed. A block in which only unidirectional prediction is allowed may be a triangular block or a block having a specific size (eg, 4x8, 8x4, etc.).

In the present specification, the present invention will be described centering on an example in which a method of inducing unidirectional motion information is applied to a triangular block.

Candidate list

The candidate list may mean a list used for inter prediction of a current block including triangular blocks. An image encoding/decoding apparatus may construct a candidate list using the same method.

The candidate list may include 1) a spatial candidate, 2) a temporal candidate, 3) a history-based candidate, 4) a pair-wise average candidate, 5) a zero motion vector candidate, and the like.

Positions of candidate blocks for a spatial candidate and a temporal candidate are shown in FIG. 7. Block 1 is located on the left side of the current block, Block 2 is located above the current block, Block 3 is located on the top right of the current block, and Block 4 is located on the bottom left of the current block. And block 5 is located on the upper left of the current block. Also, block 6 is located at the lower right of the collocated block, and block 7 is located at the center of the collocated block. Here, the collocated block refers to a block located in the collocated picture, but located at the same position as the current picture of the current block.

The apparatus for encoding/decoding an image may search for candidate blocks in the order of block 2 → block 1 → block 3 → block 4 → block 5 and include up to 5 spatial candidates in the candidate list. In addition, the apparatus for encoding/decoding an image may search for candidate blocks in the order of block 6 → block 7 and include at most one temporal candidate in the candidate list.

The image encoding/decoding apparatus may further include one or more of motion information stored in the history-based table in the candidate list. The history-based table refers to a table in which motion information of blocks that have already been reconstructed before restoration of the current block is performed is stored in a first input first output (FIFO) structure.

The image encoding/decoding apparatus may select motion information stored in the history-based table in the reverse order of the stored order and include it in the candidate list. The number of history-based candidates included in the candidate list may be one less than the maximum number of candidates that may be included in the candidate list. For example, if the maximum number of candidates that can be included in the candidate list is "6" and the number of candidates currently included in the candidate list is "3", "2" history-based candidates may be included in the candidate list. .

The image encoding/decoding apparatus may average candidates at a preset position among candidates included in the candidate list and further include them in the candidate list (pair-averaged candidates). When the maximum number of candidates that can be included in the candidate list is not satisfied, a zero motion vector may be further included in the candidate list.

Triangle block

As shown in FIG. 9, a rectangular current block is divided into two triangular blocks PU ₁ and PU ₂ , and these two triangular blocks are defined as a prediction unit. Each triangular block may be predicted by applying a skip mode or a merge mode to each of the divided triangular blocks.

It is assumed that prediction is performed using a single candidate list for the current block, only unidirectional prediction is allowed for the triangle block, and a separate index is signaled for each of the triangle blocks. That is, it is assumed that the triangular blocks share the candidate list, but do not share the index. Hereinafter, a prediction mode to which the above assumptions are applied will be referred to as a'triangle prediction mode'.

Syntaxes for the triangular prediction mode are shown in Table 5.

[Table 5]

The merge_triangle_flag is information indicating whether the triangle prediction mode is applied (om/off), and may be explicitly signaled from the image encoding device to the image decoding device or derived to a preset value. The image decoding apparatus may determine whether to apply the triangle prediction mode according to the value of merge_triangle_flag.

The merge_triangle_flag may be signaled when enable information (triangle_enabled_flag) indicating whether or not the triangle prediction mode is active/inactive indicates activity, or may be derived to a preset value. The triangle_enabled_flag may be defined in a high-level header such as sequence-level, picture-level, slice-level, and/or tile group-level.

merge_triangle_flag is signaled or “1” when the slice type or tile group type is “B type (bi-prediction)”, the maximum number of candidates for triangle prediction is 2 or more, and the inter prediction mode of the current block is the merge mode. Can be derived by value. As another example, merge_triangle_flag may be signaled or derived to a value of “1” when the size (Width*Height) of the current block is greater than or equal to a preset threshold. Here, the preset threshold value may be “64”. As another example, merge_triangle_flag may be signaled or derived as a value of “1” when the slice type or tile group type is “P type”.

When merge_triangle_flag indicates that the triangular prediction mode is applied, information (merge_triangle_split_flag or merge_triangle_split_dir) about two division directions or shapes of the triangular block as shown in FIG. 9 may be signaled from the video encoding apparatus to the video decoding apparatus. merge_triangle_split_flag=0 may indicate a split shape in the downward-right direction as shown in FIG. 9(a), and merge_triangle_split_flag=1 may indicate a division shape in the upward-right direction as shown in FIG. 9(b). The image decoding apparatus may determine the split shape of the triangular block using merge_triangle_split_flag.

Indexes (merge_triangle_idx0, merge_triangle_idx1) for each of the triangular blocks may be signaled from the image encoding apparatus to the image decoding apparatus. merge_triangle_idx0 indicates a candidate applied to the triangle prediction mode of PU ₁ from among candidates included in the candidate list, and merge_triangle_idx1 may indicate a candidate applied to the triangle prediction mode of PU ₂ from among candidates included in the candidate list. Candidates indicated by merge_triangle_idx0 and merge_triangle_idx1 are different from each other. This is because if the candidates indicated by merge_triangle_idx0 and merge_triangle_idx1 are the same (if the same motion information is used for prediction of PU ₁ and PU ₂ ), it is not necessary to divide into triangular blocks. The video decoding apparatus (decoder) may decode the index from the bitstream (S810) and specify a candidate indicated by the decoded index (S820). Here, the candidate indicated by the index is referred to as a'target candidate' or a'target candidate block'.

The image encoding/decoding apparatus (inter prediction unit) may determine whether the reference direction of the specified target candidate block is bidirectional (S830). That is, the image encoding/decoding apparatus may determine whether the target candidate block is bidirectionally predicted. The motion information of the bi-predicted target candidate block may include L0 motion information referring to a reference picture included in the L0 list and L1 motion information referring to a reference picture included in the L1 list.

When the target candidate block is predicted in one direction (L0 direction or L1 direction) (S830), motion information (unidirectional) of the target candidate block may be derived as motion information of the triangular block (S840). In contrast, when the target candidate block is bidirectionally predicted (S830), either of the L0 motion information and the L1 motion information may be derived as the motion information of the triangular block (S850). This is because it is assumed that only one-way prediction is allowed for the triangular block.

The motion information derived from the motion information of the triangular block may be determined based on'weights applied to bidirectional prediction of the target candidate block' or may be determined based on the'distance between reference pictures and the current picture'.

As described above, in the present invention, when the target candidate block is predicted in one direction, the motion information is used as it is, and when the target candidate block is predicted in both directions, motion information in either direction is used to induce one-way motion information. I can.

Meanwhile, the image encoding apparatus may construct a separate candidate list used for inter prediction of a triangular block by using the derived motion information (unidirectional motion information). In order to distinguish the candidate list for the current block and the candidate list for the triangle block, the candidate list for the current block is referred to as a'general candidate list', and the candidate list for the triangle block is referred to as a'triangular candidate list'. .

One or more candidates may be included in the triangular candidate list. Information on the maximum number of candidates included in the triangular candidate list (information on the maximum number of candidates used in the triangular prediction mode, information on the maximum number) may be signaled from the video encoding device to the video decoding device.

The information on the maximum number can be implemented in various forms. For example, 1) The information on the maximum number may indicate the number of candidates used in the triangle prediction mode (eg, max_num_triangle_cand). As another example, 2) The information on the maximum number may be a value obtained by subtracting the maximum number from a specific constant value (eg, six_minus_max_num_triangle_cand). As another example, 3) information on the maximum number is the maximum number of candidates used in the triangular prediction mode (triangle candidate list) from the maximum number of candidates used for the merge mode prediction of the current block (the maximum number included in the general candidate list). It may be a value obtained by subtracting the maximum number included in (eg, max_num_merge_cand_minus_max_num_triangle_cand).

Assuming that the maximum number of candidates used in the triangular prediction mode is “4”, the maximum number of candidates used in the merge mode prediction of the current block is “6”, and the specific constant value is “5”, the maximum number The information may be max_num_triangle_cand=4 in the example 1), six_minus_max_num_triangle_cand=1 (5-4=1) in the example 2), and max_num_merge_cand_minus_max_num_triangle_cand=2 (6-4=2) in the example 3). .

The information on the maximum number may be defined at one or more positions among high-level headers such as sequence-level, picture-level, slice-level, and/or tile group-level, and may be signaled from the image encoding device to the image decoding device. . Also, information on the maximum number may be signaled when triangle_enabled_flag indicates activation of the triangle prediction mode. Furthermore, when the information on the maximum number is implemented in the same form as in the example of 3), the maximum number of candidates used for the merge mode prediction of the current block while the triangle_enabled_flag indicates the activity of the triangle prediction mode is In the case of 2 or more, it may be signaled.

weight

Among the two-way motion information (L0 motion information and L1 motion information), motion information to be used for prediction of a triangle block may be selected or derived based on weights applied to two-way prediction of a target candidate block. Here, the weight (w0) applied to prediction in the L0 direction (applied to the reference sample) is referred to as the first weight, and the weight (w1) applied to prediction in the L1 direction (applied to the reference sample) is referred to as the second weight. It can be referred to as Conversely, w0 may be referred to as a second weight and w1 may be referred to as a first weight.

Standard weights include 1) weights applied to weighted prediction (WP), and 2) weights applied to bi-prediction with CU-level weights (BCW). Can be included. The WP function and the BCW function can be implemented exclusively from each other. It may be defined in a sequence-level and/or a picture-level so that any one of the WP function and the BCW function is applied or implemented.

1) WP is a method of performing weighted prediction by setting weights in units of slices or type groups, and a weight value and an offset value for WP may be defined in a slice header. The defined weight value and offset value may be set for each of all reference pictures referenced in the current slice (the slice including the current block or the triangle block).

As an example of WP, WP is applied (on) to a slice, the slice type is B type, the current block included in the slice is bidirectionally predicted, and the reference picture of the current block is the 0-th reference picture and the L1 list in the L0 list. In the case of the 1-th reference picture, the prediction sample of the current block can be derived as in Equation 1 below.

[Equation 1]

In Equation 1, pdSamples is a value of a weighted predicted prediction sample. pdSamplesL0 is a prediction sample value in the L0 direction, w0 is a weight value for the 0-th reference picture in the L0 list, and o0 is an offset value for the 0-th reference picture in the L0 list. pdSamplesL1 is a prediction sample value in the L1 direction, w1 is a weight value for the 1-th reference picture in the L1 list, and o1 is an offset value for the 1-th reference picture in the L1 list.

2) BCW is a method of setting a weight in units of a block to be decoded, and a weight may be specified by a weight index (bcw_idx). In the case of a low delay picture, bcw_idx may be set to 5, and in the case of a non-low delay picture, bcw_idx may be set to 3. Table 6 shows bcw_idx and weights (applicable candidate weights) set for each bcw_idx.

[Table 6]

As shown in Table 6, bcw_idx may have a value of 0 to 4 in the case of a low delay picture, and may have a value of 0 to 2 in the case of a non-low delay picture. From Table 6, if bcw_idx is even, w0 is greater than w1, if bcw_idx is odd, w0 is smaller than w1, and if bcw_idx is 0, w0 and w1 are the same, and between w0 and w1 is mathematical It can be seen that the relationship shown in Equation 2 holds.

[Equation 2]

The current block is predicted in both directions, the BCW function is enabled in the high-level header (sps_bcw_enabled_flag = “on”), the WP function is “off”, and the size of the current block (Width * Height) is preset threshold (eg: 256) or more, bcw_idx may be signaled from the image encoding device to the image decoding device. The image decoding apparatus may derive a weight value w0 applied to prediction in the L0 direction and a weight value w1 applied to prediction in the L1 direction through the signaled bcw_idx value.

As an example of BCW, when the bcw_idx value is “2”, the weight value (w0) applied to prediction in the L0 direction is set or derived as “5/8”, and the weight value applied to prediction in the L1 direction (w1) This “3/8” can be set or derived.

Using the derived weight values, a prediction sample of the current block can be derived as shown in Equation 3 below.

[Equation 3]

The video encoding/decoding apparatus may derive motion information in a direction corresponding to any one of w0 and w1 as motion information of a triangular block. For example, motion information in a direction corresponding to a weight having a larger value among w0 and w1 may be derived as motion information of a triangular block. As another example, motion information in a direction corresponding to a weight having a smaller value among w0 and w1 may be derived as motion information of a triangular block. As another example, when w0 and w1 are the same, L0 motion information or L1 motion information may be derived as motion information of a triangular block.

Distance between reference pictures and current picture

Among the two-way motion information (L0 motion information and L1 motion information), the motion information to be used for the prediction of the triangle block is selected based on the distance difference between each of the reference pictures of the target candidate block and the current picture (picture including the triangle block). Or can be derived.

For example, among the reference pictures in the L0 direction and the reference pictures in the L1 direction, motion information corresponding to the direction of the reference picture closer to the current picture may be derived as the motion information of the triangular block. As another example, among the reference pictures in the L0 direction and the reference pictures in the L1 direction, motion information corresponding to a direction of a reference picture farther from the current picture may be derived as the motion information of the triangular block.

The distance between each of the reference pictures and the current picture may be a difference value of a picture order count (POC) between each of the reference pictures and the current picture. For example, if the target candidate block is bidirectionally predicted, POC of the L0 reference picture is 3, POC of the L1 reference picture is 6, and POC of the current picture is 5, the difference between the POC of the L0 reference picture and the current picture is “ 2” and “1”, which is a POC difference between the L1 reference picture and the current picture, one of the L0 motion information and the L1 motion information may be derived as the motion information of the triangular block.

When the motion information of the triangular block is derived through the various methods described above, the image encoding/decoding apparatus may predict the triangular block based on the derived motion information (S860).

Example 1

Embodiment 1 is a method of deriving motion information of a triangular block based on weights used in the WP function. An example of a method for deriving motion information of PU ₁ from among two triangular blocks PU ₁ and PU ₂ is shown in FIG. 10. w0 and w1 denote weight values described in Equation 1.

The image decoding apparatus may decode merge_triangle_idx0 from the bitstream (S1010), and may specify a target candidate block A indicated by merge_triangle_idx0 from among candidates included in the general candidate list (S1020).

When the reference direction of the target candidate block is bidirectional (S1030), the video decoding apparatus identifies weight values (w0, w1) applied to the weighted prediction of the target candidate block (S1040), and compares the weight values of PU ₁ Motion information can be derived. Specifically, when w0> w1 (S1050), the L0 motion information may be derived as the motion information of PU ₁ (S1060), and when w0 <w1 (S1050), the L1 motion information is the motion information of PU ₁ It can be induced by (S1080). In addition, when w0 = w1 (S1050), the L1 motion information may be derived as the motion information of the PU ₁ (S1080).

When the reference direction of the target candidate block is one direction (L0 or L1) (S1030), the video decoding apparatus may derive motion information in the same direction as the reference direction of the target candidate block as the motion information of PU ₁ . Specifically, when the reference direction of the target candidate block is one direction of L0 (S1070), the motion information of L0 may be derived as the motion information of PU ₁ (S1060), and when the reference direction of the target candidate block is one direction of L1 In S1070, the motion information of L1 may be derived as the motion information of PU ₁ (S1080).

Although only an example of deriving the motion information of PU ₁ is shown in FIG. 10, the motion information of PU ₂ may also be derived through the processes shown in FIG. 10.

Meanwhile, Embodiment 1 may be implemented by replacing w0 and w1 with offset values o0 and o1 described in Equation 1. That is, motion information in the same direction as the direction of the offset having a larger value among the offsets may be derived as the motion information of the triangular block. In addition, motion information of a triangular block may be derived using both the weight values w0 and w1 and the offset values o0 and o1.

Example 2

Embodiment 2 is a method of deriving motion information of a triangular block based on weights used for the BCW function. An example of a method of inducing motion information of PU ₁ among two triangular blocks PU ₁ and PU ₂ is shown in FIG. 11. w0 and w1 denote weight values described in Equations 2 and 3.

The video decoding apparatus may decode merge_triangle_idx0 from the bitstream (S1110), and specify a target candidate block A indicated by merge_triangle_idx0 from among candidates included in the general candidate list (S1120).

When the reference direction of the target candidate block is bidirectional (S1130), the video decoding apparatus may determine a bcw_idx value applied to the weighted prediction of the target candidate block (S1140). When bcw_idx is an even number (S1150), since w0 has a larger value than w1, the L0 motion information may be derived as the motion information of PU ₁ (S1160). In contrast, when bcw_idx is an odd number (S1150), since w0 has a smaller value than w1, the L1 motion information may be derived as the motion information of PU ₁ (S1180). When bcw_idx is 0 (S1150), since w0 = w1, the L1 motion information may be derived as the motion information of PU ₁ (S1180).

When the reference direction of the target candidate block is unidirectional (L0 or L1) (S1130), the video decoding apparatus may derive motion information in the same direction as the reference direction of the target candidate block as the motion information of PU ₁ . Specifically, when the reference direction of the target candidate block is one direction of L0 (S1170), the motion information of L0 may be derived as the motion information of PU ₁ (S1160), and when the reference direction of the target candidate block is one direction of L1 In (S1170), the motion information of L1 may be derived as the motion information of PU ₁ (S1180).

Although only an example of deriving the motion information of PU ₁ is shown in FIG. 11, the motion information of PU ₂ may also be derived through the processes shown in FIG. 11.

Example 3

Embodiment 3 is a method of deriving motion information of a triangular block based on the distance between each of the reference pictures and the current picture. Here, the distance between each of the reference pictures and the current picture may be a difference value of a picture order count (POC) between each of the reference pictures and the current picture. An example of a method of deriving motion information of PU ₁ among two triangular blocks PU ₁ and PU ₂ is shown in FIG. 12.

The image decoding apparatus may decode merge_triangle_idx0 from the bitstream (S1210), and may specify a target candidate block A indicated by merge_triangle_idx0 from among candidates included in the general candidate list (S1220).

When the reference direction of the target candidate block is bidirectional (S1230), the video decoding apparatus includes a difference (diff.POC0) between the POC value (POCL0) of the L0 reference picture and the POC value (POCcurr) of the current picture (diff.POC0), and the L1 reference picture. The difference (diff.POC1) between the POC value of (POCL1) and the current picture's POC value (POCcurr) is calculated (S1240), and diff.POC0 and diff.POC1 are compared (S1250), and the motion information of PU ₁ is obtained. You can induce.

Specifically, when diff.POC0 <diff.POC1 (S1250), the L0 motion information may be derived as the motion information of the PU ₁ (S1260). If diff.POC0> diff.POC1 (S1250), the L1 motion information may be derived as the motion information of the PU ₁ (S1280). If diff.POC0 = diff.POC1 (S1250), the L1 motion information may be derived as the motion information of the PU ₁ (S1280).

When the reference direction of the target candidate block is one direction (L0 or L1) (S1230), the video decoding apparatus may derive motion information in the same direction as the reference direction of the target candidate block as the motion information of PU ₁ . Specifically, when the reference direction of the target candidate block is one direction of L0 (S1270), the motion information of L0 may be derived as the motion information of PU ₁ (S1260), and when the reference direction of the target candidate block is one direction of L1 In S1270, the motion information of L1 may be derived as the motion information of PU ₁ (S1280).

Although only an example of deriving the motion information of PU ₁ is shown in FIG. 12, the motion information of PU ₂ may also be derived through the processes shown in FIG. 12.

The above description is merely illustrative of the technical idea of the present embodiment, and those of ordinary skill in the technical field to which the present embodiment belongs will be able to make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present exemplary embodiments are not intended to limit the technical idea of the present exemplary embodiment, but are illustrative, and the scope of the technical idea of the present exemplary embodiment is not limited by these exemplary embodiments. The scope of protection of this embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

120, 440: prediction unit 130: subtractor
170, 450: adder 180, 460: filter unit

Claims (12)

As a method of inducing the motion information of a triangular block,
Decoding an index indicating any one of candidate blocks included in a candidate list from the bitstream;
Determining whether a reference direction of a target candidate block, which is a candidate block indicated by the index, is bi-prediction; And
And inducing any one of the two-way motion information of the target candidate block into motion information of the triangular block when the reference direction of the target candidate block is bidirectional. The method of claim 1,
The inducing step,
A method of inducing motion information, in which one of motion information of the target candidate block is derived as motion information of the triangular block based on weights applied to weighted prediction of the target candidate block. The method of claim 2,
The weights are,
A first weight applied to prediction in the first direction and a second weight applied to prediction in the second direction,
The inducing step,
When the first weight is greater than the second weight, the motion information in the first direction is derived from motion information of the target candidate block as motion information of the triangular block. The method of claim 2,
The weights are,
The motion information derivation method, which is set in a slice unit including the target candidate block. The method of claim 2,
The weights are,
Among applicable candidate weights, motion information derivation method indicated by a weight index set in the target candidate block. The method of claim 1,
The inducing step,
Motion information for deriving any one of motion information of the target candidate block as motion information of the triangle block based on a distance difference between each of the reference pictures of the target candidate block and the current picture including the triangle block Induction method. A decoder for decoding an index indicating any one of candidate blocks included in a candidate list from the bitstream; And
It is determined whether the reference direction of the target candidate block, which is the candidate block indicated by the index, is bi-prediction, and when the reference direction of the target candidate block is bi-directional, any of the two-way motion information of the target candidate block A video decoding apparatus comprising a prediction unit for inducing one into motion information of a triangular block. The method of claim 7,
The prediction unit,
An image decoding apparatus for inducing any one of motion information of the target candidate block as motion information of the triangular block based on weights applied to bidirectional prediction of the target candidate block. The method of claim 8,
The weights are,
Among the two directions of the target candidate block, a first weight applied to prediction in a first direction and a second weight applied to prediction in a second direction are included,
The prediction unit,
When the first weight value is greater than the second weight value, the motion information in the first direction among motion information of the target candidate block is derived as motion information of the triangular block. The method of claim 8,
The weights are,
The image decoding apparatus, which is set in a slice unit including the target candidate block. The method of claim 8,
The weights are,
Among applicable candidate weights, the video decoding apparatus is indicated by a weight index set in the target candidate block. The method of claim 7,
The prediction unit,
Based on a picture order count (POC) difference between each of the reference pictures of the target candidate block and the current picture including the triangle block, one of the motion information of the target candidate block is used as the motion information of the triangle block. Inducing, video decoding device.

Images