KR20210026015A - Fast block split method in video and encoding apparatus - Google Patents

Abstract

A fast block split method in an image includes steps of: performing, by an encoder, an evaluation of a binary tree (BT) splitting a specific CU of a frame; and performing, by the encoder, evaluation on a ternary tree (TT) splitting the specific CU. The encoder omits the evaluation of a vertical direction in a TT division process when a rate-distortion (RD) cost in a horizontal direction in a BT split is smaller than the RD cost in the vertical direction, and the encoder omits the evaluation of the horizontal direction in the TT segmentation process when the rate-distortion (RD) cost in the horizontal direction is greater than the RD cost in the vertical direction. The present invention can perform high-speed encoding by using the rate-distortion cost for blocks having overlapping regions.

Description

High-speed segmentation method and encoder device for blocks in video {FAST BLOCK SPLIT METHOD IN VIDEO AND ENCODING APPARATUS}

The technique described below relates to an image coding technique. In particular, the technique described below relates to a splitting technique for a block in an image in the encoding process.

Joint Video Exports Team (JVET), a team of ITU-T SC16 WP3 and ISO/IEC JTC 1/SC 29/WG 11, configured for the next-generation video coding standard, is HEVC (High Efficiency Video Coding)/H. Standardization work is in progress for VVC (Versatile Video coding), which has more than twice the coding performance of 265. VVC divides a frame into a plurality of blocks and performs coding in units of divided blocks. VVC divides a block using a new tree structure called MTT (Multi-Type Tree).

K. Choi, S. -H. Park, and E. S. Jang, "Coding tree pruning based CU early termination," Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T and ISO/IEC, Document JCTVC-F092, Jul. 2019.

In order to utilize the diversity of MTT, the encoder must check the prediction technique and transform/quantization technique for each block. Therefore, VVC should test the encoding process for a much larger number of cases compared to HEVC in the encoding process.

The technique described below is intended to provide a technique for omitting a test process for a specific block by using the characteristic of the block structure. The technique described below is intended to provide an encoding technique that shares a rate-distortion cost between blocks with overlapping pixels.

In the high-speed segmentation method for a block in an image, an encoder performs an evaluation of a binary tree (BT) segmentation for a specific CU (coding unit) of a frame, and the encoder performs a three-dimensional tree (TT) segmentation for the specific CU. And performing an evaluation on the. In the case where the rate-distortion (RD) cost in the horizontal direction in the BT division is smaller than the RD cost in the vertical direction, the encoder omits the evaluation in the vertical direction in the TT division process. In addition, when the rate-distortion (RD) cost in the horizontal direction is greater than the RD cost in the vertical direction, the encoder skips the evaluation of the horizontal direction in the TT division process.

An encoder device that performs high-speed segmentation on a block in an image is a storage device that stores information on a target block in a frame to be segmented, a result of a binary tree (BT) segmentation evaluation result for the target block, and a code for segmenting the target block. And a processor that executes the code to evaluate the three-dimensional tree (TT) partitioning of the target block. The BT segmentation evaluation result includes a horizontal rate-distortion (RD) cost of the BT segmentation for the target block and a vertical RD cost of the BT segmentation for the target block. The processor omits the evaluation in the horizontal direction or the vertical direction in the TT division evaluation process based on the RD cost in the horizontal direction and the RD cost in the vertical direction.

The technique described below can perform encoding at high speed using a rate-distortion cost for a block with an overlapping region among blocks divided according to MTT in VVC. The technology to be described below allows a device having limited performance, such as a mobile device, to record a VVC image having a higher complexity than HEVC in real time.

1 is an example of a structure of a multi-type tree (MTT) of VVC.
2 is an example of a prior probability for a case where the RD cost of TT calculated during the encoding process is not optimal.
3 is an example of a posterior probability that the RD cost of BT becomes valid data for TT in the same direction.
4 is an example of a process of determining a partition structure of a block in VVC.
5 is an example of a configuration of an encoder device that determines a block division structure in VVC.

The technology to be described below may be modified in various ways and may have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the technology to be described below with respect to a specific embodiment, and it should be understood to include all changes, equivalents, or substitutes included in the spirit and scope of the technology to be described below.

Terms such as 1st, 2nd, A, B, etc. may be used to describe various components, but the components are not limited by the above terms, and only for the purpose of distinguishing one component from other components. Is only used. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component without departing from the scope of the rights of the technology described below. The term and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

In terms of the terms used in the present specification, expressions in the singular should be understood as including plural expressions unless clearly interpreted differently in context, and terms such as "includes" are specified features, numbers, steps, actions, and components. It is to be understood that the presence or addition of one or more other features or numbers, step-acting components, parts or combinations thereof is not meant to imply the presence of, parts, or combinations thereof.

Prior to the detailed description of the drawings, it is intended to clarify that the division of the constituent parts in the present specification is merely divided by the main function that each constituent part is responsible for. That is, two or more constituent parts to be described below may be combined into one constituent part, or one constituent part may be divided into two or more for each more subdivided function. In addition, each of the constituent units to be described below may additionally perform some or all of the functions of other constituent units in addition to its own main function, and some of the main functions of each constituent unit are different. It goes without saying that it can also be performed exclusively by.

In addition, in performing the method or operation method, each of the processes constituting the method may occur differently from the specified order unless a specific order is clearly stated in the context. That is, each of the processes may occur in the same order as the specified order, may be performed substantially simultaneously, or may be performed in the reverse order.

The technique described below is a process of performing encoding on a VVC video. VVC video refers to a video coded according to the VVC protocol (standard). Hereinafter, it will be described that the encoder performs encoding. An encoder refers to a device that encodes an image. A computing device such as a PC, mobile device, smart device, server, etc. may be an encoder.

1 is an example of a structure of a multi-type tree (MTT) of VVC.

MTT is a new tree structure applied to VVC. In addition to the conventional QT (Quad Tree, photo tree), MTT uses BT (Binary Tree, binary tree) and TT (Ternary Tree, ternary tree). MTT can divide one block using QT, BT or TT. MTT allows one CU (Coding Unit) to be divided into various sub-trees. In particular, BT and TT may exist only under QT. BT can place BT or TT as a sub tree. In addition, TT may have TT or BT as a subtree. BT and TT can be divided into a horizontal direction (horizontal) and a vertical direction (vertical) as shown in FIG. According to the maximum depth of the tree and the minimum block size, one block can be divided into more various types. That is, the VCC can perform block division in various forms using various tree structures. VCC provides improved coding efficiency by using MTT, which provides a variety of tree structures.

The encoder calculates the RD (rate distortion) cost for all possible partition structures in the process of partitioning a CU based on MTT. Through this, the encoder determines the optimal tree structure for the CU. The encoder basically divides a specific CU to have a structure with the smallest RD cost. On the other hand, if the encoder determines that the optimal RD cost is not generated by dividing the CU, it does not divide the CU any more. Of course, as described above, the encoder divides the CU in consideration of the conditions of the maximum tree depth and the minimum block size.

Among the trees of MTT, TT has a relatively high encoding complexity. The technique described below reduces the number of RD cost evaluations for TT. The technique described below is based on Bayesian probability to reduce the number of RD cost calculations of TT.

Let's look at the RD cost calculation process performed by the VCC test model (VTM) for TT. 2 is an example of a prior probability for a case where the RD cost of TT calculated during the encoding process is not optimal. 2 is an example of a case where the RD cost τ calculated for TT division by VTM is not optimal. p(τ) represents the probability when the RD cost for TT is not optimal. That is, p(τ) can be said to be a probability that evaluation of TT is not required. In Bayes theorem, the prior probability p(τ) refers to the probability that the hypothesis τ is true until actual evidence (data) is observed.

2 is a test result of a sample image (sample sequence). The test used only the first frame of the sample sequence. The test was performed in two cases in which the quantization parameter (QP) was set to QP = 22 and QP = 37.

Referring to FIG. 2, the average prior probability p(τ _hor ) that TT will be divided horizontally (horizontally) is 87.9% when QP = 22, and 89.0% when QP = 37. The subscript'hor' means horizontal. In addition, the average prior probability p(τ _ver ) that TT will be divided vertically (vertically) is 89.6% when QP = 22, and 89.1% when QP = 37. The subscript'ver' means vertical.

According to Bayesian theory, we try to find the appropriate evidence ε and examine the posterior probability p(τ|ε). In TT and BT, structurally, most of the pixels overlap. Therefore, the RD cost of BT can be a good evidence ε for τ. BT shares a large portion of the pixels present in the TT. For example, the horizontal direction TT shares only 25% of the pixels of the parent CU, but the horizontal direction BT shares 50% of the pixels. Thus, the RD cost for BT can be evidence for the hypothesis.

Evidence ε indicates a situation where the RD cost of one direction BT is better (smaller) than the other direction. When described on the basis of a specific block, when the specific block is divided into BT because the RD cost of a specific direction of BT is less than the RD cost of another direction, the specific block represents a situation in which BT division of a specific direction is performed. Equation 1 below defines the evidence ε for a specific direction δ.

δ means a horizontal or vertical direction in BT division. The value of ε(δ) is determined according to _{the RD cost J BT of BT.} ~δ means a different direction from δ. According to Equation 1, ε(δ) is 1 when J _BT (δ) <J _BT (˜δ), and 0 in other cases. p(ε) represents the probability that ε is 1.

3 is an example of a posterior probability that the RD cost of BT becomes valid data for TT in the same direction. The posterior probability p(τ|ε) is the probability that the hypothesis τ is true after evidence ε is observed. Referring to FIG. 3, the posterior probability has an average of 95% and 94% for each of the horizontal and vertical directions. Therefore, it can be seen that the posterior probability is higher than the prior probability. According to the results of FIGS. 2 and 3, it can be said that the encoder does not perform a separate RD cost calculation for TT in the TT evaluation process, and it can be said that it is sufficiently effective to use the RD cost for BT in the same direction.

4 is an example of a process 100 of determining a block division structure in VVC. FIG. 4 corresponds to a technique used for evaluating TT based on the pixel redundancy of BT and TT. That is, FIG. 4 is a technique for lowering encoding complexity by ending TT evaluation early.

4 is an example of a partitioning process for a specific block. Referring to the VCC as a reference, FIG. 4 is an example of a partitioning process for a specific CI. The block to be divided or to be evaluated is referred to as a target block. In addition, the CU to be divided or evaluated is referred to as a target CU. Hereinafter, it will be described based on the CU.

In general, the encoder first performs intra prediction for the entire target CU (110). That is, the encoder evaluates a situation in which a child node for the target CU does not exist. This prediction mode may be referred to as a current block mode. The encoder can calculate the RD cost J _{cur for the current block.}

The encoder may perform QT division evaluation on the target CU (120). The encoder can calculate the _{RD cost J QT} in case of QT division for the target CU. The order of QT division evaluation may be different from that of FIG. 4. For example, the encoder may perform QT division evaluation after BT division evaluation.

The encoder performs BT segmentation evaluation on the target CU (130). The encoder can calculate the RD cost in case of BT division for the target CU. BT segmentation has two directions (horizontal direction and vertical direction). Encoder costs RD for horizontal BT segmentation J _{BT _ HOR} And RD cost for vertical direction BT segmentation J _{BT _ VER} Can each be calculated. The encoder completes the BT evaluation before the TT evaluation. The encoder may store BT segmentation evaluation information. The BT split evaluation information may include J _{BT_HOR} and J _{BT_VER} .

The encoder performs TT division evaluation on the target CU (140). In this process, the encoder may determine whether to perform TT evaluation based on the above-described evidence ε(δ).

The encoder may first determine whether TT partition evaluation is required for the target CU based on BT partition evaluation information for the target CU. Encoder is J _{BT _ HOR} And compares the _BT J _{VER _} (141).

J _{BT _ HOR} If <J _{BT _ VER} (YES in 141), the encoder may determine that the TT for the target CU also has a similar tendency. In this case, the encoder may omit the evaluation (test) process for vertical division and only evaluate horizontal TT division (142). The encoder calculates _{the RD cost J TT _ HOR} of the horizontal direction TT division.

J _{BT _ HOR} <J _{BT _ VER} (NO of 141) is usually J _{BT _ HOR} > J _{BT _ VER} . Also in this case, the encoder may determine that the TT for the target CU also has a similar tendency. In this case, the encoder may omit the evaluation (test) process for horizontal division and perform only evaluation on vertical TT division (143). The encoder calculates _{the RD cost J TT_VER} of TT division in the horizontal direction.

Finally, the encoder determines partition information for the target CU based on the calculated RD cost information. For example, the encoder may divide the target CU into a structure having an optimal (smallest) RD cost by comparing the RD cost calculated based on various division modes. The partitioning information indicates an optimal partitioning structure for the target CU.

The encoder may determine whether or not to re-segment a child CU of the divided CU and a partition structure.

5 is an example of a configuration of an encoder device 200 for determining a block division structure in VVC. For convenience of explanation, FIG. 5 shows only a configuration for partitioning a block (or CU) among the encoder devices. That is, the encoder device 200 may be an internal component of an encoder that performs full encoding.

The encoder device 200 divides a block of a frame in an encoding process. The encoder device 200 may be implemented as an analog circuit or a digital circuit. 5 shows an example of an apparatus in which a code for block division is embedded. 5 shows an encoder device 200 in the form of a chip in which a code or program for block division is embedded. The encoder device 200 may perform block division on an image conforming to the VVC standard.

The encoder device 200 includes a storage device 210, a processor 220, and an interface device 230.

The storage device 210 stores information required for block division. The storage device 210 may temporarily store information required for block division. In this case, the storage device 210 may be a device such as a flash memory. Furthermore, the storage device 210 may store codes or programs for block division. In this case, the storage device 210 may be a device such as a ROM. Accordingly, although one block is illustrated as the storage device 210 in FIG. 5, the storage device 210 may be divided into a memory storing temporary information and a storage medium storing a block division code.

The storage device 210 may store various pieces of information necessary for block division. The storage device 210 may receive and store information such as frame information and block information. 5 shows that information on a target block is received.

Furthermore, the storage device 210 may store an intermediate result calculated during the block division process. For example, the storage device 210 may store the RD cost J _cur , J _BT , J _TT , and the like. J _BT includes the RD cost J _{BT_HOR} and J _{BT_VER} for vertical division.

The interface device 230 refers to a component that transmits data and commands inside the encoder. The interface device 230 may include a physical device and a communication protocol for internal communication. The interface device 230 may be configured with a line for transmitting information, a configuration for controlling information transmission (such as a switch), a configuration for processing information (such as an adder), and the like.

The processor 220 refers to a component that processes given data or information. The processor 220 is a kind of computing device. The processor 220 may perform block division on an image using a code or program stored in the storage device 210.

The processor 220 performs block division for a specific block or CU. The processor 220 may perform intra prediction for the entire target CU. The processor 220 may calculate _{the RD cost J cur for the current block.}

The processor 220 may perform QT division evaluation for a specific block or CU. The processor 220 may determine whether QT division is possible. In the case of QT division, the processor 220 may calculate the _{RD cost J QT.}

The processor 220 performs BT partitioning evaluation for a specific block or CU. Processor 220 is the RD cost for the horizontal direction BT partition J _{BT _ HOR} And RD cost for vertical direction BT segmentation J _{BT _ VER} Can each be calculated. The storage device 210 is J _{BT _ HOR} And J _{BT _ VER} BT division evaluation information including a can be stored. 5 illustrates an example of storing BT division evaluation information calculated by the processor 210.

The processor 220 performs TT partition evaluation for a specific block or CU. The processor 220 may determine whether TT division evaluation for a specific block or CU is required based on the BT division evaluation information.

Processor 220 is J _{BT _ HOR} And J _{BT _ VER} can be compared. (1) J _{BT _ HOR} If <J _{BT _ VER} (YES in 141), the processor 220 may determine that TT for a specific block or CU also has a similar tendency. In this case, the processor 220 may omit the evaluation (test) process for the vertical division and only evaluate the horizontal TT division. The processor 220 calculates _{the RD cost J TT _ HOR} of TT division in the horizontal direction. (2) J _{BT _ HOR} <If you are not J _{BT _ VER} typically J _{BT _ HOR} > J _{BT _ VER} . In this case, the processor 220 may determine that the TT for a specific block or CU also has a similar tendency. The processor 220 may omit the evaluation (test) process for horizontal division and perform only evaluation on vertical TT division. The processor 220 calculates _{the RD cost J TT _ VER} of TT division in the horizontal direction.

The processor 220 determines partition information for a specific block or CU based on the calculated RD cost information. The partitioning information indicates an optimal partitioning structure for a specific block or CU. The storage device 210 may store partition information finally determined for a specific block or CU.

In addition, the block division method or the encoding method as described above may be implemented as a program (or application) including an executable algorithm that can be executed on a computer. The program may be provided by being stored in a non-transitory computer readable medium.

The non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short moment, such as a register, a cache, and a memory. Specifically, the above-described various applications or programs may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, or the like.

The present embodiment and the accompanying drawings are merely illustrative of some of the technical ideas included in the above-described technology, and those skilled in the art can easily be used within the scope of the technical idea included in the specification and drawings of the above-described technology. It will be apparent that all of the modified examples and specific embodiments that can be inferred are included in the scope of the rights of the above-described technology.

Claims (11)

In a method for an encoder to perform segmentation on a CU (coding unit) of an image,
Performing, by an encoder, an evaluation of binary tree (BT) partitioning for a particular CU of the frame; And
Including the step of performing, by the encoder, evaluation of the three-dimensional tree (TT) partitioning for the specific CU,
In the case where the rate-distortion (RD) cost in the horizontal direction in the BT division is less than the RD cost in the vertical direction, the encoder omits the evaluation in the vertical direction in the TT division process,
The encoder is a high-speed segmentation method for a block in an image in which the horizontal direction evaluation is omitted in the TT segmentation process when the horizontal rate-distortion (RD) cost is greater than the vertical direction RD cost. Performing, by the encoder, intra prediction evaluation for the entire target block to be divided;
Performing, by the encoder, a photo tree (QT) segmentation evaluation on the target block;
Performing, by the encoder, a binary tree (BT) division evaluation on the target block;
Performing, by the encoder, a three-dimensional tree (TT) division evaluation on the target block; And
Determining, by the encoder, a division structure for the target block based on a result of division evaluation for the target block,
Based on the horizontal rate-distortion (RD) cost of the BT segmentation for the target block and the RD cost in the vertical direction, the encoder is a block in an image that omits horizontal or vertical evaluation during the TT segmentation evaluation process. Fast splitting method for. The method of claim 2,
The encoder divides the target block of a VVC (Versatile Video Coding) image into a high-speed segmentation method for a block within an image. The method of claim 2,
In the case where the RD cost in the horizontal direction is smaller than the RD cost in the vertical direction in the BT division, the encoder
A high-speed segmentation method for a block in an image in which evaluation in the vertical direction is omitted in the TT segmentation process and only RD cost in the horizontal direction is calculated. The method of claim 2,
The encoder, when the RD cost in the horizontal direction is greater than the RD cost in the vertical direction,
A high-speed segmentation method for a block in an image in which evaluation in the horizontal direction is omitted in the TT segmentation process and only the RD cost in the vertical direction is calculated. A storage device for storing information on a target block in a frame to be divided, a result of a binary tree (BT) division evaluation result for the target block, and a code for dividing the target block; And
Comprising a processor that executes the code and evaluates the ternary tree (TT) partitioning of the target block,
The BT segmentation evaluation result includes a horizontal rate-distortion (RD) cost of the BT segmentation for the target block and a vertical RD cost of the BT segmentation for the target block,
The processor, based on the RD cost in the horizontal direction and the RD cost in the vertical direction, performs high-speed segmentation on a block in an image for omitting evaluation in a horizontal direction or a vertical direction in the TT segmentation evaluation process. The method of claim 6,
The processor is an encoder device that performs high-speed segmentation of a block within an image that performs segmentation of the target block of a Versatile Video Coding (VVC) image. The method of claim 6,
In the case where the RD cost in the horizontal direction is smaller than the RD cost in the vertical direction in the BT division,
An encoder device for performing high-speed segmentation on a block in an image that omits vertical direction evaluation in the TT segmentation process and calculates only the RD cost for the horizontal direction. The method of claim 6,
The processor, when the RD cost in the horizontal direction is greater than the RD cost in the vertical direction,
An encoder device for performing high-speed segmentation on a block in an image that omits horizontal direction evaluation in the TT segmentation process and calculates only an RD cost for a vertical direction. The method of claim 6,
The processor performs intra prediction evaluation on the entire target block, and performs photo tree (QT) division evaluation on the target block,
The storage device is an encoder device that performs high-speed segmentation on a block in an image that stores the intra prediction result and the QT segmentation evaluation result. The method of claim 10,
The processor is an encoder that performs high-speed segmentation on a block in an image that determines a segmentation structure for the target block based on the intra prediction result, the QT segmentation evaluation result, the BT segmentation evaluation result, and the TT segmentation evaluation result. Device.

Images