RU2786746C2 - Boundary block separation in video encoding - Google Patents

Abstract

FIELD: video encoding.

SUBSTANCE: invention relates to means for video encoding. It is determined, whether the current image block is a boundary block, and whether a size of the current block exceeds the minimum permissible size of a leaf node of a quadrant tree. Forced separation of a binary tree (BT) is applied to the current block, when the current block is the boundary block, and the size of the current block does not exceed the minimum permissible size of the leaf node of the quadrant tree (MinQTSize). A stage of forced separation of the binary tree is recursive horizontal forced binary separation, when the current block is at a lower image boundary, or recursive vertical forced boundary separation, when the current block is located at a right image boundary.

EFFECT: increase in the efficiency of video encoding due to the absence of a requirement for additional signaling for forced separation of a binary tree.

15 cl, 13 dwg

Description

The technical field to which the invention belongs

Embodiments of the present invention generally relate to the field of video coding and, in particular, to splitting and splitting a coding block.

State of the art

The amount of video data required to render even a relatively short video can be significant, which can cause a significant load on the communication medium when the data must be streamed or otherwise transmitted over a communication network with limited bandwidth. Thus, video data is typically compressed over modern communication networks before transmission. The size of the video must also be taken into account when transmitting information when the video is stored on a storage device, since memory resources may be limited. Video compression devices often use source software and/or hardware to encode video data prior to transmission or storage, thereby reducing the amount of data needed to represent digital video images. The compressed data is then received at the destination side by a video decompressor, which decodes the video data. With limited network resources and ever-increasing demands for higher quality video, there is a need to use advanced compression and decompression methods that improve compression rates with little or no loss in image quality.

Disclosure of the essence of the invention

Embodiments of the present invention provide devices and methods for encoding and decoding.

The first aspect relates to a partitioning method, comprising determining whether the current image block is a boundary block and whether the size of the current block exceeds the minimum allowable size of the leaf node of the quadtree; and, if the current box is a boundary box and the size of the current box does not exceed the minimum allowable size (MinQTSize) of the quadtree leaf node, applying forced binary tree splitting (BT) to the current box.

In a first form of implementation of the method according to the first aspect, as such, the forced binary tree split is a recursive horizontal forced binary split in case the current block is located on the bottom border of the image, or a recursive vertical forced border split in case the current block is located on the right border of the image.

In the second form of implementation of the method according to the first aspect as such or any previous form of implementation of the first aspect, forced splitting of the binary tree continues until the leaf node block is within the image.

In a third implementation form of the method according to the first aspect as such or any previous implementation form of the first aspect, forced binary splitting comprises recursively splitting the current block with a horizontal forced boundary splitting until the current block subdivision is at the bottom border of the image; and recursively splitting the subdivision with a vertical forced edge split until the leaf node is completely located on the right border of the image.

In a fourth implementation form of the method according to the first aspect as such or any previous implementation form of the first aspect, the forced partition BT comprises recursively partitioning the current block with a vertical forced boundary partition until the division of the current block is at a lower boundary; and recursively splitting the subdivision with a horizontal forced boundary split until the leaf node is completely on the right boundary.

In a fifth form of implementation of the method according to the first aspect as such or any previous form of implementation of the first aspect, the method further comprises applying a minimum allowable size of the leaf node of the quadtree to control the division of the non-boundary block.

In the sixth form of implementation of the method according to the first aspect as such or any previous form of implementation of the first aspect, the border block is a block that is not completely inside the image and not completely outside the image.

The second aspect relates to a decoding method for decoding a block by dividing the block according to the first aspect as such or any previous form of implementation of the first aspect.

In a first form of implementing the method according to the second aspect, as such, the method further comprises receiving a minimum allowable leaf node size of the quadtree via a sequence parameter set (SPS).

The third aspect relates to an encoding method for encoding a block by dividing the block according to the first aspect as such or any previous form of implementation of the first aspect.

In a first implementation form of the method according to the third aspect, as such, the method further comprises transmitting a minimum allowable quadtree leaf node size via a sequence parameter set (SPS).

The fourth aspect relates to a decoding device comprising a logic circuit configured to decode the block by dividing the block in accordance with the division method of the first aspect as such or any previous form of implementation of the first aspect.

In the first implementation form of the device for decoding according to the fourth aspect, as such, the logic circuit is further configured to receive a minimum allowable size of a leaf node of the quadtree via a sequence parameter set (SPS).

The fifth aspect relates to an encoding device comprising a logic circuit configured to encode a block by dividing the block according to the division method of the first aspect as such or any previous form of implementation of the first aspect.

In the first implementation form of the decoding apparatus according to the fifth aspect as such, the logic is further configured to communicate a minimum allowable quadtree leaf node size via a sequence parameter set (SPS).

A sixth aspect relates to a non-volatile storage medium for storing instructions that, when executed by the processor, cause the processor to execute any of the first, second, or third aspects as such, or any prior form of implementation of the first, second, or third aspect.

A seventh aspect relates to a method comprising determining that the current image block is a boundary block and that the size of the current block is less than or equal to a minimum allowable size (MinQTSize) of a quadtree (QT) leaf node; and applying, in response to the determination, a forced binary tree split (BT) to the current block.

In the first implementation form of the method according to the seventh aspect, as such, the current block is located at the bottom border of the image, and in which the BT forced separation is a recursive horizontal BT forced separation.

In the second implementation form of the method according to the seventh aspect as such or any previous implementation form of the seventh aspect, the current block is located on the right border of the image, and in which the BT forced separation is a recursive vertical BT forced separation.

In a third implementation form of the method according to the seventh aspect as such or any previous implementation form of the seventh aspect, the forced division BT comprises recursively dividing the current block with a horizontal forced boundary division until the division of the current block is at a lower boundary; and recursively splitting the subdivision by vertically forced boundary splitting until the leaf node is completely located on the right boundary.

In a fourth implementation form of the method according to the seventh aspect as such or any previous implementation form of the seventh aspect, the forced partition BT comprises recursively partitioning the current block with a vertical forced boundary partition until the division of the current block is at a lower boundary; and recursively splitting the subdivision with a horizontal forced boundary split until the leaf node is completely on the right boundary.

In the fifth implementation form of the method according to the seventh aspect as such or any previous implementation form of the seventh aspect, the method further comprises applying MinQTSize to control the division of the non-boundary block.

In a sixth implementation form of the method according to the seventh aspect as such or any previous implementation form of the seventh aspect, the method further comprises receiving MinQTSize via a sequence parameter set (SPS).

In a seventh implementation form of the method according to the seventh aspect as such or any previous implementation form of the seventh aspect, the method further comprises transmitting MinQTSize via a sequence parameter set (SPS).

The eighth aspect relates to a device containing a memory; and a processor coupled to the memory and configured to determine whether the current image block is a boundary block, determine when the current block is a boundary block, whether the size of the current block exceeds a minimum allowable size (MinQTSize) of a quadtree (QT) leaf node, and apply , when the current block size is less than or equal to MinQTSize, force the binary tree (BT) split into the current block.

In the first implementation form of the apparatus according to the eighth aspect, the forced BT split as such is a recursive horizontal forced BT split when the current block is located on the lower border of the image, or a recursive forced vertical BT split when the current block is located on the right border of the image.

In the second implementation form of the device according to the eighth aspect as such or any previous implementation form of the eighth aspect, the forced division BT comprises recursively dividing the current block with a horizontal forced boundary division until the division of the current block is at the lower boundary; and recursively splitting the subdivision by vertically forced boundary splitting until the leaf node is completely located on the right boundary.

In the third implementation form of the device according to the eighth aspect as such or any previous implementation form of the eighth aspect, forced division BT comprises recursively dividing the current block with a vertical forced boundary division until the division of the current block is at the lower boundary; and recursively splitting the subdivision with a horizontal forced boundary split until the leaf node is completely on the right boundary.

In the fourth embodiment of the apparatus according to the eighth aspect as such or any preceding embodiment of the eighth aspect, the processor is further configured to apply MinQTSize to control the division of the non-boundary block.

In a fifth embodiment of the apparatus according to the eighth aspect as such or any preceding embodiment of the eighth aspect, the apparatus further comprises a receiver coupled to the processor and configured to receive a MinQTSize via a sequence parameter set (SPS).

In a sixth embodiment of the apparatus according to the eighth aspect as such or any preceding embodiment of the eighth aspect, the apparatus further comprises a transmitter coupled to the processor and configured to transmit MinQTSize via a sequence parameter set (SPS).

A ninth aspect relates to a computer program product comprising computer-executable instructions stored on a non-volatile medium that, when executed by a processor, causes a device to determine whether the current image block is a boundary block; determine, when the current box is a boundary box, whether the size of the current box exceeds the minimum allowable size (MinQTSize) of a quadtree (QT) leaf node; and apply, when current block size 0 does not exceed MinQTSize, force binary tree splitting (BT) to the current block.

In the first implementation form of the apparatus according to the eighth aspect, the forced BT split as such is a recursive horizontal forced BT split when the current block is located at the lower boundary of the image, or a recursive vertical forced BT split. when the current block is located on the right border of the image.

In the second embodiment of the device according to the ninth aspect as such or any preceding embodiment of the ninth aspect, the forced division BT comprises recursively dividing the current block with a horizontal forced boundary division until the division of the current block is at a lower boundary; and recursively splitting the subdivision by means of vertically forced boundary splitting until the leaf node is completely located on the right boundary.

In the third embodiment of the apparatus according to the ninth aspect as such or any preceding embodiment of the ninth aspect, the forced division BT comprises recursively dividing the current block with a vertical forced boundary division until the division of the current block is at a lower boundary; and recursively splitting the subdivision with a horizontal forced boundary split until the leaf node is completely on the right boundary.

In the fourth form of device implementation according to the ninth aspect, as such, or in any previous form of implementation of the ninth aspect, the instructions further cause the device to apply MinQTSize to control the division of the non-boundary block.

In the fifth form of implementation of the device according to the ninth aspect as such or any previous form of implementation of the ninth aspect, the instruction further causes the device to receive MinQTSize via sequence parameter set (SPS).

In a sixth form of implementation of the device according to the ninth aspect as such or any previous form of implementation of the ninth aspect, the instruction further causes the device to transmit MinQTSize via a sequence parameter set (SPS).

For the sake of clarity, any of the embodiments disclosed herein may be combined with any one or more other embodiments to form a new embodiment within the scope of the present invention.

These and other features will be better understood from the following detailed description, together with the accompanying drawings and claims.

Brief description of the drawings

For a more complete understanding of the present invention, reference will be made to the following brief description, together with the accompanying drawings and a detailed description in which like reference numbers represent like parts.

1A is a block diagram illustrating an exemplary coding system that may implement embodiments of the present invention.

1B is a block diagram illustrating another example of a coding system that may implement embodiments of the present invention.

2 is a block diagram illustrating an example of a video encoder that may implement embodiments of the present invention.

3 is a block diagram illustrating an example of a video decoder that may implement embodiments of the present invention.

4 is a diagram of a network device according to an embodiment of the present invention.

FIG. 5 is a simplified block diagram of a device that can be used as source device 12 and destination device 14, or both, of FIG. 1A according to an exemplary embodiment.

6A-F show different CU partitioning modes in VVC.

7A shows the forced QT split of the lower boundary of the CTU (128x128) HD (1920x1080).

7B shows forced splitting BT (128x128) of the lower boundary of the HD CTU (1920 x 1080) according to an embodiment of the invention.

8 shows an example of defining a boundary.

9A shows an example of forced separation of the QTBT of a corner situation according to an embodiment of the present invention.

9B shows an example of forced QTBT separation for a corner block according to an embodiment of the present invention.

Fig.10 shows a variant of the definition of the border.

Implementation of the invention

It should first be understood that while an exemplary implementation of one or more embodiments is provided below, the disclosed systems and/or methods may be implemented using any number of methods currently known or existing. The present invention should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, including the illustrative constructions and implementations illustrated and described herein, but may be varied within the scope of the appended claims, along with the full scope of their equivalents.

1A is a block diagram illustrating an exemplary coding system 10, such as a video coding system 10, that may implement embodiments of the present invention. The coding system 10 may use prediction techniques, such as bidirectional prediction techniques. As shown in FIG. 1A, encoding system 10 includes a source device 12 that provides encoded video data for decoding at a later time by destination device 14. In particular, source device 12 may provide video data to destination device 14 via computer-readable medium 16. Source device 12 and destination device 14 may comprise or be any of a wide range of devices, including desktop computers, laptops (i.e., laptop computers), tablet computers, set-top boxes, handsets, so-called smart phones, so-called smart tablets, TVs, cameras, display devices, digital media players, game consoles, video streaming devices, etc. In some cases, source device 12 and destination device 14 may be equipped for wireless communication.

Source device 14 may receive encoded video data for decoding via computer readable medium 16. Computer readable medium 16 may comprise any type of media or device configured to transfer encoded video data from source device 12 to destination device 14. In one example, computer-readable medium 16 may include a communication medium allowing source device 12 to transmit encoded video data directly to destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 14 . The communication medium may comprise any wireless or wired communication medium such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet network such as a local area network, a wide area network, or a wide area network such as the Internet. The communications medium may include routers, switches, base stations, or any other equipment that may be configured to provide communications from source device 12 to destination device 14.

In some examples, the encoded data may be output via the output interface 22 to a storage device. Similarly, the encoded data can be accessed from the storage device via the input interface 28 . The storage device may include any of a variety of distributed or locally available storage media such as a hard drive, Blu-ray discs, digital video discs (DVDs), compact disc read only memory (CD-ROM), flash memory, volatile or non-volatile memory or any other suitable digital storage medium for storing encoded video data. In a further example, the storage device may correspond to a file server or other intermediate storage device that may store the encoded video generated by source device 12. The destination device 14 may access the stored video data from the storage device via streaming or download. The file server may be any type of server capable of storing encoded video data and transmitting this encoded video data to the destination device 14 . Examples of file servers include a web server (eg, for a website), a file transfer protocol (FTP) server, network attached storage (NAS), or a local drive. Destination device 14 can access the encoded video data over any standard data connection, including an Internet connection. This may include a wireless link (such as a Wi-Fi connection), a wired connection (such as a digital subscriber line (DSL), cable modem, etc.), or a combination thereof that is suitable for accessing stored encoded video data on a file. server. The transmission of the encoded video data from the storage device may be streaming, downloading, or a combination thereof.

The methods of the present invention are not necessarily limited to wireless applications or settings. These techniques can be applied to video coding to support any of a variety of multimedia applications such as terrestrial television, cable television transmissions, satellite television transmissions, Internet video streaming such as HTTP Dynamic Adaptive Streaming (DASH), digital video, which is encoded on a storage medium, decoding digital video stored on a storage medium, or other applications. In some examples, encoding system 10 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcast, and/or video telephony.

In the example of FIG. 1A, source device 12 includes video source 18, video encoder 20, and output interface 22. The destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. In accordance with this invention, video encoder 20 of source device 12 and/or video decoder 30 of destination device 14 may be configured to apply a prediction method, such as bidirectional prediction. In other examples, source device 12 and destination device 14 may include other components or arrangements. For example, source device 12 may receive video data from an external video source such as an external camera. Likewise, destination device 14 may interact with an external display device rather than include an onboard display device.

The illustrated coding system 10 in FIG. 1A is just one example. Prediction techniques, such as bi-directional prediction, may be performed by any digital video encoding and/or decoding device. Although the methods of this present invention are typically performed by a video encoding device, the methods can also be performed by a video encoder/decoder, commonly referred to as "CODEC". Moreover, the methods of the present invention can also be performed by a video preprocessor. The video encoder and/or decoder may be a graphics processing unit (GPU) or the like.

Source device 12 and destination device 14 are merely examples of such encoding devices, in which source device 12 generates encoded video data for transmission to destination device 14. In some examples, source device 12 and destination device 14 may operate substantially symmetrically such that source device 12 and destination device 14 each include video encoding and decoding components. Therefore, encoding system 10 can support one-way or two-way video transmission between video devices 12, 14, such as for video streaming, video playback, video broadcast, or video telephony.

Video source 18 of source device 12 may include a video capture device such as a video camera, a video archive containing previously captured video, and/or a video stream interface for receiving video from a video content provider. As a further alternative, video source 18 may generate computer graphics based data as original video or a combination of live video, archived video, and computer generated video.

In some cases where video source 18 is a video camera, source device 12 and destination device 14 may form so-called camera phones or video phones. However, as mentioned above, the methods described in the present invention may be applicable to video coding in general and may be applied to wireless and/or wired applications. In each case, the captured, pre-captured, or computer-generated video may be encoded by the video encoder 20. The encoded video information may then be output via the output interface 22 to the computer-readable medium 16.

The computer readable medium 16 may include temporary media such as wireless broadcast or wired network transmission, or the computer readable medium 16 may include permanent storage media such as a hard drive, flash drive, CD, digital video disc, Blu disc. -ray or other machine-readable media. In some examples, a network server (not shown) may receive encoded video data from source device 12 and provide encoded video data to destination device 14, such as via a network transmission. Similarly, a manufacturing process computing device, such as a disc stamper, may receive encoded video data from source device 12 and produce a disc containing the encoded video data. Therefore, it can be understood that the computer-readable medium 16 includes one or more computer-readable media in various forms in various examples.

The input interface 28 of the destination device 14 receives information of the computer-readable medium 16. The information of the computer-readable medium 16 may include syntax information defined by video encoder 20, which is also used by video decoder 30, which includes syntax elements that describe the characteristics and/or processing of blocks and other coded units, such as a group of pictures (GOP). The display device 32 displays the decoded video data to the user and may include any of a variety of display devices such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light emitting diode (OLED), or other display device.

Video encoder 20 and video decoder 30 may operate in accordance with a video coding standard, such as the High Efficiency Video Coding (HEVC) standard currently under development, and may comply with the HEVC test model (HM). Alternatively, video encoder 20 and video decoder 30 may operate in accordance with other proprietary or industry standards, such as the H.264 standard of the Telecommunication Standardization Sector of the International Telecommunication Union (ITU-T), also referred to as the Moving Picture Experts Group (MPEG)-4, Part 10, Advanced Video Coding (AVC), H.265/HEVC, a draft Universal Video Coding (VVC) standard currently being developed by ITU-T and MPEG, or extensions to such standards. However, the methods of the present invention are not limited to any particular coding standard. Other examples of video coding standards include MPEG-2 and ITU-T H.263. Although not shown in FIG. 1A, in some aspects, video encoder 20 and video decoder 30 may be integrated with an audio encoder and decoder and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other encoding processing hardware and software such as audio and video in a common data stream or in separate data streams. If applicable, MUX-DEMUX blocks may conform to the ITU-T H.223 multiplexer protocol or other protocols such as User Datagram Protocol (UDP).

Video encoder 20 and video decoder 30 may each be implemented as any of a variety of suitable encoder or decoder circuits, or as a common logic or encoder circuit, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs). ), field-programmable gate arrays (FPGAs), discrete logic, graphics processing units (GPUs), processors (eg, programmable, eg, software-programmable), software, hardware, firmware, or any combination thereof. When the methods are partially implemented in software, the device may store instructions for the software on a suitable non-volatile computer-readable medium and execute the instructions in hardware using one or more processors to perform the methods of the present invention. Each of video encoder 20 and video decoder 30 may be contained in one or more encoders or decoders, each of which may be integrated as part of a combined encoder/decoder (CODEC) in a respective device. The device including video encoder 20 and/or video decoder 30 may include an integrated circuit, a microprocessor, and/or a wireless communication device such as a cellular phone.

FIG. 1B is an illustrative diagram of an exemplary video coding system 40 including video encoder 20 in FIG. 2 and/or video decoder 30 in FIG. 3, in accordance with an exemplary embodiment. Video coding system 40 may implement the methods of the present invention. In the illustrated implementation, the video coding system 40 may include an imaging device(s) 41, a video encoder 20, a video decoder 30 (and/or a video encoder implemented by processing block(s) 46 logic 47), an antenna 42, one or more processors 43, one or more memory or memory stores 44 and/or a display device 45. As shown, imaging device(s) 41, antenna 42, processing unit(s) 46, logic circuit 47, video encoder 20, video decoder 30, processor(s) 43, memories and/or memory storage 44 and/or display device 45 may be configured to interact with each other. As discussed, although FIG. 1B shows both video encoder 20 and video decoder 30, video coding system 40 may include only video encoder 20 or only video decoder 30 in various examples.

As shown, in some examples, video coding system 40 may include an antenna 42. Antenna 42 may be configured to transmit or receive an encoded video bitstream, for example. In addition, in some examples, the video encoding system 40 may include a display device 45 .

The display device 45 may be configured to present video data. As shown, in some examples, logic 47 may be implemented by processing unit(s) 46 . Processing unit(s) 46 may include application specific integrated circuit (ASIC) logic, general purpose graphics processor(s). processor(s) or the like. Video encoding system 40 may also include additional processor(s) 43, which may likewise include application specific integrated circuit (ASIC) logic, graphics processor(s), general purpose processor(s), and the like. In some examples, logic circuitry 47 may be implemented in hardware, dedicated video coding hardware, and the like, and processor(s) 43 may be implemented in general purpose software, operating systems, and the like. Additionally, memory 44 may be any type of memory, such as volatile memory (eg, static random access memory (SRAM), dynamic random access memory (DRAM), etc.) or non-volatile memory (eg, flash memory, etc.). etc.) etc. In a non-limiting example, the memory 44 may be implemented by means of a cache memory. In some examples, logic circuitry 47 may access memory 44 (eg, to implement an image buffer). In other examples, logic circuitry 47 and/or processing unit(s) 46 may include memory stores (eg, cache, etc.) for implementing an image buffer or the like.

The graphics processing unit may include a video encoder 20 implemented by logic 47 for implementing various modules as described with reference to FIG. 2 and/or any other coding system or subsystem described herein. Logic circuit 47 may be configured to perform various operations discussed herein. Video decoder 30 may be implemented in a manner similar to that implemented by logic circuitry 47 to implement various modules as described with respect to decoder 30 in FIG. 3 and/or any other decoder system or subsystem described herein. In some examples, video encoder 20 and video decoder 30 may be implemented with logic and may include an image buffer (eg, via processing unit(s) 46 or memory 44) and a graphics processor. (GPU) (eg, via the processor(s) 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include a video decoder 30 implemented via logic 47 to implement various modules as discussed with reference to FIG. 3 and/or any other decoder system or subsystem described herein.

In some examples, antenna 42 of video coding system 40 may be configured to receive an encoded video bitstream. As discussed, the encoded bitstream may include data, indicators, index values, mode selection data, and the like associated with encoding a video frame as discussed herein, such as data associated with an encoding section (eg, transform coefficients or quantized transform coefficients, possible indicators (as discussed) and/or data defining the coding section). Video coding system 40 may also include a video decoder 30 coupled to antenna 42 and configured to decode the encoded bitstream. The display device 45 is configured to present video frames.

2 is a block diagram illustrating an example of a video encoder 20 that may implement the methods of the present invention. Video encoder 20 may perform intra-coding and inter-coding of video blocks in video segments. Intra-coding relies on spatial prediction to reduce or remove spatial redundancy in video within a given video frame or picture. Inter-coding relies on temporal prediction to reduce or remove temporal redundancy in video in adjacent frames or pictures of a video sequence. The intra prediction mode (mode I) may refer to any of several spatial coding modes. Inter prediction modes such as unidirectional prediction (P mode) or bidirectional prediction (B mode) may refer to any of several time-based coding modes.

As shown in FIG. 2, video encoder 20 receives the current block of video in the video frame to be encoded. In the example of FIG. 2, video encoder 20 includes a mode selector 40, a reference frame memory 64, an adder 50, a transform processing block 52, a quantization block 54, and an entropy coding block 56. The mode selection block 40 includes a motion compensation block 44 , a motion estimation block 42 , an intra prediction block 46 , and a division block 48 . For video block reconstruction, video encoder 20 also includes an inverse quantizer 58, an inverse transform block 60, and an adder 62. A deblocking filter (not shown in FIG. 2) can also be included to filter block boundaries to remove blocking artifacts from the reconstructed video. Desirably (eg, enabled or implemented at all), the deblocking filter filters the output of adder 62. Additional filters (in-loop or post-loop) may also be used in addition to the deblocking filter. Such filters are not shown for brevity, but may optionally filter, for example, the output of adder 50 (as an in-loop filter).

During the encoding process, video encoder 20 receives a video frame or segment to be encoded. A frame or segment can be divided into multiple video blocks. The motion estimator 42 and the motion compensator 44 perform inter prediction encoding of the received video block with respect to one or more blocks in one or more reference frames for temporal prediction. Intra predictor 46 may alternatively perform intra prediction encoding of the received video block with respect to one or more neighboring blocks in the same frame or slot as the block to be encoded to provide spatial prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate encoding mode for each block of video data.

In addition, the splitter 48 may split blocks of video data into subblocks or partitions based on the evaluation of previous splitting schemes in previous passes or encoding steps. For example, splitter 48 may initially split a frame or segment into largest coding units (LCUs) and split each of the LCUs into sub-coding units (sub-CUs) based on rate distortion analysis (eg, rate-distortion optimization). The mode selector 40 may further generate a quadtree data structure indicating or indicating the division of the LCU into sub-CUs. The leaf node CUs of the quadtree may include one or more prediction units (PUs) and one or more transform units (TUs).

The present invention uses the term "block" to refer to any of the CU, PU or TU, eg in the context of HEVC or VVC, or similar data structures in the context of other standards (eg, macroblocks and their subblocks in H.264/AVC). The CU includes, for example, an encoding node, PUs, and TUs associated with the encoding node. The size of the CU corresponds to the size of the coding unit and has a square shape, for example. The size of a CU can vary from 8×8 pixels to a treeblock size with a maximum of 64×64 pixels or more, such as 128x128 pixels. Each CU may contain one or more PUs and one or more TUs. Syntax data associated with a CU may describe, for example, partitioning the CU into one or more PUs. The split modes may differ depending on whether the CU is encoded in skip mode or direct mode, intra prediction mode or inter prediction mode. PUs can be divided into non-square pieces. Syntax data associated with a CU may also describe, for example, splitting the CU into one or more TUs according to a square tree. In an embodiment, the CU, PU, or TU may be square or non-square (eg, rectangular).

The mode selector 40 may select one of the coding modes, inner or outer, for example, based on the results of errors, and provide the resulting inner or outer encoding mode to adder 50 for generating residual block data and adder 62 for recovering the coded block for use as a reference. frame. The mode selector 40 also provides syntax elements such as motion vectors, intra coding mode indicators, partition information, and/or other such syntax information to the entropy coding block 56 .

The motion estimation block 42 and the motion compensation block 44 may be highly integrated, but are shown separately for conceptual purposes. Motion estimation performed by motion estimation block 42 is a process of generating motion vectors that estimate motion for blocks of video. A motion vector, for example, may indicate the movement of a PU of a video block within the current video or image frame with respect to a prediction block in a reference frame (or other coded block) with respect to the current block being coded within the current frame (or other coded block) . A prediction block is a block that is found to closely match the block to be encoded in terms of pixel difference, which may be defined by sum absolute difference (SAD), sum difference of squares (SSD), or other difference metrics. In some examples, video encoder 20 may calculate values for sub-integer pixel positions of reference pictures stored in reference frame memory 64 . For example, video encoder 20 may interpolate one-quarter pixel positions, one-eighth pixel positions, or other fractional pixel positions of a reference frame. Therefore, the motion estimator 42 can search for motion with respect to full pixel positions and fractional pixel positions, and output a motion vector accurate to fractional pixels.

The motion estimator 42 calculates a motion vector for the PU of the video block in the inter prediction encoded segment by comparing the position of the PU with the position of the reference picture prediction block. The reference picture may be selected from a first reference picture list (List 0) or from a second reference picture list (List 1), each of which identifies one or more reference pictures stored in the reference frame memory 64 . The motion estimator 42 sends the calculated motion vector to the entropy encoding block 56 and the motion compensation block 44 .

The motion compensation performed by the motion compensator 44 may include fetching or generating a prediction block based on the motion vector determined by the motion estimator 42 . Again, in some examples, motion estimation unit 42 and motion compensation unit 44 may be functionally integrated. After receiving the motion vector for the PU of the current video block, the motion compensator 44 may locate the prediction block pointed to by the motion vector in one of the reference picture lists. The adder 50 generates a residual video block by subtracting the pixel values of the prediction block from the pixel values of the current encoded video block to generate pixel difference values as described below. In general, motion estimator 42 performs motion estimation with respect to luma components, and motion compensator 44 uses motion vectors computed based on luminance components for both chrominance and luminance components. The mode selector 40 may also generate syntax elements associated with the video block and video segment for use by video decoder 30 when decoding the video blocks of the video segment.

The intra prediction unit 46 may intra predict the current block as an alternative to the inter prediction performed by the motion estimator 42 and the motion compensation unit 44 as described above. In particular, intra prediction block 46 may determine the intra prediction mode to use for encoding the current block. In some examples, intra prediction block 46 may encode the current block using different intra prediction modes, such as during separate passes or encoding steps, and intra prediction block 46 (or mode selector 40 in some examples) may select an appropriate intra prediction mode for use from the tested modes.

For example, intra prediction unit 46 may calculate rate distortion values using rate distortion analysis for the various intra prediction modes tested, and select the intra prediction mode having the best rate-distortion performance among the tested modes. Rate distortion analysis typically determines the amount of distortion (or error) between the encoded block and the original unencoded block that was encoded to generate the encoded block, and the bitrate (ie, number of bits) used to generate the encoded block. Intra prediction block 46 may calculate distortion and rate ratios for various coded blocks to determine which intra prediction mode exhibits the best rate distortion value for the block.

Additionally, intra predictor 46 may be configured to encode depth map depth blocks using a depth modeling mode (DMM). The mode selector 40 may determine if the available DMM mode provides better coding results than the intra prediction mode and other DMM modes, for example, using rate-distortion optimization (RDO). The data for the texture image corresponding to the depth map may be stored in the 64 reference frame memory. Motion estimator 42 and motion compensator 44 may also be configured to inter-predict depth blocks of a depth map.

After selecting an intra prediction mode for a block (eg, a normal intra prediction mode or one of the DMM modes), intra prediction block 46 may provide information indicative of the selected intra prediction mode for the block to entropy encoding block 56 . Entropy encoding unit 56 may encode information indicating the selected intra prediction mode. Video encoder 20 may include transmitted bitstream configuration data, which may include a plurality of intra prediction mode index tables and a plurality of modified intra prediction mode index tables (also referred to as codeword mapping tables), coding context definitions for various blocks, and most likely indicators. an intra prediction mode index table, an intra prediction mode index table, and a modified intra prediction mode index table for use in each of the contexts.

Video encoder 20 generates a residual video block by subtracting the prediction data from mode selector 40 from the original encoded video block. The adder 50 represents the component or components that perform this subtraction operation.

The transform processing unit 52 applies a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform, to the residual block, generating a video block containing residual transform coefficient values. The transform processing unit 52 may perform other transforms that are conceptually similar to the DCT. Wavelet transforms, integer transforms, subband transforms, or other types of transforms may also be used.

The transform processing unit 52 applies the transform to the residual block, generating a residual block of transform coefficients. The transform may transform the residual information from the pixel range into a transform region, such as a frequency domain. Transform processing unit 52 may send the resulting transform coefficients to quantization unit 54 . Quantizer 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be changed by adjusting the quantization parameter. In some examples, quantizer 54 may then perform a matrix scan including the quantized transform coefficients. Alternatively, the entropy encoding block 56 may perform the scanning.

After quantization, entropy encoding block 56 entropy encodes the quantized transform coefficients. For example, entropy encoding unit 56 may perform context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval division entropy coding (PIPE ), or another way of entropy coding. In the case of context-based entropy coding, the context may be based on neighboring blocks. Following entropy encoding by entropy encoding unit 56, the encoded bit stream may be transferred to another device (eg, video decoder 30) or archived for later transmission or retrieval.

An inverse quantizer 58 and an inverse transform 60 apply inverse quantization and inverse transform, respectively, to reconstruct a residual block in the pixel region, for example, for later use as a reference block. Motion compensation block 44 may determine a prediction block (also referred to as a prediction block), such as a motion compensated prediction block, based on a motion vector and a corresponding reference block, such as obtained from a reference frame of one of the reference frame memory 64 frames. Motion compensator 44 may also apply one or more interpolation filters to calculate a sub-integer pixel value of the reference block for use in motion estimation or compensation. Adder 62 adds the reconstructed residual block to the (motion compensated) prediction block generated by motion compensator 44 to generate a reconstructed video block for storage in reference frame memory 64 . The reconstructed video block may be used by the motion estimator 42 and the motion compensation block 44 as a reference block for the outer coding block in a subsequent video frame.

Other structural variations of video encoder 20 may be used to encode the video stream. For example, video encoder 20 without transform may quantize the residual signal directly without transform processor 52 for certain blocks or frames. In another implementation, encoder 20 may have quantizer 54 and inverse quantizer 58 combined into one block.

3 is a block diagram illustrating an example of a video decoder 30 that may implement the methods of the present invention. In the example of FIG. 3, video decoder 30 includes an entropy decoder 70, motion compensation 72, intra prediction 74, inverse quantizer 76, inverse transform 78, reference frame memory 82, and adder 80. Video decoder 30 may, in in some examples, perform a decoding pass or path generally reverse of the encoding pass or path described with respect to video encoder 20 in FIG. Motion compensation unit 72 may generate prediction data based on motion vectors received from entropy decoding unit 70, while intra prediction unit 74 may generate prediction data based on intra prediction mode indicators received from entropy decoding unit 70.

During the decoding process, video decoder 30 receives an encoded video bitstream that represents a block of video and an encoded video segment and their associated syntax elements generated by video encoder 20. Entropy decoding block 70 of video decoder 30 performs entropy decoding of the bitstream to generate quantized coefficients, motion vectors, or intra prediction mode indicators; and other syntax elements. Block 70 entropy decoding sends the motion vectors and other elements of the syntax in block 72 motion compensation. Video decoder 30 may receive syntax elements at the video segment level and/or the video block level.

When a video segment is encoded as an intra (I) segment, intra prediction block 74 may generate prediction data for a video block of the current video segment based on the intra prediction signaling mode and data from previously decoded blocks of the current frame or picture. When a video frame is encoded as an inter-coded segment (i.e., B, P, or GPB), motion compensator 72 generates prediction blocks for the video block of the current video segment based on the motion vectors and other received syntax elements from entropy decoder 70. The prediction blocks may be generated from one of the reference pictures in one of the reference picture lists. Video decoder 30 may generate the reference frame lists, list 0 and list 1, using default construction methods based on the reference pictures stored in the reference frame memory 82 .

The motion compensation block 72 determines prediction information for the video block of the current video segment by analyzing motion vectors and other syntax elements, and uses the prediction information to generate prediction blocks for the current video block being decoded. For example, the motion compensator 72 uses some of the received syntax elements to determine the prediction mode (eg, intra or inter prediction) used to encode the video blocks of a video segment, the inter prediction segment type (eg, B-segment, P-segment, or GPB -segment), structure information for one or more reference picture lists for the segment, motion vectors for each inter-coded video block of the video segment, inter-prediction status for each inter-segment video block, and other information for decoding video blocks in the current segment .

The motion compensation block 72 may also perform interpolation based on interpolation filters. Motion compensator 72 may use the interpolation filters that are used by video encoder 20 during video block encoding to calculate an interpolated value for the sub-integer pixels of the reference blocks. In this case, the motion compensator 72 may determine the interpolation filters used by the video encoder 20 from the received syntax elements and use the interpolation filters to generate prediction blocks.

The data for the texture image corresponding to the depth map may be stored in the 82 reference frame memory. Motion compensation block 72 may also be configured to inter-predict depth blocks of a depth map.

Returning to FIG. 1A, the video encoding system 10 may be used to implement various video encoding or compression techniques. Several video compression techniques such as inter-prediction, intra-prediction, and loop filters have been shown to be effective. Therefore, video compression techniques have been adopted into various video coding standards such as H.264/AVC and H.265/HEVC.

Various coding tools such as adaptive motion vector prediction (AMVP) and merging mode (MERGE) can be used to predict motion vectors (MV) and improve inter-prediction performance and hence overall video compression efficiency.

The above MVs can be used in bidirectional prediction. In a bidirectional prediction operation, two prediction blocks are generated. One prediction block is generated using MV list 0 (referred to here as MV0). Another prediction block is generated using List 1 MV (referred to herein as MV1). The two prediction blocks are then combined (eg, averaged) to form a single prediction signal (eg, a prediction block or a predictor block).

Other variants of video decoder 30 may be used to decode the compressed bit stream. For example, decoder 30 may generate an output video stream without a loop filter unit. For example, a non-transform-based decoder 30 can directly inversely quantize the residual signal without the inverse transform processing unit 78 for certain blocks or frames. In another implementation, video decoder 30 may have an inverse quantization unit 76 and an inverse transform processing unit 78 combined into one unit.

4 is a diagram of a network device 400 (eg, an encoder) according to an embodiment of the present invention. The network device 400 is suitable for implementing the disclosed embodiments as described herein. In an embodiment, network device 400 may be a decoder, such as video decoder 30 in FIGS. 1A and 3, or an encoder, such as video encoder 20 in FIGS. 1A and 2. In an embodiment, network device 400 may be one or more components of video decoder 30 along 1A and 3 or video encoder 20 of FIGS. 1A and 2 as described above.

The network device 400 includes input ports 410 and receiver units (Rx) 420 for receiving data; a processor, logical unit, or central processing unit (CPU) 430 for processing data; blocks (Tx) 440 transmitter and output ports 450 for data transmission; and a memory 460 for storing data. Network device 400 may also include optical-electrical (OE) components and electrical-optical (EO) components connected to input ports 410, receiver units 420, transmitter units 440, and output ports 450 for outputting or inputting optical or electrical signals.

Processor 430 may be implemented in hardware and software. Processor 430 may be implemented as one or more CPU chips, cores (eg, as a multi-core processor), FPGA, ASIC, and DSP. Processor 430 is connected to input ports 410, receiver units 420, transmitter units 440, output ports 450, and memory 460. Processor 430 includes an encoder 470. Encoding module 470 implements the disclosed embodiments described above. For example, encoding module 470 implements, processes, prepares, or provides various encoding operations. Thus, the use of the encoder 470 provides a significant improvement in the functionality of the network device 400 and, as a result, transforms the network device 400 into a different state. Alternatively, encoding module 470 is implemented as instructions stored in memory 460 and executed by processor 430.

Memory 460 includes one or more disks, tape drives, and solid state drives and can be used as a storage device for overflowing data, for storing programs when such programs are selected for execution, and for storing instructions and data that are read during program execution. Memory 460 may be volatile and/or non-volatile and may be read only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), and/or static random access memory (SRAM).

FIG. 5 is a simplified block diagram of a device 500 that can be used as source device 12 and destination device 14, or both, in FIG. 1A according to an exemplary embodiment. Device 500 may implement the methods of the present invention. Device 500 may be in the form of a computing system including multiple computing devices, or in the form of a single computing device such as a mobile phone, tablet computer, laptop computer, laptop computer, desktop computer, and the like, like, like.

Processor 502 in device 500 may be a central processing unit. Alternatively, processor 502 may be any other type of device, or a plurality of devices, configured to manipulate or process information that currently exists or will be developed in the future. Although the disclosed implementations may be implemented with a single processor, as shown with processor 502, for example, speed and efficiency advantages may be achieved using more than one processor.

Memory 504 in device 500 may be read only memory (ROM) or random access memory (RAM) in an implementation. The memory 504 may be any other suitable type of storage device. Memory 504 may include code and data 506 accessed by processor 502 using bus 512. Memory 504 may further include an operating system 508 and application programs 510, and application programs 510 include at least one program , which allows the processor 502 to perform the methods described here. For example, applications 510 may include applications 1 through N, which further include a video encoding application that performs the methods described herein. Device 500 may also include additional memory in the form of secondary storage 514, which may be, for example, a memory card used with a mobile computing device. Because video communications may contain a significant amount of information, they may be stored in whole or in part in secondary storage 514 and loaded into memory 504 as needed for processing.

The device 500 may also include one or more output devices, such as a display 518. The display 518 may be, in one example, a touch display that combines a display with a touch element that can act as a touch input. Display 518 may be connected to processor 502 via bus 512. Other output devices that allow a user to program or otherwise use device 500 may be provided in addition to or as an alternative to display 518. When the output device is operating or includes a display, the display may be implemented in various ways, including liquid crystal display (LCD), cathode ray tube (CRT) display, plasma display, or light emitting diode (LED) display, for example , organic light-emitting diode display (OLED).

The device 500 may also include or interact with an imaging device 520, such as a camera or any other imaging device 520, currently or in the future, that can capture an image, such as an image of a user operating the device 500. The imaging device 520 may be positioned to face a user operating the device 500. In an example, the position and optical axis of the imaging device 520 may be configured such that the field of view includes an area that is immediately adjacent to the display. 518 and from which the display 518 is visible.

The device 500 may also include or interact with an audio capture device 522, such as a microphone or any other existing or currently developed audio device that can sense sounds near the device 500. The audio capture device 522 may be positioned so that it has been directed at the user operating the device 500 and may be configured to receive sound, such as speech or other sounds made by the user, when the user is operating the device 500.

Although FIG. 5 shows processor 502 and memory 504 of device 500 as integrated into a single unit, other configurations may be used. The operations of the processor 502 may be distributed among multiple machines (each machine has one or more processors), which may be connected directly or through a local area or other network. Memory 504 may be distributed across multiple machines, such as network memory or memory on multiple machines performing the operations of device 500. Although shown here as a single bus, bus 512 of device 500 may be comprised of multiple buses. Additionally, the secondary storage device 514 may be directly connected to other components of the device 500, or may be accessible via a network, and may comprise a single integrated unit, such as a memory card, or multiple units, such as multiple memory cards. Thus, device 500 can be implemented in a wide variety of configurations.

The Next Generation Universal Video Coding (VVC) standard is the latest joint video project of the International Telecommunication Union (ITU-T) Video Coding Experts Group (VCEG) and the International Organization for Standardization/International Electrotechnical Commission. (ISO/IEC) Moving Picture Experts Group (MPEG) standards organizations working together in a partnership known as the Joint Video Exploration Team (JVET). In VVC, a segmentation structure with multiple tree types (binary/triple/quaternary) (BT/TT/QT or binary tree/triple tree/quaternary tree) replaces or can replace the concept of multiple partition partition block types, i.e. removes the concepts of CU, PU, and TU partitioning except when necessary for CUs that are too large for the maximum transform length, and maintains more flexibility for CU partitioning forms. [JVET-J1002].

6A-F illustrate, by way of example, the separation modes currently used in the VTM. Fig. 6A shows a non-split block (no split), Fig. 6Bb shows a quaternary split or square tree (QT), Fig. 6C shows a horizontal binary split or binary tree (BT), Fig. 6D shows a vertical binary split or binary tree (BT ), FIG. 6E shows a horizontal ternary partition or ternary tree (TT), FIG. 6F shows a vertical ternary partition or ternary tree (TT) of a block such as a CU or CTU. Embodiments may be configured to implement split modes as shown in FIGS. 6A-6F.

In embodiments, the following parameters may be defined and indicated by sequence parameter set (SPS) syntax elements for the BT/TT/QT coding tree scheme:

CTU size: the size of the root node of the quaternary tree.

MinQTSize: minimum allowable leaf node size of a quaternary tree

MaxBTTSize: The maximum allowable size of the root node of a binary and ternary tree.

MaxBTTDepth: the maximum allowed binary and ternary tree depth, and

MinBTTSize: The minimum allowed size for a binary and ternary tree leaf node.

In other embodiments, the minimum allowable leaf node size MinQTSize of a quaternary tree may also be contained in other headers or sets, such as a segment header (SH) or a picture parameter set (PPS).

In the HEVC standard, coding tree blocks (CTUs) or coding units (CUs) that are located on segment/picture boundaries will be forced to split using a square tree (QT) until the bottom right sample of a leaf node is located within the boundary. segment/image. The forced QT split or split does not need to be signaled in the bitstream because both the encoder and decoder, for example, both video encoder 20 and video decoder 30 know when to apply forced QT. The purpose of forced separation is to make the CTU/CU boundary possible with video encoder 20/video decoder 30.

International patent publication number WO 2016/090568 discloses a QTBT (square tree plus binary tree) structure and also in VTM 1.0, the CTU/CU forced splitting process at the boundary is inherited from HEVC. This means that a CTU/CU located on a frame boundary is forced to be split by a square tree (QT) structure without regard to rate-distortion (RD) optimization until the entire current CU is inside the picture boundary. These forced sections are not transmitted in the bitstream.

7A shows an example of forced splitting for a high definition (HD) lower bound (128x128) CTU (1920x1080 pixels) by forced QT splitting. 7, an HD image has or has a size of 1920x1080 pixels and a CTU has or has a size of 128x128 pixels.

In SubCE2 (image edge processing) CE1 (separation), the San Diego meeting (04.2018) [JVET-J1021] proposed 15 tests for image edge processing using BT, TT, or ABT (asymmetric BT). For example, in JVET-K0280 and JVET-K0376, the boundary is defined as shown in FIG. 8, the borders of the image are shown with dotted lines and the regions of the border cases are illustrated with straight lines, namely, the bottom border case, the corner border case, and the right border case. Lower bound can be split by horizontal forced BT or forced QT, right bound can be split by vertical forced BT or forced QT, corner case can only be split by forced QT, and the decision to use either forced BT or forced QT split is based on the optimization criterion rate distortion and is transmitted in the bitstream. Forced split means that the block must be split, for example, forced split is applied to boundary blocks that cannot be encoded using "no split" as shown in FIG. 6A.

If a forced QT split is used in a forced boundary split, the MinQTSize split constraint is ignored. For example, in FIG. 9A, if MinQTSize is passed as 32 in the SPS to match the bound with the forced QT method, it will be necessary to split the QT to a block size of 8x8, which ignores the MinQTSize constraint of 32.

According to embodiments of the invention, if a forced QT is used to separate image boundaries, then a forced QT separation is necessarily used, eg, does not ignore the separation constraint as specified in, for example, SPS. If additional forced separation is needed, only forced BT is used, which in combination can also be called forced QTBT. In embodiments of the invention, for example, the split constraint MinQTSize is taken into account to force QT split at image boundaries, and no additional signaling is required to force BT split. Embodiments also allow the separation to be negotiated for normal (non-boundary) blocks and boundary blocks. For example, traditional solutions require two "MinQTSize" parameters: one for normal block separation and another for boundary block separation. Embodiments require only one common parameter "MinQTSize" for normal block and boundary block separation, which can be flexibly set between encoder and decoder, for example signaling one parameter "MinQTSize". In addition, embodiments require fewer partitions than, for example, forced QT.

Solutions for the Lower Bound Case and the Right Bound Case

In case of bottom and right border, if the block size is larger than MinQTSize, then the division mode for the border division of the image can be selected between BT forced division and QT forced division, for example, based on RDO. Otherwise (i.e. if the block size is equal to or less than MinQTSize), only the forced separation BT is used to separate the image borders, more specifically, the forced horizontal separation BT is used for the bottom border respectively for the border box located on the bottom border of the image . image, and vertical forced BT is used for the right border, respectively, for the border box located on the right border of the image.

Forced BT splitting may comprise recursively splitting the current block with a horizontal forced border split until the subsection of the current box is at the bottom border of the image, and recursively splitting the subsection with a vertical forced border split until the entire leaf node is at the bottom of the image. right border of the image. Alternatively, forced splitting of a BT may comprise recursively splitting the current block with a vertical forced boundary splitting until the subsection of the current box is at the bottom boundary, and recursively splitting the subsection with a horizontal forced boundary splitting until the entire leaf node is will not be located on the right border. MinQTSize can also be used to control the division of a non-boundary block.

For example, in the case shown in FIG. 7A, if MinQTSize is equal to or limited to 32, while a rectangular (non-square) block size of 8 samples high or wide is needed to fit the image border, forced BT splitting will be used to split the 32x32 border located block. BT partitions can be further partitioned using the same type of forced BT splitting, for example, in case of forced vertical splitting BT, only additional forced vertical splitting BT is applied, and in case of forced horizontal splitting BT, only additional forced horizontal splitting BT is applied. Forced BT splitting continues until the leaf node is completely within the image.

7B shows an exemplary 128x128 sample lower bound CTU split according to an embodiment of the invention. The lower boundary CTU, which forms the root block or root node of the partition tree, is broken into smaller sections, such as smaller square or rectangular sized blocks. These smaller sections or blocks may be further subdivided into even smaller sections or blocks. In FIG. 7B, the CTU is the first square tree divided into four square blocks 710, 720, 730, and 740, each of which has a size of 64x64 samples. Of these blocks, blocks 710 and 720 are again lower boundary blocks, while blocks 730 and 740 are out of the image (respectively, out of the image) and are not processed.

Block 710 is further divided using a quad tree into four square blocks 750, 760, 770, and 780, each of which has a size of 32x32 samples. Blocks 750 and 760 are located inside the image, while blocks 770 and 780 again form the lower boundary blocks. Since the size of block 770 does not exceed MinQTSize, which is equal to 32, for example, a recursive horizontal forced binary split is applied to block 770 until the leaf node is completely within or completely inside the image, for example, until block 772 of leaf nodes is rectangular a non-square block having 32x16 samples is inside the image (after one horizontal bin split), or a leaf node block 774, a rectangular non-square block located at the bottom edge of the image and having 32x8 samples is inside the image (after two horizontal bin splits). The same goes for block 780.

Embodiments of the present invention allow for matching the separation for normal blocks located entirely within the image and the separation of boundary blocks. Boundary boxes are boxes that are not completely inside the image and not completely outside the image. In other words, border boxes are boxes that contain a part that is located inside the image and a part that is located outside the image. In addition, embodiments of the present invention can reduce signaling since forced BT split signaling at or below the MinQTSize level is not required.

Corner Case Solutions

In the corner case, some approaches only allow for forced QT splitting, which also ignores the MinQTSize constraint. Embodiments of the present invention provide two solutions for the corner case.

Solution 1:

The corner case is treated as a lower boundary case or as a right boundary case. 10 shows a variant of the boundary definition. 10 shows the borders of the image with dotted lines and the edge case areas are shown with straight lines. As shown, the corner case is defined as the lower boundary case. Thus, the solution is the same as described above for the lower border case and the right border case. In other words, first apply the horizontal split (as described for the bottom border case) until the blocks or sections are completely within the image (in the vertical direction) and then apply the vertical split (as described for the right border case) until the leaf nodes will not be completely inside the image (in the horizontal direction).

Solution 2:

The definition of edge cases remains as is. If forced QT is limited by MinQTSize (current block size is equal to or less than MinQTSize), use forced horizontal BT to match lower bound, when lower bound is equal, use forced vertical BT to match right bound.

For example, in Figure 9A, which shows a forced QTBT variant for a block located in the corner of an image, if MinQTSize is equal to or limited to 32 to force corner case QT splitting, additional BT splitting will be used after splitting the 32x32 block until the forced splitting is completed. .

9B shows further details of an exemplary split of an edge CTU at or in a corner of an image according to an embodiment of the invention in which the CTU has a size of 128x128 samples. The CTU is the first square tree divided into four square blocks, each 64x64 samples in size. Of these boxes, only the upper left box 910 is a boundary box, while the other three are located outside (completely outside) the image and are not further processed. Block 910 is further partitioned using square tree partitioning into four square blocks 920, 930, 940, and 950, each having a size of 32x32 samples. Block 920 is located inside the image, while blocks 930, 940 and 950 again form boundary blocks. Since the size of these blocks 930, 940 and 950 is no larger than MinQTSize, which is 32, forced binary splitting is applied to blocks 930, 940 and 950.

Block 930 is located on the right border and is split using a recursive vertical forced binary split until the leaf node is inside the image, such as block 932 located on the right border of the image (here after two vertical binary splits).

Block 940 is located on the bottom border and is split using a recursive horizontal forced binary split until the leaf node is inside the image, such as block 942 located on the right border of the image (here after two horizontal binary splits).

Block 950 is located at the corner boundary and is split using a first recursive horizontal forced binary split until the subdivision or block, here block 952, is at the bottom border of the image (here after two horizontal binary splits) and then a recursive split is performed with vertical forced boundary split until a leaf node or block, such as block 954, located on the right border of the image (here after two vertical binary splits) or, respectively, until the leaf node is inside the image.

The above approaches can be applied to both decoding and encoding. For decoding MinQTSize can be received via SPS. For encoding, MinQTSize may be transmitted via SPS. Embodiments may use boundary definitions as shown in FIG. 8 or FIG. 10 or other boundary definitions.

The following are additional embodiments of the present invention. It should be noted that the numbering used in the following section does not have to match the numbering used in the preceding sections.

Embodiment 1: separation method comprising:

determining whether the current image block is a boundary block;

if the current box is a boundary box, determining whether the size of the current box exceeds the minimum allowable size of the leaf node of the quadtree;

if the current block size does not exceed the minimum allowed quadtree leaf node size, apply forced binary tree splitting to the current block.

Embodiment 2: The splitting method of Embodiment 1, wherein the forced binary tree splitting is a recursive horizontal forced binary splitting in case the current block is at the bottom boundary of the image, or is a recursive vertical forced boundary splitting in the case that the current block is located on the right side of the image.

Embodiment 3: The division method of Embodiment 1 or 2, wherein the forced binary division comprises recursively dividing the current block by horizontally forced boundary division until the division of the current block is located directly at the lower boundary of the image, and recursively dividing the division with using a vertical forced border split until the leaf node is located entirely on the right border of the image, or vice versa.

Embodiment 4: The partitioning method of any one of Embodiments 1 to 3, wherein the minimum allowable quadtree leaf node size is the minimum allowable quadtree leaf node size also used to control the division of a non-boundary block.

Embodiment 5: A decoding method for decoding a block by splitting a block according to the splitting method of any of Embodiments 1 to 4.

Embodiment 6: The decoding method according to Embodiment 5, in which the minimum allowable leaf node size of the quadtree is received via the SPS.

Embodiment 7: An encoding method for encoding a block by splitting a block according to the splitting method of any of Embodiments 1 to 4.

Embodiment 8: The coding method of Embodiment 7, in which the minimum allowable size of the leaf node of the quadtree is transmitted via the SPS.

Embodiment 9: A decoding device comprising a logic circuit capable of implementing any of the methods of Embodiments 5 or 6.

Embodiment 10: An encoding device comprising a logic circuit configured to implement any of the methods of Embodiments 7 or 8.

Embodiment 11: A non-volatile storage medium for storing instructions causing, when executed by the processor, execution, by the processor, of the method of any one of Embodiments 1-8.

In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware processing unit. The computer-readable medium may include a computer-readable medium that corresponds to a tangible medium, such as a data medium, or a communication medium, including any medium that facilitates the transfer of a computer program from one place to another, such as according to a communication protocol. Thus, computer-readable media can typically correspond to (1) tangible computer-readable storage media that are non-volatile, or (2) communication media such as a signal or carrier wave. Storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, or data structures for implementing the methods described in the present invention. The computer program product may include a computer readable medium.

By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, electrically erasable programmable read-only memory (EEPROM), CD-ROM or other optical disk storage, magnetic disk storage, and the like. magnetic storage devices, flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer. Additionally, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable , fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are defined as the communication medium. However, it should be understood that computer-readable storage media and storage media do not include connections, carrier waves, signals, or other temporary media, but are instead directed to permanent tangible storage media. The disc and disc used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where the discs usually reproduce data in a magnetic way, and the discs reproduce data optically using lasers. Combinations of the above should also be included in the list of computer-readable media.

The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor" as used herein may refer to any of the aforementioned structures, or any other structure suitable for implementing the methods described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules configured to encode and decode or contained in a combined codec. In addition, the methods may be fully implemented in one or more circuits or logic elements.

The methods of the present invention may be implemented in a wide variety of devices or devices, including a wireless telephone, an integrated circuit (IC), or a set of ICs (eg, a chipset). The present invention describes various components, modules, or blocks to emphasize the functional aspects of devices capable of performing the disclosed methods, but not necessarily requiring implementation in different hardware blocks. Rather, as described above, the various blocks may be combined into a codec hardware block, or provided with a set of cooperating hardware blocks, including one or more processors, as described above, in combination with suitable software and/or firmware.

The device contains a memory element; and a processor element associated with the memory element and configured to determine whether the current image block is a boundary box, determine when the current block is a boundary box, determine whether the size of the current block exceeds the minimum allowable size (MinQTSize) of the quadtree leaf node (QT ) and apply, when the size of the current block does not exceed MinQTSize, force binary tree splitting (BT) to the current block.

Although the present invention presents several embodiments, it should be understood that the disclosed systems and methods can be embodied in many other specific forms without departing from the essence or scope of the present invention. The present examples are to be considered illustrative and not restrictive, and the intention is not to be limited by the details given here. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

Additionally, the methods, systems, subsystems, and methods described and illustrated in various embodiments as separate entities may be combined or integrated with other systems, modules, technologies, or methods without departing from the scope of the present invention. Other elements shown or discussed as related or directly connected or interacting with each other may be indirectly connected or interact through some interface, device or intermediate component, electrically, mechanically or otherwise. Other examples of changes, substitutions and alterations can be established by a person skilled in the art and can be made without departing from the essence and scope of the present invention.

Abbreviations definitions and glossary

HEVC - High Efficiency Video Coding

VVC - Universal Video Coding

VTM - VVC test model

JEM - collaborative research model

CTU - coding tree block

CU - coding unit

BT - binary tree

TT - ternary tree

QT - square tree or quaternary tree

ABT - asymmetric BT

MTT - multitype tree

AMP - asymmetric section

SH - segment header

SPS - Sequence Parameter Set

PPS - Image Parameter Set

CE - main experiment

SubCE - sub-main experiment (part of the main experiment)

Claims (21)

1. A separation method, comprising the steps of: determining whether the current image block is a boundary block and whether the size of the current block exceeds the minimum allowable size of the leaf node of the quadtree; and applied when the current block is a boundary block and the size of the current block does not exceed the minimum allowable quadtree leaf node size (MinQTSize), forced binary tree splitting (BT) to the current block. 2. The splitting method according to claim 1, wherein the step of forcing binary tree splitting is a recursive horizontal forced binary splitting when the current block is at the bottom border of the image, or a recursive vertical forced border splitting when the current block is located at the right border of the image. 3. The partitioning method according to claim 1 or 2, wherein the step of forcibly splitting the binary tree continues until the leaf node block is within the image. 4. The splitting method according to claim 1 or 2, wherein the step of forcing binary splitting comprises sub-steps of: performing a recursive division of the current block through a horizontal forced border division until the division of the current block is at the bottom border of the image; and performing recursive subdivision splitting by vertically forced edge splitting until the leaf node is completely on the right border of the image. 5. The splitting method according to claim 1 or 2, wherein the step of forcing BT splitting comprises sub-steps of: performing a recursive division of the current block by forcing a vertical boundary division until the division of the current block is at the bottom boundary; and performing a recursive subdivision split by horizontally forced boundary splitting until the leaf node is completely on the right boundary. 6. The partitioning method according to claim 1 or 2, further comprising applying a minimum allowable quadtree leaf node size to control partitioning of the non-boundary block. 7. The partitioning method according to claim 1 or 2, wherein the boundary block is a block that is not completely inside the image and not completely outside the image. 8. A decoding method for decoding a block by splitting the block according to the splitting method of claim 1 or 2. 9. The decoding method of claim 8, further comprising receiving a minimum allowable leaf node size of the quadtree via a sequence parameter set (SPS). 10. An encoding method for encoding a block by splitting the block according to the splitting method according to claim 1 or 2. 11. The encoding method of claim 10, further comprising transmitting a minimum allowable leaf node size of the quadtree via a sequence parameter set (SPS). 12. A decoding device, comprising a logic circuit configured to decode the block by splitting the block according to the splitting method according to claim 1 or 2. 13. The decoder of claim 12, wherein the logic circuitry is further configured to receive a minimum allowable leaf node size of the quadtree via a sequence parameter set (SPS). 14. An encoding device, comprising a logic circuit configured to encode a block by dividing the block according to the dividing method according to claim 1 or 2. 15. A non-volatile storage medium for storing instructions causing, when executed by the processor, the execution, by the processor, of the method of claim 1 or 2.

Images