CN110636298B

CN110636298B - Unified constraints for Merge affine mode and non-Merge affine mode

Info

Publication number: CN110636298B
Application number: CN201910544642.0A
Authority: CN
Inventors: 张凯; 张莉; 刘鸿彬; 王悦
Original assignee: Beijing ByteDance Network Technology Co Ltd; ByteDance Inc
Current assignee: Beijing ByteDance Network Technology Co Ltd; ByteDance Inc
Priority date: 2018-06-21
Filing date: 2019-06-21
Publication date: 2022-09-13
Anticipated expiration: 2039-06-21
Also published as: US20210352302A1; TWI739120B; TW202007156A; US20210076050A1; US11197003B2; CN110636298A; WO2019244117A1

Abstract

Devices, systems, and methods for sub-block based prediction are described. In a representative aspect, a method for video processing includes: the method includes determining block size constraints, making a determination as to whether Merge affine mode and non-Merge affine mode are allowed for video blocks in a video frame based on the block size constraints, and generating a bitstream representation of the video blocks based on making the determination.

Description

Unified constraints for Merge affine mode and non-Merge affine mode

Cross Reference to Related Applications

The present application is required to claim in time the priority and benefit of international patent application No. PCT/CN2018/092118 filed on 21.6.6.2018 according to applicable patent laws and/or according to the rules of the paris convention. The entire disclosure of International patent application No. PCT/CN2018/092118 is incorporated by reference herein as part of the disclosure of the present application.

Technical Field

This patent document relates generally to image and video coding techniques.

Background

Motion compensation is a technique in video processing that predicts frames in a video given previous and/or future frames by taking into account the motion of the camera and/or objects in the video. Motion compensation may be used for encoding and decoding of video data for video compression.

Disclosure of Invention

Devices, systems, and methods related to subblock-based prediction for image and video encoding are described.

In representative aspects, the disclosed techniques may be used to provide a method for video processing. The method includes determining a block size constraint, making a determination as to whether Merge affine mode and non-Merge affine mode are allowed for a video block in a video frame based on the block size constraint, and generating a bitstream representation of the video block based on the making the determination.

In a representative aspect, the disclosed techniques may be used to provide another method for video processing. The method includes determining a block size constraint, making a determination as to whether Merge affine mode and non-Merge affine mode are allowed for a video block in a video frame based on the block size constraint, and generating a video block from a bitstream representation of the video block based on making the determination.

In yet another representative aspect, the above-described methods are implemented in the form of processor executable code and stored in a computer readable program medium.

In yet another representative aspect, an apparatus configured or operable to perform the above-described method is disclosed. The apparatus may include a processor programmed to implement the method.

In yet another representative aspect, a video decoder device may implement the methods described herein.

The above and other aspects and features of the disclosed technology are described in more detail in the accompanying drawings, the description and the claims.

Drawings

Fig. 1 shows an example of sub-block based prediction.

Fig. 2 shows an example of a simplified affine motion model.

Fig. 3 shows an example of an affine Motion Vector Field (MVF) for each sub-block.

Fig. 4 shows an example of Motion Vector Prediction (MVP) for the AF _ INTER affine motion mode.

Fig. 5A and 5B show example candidates for the AF _ MERGE affine motion mode.

FIG. 6 shows a block diagram for use in JEM with 4: 2: example of sub-blocks of different components of the 0 format.

Fig. 7A shows a flow diagram of an example method for video processing.

Fig. 7B illustrates another flow diagram of an example method for video processing.

FIG. 8 is a block diagram illustrating an example of an architecture of a computer system or other control device that may be used to implement various portions of the presently disclosed technology.

FIG. 9 illustrates a block diagram of an example embodiment of a device that may be used to implement portions of the presently disclosed technology.

Detailed Description

Due to the increasing demand for higher resolution video, video encoding methods and techniques are ubiquitous in modern technology. Video codecs typically include electronic circuits or software that compress or decompress digital video and are continually being improved to provide higher coding efficiency. The video codec converts uncompressed video into a compressed format or converts a compressed format into uncompressed video. There is a complex relationship between video quality, the amount of data used to represent the video (determined by the bit rate), the complexity of the encoding and decoding algorithms, susceptibility to data loss and errors, ease of editing, random access, and end-to-end delay (latency). The compression format typically conforms to a standard video compression specification, such as the High Efficiency Video Coding (HEVC) standard (also referred to as h.265 or MPEG-H part 2), the general video coding standard to be finalized, or other current and/or future video coding standards.

Sub-block based prediction was first introduced into the video coding standard by the High Efficiency Video Coding (HEVC) standard. With sub-block based prediction, a block, such as a Coding Unit (CU) or a Prediction Unit (PU), is divided into non-overlapping sub-blocks. Different sub-blocks may be assigned different motion information, such as reference indices or Motion Vectors (MVs), and Motion Compensation (MC) is performed separately for each sub-block. Fig. 1 shows an example of sub-block based prediction.

Embodiments of the disclosed techniques may be applied to existing video coding standards (e.g., HEVC, h.265) and future standards to improve runtime performance. Section headings are used in this document to enhance readability of the specification, and discussion or embodiments (and/or implementations) are not limited in any way to only the corresponding sections.

1. Example of Joint Exploration Model (JEM)

In some embodiments, reference software called Joint Exploration Model (JEM) is used to explore future video coding techniques. In JEM, sub-block based prediction is employed in several coding tools, such as affine prediction, Alternative Temporal Motion Vector Prediction (ATMVP), spatio-temporal motion vector prediction (STMVP), bi-directional optical flow (BIO), frame rate up-conversion (FRUC), Locally Adaptive Motion Vector Resolution (LAMVR), Overlapped Block Motion Compensation (OBMC), Local Illumination Compensation (LIC), and decoder-side motion vector refinement (DMVR).

1.1 example of affine prediction

In HEVC, only the translational motion model is applied to Motion Compensated Prediction (MCP). However, the camera and the object may have a variety of motions, such as zoom in/out, rotation, perspective motion, and/or other irregular motions. JEM, on the other hand, applies simplified affine transform motion compensated prediction. FIG. 2 shows a motion vector V from two control points ₀ And V ₁ An example of an affine motion field of block 200 is described. The Motion Vector Field (MVF) of block 200 may be described by:

as shown in fig. 2, (v) _0x ，v _0y ) Is to the leftMotion vector of upper corner control point, and (v) _1x ，v _1y ) Is the motion vector of the upper right hand corner control point. In order to simplify motion compensated prediction, sub-block based affine transform prediction may be applied. The subblock size M × N is derived as follows:

here, MvPre is the motion vector fractional accuracy (e.g., 1/16 in JEM). (v) of _2x ，v _2y ) Is the motion vector of the lower left control point, which is calculated according to equation (1). If desired, M and N can be adjusted downward to be divisors of w and h, respectively.

Fig. 3 shows an example of affine MVF for each sub-block of block 300. To derive the motion vector for each M × N sub-block, the motion vector for the center sample of each sub-block may be calculated according to equation (1) and rounded to the motion vector fractional accuracy (e.g., 1/16 in JEM). A motion compensated interpolation filter may then be applied to generate a prediction for each sub-block using the derived motion vectors. After MCP, the high accuracy motion vector of each sub-block is rounded and saved to the same accuracy as the normal motion vector.

In JEM, there are two affine motion patterns: AF _ INTER mode and AF _ MERGE mode. For CUs with width and height greater than 8, the AF _ INTER mode may be applied. An affine flag at the CU level is signaled in the bitstream to indicate whether AF _ INTER mode is used. In AF _ INTER mode, neighboring block construction is used with motion vector pair { (v) ₀ ,v ₁ )|v ₀ ＝{v _A ,v _B ,v _c },v ₁ ＝{v _D ,v _E } of the candidate list.

Fig. 4 shows an example of Motion Vector Prediction (MVP) for a block 400 in the AF _ INTER mode. As shown in fig. 4, v is selected from the motion vectors of sub-block A, B or C ₀ . The motion vectors from the neighboring blocks may be scaled according to the reference list. The reference Picture Order Count (POC) of the neighboring block, the reference POC of the current CU, and the POC of the current CU may also be based on a correlation between the POC of the current CU and the reference POC of the neighboring blockThe motion vectors are scaled. Selecting v from adjacent sub-blocks D and E ₁ The method of (3) is similar. If the number of candidate lists is less than 2, the list is populated by pairs of motion vectors that are constructed by duplicating each Advanced Motion Vector Prediction (AMVP) candidate. When the candidate list is greater than 2, the candidates may first be filtered according to neighboring motion vectors (e.g., based on the similarity of two motion vectors in the candidates). In some embodiments, the first two candidates are retained. In some embodiments, a Rate Distortion (RD) cost check is used to determine which motion vector pair candidate to select as the Control Point Motion Vector Predictor (CPMVP) for the current CU. An index indicating the position of the CPMVP in the candidate list may be signaled in the bitstream. After determining the CPMVP of the current affine CU, affine motion estimation is applied and Control Point Motion Vectors (CPMVs) are found. The difference between CPMV and CPMVP is then signaled in the bitstream.

When a CU is applied in AF _ MERGE mode, it obtains the first block encoded in affine mode from the valid neighboring reconstructed blocks. Fig. 5A shows an example of the selection order of candidate blocks of the current CU 500. As shown in fig. 5A, the selection order may be from left (501), top (502), top right (503), bottom left (504), to top left (505) of the current CU 500. Fig. 5B shows another example of a candidate block of the current CU500 in AF _ MERGE mode. If the neighboring lower left block 501 is encoded in affine mode, as shown in fig. 5B, then the motion vectors v for the upper left, upper right and lower left corner of the CU containing sub-block 501 are derived ₂ 、v ₃ And v ₄ . Based on v ₂ 、v ₃ And v ₄ Calculating motion vector v of the top left corner on current CU500 ₀ . The motion vector v at the top right of the current CU can be calculated accordingly ₁ 。

Calculating the CPMV v of the current CU in accordance with the affine motion model in equation (1) ₀ And v ₁ Thereafter, the MVF of the current CU may be generated. To identify whether the current CU is encoded in AF _ MERGE mode, an affine flag may be signaled in the bitstream when there is at least one neighboring block encoded in affine mode.

In JEM, the non-Merge affine mode can only be used if the width and height of the current block are both greater than 8; the Merge affine mode can be used only when the area (i.e., width x height) of the current block is not less than 64.

2. Examples of existing methods for sub-block based implementations

In some prior implementations, the size of the sub-blocks (e.g., 4 × 4 in JEM) is designed primarily for the luma component. For example, in JEM, the size of the sub-block is for a block with 4: 2: 2 x 2 chroma components in 0 format, and for a chroma component having a 4: 2: 2 x 4 chrominance components in 2 format. The small size of the sub-blocks requires higher bandwidth requirements. FIG. 6 shows a JEM having 4: 2: example of sub-blocks of 16 x 16 blocks (8 x 8 for Cb/Cr) of different components of the 0 format.

In other prior implementations, in some sub-block based tools (e.g., affine prediction in JEM), the MV of each sub-block is calculated independently for each component using the affine model shown in equation (1), which may result in misalignment of the motion vector between the luma and chroma components.

In other existing implementations, in some sub-block based tools (e.g., affine prediction), the usage constraints are different for both the Merge mode and the non-Merge inter mode (also referred to as AMVP mode, or normal inter mode), which needs to be unified.

3. Exemplary method for subblock-based prediction in video coding

The sub-block based prediction method includes unifying constraints for Merge affine mode and non-Merge affine mode. The use of sub-block based prediction to improve video coding efficiency and enhance existing and future video coding standards is set forth in the examples described below for the various embodiments.

Example 1.The Merge affine mode and the non-Merge affine mode are allowed or not allowed under the same block size constraint.

(a) The block size constraint depends on the width and height compared to one or two thresholds. For example, if the width and height of the current block are both greater than M (e.g., M equals 8), or the width is greater than M0 and the height is greater than M1 (e.g., M0 equals 8 and M1 equals 4), then Merge affine mode and non-Merge affine mode are allowed; otherwise, Merge affine mode and non-Merge affine mode are not allowed. In another example, if the width and height of the current block are both greater than M (e.g., M equals 16), then both a Merge affine mode and a non-Merge affine mode are allowed; otherwise, Merge affine mode and non-Merge affine mode are not allowed.

(b) The block size constraint depends on the total number of samples within one block (i.e., the area width x height). In one example, if the area (i.e., width x height) of the current block is not less than N (e.g., N is equal to 64), then both the Merge affine mode and the non-Merge affine mode are allowed; otherwise, Merge affine mode and non-Merge affine mode are not allowed.

(c) For the Merge affine mode, it can be an explicit mode that signals a flag as in JEM, or it can be an implicit mode that does not signal a flag as in other embodiments. In the latter case, if the Merge affine mode is not allowed, the affine Merge candidates are not put into the unified Merge candidate list.

(d) For non-Merge affine modes, when affine is not allowed according to the above rules, the signaling of the indication of affine mode is skipped.

The above examples may be incorporated in the context of methods described below (e.g., methods 700 and 750), which

methods

700 and 750 may be implemented at a video encoder and a video decoder, respectively.

Fig. 7A shows a flow diagram of an exemplary method for video processing. The method 700 includes determining a block size constraint for a video block in a video frame at operation 710. At operation 720, a determination is made as to whether a Merge affine mode and a non-Merge affine mode are allowed for the video block based on the block size constraint. At operation 730, a bit stream representation of the video block is generated based on the determination.

In some embodiments, generating the bitstream representation comprises encoding the video block using a Merge affine mode by including an indication of the Merge affine mode in the bitstream representation. In some embodiments, generating the bitstream representation comprises encoding the video block using the non-Merge affine mode by including an indication of the non-Merge affine mode in the bitstream representation. In some embodiments, generating the bitstream representation comprises encoding the video block using the Merge affine mode by omitting an indication of the Merge affine mode in the bitstream representation to implicitly indicate the Merge affine mode. In some embodiments, generating the bitstream representation comprises encoding the video block using the non-Merge affine mode by omitting the indication of the non-Merge affine mode in the bitstream representation to implicitly indicate the non-Merge affine mode.

In some embodiments, generating the bitstream representation comprises generating the bitstream representation from the video block such that the bitstream omits an explicit indication of the Merge affine mode due to determining that the block size constraint does not allow the Merge affine mode. In some embodiments, generating the bitstream representation comprises generating the bitstream representation from the video block such that the bitstream omits an explicit indication of the non-Merge affine mode due to determining that the block size constraint does not allow the non-Merge affine mode to be passed. In some embodiments, the block size constraints for the video blocks include a height of the video blocks and a width of the video blocks, both of which are greater than a common threshold. In some embodiments, the common threshold is eight, and the Merge affine mode and the non-Merge affine mode are allowed in response to the height of the video block and the width of the video block being greater than the common threshold. In some embodiments, the common threshold is 16, and the Merge affine mode and the non-Merge affine mode are allowed in response to the height of the video block and the width of the video block being greater than the common threshold.

In some embodiments, the block size constraints for the video blocks include a height of the video block being greater than a first threshold and a width of the video block being greater than a second threshold, and the first threshold and the second threshold being different. In some embodiments, the block size constraints for the video blocks include a height of the video blocks and a width of the video blocks, and a product of the height and the width is greater than a threshold.

Fig. 7B illustrates another flow diagram of an exemplary method for motion compensation. The method 750 includes determining block size constraints for video blocks in a video frame at operation 760. At operation 770, a determination is made as to whether the video block allows for the Merge affine mode and the non-Merge affine mode based on the block size constraint. In operation 780, a video block is generated from the bit stream representation of the video block based on the determining.

In some embodiments, the bitstream representation includes an indication of a Merge affine mode, and the video block is generated using the Merge affine mode by parsing the bitstream representation according to the indication. In some embodiments, the bitstream representation includes an indication of a non-Merge affine mode, and the video chunk is generated using the non-Merge affine mode by parsing the bitstream representation according to the indication. In some embodiments, the bitstream representation omits an indication of the Merge affine mode, and wherein the video block is generated using the Merge affine mode by parsing the bitstream representation without the indication of the Merge affine mode. In some embodiments, the bitstream representation omits the indication of the non-Merge affine mode and the video block is generated using the non-Merge affine mode by parsing the bitstream representation without the indication of the non-Merge affine mode.

In some embodiments, the block size constraints for the video blocks include heights of the video blocks and widths of the video blocks, the heights and widths being greater than a common threshold. In some embodiments, the block size constraints for the video blocks include that the height of the video blocks is greater than a first threshold and the width of the video blocks is greater than a second threshold, and the first and second thresholds are different. In some embodiments, the block size constraints for the video blocks include a height of the video blocks and a width of the video blocks, and a product of the height and the width is greater than a threshold.

Methods

700 and 750 described in the context of fig. 7A and 7B, respectively, may also include processing the block of video data based on a motion compensation algorithm. In some embodiments, the motion compensation algorithm may be affine motion compensated prediction, Alternative Temporal Motion Vector Prediction (ATMVP), spatio-temporal motion vector prediction (STMVP), Pattern Matched Motion Vector Derivation (PMMVD), bi-directional optical flow (BIO), or decoder-side motion vector refinement (DMVR).

In a representative aspect, the methods 700 and/or 750 described above are embodied in processor executable code and stored in a computer readable program medium. In yet another representative aspect, an apparatus configured or operable to perform the above-described method is disclosed. The device may include a processor programmed to implement the methods 700 and/or 750 described above. In yet another representative aspect, a video encoder device may implement method 700. In yet another representative aspect, a video decoder device may implement method 750.

4. Example embodiments of the disclosed technology

Fig. 8 is a block diagram illustrating an example of an architecture of a computer system or other control device that may be used to implement various portions of the presently disclosed technology, including (but not limited to) methods 700 and/or 750. In fig. 8, a computer system 800 includes one or more processors 805 and memory 810 connected via an interconnect 825. Interconnect 825 may represent any one or more separate physical buses, point-to-point connections, or both, connected through appropriate bridges, adapters, or controllers. Thus, interconnect 825 may comprise, for example, a system bus, a Peripheral Component Interconnect (PCI) bus, a HyperTransport or Industry Standard Architecture (ISA) bus, a Small Computer System Interface (SCSI) bus, a Universal Serial Bus (USB), an IIC (I2C) bus, or an Institute of Electrical and Electronics Engineers (IEEE) standard 674 bus, also sometimes referred to as a "Firewire".

The processor(s) 805 may include a Central Processing Unit (CPU) to control overall operation of, for example, a host computer. In certain embodiments, the processor(s) 805 achieve this by executing software or firmware stored in memory 810. The processor(s) 805 may be or include one or more programmable general-purpose or special-purpose microprocessors, Digital Signal Processors (DSPs), programmable controllers, Application Specific Integrated Circuits (ASICs), Programmable Logic Devices (PLDs), or the like, or a combination of such devices.

The memory 810 may be or include the main memory of a computer system. Memory 810 represents any suitable form of Random Access Memory (RAM), Read Only Memory (ROM), flash memory, etc., or combination of such devices. In use, the memory 810 may contain, among other things, a set of machine instructions that, when executed by the processor 805, cause the processor 805 to perform operations to implement embodiments of the presently disclosed technology.

An (optional) network adapter 815 is also connected to the processor(s) 805 via the interconnect 825. The network adapter 815 provides the computer system 800 with the ability to communicate with remote devices, such as storage clients and/or other storage servers, and the network adapter 815 may be, for example, an ethernet adapter or a fibre channel adapter.

Fig. 9 illustrates a block diagram of an example embodiment of a mobile device 900, which mobile device 900 may be used to implement various portions of the presently disclosed technology, including (but not limited to) methods 700 and/or 750. The mobile device 900 may be a laptop computer, a smart phone, a tablet computer, a camcorder, or other device capable of processing video. The mobile device 900 includes a processor or controller 901 for processing data and memory 902 in communication with the processor 901 for storing and/or buffering data. For example, the processor 901 may include a Central Processing Unit (CPU) or a microcontroller unit (MCU). In some implementations, the processor 901 may include a Field Programmable Gate Array (FPGA). In some implementations, the mobile device 900 includes or communicates with a Graphics Processing Unit (GPU), a Video Processing Unit (VPU), and/or a wireless communication unit for various visual and/or communication data processing functions of a smartphone device. For example, the memory 902 may include and store processor-executable code that, when executed by the processor 901, configures the mobile device 900 to perform various operations, such as receiving information, commands, and/or data, processing information and data, and transmitting or providing processed information/data to another device, such as an actuator or external display.

To support various functions of the mobile device 900, the memory 902 can store information and data such as instructions, software, values, images, and other data that are processed or referenced by the processor 901. For example, various types of Random Access Memory (RAM) devices, Read Only Memory (ROM) devices, flash memory devices, and other suitable storage media may be used to implement the storage functionality of memory 902. In some implementations, the mobile device 900 includes an input/output (I/O) unit 903 to connect the processor 901 and/or the memory 902 to other modules, units, or devices. For example, the I/O unit 903 may be connected with the processor 901 and the memory 902 to utilize various types of wireless interfaces compatible with general data communication standards, for example, between one or more computers in the cloud and a user device. In some implementations, the mobile device 900 can connect with other devices using a wired connection via the I/O unit 903. The mobile device 900 may also be connected to other external interfaces, such as data storage and/or a visual or audio display device 904, to retrieve and transfer data and information which may be processed by the processor, stored in memory, or presented on an output unit of the display device 904 or an external device. For example, display device 904 may display a video frame that includes blocks (CU, PU, or TU) that apply intra block copying based on whether the blocks are encoded using a motion compensation algorithm and in accordance with the disclosed techniques.

In some embodiments, a video encoder device or a decoder device may implement the method of sub-block based prediction as described herein for video encoding or decoding. Various features of the method may be similar to

methods

700 or 750 described above.

In some embodiments, the video encoding and/or decoding methods may be implemented using decoding devices implemented on the hardware platforms described with respect to fig. 8 and 9.

From the foregoing it will be appreciated that specific embodiments of the presently disclosed technology have been described herein for purposes of illustration, but that various modifications may be made without deviating from the scope of the invention. Accordingly, the presently disclosed technology is not limited, except as by the appended claims.

Embodiments of the subject matter and the functional operations described in this patent document can be implemented in various systems, digital electronic circuitry, or computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term "data processing unit" or "data processing apparatus" includes all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. The computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

The specification, together with the drawings, should be considered exemplary only, with the examples being meant as examples. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. In addition, the use of "or" is intended to include "and/or" unless the context clearly indicates otherwise.

While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only a few embodiments and examples are described and other embodiments, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method for video processing, comprising:

determining whether a Merge affine mode and a non-Merge affine mode are allowed for a video block in a video frame based on a uniform block size constraint; and

generating the video block from a bit stream of the video block based on making the determination, wherein,

determining that the non-Merge affine mode and the Merge affine mode are not allowed in response to the height of the video block and the width of the video block being less than 16, thus omitting an indication of the non-Merge affine mode and the Merge affine mode in the bitstream,

generating the video block by parsing the bitstream without the indication of the non-Merge affine mode and the Merge affine mode and without affine mode in the bitstream if the indication of the non-Merge affine mode and the Merge affine mode is omitted.

2. The method of claim 1, wherein the first and second light sources are selected from the group consisting of,

wherein, in case it is determined that the Merge affine mode is allowed, including an indication of the Merge affine mode in the bitstream, by parsing the bitstream according to the indication and generating the video block using the Merge affine mode.

3. The method of claim 1, wherein the first and second light sources are selected from the group consisting of,

wherein, in a case where it is determined that the non-Merge affine mode is allowed, in a case where an indication of the non-Merge affine mode is included in the bitstream, the video block is generated by parsing the bitstream according to the indication and using the non-Merge affine mode.

4. The method of claim 1, wherein the block size constraints of the video block comprise a height of the video block and a width of the video block, the height and the width being greater than a common threshold.

5. The method of claim 4, wherein the first and second light sources are selected from the group consisting of,

wherein the common threshold is 16, and

wherein the Merge affine mode and the non-Merge affine mode are allowed in response to the height of the video block and the width of the video block being greater than the common threshold.

6. The method of claim 1, wherein the first and second light sources are selected from the group consisting of,

wherein the block size constraint for the video block comprises a height of the video block being greater than a first threshold and a width of the video block being greater than a second threshold, and

wherein the first threshold and the second threshold are different.

7. The method of claim 1, wherein the first and second light sources are selected from the group consisting of,

wherein the block size constraints for the video block include a height of the video block and a width of the video block, and

wherein a product of the height and the width is greater than a threshold.

8. A method for video processing, comprising:

generating a bitstream for the video block based on the determining, wherein,

generating the bitstream from the video block such that the non-Merge affine mode and the Merge affine mode are determined not to be allowed in response to the height of the video block and the width of the video block being less than 16, thereby omitting an indication of the non-Merge affine mode and the Merge affine mode in the bitstream,

encoding the video block by not using an affine mode without the indication of the non-Merge affine mode and the Merge affine mode if the indication of the non-Merge affine mode and the Merge affine mode is omitted in the bitstream.

9. The method of claim 8, wherein, if it is determined that the Merge affine mode is allowed, including an indication of the Merge affine mode in the bitstream, the video block is encoded using the Merge affine mode.

10. The method of claim 8, wherein, if the non-Merge affine mode is determined to be allowed, including an indication of the non-Merge affine mode in the bitstream, the video block is encoded using the non-Merge affine mode.

11. The method of claim 8, wherein the block size constraints of the video block comprise a height of the video block and a width of the video block, the height and the width both being greater than a common threshold.

12. The method of claim 11, wherein the first and second light sources are selected from the group consisting of,

wherein the common threshold is 16, and

13. The method of claim 8, wherein the first and second light sources are selected from the group consisting of,

wherein the first threshold and the second threshold are different.

14. The method of claim 8, wherein the first and second light sources are selected from the group consisting of,

wherein a product of the height and the width is greater than a threshold.

15. A video decoding apparatus, comprising: a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of claims 1 to 7.

16. A video encoding device, comprising: a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of claims 8-14.

17. An apparatus for video processing, comprising:

the determining module is used for determining whether the Merge affine mode and the non-Merge affine mode are allowed for the video block in the video frame based on the unified block size constraint; and

a generation module to generate the video block from a bitstream of the video block based on the determining, wherein,

in response to the height of the video block and the width of the video block being less than 16, determining that the non-Merge affine mode and the Merge affine mode are not allowed, thus omitting an indication of the non-Merge affine mode and the Merge affine mode in the bitstream,

18. An apparatus for video processing, comprising:

the determining module is used for determining whether the Merge affine mode and the non-Merge affine mode are allowed for the video blocks in the video frame based on the uniform block size constraint; and

a generation module to generate a bitstream for the video block based on the determining, wherein,

generating the bitstream from the video block such that the non-Merge affine mode and the Merge affine mode are determined not to be allowed in response to a height of the video block and a width of the video block being less than 16, thereby omitting an indication of the non-Merge affine mode and the Merge affine mode in the bitstream,

19. A non-transitory computer-readable storage medium for storing program code, wherein the program code, when executed by a computer, the computer implements the method of any of claims 1-14.

20. A method for storing a video bitstream, comprising:

determining whether a Merge affine mode and a non-Merge affine mode are allowed for a video block in a video frame based on a uniform block size constraint;

based on the determining, generating a bitstream for the video block; and

storing the bitstream in a non-transitory computer-readable storage medium, wherein,