RU2777392C1 - Method and apparatus for encoding video data - Google Patents

Abstract

FIELD: video data encoding.

SUBSTANCE: invention relates to the field of video data encoding. Methods and devices for encoding/decoding video data are proposed. The device for decoding video data includes a processing circuit that checks a condition for obtaining a specific prediction mode flag for the current block before parsing the specific prediction mode flag for the current block from the encoded video data bitstream. This particular prediction mode is one of a plurality of inter image prediction modes. When the flag inference condition indicates that the flag can be inferred, the processing circuit obtains the flag inference without parsing the flag from the encoded video bitstream. When the flag inference condition indicates that the flag inference is undefined, the processing circuit analyzes the flag from the encoded video bitstream. Then, the processing circuit reconstructs the current block in accordance with said specific prediction mode when the flag indicates the application of said specific prediction mode to the current block.

EFFECT: improving encoding efficiency.

10 cl, 10 dwg, 4 tbl

Description

INCLUSION BY LINK

[0001] The present application claims priority under U.S. Patent Application No. 16/865,052 "Method and Apparatus for Encoding Video Data", filed May 1, 2020, which claimed priority under Provisional U.S. Patent Application No. 62/849,105 " Merge Mode Signaling, filed May 16, 2019. The contents of previous applications are incorporated herein by reference in their entirety.

FIELD OF TECHNOLOGY TO WHICH THE INVENTION RELATES

[0002] This application describes embodiments of the invention generally related to video coding.

BACKGROUND OF THE INVENTION

[0003] The description of the prior art is given here to represent the overall context of the invention. The work of the authors of the invention, to the extent that it is described in this section, as well as aspects of the description that cannot be qualified as state of the art at the time of filing, neither expressly nor impliedly recognized as the state of the art for the present invention.

[0004] Encoding and decoding of video data may be performed using inter image prediction with motion compensation. Uncompressed digital video may include a sequence of images each having a spatial size of, for example, 1920×1080 luma samples and their associated chrominance samples. The image sequence may have a fixed or variable image rate (informally also referred to as frame rate), such as 60 images per second, or 60 Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (resolution of 1920x1080 luminance samples at 60Hz frame rate) requires about 1.5Gbps of bandwidth. An hour of such video requires more than 600 GB of storage.

[0005] One goal of encoding and decoding video data may be to reduce redundancy in the input video signal by compressing. Compression can help to alleviate the aforementioned bandwidth or storage requirements, in some cases by two orders of magnitude or more. You can use both lossless and lossy compression, as well as a combination of both. Lossless compression refers to techniques for reconstructing an exact copy of the original signal from the compressed original signal. When lossy compression is used, the reconstructed signal may not be identical to the original signal, but the discrepancy between the original and reconstructed signals is small enough that the reconstructed signal can be used for the intended application. Lossy compression is widely used for video data. The amount of distortion allowed depends on the application; for example, users of some custom streaming applications can tolerate higher distortion than users of broadcast TV applications. The achievable compression ratio may reflect that a higher allowed/allowable distortion may result in a higher compression ratio.

[0006] Motion compensation may be a lossy compression technique and may refer to techniques in which a block of sample data from a previously reconstructed image or a portion thereof (reference image) after a spatial shift in the direction indicated by a motion vector (MV, motion vector) is used to predict a newly reconstructed image or part of an image. In some cases, the reference image may be the same as the image currently being reconstructed. The motion vectors may have two dimensions X and Y, or three dimensions, with the third dimension indicating the reference image used (the latter, indirectly, may be a time dimension).

[0007] In some video data compression techniques, a motion vector applicable to a certain region of sample data may be predicted based on other motion vectors, such as those associated with and preceding another region of sample data adjacent to the reconstructed region. movements in decoding order. This can significantly reduce the amount of data required for motion vector encoding, which eliminates redundancy and increases the compression ratio. Motion vector prediction can work efficiently, for example, because when encoding the input video signal received from the camera (known as natural video), there is a statistical probability that areas larger than the area to which one motion vector is applied move in a similar direction and therefore, in some cases, can be predicted using a similar motion vector derived from the motion vectors of the neighboring area. This results in the motion vector found for a given region being similar or identical to the motion vector predicted from the surrounding motion vectors and, in turn, can be represented after entropy encoding with fewer bits than in the case of direct encoding. motion vector coding. In some cases, motion vector prediction may be an example of lossless compression of a signal (namely, motion vectors) derived from the original signal (namely, a stream of samples). In other cases, the motion vector prediction itself may be lossy, for example, due to roundoff errors when computing a predictor based on multiple surrounding motion vectors.

[0008] Various motion vector prediction mechanisms are described in the H.265/HEVC standard (ITU-T Rec. H.265, High Efficiency Video Coding, December 2016). Of the many motion vector prediction mechanisms that the H.265 standard offers, a method hereinafter referred to as "spatial fusion" is described.

[0009] As shown in FIG. 1, the current block (101) contains the samples found by the encoder during the motion search process to be predictable based on a previous block of the same size that is spatially shifted. Instead of encoding the motion vector directly, this motion vector may be derived from the metadata associated with one or more reference pictures, e.g. based on the most recent (in decoding order) reference picture using a motion vector associated with any of the five surrounding samples labeled A0, A1 and B0, B1, B2 (from 102 to 106, respectively). In H.265, motion vector prediction may use predictors based on the same reference picture as used for the neighboring block.

SUMMARY OF THE INVENTION

[0010] Aspects of the invention provide methods and apparatus for encoding/decoding video data. In some examples, a device for decoding video data includes a receiving circuit and a processing circuit. The processing circuit checks a condition for deriving the specific prediction mode flag for the current block before parsing the specific prediction mode flag for the current block from the encoded video bitstream. This particular prediction mode is one of a plurality of inter image prediction modes. When the flag inference condition indicates that the flag can be inferred, the processing circuit obtains the flag inference without parsing the flag from the encoded video bitstream. When the flag inference condition indicates that the flag inference is undefined, the processing circuit performs flag analysis from the encoded video bitstream. Then, the processing circuit reconstructs the current block in accordance with said specific prediction mode when the flag indicates the application of said specific prediction mode to the current block.

[0011] In some embodiments, the processing circuit checks a flag inference condition based on the validity of one or more other modes from a plurality of inter image prediction modes. In an embodiment of the invention, the validity of another mode is determined based on information about the size of the current block.

[0012] In some embodiments, the flag is one of a conventional merge/skip conventional mode flag, an MMVD motion vector difference merge (MMVD) flag, a sub-block merge/skip mode sub-block flag, a combined intra-inter prediction (CIIP) flag ) CIIP mode, triangulation mode triangle flag.

[0013] In an embodiment of the invention, a particular prediction mode is ahead of one or more modes in a tree structure for deciding on a prediction mode.

[0014] In some examples, the flag inference condition is a combination of the validity of a particular prediction mode and the validity of other modes that are behind the particular prediction modes in the tree structure to decide on the prediction mode. In one example, the flag inference condition indicates that the flag can be inferred when the validity of a particular prediction mode indicates that the particular prediction mode is not allowed. In another example, the flag inference condition indicates that the flag can be inferred when the validity of the other modes indicates that none of the other modes are allowed.

[0015] In some embodiments of the invention, the validity of another mode is determined based on at least one of the following: sequence parameter set (SPS, sequence parameter set), picture parameter set (PPS, picture parameter set), slice header, tile information (tile ) and tile group information.

[0016] Aspects of the invention also provide a computer-readable medium storing instructions that, when executed by a video decoding computer, causes the computer to perform a video decoding method.

BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Other features, principles and various advantages of the present invention will become apparent from the following detailed description and the accompanying drawings.

[0018] FIG. 1 is a schematic illustration of the current block and its surrounding candidates for spatial merging in one embodiment of the invention.

[0019] FIG. 2 is a schematic illustration of a simplified block diagram of a communication system (200) in accordance with an embodiment of the invention.

[0020] FIG. 3 is a schematic illustration of a simplified block diagram of a communications system (300) in accordance with an embodiment of the invention.

[0021] FIG. 4 is a schematic illustration of a simplified block diagram of a decoder in accordance with an embodiment of the invention.

[0022] FIG. 5 is a schematic illustration of a simplified block diagram of an encoder in accordance with an embodiment of the invention.

[0023] FIG. 6 shows a block diagram of an encoder in accordance with another embodiment of the invention.

[0024] FIG. 7 shows a block diagram of a decoder in accordance with another embodiment of the invention.

[0025] FIG. 8 shows a flowchart for signaling (parsing) an encoder (or decoder) of merge mode flags.

[0026] FIG. 9 shows a flow diagram describing an example of a process according to some embodiments of the invention.

[0027] FIG. 10 is a schematic illustration of a computer system in accordance with an embodiment of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS

[0028] FIG. 2 shows a simplified block diagram of a communication system (200) according to an embodiment of the present invention. The communication system (200) includes a plurality of terminal devices that can communicate with each other, for example, via a network (250). For example, the communication system (200) includes a first pair of terminals (210) and (220) interconnected via a network (250). In the example shown in FIG. 2, the first pair of terminals (210) and (220) performs a unidirectional data transfer. For example, a terminal device (210) may encode video data (eg, a video stream captured by a terminal device (210)) for transmission to another terminal device (220) via a network (250). The encoded video data may be transmitted as one or more encoded video bit streams. The terminal device (220) may receive the encoded video data from the network (250), decode the encoded video data to recover the video images and display them according to the recovered video data. One-way communication may be inherent in media service applications and the like.

[0029] In another example, the communication system (200) includes a second pair of terminals (230) and (240) that carry out bi-directional transmission of encoded video data occurring, for example, during a video conference. For bidirectional data transmission, for example, each terminal of terminals (230) and (240) may encode video data (eg, a video stream captured by the terminal) for transmission to the other terminal of terminals (230) and (240) via network (250). Each terminal of terminals (230) and (240) may also receive encoded video data transmitted by another terminal from terminals (230) and (240) and may decode the encoded video data to reconstruct video images and display video images on an available display device according to recovered video data.

[0030] In the example shown in FIG. 2, terminal devices (210), (220), (230), and (240) can be illustrated as servers, personal computers, and smartphones, but this does not limit the principles of the present invention. Embodiments of the present invention find application in laptop computers, tablet computers, media players and/or dedicated video conferencing equipment. Network (250) represents any number of networks that carry encoded video data between terminals (210), (220), (230) and (240), including, for example, wired and/or wireless communication networks. The communication network (250) allows data to be exchanged in a circuit-switched and/or packet-switched mode. Examples of networks include telecommunications networks, local area networks, wide area networks and/or the Internet. For the present discussion, the architecture and topology of the network (250) may not be relevant to the present invention unless specifically noted below.

[0031] In FIG. 3 shows the placement of a video encoder and a video decoder in a streaming environment as an example of the application of the present invention. The present invention is equally applicable to other video processing applications including, for example, videoconferencing, digital television, storage of compressed video on digital media such as CD, DVD, memory card, and the like.

[0032] The streaming system may include a capture subsystem (313), which may include a source (301) of video data, such as a digital camera, creating, for example, an uncompressed video stream (302). For example, the video stream (302) includes samples taken by a digital camera. The video stream (302), shown in bold to highlight the large amount of data compared to the encoded video data (304) (or encoded video bitstreams), may be processed by an electronic device (320) that includes a video encoder (303) connected to the source ( 301) video data. The video encoder (303) may include hardware, software, or a combination thereof to provide or implement aspects of the present invention, as described in more detail below. The encoded video data (304) (or encoded video bitstream (304)), shown as a thin line to emphasize the smaller amount of data compared to the video stream (302), may be stored on the streaming server (305) for future use. One or more streaming client subsystems, such as client subsystems (306) and (308) in FIG. 3 may access the streaming server (305) to retrieve copies (307) and (309) of the encoded video data (304). The client subsystem (306) may contain a video decoder (310), for example, in an electronic device (330). The video decoder (310) decodes the incoming copy (307) of the encoded video data and creates an outgoing video stream (311) that can be rendered on a display (312) (eg, a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (304), (307), and (309) (eg, video bitstreams) may be encoded according to one or another video coding/compression standard. Examples of these standards include ITU-T Rec. H.265. For example, a video coding standard is being developed officially called Versatile Video Coding (VVC). The present invention can be used in the context of VVC.

[0033] Note that the electronic devices (320) and (330) may include other components (not shown). For example, the electronic device (320) may include a video decoder (not shown) and the electronic device (330) may also include a video encoder (not shown).

[0034] FIG. 4 is a block diagram of a video decoder (410) according to an embodiment of the present invention. The video decoder (410) may be included in the electronic device (430). The electronic device (430) may include a receiver (431) (eg, receiver circuits). Video decoder (410) may be used instead of video decoder (310) in the example shown in FIG. 3.

[0035] The receiver (431) may receive one or more encoded video sequences for decoding by video decoder (410), in the same or another embodiment of the invention, one encoded video sequence at a time, with the decoding of each encoded video sequence independent of other encoded video sequences. The encoded video sequence may be received from a channel (401), which may be a hardware/software link to a storage device where the encoded video data is stored. The receiver (431) may receive encoded video data with other data, such as encoded audio data and/or auxiliary data streams, which may be redirected to their respective utilizing entities (not shown). The receiver (431) may separate the encoded video sequence from other data. To combat network jitter, a buffer memory (415) can be connected between the receiver (431) and the entropy decoder/analyzer (420) (hereinafter "analyzer (420)"). In some applications, the buffer memory (415) is included in the video decoder (410). In others, it may not be part of the video decoder (410) (not shown). In other embodiments, there may be a buffer memory outside the video decoder (410) (not shown), for example, to combat network jitter, in addition to another buffer memory (415) within the video decoder (410), for example, to control playback time. When the receiver (431) receives data from a storage/redirector with sufficient bandwidth and controllability, or from an isosynchronous network, the buffer memory (415) may not be needed or may be small. For use in best effort packet networks, such as the Internet, buffer memory (415) may be required, may be relatively large, may have a predominantly adaptive size, and may at least partially be implemented in an operating system or similar elements (not shown) outside video decoder (410).

[0036] The video decoder (410) may include an analyzer (420) for reconstructing the symbols (421) from the encoded video sequence. The categories of these symbols include information used to control the operation of the video decoder (410) and possibly information to control a rendering device, such as a rendering device (412) (e.g., a display screen) that is not an integral part of the electronic device (430) but may be connected to an electronic device (430) as shown in FIG. 4. The control information for the rendering device(s) may be Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not shown). The analyzer (420) may analyze/entropy decode the received encoded video sequence. Video sequence coding may be in accordance with a video coding technology or standard and may follow various principles, including variable run length coding, Huffman coding, arithmetic coding with or without context sensitivity, and so on. The analyzer (420) can extract from the encoded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. Subgroups may include groups of pictures (GOP, Groups of Pictures), images, tiles, slices, macroblocks, coding units (CU, Coding Units), blocks, transformation units (TU, Transform Units), prediction units (PU, Prediction Units) and etc. The analyzer (420) can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0037] The analyzer (420) may perform an entropy decoding/analysis operation of the video sequence received from the buffer memory (415) to create symbols (421).

[0038] Several different modules may be used for symbol reconstruction (421) depending on the type of encoded video image or parts thereof (eg, intra- and outer-coded images, intra- and outer-coded blocks) and other factors. Which modules are used and how can be determined by the subgroup management information obtained from the encoded video sequence by the analyzer (420). The flow of such subgroup control information between the analyzer (420) and the modules is not shown hereinafter for simplicity.

[0039] In addition to the previously mentioned functional blocks, the video decoder (410) can be fundamentally divided into several functional modules, as described below. In practical implementation under commercial constraints, many of these modules interact closely with each other and can, at least partially, be built into one another. However, for the purpose of describing the present invention, a fundamental division into the following functional modules is appropriate.

[0040] The first module is the scaling/inverse transform module (451). The scaling/inverse transform module (451) receives the quantized transform coefficient as well as control information including the transform used, block size, quantization factor, quantization scaling matrices, etc. as character(s) (421) from the analyzer (420). The scaling/inverse transform module (451) can output blocks containing sample values that can be input to the aggregator (455).

[0041] In some cases, the output samples of the block (451) scaling / inverse transformation may refer to the block with intra coding; that is, to a block that does not use predicted information from previously reconstructed images, but may use predicted information from previously reconstructed portions of the current image. Such predicted information may be provided by an intra image prediction module (452). In some cases, the intra image prediction module (452) generates a block of the same size and shape as the block to be reconstructed using previously reconstructed environment information retrieved from the current image buffer (458). The current picture buffer (458) buffers, for example, a partially reconstructed current picture and/or a fully reconstructed current picture. The aggregator (455) in some cases adds, for each sample, the prediction information generated by the intra prediction module (452) to the output sample information provided by the scaling/inverse transform module (451).

[0042] In other cases, the output samples of the scaling / inverse transform module (451) may refer to an intra-coded block, possibly with motion compensation. In such a case, the motion compensation prediction module (453) may access the reference picture memory (457) to retrieve samples used for prediction. After motion compensation has been applied to the extracted samples in accordance with the symbols (421) associated with the block, these samples can be added by the aggregator (455) to the output signal of the scaling/inverse transform module (451) (in this case called residual samples or residual signal) to generation of output sample information. The addresses in the reference picture memory (457) from where the motion compensation prediction module (453) retrieves samples for prediction may be controlled by the motion vectors available to the motion compensation prediction module (453) in the form of symbols (421), which may have, for example , X, Y components, and reference image components. Motion compensation may also include interpolation of sample values retrieved from reference picture memory (457) when accurate sub-sample motion vectors, motion vector prediction mechanisms, etc. are used.

[0043] Various loop filtering methods can be applied to the output samples of the aggregator (455) in the loop filter module (456). Video compression technologies may include loop filter technologies driven by parameters included in the encoded video sequence (also referred to as the coded video bitstream) and available to the loop filter module (456) as symbols (421) from the analyzer (420), but may also be responsive to meta-information obtained during the decoding of the previous (in decoding order) parts of the encoded image or encoded video sequence, as well as to respond to previously reconstructed and contour filtered sample values.

[0044] The loop filter module (456) may output a stream of samples that may be fed to the renderer (412) as well as stored in the reference picture memory (457) for use in future inter-picture prediction.

[0045] Some encoded pictures, when fully reconstructed, can be used as reference pictures for future prediction. For example, when the encoded picture corresponding to the current picture is fully reconstructed and identified as a reference picture (e.g., by the analyzer (420)), the current picture buffer (458) may become part of the reference picture memory (457) and a fresh current picture buffer may be re-allocated. until the reconstruction of the next encoded picture starts.

[0046] The video decoder (410) may perform decoding operations according to a predetermined video data compression technology, such as ITU-T Rec. H.265. The encoded video sequence may follow the syntax given by the video compression technology or standard used, in the sense that the encoded video sequence may follow both the syntax of the video compression technology or standard and the profiles documented in the video compression technology or standard. In particular, a profile may select certain tools as tools available for use only under that profile, from among all the tools available in a video data compression technology or standard. It may also be necessary for consistency that the complexity of the encoded video sequence remains within the limits set by the level of technology or video compression standard. In some cases, the levels limit the maximum image size, the maximum frame rate, the maximum sampling rate for reconstruction (measured in megasamples per second, for example), the maximum reference image size, and so on. The limits set by the levels, in some cases, may be further limited by the specifications of the hypothetical reference decoder (HRD, Hypothetical Reference Decoder) and metadata for managing the HRD buffer signaled in the encoded video sequence.

[0047] According to an embodiment of the invention, the receiver (431) may receive additional (redundant) data with encoded video data. Additional data may be included as part of the encoded video sequence(s). The additional data may be used by the video decoder (410) to properly decode the data and/or more accurately reconstruct the original video data. The side data may be, for example, temporal enhancement levels, spatial enhancement levels or signal noise ratio (SNR) enhancement levels, redundant slices, redundant images, forward error correction codes, and so on.

[0048] FIG. 5 is a block diagram of a video encoder (503) according to an embodiment of the present invention. The video encoder (503) is included in the electronic device (520). The electronic device (520) includes a transmitter (540) (eg, a transmission circuit). Video encoder (503) may be used instead of video encoder (303) in the example shown in FIG. 3.

[0049] The video encoder (503) may receive video samples from a video data source (501) (which is not part of the electronic device (520) in the example shown in FIG. 5) that may capture the video image(s) to be encoding with a video encoder (503). In another example, the source (501) of video data is included in the electronic device (520).

[0050] The video data source (501) may provide the source video sequence to be encoded by the video encoder (503) in the form of a digital video data sample stream of any suitable bit depth (eg: 8 bits, 10 bits, 12 bits, ...), any color space (eg, BT.601 Y CrCB, RGB, …) and any suitable sampling structure (eg, Y CrCB 4:2:0, Y CrCB 4:4:4). In a media service system, the source (501) of video data may be a storage device where previously prepared video is stored. In a videoconferencing system, the video data source (501) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual images that give a sense of movement when viewed in sequence. The images themselves may be organized as a spatial array of pixels, where each pixel may contain one or more samples depending on the sampling structure used, color space, and so on. It is not difficult for a person skilled in the art to understand the relationship between pixels and samples. The following description is devoted to readings.

[0051] According to an embodiment of the invention, the video encoder (503) can encode and compress images of the original video sequence into the encoded video sequence (543) in real time or subject to any other time constraints imposed by the application. Setting the proper coding rate is one of the functions of the controller (550). In some embodiments of the invention, the controller (550) controls other functional modules, as described below, and is operatively connected to other functional modules. The connection is not shown for simplicity. The parameters set by the controller (550) may include parameters related to frequency adjustment (picture skip, quantizer, lambda value used in rate-distortion optimization methods, etc.), image size, organization of groups of images (GOP, group of pictures), maximum motion vector search area, etc. The controller (550) may have other suitable functions related to the video encoder (503) optimized for a particular system design.

[0052] In some embodiments of the invention, the video encoder (503) is configured to operate in a coding loop. As a very simplified description, for example, an encoding loop may include a source encoder (530) (for example, responsible for creating symbols, for example, a symbol stream based on the input image to be encoded and the reference image(s)) and (local ) decoder (533) built into the video encoder (503). The decoder (533) reconstructs the symbols to create sample data in a similar manner as a (remote) decoder would (because any compression between the symbols and the encoded video bitstream is lossless in the video compression technologies discussed here). The reconstructed sample stream (sample data) is entered into the reference picture memory (534). Since decoding the symbol stream produces bit-accurate results that are independent of the position of the (local or remote) decoder, the content of the reference picture memory (534) will also be the same bit-accuracy for the local encoder and the remote encoder. In other words, the prediction portion of the encoder "sees" as reference picture samples exactly the same sample values as the decoder would "see" if using prediction during decoding. This fundamental principle of reference picture synchronism (and eventually drift if synchronism cannot be maintained due to, for example, channel errors) is also used in some related fields of technology.

[0053] A "local" decoder (533) may operate in the same manner as a "remote" decoder, such as the video decoder (410) detailed above with reference to FIG. 4. However, again, referring to FIG. 4, since the symbols are available and the encoding/decoding of the symbols into the encoded video sequence by the entropy encoder (545) and parser (420) can be lossless, the entropy decoding portions of the video decoder (410) including the buffer memory (415) and parser (420) can be not fully implemented in the local decoder (533).

[0054] In doing so, it can be concluded that any decoding technology present in a decoder, with the exception of analysis/entropy decoding, must also necessarily be present in essentially identical functional form in the corresponding encoder. For this reason, the present invention is focused on the operation of the decoder. The description of the encoding technologies can be abbreviated as they are the inverse of the decoding technologies described in detail. Only in some areas a more detailed description is required, which is given below.

[0055] According to some examples, during operation, the source encoder (530) may perform motion-compensated predictive coding, in which an input picture is predictively encoded based on one or more previously encoded pictures from a video sequence, referred to as "reference pictures". Thus, the coding engine (532) encodes the differences between the pixel blocks of the input image and the pixel blocks of the reference image(s) that can be selected as the prediction reference(s) to the input image.

[0056] The local video decoder (533) may decode the encoded video data of pictures that can be specified as reference pictures based on the symbols generated by the source encoder (530). The operations of the encoding engine (532) may be predominantly lossy processes. When the encoded video data can be decoded in a video decoder (not shown in FIG. 5), the reconstructed video sequence can typically be a copy of the original video sequence with some errors. The local video decoder (533) duplicates the decoding processes that may be performed by the video decoder on the reference pictures and may store the reconstructed reference pictures in the reference picture cache (534). Thus, the video encoder (503) can locally store copies of the reconstructed reference pictures having the same content as the reconstructed reference pictures that would be received by the remote video decoder (in the absence of transmission errors).

[0057] The predictor (535) may perform prediction searches for the encoding engine (532). Thus, for a new image to be encoded, the predictor (535) may search the reference picture memory (534) for sample data (as pixel reference block candidates) or specific metadata such as reference picture motion vectors, block shapes, etc. .d., which can serve as a proper reference for predicting new images. The predictor (535) may operate on a "sample x pixel blocks" basis to find appropriate references for prediction. In some cases, according to the search results obtained by the predictor (535), the input picture may have prediction references extracted from the plurality of reference pictures stored in the reference picture memory (534).

[0058] The controller (550) can control the encoding operations of the source encoder (530), including, for example, setting the parameters as well as the subgroup parameters used to encode the video data.

[0059] The output signal of all the above functional modules can be entropy encoded in an entropy encoder (545). An entropy encoder (545) translates symbols generated by various functional units into an encoded video sequence by lossless symbol compression according to, for example, Huffman coding, variable run length coding, arithmetic coding, and so on.

[0060] The transmitter (540) may buffer the encoded video sequence(s) created by the entropy encoder (545) in preparation for transmission over a communication channel (560), which may be a hardware/software memory link where the encoded video data is stored. The transmitter (540) may combine the encoded video data from the video encoder (503) with other data to be transmitted, such as encoded audio data and/or auxiliary data streams (sources not shown).

[0061] The controller (550) may control the operation of the video encoder (503). During encoding, the controller (550) may assign to each encoded picture an encoded picture type, which may determine the encoding methods applicable to the corresponding picture. For example, images can often be assigned the following image types.

[0062] An intra-coded picture (I-picture) that can be encoded and decoded without using any other picture in the sequence as a source for prediction. Some video codecs allow different types of intra-coded images, including, for example, images in the Independent Decoder Refresh (IDR) format. The person skilled in the art will be aware of the varieties of I-images and their respective applications and features.

[0063] A predictive picture (P-picture) that can be encoded and decoded using intra prediction or inter prediction using at most one motion vector and a reference index to predict sample values of each block.

[0064] A bidirectional predictive picture (B-picture) that can be encoded and decoded using intra prediction or inter prediction using no more than two motion vectors and reference indices to predict sample values of each block. Similarly, multi-prediction pictures may use more than two reference pictures and associated metadata to reconstruct a single block.

[0065] Source images typically allow for spatial division into multiple blocks of samples (eg, blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and coding on a block-by-block basis. The blocks may be predictively encoded with reference to other (previously coded) blocks defined by the coding assignment applied to the respective images of those blocks. For example, blocks of I-pictures may be encoded without prediction or predictively with reference to previously encoded blocks of the same picture (with spatial prediction or intra prediction). P-picture pixel blocks may be predictively encoded by spatial prediction or temporal prediction with reference to one previously encoded reference picture. Blocks of B-pictures may be predictively encoded by spatial prediction or temporal prediction with reference to one or two previously encoded reference pictures.

[0066] The video encoder (503) may perform encoding operations according to a predetermined video coding technology or standard, such as ITU-T Rec. H.265. In operation, the video encoder (503) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data may follow the syntax given by the technology used or the video coding standard.

[0067] According to an embodiment of the invention, the transmitter (540) may transmit additional data with encoded video data. The source encoder (530) may include such data as part of the encoded video sequence. The additional data may include temporal/spatial/SNR enhancement levels, other forms of redundant data such as redundant images and slices, SEI messages, VUI parameter set fragments, and so on.

[0068] The video may be captured as a plurality of source images (video images) in time sequence. Intra-image prediction (often abbreviated as intra-image prediction) uses the spatial correlation within a given image, while inter-image prediction uses (temporal or otherwise) correlation between images. For example, a specific image to be encoded/decoded, which is called the current image, is divided into blocks. When a block in the current picture is similar to a reference block in a previously encoded and still buffered reference picture in a video, the block in the current picture may be encoded with a vector called a motion vector. The motion vector points to a reference block in the reference picture and may have a third dimension identifying the reference picture if multiple reference pictures are used.

[0069] In some embodiments of the invention, a double prediction technique in inter-picture prediction may be used. According to the double prediction method, two reference pictures are used, for example, a first reference picture and a second reference picture, both of which precede the current picture in the video in decoding order (but may be in the past and future, respectively, in display order). A block in the current picture may be encoded by a first motion vector that points to a first reference block in a first reference picture and a second motion vector that points to a second reference block in a second reference picture. The block may be predicted with a combination of the first reference block and the second reference block.

[0070] Additionally, the fusion mode method can be used in inter-picture prediction to improve coding efficiency.

[0071] According to some embodiments of the invention, predictions such as inter image prediction and intra image prediction are performed block by block. For example, according to the HEVC standard, an image in a video sequence is split into coding tree units (CTUs) for compression, CTUs in an image are the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), namely one luma CTB and two chrominance CTBs. Each CTU may be recursively divided on a quadtree basis into one or more coding units (CUs). For example, a 64x64 pixel CTU can be divided into one 64x64 pixel CU, or 4 32x32 pixel CUs, or 16 16x16 pixel CUs. For example, each CU is analyzed to determine a prediction type for that CU, such as an inter prediction type or an intra prediction type. The CU unit is divided into one or more prediction units (PU) depending on temporal and/or spatial predictability. In general, each PU includes a luminance prediction block (PB) and two chrominance prediction blocks PB. According to an embodiment of the invention, the encoding (encoding/decoding) prediction operation is performed in a prediction block unit. As an example of a prediction block, a luminance prediction block includes a matrix of values (eg, luminance values) for pixels, such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, and the like.

[0072] FIG. 6 shows a diagram of a video encoder (603) according to another embodiment of the invention. The video encoder (603) is configured to receive a processing unit (eg, a prediction unit) of sample values in the current video image in the video sequence and encoding the processing unit into an encoded image that is part of the encoded video sequence. For example, video encoder (603) is used instead of video encoder (303) in the example shown in FIG. 3.

[0073] In the HEVC example, video encoder (603) receives a matrix of sample values for a processing block, such as an 8×8 sample prediction block, or the like.

The video encoder (603) determines whether the processing block is best encoded in intra prediction mode, inter prediction mode, or dual prediction mode using, for example, rate-distortion optimization. When the processing unit is to be coded in intra prediction mode, the video encoder (603) may use an intra prediction technique to encode the processing unit into an encoded picture, and when the processing unit is to be coded in inter prediction mode or dual prediction mode, the video encoder (603) may use an inter prediction technique. prediction or double prediction, respectively, to encode the processing block into the encoded picture. In some video coding technologies, the merge mode may be a sub-mode of inter-picture prediction where a motion vector is derived from one or more motion vector predictors without involving the encoded motion vector component outside of the predictors. In some other video coding technologies, there may be a motion vector component applicable to a given block. By way of example, the video encoder (603) includes other components, such as a mode determination module (not shown) for determining the mode of the processing units.

[0074] In the example shown in FIG. 6, the video encoder (603) comprises an outer encoder (630), an intra encoder (622), a residual calculator (623), a switch (626), a residual encoder (624), a common controller (621), and an entropy encoder (625), connected to each other as shown in Fig. 6.

[0075] The outer encoding encoder (630) is configured to receive samples of the current block (for example, a processing block), compare the block with one or more reference blocks in the reference pictures (for example, blocks in previous pictures and later pictures), create outer information prediction (eg, descriptions of redundant information according to the inter-coding method, motion vectors, merge mode information); and calculating inter-prediction results (eg, prediction block) based on the inter-prediction information using any suitable method. In some examples, reference pictures are decoded reference pictures that are decoded based on encoded video information.

[0076] The encoder (622) of the intra coding is configured to receive samples of the current block (for example, a processing block), in some cases, comparing the block with blocks previously encoded in the same image, generating quantized coefficients after the transformation, and in some cases also information of the intra prediction (eg, intra prediction direction information according to one or more intra coding methods). By way of example, the intra encoder (622) also calculates intra prediction results (eg, prediction block) based on intra prediction information and reference blocks in the same picture.

[0077] The common controller (621) is configured to determine common control data and control other components of the video encoder (603) based on the common control data. For example, the common controller (621) determines the block mode and outputs a control signal to the switch (626) based on the mode. For example, when the mode is the intra coding mode, the common controller (621) controls the switch (626) to select the result of the intra coding mode to be used by the residual calculator (623) and controls the entropy encoder (625) to select the intra prediction information and include the intra prediction information in bitstream, and when the mode is an inter-coding mode, the common controller (621) controls a switch (626) to select an inter-prediction result for use by a residual calculator (623) and controls an entropy encoder (625) to select inter-prediction information and turn on inter-prediction information to the bitstream.

[0078] The residual calculator (623) is configured to calculate the difference (residual data) between the received block and the prediction results selected from the intra encoder (622) or the outer encoder (630). The residual encoder (624) is configured to operate on the residual data to encode the residual data to generate transform coefficients. For example, the residual encoder (624) is configured to transform residual data from the spatial domain to the frequency domain and generate transform coefficients. Then, the transform coefficients are subjected to quantization processing to obtain quantized transform coefficients. In various embodiments of the invention, the video encoder (603) also contains a residual decoder (628). The residual decoder (628) is configured to perform inverse transformation and generate decoded residual data. The decoded residual data may be appropriately used by the intra encoder (622) and the outer encoder (630). For example, an inter encoder (630) may generate decoded blocks based on the decoded residual data and inter prediction information, and an intra encoder (622) may generate decoded blocks based on the decoded residual data and intra prediction information. The decoded blocks are properly processed to generate decoded pictures, and the decoded pictures can be buffered in a memory circuit (not shown) and used as reference pictures in some examples.

[0079] The entropy encoder (625) is configured to format the bitstream to include a coded block. The entropy encoder (625) is configured to include various information according to a suitable standard, such as the HEVC standard. For example, the entropy encoder (625) is configured to include common control data, selected prediction information (eg, intra prediction information or inter prediction information), residual information, and other appropriate information in the bitstream. It should be noted that, according to the invention, when encoding a block in the merge sub-mode for either of the inter-coding mode and the dual prediction mode, there is no residual information.

[0080] FIG. 7 shows a diagram of a video decoder (710) according to another embodiment of the invention. The video decoder (710) is configured to receive encoded pictures that are part of the encoded video sequence and decode the encoded pictures to generate reconstructed pictures. For example, video decoder (710) is used instead of video decoder (310) in the example shown in FIG. 3.

[0081] In the example shown in FIG. 7, the video decoder (710) comprises an entropy decoder (771), an outer decoder (780), a residual decoder (773), a reconstruction unit (774), and an inner decoder (772) connected to each other as shown in FIG. 7.

[0082] The entropy decoder (771) may be configured to reconstruct, from the encoded image, some symbols that represent the syntax elements that make up the encoded image.

Such symbols may include, for example, a block coding mode (eg, intra coding mode, inter coding mode, dual prediction mode, the latter two in a merge submode or other submode), prediction information (eg, intra prediction information or inter prediction information), which can identify a certain sample or metadata used for prediction by the intra decoder (772) or the outer decode decoder (780), respectively, residual information in the form of, for example, quantized transform coefficients and the like. For example, when the prediction mode is an intra or dual prediction mode, the inter prediction information is supplied to the inter prediction decoder (780), and when the prediction type is the intra prediction type, the intra prediction information is supplied to the intra decoding decoder (772). The residual information may be inverse quantized and fed to the residual decoder (773).

[0083] The inter prediction decoder (780) is configured to receive inter prediction information and generate inter prediction results based on the inter prediction information.

[0084] The intra decoding decoder (772) is configured to receive intra prediction information and generate prediction results based on the intra prediction information.

[0085] The residual decoder (773) is configured to perform inverse quantization to extract the dequantized transform coefficients and process the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (773) may also require some control information (to enable the Quantizer Parameter (QP)), and this information may be provided by the entropy decoder (771) (the data path is not shown since this may only be small volume control information) .

[0086] The reconstruction module (774) is configured to combine, in the spatial domain, the residual at the output of the residual decoder (773) and prediction results (at the output of the inter or intra prediction modules, as appropriate) to form a reconstructed block that may be included into the reconstructed image, which, in turn, can be part of the reconstructed video. Note that other suitable operations, such as a deblocking operation and the like, may be performed to improve the visual quality.

[0087] It should be noted that video encoders (303), (503) and (603) and video decoders (310), (410) and (710) may be implemented using any suitable technology. According to an embodiment of the invention, video encoders (303), (503) and (603) and video decoders (310), (410) and (710) may be implemented using one or more integrated circuits. In another embodiment of the invention, video encoders (303), (503) and (503) and video decoders (310), (410) and (710) may be implemented using one or more processors that execute program instructions.

[0088] Aspects of the invention provide methods for signaling a merge/skip mode. Various merge/skip modes are provided for inter-picture prediction coding to improve video coding for various video attributes. Skip mode generally works the same as the corresponding merge mode. The difference is that in skip mode, no residual data is added to the predicted block, while in merge mode, residual data is added to the predicted block. It should be noted that while merge modes are used to illustrate the signaling methods, the signaling methods can be suitably used to signal the skip mode.

[0089] In some examples, the merge modes may include normal merge mode, motion vector difference (MMVD) merge mode, sub-block based merge mode, combined intra-inter prediction (CIIP) mode, triangulation mode, and the like. For example, on the encoder side, the encoder selects a particular merge mode from the merge modes to encode the block, and on the decoder side, the decoder reconstructs the block samples using that particular merge mode. In some related examples, a particular merge mode is signaled by the encoder side to the decoder side by signaling, and there is redundancy in the signaling of the different merge modes. The methods presented herein eliminate signaling redundancy for various fusion modes and improve coding efficiency.

[0090] In some examples, flags are used to signal various merge/skip modes. In some embodiments, a tree structure is used by the encoder and decoder to decide on the merge mode. Based on the tree structure, the flags are checked in a predetermined order to determine the appropriate merge mode for encoding/decoding the block. Table 1 shows an example signaling method for five types of merge modes using merge mode flags. For example, in Table 1, four merge mode flags such as a normal flag, an MMVD flag, a subblock flag, and a CIIP flag are used to signal various types of merge modes.

[0091] In the example, the block is encoded/decoded using the merge mode. According to Table 1, when a normal flag for a block is true (eg, binary one), the normal merge mode is used to encode/decode the block, regardless of the values of other merge mode flags. When the normal flag for a block is false (eg, binary zero) and the MMVD flag is true, the MMVD mode is used to encode/decode the block. When the normal flag and the MMVD flag are false and the sub-block flag is true, the sub-block merge mode is used to encode/decode the block. When the normal flag, the MMVD flag, and the sub-block flag are false and the CIIP flag is true, the CIIP mode is used to encode/decode the block. When the normal flag, the MMVD flag, the sub-block flag, and the CIIP flag are false, the delta mode is used to encode/decode the block.

[0092] Table 2 illustrates the signaling method for four types of skip modes. For example, in Table 2, three flags such as a normal flag, an MMVD flag, and a subblock flag are used to signal various skip modes. It should be noted that in some examples the same flag may be used to signal a merge mode or a skip mode which is similar to the merge mode.

[0093] In the example, the block is encoded/decoded using the skip mode. According to Table 2, when a normal flag for a block is true (eg, binary one), the normal skip mode is used to encode/decode the block, regardless of the values of other skip mode flags. When the normal flag for the block is false (eg, binary zero) and the MMVD flag is true, the MMVD (skip) mode is used to encode/decode the block. When the normal flag and the MMVD flag are set to false and the sub-block flag is set to true, a sub-block based skip mode is used to encode/decode the block. When the normal flag, the MMVD flag, and the sub-block flag are false, the (skip) triangle mode is used to encode/decode the block.

[0094] It should be noted that although some embodiments of the present invention use merge modes as examples, the invention is not limited to merge modes and can be modified accordingly for skip modes.

[0095] According to some aspects of the invention, the merge mode decision depends on high-level information (above the block level) and block information. The high-level information includes information in a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, tile group information, tile information, and the like. The block information includes information about the size of the block, such as the width, height, and area of the block. Table 3 shows the syntax table for merge modes.

[0096] For example, in Table 3, a tree structure is formed in the syntax for determining the appropriate merge mode (signal flag on the encoder side or parse flag on the decoder side) to use in the block (for example, upper left corner - (x0, y0), block width cbWidth, block height - cbHeight). The tree structure checks the merge modes in order, such as normal merge mode, MMV mode, sub-block merge mode, CIIP mode, and triangulation mode. For a mode, a valid condition is first checked, and if the valid condition is satisfied, a flag may be signaled (on the encoder side) or parsed (on the decoder side).

[0097] For example, in Table 3, when the block prediction mode (eg, CuPredMode[x0][y0]) is not an intra block copy (MODE IBC) mode, a tree structure is used to signal (or parse) information from the coded video bitstream for the appropriate merge mode.

[0098] In particular, for example, in table 3, a regular flag (regular merge flag) is signaled for the normal merge mode. When the normal flag is true, the merge index (merge_index) is signaled.

[0099] In some examples (eg, VVC draft 5), normal merge mode is enabled when the MaxNumMergeCand variable in the slice header is greater than 0, and the MaxNumMergeCand variable is used to signal the maximum number of merge candidates.

[0100] The example uses the allowRegularMerge variable to indicate whether regular merge mode is allowed or not. For example, the allowRegularMerge variable could be defined as in expression (1):

Thus, when MaxNumMergeCand is greater than 0, allowRegularMerge is true; otherwise, allowRegularMerge is false.

[0101] In addition, for example, in Table 3, when the normal flag is false, the MMVD mode is checked. The MMVD mode is enabled by the spsmmvdenabledflag flag in the SPS. In the example, MMVD mode is signaled when spsmmvdenabledflag is true and the block size requirement is met. In the example, cbWidth indicates the width of the encoding block, and cbHeight indicates the height of the encoding block. The MMVD flag (mmvd_flag) is then signaled when sps mmvd enabled flag is true and cbWidth × cbHeight is not 32. When the MMVD flag is true, other pertinent information for the MMVD mode is sent.

[0102] It should be noted that when cbWidth×cbHeight is 32, only normal merge mode and MMVD mode are allowed. In the example where cbWidth×cbHeight is 32, when the normal flag is false, then the prediction mode is MMVD mode, and it can be concluded that the MMVD flag is true. In this example, it is not required to signal the MMVD flag from the encoder side to the decoder side. This example removes redundancy for MMVD mode signaling.

[0103] In some examples, the allowMMVD variable is used to indicate whether the MMVD mode is allowed or not. The variable allowMMVD can be defined as in expression (2):

[0104] In addition, for example, in Table 3, when the MMVD flag is false, the merge mode is checked based on the subblock. The subblock-based merge mode is enabled when the MaxNumSubblockMergeCand variable, which is based on the information signaled in the slice header, is greater than 0 and cbWidth and cbHeight are respectively greater than or equal to 8. In the example, when the subblock-based merge mode is enabled, the subblock flag (merge_subblock_flag) is signaled. ). When the sub-block flag is true, other pertinent information for the sub-block-based merge mode may be signaled.

[0105] In the example, the allowSubMerge variable is used to indicate whether the subblock-based merge mode is allowed or not. The variable allowSubMerge can be defined as in expression (3):

[0106] In addition, for example, in Table 3, when the sub-block flag is false, the CIIP mode is checked. CIIP mode is enabled when sps ciip enabled flag is true, the current block is not in skip mode, cbWidth × cbHeight is at least 64, cbWidth is less than 128, and cbHeight is less than 128. For example, when CIIP mode is enabled, the CIIP flag is signaled. When the CIIP flag is true, other pertinent information for the CIIP mode may be transmitted.

[0107] In the example, the allowCIIP variable is used to indicate whether the CIIP mode is allowed or not. The variable allowCIIP is defined as in expression (4):

[0108] In some examples, instead of signaling, it may be concluded that the mode is a fusion mode of the delta mode. When the normal flag, the MMVD flag, the subblock flag, the CIIP flag are all false, then the triangulation mode merge mode can be enabled. To enable merge mode of triangle mode, the sps triangle enabled flag must be true, the current slice must be slice B, MaxNumTriangleMergeCand, which is based on the information signaled in the slice header, is greater than or equal to 2, and cbWidth × cbHeight is greater than or is 64. In the example, the allowTriangle variable is used to indicate whether the triangular mode merge mode is allowed. The variable allowTriangle can be defined as in expression (5):

[0109] In some related examples, a normal flag is always signaled. However, this is not necessary because, under certain conditions, the encoder and decoder may only select the normal merge mode.

[0110] According to some aspects of the invention, at the encoder side, before signaling the merge mode flag for the merge mode, a certain condition is checked to determine whether the merge mode flag can be inferred. When the condition satisfies the requirement to infer the merge mode flag, the encoder does not need to signal the merge mode flag. On the decoder side, the decoder can check the condition to determine if the merge mode flag can be inferred. When the condition satisfies the requirement to infer the merge mode flag, the decoder does not parse the encoded video bitstream to obtain the merge mode flag, and may infer the merge mode flag. Thus, signaling redundancy can be eliminated.

[0111] In some embodiments of the invention, when the merge mode flag has only one possible value under a predetermined condition, instead of explicitly signaling, it may be concluded that the flag is a merge mode flag. Merge modes that can be inferred may include, but are not limited to, normal merge mode, MMVD mode, sub-block based merge mode, CIIP mode, and triangulation mode.

[0112] FIG. 8 shows a flowchart for signaling (analyzing) merge mode flags by an encoder (decoder). In FIG. 8, for example, under certain conditions, instead of signaling (analysis), some merge mode flags can be inferred. Thus, signaling costs are reduced and redundancy can be eliminated. In the example algorithm in Fig. 8 is performed by the encoder processing circuit. In another example, the algorithm in FIG. 8 is executed by the decoder processing circuit.

[0113] In FIG. 8, the coding order of the tree structure starts with normal merge mode, followed by MMVD mode, sub-block based merge mode, CIIP mode, and triangulation mode. It should be noted that when the tree structure is changed, the corresponding conditions can be changed accordingly.

[0114] It should also be noted that the expressions (3)-(5) are given simply as an example of defining variables for certain conditions. The conditions can be defined in other suitable ways.

[0115] For example, expression (3) defines the allowSubMerge variable as a combination of several subconditions. The variable allowSubMerge can be defined with subconditions. In an embodiment of the invention, the allowSubMerge variable is equal to MaxNumSubMergeCand>0. For example, when MaxNumSubMergeCand is greater than zero, allowSubMerge is true (for example, binary one), and when MaxNumSubMergeCand is zero, allowSubMerge is false (for example, binary zero). In another embodiment of the invention, the allowSubMerge variable is equal to (cbWidth>=8 and cbHeight>=8). For example, when both the width and height of the encoding block are greater than 8, the variable allowSubMerge is true; otherwise, allowSubMerge is false.

[0116] In another example, expression (4) defines the allowCIIP variable as a combination of several subconditions. The variable allowCIIP can be defined by subconditions. In an embodiment of the invention, the variable allowSubMerge is equal to sps ciip enabled flag. In another embodiment of the invention, the variable allowSubMerge is equal to !cu_skip_flag. For example, when cu skip flag is true, allowSubMerge is false, and when cuskipflag is false, allowSubMerge is true. In another embodiment of the invention, the variable allowSubMerge is (cbWidth × cbHeight>=64). For example, if the block area is equal to or greater than 64, allowSubMerge is true; otherwise, allowSubMerge is false. In another embodiment of the invention, the allowSubMerge variable is (cbWidth<128 and cbHeight<128). For example, when the width and height of the encoding block is less than 128, the variable allowSubMerge is true; otherwise, allowSubMerge is false.

[0117] In another example, expression (5) defines allowTriangle as a combination of several subconditions. The variable allowTriangle can be defined with subconditions. In an embodiment of the invention, the variable allowTriangle is equal to spstriangleenbaledflag. In another embodiment of the invention, the allowTriangle variable is set based on the slicetype variable. When slice type is B (bidirectional prediction), allowTriangle is true, and when slice type is not B, allowTriangle is false. In another embodiment of the invention, the allowTriangle variable is equal to (MaxTriangleMergeCand>=2). For example, when MaxTriangleMergeCand is equal to or greater than 2, allowTriangle is true; otherwise, allowTriangle is false. In another embodiment of the invention, the variable allowTriangle is (cbWidth × cbHeight>=64). For example, when the coding block area is equal to or greater than 64, the variable allowTriangle is true; otherwise, allowTriangle is false.

[0118] As shown in FIG. 8, the algorithm starts from step (S801) and proceeds to step (S810).

[0119] Step (S810) checks the first condition (allowRegularMerge && (allowMMVD || allowSubMerge || allowCIIP || allowTriangle)). When the first condition is true, the algorithm proceeds to step (S812), and when the first condition is false, the algorithm proceeds to step (S815).

[0120] In step (S812), a normal flag is signaled by the encoder (analyzed by the decoder).

[0121] In step (S815), the normal flag is not signaled, but inferred. In the example, when allowRegularMerge is false, the normal flag is deduced to be false. In another example, when (allowMMVD || allowSubMerge || allowCIIP || allowTriangle) is "false", the normal flag is inferred to be "true".

[0122] In some examples, the variables allowRegularMerge, allowMMVD, allowSubMerge, allowCIIP and allowTriangle are defined as in expressions (1), (2), (3), (4) and (5).

[0123] It should be noted that the allowSubMerge, allowCIIP, and allowTriangle variables may be defined in another suitable manner. In some examples, some merge mode conditions overlap, such as when cbWidth × cbHeight is less than 64 (e.g., box width times box height is 32), sub-block based merge mode, CIIP, and triangulation mode are not allowed. When the sps mmvd enabled flag is false and cbWidth × cbHeight is less than 64, MMVD mode, subblock-based merge mode, CIIP mode, and triangulation mode are not enabled. In this case, only normal merge mode is allowed.

[0124] In step (S820), a regular flag is checked. When the normal flag is true, the algorithm proceeds to step (S825); otherwise, the algorithm proceeds to step (S830).

[0125] In step (S825), the normal flag is true, the algorithm returns to the normal merge mode and stops. The block is then encoded (decoded) based on the normal merge mode.

[0126] At (S830), the second condition is checked (allowMMVD && (allowSubMerge | allowCIIP allowTriangle)). When the second condition is true, the algorithm proceeds to step (S832), and when the second condition is false, the algorithm proceeds to step (S835).

[0127] In step (S832), the MMVD flag is signaled by the encoder (analyzed by the decoder).

[0128] In step (S835), the MMVD flag is not signaled but inferred. In the example where allowMMVD is false, the MMVD flag is deduced to be false. In another example, when (allowSubMerge || allowCIIP || allowTriangle) is false, the MMVD flag is inferred to be true. In some examples, the variables allowMMVD, allowSubMerge, allowCIIP and allowTriangle are defined as in expressions (2), (3), (4) and (5). Note that the allowMMVD, allowSubMerge, allowCIIP, and allowTriangle variables may be defined in other appropriate ways.

[0129] In step (S840), the MMVD flag is checked. When the MMVD flag is true, the algorithm proceeds to step (S845); otherwise, the algorithm proceeds to step (S850).

[0130] In step (S845), the MMVD flag is true, the algorithm returns to the MMVD mode and stops. The block is then encoded (decoded) based on the MMVD mode.

[0131] In step (S850), the third condition (allowSubMerge && (allowCIIP || allowTriangle)) is checked. When the third condition is true, the algorithm proceeds to step (S852); when the third condition is false, the algorithm proceeds to step (S855).

[0132] In step (S852), the sub-block flag is signaled by the encoder (analyzed by the decoder).

[0133] In step (S855), the sub-block flag is not signaled, but inferred. In the example, when allowSubMerge is false, the subblock flag is deduced to be false. In another example, when (allowCIIP || allowTriangle) is false, the sub-block flag is inferred to be true. In some examples, the variables allowSubMerge, allowCIIP and allowTriangle are defined as in expressions (3), (4) and (5). Note that the allowSubMerge, allowCIIP, and allowTriangle variables may be defined in other appropriate ways.

[0134] In step (S860), the sub-block flag is checked. When the sub-block flag is true, the algorithm proceeds to step (S865); otherwise, the algorithm proceeds to step (S870).

[0135] In step (S865), the sub-block flag is true, the algorithm returns the sub-block-based merge mode, and stops. The block is then encoded (decoded) based on the sub-block merge mode.

[0136] In step (S870), the fourth condition (allowCIIP && allowTriangle) is checked. When the fourth condition is true, the algorithm proceeds to step (S872); when the fourth condition is false, the algorithm proceeds to step (S875).

[0137] In step (S872), the CIIP flag is signaled by the encoder (analyzed by the decoder).

[0138] In step (S875), the CIIP flag is not signaled but inferred. In the example, when allowSubMerge is false, the CIIP flag is deduced to be false. In another example, when allowTriangle is false, the CIIP flag is deduced to be true. In some examples, the variables allowCIIP and allowTriangle are defined as in expressions (4) and (5). It should be noted that the variables allowCIIP and allowTriangle may be defined in other suitable ways.

[0139] In step (S880), the CIIP flag is checked. When the CIIP flag is true, the algorithm proceeds to step (S882); otherwise, the algorithm proceeds to step (S885).

[0140] In step (S882), the CIIP flag is true, the algorithm returns to the CIIP mode and stops. The block is then encoded (decoded) based on the CIIP mode.

[0141] In step (S885), the algorithm returns to the triangle mode and stops. The block is then encoded (decoded) based on the delta mode.

[0142] It should be noted that in some examples for inter prediction, valid cbWidth and cbHeight values are already defined for each mode. However, in some examples, the valid cbWidth and cbHeight values may change, after which you can change the variables and conditions accordingly.

[0143] In some embodiments, N is defined as the minimum allowable width, M is defined as the minimum allowable height, K is defined as the minimum area, H is defined as the upper limit of the width, L is defined as the upper limit of the height, and O is defined as the upper limit of the area. M, N, K, H, L, O are non-negative integers. In the example, the variables allowSubMerge, allowCIIP and allowTriangle can be defined as in expressions (6) - (8).

[0144] According to another aspect of the invention, merge mode signaling redundancy can be eliminated with context. In some embodiments, more context is added for the regular flag, the MMVD flag, the subblock flag, and the CIIP flag. Context selection can be based on block size, sps enable flags, or slice type. Some or all of the above signaling conditions may be used for context selection.

[0145] In an embodiment of the invention, a conventional flag may be based on multiplying the width of the block by the height of the block, for example, when cbWidth × cbHeight is less than a size threshold, one context is used. Otherwise, another context is used.

[0146] According to another aspect of the invention, condition checks for MMVD flag signaling can be removed. In some examples, the MMVD flag is signaled without checking the block size to speed up the decoding process. Table 4 shows an example syntax for signaling the MMVD flag.

[0147] Comparing Table 4 with Table 3, it can be seen that the MMVD flag is signaled without checking the block size (e.g., block width and height) in Table 4.

[0148] In some embodiments of the invention, to avoid broken bit streams, a matching constraint may be added. In the example, the match constraint restricts the common merge flag and the skip flag. For example, when the regular flag is false, the MMVD flag is false, the subblock flag is false, the CIIP flag is false, and the triangle flag is false, then the common merge flag and flag gaps must evaluate to false according to the match constraint.

[0149] In another example, another matching constraint is added. When the value of (cbWidth×cbHeight) is 32 and the regular flag is false, then the MMVD flag is signaled as having a value of true according to the matching constraint.

[0150] FIG. 9 shows a flowchart describing a process (900) according to an embodiment of the invention. Process (900) may be used to determine a prediction mode for the block to generate a prediction block for the reconstructed block based on the prediction mode. In various embodiments of the invention, the process (900) is performed by a processing circuit, for example, a processing circuit in terminal devices (210), (220), (230) and (240), a processing circuit that performs the functions of a video encoder (303), a processing circuit that performs the functions of a video decoder (310), a processing circuit that performs the functions of a video decoder (410), a processing circuit that performs the functions of a video encoder (503), and the like. In some embodiments, the process (900) is implemented in program instructions, such that when the processing circuit executes the program instructions, the processing circuit executes the process (900). The process starts from step (S901) and proceeds to step (S910).

[0151] In step (S910), a condition for deriving a flag associated with a specific prediction mode for the current block is checked before parsing the specific prediction mode flag for the current block from the encoded video bitstream. This particular prediction mode is one of a plurality of inter image prediction modes. In the example, the particular prediction mode is one of the merge modes. In another example, a particular prediction mode is one of the skip modes. In some embodiments, a plurality of inter image prediction modes form a tree structure for deciding a prediction mode for the current block. When the flag inference obtaining condition indicates that the flag inference can be obtained, the process proceeds to step (S920); otherwise, the process proceeds to step (S930).

[0152] In step (S920), a flag associated with a particular prediction mode is inferred without parsing the flag from the encoded video bitstream. In an example where a flag can be inferred, the flag is not signaled in the encoded video bitstream to reduce signal redundancy.

[0153] In step (S930), a flag associated with a particular prediction mode is parsed from the encoded video bitstream.

[0154] In step (S940), the flag is checked to determine whether the flag indicates the application of a particular prediction mode in the current block. When the flag indicates the application of a specific prediction mode, the process proceeds to step (S950); otherwise, the process proceeds to step (S960).

[0155] In step (S950), samples of the current block are reconstructed according to the specific prediction mode. The process proceeds to step S999 and ends.

[0156] In step (S960), the process continues to decode the next flag in the tree structure.

[0157] The above methods may be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, in FIG. 10 shows a computer system (1000) suitable for practicing some embodiments of the invention.

[0158] Computer software may be coded using any suitable machine code or computer language that may be assembled, compiled, linked, or similarly processed to produce code containing instructions that can be executed directly or through interpretation, execution of microcode, etc. etc., one or more computer central processing units (CPU, central processing units), graphic processors (GPU, Graphics Processing Units), etc.

[0159] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, IoT devices, and the like.

[0160] The computer system components (1000) shown in FIG. 10 are illustrative in nature and are not intended to impose any limitation on the scope of application or functionality of computer software implementing embodiments of the present invention. The component configuration should also not be interpreted as having any dependency or requirement in connection with any component or combination of components shown in the exemplary computer system (1000).

[0161] The computer system (1000) may include some human-machine interface input devices. Such an input device may be responsible for input by one or more users through, for example, tactile input (e.g., keystrokes, waving, glove movements), audio input (e.g., voice, claps), visual input (e.g., gestures), olfactory input (not shown). Interface devices may also be used to capture certain media not necessarily directly related to conscious human input, such as sound (e.g. speech, music, external sound), images (e.g. scanned images, photographic images received from a still camera), video (eg 2D video, 3D video including stereoscopic video).

[0162] The input human-machine interface of the device may include one or more of the following (shown one at a time): keyboard (1001), mouse (1002), touch pad (1003), touch screen (1010), information glove (not shown) , joystick (1005), microphone (1006), scanner (1007) and camera (1008).

[0163] The computer system (1000) may also include some human interface output devices. Such output devices may stimulate the senses of one or more users through, for example, tactile output, sound, light, and smell/taste. Such output devices may include tactile output devices (e.g., tactile feedback via touch screen (1010), data glove (not shown) or joystick (1005), as well as tactile feedback devices that do not serve as input devices), audio output devices (for example: speakers (1009), headphones (not shown)), visual output devices (for example, screens (1010), including CRT screens, LCD screens, plasma screens, OLED screens, each with touch screen input capabilities or without, each with or without haptic feedback capabilities, some capable of 2D visual output or more than 3D output via e.g. stereographic output, virtual reality goggles (not shown), holographic displays, smoke tanks (not shown). ) and printers (not shown).

[0164] The computer system (1000) may also include human accessible storage devices and associated media, such as optical media including CD/DVD ROM/RW (1020) with CD/DVD media (1021) and the like, flash memory cards (1022), removable hard drive or solid state drive (1023), conventional magnetic media such as tape and magnetic disk (not shown), dedicated ROM/ASIC/PLD based devices such as security dongles (not shown). shown), etc.

[0165] Those skilled in the art should also understand that the term "computer-readable media" as used in connection with the present invention does not encompass transmission media, carrier waves, or other time signals.

[0166] The computer system (1000) may also include an interface to one or more communication networks. Networks can be, for example, wireless, wired, optical. Networks can also be local, global, urban, transport and industrial, real-time, latency, etc. Examples of networks include local area networks, such as Ethernet, wireless LANs, cellular networks, including GSM, 3G, 4G, 5G, LTE, etc., digital wide area networks of wired or wireless television, including cable television, satellite television and terrestrial broadcasting, transport and industrial networks, including CANBus, etc. Some networks typically require external network interface adapters that are connected to some general purpose data ports or peripheral buses (1049) (for example, computer system USB ports (1000)); others are typically built into the core of the computer system (1000) by connecting to a system bus as described below (eg, an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1000) can communicate with other entities. Such communication may be unidirectional with receive-only capability (eg broadcast), unidirectional with transmit-only capability (eg CANbus to some CANbus devices), or bidirectional, eg to other computer systems using a local area network or a wide area digital network. Several protocols and protocol stacks may be used on each of these networks and the network interfaces described above.

[0167] The aforementioned human-machine interface devices, human-accessible storage devices, and network interfaces can be connected to the core (1040) of the computer system (1000).

[0168] The core (1040) may include one or more central processing units (CPUs) (1041), graphics processing units (GPUs) (1042), specialized programmable processing modules in the form of Field Programmable Gate Arrays (FPGAs) ( 1043), hardware accelerators (1044) for some tasks, etc. These devices, together with read-only memory (ROM) (1045), random access memory (1046), high capacity internal storage such as user-inaccessible internal hard drives, SSD, etc. (1047) can be connected via the system bus (1048). In some computer systems, the system bus (1048) may be available in the form of one or more physical connectors to allow for expansions with additional CPUs, GPUs, or the like. Peripherals can be connected either directly to the system bus (1048) of the kernel, or via a peripheral bus (1049). Peripheral bus architectures include PCI, USB, and the like.

[0169] The CPU (1041), GPU (1042), FPGA (1043), and accelerators (1044) devices may execute some instructions that together may constitute the aforementioned computer code. This computer code may be stored in ROM (1045) or RAM (1046). Temporary data may also be stored in the RAM memory (1046), while permanent data may be stored, for example, in the internal mass storage (1047). Fast storage and retrieval from any storage device can be achieved through the use of cache memory, which can be closely associated with one or more processors CPU (1041), GPU (1042), mass storage (1047), ROM memory (1045) , RAM (1046), etc.

[0170] Computer-readable media may store computer code for performing various computer-executable operations. The media and computer code may be specially created for the purposes of the present invention or may be well known and available to those skilled in the art of computer software.

[0171] By way of example, and not limitation of the invention, a computer system having an architecture (1000), and in particular, a core (1040), can provide functionality by being executed by the processor(s) (including(and) CPU, GPU , FPGAs, accelerators, etc.) software embodied in one or more tangible computer-readable media. Such computer-readable media may be media associated with the user-accessible mass storage described above, as well as some persistent kernel storage (1040), such as internal mass storage (1047) or ROM (1045). Software implementing various embodiments of the present invention may be stored on such devices and executed by the core (1040). The computer-readable medium may include one or more storage devices or microcircuits, in accordance with specific needs. The software may direct the core (1040), and in particular its processors (including CPUs, GPUs, FPGAs, and the like), to execute specific processes or specific portions of the specific processes described herein, including setting data structures stored in RAM memory (1046 ), and modification of such data structures according to the processes specified by the software. Additionally or alternatively, the computer system may provide functionality through logic hardwired or otherwise embodied in circuitry (e.g., an accelerator (1044)), which may operate in place of, or in conjunction with software, to perform specific processes or specific portions of the specific processes described herein. processes. A reference to software may cover logic, and vice versa when appropriate. Reference to computer-readable media may encompass a circuit (eg, an integrated circuit (IC)) where software is stored for execution, a circuit embodying logic for execution, or both circuits, as appropriate. The present invention encompasses any suitable combination of hardware and software.

Appendix A: Abbreviations

JEM: collaborative research model

VVC: Universal Video Data Coding

BMS: a set of standards

MV: motion vector

HEVC: High Performance Video Coding

SEI: Additional Enhancement Information

VUI: Video Usability Information

GOP: group of images

TU: conversion unit

PU: prediction unit

CTU: coding tree unit

CTB: coding tree block

RW: prediction block

HRD: Hypothetical Reference Decoder

SNR: signal to noise ratio

CPU: central processing unit

GPU: GPU

CRT: kinescope

LCD: liquid crystal display

OLED: organic light emitting diode

CD: CD

DVD: Digital Video Disc

ROM: Read Only Memory

RAM: random access memory

ASIC: Application Specific Integrated Circuit

PLD: Programmable Logic Device

LAN: local area network

GSM: Global System for Mobile Communications

LTE: a project for the long-term development of communication systems

CANBus: controller network bus

USB: Universal Serial Bus

PCI: Peripheral Component Connection

FPGA: Field Programmable Gate Array

SSD: solid state drive

IC: integrated circuit

CU: coding unit

[0172] Although some embodiments of the invention have been described here, changes, permutations, and various equivalent substitutions are possible within the scope of the invention. Thus, those skilled in the art can provide numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the invention and thus fall within its spirit and scope.

Claims (11)

1. A method for decoding video data in a decoder, including: checking, by the processor, conditions for deriving a specific prediction mode flag for the current block before parsing the specific prediction mode flag for the current block from the encoded video bitstream, said specific prediction mode being one of a plurality of inter picture prediction modes; wherein said flag inference condition is based at least on whether said specific prediction mode is valid, and the validation of the specific prediction mode is determined based on current block size information; obtaining a flag output by the processor without parsing the flag from the encoded video data bitstream when the flag output obtaining condition indicates that the flag can be inferred; analysis by the processor of the flag from the bitstream of encoded video data, when the condition for obtaining the output about the flag indicates the uncertainty of the output about the flag, and the reconstruction by the processor of the current block in accordance with the mentioned specific prediction mode, when the flag indicates the application of the mentioned specific prediction mode to the current block. 2. The method of claim 1, wherein the flag inference condition is also based on whether other modes from the plurality of inter image prediction modes are valid. 3. The method of claim 1, wherein the flag is one of a normal merge/skip normal mode flag, an MMVD motion vector difference (MMVD) merge flag, a subblock merge/skip mode sub-block flag, a combined intra-inter prediction flag (CIIP) of CIIP mode, triangle flag of triangulation mode. 4. The method of claim 2, wherein said other prediction modes are at a lower level than said specific prediction modes in the tree structure. 5. The method of claim 2, wherein the flag inference condition is a combination of the validity of said specific prediction mode and the validity of other modes that are at a lower level than said specific prediction modes in a tree structure. 6. The method of claim 1, wherein the flag inference condition indicates that the flag can be inferred when said particular prediction mode is invalid. 7. The method of claim 1, wherein the flag inference condition indicates that the flag can be inferred when none of the other modes are valid. 8. The method of claim 1, wherein the validity of other modes of the plurality of inter prediction modes is based on at least one of the following: sequence parameter set (SPS), picture parameter set (PPS), slice header, tile information, and tile group information . 9. A device for decoding video data, containing a processing circuit configured to implement the method according to any one of paragraphs. 1-8. 10. A computer-readable medium that stores instructions that, when executed by a computer to decode video data, cause the computer to carry out the method according to any one of paragraphs. 1-8.

Images