RU2798316C2 - Method and equipment for external prediction - Google Patents

Abstract

FIELD: video encoding and decoding.

SUBSTANCE: result is achieved by creating a list of candidates for merging the current image block; adding motion information of history-based motion vector predictor (HMVP) candidates to the merge candidate list in case the amount of motion information of candidates in the merge candidate list is less than the maximum number of candidates in the merge candidate list minus N, where N is a positive integer; getting a merge index of the current image block; obtaining motion information of the current image block based on the merge candidate list and the merge index; and determining a predicted block of the current image block based on the motion information of the current image block.

EFFECT: increasing the accuracy of predicting motion information of an image block, which increases performance of encoding and decoding.

11 cl, 12 dwg, 1 tbl

Description

Cross-reference to related applications

[0001] This application claims the priority of US provisional application no. 62/744,106, filed October 10, 2018, the disclosure of which is hereby incorporated by reference.

Technical field

[0002] The present disclosure relates to the field of video encoding and decoding and, in particular, to an inter prediction method and apparatus for a video image, as well as to a corresponding encoder and decoder.

State of the art

[0003] Digital video capabilities can be incorporated into a wide variety of equipment, including digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording equipment, digital media players, video game equipment, game consoles, cellular or satellite radio telephones (so-called "smartphones"), video conferencing equipment, video streaming equipment, and the like. Digital video equipment implements video compression technologies, such as the video compression technologies described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263 and ITU-T H.264/MPEG-4 Part 10, Advanced Video Coding Standard (AVC), the H.265 video coding standard/High Efficiency Video Coding (HEVC) standard, and extensions to such standards. Video equipment can transmit, receive, encode, decode and/or store digital video information more efficiently by implementing such video compression technologies.

[0004] Video compression techniques perform spatial (intra-image) prediction and/or temporal (inter-image) prediction to reduce or remove intrinsic redundancy in video sequences. For block video coding, a video slice (ie, a video frame or part of a video frame) may be divided into image blocks, and an image block may also be referred to as a tree block, a coding unit (CU), and/or a coding node. An image block in an intra-coded (I) image slice is encoded by spatial prediction of reference samples in adjacent blocks in the same image. For a picture block in a picture slice to be inter-coded (P or B), spatial prediction of reference samples in neighboring blocks in the same picture or temporal prediction of reference samples in other reference pictures may be used. An image may be referred to as a frame, and a reference image may be referred to as a reference frame.

The essence of the invention

[0005] Various embodiments and aspects of the disclosures will be described with reference to the details discussed below, and the accompanying drawings will illustrate various embodiments. The following description and drawings illustrate the disclosure and should not be construed as limiting the disclosure. Numerous specific details are described to provide a thorough understanding of the various embodiments of the present disclosure. However, in some cases, well-known or conventional details are not described in order to provide a brief discussion of embodiments of the present disclosure.

[0006] Reference in the description to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the disclosure. The appearance of the phrase "in one embodiment" in various places in the description does not necessarily refer to the same embodiment.

[0007] Embodiments of the disclosure provide an inter prediction method and apparatus for a video image, as well as a corresponding encoder and decoder, to improve the prediction accuracy of image block motion information to some extent, thereby improving encoding and decoding performance.

[0008] According to a first aspect, a computer-implemented inter-prediction method in video coding is described. In one embodiment, the method generates a merge candidate list based on motion information of a block of encoded pictures. The motion information defines a motion vector predictor candidate (MVP) in the merge candidate list. The method adds new motion information as a new MVP candidate to the merge candidate list when the number of MVP candidates in the merge candidate list is less than the maximum number of candidates. The method obtains the merge index of the position of the optimal motion information in the merge candidate list. The method obtains optimal motion information based on the merge index. The method determines the predicted current image block based on the optimal motion information.

[0009] In one embodiment, the encoded image block is spatially or temporally adjacent to the current coding block.

[0010] In one embodiment, the optimal motion information is used as the motion information of the current coding block.

[0011] In one embodiment, the method further determines the optimal motion information from the merge candidate list using rate distortion overhead.

[0012] In one embodiment, when the number of MVP candidates in the merge candidate list reaches the maximum number of candidates, the method further adds pairwise average candidate movement information to the merge candidate list.

[0013] In one embodiment, the pairwise average candidate movement information defines a pairwise average candidate generated by averaging a predetermined pair of MVP candidates in the merge candidate list.

[0014] In one embodiment, the maximum number of candidates is the maximum number of candidates in the merge candidate list minus N, where N is a positive integer.

[0015] In one embodiment, new motion information is added to the merge candidate list in a predetermined candidate scanning order.

[0016] In one embodiment, temporal motion information of a coded picture block candidate is obtained after the motion vector of the corresponding block position in a reference frame is scaled based on the image order count of the reference frame and the current frame.

[0017] In one embodiment, the motion information comprises at least one of the following: reference picture or motion vector information.

[0018] In one embodiment, the reference picture information includes at least one of unidirectional or bidirectional prediction information, a reference picture list, or a reference picture index corresponding to the reference picture list.

[0019] In one embodiment, before adding new motion information as a new MVP candidate to the merge candidate list, the redundancy method checks the merge candidate list to find if there is an identical MVP candidate in the merge candidate list. In response to finding an identical MVP candidate in the merge candidate list, the method removes the identical MVP candidate from the merge candidate list and moves the remaining MVP candidates in the merge candidate list.

[0020] According to a second aspect, an embodiment of this application provides an inter prediction method (implemented on a computer), including: creating a list of merging candidates of a current image block; when the amount of motion information of the candidate in the merge candidate list is less than the maximum number of candidates in the merge candidate list minus N, adding the motion information of the HMVP candidate in the merge candidate list to obtain a new merge candidate list, where N is a positive integer ; getting the merge index of the current image block; obtaining motion information of the current image block based on the new merge candidate list and the merge index; and determining a predicted block of the current image block based on the motion information of the current image block.

[0021] Here, the predicted block may be referred to as a prediction block.

[0022] Here, the current image block may be a coding block, a coding unit, or a prediction block.

[0023] Meanwhile, the maximum number of candidates in the merge candidate list may be referred to as the maximum number of merge motion vector predictor (MVP) candidates.

[0024] Here, the movement information of the HMVP candidate may be referred to as history-based merge candidates.

[0025] Based on the second aspect, in some possible embodiments, the method further includes: when the amount of motion information of a candidate in the merge candidate list is greater than or equal to the maximum number of candidates in the merge candidate list minus N, adding the pairwise average candidate motion information to the merge candidate list to get a new merge candidate list.

[0026] Based on the second aspect, in some possible embodiments, when the amount of movement information of candidates in the merge candidate list is less than the maximum number of candidates in the merge candidate list minus N, adding HMVP candidate movement information to the merge candidate list for obtaining a new merge candidate list includes: when the amount of motion information of a candidate in the merge candidate list is less than the maximum number of candidates in the merge candidate list, adding motion information of the HMVP candidate to the merge candidate list to obtain the first candidate list to merge; and when the number of movement information of the candidate in the first merge candidate list is less than the maximum number of candidates in the merge candidate list minus N, adding the movement information of the new HMVP candidate to the first merge candidate list to obtain a new merge candidate list.

[0027] Based on the second aspect, in some possible embodiments, the method further includes: when the number of motion information of candidates in the first merge candidate list is greater than or equal to the maximum number of candidates in the merge candidate list minus N, adding motion information on the pairwise middle candidate in the first merger list of candidates to get a new list of merger candidates.

[0028] According to a third aspect, an embodiment of this application provides an inter prediction engine, where the inter prediction engine includes functional blocks for implementing any of the methods in the first aspect. For example, the inter prediction apparatus may include: a generating unit configured to create a merge candidate list of the current image block, and when the amount of motion information of a candidate in the merge candidate list is less than the maximum number of merge candidate list candidates minus N, adding information about the movement of HMVP candidates to the merge candidate list to obtain a new merge candidate list, where N is a positive integer; and a prediction block determining unit configured to obtain a merge index of the current image block, obtain motion information of the current image block based on the new merge candidate list and the merge index, and determine a prediction block of the current image block based on the motion information of the current image block.

[0029] According to a fourth aspect, an embodiment of this application provides an inter prediction method (implemented by a computer), including: creating a list of merging candidates of a current image block; adding HMVP candidate movement information to the merge candidate list if the number of candidate movement information in the merge candidate list is less than the maximum number of candidates in the merge candidate list minus N, where N is a positive integer; obtaining information about the movement of the current image block based on the list of merging candidates; and determining a predicted block of the current image block based on the motion information of the current image block.

[0030] Moreover, N can be 1, 2 or 3.

[0031] Wherein, obtaining motion information of the current image block based on the merge candidate list may comprise: obtaining a merge index of the current image block or a merge index for the merge candidate list; obtaining information about the movement of the current image block based on the list of merge candidates and the merge index. Wherein, obtaining the merge index of the current image block or the merge index for the merge candidate list may comprise: obtaining the merge index by bitstream parsing. At the same time, obtaining information about the movement of the current image block based on the list of candidates for merge and the index of the merge may include: obtaining information about the movement from the list of candidates for the merge using the merge index. Meanwhile, the merge index may be used to indicate the position of the motion information in the merge candidate list.

[0032] Here, the motion information may comprise one or two motion vectors.

[0033] Meanwhile, the motion information may further comprise one or two reference picture indexes from one or two reference picture lists related to one or more motion vectors, unidirectional or bidirectional prediction information, or one or two MVDs (motion vector differences), associated with one or more motion vectors.

[0034] Wherein, obtaining motion information of the current image block based on the merge candidate list may comprise: determining motion information from the merge candidate list using rate distortion overhead. The method may further comprise: encoding a merge index for a merge candidate list in a bitstream.

[0035] Here, the predicted block may be referred to as a prediction block.

[0036] Here, the current image block may be a coding block, a coding unit, or a prediction block.

[0037] Meanwhile, the maximum number of candidates in the merge candidate list may be referred to as the maximum number of merge motion vector predictor (MVP) candidates.

[0038] Here, the HMVP candidate movement information may be referred to as history-based merge candidates or history-based MVP merge candidates (HMVP).

[0039] In an exemplary embodiment of the method according to the fourth aspect, as such, the method further includes: adding motion information of pairwise average candidates in the merge candidate list in the case where the amount of candidate motion information in the merge candidate list is equal to the maximum number of candidates in merger candidate list minus N.

[0040] Meanwhile, information about the movement of pairwise average candidates is generated by averaging a predetermined pair of MVP candidates in the merge candidate list.

[0041] Meanwhile, the motion information about the motion information of pairwise average candidates may be called pairwise average candidates.

[0042] According to a fifth aspect, an inter prediction apparatus is described. The inter prediction apparatus includes functional blocks for implementing any of the methods in the first aspect. For example, the inter prediction apparatus may include a generating unit configured to create a merge candidate list based on the motion information of the encoded picture block, where the motion information specifies an MVP candidate in the merge candidate list, and add the new motion information as the new merge candidate MVP candidate list when the number of MVP candidates in the merge candidate list is less than the maximum number of candidates. The inter prediction apparatus may further include a prediction block determining unit configured to obtain a fusion index of a position of the optimal motion information in the fusion candidate list, obtain the optimal motion information based on the fusion index, and determine a predicted current image block based on the optimal information about movement.

[0043] In some embodiments, image prediction hardware is, for example, applied to video coding hardware (eg, a video encoder) or video decoding hardware (eg, a video decoder).

[0044] The method according to the first aspect of the invention can be performed by the apparatus according to the fifth aspect of the application. Additional features and embodiments of the apparatus according to the fifth aspect of the application correspond to the characteristics and embodiments of the apparatus according to the first aspect of the application.

According to the sixth aspect, the external prediction apparatus is described. The inter prediction apparatus includes functional blocks for implementing any of the methods in the fourth aspect. For example, the inter prediction apparatus may include: a creation unit configured to: create a list of fusion candidates of the current image block; adding HMVP candidate movement information to the merge candidate list if the number of candidate movement information in the merge candidate list is less than the maximum number of candidates in the merge candidate list minus N, where N is a positive integer; a prediction block configured to: obtain a merge index of the current image block; obtaining motion information of the current image block based on the merge candidate list and the merge index; and determining a predicted block of the current image block based on the motion information of the current image block.

In an exemplary embodiment of the apparatus according to the sixth aspect as such, wherein the generating unit is further configured to: add motion information of pairwise average candidates to the merge candidate list in case the amount of candidate motion information in the merge candidate list is equal to the maximum number of candidates in merger candidate list minus N.

Here, information about the movement of pairwise average candidates is generated by averaging a predetermined pair of MVP candidates in the merge candidate list.

In an exemplary embodiment of the apparatus according to any previous embodiment of the sixth aspect, or the sixth aspect as such, wherein the predictor is configured to: obtain motion information from the merge candidate list using the merge index.

In an exemplary embodiment of the apparatus according to any previous embodiment of the sixth aspect, or the sixth aspect as such, wherein the predictor is configured to: determine motion information from the merger candidate list using rate distortion overhead.

[0045] In some embodiments, image prediction hardware is, for example, applied to video coding hardware (eg, a video encoder) or video decoding hardware (eg, a video decoder).

[0046] The method according to the fourth aspect of the invention can be performed by the apparatus according to the sixth aspect of the application. Additional features and embodiments of the apparatus according to the sixth aspect of the application correspond to the characteristics and embodiments of the apparatus according to the fourth aspect of the application.

[0047] According to the seventh aspect, an image prediction apparatus is described. The apparatus includes a processor and a memory associated with the processor, and the processor is configured to perform the method in any of the first to fourth aspects or the first to fourth aspects.

[0048] According to the eighth aspect, a video decoding apparatus is described. The video decoding apparatus includes a non-volatile storage medium and a processor, the non-volatile storage medium stores an executable program, and the processor and the non-volatile storage medium are in communication with each other, and the processor executes the executable program to implement the method in any of the first to fourth aspects or embodiments implementation from the first to the fourth aspect.

[0049] According to a ninth aspect, a computer-readable storage medium is described. The computer-readable storage medium stores the programming instructions, and when the instructions are executed on the computer, the computer is enabled to execute the method in any of the first to fourth aspects or the first to fourth aspects.

[0050] According to a tenth aspect, a computer program product including programming instructions is described. When the computer program product is executed on the computer, the computer is allowed to perform the method in any of the first to fourth aspects, or the first to fourth aspect embodiments.

[0051] According to an eleventh aspect, an embodiment of this application provides a computer program comprising program code for executing the method according to the first aspect through the fourth aspect, or any possible embodiment of the first aspect through the fourth aspect, when executed on a computer.

[0052] According to a seventh aspect, a computer-implemented inter-prediction method in video coding is described. In one embodiment, the method generates a merge candidate list based on motion information of a block of encoded pictures. The motion information defines a motion vector predictor candidate (MVP) in the merge candidate list. When the number of MVP candidates in the merge candidate list is less than the first maximum number of candidates, the method adds motion information of a motion vector predictor (HMVP) candidate based on history to the merge candidate list to obtain the first merge candidate list. When the number of MVP candidates in the first merge candidate list is less than the second maximum number of candidates, the method adds new HMVP candidate movement information to the first merge candidate list to obtain a new merge candidate list. The method obtains the merge index of the position of the optimal motion information in the new merge candidate list. The method obtains optimal motion information based on the merge index. The method determines the predicted current image block based on the optimal motion information.

[0053] It should be understood that the positive effects obtained using various aspects and the respective implementation methods are similar and not repeated.

BRIEF DESCRIPTION OF THE DRAWINGS

[0054] To more clearly describe the technical solutions in the embodiments of the present invention or in the background, the accompanying drawings necessary to describe the embodiments of the present invention or the prior art are described below.

[0055] FIG. 1A is a block diagram of an example of a video encoding and decoding system according to one embodiment.

[0056] FIG. 1B is a block diagram of an example of a video coding system according to one embodiment.

[0057] FIG. 2 is a block diagram of an exemplary encoder structure according to one embodiment.

[0058] FIG. 3 is a block diagram of an exemplary structure of a decoder according to one embodiment.

[0059] FIG. 4 is a block diagram of an example of a video encoding device according to one embodiment.

[0060] FIG. 5 is a block diagram of another example of encoding apparatus or decoding apparatus according to one embodiment.

[0061] FIG. 6 is a schematic block diagram of candidate motion spatial and temporal information according to one embodiment.

[0062] FIG. 7 is a schematic flow diagram of a history-based motion vector predictor (HMVP) method according to one embodiment.

[0063] FIG. 8 is a schematic block diagram of a table update according to one embodiment.

[0064] FIG. 9 is a schematic flowchart of an inter prediction method according to an embodiment.

[0065] FIG. 10 is a block diagram showing an exemplary structure of a content delivery system 3100 that implements a content delivery service.

[0066] FIG. 11 is a block diagram showing the structure of an example of a terminal device.

Detailed description

[0067] Embodiments of the present disclosure are described below with reference to the accompanying drawings in embodiments of the present disclosure. In the following descriptions, reference is made to the accompanying drawings, which form part of this disclosure and which show, by way of illustration, specific aspects of embodiments of the present disclosure or specific aspects in which embodiments of the present disclosure may be used. It should be understood that embodiments of the present disclosure may be used in other aspects and may include structural or logical changes not shown in the accompanying drawings. Therefore, the following detailed descriptions should not be construed as limiting, and the scope of the present disclosure is defined by the appended claims. For example, it is to be understood that disclosures made with reference to a described method may also be true of a corresponding device or system configured to perform the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more blocks, such as functional blocks, for performing the one or more method steps described (for example, one block performing one or more steps; or a plurality of blocks , each of which performs one or more of a plurality of steps), even if such one or more units are not explicitly described or illustrated in the accompanying drawings. Accordingly, for example, if a particular hardware is described in terms of one or more blocks, such as functional blocks, the corresponding method may include one or more steps to perform the functionality of one or more blocks (for example, one step to perform the functionality of one or more blocks; or a plurality of steps, each of which performs the functionality of one or more of the plurality of blocks), even if such one or more steps are not explicitly described or illustrated in the accompanying drawings. In addition, it should be understood that the features of the various exemplary embodiments and/or aspects described herein may be combined with each other, unless specifically indicated otherwise.

[0068] The technical solutions in embodiments of the present disclosure may not only apply to existing video coding standards (such as the H.264 standard and the HEVC standard), but also apply to future video coding standards (such as the H.266 standard). The terms used in the implementation of the present disclosure are only intended to explain specific embodiments of the present disclosure, but are not intended to limit the present invention. In the following, some concepts that may be used in embodiments of the present disclosure are first briefly described.

[0069] Video coding generally refers to the processing of a sequence of images that form a video or video sequence. In the field of video coding, the terms "picture", "frame", or "picture" may be used interchangeably. Video encoding used here means video encoding or video decoding. Video encoding is performed at the source side and typically involves processing (eg, by compressing) the original video images to reduce the amount of data needed to represent the video images for more efficient storage and/or transmission. Video decoding is performed on the destination side and typically includes inverse processing with respect to the encoder to recover the video images. "Encoding" of a video image in the embodiments may be understood as "encoding" or "decoding" for a video sequence. The combination of an encoding part and a decoding part is also called a (encoding and decoding) codec.

[0070] The video sequence includes a series of images (or pictures). The image is further divided into slices, and each slice is further divided into blocks. In video encoding, encoding processing is performed for each block. In some new video coding standards, the concept of a block is further expanded. For example, in the H.264 standard, there is a macroblock (MB) and this macroblock can be further divided into a plurality of prediction blocks (or partitions) that can be used for predictive coding. The High Efficiency Video Coding (HEVC) standard uses basic concepts such as a coding unit (CU), a prediction unit (PU), and a transformation unit (TU), so that a plurality of types of block units are obtained by functional separation, and the units are described with reference to a new tree structure. For example, a CU may be divided into smaller CUs based on a quadtree, and each smaller CU may continue to be divided, thereby forming a quadtree structure. CU is the basic unit for separating and encoding an encoded picture. The PU and TU also have a similar tree structure, and a PU may correspond to a prediction block and is the basic predictive coding unit. The CU is further divided into a plurality of PUs according to the partition mode. A TU may correspond to a transform unit and is the base unit for transforming the prediction residual. Essentially, all CUs, PUs, and TUs are block (or image block) concepts.

[0071] For example, in HEVC, the CTU is divided into multiple CUs using a quadtree structure referred to as a coding tree. The decision on whether to encode an image area using inter-frame (temporal) or intra-frame (spatial) prediction is made at the CU level. Each CU may be further partitioned into one, two, or four PUs based on the partition type of the PUs. Within one PU, the same prediction process is applied and the corresponding information is transmitted to the decoder based on the PU. After obtaining a residual block by applying a prediction process based on the partition type of the PU, the CU may be partitioned into TUs based on a different quadtree structure similar to the coding tree used for the CU. Recent developments in video compression technologies use a quadtree-binary-tree (QTBT) split frame to split a coding block. In the block structure of QTBT, the CU can be square or rectangular.

[0072] Here, for ease of description and understanding, an image block to be encoded in the current encoded image may be referred to as a current block. For example, when encoding, the current block is the block that is currently being encoded, and when decoding, the current block is the block that is currently being decoded. A decoded image block as a reference image used to predict the current block is called a reference block. In other words, a reference block is a block that provides a reference signal for the current block, where the reference signal represents a pixel value in the image block. The block that is in the reference picture and outputs a prediction signal for the current block may refer to a prediction block in which the prediction signal is a pixel value, a sample value, or a sample signal within the prediction block. For example, after a plurality of reference blocks have been traversed, an optimal reference block is found, the optimal reference block provides prediction for the current block, and this block is called a prediction block.

[0073] In the case of lossless video coding, the original video images can be reconstructed, which means that the reconstructed video images have the same quality as the original video images (provided that no transmission loss or other losses occur during storage or transmission). data). In the case of lossy video coding, additional compression is performed, for example, by quantization, to reduce the amount of data required to represent the video images, and the video images cannot be fully reconstructed at the decoder side, which means the quality of the reconstructed video image is lower or worse than that of video source images.

[0074] Several video coding standards, starting with H.261, refer to "hybrid lossy video codecs" (i.e., spatial and temporal sample-domain prediction is combined with 2D transform coding to apply transform-domain quantization). Each picture of a video sequence is usually divided into a set of non-overlapping blocks, and encoding is usually done at the block level. In other words, on the encoder side, the video is usually processed, that is, encoded at the block (or video block) level. For example, to generate a prediction block through spatial (intra-frame) prediction and temporal (inter-frame) prediction, the prediction block is subtracted from the current block (the block currently being processed or to be processed) to obtain a residual block, and the residual block is transformed and quantized in the transform domain to reduce the amount of data that needs to be transmitted (compressed), while on the decoder side, a part of the back processing with respect to the encoder is applied to the encoded or compressed block in order to reconstruct the current block for presentation. Additionally, the encoder duplicates the decoder processing cycle so that the encoder and decoder generate the same predictions (eg, intra predictions and inter predictions) and/or reconstruction for processing, i.e., for encoding subsequent blocks.

[0075] The system architecture used in embodiments of the present disclosure is described below. Fig. 1A is a schematic block diagram of an example of a video encoding and decoding system according to one embodiment. As shown in FIG. 1A, video encoding and decoding system 10 may include a source device 12 and a target device 14. Source device 12 generates encoded video data, and therefore source device 12 may be referred to as a video encoding device. Target device 14 may decode encoded video data generated by source device 12, and hence target device 14 may be referred to as video decoding hardware. Embodiments of source device 12, target device 14, or both source device 12 and target device 14 may include one or more processors and memory associated with the processor(s). Memory may include, without limitation, random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory, or any other medium that can be used to store desired program code in the form of program instructions. or a data structure accessible to a computer, as described in this document. Source device 12 and destination device 14 may include a variety of equipments, including a desktop computer, mobile computing equipment, a portable computer (for example, a laptop), a tablet computer, a set-top box, a handset such as a so-called "smart" phone, a TV , camera, display equipment, digital media player, game console, trip computer, wireless communication device, etc.

[0076] Although FIG. 1A depicts source device 12 and target device 14 as separate devices, the embodiment may alternatively include both source device 12 and target device 14, or the functionality of both source device 12 and target device 14, i.e. source device 12, or corresponding functions, and the target device 14, or corresponding functions. In such embodiments, source device 12 or related functions and target device 14 or corresponding functions may be implemented using the same hardware and/or software, separate hardware and/or software, or any combination thereof.

[0077] A communication connection may be made between source device 12 and target device 14 via link 13, and target device 14 may receive encoded video data from source device 12 via link 13. Link 13 may include one or more media or apparatus capable of transporting encoded video data from source device 12 to destination device 14. In one example, link 13 may include one or more media that allows source device 12 to transmit encoded video data directly to the target device 14 in real time. In this example, source device 12 may modulate encoded video data in accordance with a communication standard (eg, a wireless communication protocol) and may transmit the modulated video data to target device 14. One or more communication media may include a wireless communication medium and/or a wired communication medium. , for example, the radio frequency (RF) spectrum or one or more physical transmission lines. One or more communication media may form part of a packet data network, and the packet data network is, for example, a local area network, a public access network, or a wide area network (eg, the Internet). One or more communication media may include a router, switch, base station, or other device that facilitates communication from source device 12 to destination device 14.

[0078] Source device 12 includes an encoder 20, and in one embodiment, source device 12 may further include an image source 16, an image preprocessor 18, and a communication interface 22. In one embodiment, encoder 20, image source 16, image preprocessor 18, and communications interface 22 may be hardware components in source device 12, or may be software programs in source device 12. Specific descriptions are as follows:

[0079] Image source 16 may include or be any type of image capture device configured to capture a real image, for example, and/or any type of device to generate an image or comment (to encode screen content, some text on the screen is also considered as part of a picture or image to be encoded), such as a computer graphics processor configured to generate a computer animated image, or any type of device configured to receive and/or present a real world image or a computer animation image (for example, screen content or a virtual reality (VR)) and/or any combination of them (for example, an image of augmented reality (AR)). Image source 16 may be a camera configured to capture an image or a memory configured to store an image. Image source 16 may further include any type of interface (internal or external) for storing a previously captured or generated image and/or for acquiring or receiving an image. When the image source 16 is a camera, the image source 16 can be, for example, a local camera or an integrated camera built into the source device. When the image source 16 is a memory, the image source 16 may be a local memory or, for example, an integrated memory integrated in the source device. When the image source 16 includes an interface, this interface may be, for example, an external interface for receiving an image from an external video source. The external video source is, for example, an external image capturing device such as a camera, an external memory, or an external imaging device. The external imaging device is, for example, an external computer graphics processor, a computer, or a server. The interface may be any type of interface, such as a wired or wireless interface, or an optical interface, in accordance with any proprietary or standardized interface protocol.

[0080] An image can be viewed as a two-dimensional array or matrix of pixels (image element). A pixel in an array can also be called a sample. The number of samples in the horizontal and vertical directions (or axes) of an array or image determines the size and/or resolution of the image. Three color components are commonly used to represent a color. The image may be represented as three arrays of samples or may include them. For example, in RBG format or color space, an image includes respective arrays of red, green, and blue color samples. However, in video encoding, each pixel is typically represented in a luma/chrominance or color space format, for example, a YUV image includes a luminance component, denoted Y (sometimes alternatively referred to as L), and two chrominance components, denoted U and V. Y luminance (luma) represents the brightness or intensity of the gray level (for example, both are the same in a grayscale image), and the two chroma components (chroma) U and V represent components of chrominance or color information. Accordingly, the YUV image includes a luma sample array of luma values (Y) and two chrominance sample arrays of chrominance values (U and V). Images in RGB format can be converted or converted to YUV format and vice versa. This process is also called color conversion or conversion. If the image is monochrome, it may only include an array of brightness samples. In one embodiment, the image transmitted by the image source 16 to the image processor may also be referred to as the raw image data 17 .

[0081] The image preprocessor 18 is configured to receive the raw image data 17 and perform pre-processing of the raw image data 17 to obtain a pre-processed image 19 or pre-processed image data 19. For example, preprocessing performed by image preprocessor 18 may include cropping, color format conversion (eg, from RGB to YUV), color correction, or denoising.

[0082] Encoder 20 (also referred to as video encoder 20) is configured to receive preprocessed image data 19 and process preprocessed image data 19 using an appropriate prediction mode (such as the predictive mode in each embodiment of this disclosure) to provide encoded image data 21 (structural details of encoder 20 are further described herein below based on FIG. 2, FIG. 4, or FIG. 5). In some embodiments, encoder 20 may be configured to perform the various embodiments described herein below to implement on the encoder side the chrominance block prediction method described in this disclosure.

[0083] Communication interface 22 may be configured to receive encoded image data 21 and transmit encoded image data 21 to target device 14 or any other device (eg, memory) via communication link 13 for storage or direct retrieval. The other device may be any device used for decoding or storage. Communication interface 22 may, for example, be configured to encapsulate encoded image data 21 in an appropriate format, such as a data packet, for transmission over communication line 13.

[0084] Target device 14 includes a decoder 30, and in one embodiment, target device 14 may further include a communication interface 28, an image post-processor 32, and a display device 34. Individual descriptions are as follows:

[0085] Communication interface 28 may be configured to receive encoded image data 21 from source device 12 or any other source. Any other source is, for example, a storage device, and the storage device is, for example, an encoded image data storage device. Communication interface 28 may be configured to transmit or receive encoded image data 21 via communication link 13 between source device 12 and destination device 14 or via any type of network. Link 13 is, for example, a direct wired or wireless connection, and any type of network is, for example, a wired or wireless network, or any combination thereof, or any type of private or public network, or any combination thereof. The communication interface 28 may, for example, be configured to decapsulate the data packet transmitted via the communication interface 22 to obtain encoded image data 21.

[0086] Both communication interface 28 and communication interface 22 may be configured as unidirectional communication interfaces or bidirectional communication interfaces, and may be configured, for example, to send and receive messages to establish a connection, and to acknowledge and exchange any other information related to to a communication link and/or data transmission, such as the transmission of encoded image data.

[0087] Decoder 30 (also referred to as video decoder 30) is configured to receive encoded image data 21 and provide decoded image data 31 or decoded image 31 (structural details of decoder 30 are further described herein below based on FIG. 3, FIG. 4, or FIG. .5). In some embodiments, decoder 30 may be configured to perform the various embodiments described herein below to implement, on the decoder side, the chrominance block prediction method described in this disclosure.

[0088] The image post-processor 32 is configured to post-process decoded image data 31 (also referred to as reconstructed image data) to obtain post-processed image data 33. Post-processing performed by image post processor 32 may include color format conversion (eg, from YUV to RGB), color correction, cropping, resampling, or any other processing. The image post-processor 32 may be further configured to transfer the post-processed image data 33 to the display device 34 .

[0089] The display device 34 is configured to receive post-processed image data 33 to display the image, for example, to a user or viewer. The display device 34 may be or include any type of display configured to present the reconstructed image, such as an integrated or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other displays.

[0090] Although FIG. 1A depicts source device 12 and target device 14 as separate devices, the embodiment may alternatively include both source device 12 and target device 14, or the functionality of both source device 12 and target device 14, i.e. source device 12, or the corresponding functionality, and the target device 14, or the corresponding functionality. In such embodiments, source device 12 or related functionality and target device 14 or corresponding functionality may be implemented using the same hardware and/or software, separate hardware and/or software, or any combination thereof.

[0091] As will be apparent to one of skill in the art based on the descriptions, the existence and (precise) separation of functionality of the various blocks or functions of source device 12 and/or target device 14 shown in FIG. 1A may differ depending on the specific device and application. Source device 12 and destination device 14 may include any of a wide range of devices, including any type of portable or stationary device such as a laptop or laptop computer, mobile phone, smartphone, tablet or tablet computer, camera, desktop computer, television set-top box, television, camera, in-vehicle device, display device, digital media player, game console, video streaming device (such as a content service server or content delivery server), broadcast receiver device, or broadcast transmitter device, and cannot use or can use operating systems of any type.

[0092] Encoder 20 and decoder 30 may each be implemented as any of various suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic , equipment, or any combination thereof. If the methods are partially implemented in software, the device may store the program instruction on a suitable and non-volatile computer-readable storage medium and may execute the instruction in hardware using one or more processors to execute the methods of this disclosure. Any of the above (including hardware, software, a combination of hardware and software, and the like) may be considered one or more processors.

[0093] In some cases, the video encoding and decoding system 10 shown in FIG. 1A is merely an example, and the techniques of this disclosure may apply to video encoding settings (eg, video encoding or video decoding) that do not necessarily involve any communication between an encoding device and a decoding device. In other examples, data may be retrieved from local memory, streamed over a network, or the like. The video encoding device may encode the data and store the data in memory, and/or the video decoding device may retrieve the data from the memory and decode the data. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data into memory and/or retrieve data from memory and decode the data.

[0094] FIG. 1B is an illustrative diagram of an example of a video coding system according to one embodiment. Referring to FIG. 1B, video coding system 40 includes encoder 20 of FIG. 2 and/or decoder 30 of FIG. 3 according to one embodiment. Video coding system 40 may implement a combination of various techniques in embodiments of the present disclosure. In FIG. 1B, video encoding system 40 may include an imager 41, an encoder 20, a decoder 30 (and/or a video encoder/video decoder implemented by logic circuit 47 of processing unit 46), an antenna 42, one or more processors 43, one or more memory devices 44 and/or display device 45.

[0095] As shown in FIG. 1B, imager 41, antenna 42, processing unit 46, logic circuit 47, encoder 20, decoder 30, processor 43, memory 44, and/or display device 45 may communicate with each other. As described, although video coding system 40 is illustrated with encoder 20 and decoder 30, video coding system 40 may include only encoder 20 or only decoder 30 in various examples.

[0096] In some embodiments, antenna 42 may be configured to transmit or receive an encoded video bitstream. In addition, in some embodiments, display device 45 may be configured to present video data. In some examples, logic circuit 47 may be implemented by processing unit 46. The processing unit 46 may include ASIC logic, a graphics processor, a general purpose processor, and the like. The video encoding system 40 may also include an additional processor 43. The additional processor 43 may likewise include ASIC logic, a graphics processor, a general purpose processor, and the like. In some embodiments, logic circuitry 47 may be implemented in hardware, such as dedicated video coding hardware, and processor 43 may be implemented in general purpose software, an operating system, or the like. In addition, memory 44 may be any type of memory, such as volatile memory (eg, static random access memory (SRAM) or dynamic random access memory (DRAM)) or non-volatile memory (eg, flash memory). In a non-limiting example, memory 44 may be implemented with a cache. In some embodiments, logic 47 may access memory 44 (eg, to implement an image buffer). In other embodiments, logic circuitry 47 and/or processing unit 46 may include a memory (eg, cache) for implementing an image buffer or the like.

[0097] In some embodiments, the logic implemented encoder 20 may include an image buffer (eg, implemented by processing unit 46 or memory 44) and a graphics processing unit (eg, implemented by processing unit 46). The graphics processor may be communicatively coupled to the image buffer. The graphics processing unit may include an encoder 20 implemented by logic 47 to implement various modules that are described with reference to FIG. 2 and/or any other encoder system or subsystem described herein. The logic circuit can be configured to perform various operations described in this document.

[0098] In some embodiments, decoder 30 may be implemented by logic 47 in a similar manner to implement various modules that are described with reference to decoder 30 in FIG. 3 and/or any other decoder system or subsystem described herein. In some embodiments, the decoder 30 implemented by the logic may include an image buffer (eg, implemented by processing unit 2820 or memory 44) and a graphics processing unit (eg, implemented by processing unit 46). The graphics processor may be communicatively coupled to the image buffer. The graphics processing unit may include a decoder 30 implemented by logic 47 to implement various modules which are described with reference to FIG. 3 and/or any other decoder system or subsystem described herein.

[0099] In some embodiments, antenna 42 may be configured to receive an encoded video bitstream. As described, the encoded bitstream may include data, an indicator, an index value, mode selection data, and the like, which is related to the encoding of video frames and which is described herein, for example, data related to coding separation (for example, coefficient transform or a quantized transform coefficient, an optional indicator (as described) and/or data specifying a coding split). Video coding system 40 may further include a decoder 30 that is connected to antenna 42 and configured to decode the encoded bitstream. The display device 45 is adapted to present a video frame.

[00100] It should be understood that in this embodiment of the present disclosure, for the example described with respect to encoder 20, decoder 30 may be configured to perform the reverse process. With regard to signaling syntax elements, decoder 30 may be configured to receive and parse such syntax elements and decode associated video data accordingly. In some embodiments, encoder 20 may entropy encode syntax elements into an encoded video bitstream. In such examples, decoder 30 may parse such syntax elements and decode associated video data accordingly.

[00101] It should be noted that the method described in this embodiment of the present disclosure is mainly used for an inter prediction process, and this process exists in both encoder 20 and decoder 30. Encoder 20 and decoder 30 in this embodiment of the present the disclosures may be an encoder and decoder conforming to a video standard protocol such as H.263, H.264, HEVV, MPEG-2, MPEG-4, VP8 and VP9 or a next generation video standard protocol (eg H.266).

[00102] FIG. 2 is a schematic/conceptual block diagram of an example of an encoder according to one embodiment. In FIG. 2, the encoder 20 includes a residual calculation block 204, a transform processing block 206, a quantization block 208, an inverse quantization block 210, an inverse transform processing block 212, a reconstruction block 214, a buffer 216, a loop filter block 20, a decoded picture buffer (DPB) 230 ), a prediction processing unit 260, and an entropy encoding unit 270 . The prediction processing unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a mode selection unit 262. Inter prediction block 244 may include a motion estimator and a motion compensation block (not shown in the diagram). Encoder 20 shown in FIG. 2 may also be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec.

[00103] For example, the residual calculation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy encoding unit 270 form a forward signal path of the encoder 20, while, for example, the inverse quantization unit 210, the inverse transform processing unit 212 , restorer 214, buffer 216, loop filter 220, DPB 230, and prediction processor 260 form an encoder return signal path, where the encoder return signal path corresponds to the decoder signal path (eg, decoder 30 in FIG. 3).

[00104] Encoder 20 receives, for example, using input 202, image 201 or image block 203 for image 201, for example, an image in a sequence of images constituting a video or video sequence. The image block 203 may also be referred to as the current image block or the image block to be encoded, and the image 201 may be referred to as the current image or the image to be encoded (particularly in video encoding, to distinguish the current image from other images, other the pictures are, for example, previously encoded and/or decoded pictures in the same video sequence, i.e. a video sequence that also includes the current picture).

[00105] In one embodiment, encoder 20 may include a splitter (not shown in FIG. 2) configured to split image 201 into a plurality of blocks, such as image block 203. Image 201 is typically divided into a plurality of non-overlapping blocks. The partition block may be configured to use the same block size for all images in a video sequence and an appropriate block size grid, or vary the block size between images or subsets or groups of images and divide each image into appropriate blocks.

[00106] In one embodiment, encoder 20 prediction processor 260 may be configured to perform any combination of the separation techniques described above.

[00107] Like image 201, image block 203 can also or may be viewed as a two-dimensional array or matrix of samples with sample values, albeit smaller than image 201. In other words, image block 203 may include, for example, one array of samples ( eg, a luma array in the case of a monochrome image 201), three sample arrays (eg, one luminance array and two chrominance arrays in the case of a color image), or any other number and/or type of arrays depending on the color format used. The number of samples in the horizontal and vertical directions (or axes) of the image block 203 determines the size of the image block 203 .

[00108] The encoder 20 shown in FIG. 2 is configured to encode the image 201 block by block, for example, to perform encoding and prediction for each block 203 of the image.

[00109] The residual block 204 is configured to compute a residual block 205 based on the image block 203 and the prediction block 265 (more details about the prediction block 265 are provided herein below), for example, to obtain a residual block 205 in the sample area by subtracting the values of the samples prediction block 265 from sample values of image block 203 sample by sample (pixel by pixel).

[00110] The transform processing unit 206 is configured to apply a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values of the residual block 205 to obtain transform coefficients 207 in the transform domain. The transform coefficients 207 may also be referred to as residual transform coefficients and represent the residual block 205 in the transform region.

[00111] Transform processing unit 206 may be configured to apply DCT/DST integer approximations, such as the transforms specified for HEVC/H.265. Compared to the orthogonal DCT, such integer approximations are typically scaled several times. In order to preserve the rate of the residual block processed through forward and inverse transforms, an additional scaling factor is applied as part of the transform process. The scaling factor is usually chosen based on some constraints, for example, the scaling factor is a power of two for the shift operation, the bit depth of the transform factor, or a trade-off between accuracy and implementation cost. For example, a specific scaling factor is specified for the inverse transform on the decoder 30 side, for example, by the inverse transform processing unit 212 (and the corresponding inverse transform on the encoder 20 side, for example, by the inverse transform processing unit 212), and accordingly, the corresponding scaling factor can be specified for direct conversion on the encoder 20 side by the conversion processing unit 206 .

[00112] The quantizer 208 is configured to quantize the transform coefficients 207 to obtain quantized transform coefficients 209, for example by applying scalar quantization or vector quantization. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209. The quantization process may reduce the bit depth related to some or all of the transform coefficients 207. For example, an n-bit transform factor may be rounded to an m-bit transform factor during quantization, where n is greater than m. The degree of quantization can be modified by adjusting the quantization parameter (QP). For example, for scalar quantization, different scaling may be applied to achieve finer or coarser quantization. A smaller quantization step corresponds to a finer quantization, while a larger quantization step corresponds to a coarser quantization. The corresponding quantization step may be indicated by QP. For example, the quantization parameter may be an index to a predetermined set of suitable quantization steps. For example, a smaller quantization parameter may correspond to a finer quantization (eg, a smaller quantization step), and a larger quantization parameter may correspond to a coarser quantization (eg, a larger quantization step), and vice versa. Quantization may include dividing by a quantization step and corresponding quantization or dequantization, such as by inverse quantization 210, or may include multiplying by a quantization step. Embodiments according to some standards, such as HEVC, may use a quantization parameter to determine the quantization step. Typically, the quantization step can be calculated based on the quantization parameter using a fixed-point approximation of an equation involving division. Additional scaling factors may be introduced for quantization and dequantization to recover the residual block rate, which may be modified due to the scaling used in the fixed-point approximation of the equation for the quantization step and quantization parameter. In one exemplary embodiment, the inverse transform scale may be combined with the dequantization scale. Alternatively, customized quantization tables may be used and the signal transmitted from the encoder to the decoder, for example, in a bit stream. Quantization is a lossy operation, where a larger quantization step indicates a larger loss.

[00113] The inverse quantizer 210 is configured to apply the inverse quantization of the quantizer 208 to the quantized coefficients to obtain the dequantized coefficients 211, for example, applying, based on or using the same quantization step as the quantizer 208, an inverse quantization scheme applied by the 208 quantization. The dequantized coefficients 211 may also be referred to as the dequantized residual coefficients 211 and correspond to, although typically differ from, the transform coefficients due to loss caused by quantization, to the transform coefficients 207.

[00114] The inverse transform processing unit 212 is configured to apply an inverse transform to the transform applied by the transform processing unit 206, such as inverse DCT or inverse DST, to obtain inverse transform block 213 in the sample area. The inverse transform block 213 may also be referred to as the inverse transform dequantization block 213 or the inverse transform residual block 213 .

[00115] Restoration block 214 (e.g., adder 214) is configured to add inverse transform block 213 (i.e., reconstructed residual block 213) to prediction block 265 to obtain reconstructed block 215 in the sample area, for example, by adding sample values of the reconstructed residual block 213 and the sample values of the predictor 265 .

[00116] In one embodiment, a buffer block (or buffer) 216, such as string buffer 216, is configured to buffer or store the reconstructed block 215 and corresponding sample values, such as for intra prediction. In other embodiments, the encoder may be configured to use the unfiltered reconstructed blocks and/or corresponding sample values stored in buffer block 216 for any type of estimation and/or prediction, such as intra prediction.

[00117] For example, one embodiment of encoder 20 may be configured such that buffer block 216 is not only used to store reconstructed block 215 for intra prediction 254, but is also used for loop filter block 220 (not shown in FIG. 2), and /or so that, for example, the buffer block 216 and the decoded picture buffer block 230 form one buffer. Other embodiments may be configured to use filtered blocks 221 and/or blocks or samples from decoded picture buffer 230 (blocks or samples not shown in FIG. 2) as input or basis for intra prediction 254.

[00118] The loop filter block (or loop filter) 220 is configured to filter the reconstructed block 215 to obtain a filtered block 221, smooth pixel transitions, or improve video quality. The loop filter block 220 is designed to represent one or more loop filters, including a deblocking filter, an adaptive offset (SAO) filter, and other filters such as a two-sided filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a collective filter. Although the loop filter unit 220 is shown in FIG. 2 as a loop filter, in other configurations, the loop filter unit 220 may be implemented as a post loop filter. The filtered block 221 may also be referred to as the filtered reconstructed block 221. The decoded picture buffer 230 may store the reconstructed coded blocks after the loop filter block 220 performs filtering operations on the reconstructed coded blocks.

[00119] One embodiment of encoder 20 (respectively, loop filter block 220) may be configured to output a loop filter parameter (such as adaptive sample bias information), for example, directly or after entropy encoding performed by entropy encoding block 270 or any other an entropy encoding unit so that, for example, decoder 30 can receive the same loop filter parameter and apply the same loop filter parameter for decoding.

[00120] DPB 230 may be a reference picture memory that stores reference picture data for use in encoding video data by encoder 20. DPB 230 may be formed by any of a variety of memories such as DRAM (including synchronous DRAM (SDRAM), magnetoresistive RAM MRAM) and resistive RAM (RRAM)) or other types of storage devices. DPB 230 and buffer 216 may be provided by the same storage device or separate storage devices. In one embodiment, the DPB 230 is configured to store a filtered block 221. The DPB 230 can be further configured to store other previously filtered blocks, such as previously restored and filtered blocks 221, the same current image, or different images, such as previously restored images. , and may provide fully previously reconstructed, i.e., decoded pictures (and corresponding reference blocks and samples) and/or partially reconstructed current picture (and corresponding reference blocks and samples), for example, for inter prediction. In one embodiment, if the recovered block 215 is recovered without in-loop filtering, the DPB 230 is configured to store the recovered block 215.

[00121] The prediction processing unit 260, also referred to as the block prediction processing unit 260, is configured to receive or obtain an image block 203 (e.g., current image block 203 for the current image 201) and reconstructed image data, such as reference samples of the same ( or current) picture from buffer 216 and/or reference pictures data 231 from one or more previously decoded pictures, from decoded picture buffer 230, and process that data for prediction, i.e., to provide prediction block 265, this may be inter prediction block 245 or block 255 intra prediction.

[00122] Mode selector 262 may be configured to select a prediction mode (e.g., intra or inter prediction mode) and/or an appropriate prediction block 245 or 255 to be used as prediction block 265, to compute the residual block 205, and to recover refurbished unit 215.

[00123] In one embodiment, mode selector 262 may be configured to select a prediction mode (e.g., from among the prediction modes supported by prediction processor 260) where the prediction mode provides the best match, in other words, minimum residual (minimum residual means better compression for transmission or storage), or provides minimal signaling overhead (lower signaling overhead means better compression for transmission or storage), or considers or balances both. The mode selector 262 may be configured to determine a rate-distortion optimization (RDO) prediction mode, that is, select a prediction mode that provides minimum rate-distortion optimization, or select a prediction mode for which the corresponding rate distortion is at least satisfies the prediction mode selection criterion.

[00124] The following describes in more detail the prediction processing performed (for example, using the prediction processing module 260) and mode selection (for example, using the mode selection module 262) using the encoder 20 as an example.

[00125] As described above, encoder 20 is configured to determine or select an optimal or optimal prediction mode from a set of prediction modes (which may be predefined). The set of prediction modes may include, for example, intra prediction modes and/or inter prediction modes.

[00126] The set of intra prediction modes may include 35 different intra prediction modes, such as non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in H.265, or may include 67 different intra prediction modes, such as non-directional modes such as DC (or medium) mode and planar mode, or directional modes such as those defined in H.266 under development.

[00127] In one embodiment, the set of inter prediction modes depends on the available reference pictures (i.e., for example, the at least partially decoded pictures stored in the DPB 230 as described above), and other inter prediction parameters, for example, depends on , the entire reference image or only part of the reference image, e.g., the area of the search box around the region of the current block, is used to find the best matching reference block and/or, for example, depends on whether pixel interpolation is applied such as half/half pixel interpolation and/ or a quarter pixel. The set of inter prediction modes may include, for example, an enhanced motion vector predictor mode (Motion Vector Predictor AMVP) and a merge mode. In one embodiment, the set of inter prediction modes may include a checkpoint-based AMVP mode and a checkpoint-based merge mode, which are improved in embodiments of the present disclosure. In one example, intra predictor 254 may be configured to perform any combination of the intra prediction methods described below.

[00128] In addition to the aforementioned prediction modes, embodiments of the present disclosure may also employ skip mode and/or direct mode.

[00129] Prediction processing unit 260 may be further configured to split image block 203 into smaller block splits or sub-blocks, for example, by iteratively using quadtree splitting (QT), binary tree splitting (BT), ternary tree splitting (TT), or any combinations thereof, and to perform, for example, prediction for each of the block partitions or subblocks, where the mode selection includes selecting a tree structure of the divided image block 203 and prediction modes applied to each of the block partitions or subblocks.

[00130] Inter prediction block 244 may include a motion estimator (ME) (not shown in FIG. 2) and a motion compensation (MC) block (also not shown in FIG. 2). The motion estimator is configured to receive or obtain an image block 203 (e.g., current image block 203 for the current image 201) and a decoded image 231, or at least one or more previously reconstructed blocks, e.g., reconstructed blocks of one or more other/different previously decoded images 231 for motion estimation. For example, the video sequence may include the current picture and previously decoded pictures 31, or in other words, the current picture and previously decoded pictures 31 may be part of or form a sequence of pictures constituting the video sequence.

[00131] For example, encoder 20 may be configured to select a reference block from a plurality of reference blocks of the same picture or different pictures from a plurality of other pictures and provide, for a motion estimator (not shown in FIG. 2), a reference picture and /or provide an offset (spatial offset) between the position (X and Y coordinates) of the reference block and the position of the current block as an inter prediction parameter. This displacement is also called motion vector (MV).

[00132] The motion compensation block is configured to obtain an inter prediction parameter and perform inter prediction based on or using the inter prediction parameter to obtain an inter prediction block 245 . Motion compensation performed by a motion compensator (not shown in FIG. 2) may include sampling or generating a prediction block based on a motion vector/block determined by motion estimation (perhaps performing interpolation for sub-pixel accuracy). Interpolation filtering can generate additional pixel samples from known pixel samples, thereby potentially increasing the number of candidate prediction blocks that can be used to encode an image block. After receiving the motion vector for the PU of the current picture block, the motion compensator 246 can find the prediction block pointed to by the motion vector in one reference picture list. Motion compensation block 246 may further generate syntax elements associated with video blocks and slices for use by decoder 30 when decoding image blocks of a video slice.

[00133] In one embodiment, inter prediction block 244 may pass syntax elements to entropy encoding block 270, and the syntax elements include an inter prediction parameter (e.g., inter prediction mode selection indication information used to predict the current block after traversing a plurality of inter prediction modes). predictions). In one embodiment, if there is only one inter-prediction mode, the inter-prediction parameter may alternatively not be carried in the syntax elements. In this case, the decoder side 30 can perform decoding directly in the default prediction mode. It can be appreciated that inter predictor 244 may be configured to perform any combination of inter prediction methods.

[00134] The intra prediction block 254 is configured to obtain, for example, receiving an image block 203 (the current image block) and one or more previously reconstructed blocks, such as reconstructed neighboring blocks, of the same image for intra estimation. Encoder 20 may, for example, be configured to select an intra prediction mode from a plurality of intra prediction modes (which may be predetermined).

[00135] In one embodiment, encoder 20 may be configured to select an intra prediction mode based on an optimization criterion, such as a minimum residual (eg, an intra prediction mode providing a prediction block 255 that is most similar to the current image block 203) or distortion with minimum speed.

[00136] The intra prediction block 254 is further configured to determine, based on an intra prediction parameter, such as the selected intra prediction mode, the intra prediction block 255. In any case, after selecting the intra prediction mode for the block, the intra prediction block 254 is further configured to provide an intra prediction parameter, that is, information indicating the selected intra prediction mode for the block, to the entropy encoding block 270 . In one example, intra prediction block 254 may be configured to perform any combination of intra prediction methods.

[00137] In one embodiment, the intra prediction block 254 may pass the syntax elements to the entropy encoding block 270, and the syntax elements include an intra prediction parameter (e.g., information indicating selection of an intra prediction mode used to predict the current block after a traversal) from a set intra-prediction modes). In one embodiment, if there is only one intra prediction mode, the intra prediction parameter may alternatively not be carried in the syntax elements. In this case, the decoder side 30 can perform decoding directly in the default prediction mode.

[00138] The entropy encoding unit 270 is configured to apply an entropy encoding algorithm or scheme (e.g., variable length coding (VLC) scheme, context adaptive VLC scheme (CAVLC), arithmetic coding scheme, context adaptive binary arithmetic coding (CABAC), based on the syntax of context adaptive binary arithmetic coding (SBAC), probability interval entropy coding (PIPE), or other entropy coding method or technique) to the quantized residual coefficients 209, an inter prediction parameter, an intra prediction parameter, and/or a loop filter parameter individually or together (or incompletely) to obtain encoded picture data 21 that can be output by output 272, for example, in the form of an encoded bitstream 21. The encoded bitstream may be transmitted to video decoder 30 or archived for later transmission or retrieval by video decoder 30. Block 270 The entropy encoding may be further configured to entropy encode other syntax elements for the currently encoded video slice.

[00139] Other structural variants of video encoder 20 may be used to encode the video stream. For example, non-transform-based encoder 20 may quantize the residual signal directly without transform processing unit 206 for some blocks or frames. In another embodiment, the encoder 20 may have a quantizer 208 and an inverse quantizer 210 combined into a single block.

[00140] In one embodiment, the encoder 20 may be configured to implement the inter prediction method described in the following embodiment.

[00141] It should be understood that other structural variations of video encoder 20 may be used to encode the video stream. For example, for some image blocks or image frames, video encoder 20 may quantize the residual signal directly without being processed by transform processing block 206 and, accordingly, without being processed by inverse transform processing block 212 . Alternatively, for some image blocks or image frames, video encoder 20 does not generate residual data, and accordingly, there is no need for transform processing block 206, quantization block 208, inverse quantization block 210, and inverse transformation processing block 212 to perform processing. Alternatively, video encoder 20 may directly store the reconstructed image block as a reference block, without being processed by filter 220. Alternatively, quantizer 208 and inverse quantizer 210 in video encoder 20 may be combined together. The loop filter 220 is optional, and in the case of lossless compression coding, the transform processing block 206, the quantization block 208, the inverse quantization block 210, and the inverse transform processing block 212 are optional. It should be understood that in different application scenarios, inter predictor 244 and intra predictor 254 may be selectively activated.

[00142] FIG. 3 is a schematic/conceptual block diagram of an example of a decoder according to one embodiment. Referring to FIG. 3, video decoder 30 is configured to receive encoded image data (e.g., an encoded bitstream) 21, such as encoded by encoder 20, to obtain a decoded image 231. video slice and associated syntax elements, from video encoder 20.

[00143] In the example of FIG. 3, the decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., adder 314), a buffer 316, a loop filter 320, a decoded picture buffer 330, and a prediction processing unit 360. The prediction processing block 360 may include an inter prediction block 344, an intra prediction block 354, and a mode selection block 362. In some examples, video decoder 30 may perform a decoding step that is generally the reverse of the encoding step described with reference to video encoder 20 in FIG. 2.

[00144] The entropy decoding module 304 is configured to perform entropy decoding of the encoded image data 21 to obtain, for example, quantized coefficients 309 and/or decoded coding parameters (not shown in FIG. 3), for example, any one or all of the inter prediction parameter, an intra prediction parameter, a loop filter parameter, and/or other syntax elements (which are decoded). The entropy decoding module 304 is further configured to forward the inter prediction parameter, the intra prediction parameter, and/or other syntax elements to the prediction processing module 360 . Video decoder 30 may receive syntax elements at the video slice level and/or video block level.

[00145] The inverse quantization block 310 may have the same function as the inverse quantization block 110, the inverse transform processing block 312 may have the same function as the inverse transform processing block 212, the reconstruction block 314 may have the same function as decoder 214, buffer 316 may have the same function as buffer 216, loop filter 320 may have the same function as loop filter 220, and decoded picture buffer 330 may have the same function as decoded picture buffer 230.

[00146] Prediction processing unit 360 may include inter prediction unit 344 and intra prediction unit 354, where inter prediction unit 344 may resemble inter function prediction unit 244 and intra prediction unit 354 may resemble intra function prediction unit 254. Prediction processing unit 360 is typically configured to perform block prediction and/or obtain prediction block 365 from encoded data 21, and to receive or obtain (explicitly or implicitly) prediction-related parameters and/or information about the selected prediction mode, for example, from module 304 entropy decoding.

[00147] When a video slice is encoded as an intra-coded (I) slice, the intra prediction block 354 of the prediction processing block 360 is configured to generate a prediction block 365 for a picture block of the current video slice based on the signaled intra prediction mode and data from previously decoded blocks of the current frame or picture . When a video frame is encoded as an inter-coded (i.e., B or P) slice, the inter prediction block 344 (e.g., motion compensation block) of the prediction processing block 360 is configured to generate a predictor 365 for the video block of the current video slice based on the motion vector and other syntax elements received from block 304 entropy decoding. For inter prediction, a prediction block may be generated from one of the reference pictures in one reference picture list. Video decoder 30 may create reference frame lists, list 0 and list 1 using a default creation method based on reference pictures stored in DPB 330.

[00148] The prediction processing unit 360 is configured to determine prediction information for a video block of the current video slice by parsing the motion vector and other syntax elements, and using the prediction information to generate a prediction block for the current video block being decoded. In one example of the present invention, prediction processing unit 360 uses some of the received syntax elements to determine the prediction mode (eg, intra or inter prediction) used to encode video blocks of a video slice, type of inter prediction slice (eg, B-slice, P-slice, or GPB slice), creation information for one or more reference picture lists for the slice, motion vector for each intercoded video block of the slice, interframe prediction status for each intercoded video block of the slice, and other information for decoding video blocks in the current video slice. In another embodiment, syntax elements received by video decoder 30 from the bitstream include syntax elements in one or more of an adaptive parameter set (APS), sequence parameter set (SPS), picture parameter set (PPS), or slice header.

[00149] Inverse quantizer 310 may be configured to inverse quantize (ie, dequantize) the quantized transform coefficients provided in the bitstream and decoded by entropy decoding module 304. The inverse quantization process may include: using a quantization parameter calculated by video encoder 20 for each video block in a video slice to determine the amount of quantization to be applied and, likewise, the amount of inverse quantization to be applied.

[00150] The inverse transform processing unit 312 is configured to apply an inverse transform (eg, inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to generate residual blocks in the pixel region.

[00151] Reconstructor 314 (e.g., adder 314) is configured to add inverse transform block 313 (i.e., reconstructed residual block 313) to prediction block 365 to obtain reconstructed block 315 in the sample area, for example, by adding sample values of the reconstructed residual block 313 and sample values of prediction block 365 .

[00152] The loop filter block 320 (either in the encoding loop or after the encoding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321, smooth pixel transitions, or improve video quality. In one example, the loop filter block 320 may be configured to perform any combination of the filtering methods described below. The loop filter block 320 is designed to represent one or more loop filters, including a deblocking filter, an adaptive offset (SAO) filter, and other filters such as a two-sided filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a collective filter. Although the loop filter unit 320 is shown in FIG. 3 as a loop filter, in other configurations, the loop filter unit 320 may be implemented as a post loop filter.

[00153] The decoded video blocks 321 in a given frame or picture are then stored in the decoded picture buffer 330, which stores reference pictures used for subsequent motion compensation.

[00154] The decoder 30 is configured, for example, to output the decoded image 31 using the output 332 to be presented to or viewed by the user.

[00155] Other embodiments of video decoder 30 may be configured to decode the compressed bitstream. For example, decoder 30 may generate an output video stream without loop filter block 320 . For example, a non-transform-based decoder 30 may inversely quantize the residual signal directly without the inverse transform processing unit 312 for some blocks or frames. In another embodiment, video decoder 30 may have an inverse quantizer 310 and an inverse transform processor 312 combined into one unit.

[00156] In one embodiment, the decoder 30 is configured to implement the inter prediction method described in the following embodiment.

[00157] It should be understood that other structural variations of video decoder 30 may be configured to decode the encoded video bitstream. For example, video decoder 30 may generate an output video stream without processing by filter 320. Alternatively, for some image blocks or image frames, entropy decoding block 304 of video decoder 30 does not obtain quantized coefficients through decoding, and accordingly, there is no need for inverse quantization block 310 and block 312 inverse transformation processing to perform processing. The loop filter 320 is optional, and in the case of lossless compression, the inverse quantizer 310 and the inverse transform processor 312 are optional. It should be understood that in different application scenarios, the inter prediction block and the intra prediction block may be selectively activated.

[00158] It should be understood that at encoder 20 and decoder 30, the processing result for a procedure may be output to the next procedure after further processing. For example, after a procedure such as interpolation filtering, motion vector derivation, or loop filtering, an operation such as clipping or shifting is further performed on the processing result of the corresponding procedure.

[00159] For example, the motion vector of the checkpoint of the current image block, derived from the motion vector of the neighboring affine-coded block, may be further processed. It's not limited here. For example, the range of motion vector values is limited to a particular bit depth. Assuming that the allowed bit depth of the motion vector is bitDepth, the range of the motion vector is -2^(bitDepth-1) to 2^(bitDepth-1)-1, where the symbol "^" represents the degree. If bitDepth is 16, the range of values is -32768 to 32767. If bitDepth is 18, the range of values is -131072 to 131071. The range of values can be limited in the following two ways.

[00160] Method 1: Remove the overflowed MSB of the motion vector:

ux= (vx+2 ^bitDepth ) % 2 ^bitDepth

vx = (ux >= 2 ^bitDepth-1 ) ? (ux − 2 ^bitDepth ): ux

uy= (vy+2 ^bitDepth ) % 2 ^bitDepth

vy = (uy >= 2 ^bitDepth-1 ) ? (uy − 2 ^bitDepth ): uy

[00161] For example, the vx value is -32769, 32767 is obtained using the formulas above. The value is stored in the computer as two's complement, the two's complement of -32769 is 1.0111,1111,1111,1111 (17 bits), and the computer handles the overflow by discarding the most significant bit. Therefore, the value of vx is 0111,1111,1111,1111, that is, 32767, which is consistent with the result obtained by processing using the above formulas.

[00162] Method 2: Perform motion vector clipping as shown in the following formulas:

vx = Clip3(-2 ^bitDepth-1 , 2 ^bitDepth-1 -1, vx)

vy = Clip3(-2 ^bitDepth-1 , 2 ^bitDepth-1 -1, vy)

where Clip3 is defined to specify clipping of the z value to the range [x, y]:

"

Clip3(x, y, z) =

"

[00163] FIG. 4 is a schematic structural diagram of a video encoding device according to one embodiment. In FIG. 4, a video encoding device 400 (eg, a video encoding device or a video decoding device) is suitable for implementing the embodiment described here. In one embodiment, video encoding device 400 may be a video decoder (eg, decoder 30 in FIG. 1A) or a video encoder (eg, encoder 20 in FIG. 1A). In another embodiment, video encoding device 400 may be one or more components of decoder 30 in FIG. 1A or encoder 20 in FIG. 1A.

[00164] The video encoding device 400 includes: input ports 410 and a receiver (Rx) unit 420 that are configured to receive data; a processor, logical unit, or central processing unit (CPU) 430 that is configured to process data; a transmitter (Tx) unit 440 and output ports 450 that are configured for data transmission; and a memory 460 configured to store data. The video encoding device 400 may further include optical-electrical components and electrical-optical (EO) components connected to input ports 410, receiver unit 420, transmitter unit 440, and output ports 450 for output or input of optical or electrical signals.

[00165] The processor 430 is implemented in hardware and software. Processor 430 may be implemented as one or more CPU chips, cores (eg, a multi-core processor), FPGA, ASIC, and DSP. Processor 430 communicates with input ports 410, receiver unit 420, transmitter unit 440, output ports 450, and memory 460. Processor 430 includes an encoding module 470 (eg, an encoding module or a decoding module). The encoding unit 470 implements the embodiments disclosed herein and implements the chrominance block prediction method provided in the embodiments of the present disclosure. For example, encoding module 470 implements, processes, or provides various encoding operations. Therefore, the inclusion of the encoding module 470 provides a significant improvement in the functions of the video encoding device 400 and affects the transformation of the video encoding device 400 into another state. Alternatively, encoding module 470 is implemented as instructions stored in memory 460 and executed by processor 430.

[00166] Memory 460 includes one or more disks, tape drives, and solid state drives and can be used as an overflow storage device, for storing programs when such programs are selectively executed, and for storing instructions and data that are read at run time. programs. Memory 460 may be volatile and/or non-volatile and may be ROM, RAM, ternary content addressable memory (TCAM), and/or static random access memory (SRAM).

[00167] FIG. 5 is a simplified block diagram of a hardware according to one embodiment. In FIG. 5, hardware 500 can be used as one or both of source device 12 and target device 14 in FIG. 1A according to one embodiment. Apparatus 500 may implement the methods of this disclosure. In other words, FIG. 5 is a schematic block diagram of an embodiment of an encoding device or a decoding device (called an encoding device) according to one embodiment. Continuing to refer to FIG. 5, encoding hardware 500 may include a processor 510, a memory 530, and a bus system 550. The processor is connected to memory using a bus system, the memory is configured to store an instruction, and the processor is configured to execute an instruction stored in memory. The program code is stored in the memory of the encoder. The processor may call program code stored in memory to perform video encoding or decoding techniques described herein, and in particular, various new inter prediction techniques. To avoid repetition, details are again omitted here.

[00168] In one embodiment, processor 510 may be a central processing unit (CPU), or processor 510 may be another general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA), or other programmable a logic device, a discrete gated or transistorized logic device, a discrete hardware component, and the like. A general purpose processor may be a microprocessor, or the processor may be any conventional processor or the like.

[00169] The memory 530 may include a read only memory (ROM) or a random access memory (RAM) device. Any other suitable type of storage device may also be used as memory 530. Memory 530 may include code and data 531 accessed by processor 510 using bus 550. Memory 530 may further include operating system 533 and application program 535 The application program 535 includes at least one program that allows the processor 510 to perform the video encoding or decoding method (in particular, the inter prediction method described herein) described herein. For example, application program 535 may include applications 1 to N, and further includes a video encoding or decoding application (referred to as a video encoding application) that performs the video encoding or decoding method described herein.

[00170] The bus system 550 may further include a power bus, a control bus, a status signal bus, and the like. in addition to the data bus. However, for a clear description, the various types of tires are referred to in the drawing as busbar system 550.

[00171] In one embodiment, encoding hardware 500 may further include one or more output devices, such as a display 570. In one example, display 570 may be a touch display that combines a display and a touch unit that operatively recognizes touch input. Display 570 may be connected to processor 510 via bus 550.

[00172] Forward prediction means selecting, for the current coding block, a reference picture from a plurality of reference pictures from a set of reference pictures in direct order to obtain a reference block. Reverse prediction means selecting, for the current coding block, a reference picture from the reference picture set in reverse order to obtain the reference block. When the bidirectional prediction method is used, the current coding block has two reference blocks. Each reference block requires a motion vector and a reference frame index to specify. Then, a pixel value prediction value for the pixel in the current block is determined based on the pixel values for the pixels in the two reference blocks.

[00173] The inter prediction mode may include, without limitation, an enhanced motion vector predictor mode (Motion Vector Predictor AMVP) and a merge mode.

[00174] In AMVP mode, a list of candidate motion vectors is first created using the motion information of a coded block that is spatially or temporally adjacent to the current coding block, then the optimal motion vector is determined from the list of candidate motion vectors as a Motion Vector Predictor (MVP) of the current coding block . The rate distortion cost is calculated using formula (1) where J is the rate distortion cost RD Cost, SAD is the sum of the absolute differences (SAD) between the predicted pixel values and the original pixel values obtained by motion estimation performed using a candidate vector predictor motion, R is the bit rate, and λ is the Lagrange multiplier. The encoder side receives and transmits, to the decoder side, the index value of the selected motion vector predictor in the list of candidate motion vectors and the index value of the reference frame. Additionally, a motion search is performed in a neighborhood centered on the MVP to obtain the actual motion vector of the current coding block. The encoder side transmits the difference (eg, motion vector difference) between the MVP and the actual motion vector to the decoder side.

J=SAD+λR (1)

[00175] In the merge mode, a candidate motion information list is first created using motion information of an encoded block that is spatially or temporally adjacent to the current coding block, then optimal motion information is determined from the candidate motion information list using distortion rate overhead and is used as the motion information of the current coding block, and then an index value (denoted as a merge index) of the position of the optimal motion information in the candidate motion information list is obtained and transmitted to the decoder side. The fusion index can be used to obtain optimal traffic information. The spatial and temporal movement information of the current coding block candidate is shown in FIG. 6 which is a schematic block diagram of the spatial and temporal candidate motion information according to one embodiment. The spatial candidate motion information comes from five spatially contiguous blocks (A0, A1, B0, B1 and B2). If the adjacent block is not available or is in intra-coding mode, the adjacent block is not added to the Candidate Movement Information List. The temporal information about the motion candidate of the current coding block is obtained after the MV of the corresponding block position in the reference frame is scaled based on the Picture Order Count (POC) of the reference frame and the current frame. First, it is determined whether the block whose position is T in the reference frame is available. If the block is not available, the block with position C is selected.

[00176] The motion information includes at least one of the reference picture information and a motion vector. The reference picture information may include at least one of unidirectional/bidirectional prediction information, a reference picture list, and/or a reference picture index corresponding to the reference picture list. The motion vector refers to the positional displacement in the horizontal and vertical directions.

[00177] A set of history-based motion vector predictor candidates (HMVP motion vector predictor) is entered into the merge candidate list. The introduction of HMVP is as follows:

[00178] HMVP Motion Vector Predictor Method for Outer Coding: In HMVP, an HMVP candidate table is maintained and updated on the fly. After decoding the non-affine outer-coded block, the table is updated by adding the associated new motion information as a new HMVP candidate to the last table entry. A restricted FIFO rule is used to remove and add records to a table. HMVP candidates can apply to either the merger candidate list or the AMVP candidate list.

[00179] In universal video coding (VVC), motion vectors of inter-coded non-affine blocks can be signaled in two ways: in enhanced motion vector predictor (AMVP) mode or in fusion mode. For the AMVP mode, the difference between the real motion vector and the motion vector predictor (MVP), the reference index, and the MVP index related to the AMVP candidate list is signaled. For the merge mode, a merge index relating to the list of merge candidates is specified, and all traffic information associated with the merge candidate is inherited.

[00180] Both the AMVP candidate list and the merge candidate list are derived from temporally or spatially adjacent encoded blocks. Under normal testing conditions, up to six merger candidates and up to two AMVP candidates can be added to the motion vector predictor candidate list.

[00181] An HMVP candidate is defined as motion information of a previously encoded block. A multi-candidate HMVP table is maintained during encoding/decoding. The table is cleared when a new slice is found. Whenever there is an outer-coded non-affine block, the corresponding motion information is added to the last table entry as a new HMVP candidate. The overall coding stream is shown in FIG. 7.

[00182] In this description, with reference to FIG. 8 (which is a schematic flowchart of updating a table according to one embodiment), the size S of the table is set to 6, indicating that up to six HMVP candidates can be added to the table. When inserting a new motion candidate into a table, a limited FIFO rule is used where a redundancy check is first applied to determine if there is an identical HMVP in the table. If found, the identical HMVP is removed from the table and all HMVP candidates are subsequently moved forward, i.e. with indexes decremented by 1.

[00183] HMVP candidates may be used in the process of creating a merge candidate list. The last few HMVP candidates in the table are checked in order and inserted into the list of candidates after the TMVP candidate. The reduction applies to HMVP candidates for a spatial or temporal merge candidate, with the exception of a subblock motion candidate (ie, ATMVP).

[00184] To reduce the number of reduction operations, three simplifications are introduced:

[00185] 1) The number of HMPV candidates to be tested, denoted by the letter L, is given as follows:

L = (N <=4) ? M: (8 - N) (1),

where N indicates the number of non-subblock merge candidates available and M indicates the number of available HMVP candidates in the table.

[00186] 2) Additionally, once the total number of available merge candidates reaches the designated maximum allowable number of merge candidates minus 1, the process of creating the merge candidate list from the HMVP list ends.

[00187] 3) Additionally, the number of pairs for combined derivation of dual prediction merge candidates is reduced from 12 to 6.

[00188] Similarly, HMVP candidates may also be used in the process of creating an AMVP candidate list. The motion vectors of the last K HMVP candidates in the table are inserted after the TMVP candidate. Only HMVP candidates with the same reference picture as the target AMVP reference picture are used to create the list of AMVP candidates. A reduction applies to HMVP candidates. In this padding, K is set to 4 while the size of the AMVP list remains unchanged, i.e. 2.

[00189] The HMVP candidates in the HMVP candidate set are filled in the merge candidate list until the number of candidates in the merge candidate list reaches the maximum number of candidates.

[00190] History Based MVP Merge Candidates (HMVPs) are added to the merge list after the spatial MVPs and TMVPs. In this method, motion information of a previously encoded block is stored in a table and used as the MVP for the current CU. A multi-candidate HMVP table is maintained during the encoding/decoding process. The table is reset (cleared) when a new CTU row is encountered. Whenever there is an outer-coded CU that is not a sub-unit, the corresponding motion information is added to the last table entry as a new HMVP candidate.

[00191] The size S of the HMVP table is set to 6, indicating that up to 6 history-based MVP candidates (HMVP) can be added to the table. When inserting a new motion candidate into a table, a limited first-in, first-served (FIFO) rule is used that first applies a redundancy check to determine if there is an identical HMVP in the table. If found, the identical HMVP is removed from the table and all HMVP candidates are subsequently moved forward.

[00192] HMVP candidates may be used in the process of creating a merge candidate list. The last few HMVP candidates in the table are checked in order and inserted into the list of candidates after the TMVP candidate. A redundancy check is applied to HMVP candidates for a spatial or temporal merge candidate.

[00193] To reduce the number of redundancy check operations, the following simplifications are introduced:

[00194] The number of HMPV candidates used to generate the merge list is set to ((N <= 4 ) × M: (8 - N), where N indicates the number of existing candidates in the merge list and M indicates the number of available HMVP candidates in the table .

[00195] Once the total number of available merge candidates reaches the maximum allowed merge candidates minus 1, the process of generating the merge candidate list from the HMVP ends.

[00196] However, a pairwise average candidate is also entered into the merge candidate list. The introduction of a pairwise mean candidate looks like this:

[00197] Pairwise average candidates are generated by averaging the predefined candidate pairs in the current merge candidate list, and the predefined pairs are defined as {(0, 1), (0, 2), (1, 2), (0, 3), ( 1, 3), (2, 3)}, where the numbers represent the merge indices in the merge candidate list. The average motion vectors are calculated separately for each reference list. If both motion vectors are available in the same list, the two motion vectors are averaged even if they point to different reference pictures; if only one motion vector is available, one motion vector is used directly; if no motion vector is available, this list remains invalid. Pairwise average candidates replace combined candidates in the HEVC standard. The complexity analysis of the candidate pairwise means is presented in Table 1. For the worst case additional calculations for averaging (last column in Table 1), four additions and four shifts are needed for each pair (MVx and MVy in L0 and L1), as well as four reference index comparisons for each pair (refIdx0 is valid and refIdx1 is valid in L0 and L1). There are six pairs, resulting in 24 additions, 24 shifts, and 24 pivot index comparisons. The combined candidates in the HEVC standard use two reference index comparisons for each pair (refIdx0 is valid in L0 and refIdx1 is valid in L1) and there are 12 pairs, resulting in a total of 24 reference index comparisons.

Table 1 Analysis of operations for pairwise average candidates

Merge list size Maximum number of potential candidates Maximum number of candidate comparisons Maximum number of MV calculations Maximum number of temporary candidates Additional local buffer Maximum number of memory accesses Other 6, 8, 10 6 0 0 0 0 0 Replacing combined HEVC candidates, additional calculations required for averaging

[00198] Pairwise average candidates are generated by averaging the predefined candidate pairs in the existing merge candidate list, and the predefined pairs are defined as {(0, 1), (0, 2), (1, 2), (0, 3), ( 1, 3), (2, 3)}, where the numbers represent the merge indices in the merge candidate list. The average motion vectors are calculated separately for each reference list. If both motion vectors are available in the same list, the two motion vectors are averaged even if they point to different reference pictures; if only one motion vector is available, it is directly used; if no motion vector is available, this list is invalid.

[00199] When the merge list is not populated after pairwise average merge candidates are added, zero MVPs are inserted at the end until the maximum number of merge candidates is found.

[00200] FIG. 9 is a schematic flowchart of an inter prediction method according to an embodiment. The method shown in Figure 9 allows the encoder to process image blocks larger than a preset size associated with the encoder (eg buffer size). The method may be implemented using hardware, software, or any combination thereof. The method may be implemented by block 244 or 344 inter prediction. The method may be a decoding method or an encoding method. As shown in FIG. 9, the method includes the following steps.

[00201] Operation S901. (An encoder (such as encoder 20 or decoder 30 in FIG. 1) or a video coding system) creates a list of merging candidates for the current image block. The current picture block may be a coding block, CU, PU, or TU, and so on. The current image block can be of any size.

[00202] Operation S902. (The system) adds motion information of the HMVP candidate to the merge candidate list if the amount of motion information of the candidate in the merge candidate list is less than the maximum number of candidates in the merge candidate list minus N, where N is a positive integer.

[00203] Operation S903. (System) obtains information about the movement of the current image block based on the list of merging candidates.

[00204] Operation S904. (System) determines a predicted block of the current image block based on motion information of the current image block.

[00205] In one embodiment, N may be 1, 2, or 3.

[00206] In one embodiment, obtaining motion information of the current image block based on the merge candidate list may comprise: obtaining a merge index of the current image block or a merge index for the merge candidate list; obtaining information about the movement of the current image block based on the list of merge candidates and the merge index. Wherein, obtaining the merge index of the current image block or the merge index for the merge candidate list may comprise: obtaining the merge index by bitstream parsing. At the same time, obtaining information about the movement of the current image block based on the list of candidates for merge and the index of the merge may include: obtaining information about the movement from the list of candidates for the merge using the merge index. Meanwhile, the merge index may be used to indicate the position of the motion information in the merge candidate list.

[00207] In one embodiment, obtaining motion information of the current image block based on the merge candidate list may comprise: determining motion information from the merge candidate list using rate distortion overhead. The method may further comprise: encoding a merge index for a list of merge candidates in the bitstream.

[00208] In one embodiment, the predicted block may be referred to as a prediction block.

[00209] In one embodiment, the current image block may be a coding block, a coding unit, or a prediction block.

[00210] In one embodiment, the maximum number of candidates in the merge candidate list may be referred to as the maximum number of merge motion vector predictor (MVP) candidates.

[00211] In one embodiment, the HMVP candidate movement information may be referred to as history-based merge candidates or history-based merge candidates (HMVPs).

[00212] In one embodiment, the method further includes: adding movement information of pairwise average candidates to the merge candidate list in the case where the amount of movement information of candidates in the merge candidate list is equal to the maximum number of candidates in the merge candidate list minus N .

[00213] In one embodiment, candidate pairwise average motion information is generated by averaging a predetermined pair of MVP candidates in the merge candidate list.

[00214] In one embodiment, the movement information of a pairwise average candidate may be referred to as a pairwise average candidate.

[00215] In one embodiment, in a merge candidate list, the available candidates are imported into the list in a predefined candidate scan order. When the merge candidate list does not reach the maximum number of candidates in the merge list minus N (N≥1) after one HMVP candidate in the HMVP candidate set is imported into the merge list, the HMVP candidate in the HMVP candidate set continues to be imported into the merge list.

[00216] In another embodiment, in a merge candidate list, the available candidates are imported into the list in a predefined candidate scan order. When the merge candidate list does not reach the maximum number of candidates in the merge list minus N (N≥1) after one HMVP candidate in the HMVP candidate set is imported into the merge list, the HMVP candidate in the HMVP candidate set continues to be imported into the merge list. When the number of candidates in the merge list is equal to the maximum number of candidates minus N (N≥1), the pairwise average candidate is imported into the merge list.

[00217] Based on the same inventive idea as the above method, an embodiment of the present invention further provides an inter prediction apparatus, where the inter prediction apparatus includes a creation unit and a prediction block determination unit. Note that the creation block and the prediction block determination block may be applied to the inter prediction process on the encoder side or the decoder side. In particular, on the encoder side, these blocks can be applied to the inter prediction block 244 in the prediction processing block 260 of the aforementioned encoder 20; on the decoder side, these blocks may be applied to the inter prediction block 344 in the prediction processing block 360 of the aforementioned decoder 30.

[00218] It should be further noted that for the specific contents of the motion information determination block and the prediction block determination block, refer to the descriptions of the above contents, including the Summary of the Invention. For the sake of brevity of this disclosure, the details are again not described here.

[00219] In one embodiment, the creation block and the prediction block may be implemented in hardware, software, or any combination thereof.

In one embodiment, the create block is configured to: create a list of merge candidates of the current image block; adding HMVP candidate movement information to the merge candidate list if the number of candidate movement information in the merge candidate list is less than the maximum number of candidates in the merge candidate list minus N, where N is a positive integer; a prediction block configured to: obtain a merge index of the current image block; obtaining motion information of the current image block based on the merge candidate list and the merge index; and determining a predicted block of the current image block based on the motion information of the current image block.

[00220] It should be further noted that for the specific contents of the motion information determination block and the prediction block determination block, refer to the descriptions of the above contents, including the Summary of the Invention. For the sake of brevity of this description, the details are again not described here.

[00221] The following is an explanation of the applications of the encoding method as well as the decoding method as shown in the above embodiments, and the system using them.

[00222] FIG. 10 is a block diagram showing a content supply system 3100 for realizing a content distribution service. This content delivery system 3100 includes a capture device 3102, a terminal device 3106, and optionally includes a display 3126. The capture device 3102 communicates with the terminal device 3106 over a communication line 3104. The communication link may include the communication channel 13 described above. Link 3104 includes, but is not limited to, WIFI, Ethernet, cable, wireless (3G/4G/5G), USB, or any combination thereof, and the like.

[00223] The capture device 3102 generates data and may encode the data in an encoding manner as shown in the above embodiments. Alternatively, the capture device 3102 may distribute the data to a streaming server (not shown in the figures) and the server encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 includes, without limitation, a camera, a smartphone or tablet, a computer or a laptop computer, a video conferencing system, a PDA, a vehicle-mounted device, or a combination of any of these, or the like. For example, capture device 3102 may include source device 12 as described above. When the data includes video, the video encoder 20 included in the capture device 3102 may actually perform video encoding processing. When the data includes audio (ie, voice), the audio encoder included in the capture device 3102 may actually perform audio encoding processing. For some practical scenarios, the capture device 3102 distributes encoded video and audio data by multiplexing them together. For other practical scenarios, such as in a videoconferencing system, encoded audio data and encoded video data are not multiplexed. The capture device 3102 separately distributes the encoded audio data and the encoded video data to the terminal device 3106.

[00224] In the content delivery system 3100, the terminal device 310 receives and reproduces encoded data. The terminal device 3106 may be a device with the ability to receive and retrieve data, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR)/digital video recorder (DVR) 3112, a television 3114, a set-top box (STB) 3116, a videoconferencing system 3118 , a video surveillance system 3120, a personal digital assistant (PDA) 3122, a vehicle-mounted device 3124, or a combination thereof, or the like capable of decoding the aforementioned encoded data. For example, terminal device 3106 may include target device 14 as described above. When the encoded data includes video, the video decoder 30 included in the terminal apparatus is given priority to perform video decoding. When the encoded data includes audio, an audio decoder included in the terminal device is given priority to perform audio decoding processing.

[00225] For a display terminal device such as a 3108 smartphone or tablet, a 3110 computer or laptop, a 3112 Network Video Recorder (NVR)/Digital Video Recorder (DVR), a 3114 TV, a 3122 Personal Digital Assistant (PDA), or a 3124 Vehicle Mounted Device , the terminal device can transmit the decoded data to its display. For a terminal device not equipped with a display, such as STB 3116, video conferencing system 3118, or video surveillance system 3120, it establishes contact with an external display 3126 to receive and display decoded data.

[00226] When each device in this system performs encoding or decoding, an image encoding device or an image decoding device may be used as shown in the above embodiments.

[00227] FIG. 11 is a diagram showing the structure of an example of a terminal device 3106. After the terminal device 3106 receives a stream from the capturing device 3102, the protocol execution unit 3202 parses the transmission protocol of the stream. The protocol includes, without limitation, Real Time Streaming Protocol (RTSP), Hypertext Transfer Protocol (HTTP), HTTP Real Time Streaming Protocol (HLS), MPEG-DASH, Real Time Transport Protocol (RTP), real-time messaging (RTMP) or any combination thereof or the like.

[00228] After the protocol execution block 3202 processes the stream, a stream file is created. The file is output to the demultiplexer 3204 . A demultiplexer 3204 may separate the multiplexed data into encoded audio data and encoded video data. As described above, for some practical scenarios, such as in a videoconferencing system, encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is transmitted to the video decoder 3206 and audio decoder 3208 without using the demultiplexer 3204.

[00229] Through the demultiplexing processing, a video elementary stream (ES), audio ES, and optionally subtitles are generated. The video decoder 3206, which includes the video decoder 30 as explained in the above embodiments, decodes the video ES using the decoding method as shown in the above embodiments to generate a video frame, and supplies this data to the synchronous block 3212. The audio decoder 3208 decodes the audio ES, to generate an audio frame and feeds the data to synchronous block 3212. Alternatively, a video frame may be stored in a buffer (not shown in FIG. 11) before being fed to synchronous block 3212. Similarly, an audio frame may be stored in a buffer (not shown in FIG. 11 ) before feeding it to the synchronous block 3212.

[00230] The synchronous block 3212 synchronizes the video frame and the audio frame and supplies video/audio to the video/audio display 3214. For example, the synchronous block 3212 synchronizes the presentation of video and audio information. The information may be encoded in a syntax using timestamps regarding the presentation of encoded audio and visual data and timestamps regarding the delivery of the data stream itself.

[00231] If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles and synchronizes them with the video frame and audio frame and transmits the video/audio/subtitles to the video/audio/subtitle display 3216.

[00232] The present invention is not limited to the above system, and either the image encoding device or the image decoding device in the above embodiments may be included in another system such as an automobile system.

[00233] Those skilled in the art can appreciate that the functions described with reference to the various illustrative logical blocks, modules, and algorithm steps disclosed and described herein may be implemented in hardware, software, firmware, or any of them. combination. If implemented in software, the functions described with reference to various illustrative logical blocks, modules, and steps may be stored or transmitted on a computer-readable medium as one or more instructions or code and executed by a hardware processing unit. A computer-readable medium may include a computer-readable medium that corresponds to a tangible medium, such as a storage medium, or a communication medium, including any medium that facilitates transfer of a computer program from one place to another (eg, a communication protocol). Thus, a computer-readable medium can generally correspond to (1) a tangible computer-readable storage medium that is non-volatile, or (2) a communication medium such as a signal or a medium. A storage medium can be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the techniques described herein. The computer program product may include a computer readable medium.

[00234] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage device, magnetic disk storage device or other magnetic storage device, flash -memory or any other medium that can be used to store the desired program code in the form of an instruction or data structure and that can be accessed by a computer. Additionally, any connection is properly referred to as a computer-readable medium. For example, if an instruction is transmitted from a website, server, or other remote source via coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, microwave, coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the media definition. However, it should be understood that a computer-readable storage medium and a storage medium do not include connections, media, signals, or other temporary media, but actually mean non-volatile tangible storage media. Discs and floppy disks used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), and Blu-ray disc. Floppy disks usually play data magnetically, while disks play data optically using lasers. Combinations of the foregoing should also be included within the scope of the computer-readable medium.

[00235] An instruction may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein may refer to any of the aforementioned structures, or any other structures suitable for implementing the techniques described herein. Furthermore, in some aspects, the functions described with reference to various illustrative logical blocks, modules, and steps described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or may be included in combined codec. In addition, the methods may be fully implemented in one or more circuits or logic elements.

[00236] The techniques of this disclosure may be implemented in a wide variety of apparatuses or devices, including a wireless telephone, an integrated circuit (IC), or an IC stack (eg, a chipset). This document describes various components, modules, or blocks to emphasize the functional aspects of devices configured to perform the disclosed technologies, but are not necessarily implemented in different hardware blocks. In fact, as described above, various blocks may be combined in combination with suitable software and/or firmware into a codec hardware block, or may be provided by cooperating hardware blocks (including one or more of the processors described above).

[00237] In the above embodiments, the descriptions of each embodiment have appropriate emphasis. For the part that is not described in detail in the embodiment, refer to the corresponding descriptions in other embodiments.

[00238] The above descriptions are merely examples of specific implementations of this disclosure, but are not intended to limit the scope of protection of this disclosure. Any change or replacement easily discovered by one of ordinary skill in the art within the technical scope disclosed herein should fall within the protection scope of this disclosure. Therefore, the scope of this disclosure is subject to the scope of the claims.

Claims (29)

1. An inter prediction method, the method comprising: creating a list of candidates for merging the current image block; adding at least one history-based motion vector predictor (HMVP) candidate to the merge candidate list to obtain a new merge candidate list in case the number of merge candidates in the merge candidate list is less than the maximum number of candidates minus N, where N is a positive integer; obtaining motion information of the current image block based on the new merge candidate list; And determining a predicted block of the current image block based on the motion information of the current image block. 2. The method according to claim 1, wherein obtaining information about the movement of the current image block based on the new list of merging candidates comprises: adding a pairwise average merge candidate to the new merge candidate list to obtain a modified merge candidate list when the number of merge candidates in the new merge candidate list is equal to the maximum number of candidates minus N; obtaining information about the movement of the current image block based on the modified list of merging candidates. 3. The method of claim 2, wherein the pairwise average merge candidate is generated by averaging a predetermined pair of merge candidates in the new merge candidate list. 4. The method according to any one of paragraphs. 2, 3, in which obtaining information about the movement of the current image block based on the modified list of candidates for merging contains: getting the merger candidate index; obtaining movement information from the modified merger candidate list using the merger candidate index. 5. The method according to any one of paragraphs. 2, 3, wherein obtaining the motion information of the current image block based on the modified merge candidate list comprises: determining motion information from the modified merge candidate list using rate distortion overhead. 6. External prediction equipment, comprising: CPU; And memory attached to the processor to store instructions that, when executed by the processor, cause the processor to perform operations, the operations comprising: creating a list of candidates for merging the current image block; adding at least one history-based motion vector predictor (HMVP) candidate to the merge candidate list to obtain a new merge candidate list, in case the number of merge candidates in the merge candidate list is less than the maximum number of candidates minus N , where N is a positive integer; obtaining information about the movement of the current image block based on the list of merging candidates; And determining a predicted block of the current image block based on the motion information of the current image block. 7. The apparatus according to claim 6, wherein obtaining information about the movement of the current image block based on the new list of merging candidates comprises: adding a pairwise average merge candidate to the new merge candidate list to obtain a modified merge candidate list when the number of merge candidates in the new merge candidate list is equal to the maximum number of candidates minus N; obtaining information about the movement of the current image block based on the modified list of merging candidates. 8. The apparatus of claim 7, wherein the pairwise average merge candidate is generated by averaging a predetermined pair of merge candidates in the new merge candidate list. 9. Equipment according to any one of paragraphs. 7, 8, and the operation of obtaining information about the movement of the current image block based on the modified list of candidates for merging contains: getting the merger candidate index; obtaining information about the movement from the list of merge candidates using the merge candidate index. 10. Equipment according to any one of paragraphs. 7, 8, wherein the operations of obtaining motion information of the current image block based on the modified merge candidate list comprises: determining motion information from the modified merge candidate list using rate distortion overhead. 11. A non-volatile, tangible, computer-readable storage medium that stores program code that, when executed by a computing device, causes the computing device to perform the method of any one of claims. 1-5.

Images