RU2797574C2 - Encoder, decoder and corresponding methods using intra-frame block copy merge list - Google Patents

Abstract

FIELD: image processing.

SUBSTANCE: invention relates to a shared list for prediction when encoding and decoding video. Methods and devices for constructing a list of merge candidates for intraframe block copying (IBC) mode are proposed, and they comprise the following steps: inserting the block vector of the left neighbour block of the current block into the initial merge list of the current block, when the left neighbour block is available and the left neighbour block uses IBC mode; inserting the block vector of the aforementioned neighbour block of the current block into the initial merge list, when the aforementioned neighbour block is available, the aforementioned neighbour block uses the IBC mode, and the block vector of the aforementioned neighbour block is not the same as the block vector of the left neighbour block; inserting the block vector of the last candidate in the history-based motion vector predictor (HMVP) into the initial merge list when the block vector of the aforementioned neighbour block is not the same as the block vector of the last HMVP candidate, and when the block vector of the left neighbour block is not the same as the block vector of the latest HMVP candidate; obtaining a block vector of the current block corresponding to the initial merge list after the above insertion processes, and a merge candidate index for the current block.

EFFECT: increased compression ratio with little or no loss of image quality.

16 cl, 10 dwg, 2 tbl

Description

The field of technology to which the invention belongs

Embodiments of the present application (disclosure) generally relate to the field of image processing and, more specifically, to a shared list for prediction.

State of the art

Video encoding (video encoding and decoding) is used in a wide range of digital video applications, such as digital broadcast television, video transmission over the Internet and mobile networks, real-time conversational applications such as video chat, video conferencing, DVD and Blu-ray discs, in video content acquisition and editing systems and in security camera applications.

The amount of video data required to represent even a relatively short video can be significant, which can lead to difficulties when the data must be streamed or otherwise communicated over a communications network having limited bandwidth capabilities. Therefore, video data is usually compressed before transmission over modern communication networks. Video size can also be an issue when videos are stored in a storage device, as memory resources can be limited. Video compression devices often use software and/or hardware at the data source to encode video data prior to transmission or storage, thereby reducing the amount of data required to represent digital video images. The compressed data is then received at the destination by a video decompressor that decodes the video data. With limited network resources and ever-increasing demands for higher video quality, advanced compression and decompression technologies are required that increase compression rates with little or no loss in image quality.

The essence of the invention

Embodiments of the present application provide encoding and decoding devices and methods corresponding to the independent claims.

The above and other problems are solved by the subject matter of the independent claims. Additional forms of implementation are obvious from the dependent claims, description and drawings.

The first embodiment of the invention discloses a method for constructing a candidate merge list for Intra block copy (IBC) mode, where said method comprises the steps of:

inserting the block vector of the current block's left neighbor block into the current block's initial merge list (in this example, before this insertion step, the initial merge list is an empty list) when the left neighbor is available and the left neighbor is using IBC mode;

insert the block vector of the aforementioned adjacent block of the current block into the initial merge list (in this example, the initial merge list before this step is an empty list or the initial merge list contains the block vector of the left adjacent block of the current block), when the aforementioned adjacent block is available, the aforementioned the neighboring block uses the IBC mode and the block vector of the above neighboring block is not the same as the block vector of the left neighboring block;

inserting the block vector of the last candidate in the history based motion vector predictor (HMVP) into the initial merge list when the block vector of the aforementioned neighboring block is not the same as the block vector of the last HMVP candidate, and when the block vector of the left adjacent block is not the same as the block vector of the last HMVP candidate.

A second embodiment of the invention discloses a method for constructing a merge candidate list for an Intra block copy (IBC) candidate mode, said method comprising the steps of:

inserting a neighbor block vector of the current block into the initial merge list of the current block when the neighbor block is available and the neighbor block uses the IBC mode;

inserting a block vector of the last candidate in the history based motion vector predictor (HMVP) into the initial merge list when the block vector of the neighboring block is not the same as the block vector of the last HMVP candidate;

inserting the block vector of the other HMVP candidate into the initial merge list in which the reduction of the block vector of the other HMVP candidate is removed.

The third variant of the invention discloses a method for constructing a list of possible blocks for merging for the mode of intra-frame copying of prediction blocks (Intra block copy, IBC), where the said method contains the steps in which:

inserting the block vector of the current block's left neighbor block into the current block's initial merge list (in this example, before this insertion step, the initial merge list is an empty list) when the left neighbor is available and the left neighbor is using IBC mode;

insert the block vector of the aforementioned adjacent block of the current block into the initial merge list (in this example, the initial merge list before this step is an empty list or the initial merge list contains the block vector of the left adjacent block of the current block), when the aforementioned adjacent block is available, the aforementioned the neighboring block uses the IBC mode and the block vector of the above neighboring block is not the same as the block vector of the left neighboring block;

inserting the block vector of the last candidate in the history based motion vector predictor (HMVP) into the initial merge list when the block vector of the above neighbor block is not the same as the block vector of the last HMVP candidate and when the block vector the left adjacent block is not the same as the block vector of the last HMVP candidate;

inserting the block vector of the last candidate in the archive-based motion vector predictor HMVP into the initial merge list when the block vector of the aforementioned neighbor block is not the same as the block vector of the last HMVP candidate and when the block vector of the left neighbor block is not the same as the block vector of the latest HMVP candidate;

inserting the block vector of the other HMVP candidate into the initial merge list in which the reduction of the block vector of the other HMVP candidate is removed.

The fourth embodiment of the invention discloses a method for constructing a merge candidate list for Intra block copy (IBC) mode, said method comprising the steps of:

inserting a neighbor block vector of the current block into the initial merge list of the current block when the neighbor block is available and the neighbor block uses the IBC mode;

inserting a candidate block vector in a history based motion vector predictor (HMVP) into the initial merge list when the neighboring block block vector is not the same as the block vector of the last HMVP candidate;

in which the last block vector in the initial merge list of the current block is the block vector of one HMVP candidate.

The fifth embodiment of the invention discloses a method for constructing a merge candidate list for Intra block copy (IBC) mode, said method comprising the steps of:

inserting the block vector of the current block's left neighbor block into the current block's initial merge list (in this example, before this insertion step, the initial merge list is an empty list) when the left neighbor is available and the left neighbor is using IBC mode;

insert the block vector of the aforementioned adjacent block of the current block into the initial merge list (in this example, the initial merge list before this step is an empty list or the initial merge list contains the block vector of the left adjacent block of the current block), when the aforementioned adjacent block is available, the aforementioned the neighboring block uses the IBC mode and the block vector of the above neighboring block is not the same as the block vector of the left neighboring block;

inserting the block vector of the last candidate in the HMVP archive-based motion vector predictor into the initial merge list when the block vector of the aforementioned neighbor block is not the same as the block vector of the last HMVP candidate, and when the block vector of the left neighbor block is not is the same as the block vector of the last HMVP candidate;

inserting the block vector of the other HMVP candidate into the initial merge list in which the reduction of the block vector of the other HMVP candidate is removed, and

where the last block vector in the current block's initial merge list is a block vector of another HMVP candidate.

In one of the possible implementations of any of the first to fifth options, the method further comprises the step of: obtaining a block vector of the current block corresponding to the initial merge list after the insertion processes described above, and a possible candidate merge index for the current block.

In one of the possible implementations of any of the first to fifth options, in which the initial merge list is an empty list before the first insertion process.

In one of the possible implementations of any of the first to fifth options, the above insertion processes are performed in the appropriate order.

The sixth embodiment of the invention discloses an encoder (20) containing a processing circuit for performing any of the previous embodiments of the method.

The seventh embodiment of the invention discloses a decoder (30) containing a processing circuit for performing any of the previous embodiments of the method.

An eighth embodiment of the invention discloses a computer program product containing program code for executing any of the preceding method embodiments.

The ninth embodiment of the invention discloses a decoder or encoder comprising:

one or more processors; And

a non-portable, computer-readable medium associated with the processors and storing programming for execution by the processors, wherein the programming, when executed by the processors, configures a decoder to perform any of the preceding method embodiments.

Details of one or more embodiments are provided below in the accompanying drawings and in the description. Other features, objectives and advantages will become apparent from the description, drawings and claims.

Brief description of the drawings

Further embodiments of the invention are described in more detail with reference to the attached drawings, in which:

fig. 1A is a block diagram of an example of a video coding system capable of implementing embodiments of the invention;

fig. 1B is a block diagram of another example of a video coding system capable of implementing embodiments of the invention;

fig. 2 is a block diagram of an example of a video encoder capable of implementing embodiments of the invention;

fig. 3 is a block diagram of an exemplary structure of a video decoder capable of implementing embodiments of the invention;

fig. 4 is a block diagram of an example of an encoder or decoder;

fig. 5 is a block diagram of another example of an encoder or decoder;

fig. 6 is a block diagram of an example of neighboring blocks of the current block;

fig. 7 is an example of an embodiment of the present application;

fig. 8 is a block diagram of an exemplary structure of a content providing system 3100 implementing a content delivery service;

fig. 9 is a block diagram of the structure of an example terminal.

In the following, identical reference numerals refer to identical or at least functionally equivalent features, unless otherwise expressly defined otherwise.

Detailed description of embodiments

In the following description, reference is made to the accompanying drawings, which form part of the disclosure and show, by way of illustration, specific configurations of embodiments of the invention or specific configurations in which embodiments of the present invention may be used. It should be understood that embodiments of the invention can be used in other ways and contain structural or logical changes not shown in the drawings. The following detailed description, therefore, should not be construed in a limiting sense and the scope of the present invention is defined by the appended claims.

For example, it should be understood that the disclosure associated with the described method may also remain true for the corresponding device or system configured to carry out the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may comprise one or more blocks, such as functional blocks, for performing the one or more method steps described (for example, one block performing one or more steps, or a plurality of blocks each of which performs one or more of the plurality of steps), even if such one or more units are not explicitly described or shown in the drawings. On the other hand, for example, if a particular device is described based on one or more blocks, for example, functional blocks, the corresponding method may contain one step to perform the functionality of one or more blocks (for example, one step performs the functionality of one or more blocks or a plurality of steps, each of which performs the functionality of one or more of the plurality of blocks), even if such one or more steps are not explicitly described or shown in the drawings. Additionally, it should be understood that features of the various exemplary embodiments and/or variants described herein may be combined with one another unless specifically indicated otherwise.

Video coding generally refers to the processing of a sequence of images that form a video or video sequence. Instead of the term “image” (picture) in the field of video coding, the term “frame” (frame) or “image” (image) can be used as a synonym. Video coding (or, more generally, coding) contains two parts: video coding and video decoding. Video coding is performed on the side of the data source, usually involves processing (eg, compressing) the original video images to reduce the amount of data required to represent the video images (for more efficient storage and/or transmission of data). Video decoding is performed on the destination side and usually contains inverse processing compared to the encoder for reconstructing video images. Embodiments relating to “coding” video images (or images in general) should be understood as referring to “coding” or “decoding” video images or corresponding video sequences. The combination of an encoding part and a decoding part is also referred to as CODEC (Coding and Decoding).

In the case of lossless video coding, the original video images can be reconstructed, i.e. reconstructed as video images having the same quality as the original video images (assuming no transmission loss or other data loss during data storage or transmission). In the case of lossy video coding, additional compression is performed, for example by quantization, to reduce the amount of data representing video images that cannot be completely reconstructed in the decoder, i.e. the quality of the reconstructed video images will be lower or worse than the quality of the original video images.

Several video coding standards belong to the group of "hybrid lossy video codecs" (ie, combine spatial and temporal sample-domain prediction and two-dimensional (2D) transform coding to apply transform-domain quantization). Each image of a video sequence is typically divided into a set of non-overlapping blocks, and encoding is typically performed at the block level. In other words, in the encoder, the video is usually processed, i.e. encode, at the block (video block) level, for example, using spatial (intra-frame) prediction and/or temporal (inter-frame) prediction to generate a prediction block by subtracting the prediction block from the current block (block currently being processed/block to be processed), to obtain the residual block, transform the residual block, and quantize the residual block in the transform domain to reduce the amount of data to be transmitted (compressed), while in the decoder, compared to the encoder, the encoded or compressed block is applied to the encoded or compressed block to reconstruct the current block to represent it. processing. Additionally, the encoder duplicates the decoder's processing cycle so that they both generate identical predictions (eg, intra and inter predictions) and/or reconstructions for the process, i.e. coding, subsequent blocks.

In the following embodiments of video coding system 10, video encoder 20 and video decoder 30 are described based on FIG. 1-3.

In FIG. 1A is a schematic block diagram of an exemplary coding system 10, such as a video coding system 10 (or short coding system 10), that may use the technologies presented herein. Video encoder 20 (or short encoder (short encoder) 20) and video decoder 30 (or short decoder (short decoder) 30) video coding system 10 are examples of devices that can be configured to execute methods in accordance with various examples described in this application .

As shown in FIG. 1A, the encoding system 10 includes a data source device 12 configured to provide the encoded image data 21 to, for example, the destination device 14 to decode the encoded image data 13. The data source device 12 includes an encoder 20 and may additionally, i. e. optionally, contain an image source 16, a preprocessor (or preprocessor) 18, such as an image preprocessor 18, and a communication interface or communication unit 22.

The image source 16 may comprise any kind of image acquisition device, such as a camera for capturing a real image, and/or any kind of imaging device, such as a computer graphics processor for generating an animated computer image, or any kind of other device for capturing and/or providing a real image. an image generated by an imaging computer (eg, screen content, virtual reality (VR) image) and/or any combination thereof (eg, augmented reality (AR) images). The image source may be any kind of memory or storage device storing any of the above images.

Unlike the pre-processor 18 and the process performed by the pre-processor 18, the image or image data 17 may also be referred to as a raw image or raw image data 17.

The pre-processor 18 is configured to receive (raw) image data 17 and perform pre-processing of the image data 17 to obtain a pre-processed image 19 or pre-processed image data 19. The pre-processing performed by the pre-processor 18 may, for example, include cropping, color format conversion (eg from RGB to YCbCr), color correction or noise reduction. It should be understood that the preprocessing unit 18 may be an optional component.

Video encoder 20 is configured to receive pre-processed image data 19 and provide encoded image data 21 (more details will be described below, for example, based on FIG. 2).

Communication interface 22 of source device 12 may be configured to receive encoded image data 21 and transmit encoded image data 21 (or any further processed version thereof) over communication channel 13 to another device, such as destination device 14 or any other storage device. or direct reconstruction.

Destination device 14 includes a decoder 30 (eg video decoder 30) and may additionally, i. e. optionally, contain a communication interface or communication unit 28, postprocessor 32 (or postprocessing unit 32) and display 34.

The communication interface 28 of the destination device 14 is configured to receive the encoded image data 21 (or any further processed version thereof), for example, directly from the data source device 12 or from any other source, for example, from a storage device, for example, an encoded image data storage device. , and providing the encoded image data 21 to the decoder 30.

Communication interface 22 and communication interface 28 may be configured to transmit or receive encoded image data 21 or encoded data 13 over a direct link between source device 12 and destination device 14, such as over a direct wired or wireless connection, or over any kind of network, for example, through a wired or wireless network, or any combination thereof, or through any kind of private network and public network, or any combination thereof.

The communication interface 22 may, for example, be configured to package the encoded image data 21 into an appropriate format, such as packets, and/or process the encoded image data with any kind of transmission encoding or transmission processing for transmission over a communication channel or over a communication network.

The communication interface 28, which duplicates the communication interface 22, may, for example, be configured to receive the transmitted data and process the transmission data with any kind of appropriate transmission decoding or processing and/or decompression to obtain encoded image data 21.

Both interfaces, communication interface 22 and communication interface 28, can be implemented as unidirectional communication interfaces, as indicated by the arrow for transmission channel 13 in FIG. 1A pointing from source device 12 to destination device 14, or as bidirectional communication interfaces, and can be configured, for example, to send and receive messages, for example, to set up a connection, acknowledge receipt, and exchange any other information associated with a communication channel. and/or with data transmission, such as transmission of encoded image data.

The decoder 30 is configured to receive encoded image data 21 and provide decoded image data 31 or decoded image 31 (more details will be described below based on FIG. 3 or FIG. 5, for example).

The post processor 32 of the destination device 14 is configured to post-process the decoded image data 31 (also referred to as reconstructed image data), such as the decoded image 31, to obtain data 33 after post-processing the image, such as the post-processed image 33. The post-processing performed by the post-processing unit 32, may comprise, for example, color format conversion (e.g., from YCbCr to RGB), color correction, cropping or resampling, or any other processing, such as preparing decoded image data 31 for display, for example, on display 34.

The display 34 of the destination device 14 is configured to receive post-processed image data 33 for displaying the image to, for example, a user or viewer. The display 34 may be or include any kind of display for presenting the rendered image, such as an integrated or external display or monitor. The displays may, for example, include liquid crystal displays (liquid crystal display (LCD), organic light emitting diode (OLED) displays, plasma displays, projectors, micro LED displays, silicon liquid crystal displays (LCoS), digital optical processor (digital light processor, DLP) or any other kind of display.

Although in FIG. 1A, source device 12 and destination device 14 are shown as separate devices, device embodiments may also comprise both devices or both functionality, i.e., source device 12 or its corresponding functionality and destination device 14 or its corresponding functionality. In such embodiments, source device 12 or its respective functionality and destination device 14 or its corresponding functionality may be implemented using the same hardware and/or software, or individual hardware and/or software, or any combination thereof. .

As should be apparent to one skilled in the art based on the description, the existence and (precise) division of functionality of various blocks or functionality within source device 12 and/or destination device 14 as shown in FIG. 1A may vary depending on the existing device and application.

Encoder 20 (eg, video encoder 20) or decoder 30 (eg, video decoder 30), or encoder 20 and decoder 30 together may be implemented by a processing circuit as shown in FIG. 1B, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, dedicated video encoders, or any combination thereof. Encoder 20 may be implemented by processing circuitry 45 to implement various modules, as discussed with respect to encoder 20 shown in FIG. 2 and/or any other encoder system or subsystem described herein. Decoder 30 may be implemented by processing circuitry 46 to implement various modules, as discussed with respect to decoder 30 in FIG. 3 and/or any other decoder system or subsystem described herein. The processing circuit may be configured to implement various operations, as discussed later. As shown in FIG. 5, if the technologies are implemented in part by software, a device may store software instructions on an appropriate non-portable machine-readable medium and may execute instructions on hardware using one or more processors to execute the technologies of this disclosure. Either of video encoder 20 or video decoder 30 may be integrated as part of a combined encoder/decoder (CODEC) in a single device, such as shown in FIG. 1b.

Source device 12 and destination device 14 may comprise any of a wide variety of devices, including any kind of handheld or stationary devices, such as laptops or laptops, mobile phones, smartphones, tablets or tablet computers, cameras, desktop computers, set-top boxes. , televisions, displays, digital media players, video game consoles, video streaming devices (such as content service servers or content delivery servers), radio broadcast receiver, radio broadcast transmitter, etc., and may not use any or use an operating system any kind. In some cases, source device 12 and destination device 14 may be designed for wireless communication. Thus, source device 12 and destination device 14 may be wireless communication devices.

In some cases, the video coding system 10 shown in FIG. 1A is merely an example, and the techniques of the present application may apply to video encoding settings (eg, video encoding or video decoding) that do not necessarily involve any communication between encoding and decoding devices. In other examples, data is retrieved from local memory, streamed over a network, and the like. The video encoder may encode and store data in memory and/or the video decoder may retrieve data from memory and decode it. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data into memory and/or retrieve and decode data from memory.

For ease of description, embodiments of the invention are described here, for example, with reference to High-Efficiency Video Coding (HEVC) or Versatile Video coding (VVC) reference software, the next generation video coding standard developed by the Joint Expert Group on ITU-T Video Coding Experts Group (VCEG) and ISO/IEC Motion Picture Experts Group (MPEG). Each of those skilled in the art should understand that embodiments of the invention are not limited to HEVC or VVC.

Encoder and encoding method

In FIG. 2 schematically shows an exemplary block diagram of a video encoder 20 capable of implementing the technologies of the present application. In the example shown in FIG. 2, the video encoder 20 comprises an input 201 (or input interface 201), a residual calculation block 204, a transformation process block 206, a quantization block 208, an inverse quantization block 210, and an inverse transform process block 212, a reconstruction block 214, a loop filter block 220, a buffer 230 decoded picture buffer (DPB), mode selection block 260, entropy encoding block 270, and output 272 (or output interface 272). The mode selector 260 may comprise an inter-predictor 244 and an intra-predictor 254, and a splitter 262. The inter prediction block 254 may comprise a motion estimator and a motion compensation block (not shown). Video encoder 20 shown in FIG. 2 may also be referred to as a hybrid video encoder or a video encoder corresponding to a hybrid video codec.

Residual calculator 204, transform process block 206, quantizer 208, mode selector 260 may be referred to as forming the signal path of the encoder 20, while inverse quantizer 210, inverse transform process block 212, reconstruction block 214, buffer 216, loop filter 220, decoded picture buffer (DPB) 230, inter prediction block 244, and intra prediction block 254 may be referred to as forming the return signal path of video encoder 20, where the return signal path of video encoder 20 corresponds to the signal path of the decoder (see video decoder 30 in Fig. 3). An inverse quantizer 210, an inverse transform 212 212, a reconstructor 214, a loop filter 220, a picture buffer (DPB) 230, an inter-frame predictor 244, and an intra-frame predictor 254 are also related to forming the “built-in decoder” of the video encoder 20.

Separation of images and images (images and blocks)

Encoder 20 may be configured to receive, for example, via input 201, an image 17 (or image data 17), for example an image of a sequence of images forming a video or video sequence. The received image or image data may also be a pre-processed image 19 (or pre-processed image data 19). For simplicity, the following description refers to picture 17. Picture 17 may also be referred to as the current picture or the picture to be encoded (particularly in video coding, to distinguish the current picture from other pictures, e.g., previously encoded and/or decoded pictures of one and the same video sequence, i.e. a video sequence that also contains the current image).

The (digital) image is, or may be, viewed as a two-dimensional matrix or matrix of samples with intensity values. A sample in a matrix may also be referred to as a pixel (short form of picture element) or picture element. The number of samples in the horizontal and vertical directions (or axes) of a matrix or image determines the size and/or resolution of the image. Three color components are commonly used to represent a color, i.e. the image may be represented by or contain three sample matrices. In RBG format or color space, an image contains a corresponding matrix of red, green, and blue samples. However, in video coding, each pixel is usually represented in a luminance and chrominance format or color space, such as YCbCr, which contains a luminance component, denoted Y (sometimes L is also used instead), and two chrominance components, denoted Cb and Cr. The luminance Y component (or luminance for short) represents the intensity of the luminance level or gray level (eg, as in a grayscale image), while the two color components Cb and Cr (or chrominance for short) represent color or chrominance information components. Accordingly, a YCbCr image contains a matrix of luminance samples (Y) and two matrixes of chrominance samples with chrominance values (Cb and Cr). Images in RGB format can be converted or transformed to YCbCr format and vice versa, the process is also known as color conversion or conversion. If the image is monochrome, the image may only contain a matrix of luminance samples. Accordingly, the image may be, for example, a matrix of luminance samples in monochrome format, or a matrix of luminance samples and two corresponding chrominance sample matrices in 4:2:0, 4:2:2 and 4:4:4 color format.

Embodiments of video encoder 20 may include an image splitter (not shown in FIG. 2) configured to split image 17 into multiple (typically non-overlapping) image blocks 203. These blocks may also be referred to as root blocks, macroblocks (H.264/AVC), or coding tree blocks (CTBs) or coding tree units (CTUs) (H.265/HEVC and VVC). The image dividing block may be configured to use the same block size for all images of the video sequence and the corresponding grid defining the block size, or to change the block size between images or subsets or groups of images and divide each image into appropriate blocks.

In further embodiments, the video encoder may be configured to directly receive image block 203 17, such as one, more or all of the blocks forming image 17. Image block 203 may also be referred to as the current image block or the image block to be encoded.

Like image 17, image block 203 is again, or may be considered, a two-dimensional or sample matrix with intensity values (sample values), albeit smaller than image 17. In other words, block 203 may contain, for example, one matrix of samples (for example , a luma matrix in the case of a monochrome image 17, or a luminance or chrominance matrix in the case of a color image) or three sample matrices (for example, a luminance matrix and two chrominance matrices in the case of a color image 17) or any other number and/or types of matrices depending on applied color format. The number of samples in the horizontal and vertical directions (or axes) of block 203 determines the size of block 203. Accordingly, the block may contain, for example, an MxN matrix of samples (M columns by N rows) or an MxN matrix of transform coefficients.

Embodiments of video encoder 20, as shown in FIG. 2 may be configured to encode the image 17 block by block, for example, encoding and prediction are performed for each block 203.

Embodiments of video encoder 20, as shown in FIG. 2 may be further configured to divide and/or encode the image using segments (also referred to as video segments), where the image may be divided into one or more segments, or encoded using one or more segments (typically non-overlapping), and each segment may contain one or more units (eg CTU).

Embodiments of video encoder 20, as shown in FIG. 2 may be further configured to split and/or encode an image using tiles (also referred to as video tiles) and/or tiles (also referred to as video tiles), where an image may be split or encoded using one or more (typically non-overlapping) tiles. groups, and each tile group may contain, for example, one or more blocks (eg, CTU) or one or more tiles, where each tile may, for example, have a rectangular shape and may contain one or more blocks (eg, CTU), for example , full or fractional blocks.

Remainder Calculation

Residual block 204 can be configured to calculate residual block 205 (also referred to as residual 205) based on image block 203 and prediction block 265 (more details about predictor 265 are provided below), for example, by subtracting the sample values of prediction block 265 from sample values of the image block 203 sample by sample (pixel by pixel) to obtain a residual block 205 in the sample area.

transformation

Transform block 206 206 can be configured to apply a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), on the sample values of the residual block 205 to obtain transform coefficients 207 in the transform domain. The transform coefficients 207 may also be referred to as residual transform coefficients and represent the residual block 205 in the transform domain.

Transform block 206 may be configured to apply DCT/DST integer approximations such as the transforms defined in H.265/HEVC. Compared to the orthogonal DCT, such integer approximations are usually scaled by a certain factor. To preserve the norm of the residual block that is processed by forward and backward transformation, additional scaling factors are applied as part of the transformation process. Scale factors are usually chosen based on certain constraints, such as scale factors being the second power of shift operations, color depth of transform coefficients, trade-off between accuracy and implementation cost, and so on. Specific scaling factors are determined, for example, for the inverse transform, for example, by the inverse transform block 212 (and for the corresponding inverse transform, for example, by the inverse transform block 312 in the video decoder 30), and the corresponding scaling factors for the forward transform can be determined, for example, by the transform block 206 in encoder 20, respectively.

Embodiments of video encoder 20 (respectively, transform block 206) may be configured to output transform parameters, such as the type of transform or transforms, for example, directly or encoded or compressed by entropy encoding block 270, so that, for example, video decoder 30 can receive and use transform options for decoding.

Quantization

Quantizer 208 may be configured to quantize transform coefficients 207 to obtain quantized coefficients 209, for example, using scalar quantization or vector quantization. The quantized coefficients 209 may also be referred to as quantized transform coefficients 209 or as quantized residual coefficients 209.

The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207 . For example, an n-bit transform factor may be rounded down to an m-bit transform factor during quantization, where n is greater than m. The degree of quantization can be changed by adjusting the quantization parameter (QP). For example, for scalar quantization, different scaling may be applied to achieve finer or coarser quantization. Smaller quantization step sizes correspond to finer quantization, while larger quantization step sizes correspond to coarser quantization. The applied quantization step size may be indicated by a quantization parameter (QP). The quantization parameter may be, for example, an index into a given set of applicable quantization step sizes. For example, small quantization parameters may correspond to fine quantization (small quantization step sizes), and large quantization parameters may correspond to coarse quantization (large quantization step sizes), or vice versa. The quantization may comprise quantization step size division and corresponding and/or inverse dequantization, for example by inverse quantizer 210, and may comprise multiplying by the quantization step size. Embodiments conforming to any standards, such as HEVC, may be configured to use a quantization parameter to determine the quantization step size. Typically, the quantization step size can be calculated based on the quantization parameter using a fixed-point approximation for an equation containing separation. Additional scaling factors may be introduced for quantization and inverse dequantization to reconstruct the residual block norm, which may be changed due to the scaling used in the fixed-point approximation for the equation for quantization step size and quantization parameter. In one exemplary embodiment, inverse transform scaling and inverse dequantization may be combined. On the other hand, for example, in a bitstream, special quantization tables can be used and communicated from the encoder to the decoder. Quantization is a lossy operation where the loss increases with increasing quantization step sizes.

Embodiments of video encoder 20 (respectively, quantization block 208) may be configured to output quantization parameters (QP), for example, directly or encoded by entropy encoding block 270, so that, for example, video decoder 30 can receive and apply quantization parameters for decoding.

Inverse quantization

The inverse quantizer 210 is configured to apply the inverse quantization of the quantizer 208 to the quantized coefficients to obtain the inverse quantized coefficients 211, for example, by inverting the quantization scheme applied by the quantizer 208 based on or using the same quantization step size as the quantizer 208 quantization. The inversely quantized coefficients 211 may also be referred to as inversely quantized residual coefficients 211 and correspond to—though not generally identical transform coefficients due to quantization loss—transform coefficients 207.

Reverse transformation

The inverse transform block 212 is configured to apply an inverse transform to the transform applied by the transform block 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), or other inverse transform to obtain a reconstructed residual block 213 (or corresponding inverse quantized coefficients 213) in the sample area. The reconstructed residual block 213 may also be referred to as the transform block 213.

Reconstruction

Reconstructor 214 (e.g., adder 214) is configured to add transform block 213 (i.e., reconstructed residual block 213) to prediction block 265 to obtain reconstructed block 215 in the sample area, such as by adding, sample by sample, the values samples of reconstructed residual block 213 and sample values of prediction block 265 .

Filtration

The loop filter block 220 (or "loop filter" 220 for short) is configured to filter the reconstructed block 215 to obtain a filtered block 221, or, in general, filter the reconstructed samples to obtain filtered samples. The loop filter unit, for example, is configured to smooth transitions between pixels or otherwise improve video quality. The loop filter block 220 may comprise one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) bias filter, or one or more other filters, such as a bidirectional filter, an adaptive loop filter, ALF), sharpening, anti-aliasing or joint filters, or any combination of them. Although the loop filter unit 220 is shown in FIG. 2 as a loop filter, in other configurations, the loop filter unit 220 may be implemented as a post loop filter. Filtered block 221 may also be referred to as filtered reconstructed block 221.

Embodiments of video encoder 20 (respectively, loop filter block 220) may be configured to output loop filter parameters (such as adaptive sample offset information), for example, directly or encoded by entropy encoding block 270, so that, for example, decoder 30 can receive and apply the same loop filter parameters or corresponding loop filters for decoding.

Decoded image buffer

The decoded picture buffer (DPB) 230 may be a memory that stores reference pictures, or reference picture data in general, for video encoder 20 to encode video data. memory (dynamic random access memory, DRAM), including synchronous DRAM (synchronous DRAM, SDRAM), magnetoresistive RAM (magnetoresistive RAM, MRAM), resistive RAM (resistive RAM, RRAM) or other types of memory devices. Decoded picture buffer (DPB) 230 may be configured to store one or more filtered blocks 221. Decoded picture buffer 230 may further be configured to store other previously filtered blocks, such as previously reconstructed and filtered blocks 221, of the same current picture. or other images, such as previously rendered images, and may provide complete previously rendered images, i.e. decoded pictures (and corresponding reference blocks and samples) and/or a partially reconstructed current picture (and corresponding reference blocks and samples), for example, for inter-picture prediction. The decoded picture buffer (DPB) 230 may also be configured to store one or more unfiltered reconstructed blocks 215 or, in general, unfiltered reconstructed samples, for example, if the reconstructed block 215 is not filtered by the loop filter block 220 or no other additional processed version is used. reconstructed blocks or samples.

Mode selection (division and prediction)

The mode selector 260 comprises a splitter 262, an inter-picture predictor 244, and an intra-predictor 254, and is configured to receive or obtain original image data, such as the original block 203 (the current block 203 of the current image 17) and reconstructed image data, such as filtered and/or unfiltered reconstructed samples or blocks of the same (current) picture and/or their one or more previously decoded pictures, eg from decoded picture buffer 230 or other buffers (eg, linear buffer, not shown). The reconstructed image data is used as reference image data for prediction, such as inter-picture prediction or intra-picture prediction, to obtain a prediction block 265 or a predictor 265.

The mode selector 260 may be configured to determine or select a split for the current block prediction mode (including no prediction) and the prediction mode (e.g., intra frame or inter frame prediction mode) and generate a corresponding prediction block 265 that is used to compute the residual block. 205 and for the reconstruction of the reconstructed block 215.

Embodiments of mode selector 260 may be configured to select a split and prediction mode (e.g., those supported by or available to mode selector 260) that provides the best fit, or in other words, minimum residual (lowest residual means best compression for transmission or storage), or minimum overhead (minimum overhead means better compression or storage), or which takes into account or balances both. The mode selector 260 may be configured to determine the partitioning and prediction mode based on the rate distortion optimization (RDO), i. selecting the prediction mode that provides the minimum distortion at a given rate. Terms such as “best”, “minimum”, “optimal”, etc., in this context do not necessarily refer, in general, to “best”, “minimum”, “optimal”, etc., but may also refer to the fulfillment of a termination or selection criterion, such as a value exceeding or falling below a threshold, or other constraints potentially leading to a “sub-optimal selection” that reduces complexity and processing time.

In other words, partitioning block 262 can be configured to partition block 203 into smaller block parts or subblocks (which again form blocks), for example, by repeatedly using quad-tree-partitioning (QT), binary partitioning, BT) or triple tree-partitioning (TT) or any combination thereof, and performing, for example, prediction for each of the parts of the partition of the block or subblocks, where the selection mode contains the selection of the tree structure of the divided block 203, and the prediction modes are applied to each of the divisions of the block or subblocks.

In the following, the separation (eg, by the splitter 260) and the prediction process (by the inter prediction 244 244 and the intra prediction 254 254) performed by the exemplary video encoder 20 will be explained in more detail.

Separation

Divider 262 may divide (or split) the current block 203 into smaller pieces, such as smaller square or rectangular blocks. These smaller blocks (which may also be referred to as sub-blocks) can be further subdivided into even smaller pieces. This is also referred to as tree splitting or hierarchical tree splitting, in which a root box, such as at the root zero level of the tree (hierarchical level 0, depth 0), can split recursively, such as split into two or more boxes of the next lower level of the tree, e.g. nodes at tree level 1 (hierarchical level 1, depth 1), where these blocks can again be split into two or more next lower level blocks, e.g. tree level 2 (hierarchical level 2, depth 2), etc. d. until the split is complete, for example because a termination criterion is met, such as the maximum tree depth or minimum block size is reached. Blocks that are not further divided are also referred to as leaf blocks or tree leaf nodes. A tree using splitting into two parts is referred to as a binary tree (binary-tree, BT), and a tree using splitting into three parts is referred to as a ternary tree (ternary-tree, TT) and a tree using splitting into four parts is referred to as quad-tree (QT).

As mentioned before, the term "block", as used here, may be a part, in particular, a square or rectangular part of the image. With reference to, for example, HEVC and VVC, a block can be or correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit TU) and/or corresponding blocks, such as a coding tree block (CTB), a coding block (CB), a transform block (TB), or a prediction block (PB).

For example, a coding tree block (CTU) can be or contain luma samples CTBs, two corresponding chrominance sample CTBs of an image that has three sample matrices, or a monochrome image sample CTB or an image that is encoded using three separate color planes and syntax structures. , used to encode the samples. Accordingly, a coding tree block (CTB) may be a block with a size of NxN samples for some value of N, such that dividing a component by CTB is a partitioning. A coding unit (CU) may be or comprise a luminance sample coding unit, two corresponding chrominance sample coding units of an image having three sample matrices, or a monochrome image sample coding unit or an image encoded with three separate color planes, and a syntax structure. , used to encode the samples. Accordingly, a coding block (CB) may be a block of size MxN samples for some values of M and N, so that dividing the CTB into coding blocks is a partitioning.

In HEVC-compliant embodiments, for example, a coding tree unit (CTU) may be partitioned into CUs using a quadtree structure referred to as a coding tree. The decision whether to encode an image area using inter-frame (temporal) prediction or intra-frame (spatial) prediction is made at the CU layer. Each CU may be further divided into one, two or four PUs corresponding to the division type of the PU. Within one PU, the same prediction process is applied and the corresponding information is transmitted to a decoder based on the PU. After obtaining the residual block by applying a prediction process based on the division type of the PU, the CU can be divided into transform units (TUs) corresponding to a different quadtree structure similar to the coding tree for the CU.

In embodiments, for example, corresponding to the latest video coding standard currently being developed, which is referred to as universal video coding (Versatile Video Coding, VVC), for example, separation of the combined Quad-tree and binary tree (combined Quad-tree and binary tree, QTBT). In the QTBT block structure, the CU can be square or rectangular. For example, a coding tree unit (CTU) is first divided by a quadtree structure. The leaf nodes of the quadtree are further divided by a binary tree structure or a ternary (or triple) tree structure. The leaf nodes of the partitioning tree are called coding units (CU) and such segmentation is used for the prediction and transformation process without any additional partitioning. This means that in a QTBT coding block structure, CUs, PUs, and TUs have the same block size. In parallel, multi-partitioning may be used, such as the ternary tree partitioning used in conjunction with the QTBT block structure.

In one example, mode selector 260 of video encoder 20 may be configured to implement any combination of the separation techniques described herein.

As described above, video encoder 20 is configured to determine or select the best or optimal prediction mode from a set of (e.g., predetermined) prediction modes. The set of prediction modes may comprise, for example, intra-prediction modes and/or inter-prediction modes.

Intra-frame prediction

The set of intra-prediction modes may contain 35 different intra-prediction modes, such as non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in HEVC, or may contain 67 different intra-prediction modes , for example, non-directional modes such as DC (or medium) mode and planar mode, or directional modes such as those defined for VVC, for example.

The intra prediction block 254 is configured to use reconstructed samples of neighboring blocks of the same current image to generate an intra prediction block 265 corresponding to an intra prediction mode from a set of intra prediction modes.

The intra prediction block 254 (or mode selector 260 generally) is further configured to output intra prediction parameters (or information indicative of the selected intra prediction mode for the block in general) to the entropy encoding block 270 in the form of syntax elements 266 for including an encoded image in data 21 so that, for example, video decoder 30 can receive and use prediction parameters for decoding.

Interframe prediction

The set (or possible) inter prediction modes depends on the available reference pictures (i.e., previous at least partially decoded pictures, such as those stored in DBP 230) and other inter prediction options, such as whether the entire reference picture is used or only a portion of the reference image, such as a search window area around the current block area, to find the best matching reference block, and/or, for example, whether pixel interpolation is applied, such as half/half-element interpolation and/or quarter-pixel interpolation.

In addition to the aforementioned prediction modes, skip mode and/or direct mode may be applied.

The inter prediction block 244 may comprise a motion estimation (ME) block and a motion compensation (MC) block (both not shown in FIG. 2). The motion estimator may be configured to receive or obtain image block 203 (current image block 203 from current image 17) and decoded image 231, or at least one or more previously reconstructed blocks, e.g., reconstructed blocks of one or more other/different previously decoded images 231 for motion estimation. For example, the video sequence may comprise the current picture and previously decoded pictures 231 or, in other words, the current picture and previously decoded pictures 231 may be part of or form a sequence of pictures to form a video sequence.

The encoder 20 may, for example, be configured to select a reference block from a plurality of reference blocks of the same or different pictures from a plurality of other pictures and provide a reference picture (or reference picture index) and/or an offset (spatial offset) between position (coordinates x, y) of the reference block and the position of the current block as inter-prediction parameters to feed to the motion estimation block. This displacement is also called the motion vector (MV).

The motion compensation block is configured to obtain, for example, receive, an inter prediction parameter, and perform inter prediction based on or using the inter prediction parameter to obtain the inter prediction block 265 . The motion compensation performed by the motion compensator may comprise sampling or generating a prediction block based on the motion vector/block vector determined by the motion estimate, possibly performing sub-pixel precision interpolations. Interpolation filtering can generate additional pixel samples from known pixel samples, thus potentially increasing the number of possible prediction blocks that can be used to encode an image block. After receiving the motion vector for the PU of the current picture block, the motion compensator may locate the prediction block pointed to by the motion vector in one of the reference picture lists.

The motion compensation block may also generate syntax elements associated with video blocks and segments for use by video decoder 30 when decoding image blocks of a video segment. In addition, or as an alternative to segments and corresponding syntax elements, tile groups and/or tiles and corresponding syntax elements may be formed or used.

Entropy coding

Entropy encoding unit 270 is configured to apply, for example, an entropy coding algorithm or scheme (for example, variable length coding (VLC), context adaptive VLC (context adaptive VLC, CAVLC), context adaptive binary arithmetic coding (context adaptive binary arithmetic coding (CABAC), based on the syntax of context-based adaptive binary arithmetic coding (syntax-based context-adaptive binary arithmetic coding, SBAC), entropy coding with probability interval partitioning entropy (PIPE), or other methodology or technology of entropy coding) or bypass (uncompressed) quantized coefficients 209, inter prediction parameters, intra prediction parameters, loop filter parameters, and/or other syntax elements to obtain encoded image data 21 that can be output via output 272, for example, in the form of an encoded bitmap. stream 21 so that, for example, video decoder 30 can receive and use parameters for decoding. The encoded bitstream 21 may be transmitted to video decoder 30 or stored in memory for later transmission or retrieval by video decoder 30.

Other structural variants of video encoder 20 may be used to encode the video stream. For example, encoder 20 with no transform for certain blocks or frames may quantize the residual signal directly without transform block 206 . In another implementation, encoder 20 may have quantizer 208 and inverse quantizer 210 combined into a single unit.

Decoder and decoding method

In FIG. 3 shows an example of a video decoder 30 that is capable of implementing the technologies of the present application. Video decoder 30 is configured to receive encoded image data 21 (e.g., encoded bitstream 21), such as encoded by encoder 20, to obtain a decoded image 331. The encoded image data or bitstream contains information for decoding encoded image data, such as data representing image blocks of the encoded video segment (and/or tile group or tile) and associated syntax elements.

In the example shown in FIG. 3, the decoder 30 includes an entropy decoder 304, an inverse quantizer 310, an inverse transform 312, a reconstructor 314 (e.g., adder 314), a loop filter 320, a decoded picture buffer (DBP) 330, a mode application 360 , block 344 inter-prediction and block 354 intra-prediction. The inter prediction block 344 may comprise or include a motion compensation block. Video decoder 30, in some examples, may perform decoding generally in the reverse direction of the encoding described with respect to video encoder 100 shown in FIG. 2.

As has been explained with respect to encoder 20, inverse quantizer 210, inverse transform 212, reconstructor 214 and loop filter 220, decoded picture buffer (DPB) 230, inter prediction 344, and intra prediction 354 are also referred to as constituting an “embedded decoder”. ” of the video encoder 20. Accordingly, inverse quantizer 310 may be functionally identical to inverse quantizer 110, inverse transform block 312 may be functionally identical to inverse transform block 212, reconstructor 114 may be functionally identical to reconstructor 214, loop filter 320 may be functionally identical loop filter 220 and decoded image buffer 330 may be functionally identical to decoded image buffer 230. Therefore, the explanations provided for the respective blocks and functions of video encoder 20 apply properly to the corresponding blocks and functions of video decoder 30.

Entropy decoding

The entropy decoding unit 304 is configured to parse the bitstream 21 (or, in general, the encoded image data 21) and perform, for example, entropy decoding of the encoded image data 21 to obtain, for example, quantized coefficients 309 and/or decoded coding parameters (not shown in Figure 3), for example, any or all of the inter-prediction parameters (eg, reference picture index and motion vector), intra-prediction parameter (eg, intra-prediction mode or index), transform parameters, quantization parameters, loop filter parameters, and/ or other syntax elements. Entropy decoding unit 304 may be configured to apply decoding algorithms or schemes corresponding to the coding schemes described with respect to entropy encoding unit 270 of encoder 20. Entropy decoding unit 304 may be further configured to provide inter-prediction parameters, an intra-prediction parameter, and/or other syntax elements for mode application module 360; and other parameters for other blocks of decoder 30. Video decoder 30 may receive syntax elements at the video segment level and/or at the video block level. In addition, or as an alternative to segments and corresponding syntax elements, tile groups and/or tiles and corresponding syntax elements may be accepted and/or used.

Inverse quantization

The inverse quantizer 310 may be configured to receive quantization parameters (QP) (or inverse quantization related information in general) and quantized coefficients from the encoded image data 21 (e.g., by parsing and/or decoding, for example, entropy decoding block 304) and applying, based on the quantization parameters, inverse quantization on the decoded quantized coefficients 309 to obtain dequantized coefficients 311, which may also be referred to as transform coefficients 311. The inverse quantization process may comprise using a quantization parameter determined by video encoder 20 for each video block in a video segment (or tile or tile group) to determine the amount of quantization and, likewise, the amount of inverse quantization that may be applied.

Reverse transformation

Inverse transform block 312 may be configured to receive inversely quantized coefficients 311, also referred to as transform coefficients 311, and apply a transform to inversely quantized coefficients 311 to obtain reconstructed residual blocks 213 in the sample area. The reconstructed residual blocks 213 may also be referred to as transform blocks 313 . The transformation may be an inverse transformation, eg, inverse DCT, inverse DST, inverse integer transformation, or a conceptually similar inverse transformation process. The inverse transform block 312 may further be configured to receive transform parameters or corresponding information from the encoded image data 21 (eg, by parsing and/or decoding, for example, entropy decoding block 304) to determine the transform to be applied to the dequantized coefficients 311 .

Reconstruction

Reconstructor 314 (e.g., adder 314) can be configured to add reconstructed residual block 313 to predictor 365 to obtain reconstructed block 315 in the sample region, for example, by adding sample values of reconstructed residual block 313 and sample values of predictor 365.

Filtration

The loop filter block 320 (either in the encoding loop or after the encoding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321, for example, to smooth pixel transitions or otherwise improve video quality. The loop filter block 320 may comprise one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) bias filter, or one or more other filters, such as a bidirectional filter, an adaptive loop filter, ALF), sharpening, anti-aliasing or joint filters, or any combination of them. Although the loop filter unit 320 is shown in FIG. 3 as a loop filter, in other configurations, the loop filter unit 320 may be implemented as a post loop filter.

Decoded image buffer

The decoded video blocks 321 are then stored in the decoded picture buffer 330, which stores the decoded pictures 331 as reference pictures for subsequent motion compensation in other pictures and/or for display, respectively.

The decoder 30 is configured to output the decoded image 311, for example, through the output 312, for presentation or viewing by the user.

Prediction

The inter prediction block 344 may be identical to the inter prediction block 244 (in particular, the motion compensation block), and the intra prediction block 354 may be functionally identical to the inter prediction block 254 and perform division or division decision and prediction based on division and/or prediction parameters. or corresponding information obtained from the data 21 of the encoded image (for example, by parsing and/or decoding, for example, block 304 entropy decoding). Mode apply block 360 may be configured to perform prediction (intraframe or interframe) for each block based on reconstructed images, blocks, or corresponding samples (filtered or unfiltered) to obtain prediction block 365 .

When a video segment is encoded as an intra-frame coded segment (I), the intra-frame prediction block 354 of the mode applying block 360 is configured to generate a prediction block 365 for a picture block of the current video segment based on the reported intra-frame prediction mode and data from previously decoded blocks of the current picture. When a video image is encoded as an inter-frame coded segment (i.e., B or P), the inter-prediction block 344 (eg, motion compensation block) of the mode apply block 360 is configured to generate prediction blocks 365 for the video block of the current video segment based on motion vectors and others. syntax elements received from block 304 entropy decoding. For inter prediction, prediction blocks may be created from one of the reference pictures within one of the reference pictures contained in the reference picture lists. Video decoder 30 may create reference frame lists, list 0 and list 1 using default construction techniques based on reference pictures stored in DPB 330. The same or similar may be applied by embodiments using tiled groups (eg, video tiles) and/ or tiles (eg, video tiles) in addition to or alternatively to segments (eg, video segments), for example, video may be encoded using tiles I, P, or B and/or tiles.

The mode applying block 360 is configured to determine prediction information for a video block of the current video segment by parsing motion vectors or related information and other syntax elements, and uses the prediction information to create prediction blocks of the current video block being decoded. For example, the mode applicator 360 uses some of the received syntax elements to determine the prediction mode (eg, intra or inter prediction) used to encode the video blocks of a video segment, the type of inter prediction segment (eg, B segment, P segment, or GPB segment), information about building one or more lists of segment reference pictures, motion vectors for each video block of the segment with interframe coding, and other information for decoding video blocks in the current video segment. The same or similar may be applied in embodiments using tiles (eg, video tiles) and/or tiles (eg, video tiles) in addition to or alternatively to segments (eg, video segments), e.g., video may be encoded using tiles I , P or B and/or tilings.

Embodiments of video decoder 30, as shown in FIG. 3 may be configured to split and/or decode a picture using segments (also referred to as video segments), where an image may be split or decoded using one or more segments (usually non-overlapping), and each segment may contain one or more blocks ( e.g. CTU).

Embodiments of video decoder 30, as shown in FIG. 3 may be configured to split and/or decode an image using tiles (also referred to as video tiles) and/or tiles (also referred to as video tiles), where an image may be split or decoded using one or more (typically non-overlapping) tiles. groups, and each tile group may contain, for example, one or more blocks (eg, CTU) or one or more tiles, where each tile may be, for example, rectangular in shape and may contain one or more blocks (eg, CTU), for example , full or fractional blocks.

Other variants of video decoder 30 may be used to decode encoded picture data 21 . For example, decoder 30 may produce an output video stream without loop filtering unit 320 . For example, decoder 30 with no transform for certain blocks or frames may inversely quantize the residual signal directly without inverse transform block 312 . In another implementation, video decoder 30 may have an inverse quantizer 310 and an inverse transform 312 combined into a single unit.

It should be understood that in encoder 20 and decoder 30, the processing result of the current step may be further processed and then output to the next step. For example, after interpolation filtering, motion vector acquisition, or loop filtering of the interpolation filter result, motion vector acquisition, or loop filtering, additional operations such as clipping or shifting may be performed.

It should be noted that additional operations can be applied to the received motion vectors of the current block (including, but not limited to, affine mode controlled point motion vectors, sub-block motion vectors in affine, planar ATMVP modes, temporal motion vectors, and so on) . For example, the value of a motion vector is limited to a predetermined range corresponding to its representing bit. If the representing bit of the motion vector is bitDepth, then the range is -2^(bitDepth-1) ~ 2^(bitDepth-1)-1, where “^” means exponentiation. For example, if bitDepth is set to 16, the range is -32768 ~ 32767; if bitDepth is set to 18, the range is -131072~131071. For example, the value of the obtained motion vector (e.g., MV of four 4x4 subblocks in one 8x8 block) is limited such that the maximum difference between integer parts of four 4x4 MV subblocks does not exceed N pixels, for example, not more than 1 pixel. Here, two ways of limiting the motion vector corresponding to bitDepth are provided.

Method 1: Removing the MSB (Most Significant Bit) of the overflow through stream pass operations.

ux = (mvx+2 ^bitDepth ) % 2 ^bitDepth (1) mvx = (ux>= 2 ^bitDepth-1 ) ? (ux − 2 ^bitDepth ): ux (2) uy = (mvy+2 ^bitDepth ) % 2 ^bitDepth (3) mvy = (uy>= 2 ^bitDepth-1 ) ? (uy − 2 ^bitDepth ): uy (4),

where mvx is the horizontal component of the motion vector of the image block or sub-block, mvy is the vertical component of the motion vector of the image block or sub-block, and ux and uy are intermediate values.

For example, if the value of mvx after formulas (1) and (2) is -32769, the resulting value is 32767. In a computer system, decimal numbers are stored as two's complement. The two's complement of -32769 is 1.0111,1111,1111,1111 (17 bits), then the MSB is discarded, so the resulting two's complement is 0111,1111,1111,1111 (decimal 32767), which is the same , as the result obtained by applying formulas (1) and (2).

ux = (mvpx + mvdx +2 ^bitDepth ) % 2 ^bitDepth (5) mvx = (ux >= 2 ^bitDepth-1 )? (ux − 2 ^bitDepth ): ux (6) uy = (mvpy + mvdy +2 ^bitDepth ) % 2 ^bitDepth (7) mvy = (uy >= 2 ^bitDepth-1 )? (uy − 2 ^bitDepth ): uy (8)

Operations can be applied during the summation of mvp and mvd, as shown in formulas (5)-(8).

Method 2: Removing the MSB Overflow by Trimming the Value

vx = Clip3 (-2 ^bitDepth-1 , 2 ^bitDepth-1 - 1, vx)

vy = Clip3 (-2 ^bitDepth-1 , 2 ^bitDepth-1 - 1, vy),

where vx is the horizontal component of the motion vector of the image block or subblock, vy is the vertical component of the motion vector of the image block or subblock; x, y and z correspond to the three input values of the clipping process MV, respectively, and the definition of the Clip3 function is as follows:

In FIG. 4 schematically shows a video encoding device 400 according to an embodiment of the disclosure. The video encoder 400 is suitable for implementing the disclosed embodiments as described herein. In an embodiment, video encoder 400 may be a decoder, such as video decoder 30 shown in FIG. 1A, or may be an encoder such as video encoder 20 shown in FIG. 1A.

The video encoder 400 includes input ports 410 (or input ports 410) and receiver (Rx) units 420 for receiving data; a processor, logical unit, or central processing unit (CPU) 430 for data processing; transmitter (Tx) blocks 440 and output ports 450 (or output ports 450) for data transmission; and a memory 460 for storing data. Video encoder 400 may also include optical-to-electrical (OE) components and electrical-to-optical (EO) components associated with input ports 410, receiver units 420, transmitter unit 440, and output ports 450 for input. and output of optical or electrical signals.

Processor 430 is implemented in hardware and software. Processor 430 may be implemented as one or more CPU chips, a core (eg, as a multi-core processor), an FPGA, an ASIC, and a DSP. Processor 430 communicates with input ports 410, receiver units 420, transmitter units 440, output ports 450, and memory 460. Processor 430 includes an encoder 470. Encoding module 470 implements the embodiments disclosed and described above. For example, encoding module 470 implements, processes, prepares, or provides various encoding operations. The introduction of the encoding module 470 therefore provides a significant improvement in the functionality of the video encoding device 400 and has the effect of converting the video encoding device 400 to another state. Alternatively, encoding module 470 is implemented as instructions stored in memory 460 and executed by processor 430.

Memory 460 may comprise one or more disks, tape drives, and solid state drives, and may be used as an overflow storage device for storing programs when such programs are selected for execution, and storing instructions and data that are read during program execution. Memory 460 may be, for example, volatile and/or non-volatile and may be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and /or static random-access memory (SRAM).

In FIG. 5 is a simplified block diagram of a device 500 that can be used separately as data source device 12 and destination device 14 shown in FIG. 1, or together, in accordance with an exemplary embodiment.

Processor 502 of device 500 may be a central processing unit. Alternatively, processor 502 may be any other type of device or multiple devices capable of managing or processing existing or future information. While the disclosed implementations may be practiced by a single processor, as illustrated by processor 502, for example, the speed and efficiency advantages can be achieved with multiple processors.

Memory 504 in device 500, when implemented, may be a Read Only Memory (ROM) device or a Random Access Memory (RAM) device. The memory 504 may be any other suitable type of storage device. Memory 504 may contain code and data 506 that is accessed by processor 502 using bus 512. Memory 504 may further comprise an operating system 508 and application programs 510, application programs 510 comprising at least one program that allows processor 502 to execute the functions described herein. ways. For example, application programs 510 may comprise applications 1-N, which further comprise a video encoding application that performs the methods described herein.

The device 500 may also include one or more output devices, such as a display 518. The display 518, in one example, may be a touch display that combines a display with a touch element capable of acting upon its touch as input. Display 518 may be coupled to processor 502 via bus 512.

Although a single bus is shown in the drawing, bus 512 of device 500 may be comprised of multiple buses. Additionally, the external storage device 514 may be directly connected to other components of the device 500 or accessed via a network and may comprise a single integrated unit, such as a memory card, or multiple units, such as multiple memory cards. Device 500 may thus be implemented in a wide variety of configurations.

The inter prediction block 244 may comprise a motion estimation (ME) block and a motion compensation (MC) block (not shown in FIG. 2). The motion estimator is configured to receive or obtain an image block 203 (current image block 203 from the current image 201) and a decoded image 331, or at least one or more of previously reconstructed blocks, e.g., reconstructed blocks of one or a plurality of other/different previously 331 decoded pictures for motion estimation. For example, the video sequence may comprise the current picture and previously decoded pictures 331 or, in other words, the current picture and previously decoded pictures 331 may be part of or form a sequence of pictures that make up the video sequence. The encoder 200 may, for example, be configured to select a reference block from a plurality of reference blocks of the same or different pictures from a plurality of other pictures, and provide the reference picture motion estimate (or reference picture index, ...) and/or offset (spatial offset) to the block. ) between the position (x, y coordinates) of the reference block and the position of the current block as inter prediction parameters (not shown in FIG. 2). This displacement is also called the motion vector (MV). Merging is an important motion estimation tool used in HEVC and transmitted by VVC.

To perform the merge evaluation, a list of possible merge candidates is created, where each candidate in the candidate list contains motion data, this motion data contains information that indicates whether one or two reference picture lists are used, and also contains other information such as a reference index and motion vector for each list. In this example, a list of possible merge candidates is created based on the following candidates:

a. up to four spatial merger candidates, which are derived from five spatial neighboring blocks;

b. one temporary merger candidate derived from two temporary, merged blocks;

c. additional merge candidates, including joint bi-predictable candidates and null motion vector candidates.

The first candidates in the merge candidate list are spatial neighbors. Up to four candidates are entered into the merge list by successively checking A1, B1, B0, A0, and B2, in that order, according to the right side of FIG. 6.

In addition to checking whether a coding block is available and contains motion information, some additional redundancy checks are performed before accepting all motion data of a coding block as a merge candidate. These redundancy checks can be divided into two categories: a. To avoid redundant motion data in the list; b. To prevent two sections from being merged, which may be expressed by other means capable of creating redundant syntax.

In this example, N is the number of spatial merge candidates, a full redundancy check may contain (N • (N-1))/2 motion data comparisons. In the case of five potential spatial merge candidates, ten motion data comparisons are needed to ensure that all candidates in the merge list have different motion data. During the development of HEVC, checks for redundant motion data were reduced to a subset in a manner that preserves coding efficiency while the comparison logic is greatly reduced. As a result, no more than two comparisons are performed per candidate, resulting in five total comparisons. Given the order {A1, B1, B0, A0, B2}, B0 only checks B1, A0 only checks A1, and B2 only checks A1 and B1. In the split redundancy test example, the lower PU of the 2N×N split is merged with the upper PU by selecting candidate B1. This may result in one CU with two PUs having the same traffic data, which may be reported as 2N×2N CUs in the same way. In general, this check is applied to all second PUs of 2N×N, 2N×nU, 2N×nD, N×2N, nR×2N and nL×2N rectangular and asymmetric parts. Note that for spatial merge candidates, only redundancy checks are performed and motion data is copied from candidate candidate blocks. Therefore, no scaling of the motion vectors is required here.

The derivation of motion vectors for a temporal merge candidate is the same as the Temporal Motion Vector Prediction (TMVP) vector. Because the merge candidate contains all motion data and the TMVP is only one motion vector (meaning that only one TMVP candidate is allowed at a regular merge), the acquisition of motion data depends on the segment type. For bidirectional predictive segments, a TMVP is obtained for each reference picture list. Depending on the availability of TMVP for each list, the prediction type is set to bidirectional prediction or to the list for which TMVP is available. All associated reference picture indexes are set to zero. For segments with TMVP unidirectional prediction for the list, 0 is obtained along with a reference picture index of zero.

When at least one TMVP is available and a temporary merge candidate is added to the list, no redundancy check is performed. This makes the construction of the merge list independent of the merged image, which improves robustness. Consider the case where a temporary merge candidate may be redundant and therefore not included in the list of merge candidates. In the case of a lost merged picture, the decoder could not obtain temporary candidates and therefore cannot check if they are redundant. This may affect the indexing of all subsequent candidates.

To parse the reasons for stability, the number of candidates in the list of candidates for merging is fixed. After the spatial and temporal merge candidates are added, the list may be incomplete (the number of candidates in the merge candidate list is less than a fixed number). Additional candidates are generated to compensate for the loss in coding efficiency that occurs when signaling the incomplete adaptive list index. Depending on the type of segment, up to two types of candidates are used to complete the list: a. Joint candidates with bidirectional prediction; b. Candidates with zero motion vector.

In bidirectional prediction segments, additional candidates can be generated from existing ones by combining one candidate's reference picture list 0 motion data with another candidate's list 1 motion data. This is done by copying Δx0, Δy0, Δt0 from one candidate, eg the first candidate, and Δx1, Δy1, Δt1 from another, eg the second candidate. Various combinations are defined and are listed in Table 1.

Table 1

Combination order 0 1 2 3 4 5 6 7 8 9 10 eleven Δx0, Δy0, Δt0 from candidate 0 1 0 2 1 2 0 3 1 3 2 3 Δx0, Δy0, Δt0 from candidate 1 0 2 0 2 1 3 0 3 1 3 2

When the list is still incomplete after adding combined bidirectional predictive candidates or for unidirectional predictive segments, null motion vector candidates are added. The null motion vector candidates have one motion vector with zero displacement for unidirectional prediction segments and two motion vectors with zero displacement displacement for bidirectional prediction segments. The pivot indices are set to zero and incremented by one for each additional candidate until the maximum pivot index is reached. If more additional candidates need to be added, a reference index of zero is used to create additional candidates. For additional candidates, no redundancy checks are performed, since omission of these checks does not introduce a loss in coding efficiency.

For each PU encoded in the inter prediction mode, a merge flag merge_flag is used to indicate whether a merge block is used to obtain motion data. merge_idx is used to determine a merge list candidate that provides the motion data needed for motion compensated prediction (MCP). In addition to signaling the PU layer, the number of merge list candidates is reported in the segment header. In this example, the default value is five, which is represented as the difference up to five (five_minus_max_num_merge_cand). As for the process of building the list of merge candidates, the whole process remains the same, although it ends after the maximum number of merge candidates has been included in the list. In the initial construction, the maximum merge index encoding value was given by the number of available spatial and temporal candidates in the list. When only two candidates are available, the index can be effectively encoded as a flag. In order to parse the merge index, the entire list of merge candidates must be built to know the actual number of candidates.

The use of block merging in HEVC is combined with the skip mode. The skip mode is used for a block to indicate that the motion data for the block is finally received instead of being explicitly reported in the bitstream, and that the prediction residual for the block is zero, i.e. no transform coefficients are transmitted. In HEVC, a skip_flag is reported at the beginning of each CU in an interframe segment, which implies the following: a. CU contains only one PU (partition type 2N×2N); b. Merge mode is used to get motion data (merge_flag is 1); c. No residual data is present in the bitstream.

Another motion estimation tool introduced in HEVC and native to VVC is called Advanced Motion Vector Prediction (AMVP). In AMVP mode, motion vectors are encoded in terms of horizontal (x) and vertical (y) components as differences, resulting in a so-called motion predictor (MVP). The calculation of the components of the motion vector difference (MVD) is represented as MVDx = MVx – MVPx, MVDy = MVy – MVPy.

The motion vectors of the current block are usually correlated with the motion vectors of neighboring blocks in the current picture or in previously encoded pictures. Since adjacent blocks are likely to correspond to the same moving object with similar motion, it is unlikely that the object's motion will change dramatically over time. Therefore, using motion vectors in adjacent blocks as predictors reduces the size of the reported motion vector difference. The MVP is typically derived from decoded motion vectors from spatial neighbors or from temporally neighbors in a merged image. In HEVC, the implicit MVP extraction approach used to derive the motion vector has been replaced by a technology known as motion vector contention, which explicitly reports the MVP from the MVP list. The variable block structure of the coding quadtree in HEVC may result in a single block having multiple neighboring blocks with motion vectors as potential MVP candidates. Taking the left neighbor as an example, in the case where a 64x64 luminance prediction block can have 16 8x4 luminance prediction blocks on the left, when the 64x64 luma coding tree block is not further divided, the remaining block is divided by the maximum depth. Advanced Motion Vector Prediction (AMVP) has been introduced to modify motion vector contention to account for such a flexible block structure. During the development of HEVC, the initial construction of AMVP was greatly simplified to provide a good trade-off between coding efficiency and user-friendly implementation construction.

The initial AMVP build contains five MVPs from three different classes of predictors: three motion vectors from spatial neighbors, a median of three spatial predictors, and a scaled motion vector from a co-located, temporally adjacent block. Additionally, the list of predictors has been modified by ordering to place the most likely motion predictor in first place and by removing redundant candidates to ensure minimal auxiliary signaling. Exhaustive experimentation throughout the standardization process has made it possible to study how the complexity of this motion vector prediction and message scheme can be reduced without sacrificing too much coding efficiency. This leads to significant simplifications in the construction of AMVP, such as removing the median predictor, reducing the number of candidates in the list from five to two, fixing the order of candidates in the list, and reducing the number of redundancy checks. Building a list of AMVP candidates contains the following two MVP candidates:

up to two spatial MVP candidates, which are derived from five spatial neighboring blocks;

one temporal MVP candidate derived from two temporal, co-located blocks when both spatial MVP candidates are unavailable or identical;

null motion vectors when spatial, temporal, or both candidates are not available.

In describing the spatial candidates, the production path for two spatial candidates A and B is shown in FIG. 13. For candidate A, the movement data from these two blocks A0 and A1 in the lower left corner is taken into account in the two-pass approach. In the first pass, it is checked whether any of the candidate blocks contains a reference index equal to the reference index of the current block. The first found motion vector will be taken as candidate A. When all reference indices from A0 and A1 point to a reference picture other than the reference index of the current block, the corresponding motion vector cannot be used as it is. Therefore, on the second pass, the motion vectors must be scaled according to the temporal distances between the candidate reference picture and the current reference picture. The temporal distance is expressed in terms of the difference between the picture order count (POC) values, with the POC values used to determine the display order of the pictures.

For candidate B, candidates B0-B2 are tested sequentially in the same way as A0 and A1. The second pass, however, is performed when blocks A0 and A1 do not contain any motion information, i.e. not available or encoded using intra prediction. Candidate A is then set equal to the unscaled candidate B, if found, and candidate B is set equal to the second unscaled or scaled candidate B candidate. MV received from candidates B0-B2. In general, this construction allows processing A0 and A1 independently of B0, B1 and B2. When B is received, both A0 and A1 must be aware of the availability to search for the scaled or additional unscaled MV received from B0-B2. This dependency greatly reduces the complex motion vector scaling operations for candidate B. Reducing the number of motion vector scalings represents a significant reduction in the complexity of the motion vector predictor derivation process.

In the process of choosing an interim candidate. As can be seen from FIG. 6, motion vectors from spatial neighboring blocks to the left and above the current block are considered as spatial MVP candidates. This can be explained by the fact that the blocks to the right and below the current block have not yet been decoded and therefore their motion data is missing. Since the merged picture is a reference picture that has already been decoded, one can also consider motion data from a block at the same position, from blocks to the right of the merged block, or from lower blocks. In HEVC, the block at the bottom right and center of the current block was determined to be fully capable of providing a good temporal motion vector predictor (TMVP). These candidates are shown in FIG. 6, where C0 represents the bottom right neighbor and C1 represents the center block. Motion data C0 is considered first, if motion data C0 is not available, the motion data of the co-located candidate block at the center is used to derive the temporary MVP candidate C. C0 motion data is also considered to be unavailable when the companion PU belongs to a CTU outside of the current CTU row. This minimizes the memory bandwidth requirement to store the combined motion data. Unlike spatial MVP candidates where motion vectors can refer to the same reference picture, motion vector scaling is mandatory for TMVP.

Building a merge list and an AMVP list at the same time, a history based motion vector predictor (HMVP) must merge or enter the AMVP merge list. History-based MVP (HMVP) merge candidates are added to the merge/AMVP list after the spatial MVP and TMVP. With this method, the motion information of the previously encoded block is stored in a table and used as the MVP for the current CU. A table with multiple HMVP candidates is maintained during the encoding/decoding process. The table is reset (emptied) when a new CTU row is inserted. Each time a non-subblock interframe coded CU exists, the associated motion information is added to the last entry in the table as a new HMVP candidate.

In VTM4, the size S of the HMVP table is set to 6, which indicates that up to 6 HMVP candidates can be added to the table. When inserting a new motion candidate into a table, a limited first-in-first-out (FIFO) rule is used in which a redundancy check is first applied to determine if there is an identical HMVP in the table. If an identical HMVP exists in the table, the identical HMVP is removed from the table and all HMVP candidates are further promoted.

HMVP candidates may be used in the merge candidate list/AMVP list building process. The most recent few HMVP candidates in the table are checked in order and listed after the TMVP candidate. A redundancy check is applied on HMVP candidates for a spatial or temporal merge candidate.

To reduce the number of redundancy check operations, the following simplifications are introduced:

Is the number of HMPV candidates used to form the merge list set to (N <= 4)? M: (8 - N) where N indicates the number of existing candidates in the merge list and M indicates the number of available HMVP candidates in the table.

When the total number of available merge candidates is equal to the maximum number of merge candidates allowed minus 1, the process of building the merge candidate list from the HMVP ends.

In the VVC Draft project, similar to the Inter mode, the IBC mode is introduced.

Block copy with intra-frame prediction (Intra block copy, IBC) is a tool adopted in the HEVC extensions on SCC. It greatly improves the efficiency of encoding screen content materials. Since the IBC mode is implemented as a block-level coding mode, block matching (BM) is performed in the encoder to find the optimal block vector (or motion vector) for each CU. Here, the motion vector is used to indicate the offset from the current box to a reference box that has already been reconstructed inside the current image. The luminance motion vector of a CU encoded using IBC has integer precision. The chroma motion vector is also truncated with integer precision. When combined with AMVR, the IBC mode can switch between 1-element precision and 4-element motion vector precision. A CU encoded using IBC is regarded as a third prediction mode other than intra or inter modes.

To reduce memory consumption and decoder complexity, IBC in VTM4 allows the use of a reconstructed section of a given area containing the current CTU. This limitation allows hardware implementations to implement IBC mode using local on-chip memory.

On the encoder side, the hash-based motion estimation is performed for the IBC. The encoder performs an RD check for blocks that have a width or height of no more than 16 luma samples. For non-merge mode, block vector searches are performed using a hash-based search first. If the hash lookup does not yield a valid candidate, block matching will be performed based on the local lookup.

In a hash-based search, a hash key match (32-bit CRC) between the current block and the reference block extends to all allowed block sizes. The hash key calculation for each position in the current image is based on 4x4 sub-blocks. For the current larger block, the hash key is determined to match the hash key of the reference block when all hash keys of all 4×4 sub-blocks match the hash keys at the respective reference locations. If it is determined that the hash keys of the set of reference blocks match the current block, the vector block cost for each of the matching reference blocks is calculated and the block with the lowest cost is selected.

When searching for a block match, the search range is set to N samples to the left and up of the current block within the current CTU. At the start of the CTU, the value of N is initialized to 128 if there is no temporal reference picture, or is initialized to 64 if at least one temporal reference picture exists. The hash hit rate is defined as the percentage of samples in the CTU that are matched using a hash-based search. When encoding the current CTU, if the hash hit rate is below 5%, N is halved.

At the CU level, the IBC mode is signaled with a flag and may be signaled as the AMVP IBC mode or the IBC pass/merge mode as follows:

IBC Skip/Merge Mode: The merge candidate index is used to indicate which block vectors in the list of neighboring encoded IBC candidate blocks are used to predict the current block. The merge list consists of spatial, HMVP, and pairwise candidates.

IBC AMVP mode: block vector difference is encoded in the same way as motion vector difference. The block vector prediction method uses two candidates as predictors, one from the left neighbor and one from the top neighbor (if IBC coded). When one of the neighbors is unavailable, the default block vector will be used as a predictor. The flag is reported to indicate the block vector predictor index.

Since IBC introduced IBC merge/skip mode and IBC AMVP mode, there is an additional IBC merge list and AMVP list that will be created in VVC Draft 4.0.

Since only the spatial candidates (Left Neighbor A1, Top Neighbor B1, Left Bottom Neighbor A0, Right Top Neighbor B0, and Left Top Neighbor B2 shown in Fig. 6), the HMVP candidates (H1 ... Hk, where k is maximum HMVP list size) and pairwise candidates are used when building the IBC merge list in VVC Draft 4.0, the following reductions are possible when building the IBC merge list:

Reduction between A1 and B1

Reduction between A0 and A1

Reduction between B0 and B1

Reduction between B2 and A1

Reduction between B2 and B1

Reduction between last candidate Hk HMVP and A1

Reduction between last candidate Hk HMVP and B1

Reduction between penultimate candidate H _k-1 HMVP and A1

Reduction between penultimate candidate H _k-1 HMVP and B1

The reduction process means comparing whether two IBC merger candidates are the same. Namely, the reduction process compares whether the block vectors between two IBC merge candidates are the same.

As a result, to build the IBC merge list of the current block, a maximum of 9 reductions are needed.

In the encoder and decoder, the reduction is a delay in the merge list building process. Since the “if” condition is checked at each step of the reduction, the further process of building the merge list is performed depending on the checks of this “if” condition. The more shortcuts there are in constructing the merge candidate list, the more complex the processes in the encoder and decoder are. To reduce the complexity when building a merge list, the following solutions are introduced.

Solution 1

Further, the left (A1) and top (B1) spatial neighbor blocks are important for predicting the current block using the IBC mode. In Solution 1, the spatial neighbor reduction between A1 and B1 is retained, the remaining spatial neighbor reduction is removed. The reduction in HMVP candidates is maintained. In an embodiment, the following reductions are possible during construction of the IBC merge list:

Reduction between A1 and B1

Reduction between last candidate H _k HMVP and A1

Reduction between last candidate H _k HMVP and B1

Reduction between penultimate candidate H _k-1 HMVP and A1

Reduction between penultimate candidate H _k-1 HMVP and B1.

In this case, the maximum number of reductions for building the IBC merge candidate list is reduced from 9 to 5. The solution greatly reduces the complexity of building the IBC merge list for both the encoder and the decoder.

In this example, a specific IBC merge list is created as follows:

If neighboring block A1 is available and using IBC mode, the block vector of block A1 is inserted into the current block's IBC merge candidate list. Otherwise (A1 is not available or A1 is not using IBC mode), the block vector of block A1 is not inserted into the current block's IBC merge candidate list (first candidate, no reduction).

If neighboring block B1 is available and using IBC mode, reduce (or determine) whether block vector B1 is the same as A1. If block vector B1 is not the same as block vector A1, the block vector of block B1 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B1 is the same as block vector A1, or B1 is not available, or B1 does not use IBC mode), the block vector of block B1 is not inserted into the current block's IBC merge candidate list (abbreviation A1 and B1).

If a neighboring block B0 is available and using the IBC mode, the block vector of block B0 is inserted into the current block's IBC merge candidate list. Otherwise (B0 is not available or B0 is not using IBC mode), the block vector of block B0 is not inserted into the current block's IBC merge candidate list (no pruning).

If a neighboring block A0 is available and using the IBC mode, the block vector of block A0 is inserted into the current block's IBC merge candidate list. Otherwise (A0 is not available or A0 is not using IBC mode), the block vector of block A0 is not inserted into the current block's IBC merge candidate list (no pruning).

If neighboring block B2 is available and using the IBC mode and the current size of the IBC merge candidate list is less than 4, the block vector of block B2 is inserted into the current block's IBC merge candidate list. Otherwise (B2 is not available, or B2 is not using IBC mode, or the current IBC merge candidate list size is at least 4), B2's block vector is not inserted into the current block's IBC merge candidate list (no reduction).

If the last candidate H _k HMVP is available and using the IBC mode and the current size of the IBC merge candidate list is less than the value equal to the maximum number of IBC merge candidates minus one; and the block vector H _k is not the same as A1 and B1, insert the block vector H _k into the IBC merge candidate list of the current block. Otherwise (H _k is not available or H _k is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector H _k is the same as block vector A1, or block the vector H _k is the same as the block vector B1), do not insert the block vector Hk into the IBC merge candidate list of the current block (abbreviation H _k and A1, H _k and B1).

If the penultimate Hk-1 HMVP candidate is available and using IBC mode and the current size of the IBC merge list is less than the value equal to the maximum number of IBC merge candidates minus one and the block vector H _k-1 is not the same as A1 and B1, inserting the block vector H _k-1 into the IBC merge candidate list of the current block. Otherwise (H _k-1 is not available or H _k-1 is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector H _k-1 is the same block vector A1, or block vector H _k-1 is the same as block vector B1), do not insert block vector H _k-1 into the IBC merge candidate list of the current block (abbreviation H _k-1 and A1, H _k-1 and B1).

The remaining HMVP candidates are entered one at a time if the candidate is available and using the IBC mode and the current size of the IBC merger candidate list of candidates is less than the maximum number of IBC merger candidates minus one (no reduction).

Pairwise candidates are entered if the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates (no reduction).

Solution 2

Further, the left (A1) and top spatial neighboring blocks (B1) are important for predicting the current block using the IBC mode. In Solution 2, spatial neighbor block A1 and B1 are retained for inclusion in the IBC merge candidate list, the remaining spatial neighbor block candidates are removed. The reduction of HMVP candidates is kept as is. In an embodiment, the following reduction may be performed during the construction of the IBC merge list:

Reduction between A1 and B1

Reduction between last HMVP candidate H _k and A1

Reduction between last candidate H _k HMVP and B1

Reduction between penultimate candidate H _k-1 HMVP and A1

Reduction between penultimate candidate H _k-1 HMVP and B1

In this case, the maximum number of reductions for building the IBC merge candidate list of the current block is reduced from 9 to 5. The solution greatly reduces the complexity of building the IBC merge list for both the encoder and the decoder.

In this example, a specific list of IBC merger candidates is created as follows:

If neighboring block A1 is available and using IBC mode, insert the block vector of block A1 into the IBC merge candidate list of the current block. Otherwise (A1 is not available or A1 is not using IBC mode), do not insert the block vector of block A1 into the current block's IBC merge candidate list (first candidate, no reduction).

If neighboring block B1 is available and using the IBC mode, reduce (or determine) whether block vector B1 is the same as A1. If block vector B1 is not the same as block vector A1, the block vector of block B1 is inserted into the IBC merge candidate list of the current block. Otherwise (B1's block vector is the same as A1's block vector, or B1 is not available, or B1 does not use IBC mode), do not insert A1's block vector into the current block's IBC merge candidate list (A1 and B1 abbreviation).

If the last HMVP candidate H _k is available and uses the IBC mode and the size of the current IBC merge candidate list is less than a value equal to the maximum number of IBC merge candidates minus one, and the block vector H _k is not the same as A1 and B1, insert block vector H _k in the list of IBC merge candidates of the current block. Otherwise, (H _k is not available or H _k is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector Hk is the same as the block vector A1 , or block vector H _k is the same as block vector B1), do not insert block vector H _k into the IBC merge candidate list of the current block (abbreviated as H _k and A1, H _k and B1).

If the penultimate HMVP candidate H _k -1 is available and using IBC mode, and the current size of the IBC merge candidate list is less than the value equal to the maximum number of IBC merge candidates minus one, and the block vector H _k-1 is not the same as A1 and B1 insert the block vector H _k-1 into the IBC merge candidate list of the current block. Otherwise (H _k-1 is not available, or H _k-1 is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector H _k-1 is the same as block vector A1, or block vector H _k-1 is the same as block vector B1), do not insert block vector H _k-1 into the IBC merge candidate list of the current block (abbreviation H _k-1 and A1, H _{k- 1} and B1).

Enter the remaining HMVP candidates one by one if the candidate is available and using the IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one (no reduction).

A pairwise candidate is entered if the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates (no reduction).

Solution 3

According to solution 3, the reduction of the space neighbor block vectors remains the same. To reduce the HMVP candidates, the block vector of the last HMVP candidate H _k is reduced by the space neighbor block A1 and B1 of the vector block. The reduction of the remaining HMVP candidates is removed. In an embodiment, the following reduction is possible during construction of the IBC merge list:

Reduction between A1 and B1

Reduction between A0 and A1

Reduction between B0 and B1

Reduction between B2 and A1

Reduction between B2 and B1

Reduction between last candidate H _k HMVP and A1

Reduction between the last candidate H _k HMVP and B1.

In this case, the maximum number of cuts for building the IBC merge candidate list is reduced from 9 to 7. The solution greatly reduces the complexity of building the IBC merge list for both the encoder and the decoder.

In this example, a specific list of IBC merger candidates is created as follows:

If neighboring block A1 is available and using IBC mode, insert the block vector of block A1 into the IBC merge candidate list of the current block. Otherwise (A1 is not available or A1 is not using IBC mode), do not insert the block vector of block A1 into the current block's non-IBC candidate list (the first candidate, no reduction).

If neighboring block B1 is available and using IBC mode, reduce (or determine) whether block vector B1 is the same as A1. If block vector B1 is not the same as block vector A1, insert the block vector of block B1 into the IBC merge candidate list of the current block. Otherwise (B1's block vector is the same as A1's block vector, or B1 is not available, or B1 does not use IBC mode), do not insert A1's block vector into the current block's IBC merge candidate list (A1 and B1 abbreviation).

If neighboring block B0 is available and using IBC mode, reduce (or determine) whether block vector B0 is the same as B1. If block vector B0 is not the same as block vector B1, insert the block vector of block B0 into the IBC merge candidate list of the current block. Otherwise (block vector B0 is the same as block vector B1, or B0 is not available, or B0 does not use IBC mode), do not insert block vector B0 into the IBC merge candidate list of the current block (B0 and abbreviation B1).

If neighboring block A0 is available and using the IBC mode, reduce (or determine) whether block vector A0 is the same as A1. If block vector A0 is not the same as block vector A1, insert the block vector of block A0 into the IBC merge candidate list of the current block. Otherwise (block vector A0 is the same as block vector A1, or A0 is not available, or A0 is not in IBC mode), do not insert block vector A0 into the current block's IBC merge candidate list (A0 and A1 abbreviation).

If neighboring block B2 is available and using IBC mode and the current IBC merge list size is less than 4, reduce (or determine) whether block vector B2 is the same as A1 and whether block vector B2 is the same as B1. If block vector B2 is not the same as block vector A1 and block vector B2 is not the same as B1, the block vector of block B2 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B2 is the same as block vector A1 or B1 or B0 is not available or B0 does not use IBC mode), do not insert block vector B0 into the current block's IBC merge candidate list (reduce B2 and A1 and reduce B2 B1 ).

If the last HMVP candidate H _k is available and using IBC mode and the current list size IBC candidate list size is less than a value equal to the maximum number of IBC merge candidates minus one, and the block vector H _k is not the same as A1 and B1, insert the block vector H _k in the list of IBC merger candidates of the current block. Otherwise (H _k is not available or H _k is not using IBC mode or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector H _k is the same as block vector A1, or block vector H _k is the same as block vector B1), do not insert block vector H _k into the IBC merge candidate list of the current block (abbreviation H _k and A1, H _k and B1).

Enter the remaining HMVP candidates one by one if the candidate is available and using the IBC mode and the current size of the IBC candidate list is less than the number equal to the maximum number of IBC merger candidates minus one (no reduction).

A pairwise candidate is entered if the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates (no reduction).

Solution 4

According to decision 4, the vector block reduction for the spatial neighboring block remains the same. To reduce the HMVP candidates, the block vector of the last HMVP candidate H _k and the penultimate HMVP candidate H _k-1 are reduced by the spatial neighbor block A1 of the vector block. The remaining HMVP candidate abbreviations are removed. In an embodiment, the following reduction is possible during construction of the IBC merge list:

Abbreviation between A1 and B1

Abbreviation between A0 and A1

Reduction between B0 and B1

Abbreviation between B2 and A1

Reduction between B2 and B1

Reduction between last candidate H _k HMVP and A1

Reduction between last candidate H _k-1 HMVP and A1

In this case, the maximum number of reductions for building the IBC merge candidate list is reduced from 9 to 7. The solution greatly reduces the complexity of building the IBC merge list for both the encoder and the decoder.

In this example, a specific list of IBC merger candidates is created as follows:

If neighboring block A1 is available and using IBC mode, insert the block vector of block A1 into the IBC merge candidate list of the current block. Otherwise (A1 is not available or A1 is not using IBC mode), do not insert block block vector A1 into the current block's IBC merge candidate list (first candidate, no reduction).

If neighboring block B1 is available and using IBC mode, reduce (or determine) whether vector block B1 is the same as A1. If block vector B1 is not the same as block vector A1, block vector B1 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B1 is not the same as block vector A1, or B1 is not available, or B1 does not use IBC mode), do not insert block vector A1 into the current block's IBC merge candidate list (abbreviation A1 and B1).

If neighboring block B0 is available and using IBC mode, reduce (or determine) whether vector block B0 is the same as B1. If block vector B0 is not the same as block vector B1, insert the block vector of block B0 into the IBC merge candidate list of the current block. Otherwise (block vector B0 is the same as block vector B1, or B0 is not available, or B0 does not use IBC mode), do not insert block vector B0 into the current block's IBC merge candidate list (abbreviation B0 and B1).

If neighboring block A0 is available and using IBC mode, reduce (or determine) whether vector block A0 is the same as A1. If block vector A0 is not the same as block vector A1, insert block vector A0 into the IBC merge candidate list of the current block. Otherwise, (block vector A0 is the same as block vector A1, or A0 is not available, or A0 does not use IBC mode), do not insert block vector A0 into the current block's IBC merge candidate list (abbreviation A0 and A1).

If adjacent block B2 is available and using IBC mode and current IBC merge list size is less than 4, reduce (or determine) whether vector block B2 is the same as A1 and whether block vector B2 is the same as B1. If block vector B2 is not the same as block vector A1 and block vector B2 is not the same as B1, block vector B2 is inserted into the IBC merge candidate list of the current block. Otherwise, (block vector B2 is the same as block vector A1 or B1, or B0 is not available, or B0 does not use IBC mode), do not insert block vector B0 into the current block's IBC merge candidate list (reduce B2 and A1 and reduce B2 and B1).

If the last HMVP candidate H _k is available and using the IBC mode and the current size of the IBC merge candidate list is less than a value equal to the maximum number of IBC merge candidates minus one, and the block vector H _k is not the same as A1, insert the block vector H _k into list of IBC merger candidates of the current bloc. Otherwise ( _Hk is not available, or _Hk is not using IBC mode, or the current size of the IBC merge candidate list is greater than or equal to the maximum number of IBC merge candidates minus one, or the block vector _Hk is the same as block vector A1), do not insert the block vector H _k into the IBC merge candidate list of the current block (abbreviation H _k and A1).

If the last HMVP candidate H _k-1 is available and using IBC mode and the current size of the IBC merge candidate list is less than the value equal to the maximum number of IBC merge candidates minus one, and the block vector H _k-1 is not the same as A1, insert the block vector vector H _k-1 in the list of IBC merge candidates of the current block. Otherwise (H _k-1 is not available or H _k-1 is not using IBC mode or the current size of the IBC merge candidate list is not less than a value equal to the maximum number of IBC merge candidates minus one, or the block vector H _k-1 is the same as as the block vector A1), do not insert the block vector H _k-1 into the IBC merge candidate list of the current block (abbreviation H _k-1 and B1).

Insert the remaining HMVP candidates one on top of the other if the candidate is available and using the IBC mode and the current size of the IBC merge candidate list is less than the number equal to the maximum number of IBC merge candidates minus one (no reduction).

Insert a pairwise candidate if the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates (no reduction).

Solution 5

According to decision 5, the vector block reduction for the spatial neighboring block remains the same. In order to reduce HMVP candidates, the block vector of the last HMVP candidate H _k and the penultimate HMVP candidate H _k-1 are reduced by the spatial neighbor block B1 of the vector block. The remaining HMVP candidate abbreviations are deleted. In an embodiment, the following reduction is possible during construction of the IBC merge list:

Abbreviation between A1 and B1

Abbreviation between A0 and A1

Reduction between B0 and B1

Abbreviation between B2 and A1

Reduction between B2 and B1

Reduction between last candidate H _k HMVP and B1

Reduction between last candidate H _k-1 HMVP and B1

In this case, the maximum number of reductions for building the IBC merge candidate list is reduced from 9 to 7. The solution greatly reduces the complexity of building the IBC merge list for both the encoder and the decoder.

In this example, a specific list of IBC merger candidates is created as follows:

If neighboring block A1 is available and using IBC mode, insert the block vector of block A1 into the IBC merge candidate list of the current block. Otherwise (A1 is not available or A1 is not using IBC mode), do not insert block block vector A1 into the current block's IBC merge candidate list (first candidate, no reduction).

If neighboring block B1 is available and using IBC mode, reduce (or determine) whether vector block B1 is the same as A1. If block vector B1 is not the same as block vector A1, block vector B1 is inserted into the IBC merge candidate list of the current block. Otherwise, (block vector B1 is not the same as block vector A1, or B1 is not available, or B1 does not use IBC mode), do not insert block vector A1 into the current block's IBC merge candidate list (abbreviation A1 and B1).

If neighboring block B0 is available and using IBC mode, reduce (or determine) whether vector block B0 is the same as B1. If block vector B0 is not the same as block vector B1, insert the block vector of block B0 into the IBC merge candidate list of the current block. Otherwise, (block vector B0 is the same as block vector B1, or B0 is not available, or B0 does not use IBC mode), do not insert block vector B0 into the current block's IBC merge candidate list (abbreviation B0 and B1).

If neighboring block A0 is available and using IBC mode, reduce (or determine) whether vector block A0 is the same as A1. If block vector A0 is not the same as block vector A1, insert block vector A0 into the IBC merge candidate list of the current block. Otherwise, (block vector A0 is the same as block vector A1, or A0 is not available, or A0 does not use IBC mode), do not insert block vector A0 into the current block's IBC merge candidate list (abbreviation A0 and A1).

If neighboring block B2 is available and using IBC mode and the current IBC merge list size is less than 4, reduce (or determine) whether block vector B2 is the same as A1 and whether block vector B2 is the same as B1. If block vector B2 is not the same as block vector A1 and block vector B2 is not the same as B1, the block vector of block B2 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B2 is the same as block vector A1 or B1, or B0 is not available, or B0 does not use IBC mode), do not insert block vector B0 into the current block's IBC merge candidate list (reduce B2 and A1 and reduce B2 B1).

If the last HMVP candidate H _k is available and using the IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, and the block vector H _k is not the same as B1, insert the block vector H _k into list of IBC merger candidates of the current bloc. Otherwise ( _Hk is not available, or _Hk is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector _Hk is the same as block vector B1), do not insert block vector H _k in the current block's IBC merge candidate list (abbreviation H _k and B1).

If the penultimate HMVP candidate H _k-1 is available and using the IBC mode and the current size of the IBC merge candidate list is less than a value equal to the maximum number of IBC merge candidates minus one, and the block vector H _k-1 is not the same as A1, insert a block vector H _k-1 into the IBC merge candidate list of the current block. Otherwise (H _k-1 is not available or H _k-1 is not using IBC mode or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector H _k-1 is the same as the block vector vector B1), do not insert block vector H _k-1 into the current block merge candidate list IBC (abbreviation H _k-1 and B1).

Insert the remaining HMVP candidates one by one if the candidate is available and using IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one (no reduction).

A paired candidate is inserted, the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates (no reduction).

Solution 6

According to solution 6, this solution resolves the last candidate of the IBC merge candidate list using HMVP mode, the merge candidate list building reduction process is not changed in this solution, but this method introduces a more efficient way to build the merge list.

In this example, a specific list of IBC merger candidates is created as follows.

If neighboring block A1 is available and using IBC mode, insert the block vector of block A1 into the IBC merge candidate list of the current block. Otherwise (A1 is not available or A1 is not using IBC mode), do not insert block block vector A1 into the current block's IBC merge candidate list (first candidate, no reduction).

If neighboring block B1 is available and using IBC mode, reduce (or determine) whether block vector B1 is the same as A1. If block vector B1 is not the same as block vector A1, the block vector of block B1 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B1 is the same as block vector A1, or B1 is not available, or B0 does not use IBC mode), do not insert block vector A1 into the current block's IBC merge candidate list (abbreviation A1 and B1).

If neighboring block B0 is available and using IBC mode, reduce (or determine) whether block vector B0 is the same as B1. If block vector B0 is not the same as block vector B1, the block vector of block B0 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B0 is the same as block vector B1, or B0 is not available, or B0 does not use IBC mode), do not insert block vector B0 into the current block's IBC merge candidate list (abbreviation B0 and B1).

If neighboring block A0 is available and using IBC mode, reduce (or determine) whether block vector A0 is the same as A1. If block vector A0 is not the same as block vector A1, insert the block vector of block A0 into the IBC merge candidate list of the current block. Otherwise (block vector A0 is the same as block vector A1, or A0 is not available, or A0 does not use IBC mode), do not insert block vector A0 into the current block's IBC merge candidate list (abbreviation A0 and A1).

If neighboring block B2 is available and using IBC mode and the current IBC merge list size is less than 4, reduce (or determine) whether block vector B2 is the same as A1 and whether block vector B2 is the same as B1. If block vector B2 is not the same as block vector A1 and block vector B2 is not the same as B1, the block vector of block B2 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B2 is the same as block vector A1 or B1, or B0 is not available, or B0 does not use IBC mode), do not insert block vector B0 into the current block's IBC merge candidate list (reduce B2 and A1 and reduce B2 and B1).

If the last HMVP candidate H _k is available and using the IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, and the block vector H _k is not the same as A1 and B1, insert the block vector H _k into list of IBC merger candidates of the current bloc. Otherwise (H _k is not available, or H _k is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates, or the block vector H _k is the same as the block vector A1, or the block vector H _k is the same as the block vector B1), do not insert the block vector H _k into the IBC merge candidate list of the current block (abbreviation H _k and A1, H _k and B1).

If the penultimate HMVP candidate H _k-1 is available and using IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates and the block vector H _k-1 is not the same as A1 and B1, insert the block vector H _k-1 to the IBC merger candidate list of the current block. Otherwise (H _k-1 is not available or H _k-1 is not using IBC mode or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates, or block vector H _k-1 is the same as block vector A1 , or block vector H _k-1 is the same as block vector B1), do not insert block vector H _k-1 into the IBC merge candidate list of the current block (abbreviation H _k-1 and A1, H _k-1 and B1) .

Insert the remaining HMVP candidates one by one if the candidate is available and using IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one (no reduction).

A paired candidate is inserted, the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates (no reduction).

Solution 7

According to solution 7, the reduction of block vectors of the spatial neighboring block is kept the same. To reduce HMVP candidates, the block vector of the last HMVP candidate H _k and the penultimate HMVP candidate H _k-1 is reduced by the block vector of the first spatial neighboring block A1 and B1. If neither of the blocks A1 or B1 is already present in the IBC merge list, then no HMVP candidate is reduced by the spatial candidate.

Abbreviations of other HMVP candidates are removed. In an embodiment, during the creation of the IBC merge list during the reduction process, the following reductions are possible in the worst case

Reduction between A1 and B1

Reduction between A0 and A1

Reduction between B0 and B1

Reduction between B2 and A1

Reduction between B2 and B1

Reduction between last candidate H _k HMVP and A1 or H _k and B1

Reduction between penultimate candidate H _k-1 HMVP and A1 or H _k-1 and B1

In this case, the maximum number of reductions to create the IBC merge candidate list of the current block is reduced from 9 to 7. The solution significantly reduces the complexity of creating the IBC merge list for both the encoder and decoder.

In this example, a specific list of IBC merger candidates is created as follows.

If neighboring block A1 is available and using IBC mode, insert the block vector of block A1 into the IBC merge candidate list of the current block. Otherwise (A1 is not available or A1 is not using IBC mode), do not insert block block vector A1 into the current block's IBC merge candidate list (first candidate, no reduction).

If neighboring block B1 is available and using IBC mode, reduce (or determine) whether block vector B1 is the same as A1. If block vector B1 is not the same as block vector A1, the block vector of block B1 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B1 is the same as block vector A1, or B1 is not available, or B0 does not use IBC mode), do not insert block vector A1 into the current block's IBC merge candidate list (abbreviation A1 and B1).

If neighboring block B0 is available and using IBC mode, reduce (or determine) whether block vector B0 is the same as B1. If block vector B0 is not the same as block vector B1, the block vector of block B0 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B0 is the same as block vector B1, or B0 is not available, or B0 does not use IBC mode), do not insert block vector B0 into the current block's IBC merge candidate list (abbreviation B0 and B1).

If neighboring block A0 is available and using IBC mode, reduce (or determine) whether block vector A0 is the same as A1. If block vector A0 is not the same as block vector A1, insert the block vector of block A0 into the IBC merge candidate list of the current block. Otherwise (block vector A0 is the same as block vector A1, or A0 is not available, or A0 does not use IBC mode), do not insert block vector A0 into the current block's IBC merge candidate list (abbreviation A0 and A1).

If adjacent block B2 is available and using IBC mode and the current IBC merge list size is less than 4, reduce (or determine) whether block vector B2 is the same as A1 and whether block vector B2 is the same as B1. If block vector B2 is not the same as block vector A1 and block vector B2 is not the same as B1, the block vector of block B2 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B2 is the same as block vector A1 or B1, or B0 is not available, or B0 does not use IBC mode), do not insert block vector B0 into the current block's IBC merge candidate list (reduce B2 and A1 and reduce B2 B1).

If block A1 is already on the IBC merge list, then

If the last HMVP candidate H _k is available and using the IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, and the block vector H _k is not the same as A1, insert the block vector H _k into the candidate list at merge the IBC of the current block. Otherwise ( _Hk is not available, or _Hk is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector _Hk is the same as block vector A1), do not insert block vector H _k in the current block's IBC merge candidate list (abbreviation H _k and A1).

If the penultimate HMVP candidate H _k-1 is available and using IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, and the block vector H _k-1 is not the same as A1, insert the block vector H _k-1 to the IBC merger candidate list of the current block. Otherwise (H _k-1 is not available or H _k-1 is not using IBC mode or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector H _k-1 is the same as the block vector vector A1), do not insert the block vector H _k-1 into the IBC merge candidate list of the current block (abbreviation H _k-1 and A1).

Otherwise, if A1 is not on the IBC merge list and B1 is already on the IBC merge list, then:

If the last HMVP candidate H _k is available and using the IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, and the block vector H _k is not the same as B1, insert the block vector H _k into list of IBC merger candidates of the current bloc. Otherwise ( _Hk is not available, or _Hk is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector _Hk is the same as block vector B1), do not insert block vector H _k in the current block's IBC merge candidate list (abbreviation H _k and B1).

If the penultimate HMVP candidate H _k-1 is available and using the IBC mode and the current size of the IBC merge candidate list is less than a value equal to the maximum number of IBC merge candidates minus one, and the block vector H _k-1 is not the same as A1, insert a block vector H _k-1 into the IBC merge candidate list of the current block. Otherwise (H _k-1 is not available or H _k-1 is not using IBC mode or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector H _k-1 is the same as the block vector vector B1), do not insert block vector H _k-1 into the current block merge candidate list IBC (abbreviation H _k-1 and B1).

Otherwise, if A1 and B1 are already on the IBC merge list, then:

If the last HMVP candidate H _k is available and using the IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, insert the block vector H _k into the current block's IBC merge candidate list. Otherwise ( _Hk is not available or _Hk is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one), do not insert the block vector _Hk into the current block's IBC merge candidate list (without abbreviations).

If a penultimate HMVP candidate H _k-1 is available and using the IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, insert the block vector H _k-1 into the current block's IBC merge candidate list. Otherwise (H _k-1 is not available, or H _k-1 is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one), do not insert the block vector H _k-1 into the IBC merge candidate list. merging the IBC of the current block (no reduction).

Insert the remaining HMVP candidates one by one if the candidate is available and using IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one (no reduction).

A paired candidate is inserted, the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates (no reduction).

Solution 8

According to solution 8, the reduction of the block vectors of the spatial neighboring block is kept the same. To reduce HMVP candidates:

the block vector of the last HMVP candidate H _k and the penultimate HMVP candidate H _k-1 are reduced by the block vector of the space neighbor block A1 when A1 is already in the IBC merge list and B1 is not in the IBC merge list;

the block vector of the last HMVP candidate H _k and the penultimate HMVP candidate H _k-1 are reduced by the block vector of the space neighbor block B1 when B1 is already in the IBC merge list and A1 is not in the IBC merge list;

the block vector of the last candidate H _k HMVP is reduced by the block vector of the spatial neighboring block B1 when both blocks A1 and B1 are already in the IBC merge list.

If neither of the blocks A1 or B1 is already present in the IBC merge list, then no HMVP candidate is reduced by the spatial candidate.

In this case, the maximum number of reductions to create the list of candidates for the IBC merger of the current block is reduced from 9 to 7.

The solution greatly reduces the complexity of creating an IBC merge list for both the encoder and decoder.

In this example, a specific list of IBC merger candidates is created as follows.

If neighboring block A1 is available and using IBC mode, insert the block vector of block A1 into the IBC merge candidate list of the current block. Otherwise (A1 is not available or A1 is not using IBC mode), do not insert block block vector A1 into the current block's IBC merge candidate list (first candidate, no reduction).

If neighboring block B1 is available and using IBC mode, reduce (or determine) whether block vector B1 is the same as A1. If block vector B1 is not the same as block vector A1, the block vector of block B1 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B1 is the same as block vector A1, or B1 is not available, or B0 does not use IBC mode), do not insert block vector A1 into the current block's IBC merge candidate list (abbreviation A1 and B1).

If neighboring block B0 is available and using IBC mode, reduce (or determine) whether block vector B0 is the same as B1. If block vector B0 is not the same as block vector B1, the block vector of block B0 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B0 is the same as block vector B1, or B0 is not available, or B0 does not use IBC mode), do not insert block vector B0 into the current block's IBC merge candidate list (abbreviation B0 and B1).

If neighboring block A0 is available and using IBC mode, reduce (or determine) whether block vector A0 is the same as A1. If block vector A0 is not the same as block vector A1, insert the block vector of block A0 into the IBC merge candidate list of the current block. Otherwise (block vector A0 is the same as block vector A1, or A0 is not available, or A0 does not use IBC mode), do not insert block vector A0 into the current block's IBC merge candidate list (abbreviation A0 and A1).

If adjacent block B2 is available and using IBC mode and the current IBC merge list size is less than 4, reduce (or determine) whether block vector B2 is the same as A1 and whether block vector B2 is the same as B1. If block vector B2 is not the same as block vector A1 and block vector B2 is not the same as B1, the block vector of block B2 is inserted into the IBC merge candidate list of the current block. Otherwise (block vector B2 is the same as block vector A1 or B1, or B0 is not available, or B0 does not use IBC mode), do not insert block vector B0 into the current block's IBC merge candidate list (reduce B2 and A1 and reduce B2 B1).

If A1 is already on the IBC merge list and B1 is not on the list, then:

if the last HMVP candidate H _k is available and using the IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, and the block vector H _k is not the same as A1, insert the block vector H _k into the candidate list at merge the IBC of the current block. Otherwise ( _Hk is not available, or _Hk is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector _Hk is the same as block vector A1), do not insert a block vector H _k in the IBC merge candidate list of the current block (abbreviation H _k and A1);

if the penultimate HMVP merge candidate H _k-1 is available and using IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, and the block vector H _k-1 is not the same as A1, insert the block vector H _k-1 to the IBC merger candidate list of the current block. Otherwise (H _k-1 is not available or H _k-1 is not using IBC mode or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector H _k-1 is the same as the block vector vector A1), do not insert the block vector H _k-1 into the IBC merge candidate list of the current block (abbreviation H _k-1 and A1).

Otherwise, if A1 is not on the IBC merge list and B1 is already on the IBC merge list, then:

if the last HMVP candidate H _k is available and using the IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, and the block vector H _k is not the same as B1, insert the block vector H _k into list of IBC merger candidates of the current bloc. Otherwise ( _Hk is not available, or _Hk is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector _Hk is the same as block vector B1), do not insert a block vector H _k in the IBC merge candidate list of the current block (abbreviation H _k and B1);

if the penultimate HMVP candidate H _k-1 is available and using IBC mode and the current size of the IBC merge candidate list is less than the value equal to the maximum number of IBC merge candidates minus one, and the block vector H _k-1 is not the same as A1, insert a block vector H _k-1 into the IBC merge candidate list of the current block. Otherwise (H _k-1 is not available or H _k-1 is not using IBC mode or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector H _k-1 is the same as the block vector vector B1), do not insert block vector H _k-1 into the current block merge candidate list IBC (abbreviation H _k-1 and B1).

Otherwise, if A1 and B1 are already on the IBC merge list, then:

if the last HMVP candidate H _k is available and using the IBC mode and the current size of the IBC merge candidate list is less than a value equal to the maximum number of IBC merge candidates minus one, and the block vector H _k is not the same as A1, and the block vector H _k is other than B1 insert the block vector H _k into the IBC merge candidate list of the current block. Otherwise (H _k is not available or H _k is not using the IBC mode or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector H _k is the same as the block vector A1, or the block vector H _k is the same as the block vector B1), do not insert the block vector H _k into the IBC merge candidate list of the current block (abbreviation H _k and A1 and B1).

Otherwise, if A1 and B1 are already on the IBC merge list, then

If the last HMVP candidate H _k is available and using the IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, insert the block vector H _k into the current block's IBC merge candidate list. Otherwise ( _Hk is not available or _Hk is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one), do not insert the block vector _Hk into the current block's IBC merge candidate list (without abbreviations).

If a penultimate HMVP candidate H _k-1 is available and using the IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, insert the block vector H _k-1 into the current block's IBC merge candidate list. Otherwise (H _k-1 is not available, or H _k-1 is not using IBC mode, or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one), do not insert the block vector H _k-1 into the IBC merge candidate list. merging the IBC of the current block (no reduction).

Insert the remaining HMVP candidates one by one if the candidate is available and using IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one (no reduction).

A paired candidate is inserted, the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates (no reduction).

Solution 9

In accordance with decision 9, over and above decision 1, the spatial neighboring block B0, based on the availability of B0 and B1; spatial neighbor block A0 based on the availability of A0 and A1; spatial neighboring block B2, based on the availability of B2, B1 and A1,

In this example,

If neighboring block B0 is available and uses IBC mode and B1 is not available, the block vector of block B0 is inserted into the current block's IBC merge candidate list. Otherwise (B0 is not available or B0 is not using IBC mode or B1 is available), the block vector of block B0 is not inserted into the current block's IBC merge candidate list (no reduction).

If neighboring block A0 is available and using IBC mode, and A1 is not available, the block vector of block A1 is inserted into the current block's IBC merge candidate list. Otherwise (A0 is not available or A0 is not using IBC mode or A1 is available), the block vector of block A0 is not inserted into the current block's IBC merge candidate list (no reduction).

If adjacent block B2 is available and using IBC mode, and A1 and B1 are not available, and the current IBC merge candidate list size is less than 4, block vector B2 is inserted into the current block's IBC merge candidate list. Otherwise (B2 is not available, or B2 is not using IBC mode, or the current IBC merge candidate list size is less than 4, or A1 is available and B1 is available), block vector B2 is not inserted into the current block's IBC merge candidate list (no reduction).

In this solution, no additional reduction is added, but compared to solution 1, a higher coding efficiency is achieved.

Solution 10

Consistent with Solution 10, any of Solutions 1-9 can be combined together to reduce the complexity of creating an IBC merge list.

In the example given for Solution 10, Solution 2 and Solution 3 are combined. To predict the current block using the IBC mode, both the left (A1) and top (B1) spatial neighbor blocks are important. In this example, _the reduction of spatial neighbor blocks between A1 and B is kept, the block vectors of the remaining spatial neighbor blocks are not inserted into the IBC merge list. . In an embodiment, the following reduction is possible during construction of the IBC merge list:

Reduction between last candidate H _k HMVP and A1

Reduction between last candidate H _k HMVP and B1

In this case, the maximum number of reductions to create the IBC merge candidate list of the current block is reduced from 9 to 3. The solution significantly reduces the complexity of creating the IBC merge list for both the encoder and decoder.

In this example, the last candidate from the IBC merge candidate list is resolved using the HMVP mode.

In this example, a specific list of IBC merger candidates is created as follows.

If neighboring block A1 is available and using IBC mode, insert the block vector of block A1 into the IBC merge candidate list of the current block. Otherwise (A1 is not available or A1 is not using IBC mode), do not insert block block vector A1 into the current block's IBC merge candidate list (first candidate, no reduction).

If neighboring block B1 is available and using IBC mode and the block vector of block A1 is inserted into the current block's IBC merge candidate list, reduce (or determine) whether block vector B1 is the same as A1. If block vector B1 is not the same as block vector A1, the block vector of block B1 is inserted into the IBC merge candidate list of the current block. Otherwise (B1's block vector is the same as A1's block vector, or B1 is not available, or B1 does not use IBC mode), do not insert B1's block vector into the current block's IBC merge candidate list (abbreviation A1 and B1).

If the last HMVP candidate H _k is available and using IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one, and the block vector H _k is not the same as A1 and B1, insert the block vector H _k to the list of IBC merger candidates of the current block. Otherwise (H _k is not available or H _k is not using the IBC mode or the current size of the IBC merge candidate list is not less than the maximum number of IBC merge candidates minus one, or the block vector H _k is the same as the block vector A1, or the block vector H _k is the same as block vector B1), do not insert block vector H _k into the IBC merge candidate list of the current block (abbreviation H _k and A1, H _k and B1).

The remaining HMVP candidates are inserted one by one if the candidate is available and using the IBC mode and the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates minus one (no reduction).

In some examples, a paired candidate is additionally inserted if the current size of the IBC merge candidate list is less than the maximum number of IBC merge candidates (no reduction).

This example shows the decoding process for coding blocks encoded in IBC prediction mode.

8.6.1 General description of the decoding process for coding blocks encoded in IBC prediction mode

The inputs to this process are:

luma coding block location (xCb, yCb ) indicating the top-left sample of the current coding block relative to the top-left luma sample of the current picture,

cbWidth variable indicating the width of the current coding block in luma samples,

cbHeight variable indicating the height of the current encoding block in luma samples,

a treeType variable indicating whether a single or double tree is used, and if a double tree is used, this indicates whether the current tree matches the luma or chrominance components.

The result of this process is a modified reconstructed image that precedes contour filtering.

The quantization parameter derivation process is initiated by using the luminance location (xCb, yCb), the width of the current coding block in luma samples cbWidth, the height of the current coding block in luminance samples cbHeight, and the variable treeType as input.

The IsGt4by4 variable is obtained as follows:

IsGt4by4 = (cbWidth * cbHeight) > 16 (1111)

The decoding process for coding blocks encoded in IBC prediction mode consists of the following steps, performed in order:

1. The block vector components of the current coding block are obtained as follows:

The block vector component derivation process is initiated with the luminance coding block location (xCb, yCb), the luminance coding block width cbWidth, the coding block height cbHeight, and the luminance block vector bvL as output.

When treeType is SINGLE_TREE, the chrominance block derivation process is initiated by the luma block vector bvL as input and the chroma block vector bvC as output.

2. Prediction samples of the current coding block are obtained as follows:

The decoding process for IBC blocks as specified in clause 8.6.3.1 is initiated with the location of the luma coding block (xCb, yCb), the width cbWidth of the luma coding block and the height cbHeight of the luma coding block, the luma block vector bvL, the variable cIdx set to 0 as input, and IBC prediction samples (predSamples), which is a predictive matrix of luminance samples predSamplesL (cbWidth)x(cbHeight) as output.

When treeType is SINGLE_TREE, prediction samples of the current coding block are obtained as follows:

- The IBC block decoding process as specified in clause 8.6.3.1 is initiated with the luminance coding block location (xCb, yCb), the luma coding block width cbWidth and the luma coding block height cbHeight, the chrominance block vector bvC and the variable cIdx set to 1 , as input, and IBC prediction samples (predSamples), which is the prediction matrix of chrominance samples predSamplesCb (cbWidth/SubWidthC)x(cbHeight/SubHeightC) for chrominance components Cb, as output.

- The IBC block decoding process is initiated according to clause 8.6.3.1 with the luminance coding block location (xCb, yCb), the luma coding block width cbWidth and the luma coding block height cbHeight, the chrominance block vector bvC and cIdx variable set to 2 as input data, and IBC prediction samples (predSamples), which is a matrix predSamplesCb (cbWidth/SubWidthC)x(cbHeight/SubHeightC) of chrominance sample prediction for chrominance components Cb as output.

3. Residual samples of the current coding block are obtained as follows:

- The process of decoding the residual signal of coding blocks encoded in the inter-prediction mode is initiated with the location (xTb0, yTb0) set equal to the location of the luminance coding block (xCb, yCb), the width nTbW set equal to the luminance coding block width cbWidth, the height nTbH, set equal to the height of the luminance coding block cbHeight, and the variable cIdx set equal to 0 as input, and the resSamplesL matrix as output.

- The process of decoding the residual signal of coding blocks encoded in the inter-prediction mode is initiated with the location (xTb0, yTb0) set equal to the location of the luminance coding block (xCb, yCb), the width nTbW set equal to the luminance coding block width cbWidth, the height nTbH, set equal to the height of the luminance coding block cbHeight, and the variable cIdx set equal to 1 as input, and the resSamplesL matrix as output.

- The process of decoding the residual signal of coding blocks encoded in the inter-prediction mode is initiated with the location (xTb0, yTb0) set equal to the location of the luminance coding block (xCb, yCb), the width nTbW set equal to the luminance coding block width cbWidth, the height nTbH, set equal to the height of the luminance coding block cbHeight, and the variable cIdx set equal to 2 as input, and the resSamplesL matrix as output.

The residual samples of the current coding block are obtained as follows:

- The image reconstruction process for the color component is initiated with the block location (xCurr, yCurr ) set equal to ( xCb, yCb ), the block width nCurrSw set equal to cbWidth, the block height nCurrSh set equal to cbHeight, the cIdx variable set equal to 0, the matrix predSamples (cbWidth)x(cbHeight) set to predSamplesL and resSamples (cbWidth)x(cbHeight) matrix set to resSamplesL as input, and the output is a modified reconstructed image prior to in-loop filtering.

- When treeType is equal to SINGLE_TREE, the color component image reconstruction process is initiated with a box location (xCurr, yCurr ) set to ( xCb/SubWidthC, yCb/SubHeightC ), a box width nCurrSw set to cbWidth/SubWidthC, a box height nCurrSh set to cbHeight/SubHeightC, cIdx variable set to 1, predSamples (cbWidth)x(cbHeight) matrix set to predSamplesL, and resSamples (cbWidth/SubWidthC)x(cbHeight/SubHeightC) matrix set to predSamplesCb, and resSamples (cbWidth/ SubWidthC)x(cbHeight/SubHeightC) set to resSamplesCb as input and the output is a modified reconstructed image prior to in-loop filtering.

- When treeType is SINGLE_TREE, the image reconstruction process for the color component is initiated with a box position (xCurr, yCurr ) set to ( xCb/SubWidthC, yCb/SubHeightC ), a box width nCurrSw set to cbWidth/SubWidthC, a box height nCurrSh set to cbHeight/SubHeightC, cIdx set to 2, predSamples (cbWidth/SubWidthC)x(cbHeight/SubHeightC) matrix set to predSamplesCr, and resSamples (cbWidth/SubWidthC)x(cbHeight/SubHeightC) matrix set to predSamplesCb, and matrix resSamples (cbWidth/SubWidthC)x(cbHeight/SubHeightC) set to resSamplesCr as input, and the output is a modified reconstructed image prior to in-loop filtering.

8.6.2 Process for deriving block vector components for IBC blocks

8.6.2.1 General

The inputs to this process are:

luminance coding block location (xCb, yCb) indicating the top-left sample of the current luminance coding block relative to the top-left sample of the luminance of the current image,

cbWidth variable indicating the width of the current coding block in luma samples,

cbHeight variable indicating the height of the current encoding block in luma samples.

The output of this process is:

luma block vector bvL with 1/16th fraction of the sample precision.

The block luma vector bvL is obtained as follows:

The derivation process for predicting the IBC luminance block vector is initiated according to clause 8.6.2.2 with the input luminance coding block location (xCb, yCb), the variables cbWidth and cbHeight, and the output is the luminance block vector bvL.

When general_merge_flag[ xCb ][ yCb ] is 0, the following applies:

The bvd variable is obtained as follows:

bvd[ 0 ] = MvdL0[ xCb ][ yCb ][ 0 ] (1112) bvd[ 1 ] = MvdL0[ xCb ][ yCb ][ 1 ] (1113)

2. The rounding process for motion vectors is initiated with mvX set to bvL, rightShift set to AmvrShift, and leftShift set to AmvrShift as input, and the output is obtained as rounded bvL.

3. The luminance block vector bvL is modified as follows:

u[ 0 ] = ( bvL[ 0 ] + bvd[ 0 ] + 2 ¹⁸ ) % 2 ¹⁸ (1114) bvL[ 0 ] = ( u[ 0 ] >= 2 ¹⁷ ) ? ( u[ 0 ] − 2 ¹⁸ ) : u[ 0 ] (1115) u[ 1 ] = ( bvL[ 1 ] + bvd[ 1 ] + 2 ¹⁸ ) % 2 ¹⁸ (1116) bvL[ 1 ] = ( u[ 1 ] >= 2 ¹⁷ ) ? ( u[ 1 ] − 2 ¹⁸ ) : u[ 1 ] (1117)

Note 1.— The resulting values bvL[ 0 ] and bvL[ 1 ] above will always be between −2 ¹⁷ and 2 ¹⁷ − 1, inclusive.

When IsGt4by4 is TRUE, the process of updating the list of block vector predictors based on archived data, as specified in clause 8.6.2.6, is initiated with the luma block vector bvL.

The bitstream consistency requirement is that the luma block vector bvL satisfy the following conditions:

CtbSizeY is greater than or equal to (( yCb + ( bvL[ 1 ] >> 4 ) ) & ( CtbSizeY − 1 ) ) + cbHeight.

IbcVirBuf[ 0 ][ ( x + (bvL[ 0 ] >> 4 ) ) & ( IbcBufWidthY − 1 ) ][ ( y + (bvL[1] >> 4 ) ) & ( CtbSizeY − 1) ] must not be equal to −1 for x = xCb..xCb + cbWidth − 1 and y = yCb..yCb + cbHeight − 1.

8.6.2.2 IBC luma block vector prediction process

The process is initiated only when CuPredMode[ 0 ][ xCb ][ yCb ] is equal to MODE_IBC, where ( xCb, yCb ) indicates the top-left sample of the current luminance coding block relative to the top-left sample of the luma of the current picture.

The inputs to this process are:

luminance coding block location (xCb, yCb) indicating the top left sample of the current luminance coding block relative to the top left luminance sample of the current picture,

cbWidth variable indicating the width of the current coding block in luma samples,

cbHeight variable indicating the height of the current encoding block in luma samples.

The output of this process is:

luma block vector bvL with 1/16th fraction of the sample precision.

The luma block vector bvL is obtained in the following order:

1. When IsGt4by4 is TRUE, the process of deriving spatial block vector candidates from neighboring coding blocks as described in clause 8.6.2.3 is initiated with the location (xCb, yCb) of the luminance coding block, the width of the luminance coding block cbWidth and the height cbHeight as inputs, and the outputs are availability flags availableFlagA1, availableFlagB1 and block vectors bvA1 and bvB1.

2. When IsGt4by4 is TRUE, the block vector candidate list, bvCandList, is constructed as follows:

i = 0
if (availableFlagA1)
bvCandList [ i++ ] = bvA1 i = 0
if (availableFlagB1)
bvCandList [ i++ ] = bvB1

3. The numCurrCand variable is obtained as follows:

When IsGt4by4 is TRUE, numCurrCand is set to the number of merge candidates in bvCandList.

Otherwise, (IsGt4by4 is FALSE), numCurrCand is set to 0.

4. When numCurrCand is less than MaxNumIbcMergeCand and NumHmvpIbcCand is greater than 0, the process of deriving archive data-based IBC block vector candidates according to clause 8.6.2.4 is initiated with bvCandList and numCurrCand as input, and modified bvCandList and numCurrCand are obtained in as the output.

5. When numCurrCand is less than MaxNumIbcMergeCand, the following applies until numCurrCand equals MaxNumIbcMergeCand:

bvCandList[ numCurrCand ][ 0 ] is set to 0.

bvCandList[ numCurrCand ][ 1 ] is set to 0.

numCurrCand increment by 1.

6. The bvIdx variable is obtained as follows:

bvIdx = general_merge_flag[ xCb ][ yCb ] ? merge_idx[ xCb ][ yCb ] : mvp_l0_flag[ xCb ][ yCb ] (1119)

7. The following appointments are made:

bvL[ 0 ] = bvCandList[ mvIdx ][ 0 ] (1120) bvL[ 1 ] = bvCandList[ mvIdx ][ 1 ] (1121)

8.6.2.3 Process for deriving IBC spatial block vector candidates

The inputs to this process are:

luminance coding block location (xCb, yCb) indicating the top left sample of the current luminance coding block relative to the top left luminance sample of the current picture,

cbWidth variable indicating the width of the current coding block in luma samples,

cbHeight variable indicating the height of the current encoding block in luma samples.

The output of this process is:

availability flags availableFlagA1 and availableFlagB1 of adjacent coding blocks,

block vectors bvA1 and bvB1 with an accuracy of 1/16 of the fraction of the sample of neighboring coding blocks.

To obtain availableFlagA1 and mvA1, use the following:

The location of the luminance coding block (xNbA1, yNbA1) within the adjacent luminance coding block is set to (xCb − 1, yCb + cbHeight − 1).

The neighbor block accessibility process is initiated with the current luminance coding block location ( xCurr, yCurr ) set to ( xCb, yCb ), the neighboring luminance coding block location (xNbA1, yNbA1), checkPredModeY set to TRUE, and cIdx, set to 0 as input, and the output is assigned to the block availability flag availableA1.

The availableFlagA1 and bvA1 variables are obtained as follows:

If availableA1 is FALSE, availableFlagA1 is set to 0 and both bvA1 components are set to 0.

Otherwise, availableFlagA1 is set to 1 and the following assignments are made:

bvA1 = MvL0[ xNbA1 ][ yNbA1 ] (1122)

To obtain availableFlagB1 and bvB1, use the following:

The location of the luminance coding block (xNbB1, yNbB1) within the adjacent luminance coding block is set to (xCb + cbWidth − 1, yCb − 1).

The neighbor block accessibility process is initiated with the current luminance coding block location ( xCurr, yCurr ) set to ( xCb, yCb ), the neighboring luma coding block location (xNbB1, yNbB1), checkPredModeY set to TRUE (true), and cIdx, set to 0 as input, and the output is assigned to the block availability flag availableB1.

The availableFlagB1 and bvB1 variables are obtained as follows:

If one or more of the following conditions are valid, availableB1 is set to 0 and both bvB1 components are set to 0:

availableB1 is FALSE.

availableA1 is TRUE and the luminance coding block locations (xNbA1, yNbA1) and (xNbB1, yNbB1) have the same block vectors.

Otherwise, availableFlagB1 is set to 1 and the following assignments are made:

bvB1 = MvL0[ xNbB1 ][ yNbB1 ] (1123).

8.6.2.4 Process for deriving IBC block vector candidates based on archived data

The inputs to this process are:

block vector candidate list bvCandList,

the number of available block vector candidates in the numCurrCand list.

The output of this process is:

modified block vector candidate list bvCandList,

the modified number of available block vector candidates in the numCurrCand list.

The variables isPrunedA1 and isPrunedB1 are both set to FALSE.

For each HmvpIbcCandList[ hMvpIdx ] candidate with index hMvpIdx = 1..NumHmvpIbcCand, the following successive steps are repeated until numCurrCand equals MaxNumIbcMergeCand:

1. The sameMotion variable is obtained as follows:

If all of the following conditions are valid for any block vector candidate N, where N is A1 or B1, sameMotion and isPrunedN are both set to TRUE:

IsGt4by4 is TRUE.

hMvpIdx is 1.

Candidate HmvpIbcCandList[NumHmvpIbcCand − hMvpIdx] is equal to block vector candidate N .

isPrunedN is FALSE.

Otherwise, sameMotion is set to FALSE.

2. When same Motion is FALSE, candidate HmvpIbcCandList [NumHmvpIbcCand − hMvpIdx] is added to the block vector candidate list as follows:

bvCandList[ numCurrCand++ ]= HmvpIbcCandList[ NumHmvpIbcCand − hMvpIdx ] (1124).

8.6.2.5 Chromaticity block vector derivation process

The inputs to this process are:

luma block vector bvL with 1/16th fraction of the sample precision.

The result of this process is a block chromaticity vector bvC with an accuracy of 1/32 of the sample fraction.

The chrominance block vector is derived from the corresponding luma block vector.

The block chroma vector bvC is obtained as follows:

bvC[ 0 ] = ( ( bvL[ 0 ] >> ( 3 + SubWidthC ) ) * 32 (1125) bvC[ 1 ] = ( ( bvL[ 1 ] >> ( 3 + SubHeightC ) ) * 32 (1126)

8.6.2.6 Process for updating a list of block vector predictor candidates based on archived data

The inputs to this process are:

luma block vector bvL with 1/16th fraction of the sample precision.

The HmvpIbcCandList candidate list is modified in the following steps, in order:

1. The variable identicalCandExist is set to FALSE (false) and the variable removeIdx is set to 0.

2. When NumHmvpIbcCand is greater than 0, for each hMvpIdx index with hMvpIdx = 0..NumHmvpIbcCand − 1, subsequent steps are applied until identicalCandExist is TRUE:

When bvL is equal to HmvpIbcCandList[ hMvpIdx ], identicalCandExist is set to TRUE (true) and removeIdx is set to hMvpIdx.

3. The HmvpIbcCandList candidate list is updated as follows:

If identicalCandExist is TRUE (true) or NumHmvpIbcCand is 5, the following apply:

For each index i with i = ( removeIdx + 1 )..( NumHmvpIbcCand − 1 ), HmvpIbcCandList[ i − 1] is set equal to HmvpIbcCandList [ i ].

HmvpIbcCandList[ NumHmvpIbcCand − 1 ] is set to bvL.

Otherwise (identicalCandExist is FALSE and NumHmvpIbcCand is less than 5), the following applies:

HmvpIbcCandList[ NumHmvpIbcCand ++ ] is set to bvL.

8.6.3 Decoding process for IBC blocks

8.6.3.1 General

This process is initiated when decoding coding blocks encoded in IBC prediction mode.

The inputs to this process are:

luma coding block location (xCb, yCb ) indicating the top-left sample of the current coding block relative to the top-left luma sample of the current picture,

cbWidth variable indicating the width of the current coding block in luma samples,

cbHeight variable indicating the height of the current encoding block in luma samples,

block Vector bv,

variable cIdx indicating the index of the chrominance components of the current block.

The output of this process is:

the predSamples matrix of prediction samples.

When cIdx is 0, for x = xCb..xCb + cbWidth − 1 and y = yCb..yCb + cbHeight − 1, the following apply:

xVb = ( x + ( bv[ 0 ] >> 4 ) ) & ( IbcBufWidthY − 1 ) (1127) yVb = ( y + ( bv[ 1 ] >> 4 ) ) & ( CtbSizeY − 1 ) (1128) predSamples[ x ][ y ] = ibcVirBuf[ 0 ][ xVb ][ yVb ] (1129)

When cIdx is not 0, for x = xCb/subWidthC..xCb/subWidthC + cbWidth/subWidthC − 1 and y = yCb/subHeightC..yCb/subHeightC + cbHeight/subHeightC − 1, the following apply:

xVb = ( x + ( bv[ 0 ] >> 5 ) ) & ( IbcBufWidthC − 1 ) (1130) yVb = ( y + ( bv[ 1 ] >> 5 ) ) & ( ( CtbSizeY/subHeightC ) − 1 ) (1131) predSamples[ x ][ y ] = ibcVirBuf[ cIdx ][ xVb ][ yVb ] (1132)

When cIdx is 0, the following assignments are made for x = 0..cbWidth − 1 and y = 0..cbHeight − 1:

MvL0[ xCb + x ][ yCb + y] = bv (1133) MvL1[ xCb + x ][ yCb + y] = 0 (1134) RefIdxL0[ xCb + x ][ yCb + y ] = −1 (1135) RefIdxL1[ xCb + x ][ yCb + y ] = −1 (1136) PredFlagL0[ xCb + x ][ yCb + y] = 0 (1137) PredFlagL1[ xCb + x ][ yCb + y] = 0 (1138) BcwIdx[ xCb + x ][ yCb + y] = 0 (1139).

The following is an explanation of the applications of the encoding method as well as the decoding method as described in the above embodiments, and the system using them.

In FIG. 8 is a block diagram of a content providing system 3100 for implementing a content distribution service. This content delivery system 3100 includes an image acquisition device 3102, a terminal device 3106, and optionally a display 3126.

The image acquisition device 3102 communicates with the terminal device 3106 via the communication line 3104.

The communication line may comprise the communication channel 13 described above. Link 3104 includes, but is not limited to, WI-FI, Ethernet, cable line, wireless link (3G/4G/5G), USB, or any combination thereof, and the like.

The image pickup device 3102 generates data and can encode the data in an encoding method as shown in the above embodiments. Alternatively, the image acquisition device 3102 may distribute the data to a streaming server (not shown in the drawings) and the server encodes the data and transmits the encoded data to the terminal device 3106.

The image acquisition device 3102 includes, but is not limited to, a camera, a smartphone or tablet, a computer or laptop, a video conferencing system, a PDA, a vehicle-mounted device, or a combination of any of these, and the like. For example, the image acquisition device 3102 may include a source device 12 as described above. When the data includes video data, the video encoder 20 included in the image pickup device 3102 can actually perform a video encoding process. When the data contains audio data (eg, voice), the audio encoder contained in the image pickup device 3102 can actually perform the audio encoding process. For some practical scenarios, the image acquisition device 3102 distributes encoded video data and audio data by multiplexing them together. For other practical scenarios, such as in a video conferencing system, encoded audio data and encoded video data are not multiplexed. The image pickup device 3102 separately distributes the encoded audio data and the encoded video data to the terminal device 3106.

In the content delivery system 3100, the terminal device 3106 receives and reproduces the encoded data. The terminal device 3106 may be a device with receive and retrieve capabilities such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR)/digital video recorder (DVR) 3112, a television receiver 3114, a set-top box ( set top box (STB) 3116, a videoconferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, a vehicle-mounted device 3124, or a combination of any of them, etc., capable of decoding the above encoded data. For example, terminal device 3106 may include a destination device 14 as described above. When the encoded data includes video data, the video encoder 30 included in the terminal has priority to perform video decoding. When the encoded data contains audio data, the audio decoder 30 contained in the terminal has priority to perform audio decoding.

For a terminal device that has a display, such as a 3108 smartphone or tablet, a 3110 computer or laptop, a 3112 Network Video Recorder (NVR)/DVR (Digital Video Recorder) 3112, a 3114 TV, a 3122 Personal Digital Assistant (PDA), or a 3124 mountable on the vehicle, the terminal may provide the decoded data to its display. For a terminal device that does not have a display, such as STB 3116, video conferencing system 3118 or video surveillance system 3120, an external display 3116 communicates with them and shows the decoded data.

When any of the devices in this system performs encoding or decoding, the image encoding device or the image decoding device shown in the above embodiments may be used.

In FIG. 9 is a diagram of the structure of an exemplary terminal device 3106.

After the terminal device 3106 receives the stream from the image pickup device 3102, the protocol processing unit 3202 parses the protocol for transmitting the stream. The protocol includes, but is not limited to, Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG -DASH, Real-time Transport protocol (RTP), Real Time Messaging Protocol (RTMP), or any combination thereof, etc.

After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexer 3204 . A demultiplexer 3204 may separate the multiplexed data into encoded audio data and encoded video data. As described above, for some practical scenarios, such as in a videoconferencing system, encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is transmitted to the video decoder 3206 and the audio decoder 3208 without passing through the demultiplexer 3204.

The demultiplexing process generates an elementary video stream (ES), audio ES, and optionally subtitles. The video decoder 3206, which includes the video decoder 30 as explained in the above embodiments, decodes the video ES by the decoding method shown in the above embodiments to form a video frame, and supplies this data to the synchronization block 3212.

An audio decoder 3208 decodes the audio ES to form an audio frame and provides this data to a sync block 3212.

Alternatively, the video frame may be stored in a buffer (not shown in FIG. 9) before being fed to the sync block 3212.

Likewise, the audio frame may be stored in a buffer (not shown in FIG. 9) before being fed to the sync block 3212.

Block 3212 synchronization synchronizes the video frame and audio frame and supplies video/audio data to the display 3214 video/audio.

For example, block 3212 synchronization synchronizes the presentation of video and audio information. The information may be encoded into a syntax using timestamps related to the presentation of the encoded audio and video data and timestamps related to the transmission of the data stream itself.

If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles and synchronizes them with a video frame and an audio frame and supplies video/audio/subtitle data to the video/audio/subtitle display 3216.

The present invention is not limited to the above system, and any of the devices such as an image encoding device or an image decoding device can be introduced into another system such as an automobile system.

Example 1. A method for constructing a list of merge candidates for an intra block copy (IBC) mode, said method comprising the steps of:

inserting the block vector of the current block's left neighbor block into the initial merge list of the current block (in this example, before this insertion step, the initial merge list is an empty list) when the left neighbor is available and the left neighbor is using IBC mode;

insert the block vector of the current block's top neighbor block into the initial merge list (in this example, before this step, the initial merge list is an empty list, or the initial merge list contains the block vector of the current block's left neighbor block), when the top neighbor block is available, the top neighbor the block uses IBC mode and the block vector of the top neighbor block is different from the block vector of the left neighbor block;

insert the block vector of the current block's top right neighbor block into the initial merge list (in this example, before this step, the initial merge list is an empty list, or the initial merge list contains the block vector of the current block's top neighbor block, or the initial merge list contains the block vector of the left block vector of the current block's top neighbor or the initial merge list contains the block vector of the top neighbor of the current block and the block vector of the left neighbor of the current block) when the right top neighbor is available and the right top neighbor is using IBC mode;

Example 2 The method of Example 1, wherein the method further comprises:

insert the block vector of the current block's left bottom neighbor block into the initial merge list (in this example, before this step, the initial merge list is an empty list, or the initial merge list contains the block vector of the current block's top neighbor block, or the initial merge list contains a block vector the left neighbor block of the current block, or the initial merge list contains the block vector of the top neighbor block of the current block and the block vector of the left neighbor block of the current block, or the initial merge list contains the block vector of the top neighbor block of the current block, the block vector of the left neighbor block of the current block, and top right neighbor block vector) when a bottom left neighbor is available and the bottom left neighbor is using IBC mode;

Example 3 A method according to example 1 or 2, wherein the method further comprises:

insert the block vector of the current block's top left neighbor block into the initial merge list (in this example, before this step, the initial merge list is an empty list, or the initial merge list contains the block vector of the current block's top neighbor block, or the initial merge list contains a block vector the left neighbor block of the current block, or the initial merge list contains the block vector of the top neighbor block of the current block and the block vector of the left neighbor block of the current block, or the initial merge list contains the block vector of the top neighbor block of the current block, the block vector of the left neighbor block of the current block, and the block vector of the top right neighbor block, or the initial merge list contains the block vector of the top neighbor block of the current block, the block vector of the left neighbor block of the current block, the block vector of the right top neighbor block, and the block vector of the left bottom neighbor block) when the left upper neighbor block is available , the top-left neighbor block uses IBC mode and the number of block vectors in the initial merge list is less than a threshold (for example, the threshold is 4).

Example 4 Encoder (20) containing a processing circuit for performing the method corresponding to any of Examples 1-3.

Example 5 A decoder (30) containing a processing circuit for performing a method corresponding to any of Examples 1-3.

Example 6 A computer program product containing a control program for executing a method according to any one of Examples 1-3.

Example 7: A decoder or encoder comprising:

one or more processors; And

a non-portable computer-readable storage medium associated with the processors and storing programming for execution by the processors, wherein the programming, when executed by the processors, configures a decoder to perform the method corresponding to any of Examples 1-3.

Mathematical operators

The mathematical operators used in this application are similar to those used in the "C" programming language. However, the results of integer division and arithmetic shifts are defined more precisely, and additional operations such as exponentiation and division of real values are defined. Numbering and counting rules usually start at 0, for example, "first" is equivalent to 0, "second" is equivalent to 1, and so on.

Arithmetic operators

The following arithmetic operators are defined as follows:

+ Addition

- Subtraction (as a two-argument operator) or negation (as a unary prefix operator)

* Multiplication, including matrix multiplication

x ^Y Exponentiation. Specifies x to the power of y. In other contexts, this notation is used to provide superscripts not intended to be interpreted as exponentiation.

/Integer division truncating the result towards zero. For example, 7/4 and -7/-4 are truncated to 1, and -7/4 and 7/-4 are truncated to -1.

÷ Used to denote division in mathematical equations in which no truncation or rounding is expected.

Используют для обозначения деления в математических уравнениях, в которых никакое усечение или округление не предполагается. $$\frac{X}{y}$$

Used to indicate division in mathematical equations in which no truncation or rounding is expected.

Суммирование f( i ), где i принимает любые целочисленные значения от x до и включая y. $${\displaystyle \sum _{i=x}^{y}f(i)}$$

The sum of f( i ), where i takes any integer value from x to and including y.

x%y Module. The remainder of x divided by y is only defined for integers x and y where x >= 0 and y > 0

Logical operators

The logical operators below are defined as follows:

x && y Boolean logical "AND" x and y

x | | y Boolean logical "OR" x and y

! Boolean logical "NOT"

x? y : z If x is TRUE (true) or not equal to 0, evaluate the value of y; otherwise, the z-value is evaluated.

Ratio Operators

The ratio operators below are defined as follows:

> More

>= Greater than or equal

< Less

<= Less than or equal

= = Equal

!= Not equal

When a ratio operator is applied to a syntax element or variable that has been assigned the value "na" (not applicable), the value "na" for the syntax element or variable is treated as a single value. The value "na" is treated as unequal to any other value.

Bitwise Operators

The following bitwise operators are defined as follows:

& Bitwise "AND". When used with integer arguments, acts as a complementary representation up to 2 for an integer value. When dealing with a binary argument containing fewer bits than another argument, the shorter argument is expanded to include more significant bits of 0.

| Bitwise "OR". When working with integer arguments, the complementary representation up to 2 for the integer value is in effect. When dealing with a binary argument containing fewer bits than another argument, the shorter argument is expanded to include more significant bits of 0.

^ Bitwise "exclusive OR" When working with integer arguments, the complementary representation up to 2 for the integer value is in effect. When dealing with a binary argument containing fewer bits than another argument, the shorter argument is expanded to include more significant bits of 0.

x >> y Arithmetic right shift of the complementary integer representation up to 2 for the binary digits x * y. This function is only defined for non-negative integer values of y. Bits shifted to the most significant bit (MSB) as a result of a right shift have a value equal to the MSB for x preceding the shift operation.

x << y Arithmetic left shift of complementary integer representation up to 2 for binary digits x * y. This function is only defined for non-negative integer values of y. Bits shifted to the least significant bit (LSB) as a result of a left shift have a value of 0.

Assignment operators

The following arithmetic operators are defined as follows:

= assignment operator

+ + Increment up, that is, x+ + is equivalent to x = x + 1; when used in a matrix index, evaluates the value of the variable before incrementing up.

− − Increment down, that is, x− − is equivalent to x = x - 1; when used in a matrix index, evaluates the value of the variable before incrementing down.

+= Increment up by the specified amount, that is, x += 3 is equivalent to x = x + 3, and x += (−3) is equivalent to x = x + (−3).

−= Increment down by the specified amount, that is, x −= 3 is equivalent to x = x − 3, and x −= (−3) is equivalent to x = x − (−3).

Range designation

The following notation is used to indicate a range of values.

x = y..z x x takes integer values from y to z, inclusive, where x, y, and z are integers and z is greater than y.

Math functions

The following mathematical functions are defined:

Asin( x ) is a trigonometric inverse sine function acting on an argument x that is in the range −1.0 to 1.0, inclusive, with an output value in the range −rp2 to rp2, inclusive, in units of radians.

Atan( x ) is a trigonometric inverse tangent function, acting on the argument x, with an output value in the range −rp2 to rp2, inclusive, in units of radians.

Ceil( x ) is the smallest integer greater than or equal to x.

b, d = Clip1((F + 0) >> 1)

Clip1 _C ( x ) = Clip3( 0, ( 1 << BitDepth _C ) − 1, x )

Cos( x ) is a trigonometric cosine function acting on the argument x in units of radians.

Floor( x ) is the largest integer less than or equal to x.

Ln( x ) is the natural logarithm of x (log base e, where e is the base constant of the natural logarithm 2.718 281 828...).

Log2( x ) base 2 logarithm of x.

Log10( x ) is the base 10 logarithm of x.

Round( x ) = Sign( x ) * Floor( Abs( x ) + 0.5 )

Sin( x ) is a trigonometric sine function acting on the argument x in units of radians.

Sqrt( x ) =

Swap( x, y ) = ( y, x )

Tan( x ) is a trigonometric tangent function acting on the argument x in units of radians.

Order of precedence of operations

When the order of operations in an expression is not explicitly specified using parentheses, the following rules apply.

– Operations with a higher priority are evaluated before any operation with a lower priority.

– Operations of equal precedence are evaluated sequentially from left to right.

The table below lists the priority of operations in order from highest to lowest; a higher position in the table indicates a higher priority.

For those operations that are also used in the C programming language, the order of precedence used in this specification is the same as that used in the C programming language.

Table: Order of execution of operations from highest priority (top of the table) to lowest priority (bottom of the table).

operations (with operands x, y and z) "x++", "x−−" "!x", "−x" (as unary prefix operator) x ^y

", "x % y""x * y", "x/y", "x h y", "

", "x % y" "x + y", "x − y" (as a two-argument operator), "

" "x << y", "x >> y" "x < y", "x <= y", "x > y", "x >= y" "x == y", "x != y" "x & y" "x|y" "x && y" "x||y" "x? y: z" "x..y" "x = y", "x += y", "x −= y"

Text description of logical operations

In the text, the operator of logical operations must be described mathematically in the following form:

if( condition 0)

statement 0

else if( condition 1 )

statement 1

...
else /* informative remark on remaining condition */

statement n

can be described as follows:

... as follows/... the following applies...:

– If condition 0, statement 0

– Otherwise, if condition 1, statement 1

- ...

– Otherwise (informative remark on remaining condition), statement n

Each "If ... Otherwise, if ... Otherwise, ..." statement in the text is entered with "... as follows" or "... the following applies", immediately followed by "If ... ". The last "If ... Otherwise, if ... Otherwise, ..." conditions are always "Otherwise, ..." statements. Interleaved "If ... Otherwise, if ... Otherwise, ...", which can be identified by matching "... as follows" or "... the following applies" ending in "Otherwise, ...".

In the text, the operator of logical operations must be described mathematically in the following form:

if( condition 0a && condition 0b )

statement 0

else if( condition 1a | | condition 1b )

statement 1

...
else
statement n

can be described in the following way:

... as follows/... the following applies:

– If all of the following conditions are true, statement 0:

– condition 0a

– condition 0b

– Otherwise, if one or more of the following conditions are true, statement 1:

– condition 1a

– condition 1b

- ...

– Otherwise, statement n

In the text, the operator of logical operations must be described mathematically in the following form:

if( condition 0 )

statement 0

if( condition 1 )

statement 1

can be described in the following way:

When condition 0, statement 0

When condition 1, statement 1.

Although embodiments of the invention have been described primarily based on video coding, it should be noted that embodiments of encoding system 10, encoder 20 and decoder 30 (and thus system 10) and other embodiments described herein may also be performed with the possibility of application in photo processing or in encoding, that is, in the processing or encoding of an individual image, regardless of any previous or subsequent image, as in video encoding. In general, only the inter prediction blocks 244 (encoder) and 344 (decoder) may be unavailable in the case where image processing coding is limited to a single image 17.

All other functionality (also referred to as tools or technologies) of video encoder 20 and video decoder 30 can equally be used for photo processing, e.g. ) transform 212/312, partitioning 262/362, intra prediction 254/354, and/or loop filtering 220, 320, entropy coding 270, and entropy decoding 304.

Embodiments of, for example, encoder 20 and decoder 30, and the functions described herein, for example, with reference to encoder 20 and decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on a computer-readable medium or transmitted via communications as one or more instructions or code and executed by a processor in hardware. The computer-readable medium may comprise a computer-readable storage medium corresponding to a physical medium, such as a storage medium for data storage, or a communication medium comprising any medium facilitating the transfer of a computer program from one place to another, such as according to a communication protocol. Thus, computer-readable media may generally correspond to (1) physical, computer-readable storage media that is non-portable, or (2) a communication medium such as a signal or carrier. A storage media can be any available media that can be accessed by one or more computers, one or more computers, or one or more processors to retrieve instructions, control program, and/or data structures for implementing the technologies described in this disclosure. . The computer program product may include a computer-readable medium.

By way of example, and not limitation, such computer-readable storage media may comprise random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM), or other storage device on optical disks, magnetic disk storage or other magnetic storage devices, flash memory or any other medium that can be used to store the desired control program in the form of instructions or data structures and can be accessed by a computer. In addition, any connection is appropriately referred to as a computer-readable medium. For example, if commands are transmitted from a website, server, or other remote source using coaxial cable, optical cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio wave, or microwave, then then coaxial cable, optical cable, twisted pair, DSL, or wireless technologies such as infrared, radio wave, or microwave are contained in the media definition. However, it should be understood that computer-readable storage media and storage media may not contain connections, carriers, signals, or other portable media, but be non-transferable physical storage media. The discs include compact discs (CD) black discs, optical discs, digital versatile discs (DVDs), floppy disks and Blu-ray discs, where some discs typically reproduce data magnetically while other discs reproduce data optically using lasers. Combinations of the above may also be included within the scope of computer-readable media.

Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application-specific integrated circuits (ASICs), field programmable gate arrays, FPGA) or other equivalent integrated or discrete logic circuits. Accordingly, the term "processor" as used herein may refer to any of the above structures, or to any other structure suitable for implementing the technologies described here. In addition, in some embodiments, the functionality presented herein may be provided within dedicated hardware and/or software modules capable of encoding and decoding, or may be contained in a combined codec. In addition, technologies may be fully implemented in one or more circuits or logic elements.

The technologies described in this disclosure may be implemented by a wide variety of devices, including a wireless mobile phone, an integrated circuit (IC), or an IC stack (eg, a stack of chips). Various components, modules, or blocks are described in this disclosure to emphasize the functional aspects of devices capable of implementing the disclosed technologies, but not necessarily requiring implementation with different hardware blocks. More specifically, as described above, the various blocks may be combined into an encoder/decoder hardware block, or may be provided by a set of interoperable hardware comprising one or more processors as described above, in combination with appropriate software and firmware.

Claims (42)

1. A method for constructing a merge candidate list for Intra block copy (IBC) mode, said method comprising the steps of: inserting a left neighbor block vector of the current block into the initial merge list of the current block when the left neighbor is available and the left neighbor is using IBC mode; inserting the block vector of the aforementioned neighbor block of the current block into the initial merge list, when the aforementioned neighbor block is available, the aforementioned neighbor block uses the IBC mode, and the block vector of the aforementioned neighbor block is not the same as the block vector of the left neighbor block; And inserting the block vector of the last candidate in the archive-based motion vector predictor (HMVP) into the initial merge list when the block vector of the aforementioned neighbor block is not the same as the block vector of the last HMVP candidate, and when the block vector of the left neighbor block is not the same as the block vector of the last HMVP candidate. 2. The method of claim 1, wherein the method further comprises: obtaining a block vector of the current block corresponding to the initial merge list after the above insertion processes, and a merge candidate index for the current block. 3. A method for constructing a merge candidate list for Intra block copy (IBC) mode, said method comprising the steps of: inserting a neighbor block vector of the current block into the initial merge list of the current block when the neighbor block is available and the neighbor block uses the IBC mode; inserting a block vector of the last candidate in the history-based motion vector predictor (HMVP) into the initial merge list when the block vector of the adjacent block is not the same as the block vector of the last HMVP candidate; And inserting the block vector of the other HMVP candidate into the initial merge list in which the abbreviation for the block vector of the other HMVP candidate is removed. 4. The method of claim 3, wherein the method further comprises: obtaining a block vector of the current block corresponding to the initial merge list after the above insertion processes, and a merge candidate index for the current block. 5. A method for constructing a merge candidate list for Intra block copy (IBC) mode, said method comprising the steps of: inserting a left neighbor block vector of the current block into the initial merge list of the current block when the left neighbor is available and the left neighbor is using IBC mode; inserting the block vector of the aforementioned neighbor block of the current block into the initial merge list, when the aforementioned neighbor block is available, the aforementioned neighbor block uses the IBC mode, and the block vector of the aforementioned neighbor block is not the same as the block vector of the left neighbor block; inserting the block vector of the last candidate in the archive-based motion vector predictor (HMVP) into the initial merge list when the block vector of the aforementioned neighbor block is not the same as the block vector of the last HMVP candidate, and when the block vector of the left neighbor block is not the same as the block vector of the latest HMVP candidate; And inserting the block vector of the other HMVP candidate into the initial merge list in which the abbreviation for the block vector of the other HMVP candidate is removed. 6. The method of claim 5, wherein the method further comprises: obtaining a block vector of the current block corresponding to the initial merge list after the above insertion processes, and a merge candidate index for the current block. 7. The method of claim 5 or 6, wherein the initial merge list before the first insertion process is an empty list. 8. The method according to any one of paragraphs. 5-7, in which the above-mentioned insertion processes are performed in order. 9. A method for constructing a merge candidate list for Intra block copy (IBC) mode, said method comprising the steps of: inserting a neighbor block vector of the current block into the initial merge list of the current block when the neighbor block is available and the neighbor block uses the IBC mode; And inserting a block vector of the last candidate in the history-based motion vector predictor (HMVP) into the initial merge list when the block vector of the adjacent block is not the same as the block vector of the last HMVP candidate; in which the last block vector in the initial merge list of the current block is the block vector of one HMVP candidate. 10. The method of claim 9, wherein the method further comprises: obtaining a block vector of the current block corresponding to the initial merge list after the above insertion processes, and a merge candidate index for the current block. 11. A method for constructing a merge candidate list for Intra block copy (IBC) mode, said method comprising the steps of: inserting a left neighbor block vector of the current block into the initial merge list of the current block when the left neighbor is available and the left neighbor is using IBC mode; inserting the block vector of the aforementioned neighbor block of the current block into the initial merge list, when the aforementioned neighbor block is available, the aforementioned neighbor block uses the IBC mode, and the block vector of the aforementioned neighbor block is not the same as the block vector of the left neighbor block; inserting the block vector of the last candidate in the archive-based motion vector predictor (HMVP) into the initial merge list when the block vector of the aforementioned neighbor block is not the same as the block vector of the last HMVP candidate, and when the block vector of the left neighbor block is not the same as the block vector of the latest HMVP candidate; And inserting the block vector of the other HMVP candidate into the initial merge list in which the abbreviation for the block vector of the other HMVP candidate is removed, and where the last block vector in the current block's initial merge list is a block vector of another HMVP candidate. 12. Encoder (20), containing the processing circuit for performing the method corresponding to any one of paragraphs. 1-11. 13. Decoder (30) containing the processing circuit for performing the method corresponding to any one of paragraphs. 1-11. 14. Computer-readable media containing program code for performing the method according to any one of paragraphs. 1-11. 15. Decoder, containing: one or more processors; And a non-portable, computer-readable storage medium associated with the processors and storing programming for execution by the processors, wherein the programming, when executed by the processors, configures a decoder to perform the method of any one of claims. 1-11. 16. A video data decoding device, comprising: a non-volatile storage device configured to store video data in the form of a bit stream; and a video decoder configured to perform the method according to any one of paragraphs. 1-11.

Images