RU2801112C1 - Coder, decoder and corresponding methods related to intra-frame prediction mode - Google Patents

Abstract

FIELD: image processing.

SUBSTANCE: invention relates to performing an intraframe chrominance prediction mode derivation using the intra-frame prediction mode from a corresponding luminance component. The technical result is to increase the compression ratio with minimal or no effect on image quality. The result is achieved in that the encoding implemented by the decoding device or the encoding device comprises obtaining indication information for the position (cbWidth/2, cbHeight/2) of the brightness of the current encoding block relative to the top-left position (xCb, yCb) of the brightness sample of the current encoding block, in wherein cbWidth represents the width of the current coding block in luma samples, cbHeight represents the height of the current coding block in luma samples; setting the value of the luminance intraframe prediction mode associated with the current coding block as the first default value when the indication information indicates that the matrix-based Intra Prediction (MIP) mode is applied to the luminance samples at the position (cbWidth/2, cbHeight/2) brightness relative to the upper left position (xCb, yCb) of the brightness sample of the current coding block; and deriving an intra chroma prediction mode value based on the intra chroma prediction mode value of the current coding block.

EFFECT: increased compression ratio with minimal or no effect on image quality.

30 cl, 13 dwg, 4 tbl

Description

Cross-reference to related applications

This application claims priority over PCT Application No. PCT/EP 2019/072611 filed on August 23, 2019, which claims priority over PCT Application No. PCT/EP 2019/069944 filed on July 24, 2019. Both applications are hereby incorporated by reference .

The technical field to which the invention belongs

The embodiments presented in the present application (disclosure) generally relate to the field of image processing, and more specifically to performing intra-frame chrominance prediction mode derivation using intra-frame prediction mode from a corresponding luminance component.

State of the art

Video coding (video coding and video decoding) is used in a wide range of digital video applications, such as digital television broadcast, video transmission over the Internet and mobile networks, real-time conversational applications such as video chat, video conferencing, DVD and Blu-ray discs, content acquisition and editing systems; and security application cameras.

The amount of video data required to display even a relatively short video can be significant, which can result in difficulties when the data must be streamed or otherwise over a communication network with limited bandwidth. Thus, video data is usually compressed before being transmitted over modern communication networks. Video size can also be an issue when the video is stored on a storage device because memory resources can be limited. Video compression devices often use software and/or hardware at the signal source to encode video data prior to transmission or storage, thus reducing the amount of data required to represent digital video images. The compressed data is then received at the destination by a video decompressor which decodes the video data. With limited network resources and ever-increasing demand for higher quality video, improved compression and decompression technologies are desirable that increase compression rates with little or no impact on image quality.

Disclosure of the essence of the invention

Embodiments of the present application provide encoding and decoding devices and methods corresponding to the independent claims.

The above and other problems are solved by the subject of independent claims. Additional forms of implementation are obvious from the dependent claims, description and drawings.

The first approach of the invention relates to an encoding method implemented by a decoding device or an encoding device. The method comprises the steps of: obtaining indication information for the position (cbWidth/2, cbHeight/2) of the brightness of the current coding block relative to the top-left position (xCb, yCb) of the brightness sample of the current coding block, where cbWidth represents the width of the current coding block in the component brightness, cbHeight represents the height of the current encoding block in the brightness component. Accordingly, cbWidth/2 represents half the width of the current coding block in the luminance component, and cbHeight/2 represents half the height of the current coding block in the luma component. The absolute position (cbWidth/2, cbHeight/2) of the luminance is (xCb+cbWidth/2, yCb+cbHeight/2), that is, the "middle" of the corresponding luminance prediction block.

The method further comprises: setting the value of the intra-frame luminance prediction mode associated with the current coding block as a default value when the indication information indicates that Matrix-based Intra Prediction (MIP) is applied to the component luminance at the luminance position (cbWidth/2, cbHeight/2) relative to the top-left position (xCb, yCb) of the luminance sample of the current coding block; and obtaining an intra chroma prediction mode value based on the intra luma prediction mode value of the current coding block.

Obtaining prediction mode information from a fixed position (cbWidth/2, cbHeight/2) in the corresponding luminance component ensures that the MIP mode position and the intra frame prediction mode position match when, for a given block size, the luminance component separation is different from the chrominance component separation (e.g., when the binary tree coding method is enabled), where the position of the MIP mode replaces the position at which the MIP mode is obtained, and the position of the intra luminance prediction mode replaces the position at which the intra luminance prediction mode is obtained.

The second approach of the invention relates to an encoding method implemented by a decoding device or an encoding device. The method comprises the steps of: obtaining indication information for the position (cbWidth/2, cbHeight/2) of the brightness of the current coding block relative to the top-left position (xCb, yCb) of the brightness sample of the current coding block, where cbWidth represents the width of the current coding block in the component brightness, cbHeight represents the height of the current encoding block in the brightness component. Accordingly, cbWidth/2 represents half the width of the current coding block in the luminance component, and cbHeight/2 represents half the height of the current coding block in the luma component. The absolute position of the luminance (cbWidth/2, cbHeight/2) is (xCb+cbWidth/2, yCb+cbHeight/2), that is, the "middle" of the corresponding luminance prediction block.

The method comprises the steps of: setting the value of the intra block luminance prediction mode associated with the current coding block as the first default value when the indication information indicates that the intra block copy (Intra Block Copy, IBC) mode or palette mode is applied to the luminance component at the position (cbWidth/2, cbHeight/2) of the brightness relative to the top-left position (xCb, yCb) of the brightness sample of the current coding block; and obtaining an intra chroma prediction mode value based on the intra luma prediction mode value of the current coding block.

Obtaining prediction mode information from a fixed position (cbWidth/2, cbHeight/2) in the corresponding luminance component ensures that the IBC mode position and the intra frame prediction mode position match when, for a given block size, the luma component separation is different from the chrominance component separation (e.g., when the binary tree coding method is enabled), where the position of the IBC mode replaces the position at which the IBC mode is obtained, and the position of the intra luma prediction mode replaces the position at which the intra luminance prediction mode is obtained.

Alternatively, obtaining prediction mode information from a fixed position (cbWidth/2, cbHeight/2) in the corresponding luminance component ensures that the mode palette position and the intra-frame luminance prediction mode position match when, for a given block size, the luminance component separation is different from the chrominance component separation (for example, when the binary tree coding method is enabled), where the mode palette position replaces the position at which the palette mode is obtained, and the position of the intra-luma prediction mode replaces the position at which the intra-luma prediction mode is obtained.

Coincidence of obtaining information about the mode of obtaining from position (cbWidth/2, cbHeight/2) is necessary when, for a given block size, the separation of the luminance component differs from the separation of the chrominance component. Otherwise, it may cause undefined behavior, as shown in FIG. 7.

The method according to the first approach of the invention may be carried out by an apparatus according to the third approach of the invention. Additional features and embodiments of the device according to the third approach correspond to the features and embodiments of the method according to the first approach of the invention.

The method according to the second approach of the invention can be performed by a device corresponding to the fourth approach of the invention. Additional features and embodiments of the device according to the fourth approach correspond to the features and embodiments of the method according to the second approach of the invention.

According to the fifth approach, the embodiment is associated with a video stream decoding or encoding device, comprising a processor and a memory. The memory stores instructions that cause the processor to execute the method in accordance with the first approach.

According to the sixth approach, an embodiment of the invention is associated with a device for decoding or encoding a video stream, comprising a processor and a memory. The memory stores instructions that cause the processor to execute the method in accordance with the second approach.

According to a seventh approach, a computer-readable storage medium is provided on which instructions are stored that, when executed, cause one or more configured processors to encode video data. The instructions cause one or more processors to execute the method corresponding to the first or second approach, or any possible embodiment of the first or second approach.

According to an eighth approach, an embodiment of the invention is associated with a computer program containing a control program for executing the method corresponding to the first or second approach, or any possible embodiment of the first or second approach, when executed on the computer.

According to the ninth approach, an embodiment of the invention is associated with an apparatus for obtaining an intra-frame chrominance prediction mode, comprising: one or more processors; and a computer-readable persistent storage medium associated with the processors and storing programming for execution by the processors, where the programming, when executed by the processors, configures the decoder to: obtain first indication information for the luminance position (cbWidth/2, cbHeight/2) of the current coding block relative to the top-left position (xCb, yCb) of the luminance sample of the current coding block, where cbWidth represents the width of the current coding block in the luminance component, cbHeight represents the height of the current coding block in the luminance component. Accordingly, cbWidth/2 represents half the width of the current coding block in the luminance component, and cbHeight/2 represents half the height of the current coding block in the luma component. The absolute position of the luminance (cbWidth/2, cbHeight/2) is (xCb+cbWidth/2, yCb+cbHeight/2), that is, the "middle" of the corresponding luminance prediction block.

The one or more processors are further configured to set the value of the intra-frame luminance prediction mode associated with the current coding block as the first default value when the first indication information indicates that Matrix-based Intra Prediction (MIP) , applied to the luminance component at the position (cbWidth/2, cbHeight/2) of the luminance relative to the top-left position (xCb, yCb) of the luminance sample of the current coding block; or obtaining the second indication information for the luminance position (cbWidth/2, cbHeight/2) of the current coding block, when the first indication information indicates that MIP is not applied to the luminance component at the luminance sample position (cbWidth/2, cbHeight/2) relative to the top- left position (xCb, yCb) of the brightness sample of the current block.

The one or more processors are further configured to set the value of the intra-frame luminance prediction mode associated with the current coding block as a second default value when the second indication information indicates that an intra block copy (IBC) mode or a palette mode is applied. for the luminance component at the luminance position (cbWidth/2, cbHeight/2) relative to the top-left position (xCb, yCb) of the luminance sample of the current coding block; and obtaining an intra chroma prediction mode value based on the intra luma prediction mode value of the current coding block.

Obtaining the first indication information from a fixed position (cbWidth/2, cbHeight/2) in the corresponding luminance component ensures that the position of the MIP mode and the position of the intra-frame luminance prediction mode match when, for a given block size, the separation of the luma component differs from the separation of the chrominance component (for example, when the binary tree encoding method is allowed). When the first indication information does not indicate that MIP is applied to the luminance component at the position (cbWidth/2, cbHeight/2) of the luminance, obtaining the second indication information from the fixed position (cbWidth/2, cbHeight/2) at the corresponding luminance component ensures that the position The mode IBC and the position of the intra-frame luminance prediction mode are the same when, for a given block size, the luma component separation is different from the chrominance component separation (eg, when the binary tree coding method is enabled). Alternatively, obtaining the second indication information from a fixed position (cbWidth/2, cbHeight/2) in the corresponding luminance component ensures that the mode palette position and the intra-frame luminance prediction mode position match when, for a given block size, the luminance component separation is different from the chrominance component separation ( for example, when a binary tree encoding method is allowed).

Details of one or more embodiments are set forth in the accompanying drawings and description below. Other features, objectives and advantages will become apparent from the description, drawings and claims.

Brief description of the drawings

In the following, embodiments of the invention are described in more detail with reference to the attached drawings, in which:

fig. 1A is a block diagram of an example of a video coding system capable of implementing embodiments of the invention;

fig. 1B is a block diagram of another example of a video coding system capable of implementing embodiments of the invention;

fig. 2 is a block diagram of an example video encoder capable of implementing embodiments of the invention;

fig. 3 is a block diagram of an exemplary structure of a video decoder capable of implementing embodiments of the invention;

fig. 4 is a block diagram of an example of an encoder or decoder;

fig. 5 is a block diagram of another example of an encoder or decoder;

fig. 6 is an example of intra-frame coding modes;

fig. 7 is a block diagram showing a potential problem of undefined behavior when obtaining an intra-frame chrominance prediction mode using the current specification;

fig. 8 is an example of a luminance location value of a coding block;

fig. 9 shows embodiments of a method 900 according to the present invention;

fig. 10 shows embodiments of an apparatus 1000 for use in accordance with the present invention;

fig. 11 is a block diagram of an exemplary structure of a content providing system 3100 implementing a content delivery service;

fig. 12 is a block diagram of the structure of an exemplary terminal.

In the following, identical reference numerals refer to identical or at least functionally equivalent features, unless expressly defined otherwise.

Implementation of the invention

In the following description, reference is made to the accompanying drawings, which form part of the disclosure and which, by way of illustration, show specific approaches of embodiments of the invention or specific approaches in which embodiments of the present invention may be used. It should be understood that embodiments of the invention may be used in other ways and contain structural or logical changes not shown in the drawings. The following detailed description is therefore not to be construed in a limiting sense and the protection scope of the present invention is defined by the appended claims.

For example, it should be understood that the disclosure in connection with the described method may also remain valid for the corresponding device or system configured to carry out the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may comprise one or more blocks, such as functional blocks, for performing the one or more method steps described (for example, one block performs one or more steps or multiple blocks, each of which performs one or more of a plurality of steps), even if such one or more blocks are not explicitly described or illustrated in the drawings. On the other hand, for example, if a particular device is described in terms of one or more blocks, such as functional blocks, the corresponding method may include one step to perform the functionality of one or more blocks (for example, one step performs the functionality of one or more blocks, or each from a plurality of steps performs the functionality of one or more of the plurality of blocks), even if such one or more steps are not explicitly described or shown in the drawings. It is further contemplated that the features of the various exemplary embodiments and/or approaches described herein may be combined with one another unless specifically noted otherwise.

Video coding generally refers to the processing of a sequence of images that form a video or video sequence. Instead of the term "image", as synonyms in the field of video coding, the term "frame" or "picture" can be used. Video coding (or, more generally, coding) contains two parts: video coding and video decoding. Video coding is performed at the source side, typically comprising processing (eg, by compressing) the original video images to reduce the amount of data required to represent the video images (for more efficient storage and/or transmission). Video decoding is performed on the destination side and usually contains inverse processing compared to the encoder for reconstructing video images. Embodiments relating to “coding” of video images (or, in general, frames) should be understood as referring to “coding” or “decoding” of video images or corresponding video sequences. The combination of the coding part and the decoding part is also referred to as CODEC (Coding and Decoding, encoding and decoding, codec).

In the case of lossless video coding, the original video images can be reconstructed, i.e. the reconstructed video images have the same quality as the original video images (assuming no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, additional compression is performed, for example by quantization, to reduce the amount of data representing video images that cannot be fully reconstructed in the decoder, i.e. the quality of the reconstructed video images is lower or worse than the quality of the original video images.

Several video coding standards belong to the group of "hybrid lossy video codecs" (ie, combine spatial and temporal sample-domain prediction and 2D transform coding to apply transform-domain quantization). Each picture of a video sequence is typically divided into a number of non-overlapping blocks, and encoding is usually done at the block level. In other words, in the encoder, the video is usually processed, i.e. encoded at the block (video block) level, for example, using spatial (intra-frame) prediction and/or temporal (inter-frame) prediction to form a prediction block, subtracting the prediction block from the current block (the block currently being processed/to be processed) to obtain a residual block by transforming the residual block and quantizing the residual block in the transform domain to reduce the amount of data that must be transmitted (compression), while in the decoder, compared to the encoder, inverse processing is applied to the encoded or compressed block to reconstruct the current block for presentation. Additionally, the encoder copies the decoder processing cycle such that they together form identical predictions (eg, intra and inter predictions) and/or reconstructions for processing, i. coding of subsequent blocks.

In the following embodiments of video coding system 10, video encoder 20 and video decoder 30 are described based on FIG. 1-3.

In FIG. 1A is a block diagram of an exemplary coding system 10, such as a video coding system 10 (or coding system 10 for short), that may use the technology of the present application. Video encoder 20 (or encoder 20 for short) and video decoder 30 (or decoder 30 for short) of video coding system 10 represent examples of devices that can be configured to implement the technologies in accordance with the various examples described herein.

As shown in FIG. 1A, the encoding system 10 includes a source device 12 configured to provide the encoded image data 21 to, for example, the destination device 14 to decode the encoded image data 13.

The source device 12 includes an encoder 20 and may additionally, i. e. optionally comprise an image source 16, a preprocessor (or preprocessor) 18, such as an image preprocessor 18, and a communication interface or communication unit 22.

The image source 16 may comprise or be any kind of imaging device, such as a camera for capturing a real world image and/or some sort of imaging device, such as a computer graphics processor for generating a computer image with animation, or any kind of other device for capturing and/ or providing an image of the real world, a computer-generated image (e.g., screen content, virtual reality (VR) image), and/or any combination thereof (e.g., augmented reality (AR) image). The image source may be any kind of memory or storage device storing any of the above images.

Unlike the pre-processor 18 and the processing performed by the pre-processor 18, the image or image data 17 may also be referred to as raw image or raw image data 17.

The pre-processor 18 is configured to receive (raw) image data 17 and perform pre-processing of the image data 17 to obtain a pre-processed image 19 or pre-processed image data 19. The preprocessing performed by the preprocessor 18 may include, for example, cropping, color format conversion (eg from RGB to YCbCr), color correction, or noise reduction. It should be understood that the pre-processing unit 18 may be an optional component.

Video encoder 20 is configured to receive pre-processed image data 19 and provide encoded image data 21 (more details will be described below, for example, based on Fig. 2).

The communication interface 22 of the source device 12 may be configured to receive the encoded image data 21 and transmit the encoded image data 21 (or any further processed version thereof) over the transmission channel 13 to another device, such as the destination device 14 or any other device for storage or direct reconstruction.

Destination device 14 includes a decoder 30 (eg, video decoder 30) and may additionally, i. e. optionally, include a communication interface or communication unit 28, a post-processor 32 (or post-processing unit 32) and a display 34.

The communication interface 28 of the destination device 14 is configured to receive the encoded image data 21 (or any other further processed version thereof), for example, directly from the source device 12 or from any other source, for example, from a storage device, for example, from an encoded data storage device. image, and providing the encoded image data 21 to the decoder 30.

Communication interface 22 and communication interface 28 may be configured to transmit or receive encoded image data 21 or encoded data 13 via a direct link between source device 12 and destination device 14, such as via a direct wired or wireless connection, or via any kind of network. , for example, through a wired or wireless network, or any combination thereof, or through any kind of private network and public network, or any kind of combination thereof.

The communication interface 22 may be configured, for example, to package the encoded image data 21 into an appropriate format, such as packets, and/or process the encoded image data with any kind of transmission encoding or processing for transmission over a communication channel or over a communication network.

The communication interface 28 representing a copy of the communication interface 22 may be configured to, for example, receive the transmission data and process the transmission data with any kind of appropriate transmission decoding or processing and/or decompression to obtain the encoded image data 21.

Both interfaces, communication interface 22 and communication interface 28, can be implemented as unidirectional communication interfaces, as indicated by the arrow for communication channel 13 in FIG. 1A indicating the direction from the source device 12 to the destination device 14, or as bidirectional communication interfaces, and can be configured to, for example, send and receive messages, for example, to establish a connection, acknowledge reception, and exchange any other information associated with the channel communication and/or data transmission, for example, transmission of encoded image data.

The decoder 30 is configured to receive encoded image data 21 and provide decoded image data 31 or decoded image 31 (more details will be described below, for example, based on Fig. 3 or Fig. 5).

The post-processor 32 of destination device 14 is configured to post-process decoded image data 31 (also referred to as reconstructed image data), such as decoded image 31, to obtain post-processed image data 33, such as post-processed image 33. Post-processing performed by the block 32 post-processing may include, for example, color format conversion (for example, from YCbCr to RGB), color correction, cropping or resampling, or any other processing, for example, to prepare the decoded image data 31 for display, for example, on the display 34.

The display 34 of the destination device 14 is configured to receive post-processed image data 33 for displaying the image to, for example, a user or viewer. The display 34 may be or include any kind of display for presenting the rendered image, such as an integrated or external display or monitor. The displays may, for example, be liquid crystal displays (LCD), organic light emitting diode (OLED) displays, plasma displays, projectors, micro-LED displays, liquid crystal silicon (LCoS) liquid crystal displays, digital light processor (digital light processor, DLP) or any other type of display.

Although in FIG. 1A, source device 12 and destination device 14 are shown as separate devices, device embodiments may also comprise both devices or both capabilities, source device 12 or its corresponding functionality, and destination device 14 or its corresponding functionality. In such embodiments, source device 12 or its respective functionality and destination device 14 or its corresponding functionality may be implemented using the same hardware and/or software, or separate hardware and/or software, or any combination thereof. .

As will be apparent to those skilled in the art, based on the description, the existence and (precise) separation of functionality of various blocks or functionality within source device 12 and/or destination device 14 as shown in FIG. 1A may vary depending on the actual device and application.

Encoder 20 (eg, video encoder 20) or decoder 30 (eg, video decoder 30), or encoder 20 and decoder 30 together may be implemented by a processing circuit as shown in FIG. 1B, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, custom video coding, or any combination thereof. Encoder 20 may be implemented by processing circuitry 46 to implement various modules as discussed with respect to encoder 20 shown in FIG. 2 and/or any other encoder system or subsystem described herein. Decoder 30 may be implemented by processing circuitry 46 to implement various modules as discussed with respect to decoder 30 shown in FIG. 3 and/or any other decoder system or subsystem described herein. The processing circuit may be configured to perform various operations, as discussed below. As shown in FIG. 5, if the technologies are implemented in part in software, the device may store the software instructions on a suitable machine-readable non-volatile storage medium and may execute the instructions in hardware using one or more processors to execute the technologies of the present disclosure. Video encoder 20 or video decoder 30 may be integrated as part of a combined encoder/decoder (CODEC) in a single device, for example, as shown in FIG. 1b.

Source device 12 and destination device 14 may comprise any of a wide range of devices, including any kind of portable or stationary devices such as laptops or laptops, mobile phones, smartphones, tablets or tablet computers, cameras, desktop computers, indoor installations, televisions, displays, digital media players, video game consoles, video streaming devices (such as content service servers or content delivery servers), home radio, broadcast transmitter, and the like. and may or may not use any kind of operating system. In some cases, source device 12 and destination device 14 may be equipped for wireless communication. Thus, source device 12 and destination device 14 may be wireless communication devices.

In some cases, the video coding system 10 shown in FIG. 1A is merely an example, and the techniques presented herein may apply to video encoding settings (eg, video encoding or video decoding) that do not necessarily involve data communication between encoding and decoding devices. In other examples, data is retrieved from local memory, streamed over a network, and the like. The video encoder may encode and store data in memory and/or the video decoder may retrieve and decode data from memory. In some examples, encoding and decoding are performed by devices that are not connected to each other, but simply encode data into memory and/or retrieve and decode data from memory.

For convenience of description, embodiments of the disclosure are described herein, for example, with reference to High-Efficiency Video Coding (HEVC) or with reference to Versatile Video coding (VVC) software, the next generation video coding standard developed by the Joint Collaboration Group. ITU-T Video Coding Experts Group (VCEG) and ISO/IEC Motion Picture Experts Group (Joint Collaboration Team on Video Coding, JCT-VC) MPEG). Any person skilled in the art should understand that embodiments of the invention are not limited to HEVC or VVC.

Encoder and encoding method

In FIG. 2 shows a block diagram of an example video encoder 20 capable of implementing the technologies of the present application. In the example shown in FIG. 2, the video encoder 20 includes an input 201 (or input interface 201), a residual calculation block 204, a transform block 206, a quantizer 208, an inverse quantizer 210 and an inverse transform block 212, a reconstruction block 214, a loop filter block 220, a decoded image buffer 230 (decoded picture buffer, DPB) 230, mode selection block 260, entropy encoding block 270, and output 272 (or output interface 272).

Block 260 mode selection may contain block 244 inter-prediction, block 254 intra-prediction and block 262 division.

The inter prediction block 244 may comprise a motion estimator and a motion compensation block (not shown). Video encoder 20, as shown in FIG. 2 may also be referred to as a hybrid video encoder or a video encoder corresponding to a hybrid video codec.

Residual calculator 204, transform 206, quantizer 208, mode selector 260 may be referred to as forming the forward signal path of encoder 20, while quantizer 210, inverse transform 212, reconstructor 214, buffer 216, loop filter 220, decoded picture buffer (DPB) 230, inter prediction block 244, and intra prediction block 254 may be referred to as forming the reverse signal path of video encoder 20, where the reverse signal path of video encoder 20 corresponds to the signal path of the decoder (see video decoder 30 in FIG. 3). An inverse quantizer 210, an inverse transform 212, a reconstructor 214, a loop filter 220, a decoded image buffer (DPB) 230, an inter-predictor 244, and an intra-predictor 254 are also related to the formation of the "built-in decoder" of the video encoder 20.

Images and image division (images and blocks)

The encoder 20 may be configured to receive, for example, via input 201, an image 17 (or image data 17), such as an image of a sequence of images forming a video or video sequence. The received image or image data may also be a pre-processed image 19 (or pre-processed image data 19). For simplicity, the following description refers to picture 17. Picture 17 may also be referred to as the current picture or the picture to be encoded (in particular video coding to distinguish the current picture from other pictures, e.g., previously encoded and/or decoded pictures from the same video sequence , i.e. a video sequence that also contains the current image).

A (digital) image is, or can be viewed as, a two-dimensional array or matrix of samples with intensity values. A sample in an array can also be referred to as a pixel (short form of picture element) or picture element. The number of samples in the horizontal and vertical direction (or axis) of the array or image determines the size and/or resolution of the image. Three color components are commonly used to represent a color, i.e. the image can be represented or contain three arrays of samples. In RBG format or color space, an image contains a corresponding red, green, and blue array of samples. However, in video coding, each pixel is typically represented in a luma and chrominance format or color space, such as YCbCr, which contains a luma component labeled Y (sometimes L is also used instead) and two chrominance components labeled Cb and Cr. The Y component of luminance (luminance or luma for short) represents the luminance level or gray level intensity (e.g. in a grayscale image), while the two chrominance components (chrominance or chroma for short) Cb and Cr represent color information components or chromaticity. Accordingly, an image in YCbCr format contains an array of luma samples (Y) and two arrays of chroma samples of chrominance values (Cb and Cr). RGB images can be converted or transformed to YCbCr format and vice versa, this process is also known as color conversion or transformation. If the image is monochrome, the image can only contain an array of brightness samples. Accordingly, the image may be, for example, an array of luminance samples in monochrome format, or an array of luminance samples and two corresponding arrays of chrominance samples in 4:2:0, 4:2:2 and 4:4:4 color format.

Embodiments of video encoder 20 may comprise an image splitter (not shown in FIG. 2) configured to split image 17 into multiple (typically non-overlapping) image blocks 203.

These blocks may also be referred to as root blocks, macroblocks (H.264/AVC), or coding tree blocks (CTBs) or coding tree units (CTUs) (H.265/ HEVC and VVC). The image dividing block may be configured to use the same block size for all images of the video sequence and the corresponding grid defining the block size, or to change the block size between images or subsets or groups of images and divide each image into appropriate blocks.

In additional embodiments, the video encoder may be configured to receive directly the image block 203 17, such as one, several or all of the blocks forming the image 17. The image block 203 may also be referred to as the current image block or the image block to be encoded.

Like image 17, image block 203 is or can be thought of as a two-dimensional array or matrix of samples with intensity values (sample values), albeit smaller than image 17. In other words, block 203 may contain, for example, one array of samples (for example, luminance array luma in the case of a monochrome image 17 or chroma array in the case of a color image) or three arrays of samples (for example, a luma array and two chrominance arrays in the case of a color image 17) or any other number and / or types of arrays depending on the applied color format. The number of samples in the horizontal and vertical direction (or axis) of block 203 determines the size of block 203. Accordingly, the block may be an array of samples, such as MxN (M columns and N rows) or an MxN array of transform coefficients.

Embodiments of video encoder 20, as shown in FIG. 2 may be configured to encode the image 17 block by block, for example, encoding and prediction are performed for each block 203.

Embodiments of video encoder 20 shown in FIG. 2 may further be configured to split and/or encode the image using slices (also referred to as video slices), where the picture may be sliced or encoded using one or more slices (usually non-overlapping) and each slice may contain one or more blocks (for example, CTU).

Embodiments of video encoder 20 shown in FIG. 2 may further be configured to split and/or encode the image using tile groups (also referred to as video tile groups) and/or tiles (also referred to as video tiles) in which the image may be split or encoded using one or more (typically non-overlapping) tile groups, and each tile group may contain one or more blocks (eg, CTUs) or one or more tiles, where each tile may be, for example, rectangular in shape and may contain one or more blocks (eg, CTUs), for example, full or fractional blocks.

Remainder Calculation

Residual block 204 can be configured to calculate residual block 205 (also referred to as residual 205) based on image block 203 and prediction block 265 (more on predictor 265 is provided below), for example, by subtracting the sample values of predictor 265 from the sample values of the image block 203 sample-by-pixel (pixel-by-pixel) to obtain a residual block 205 in the sample area.

transformation

The transform block 206 can be configured to apply a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), on the sample values of the residual block 205 to obtain transform coefficients 207 in the transform domain. The transform coefficients 207 may also be referred to as residual transform coefficients and represent the residual block 205 in the transform domain.

Transformer 206 may be configured to apply DCT/DST integer approximations such as those specified for H.265/HEVC. Compared to the orthogonal DCT transform, such integer approximations are usually scaled by a certain factor. To preserve the residual block norm that is handled by the forward and inverse transforms, additional scale factors are applied as part of the transform process. The scaling factors are usually chosen based on some constraints such as the scaling factors being a power of two for shift operations, the bit depth of the transform coefficients, the trade-off between accuracy and implementation cost, and so on. Specific scaling factors are determined, for example, for inverse transform, for example, by inverse transform block 212 (and the corresponding inverse transform, for example, by inverse transform block 312 in video decoder 30), and corresponding scaling factors for direct transformation, for example, by transform block 206 in encoder 20 , can be defined accordingly.

Embodiments of video encoder 20 (respectively, transform block 206) may be configured to output transform parameters, such as the type of transform or transforms, for example, directly or encoded or compressed via entropy encoding block 270, so that, for example, video decoder 30 can receive and use transform options for decoding.

Quantization

Quantizer 208 may be configured to quantize transform coefficients 207 to obtain quantized coefficients 209, for example by applying scalar quantization or vector quantization. The quantized coefficients 209 may also be referred to as quantized transform coefficients 209 or quantized residual coefficients 209.

The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207 . For example, an n-bit transform factor during quantization may be rounded down to an m-bit transform factor, where n is greater than m. The degree of quantization can be modified by adjusting the quantization parameter (QP). For example, for scalar quantization, different scaling may be applied to achieve finer or coarser quantization. Smaller quantization step sizes correspond to finer quantization, while larger quantization step sizes correspond to coarser quantization. The applied quantization step size may be indicated by a quantization parameter (QP). The quantization parameter may be, for example, an index of a given set of applied quantization step sizes. For example, small quantization parameters may correspond to fine quantization (small quantization step sizes), and large quantization parameters may correspond to coarse quantization (large quantization step sizes), or vice versa. The quantization may comprise dividing by the quantization step size and corresponding and/or inverse dequantization, for example by inverse quantizer 210, may comprise multiplying by the quantization step size. Embodiments following some standards, such as HEVC, may be configured to use a quantization parameter to determine the quantization step size. Typically, the quantization step size can be calculated based on the quantization parameter using a fixed-point approximation of an equation involving division. Additional scaling factors may be introduced for quantization and dequantization to recover the residual block rate, which may be modified by the scaling used in the fixed-point approximation of the equation for quantization step size and quantization parameter. In one exemplary implementation, inverse transform scaling and dequantization may be combined. Alternatively, specialized quantization tables may be used and communicated from the encoder to the decoder, eg in a bitstream. Quantization is a lossy operation where the loss increases as the quantization step size increases.

Embodiments of video encoder 20 (respectively, quantizer 208) may be configured to output quantization parameters (QPs), for example, directly or encoded via entropy coding unit 270, so that, for example, video decoder 30 can receive and apply quantization parameters for decoding .

Inverse quantization

Inverse quantizer 210 is configured to apply inverse quantization of quantizer 208 to quantization coefficients to obtain dequantized coefficients 211, for example, by inverting the quantization scheme applied by quantizer 208 based on or using the same quantization step size as quantizer 208 . The dequantization coefficients 211 may also be referred to as the dequantized residual coefficients 211 and correspond - although they are not usually identical to the transform coefficients due to quantization loss - to the transform coefficients 207.

Reverse transformation

The inverse transform block 212 is configured to apply an inverse transform to the transform applied by the transform block 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST), or another inverse transform for obtaining a reconstructed residual block 213 (or corresponding dequantized coefficients 213) in the sample area. The reconstructed residual block 213 may also be referred to as the transformation block 213 .

Reconstruction

Reconstructor 214 (e.g., adder or adder 214) is configured to add transform block 213 (i.e., reconstructed residual block 213) to prediction block 265 to obtain reconstructed block 215 in the sample domain, for example, by adding - sample with sample - sample values of the reconstructed residual block 213 with the sample values of the block 265 prediction.

Filtration

The loop filter block 220 (or "loop filter" 220 for short) is configured to filter the reconstructed block 215 to obtain a filtered block 221, or, in general, filter the reconstructed samples to obtain filtered samples. The loop filter block, for example, is configured to smooth out pixel transitions or otherwise improve video quality. The loop filter block 220 may comprise one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF). ), a sharpening filter, an anti-aliasing filter, or collaborative filters, or any combination thereof. Although the loop filter block 220 in FIG. 2 is shown as being a loop filter, in other configurations the loop filter unit 220 may be implemented as a post-processing loop filter. Filtered block 221 may also be referred to as filtered reconstructed block 221.

Embodiments of video encoder 20 (respectively, loop filter block 220) may be configured to output loop filter parameters (such as adaptive sample offset information), for example, directly or encoded by entropy encoding block 270, so that, for example, decoder 30 can receive and apply the same loop filter parameters or corresponding loop filters for decoding.

Decoded image buffer

Decoded picture buffer (DPB) 230 230 may be a memory that stores reference pictures, or reference picture data in general, for encoding video data by video encoder 20. DPB 230 may be formed by any of a variety of memories, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetically resistant RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. Picture decoding buffer (DPB) 230 230 may be configured to store one or more filtered blocks 221. Decoded picture buffer 230 may be further configured to store other previously filtered blocks, such as previously reconstructed and filtered blocks 221 of the same current images or different images, such as previously rendered images, and may provide fully previously reconstructed, ie. decoded pictures (and corresponding reference blocks and samples) and/or a partially reconstructed current picture (and corresponding reference blocks and samples), for example, for inter-picture prediction. The buffer 230 decoded image (decoded picture buffer, DPB) can also be configured to store one or more unfiltered reconstructed blocks 215 or, in general, unfiltered reconstructed samples, for example, if the reconstructed block 215 is not filtered by the loop filter block 220 or there is no other an additionally processed version of the reconstructed blocks or samples.

Mode selection (division and prediction)

The mode selector 260 comprises a separating block 262, an inter-prediction block 244, and an intra-prediction block 254, and is configured to receive or obtain the original image data, for example, using the original block 203 (the current block 203 of the current image 17), and the reconstructed image data, e.g. filtered and/or unfiltered reconstructed samples or blocks of the same (current) picture and/or one or more previously decoded pictures, e.g. obtained from decoded picture buffer 230 or other buffers (e.g. not shown). The reconstructed image data is used as reference image data for prediction, such as inter-picture prediction or intra-picture prediction, to obtain prediction block 265 or predictor 265.

The mode selector 260 may be configured to determine or select a split for the current block prediction mode (including no split) and the prediction mode (e.g., intra frame or inter frame prediction mode) and generate a corresponding prediction block 265 that is used to calculate the residual block 205 and for the reconstruction of the reconstructed block 215.

Embodiments of mode selector 260 may be configured to select a splitting and prediction mode (e.g., from those supported by or available to mode selector 260) that provides the best match, or in other words, minimum residual (lowest residual means best compression for transmission or storage) or minimum signaling overhead (lowest signaling overhead means better compression for transmission or storage), or which consider or balance both. The mode selector 260 may be configured to determine the partitioning and prediction mode based on rate distortion optimization (RDO), i. selecting a prediction mode that provides the minimum distortion-rate ratio. The terms “best”, “minimum”, “optimal”, etc. in this context does not necessarily refer to the general understanding of “best”, “minimum”, “optimal”, etc., but may also refer to the fulfillment of a completion or selection criterion, such as exceeding a threshold or falling below a threshold, or other constraints , potentially leading to a “suboptimal choice”, but reducing complexity and processing time.

In other words, partitioner 262 may be configured to partition block 203 into smaller blocks or sub-blocks (which again form blocks), for example, by repeatedly using quad-tree-partitioning (QT), binary tree partitioning (binary-tree partitioning, BT) or ternary tree division (triple-tree-partitioning, TT), or any combination thereof, and performing prediction for each of the block parts or for each sub-block, where the mode selection contains the selection of the tree structure of the divided block 203 and the prediction modes are applied to each part of the block or to a subblock.

In the following, the separation process (for example, by the splitter 260) and prediction (by the inter predictor 244 244 and the intra predictor 254) performed by the video encoder 20 as an example will be explained in more detail.

Partitioning

Divider 262 may divide or split the current block 203 into smaller pieces, such as smaller square or rectangular blocks. These smaller blocks (which may also be referred to as sub-blocks) may further be subdivided into even smaller pieces. This is also referred to as tree division or hierarchical tree division, where a root block, e.g. at level 0 of the tree root (hierarchy-level 0, depth 0) can be recursively split, e.g. divided into two or more blocks of the next lower level of the tree, e.g., nodes at level 1 of the tree (hierarchy-level 1, depth 1), where these blocks can be split again into two or more blocks of the next lower level, e.g., level 2 of the tree (hierarchy-level 2, depth 2), and so on, until the division ends, eg because a termination criterion is met, eg the maximum tree depth or minimum block size is reached. Blocks that are not further divided are also referred to as leaf blocks or tree nodes. Division using a tree into two parts is referred to as a binary tree (binary-tree, BT), division using a tree into three parts is referred to as a ternary tree (ternary-tree, TT) and division using a tree into four parts is referred to as a quad tree (quad). -tree, QT).

As mentioned above, the term "block", as used here, may be a part, in particular, a square or rectangular part of the image. With reference to, for example, HEVC and VVC, a block can either be a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), or a transform unit. TU) and/or corresponding blocks, such as a coding tree block (CTB), a coding block (CB), a transform block (TB), or a prediction block (PB).

For example, a coding tree unit (CTU) may be or contain a luminance sample CTB, two corresponding image chrominance sample CTBs that have three sample arrays, or a monochrome image sample CTB or an image that is encoded with three separate color planes and syntax structures used to sample coding. Accordingly, a coding tree block (CTB) may be an NxN block of samples for some value of N, such that splitting a component into CTBs is a division. A coding unit (CU) can be either a luminance sample coding unit, two corresponding chrominance sample coding units of an image that has three sample arrays, or a monochrome image sample coding unit or an image that is encoded with three separate color planes and structures. syntax used to encode selections. Accordingly, a coding block (CB) may be an MxN block of samples for some values of M and N, so dividing the CTB into coding blocks is a division.

In HEVC-compliant embodiments, for example, a coding tree unit (CTU) may be partitioned into CUs using a quadtree structure referred to as a coding tree. The decision whether to encode a region of an image using inter-frame (temporal) or intra-frame (spatial) prediction is made at the CU level. Each CU may be further partitioned into one, two, or four PUs, according to the partition type of the PU. Within one PU, the same prediction process is applied and the corresponding information is transmitted to the decoder based on the PU. After obtaining the residual block, using a prediction process based on the partition type of the PU, the CU can be divided into transform units (TUs) according to another quadtree structure similar to the coding tree for the CU.

In embodiments, for example, in accordance with the latest video coding standard currently in development, referred to as Versatile Video Coding (VVC), combined quad-tree and binary- tree, QTBT) is used, for example, to separate an encoding block. In the QTBT block structure, the CU can be square or rectangular. For example, a coding tree unit (CTU) is first divided by a quadratic tree structure. The nodes of a quadratic tree are further divided by a binary tree structure or a ternary tree structure. The nodes of the tree when divided are called coding units (CU) and such segmentation is used for the prediction and transformation process without any additional division. This means that in the QTBT coding block structure, CUs, PUs and TUs have the same block size. In parallel, multiple partitioning, such as ternary tree partitioning, can be used in conjunction with the QTBT box structure.

In one example, mode selector 260 of video encoder 20 may be configured to implement any combination of the separation technologies described herein.

As described above, video encoder 20 is configured to determine or select the best or optimal prediction mode from a set (eg, predetermined) of prediction modes. The set of prediction modes may comprise, for example, intra-prediction modes and/or inter-prediction modes.

Intra-frame prediction

The set of intra-prediction modes may contain 35 different intra-prediction modes, for example, non-directional modes such as DC (or average) mode and planar mode, or directional modes, for example, as defined in HEVC, or may contain 67 different intra-prediction modes, for example , non-directional modes such as DC (or medium) mode and planar mode, or directional modes such as those defined for VVC.

The intra prediction block 254 is configured to use reconstructed samples of neighboring blocks of the same current image to generate an intra prediction block 265 corresponding to an intra prediction mode from a set of intra prediction modes.

The intra prediction block 254 (or mode selector 260 generally) is further configured to output intra prediction parameters (or information indicative of the selected intra prediction mode for the block in general) to the entropy encoding block 270 in the form of syntax elements 266 for included in the encoded image data 21 so that, for example, video decoder 30 can receive and use the prediction parameters for decoding.

Interframe prediction

The set of inter prediction modes (or possible modes) depends on the available reference pictures (i.e., previous at least partially decoded pictures, such as those stored in DPB 230) and other inter prediction parameters, such as whether the reference picture is fully used or only partially, such as the area of the search window around the area of the current reference picture block to search for the best matching reference block, and/or, for example, whether pixel interpolation is applied, such as half/half-element, quarter-element, and/or 1/16-element interpolation.

In addition to the prediction modes mentioned above, skip mode and/or direct mode may be applied.

The inter prediction block 244 may comprise a motion estimation (ME) block and a motion compensation (MC) block (both not shown in FIG. 2). The motion estimator may be configured to receive or obtain an image block 203 (the current image block 203 of the current image 17) and a decoded image 231 or at least one or a plurality of previously reconstructed blocks, e.g., reconstructed blocks from one or a plurality of other/different previously decoded images 231, for motion compensation. For example, the video sequence may comprise the current picture and previously decoded pictures 231 or, in other words, the current picture and previously decoded pictures 231 may be part of or form a sequence of pictures that make up the video sequence.

Encoder 20 may, for example, be configured to select a reference block from a plurality of reference blocks of the same or different pictures from a plurality of other pictures and provide the reference picture (or reference picture index) motion estimate and/or offset (spatial offset) between the position (with x, y coordinates) of the reference block and the position of the current block as inter-prediction parameters. This displacement is also called the motion vector (MV).

The motion compensation block is configured to obtain, for example, receive, an inter prediction parameter, and perform inter prediction based on or using the inter prediction parameter to obtain the inter prediction block 265 .

The motion compensation performed by the motion compensator may comprise sampling or generating a prediction block based on a motion/block vector determined by motion estimation, possibly performing interpolations to sub-pixel precision. Interpolation filtering can generate additional pixel samples from known pixel samples, thus potentially increasing the number of possible prediction blocks that can be used to encode an image block. After receiving the motion vector for the PU of the current picture, the motion compensator may locate the prediction block to which the motion vector is directed in one of the reference picture lists.

The motion compensation block may also generate syntax elements associated with blocks and video sections for use by video decoder 30 when decoding image blocks of a video section. In addition, or as an alternative to sections and corresponding syntax elements, groups of tiles and/or tiles and corresponding syntax elements can be formed or used.

Entropy coding

Entropy encoding unit 270 is configured to apply, for example, an entropy coding algorithm or scheme (for example, variable length coding (VLC) coding scheme, context adaptive VLC (CAVLC) context adaptive scheme), arithmetic coding scheme, binarization , context adaptive binary arithmetic coding (CABAC), based on the syntax of context-adaptive binary arithmetic coding (syntax-based context-adaptive binary arithmetic coding, SBAC), entropy coding with probability interval partitioning entropy , PIPE) or other entropy coding methodology or technology) or traversal (no compression) on quantized coefficients 209, inter-prediction parameters, intra-prediction parameters, loop filter parameters, and/or other syntax elements to obtain encoded image data 21 that can be output via output 272, for example in the form of an encoded bitstream 21, so that, for example, video decoder 30 can receive and use the parameters for decoding. The encoded bitstream 21 may be transmitted to video decoder 30 or stored in memory for later transmission or retrieval by video decoder 30.

Other structural changes to video encoder 20 may also be used to encode the video stream. For example, a no-transform encoder 20 may quantize the residual signal directly without a transform unit 206 for certain blocks or frames. In another implementation, encoder 20 may have quantizer 208 and inverse quantizer 210 combined into a single block.

Decoder and decoding method

In FIG. 3 shows an example of a video decoder 30 capable of implementing the technologies of the present application. Video decoder 30 is configured to receive encoded image data 21 (e.g., encoded bitstream 21), such as data encoded by encoder 20 to obtain a decoded image 331.

The encoded image data or bitstream contains information for decoding encoded image data, for example, data that represents image blocks of an encoded video section (and/or groups of tiles or tiles) and associated syntax elements.

In the example in FIG. 3, decoder 30 includes an entropy decoder 304, an inverse quantizer 310, an inverse transform 312, a reconstructor 314 (e.g., adder 314), a loop filter 320, a decoded picture buffer (DPB) 330, a mode application 360, block 344 inter-prediction and block 354 intra-prediction. The inter prediction block 344 may be or comprise a motion compensation block.

Video decoder 30, in some examples, may perform a decoding loop, typically the reverse of the encoding loop described with reference to video encoder 100 shown in FIG. 2.

As has been explained with reference to encoder 20, inverse quantizer 210, inverse transform 212, reconstructor 214, loop filter 220, decoded picture buffer (DPB) 230, inter prediction 344, and intra prediction 354 are also referred to as generating “inline decoder" of the video encoder 20. Accordingly, inverse quantizer 310 may be functionally identical to inverse quantizer 110, inverse transform block 312 may be functionally identical to inverse transform block 212, reconstructor 314 may be functionally identical to reconstructor 214, loop filter 320 may be functionally identical to is identical to the loop filter 220, and the decoded image buffer 330 may be functionally identical to the decoded image buffer 230. Therefore, the explanations provided for the respective blocks and functions of video encoder 20 apply properly to the corresponding blocks and functions of video decoder 30.

Entropy decoding

The entropy decoding unit 304 is configured to parse the bitstream 21 (or, generally speaking, the encoded image data 21) and perform, for example, entropy decoding of the encoded image data 21 to obtain, for example, quantized coefficients 309 and/or decoded coding parameters (not shown in Figure 3), such as any or all of the inter-prediction parameters (eg, reference picture index and motion vector), intra-prediction parameter (eg, intra-prediction mode or index), transform parameters, quantization parameters, loop filter parameters, and/ or other syntax elements. Entropy decoding unit 304 may be configured to apply decoding algorithms or schemes corresponding to the coding schemes described with reference to entropy encoding unit 270 of encoder 20. Entropy decoding unit 304 may be further configured to present to application unit 360 of the inter-frame prediction parameter mode, parameters intra-frame prediction and/or other syntax elements, as well as other parameters to other blocks of decoder 30. Video decoder 30 may receive syntax elements at the video section level and/or at the video block level. In addition, or as an alternative to sections and corresponding syntax elements, groups of tiles and/or tiles and corresponding syntax elements may be accepted and/or used.

Inverse quantization

The inverse quantizer 310 may be configured to receive quantization parameters (QP) (or, in general, information related to inverse quantization) and quantized coefficients from the encoded image data 21 (for example, by parsing and/or decoding, for example, entropy decoding block 304) and applying, based on the quantization parameters, inverse quantization on the decoded quantized coefficients 309 to obtain dequantized coefficients 311, which may also be referred to as transform coefficients 311. The inverse quantization process may comprise using a quantization parameter determined by video encoder 20 for each video block in a video slice (or tile or group of tiles) to determine the amount of quantization and, likewise, the amount of inverse quantization to be applied.

Reverse transformation

Inverse transform block 312 may be configured to receive the dequantized coefficients 311, also referred to as transform coefficients 311, and apply a transform to the dequantized coefficients 311 to obtain reconstructed residual blocks 213 in the sample area. The reconstructed residual blocks 213 may also be referred to as transform blocks 313 . The transformation may be an inverse transformation, eg, inverse DCT, inverse DST, inverse integer transformation, or a conceptually similar inverse transformation process. The inverse transform block 312 may be further configured to receive transform parameters or corresponding information from the encoded image data 21 (eg, by parsing and/or decoding, eg by entropy decoding block 304) to determine the transform to be applied to the dequantized coefficients 311 .

Reconstruction

Reconstructor 314 (e.g., adder or adder 314) may be configured to add reconstructed residual block 313 to predictor 365 to obtain reconstructed block 315 in the sample region, for example, by adding the sample values of the reconstructed residual block 313 and the sample values of block 365 predictions.

Filtration

The loop filter block 320 (in the encoding cycle or after the encoding cycle) is configured to filter the reconstructed block 315 to obtain a filtered block 321, for example, to smooth pixel transitions or otherwise improve video quality. The loop filter block 320 may comprise one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a bilateral filter, an adaptive loop filter (ALF). ), a sharpening filter, an anti-aliasing filter, or collaborative filters, or any combination thereof. Although the loop filter block 320 in FIG. 3 is shown as a loop filter, in other configurations the loop filter unit 320 may be implemented as a post-processing loop filter.

Decoded image buffer

The decoded video blocks 321 of the picture are then stored in the decoded picture buffer 330, which stores the decoded pictures 331 as reference pictures for subsequent motion compensation in other pictures and/or for display, respectively.

The decoder 30 is configured to output the decoded image 311, for example, through the output 312, for presentation or viewing by the user.

Prediction

The inter prediction block 344 may be identical to the inter prediction block 244 (in particular, the motion compensation block), and the intra prediction block 354 may be functionally identical to the intra prediction block 254, and makes splitting or splitting and prediction decisions based on splitting parameters and/or prediction or on the corresponding information obtained from the encoded image data 21 (for example, by parsing and/or decoding, for example, block 304 entropy decoding). Mode apply block 360 may be configured to perform prediction (intraframe or interframe) for each block based on reconstructed images, blocks, or corresponding samples (filtered or unfiltered) to obtain prediction block 365 .

When a video section is encoded as an intra-frame (I)-coded section, intra-prediction block 354 from mode apply block 360 is configured to generate prediction block 365 for a picture block of the current video section based on the reported intra-frame prediction mode and data from previously decoded blocks of the current picture. When a video image is encoded as an inter-frame coded (i.e., B or P) section, the inter-prediction block 344 (e.g., motion compensation block) of the mode apply block 360 is configured to create prediction blocks 365 for the video block of the current video section based on the motion vectors and other elements. syntax received from block 304 entropy decoding. For inter-picture prediction, prediction blocks may be generated from one of the reference pictures within one of the reference picture lists. Video decoder 30 may construct lists of reference frames, List 0 and List 1, using default construction techniques based on reference pictures stored in DPB 330. The same or similar approach may be taken by embodiments using tile groups (e.g., video tile groups) and/or tiles (eg, video tiles) in addition to or alternatively to sections (eg, video sections), for example, video may be encoded using groups I, P, or B of tiles and/or tiles.

The mode applicator 360 is configured to determine prediction information for a video block of the current video section by parsing motion vectors or related information and other syntax elements, and uses the prediction information to generate prediction blocks for the current video block being decoded. For example, the usage mode application block 360 uses some of the received syntax elements to determine the prediction mode (eg, intra frame or inter frame) used to encode the video blocks of the video section, the type of inter prediction section (eg, section B, section P, or section GPB), information about building one or more lists of reference pictures for a section, motion vectors for each interframe coded video block of the section, and other information for decoding video blocks in the current video section. The same or similar approach can be taken by embodiments using tile groups (eg, video tile groups) and/or tiles (eg, video tiles) in addition to or alternatively to sections (eg, video sections), e.g., video may be encoded using groups I , P or B tiles and/or tiles.

Embodiments of video encoder 30 shown in FIG. 3 may further be configured to split and/or encode a picture using slices (also referred to as video slices), where the picture may be sliced or decoded using one or more slices (usually non-overlapping) and each slice may contain one or more blocks (for example, CTU).

Embodiments of video decoder 30 shown in FIG. 3 may be configured to split and/or decode an image using groups of tiles (also referred to as video tile groups) and/or tiles (also referred to as video tiles) in which an image may be split or decoded using one or more (typically non-overlapping) groups tiles, and each group of tiles may contain, for example, one or more blocks (eg, CTU) or one or more tiles, where each tile may be, for example, rectangular in shape and may contain one or more blocks (eg, CTU), for example , full or fractional blocks.

Other versions of the video decoder 30 may be used to decode the encoded image data 21 . For example, decoder 30 may produce an output video stream without loop filtering unit 320 . For example, no transform decoder 30 may inversely quantize the residual signal directly, without inverse transform block 312, for some blocks or frames. In another implementation, video decoder 30 may have an inverse quantizer 310 and an inverse transform 312 combined into a single unit.

It should be understood that in encoder 20 and decoder 30, the processing result of the current stage may be further processed and then output to the next stage. For example, after the interpolation filtering, motion vector acquisition, or contour filtering, the next operation such as Clip compression or shifting may be performed on the result of the interpolation filtering, motion vector acquisition, or contour filtering process.

It should be noted that further operations may be applied to the received motion vectors of the current block (including, but not limited to, affine mode point motion vector control, sub-block motion vectors in affine mode, planar mode, ATMVP mode, temporal motion vectors and so on). For example, the value of a motion vector is limited to a predetermined range corresponding to the bit that represents it. If the representing bit of the motion vector is bitDepth, then the range is -2^ (bitDepth - 1) ~ 2^ (bitDepth -1)-1, where “^” means exponentiation. For example, if bitDepth is set to 16, the range is -32768 ~ 32767; if bitDepth is set to 18, the range is -131072~131071. For example, the value of the obtained motion vector (eg, MV of four 4x4 subblocks in one 8x8 block) is limited such that the maximum difference between the integer parts of the four 4x4 MV subblocks does not exceed N pixels, for example, not more than 1 pixel. Two methods are provided here to limit the motion vector corresponding to bitDepth.

Method 1: remove the MSB (most significant bit) of the overflow through the following operations:

ux = (mvx+2 ^bitDepth ) % 2 ^bitDepth … (1)

mvx = (ux >= 2 ^bitDepth-1 ) ? (ux - 2 ^bitDepth ) : ux … (2)

uy = (mvy+2 ^bitDepth ) % 2 ^bitDepth … (3)

mvy = (uy >= 2 ^bitDepth-1 ) ? (uy - 2 ^bitDepth ) : uy … (4)

where mvx is the horizontal component of the motion vector of the image block or sub-block, mvy is the vertical component of the motion vector of the image block or sub-block, and ux and uy are the respective intermediate values.

For example, if the value of mvx is -32769, after formulas (1) and (2) are applied, the resulting value is 32767. In a computer system, decimal numbers are stored as binary components. The two's complement of -31769 is 1,0111,1111,1111,1111 (17 bits), then the MSB is discarded so that the resulting two's complement is 0111,1111,1111,1111 (decimal 32767), which is the same as the output result of applying formulas (1) and (2).

ux = (mvpx + mvdx + 2 ^bitDepth ) % 2 ^bitDepth … (5)

mvx = (ux >= 2 ^bitDepth-1 ) ? (ux - 2 ^bitDepth ) : ux … (6)

uy = (mvpy + mvdy + 2 ^bitDepth ) % 2 ^bitDepth … (7)

mvy = (uy >= 2 ^bitDepth-1 ) ? (uy - 2 ^bitDepth ) : uy … (8)

Operations can be applied during the summation of mvp and mvd, as shown in formulas (5)-(8).

Method 2: Remove the overflow MSB by limiting the value:

vx = Clip3(-2 ^bitDepth-1 , 2 ^bitDepth-1 -1, vx)

vy = Clip3(-2 ^bitDepth-1 , 2 ^bitDepth-1 -1, vy)

where vx is the horizontal component of the motion vector of the image block or sub-block, vy is the vertical component of the motion vector of the image block or sub-block, x, y and z are respectively the three input values of the compression process MV and the definition of the Clip3 function is as follows:

In FIG. 4 schematically shows a video encoding device 400 according to an embodiment of the disclosure. The video encoding device 400, as described here, is suitable for implementing the disclosed embodiments. In an embodiment, video encoder 400 may be a decoder, such as video decoder 30 shown in FIG. 1A or an encoder such as video encoder 20 shown in FIG. 1A.

The video encoder 400 includes input ports 410 (or input ports 410) and receiver (Rx) units 420 for receiving data; a processor, logical unit, or central processing unit (CPU) 430 for data processing; transmitter (Tx) blocks 440 and output ports 450 (or output ports 450) for data transmission; and a memory 460 for storing data. Video encoder 400 may also include optical-to-electrical (OE) components and electrical-to-optical (EO) components associated with input ports 410, receiver units 420, transmitter units 440 440, and output ports 450 for input. or output of optical or electrical signals.

Processor 430 is implemented in hardware and software. Processor 430 may be implemented as one or more CPU chips, cores (eg, as a multi-core processor), FPGA, ASIC, and DSP. Processor 430 communicates with input ports 410, receive units 420, transmit units 440, output ports 450, and memory 460. Processor 430 includes an encoder 470. Encoding module 470 implements the disclosed embodiments described above. For example, encoding module 470 implements, processes, prepares, or provides various encoding operations. The introduction of the encoding module 470 therefore provides a significant improvement in the functionality of the video encoding device 400 and converts the video encoding device 400 to a different state. Alternatively, encoding module 470 is implemented as instructions stored in memory 460 and executed by processor 430.

Memory 460 may contain one or more disks, tape drives, and solid state drives, and may be used as an overflow storage device to store programs when such programs are selected for execution, and to store instructions and data that are read during program execution. Memory 460 may be, for example, volatile and/or non-volatile and may be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and/ or static random-access memory (SRAM).

In FIG. 5 shows a simplified block diagram of a device 500 that can be used as one of the source device 12 and destination device 14, or both, shown in FIG. 1 in accordance with an exemplary embodiment.

Processor 502 in device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or multiple devices capable of managing or processing information currently in existence or later acquired. Although the disclosed implementations may be implemented with a single processor as illustrated, for example processor 502, speed and efficiency advantages may be achieved with multiple processors.

Memory 504 in device 500, when implemented, may be a Read Only Memory (ROM) device or a Random Access Memory (RAM) device. The memory 504 may be any other suitable type of storage device.

Memory 504 may contain program code and data 506 that processor 502 accesses using bus 512.

Memory 504 may further comprise an operating system 508 and application programs 510, where application programs 510 include at least one program that allows processor 502 to perform the methods described herein. For example, application programs 510 may comprise applications 1-N, which further comprise a video encoding application that performs the methods described herein.

The device 500 may also include one or more output devices such as a display 518. The display 518 in one example may be a touch display that combines a display with a touch element that enables touch input to be received. Display 518 may be coupled to processor 502 via bus 512.

Although shown as a single bus in the drawing, bus 512 of device 500 may be comprised of multiple buses. Additionally, the external memory 514 may be directly connected to other components of the device 500 or may be accessed via a network and may comprise a single integrated unit, such as a memory card, or multiple units, such as multiple memory cards. Device 500 may thus be implemented in a wide variety of configurations.

The "Background Art" section refers to the intra-prediction mode.

MIP (Matrix-based Intra Prediction) and IBC (Intra Block Copy) are two prediction methods. MIP performs intra-frame prediction based on given coefficients. In IBC, sample values are predicted from other samples in the same picture using a displacement vector vector, called a block vector, in a manner conceptually similar to motion-compensated prediction.

The palette mode is a coding tool for screen content coding (SCC) to improve the efficiency of coding screen contents such as computer-generated video with a significant amount of text and graphics. In palette mode, pixels in a coding unit (CU) are represented by selected representative colors corresponding to the characteristics of the screen contents, in which pixel values typically concentrate on a few color values.

In some examples, a MIP (or IBC or paletted) mode may be applied to the luma component. When performing intra-frame chrominance prediction mode acquisition using the DM mode (acquisition mode from the corresponding luma component), special modes (lumaIntraPredMode) are assigned to the DM mode if the corresponding luma block is applied with the MIP or IBC or palette mode.

There are numerous documents related to the process of obtaining the intra-frame chrominance prediction mode, for example, ITU JVET-O0925 discloses the established derive mode (DM) as Planar mode if matrix-based intra-frame prediction is enabled. Intra Prediction, MIP), ITU JVET-O0258 exposes the inhibited Intra Block Copy, IBC mode for the chrominance component, ITU JVET-O0651 exposes the set chroma acquisition mode (Chroma DM) as DC if IBC is enabled.

In this example, the chroma acquisition process is as follows:

The inputs to this process are:

- luma location (xCb, yCb) indicating the top-left sample of the current chrominance coding block relative to the top-left sample of the luma of the current picture,

- variable cbWidth indicating the width of the current coding block, expressed in luma samples,

is a cbHeight variable indicating the height of the current encoding block, expressed in luma samples.

In this process, the intra-frame chroma prediction mode IntraPredModeC[xCb][yCb] is obtained.

The corresponding intra-frame luminance prediction mode lumaIntraPredMode is obtained as follows:

- If intra_mip_flag[xCb][yCb] is 1, lumaIntraPredMode is set to INTRA_PLANAR.

- Otherwise, if CuPredMode[xCb][yCb] is equal to MODE_IBC, lumaIntraPredMode is set to INTRA_DC.

- Otherwise, lumaIntraPredMode is set to IntraPredModeY[xCb+cbWidth /2][yCb + cbHeight/2].

In the above process, intra_mip_flag[xCb][yCb] equal to 1 indicates that the type of intra-prediction for luminance samples is matrix-based intra-prediction, and intra_mip_flag[xCb][yCb] equal to 0 indicates that the type of intra-prediction prediction for luminance samples is not matrix-based intra-frame prediction.

CuPredMode[xCb][yCb] equal to MODE_IBC indicates that the current prediction block is applied with IBC mode.

IntraPredModeY[xCb + cbWidth / 2][yCb + cbHeight / 2] defines the intra-frame luminance prediction mode for the prediction block containing the position (xCb + cbWidth / 2, yCb + cbHeight / 2), i.e., the "middle" of the corresponding luma prediction block .

In this example, the intra-frame chroma prediction mode IntraPredModeC[xCb][yCb] is obtained using intra_chroma_pred_mode[xCb][yCb] and lumaIntraPredMode as shown in Table 8-5 or Table 8-6, but there are also many other examples or tables, which can be used to obtain the intra-frame chrominance prediction mode IntraPredModeC[xCb][yCb]. In addition to the above process,

intra_chroma_pred_mode[x0][y0] defines the intra-frame prediction mode (index) for chrominance samples. Note that this is not the final intra prediction mode, it would be more correct to take this value as an intermediate index as input to obtain the final intra prediction mode for chrominance samples.

Table 8-5

IntraPredModeC[xCb][yCb] specification dependent on intra_chroma_pred_mode[xCb][yCb]lumaIntraPredMode where sps_cclm_enabled_flag is 0

intra_chroma_pred_mode[xCb][yCb] lumaIntraPredMode 0 50 18 1 X (0 <= X <= 66) 0 66 0 0 0 0 1 50 66 50 50 50 2 18 18 66 18 18 3 1 1 1 66 1 4 0 50 18 1 X

Table 8-6

IntraPredModeC[xCb][yCb] specification dependent on intra_chroma_pred_mode[xCb][yCb]lumaIntraPredMode where sps_cclm_enabled_flag is 1

intra_chroma_pred_mode[xCb][yCb] lumaIntraPredMode 0 50 18 1 X (0 <= X <= 66) 0 66 0 0 0 0 1 50 66 50 50 50 2 18 18 66 18 18 3 1 1 1 66 1 4 81 81 81 81 81 5 82 82 82 82 82 6 83 83 83 83 83 7 0 50 18 1 X

In Tables 8-5 and 8-6 above, the sps_cclm_enabled_flag set to 0 specifies that intra-frame prediction of a Cross-Component Linear Model (CCLM) from a luminance component to a chrominance component is disabled. The flag sps_cclm_enabled_flag equal to 1 specifies that intra-frame prediction of a linear model with cross components from the luminance component to the chrominance component is enabled.

In the example disclosed in the above process, the intra-frame luminance prediction mode (lumaIntraPredMode) is obtained, and then the intra-frame chroma prediction mode (intra_chroma_pred_mode) is obtained from the bitstream, for example. According to the value of intra-frame luminance prediction mode (lumaIntraPredMode) and the value of syntax intra_chroma_pred_mode, the output prediction mode is obtained in accordance with Tables 8-5 and 8-6.

In some examples corresponding to tables 8-5 and 8-6, the output mode may be one of 70 modes. The 70 modes can be classified as 67 normal modes and three cross-component linear model (CCLM) modes. The 67 normal modes can be further divided into non-angular modes (planar and DC modes) and 65 angular modes (modes 2-66) as shown in FIG. 6.

Modes 81, 82, 83 correspond to the three CCLM (Cross Component Linear Model) modes, which correspond to Left and Top Linear Mode (INTRA_LT_CCLM), Left Linear Mode (INTRA_L_CCLM), and Top Linear Mode (INTRA_T_CCLM). These modes are summarized in Table 8-3.

Table 8-3

Intra-frame specification and related names

Intra prediction mode Appropriate title 0 INTRA_PLANAR 1 INTRA_DC 2..66 INTRA_ANGULAR2..INTRA_ANGULAR66 81..83 INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

A mismatch between the position of the MIP (or IBC) flag and the position of the intra-frame luminance prediction mode can cause a potential problem in obtaining the intra-frame chrominance prediction mode.

Namely, the intra_mip_flag (or CuPredMode) flag is selected from the position (xCb, yCb) of the luminance, while the intra-frame prediction mode of the luma block is selected from the position (xCb + cbWidth / 2, yCb + cbHeight / 2), as follows (defined as the process VVC receive intra-frame chroma prediction mode).

- If intra_mip_flag[xCb][yCb] is 1, lumaIntraPredMode is set to INTRA_PLANAR.

- Otherwise, if CuPredMode[xCb][yCb] is equal to MODE_IBC, lumaIntraPredMode is set to INTRA_DC.

- Otherwise, lumaIntraPredMode is set to IntraPredModeY[xCb + cbWidth / 2][yCb + cbHeight / 2].

When CUPredMode[xCb][yCb] is equal to MODE_IBC, this indicates that no intra prediction mode is defined in the VVC working draft.

In the example shown in FIG. 7, the division of the luminance prediction block and the chrominance prediction block in the coding region (eg, the coding region is CTU) does not match when the binary tree encoding method (one tree division for luma and one tree division for chroma) is enabled. For simplicity, it is assumed that the chrominance component in the CTU is divided into two parts, a left and a right subdivision (left CTU in FIG. 7). The luma component in the CTU is also split into two parts, however, this is an upper and lower subdivision. The upper subdivision of the luma component is applied with the normal intra prediction mode, and the lower subdivision is applied with the IBC mode (right CTU in FIG. 7).

In this example, shown in FIG. 7, the chrominance component performs the acquisition of the intra-frame chrominance prediction mode. According to the VVC process of obtaining the intra-frame chrominance prediction mode as described above, the value of intra_mip_flag[xCb][yCb] is 0 because the normal intra-frame prediction mode is applied to the upper subdivision within the luminance component whose left-upper position is (xCb, yCb) . For the same reason, CuPredMode[xCb][yCb] is also not equal to MODE_IBC, where the position (xCb, yCb) belongs to the upper subdivision of the luma component and where the block is applied with normal intra prediction. Therefore, deriving the intra-frame chrominance prediction mode will retrieve the normal intra-luma prediction mode value in (xCb + cbWidth / 2, yCb + cbHeight / 2) (i.e., IntraPredModeY[xCb + cbWidth/2][yCb + cbHeight/2]). However, the position (xCb + cbWidth / 2, yCb + cbHeight / 2) now points to a location in the lower subdivision of the luma component (as shown in the lower CTU in FIG. 7). Luma component lower sub-division is applied in conjunction with IBC and no intra-luma prediction mode is defined. Therefore, in the current specification, the value IntraPredModeY[xCb + cbWidth / 2][yCb + cbHeight / 2] is an undefined value. In this case of going beyond the usual limits, the process of obtaining the intra-frame chrominance prediction mode is violated.

The example shown in FIG. 7 is an overrun case where two subdivisions in the luma component are applied with the conventional intra prediction mode and the IBC mode, respectively. It can be found that the color acquisition process is also broken when the IBC mode is changed to the MIP mode or the palette mode.

The luma location (xCb, yCb) indicates the top-left sample of the current chroma coding block relative to the top-left luminance sample of the current picture.

The luminance location value (xCb, yCb) of the top-left chrominance component location of the current coding block is obtained, the luminance location value (xCb, yCb) is determined in the luminance sample of the current coding block; get the value of the first indication information (eg, intra_mip_flag) for the current coding block, obtain the value of the first indication information for the current coding block, obtain according to the luminance position (cbWidth/2, cbHeight/2) relative to the top-left luminance sample of the current coding block, cbWidth represents the width of the current coding block in luma samples, cbHeight represents the height of the current coding block in luma samples (xCb+cbWidth/2, yCb+cbHeight/2)).

As an example shown in FIG. 8, the location value for the top-left position of the current picture is (0, 0), and the location value for the top-left position of the current coding block is (128, 64). The width of the current coding block is 64, the height of the current coding block is 32. Therefore, the location value that is used to obtain the intra prediction mode is ((128+ 64/2), 64+32/2)), i.e., (160, 80).

In one embodiment of the present invention, the position of the IBC or MIP flags is the same as the position of the intra-frame luminance prediction mode, and the following process is applied:

- If intra_mip_flag[xCb + cbWidth / 2][yCb + cbHeight / 2] is 1, lumaIntraPredMode is set to INTRA_PLANAR.

- Otherwise, if CuPredMode[xCb + cbWidth / 2][yCb + cbHeight / 2] is MODE_IBC, lumaIntraPredMode is set to INTRA_DC

- Otherwise, lumaIntraPredMode is set to IntraPredModeY[xCb + cbWidth / 2][yCb + cbHeight / 2].

In such a case, again taking FIG. 7 as an example, obtaining the intra-frame chrominance prediction mode should fall on the second branch (that is, otherwise, if CuPredMode[xCb + cbWidth / 2][yCb + cbHeight / 2] is equal to MODE_IBC), lumaIntraPredMode is set to DC mode.

The method ensures that the IBC and MIP modes are detected in the first place using the matching position, and thus the variable lumaIntraPredMode must always be assigned along with the actual intra-luma prediction mode.

In one embodiment, when the position of the IBC or MIP flags is the same as the position of the luminance intra prediction mode and their respective intra prediction mode is set to planar mode in both cases, the following process is applied:

- If intra_mip_flag[xCb + cbWidth / 2][yCb + cbHeight / 2] is 1 or CuPredMode[xCb + cbWidth / 2][yCb + cbHeight / 2] is MODE_IBC, lumaIntraPredMode is set to INTRA_PLANAR.

- Otherwise, lumaIntraPredMode is set to IntraPredModeY[xCb + cbWidth / 2][yCb + cbHeight / 2].

In one embodiment, when the position of the IBC or MIP flags is the same as the position of the luminance intra prediction mode and their respective intra prediction mode is set to DC mode in both cases, the following process is applied:

- If intra_mip_flag[xCb + cbWidth / 2][yCb + cbHeight / 2] is 1 or CuPredMode[xCb + cbWidth / 2][yCb + cbHeight / 2] is MODE_IBC, lumaIntraPredMode is set to INTRA_DC.

- Otherwise, lumaIntraPredMode is set to IntraPredModeY[xCb + cbWidth / 2][yCb + cbHeight / 2].

In one embodiment of the present invention, the position of the IBC or MIP flags is the same as the position of the intra-frame luminance prediction mode, and the following process is applied:

- If intra_mip_flag[xCb + cbWidth / 2][yCb + cbHeight / 2] is 1, lumaIntraPredMode is set to INTRA_PLANAR.

- Otherwise, if CuPredMode[0][xCb + cbWidth / 2][yCb + cbHeight /2] is MODE_IBC, lumaIntraPredMode is set to INTRA_DC

- Otherwise, lumaIntraPredMode is set to IntraPredModeY[xCb + cbWidth / 2][yCb + cbHeight / 2].

In one embodiment of the present invention, the position of the IBC, MIP, or palette flags is the same as the position of the intra-frame luminance prediction mode, and the following process is applied:

- If intra_mip_flag[xCb + cbWidth / 2][yCb + cbHeight / 2] is 1, lumaIntraPredMode is set to INTRA_PLANAR.

- Otherwise, if CuPredMode[0][xCb + cbWidth / 2][yCb + cbHeight / 2] is MODE_IBC or MODE_PLT, lumaIntraPredMode is set to INTRA_DC.

- Otherwise, lumaIntraPredMode is set to IntraPredModeY[xCb + cbWidth / 2][yCb + cbHeight / 2].

In one embodiment of the present invention, the position of the IBC, MIP, or palette flags is the same as the position of the intra-frame luminance prediction mode, and the following process is applied:

- If intra_mip_flag[xCb + cbWidth / 2][yCb + cbHeight / 2] is 1 or if CuPredMode[0][xCb + cbWidth / 2][yCb + cbHeight / 2] is MODE_IBC or MODE_PLT, lumaIntraPredMode is set to INTRA_PLANAR.

- Otherwise, lumaIntraPredMode is set to IntraPredModeY[xCb + cbWidth / 2][yCb + cbHeight / 2].

In one embodiment of the present invention, the position of the IBC, MIP, or palette flags is the same as the position of the intra-frame luminance prediction mode, and the following process is applied:

- If intra_mip_flag[xCb + cbWidth / 2][yCb + cbHeight / 2] is 1 or if CuPredMode[0][xCb + cbWidth / 2][yCb + cbHeight / 2] is MODE_IBC or MODE_PLT, lumaIntraPredMode is set to INTRA_DC.

- Otherwise, lumaIntraPredMode is set to IntraPredModeY[xCb + cbWidth / 2][yCb + cbHeight / 2].

In one embodiment of the present invention, the position of the IBC, MIP, or palette flags is the same as the position of the intra-frame luminance prediction mode, and the following process is applied:

- if CuPredMode[0][xCb + cbWidth / 2][yCb + cbHeight / 2] is MODE_IBC or MODE_PLT, lumaIntraPredMode is set to INTRA_DC

- Otherwise, if intra_mip_flag[xCb + cbWidth / 2][yCb + cbHeight / 2] is 1, lumaIntraPredMode is set to INTRA_PLANAR.

- Otherwise, lumaIntraPredMode is set to IntraPredModeY[xCb + cbWidth / 2][yCb + cbHeight / 2].

The above embodiments use CuPredMode[xCb + cbWidth / 2][yCb + cbHeight / 2] or CuPredMode[i][xCb + cbWidth / 2][yCb + cbHeight / 2]. In fact, they define the same thing, i.e., the prediction mode at the position (xCb + cbWidth / 2, yCb + cbHeight / 2) of the luma components.

Note that CuPredMode[i][xCb + cbWidth / 2][yCb + cbHeight / 2] is used with one or more luma or chrominance component definition sizes, where i=0 or 1.

CuPredMode[0][xCb + cbWidth / 2][yCb + cbHeight / 2] represents the luma component prediction mode because its size index is 0.

If a chrominance channel is used, the corresponding variable must be CuPredMode[1][xCb + cbWidth / 2][yCb + cbHeight / 2].

In the above embodiment, when CuPredMode[0][xCb + cbWidth / 2][yCb + cbHeight / 2] is equal to MODE_PLT, this indicates that the luminance component at the luminance position (cbWidth / 2, cbHeight / 2) uses palette mode. The position (cbWidth / 2, cbHeight / 2) of the sample specifies the position relative to the top-left luma sample of the current coding block. The top left sample of the current coding block (xCb, yCb) defines a position relative to the top left sample of the current picture.

An example of the position (xCb+cbWidth/2, yCb+cbHeight/2) of the brightness is shown in FIG. 7, where xCb = 128, yCb = 64, cbWidth = 64, cbHeight = 32.

In particular, the following methods and embodiments are implemented by an encoder or decoder. The decoding device may be the video decoder 30 shown in FIG. 1A or decoder 30 shown in FIG. 3.

The encoder may be the video encoder 20 shown in FIG. 1A or encoder 20 shown in FIG. 2.

According to embodiment 900 (FIG. 9), the device obtains first indication information for the luminance position (cbWidth/2, cbHeight/2) of the current coding block relative to the upper left luminance sample position (xCb, yCb) of the current coding block, where cbWidth represents the width of the current coding block in the luminance component, cbHeight represents the height of the current coding block in the luminance component, in step 901.

Accordingly, cbWidth/2 represents half the width of the current coding block in the luminance component, and cbHeight/2 represents half the height of the current coding block in the luma component. The absolute position of the luminance position (cbWidth/2, cbHeight/2) is (xCb+cbWidth/2, yCb+cbHeight/2), that is, the "middle" of the corresponding luminance prediction block.

For example, the first indication information for the brightness position (cbWidth/2, cbHeight/2) may be intra_mip_flag[xCb + cbWidth / 2][yCb + cbHeight / 2].

In step 902, when the first indication information indicates that Matrix-based Intra Prediction (MIP) is applied to the luminance component at the luminance position (cbWidth/2, cbHeight/2) relative to the top-left position (xCb , yCb) luminance samples of the current coding block, the device sets the intra-frame luminance prediction mode value associated with the current coding block as the first default value. For example, the first default value is the planar mode value.

When intra_mip_flag[xCb + cbWidth / 2][yCb + cbHeight / 2] is equal to 1, the first indication information indicates that MIP is applied to the luma component.

In step 903, when the first indication information indicates that MIP is not applied to the luminance component at the luminance position (cbWidth/2, cbHeight/2) relative to the top-left position (xCb, yCb) of the luminance sample of the current coding block, the device acquires the second indication information for the position (cbWidth/2, cbHeight/2) of the luminance of the current coding block relative to the top-left position (xCb, yCb) of the luminance sample of the current coding block.

When intra_mip_flag[xCb + cbWidth / 2][yCb + cbHeight / 2] is 0, the first indication information indicates that MIP is not applied to the luma component.

For example, the first indication information for the brightness position (cbWidth/2, cbHeight/2) may be intra_mip_flag[xCb + cbWidth / 2][yCb + cbHeight / 2].

In step 905, when the second indication information indicates that the intra block copy (Intra Block Copy, IBC) copy mode is applied to the luminance component at the luminance position (cbWidth/2, cbHeight/2) relative to the top-left position (xCb, yCb) of the sample luminance of the current coding block, the device sets the intra-frame luminance prediction mode value associated with the current coding block as the second default value. For example, the first default value is the planar mode value.

When CuPredMode[0][xCb + cbWidth / 2][yCb + cbHeight / 2] is MODE_IBC, the second indication information indicates that the IBC mode is applied to the luma component. When CuPredMode[0][xCb + cbWidth / 2][yCb + cbHeight / 2] is MODE_PLT, the second indication information indicates that a palette mode is applied to the luminance component.

When the second indication information indicates that the IBC mode or palette mode is not applied to the luminance component at the luminance position (cbWidth/2, cbHeight/2), the intra luminance prediction mode associated with the current coding block is set as the intra luminance prediction mode at the position [ xCb +cbWidth/2][yCb + cbHeight/2], that is, IntraPredModeY[xCb + cbWidth/2][yCb + cbHeight / 2].

In step 907, the device obtains the intra chroma prediction mode value based on the intra luma prediction mode value of the current coding block. If the intra-luma prediction mode value of the current coding block is the first default value shown in step 902, the device obtains the intra-luma prediction mode value based on the first default value. If the intra-luma prediction mode value of the current coding block is the second default value shown in step 905, the device obtains the intra-luma prediction mode value based on the second default value.

Detailed information on obtaining the intra-frame chrominance prediction mode using the intra-frame prediction mode from the corresponding luminance component is given in the above-mentioned embodiments.

In FIG. 10 shows an embodiment of device 1000. Device 1000 may be the video decoder 30 shown in FIG. 1A or decoder 30 shown in FIG. 3, or may be the video encoder shown in FIG. 1A or encoder 20 shown in FIG. 2. Device 1000 may be used to implement embodiment 900 and other embodiments described above.

The device 1000 for obtaining the intra-frame chrominance prediction mode includes an acquisition block 1001, a setting block 1002, and an intra-frame chrominance prediction mode block 1003. An obtaining unit 1001, configured to obtain first indication information for the position (cbWidth/2, cbHeight/2) of the brightness of the current coding block relative to the upper left position (xCb, yCb) of the brightness sample of the current coding block, where cbWidth represents the width of the current coding block in the component luminance, cbHeight represents the height of the current encoding block in the luma component. A setting unit 1002, configured to set the value of the intra-frame luminance prediction mode associated with the current coding block as the first default value when the first indication information indicates that Matrix-based Intra Prediction (MIP) is applied for the luminance component at the position (cbWidth/2, cbHeight/2) of the luminance relative to the top-left position (xCb, yCb) of the luminance sample of the current coding block.

Obtaining unit 1001, further configured to obtain second indication information for the luminance position (cbWidth/2, cbHeight/2) of the current coding block when the first indication information indicates that MIP is not applied to the luminance component at position (cbWidth/2, cbHeight/ 2) luminance relative to the top-left position (xCb, yCb) of the luminance sample of the current coding block.

A setting unit 1002, further configured to set the value of the intra-frame prediction luminance prediction mode associated with the current coding block as the second default value when the second indication information indicates that the copy mode of the intra-frame prediction block (Intra Block Copy, IBC) or the palette mode is applied for the luminance component at the position (cbWidth/2, cbHeight/2) of the luminance relative to the top-left position (xCb, yCb) of the luminance sample of the current coding block.

The intra chroma prediction mode block 1003 is configured to obtain an intra chroma prediction mode value based on the intra luma prediction mode value of the current coding block.

The present disclosure provides the following set of embodiments or approaches:

According to the first approach, the invention is related to an encoding method implemented by a decoding device and comprising the steps of:

obtaining the value of the first indication information for the current coding block, in which the value of the first indication information for the current coding block is obtained from the luminance component corresponding to the position (cbWidth/2, cbHeight/2) of the luminance relative to the upper-left sampling position of the luminance of the current coding block (xCb, yCb), where cbWidth represents the width of the current coding block, expressed in luma samples, cbHeight represents the height of the current coding block, expressed in luma samples;

set the value of the luminance intra-frame prediction mode associated with the current coding block as the first default value when the value of the first indication information indicates that Matrix-based Intra Prediction (MIP) is applied to the luminance component at position ( cbWidth/2, cbHeight/2) luminance relative to the top-left luminance sample of the current coding block (xCb, yCb).

obtaining a chrominance intra-frame prediction mode value based on the luma intra-frame prediction mode value of the current coding block.

According to the second approach, the invention is related to an encoding method implemented by a decoding device and comprising the steps of:

obtaining the value of the first indication information for the current coding block, in which the value of the first indication information for the current coding block is obtained from the luminance component corresponding to the position (cbWidth/2, cbHeight/2) of the luminance relative to the top-left luminance sample of the current coding block (xCb, yCb ), where cbWidth represents the width of the current coding block, expressed in luma samples, cbHeight represents the height of the current coding block, expressed in luma samples;

set the value of the intra block prediction luminance mode associated with the current coding block as the first default value when the value of the first indication information indicates that the copy mode of the intra block copy (Intra Block Copy, IBC) or palette mode is applied to the luminance component at position (cbWidth /2, cbHeight/2) brightness relative to the top-left brightness sample of the current coding block (xCb, yCb);

obtaining a chrominance intra-frame prediction mode value based on the luma intra-frame prediction mode value of the current coding block.

As discussed above, in conventional MIP or IBC or palette related cases (as in the example shown in FIG. 7), there is a mismatch between the position of the MIP (or IBC or palette) mode and the position of the intra-frame luminance prediction mode when separating the luminance component different from splitting the chrominance component (eg, when the binary tree encoding method is allowed). In the approaches and forms of the invention, obtaining the first indication information from a fixed position (cbWidth/2, cbHeight/2) in the corresponding luma component ensures that the position of the MIP mode and the position of the intra-frame luminance prediction mode are the same when, for a given block size, the division of the luminance component is different from separating the chrominance component (for example, when a binary tree encoding method is allowed). When the first indication information does not indicate that MIP is applied to the luminance component at the luminance position (cbWidth/2, cbHeight/2), obtaining the second indication information from the fixed position (cbWidth/2, cbHeight/2) at the corresponding luminance component ensures that the position The mode IBC and the intra-luma prediction mode position are aligned when, for a given block size, the luminance component separation is different from the chrominance component separation (eg, when the binary tree coding method is enabled). Alternatively, obtaining the second indication information from a fixed position (cbWidth/2, cbHeight/2) in the corresponding luminance component ensures that the mode palette position and the intra-frame luminance prediction mode position are aligned when, for a given block size, the luminance component separation is different from the chrominance component separation ( for example, when a binary tree encoding method is allowed).

The following is an explanation of the applications of the encoding method as well as the decoding method as described in the above embodiments, and the system using them.

In FIG. 11 is a block diagram of a content providing system 3100 for implementing a content distribution service. This content support system 3100 includes an image acquisition device 3102, a terminal device 3106, and optionally a display 3126. The image acquisition device 3102 communicates with the terminal device 3106 over a communication channel 3104. The communication channel may comprise the communication channel 13 described above. Link 3104 includes, but is not limited to, WI-FI, Ethernet, cable line, wireless link (3G/4G/5G), USB, or any combination thereof, and the like.

The image pickup device 3102 generates data and may encode the data in an encoding method as shown in the above embodiments. Alternatively, the image pickup device 3102 may distribute the data to a streaming server (not shown in the drawings), and the server encodes the data and transmits the encoded data to the terminal device 3106. The image pickup device 3102 includes, but is not limited to, a camera, a smartphone or tablet, a computer, or a laptop. , videoconferencing system, PDA, vehicle-mounted device, or a combination of any of them, etc. For example, the image acquisition device 3102 may include a source device 12 as described above. When the data includes video data, the video encoder 20 included in the image pickup device 3102 can actually perform a video encoding process. When the data contains audio data (eg, voice), the audio encoder contained in the image pickup device 3102 can actually perform the audio encoding process. For some practical scenarios, the image acquisition device 3102 distributes encoded video data and audio data by multiplexing them together. For other practical scenarios, such as in a video conferencing system, encoded audio data and encoded video data are not multiplexed. The image pickup device 3102 separately distributes the encoded audio data and the encoded video data to the terminal device 3106.

In the content delivery system 3100, the terminal device 310 receives and reproduces the encoded data. Terminal device 3106 may be a device with data reception and reconstruction capabilities, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (digital video recorder, DVR) 3112, a television receiver 3114, a set-top box ( set top box (STB) 3116, a videoconferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, a vehicle-mounted device 3124, or a combination of any of them, etc., capable of decoding the above encoded data. For example, terminal device 3106 may include a destination device 14 as described above. When the encoded data includes video data, the video encoder 30 included in the terminal has priority to perform video decoding. When the encoded data contains audio data, the audio decoder 30 contained in the terminal has priority to perform audio decoding.

For a terminal device having a display, such as a 3108 smartphone or tablet, a 3110 computer or laptop, a 3112 Network Video Recorder (NVR)/Digital Video Recorder (DVR), a 3114 TV, a 3122 Personal Digital Assistant (PDA), or a 3124 mountable on the vehicle, the terminal may provide the decoded data to its display. For a terminal device that does not have a display, such as STB 3116, video conferencing system 3118 or video surveillance system 3120, an external display 3116 communicates with them and shows the decoded data.

When any of the devices in this system performs encoding or decoding, the image encoding device or the image decoding device shown in the above embodiments may be used.

In FIG. 12 shows the structure of an example terminal device 3106. After the terminal device 3106 receives a stream from the image pickup device 3102, the protocol block 3202 parses the transmission protocol of the stream. The protocol includes, but is not limited to, Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG -DASH, Real-time Transport protocol (RTP), Real Time Messaging Protocol (RTMP), or any combination thereof, etc.

After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexer 3204 . A demultiplexer 3204 may separate the multiplexed data into encoded audio data and encoded video data. As described above, for some practical scenarios, such as in a videoconferencing system, encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is transmitted to the video decoder 3206 and the audio decoder 3208 without passing through the demultiplexer 3204.

The demultiplexing process generates an elementary video stream (ES), audio ES, and optionally subtitles. The video decoder 3206, which includes the video decoder 30 as explained in the above embodiments, decodes the video ES by the decoding method shown in the above embodiments to form a video frame, and supplies this data to the synchronization block 3212. An audio decoder 3208 decodes the audio ES to form an audio frame and provides this data to a sync block 3212. Alternatively, the video frame may be stored in a buffer (not shown in FIG. 12) before being fed to the sync block 3212. Likewise, an audio frame may be stored in a buffer (not shown in FIG. 12) before being fed to the sync block 3212.

The sync block 3212 synchronizes the video frame and the audio frame and provides video/audio to the video/audio display 3214. For example, the sync block 3212 synchronizes the presentation of video information and audio information. The information may be encoded in syntax using timestamps related to the presentation of the encoded audio and video data and timestamps related to the transmission of the data stream itself.

If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles and synchronizes them with the video frame and the audio frame, and supplies the video/audio/subtitle data to the video/audio/subtitle display 3216.

The present invention is not limited to the above system, and any of the devices such as an image encoding device or an image decoding device can be introduced into another system such as an automobile system.

Mathematical Operators

The mathematical operators used in this application are similar to those used in the "C" programming language. However, the results of integer division operations and arithmetic shifts are defined more precisely and additional operations such as exponentiation and division of real values are defined. Numbering and counting rules usually start at 0, for example, "first" is equivalent to 0, "second" is equivalent to 1, and so on.

Arithmetic operators

The following arithmetic operators are defined as follows:

+ Addition - Subtraction (as a two-argument operator) or negation (as a one-argument prefix operator) * Multiplication, including matrix multiplication x ^y Exponentiation. Specifies x to the power of y. In other contexts, this notation is used to provide superscripts not intended to be interpreted as exponentiation. / Integer division truncating the result towards zero. For example, 7/4 and -7/-4 are truncated to 1, and -7/4 and 7/-4 are truncated to -1. ÷ Used to indicate division in mathematical equations in which no truncation or rounding is expected. Used to indicate separation in mathematical equations in which no truncation or rounding is expected. Summation f(i), where i takes any integer value from x up to and including y. x % y Module. The remainder of x divided by y is only defined for integers x and y where x >= 0 and y > 0.

Logical operators

The logical operators below are defined as follows:

x && y Boolean logical "AND" x and y x | | y Boolean logical "OR" x and y ! Boolean logical "NOT" x? y : z If x is TRUE (true) or not equal to 0, evaluate the value of y; otherwise, the z-value is evaluated.

Relationship operators

The ratio operators below are defined as follows:

> More than >= Greater than or equal < Less than <= Less or equal == Equals != Not equal

When a ratio operator is applied to a syntax element or variable that has been assigned the value "na" (not applicable), the value "na" for the syntax element or variable is treated as a single value. The value "na" is treated as unequal to any other value.

Bitwise Operators

The ratio operators below are defined as follows:

& Bitwise "and". When working with integer arguments, the complementary representation of 2 for the integer value is in effect. When dealing with a binary argument containing fewer bits than another argument, the shorter argument is expanded to include more significant bits of 0. | Bitwise "OR". When working with integer arguments, the complementary representation of 2 for the integer value is in effect. When dealing with a binary argument containing fewer bits than another argument, the shorter argument is expanded to include more significant bits of 0. ^ Bitwise XOR When working with integer arguments, the complementary representation of 2 for the integer value is in effect. When dealing with a binary argument containing fewer bits than another argument, the shorter argument is expanded to include more significant bits of 0. x >> y Arithmetic right shift of the complementary integer representation up to 2 for the binary digits x * y. This function is only defined for non-negative integer values of y. Bits shifted to the most significant bit (MSB) as a result of a right shift have a value equal to the MSB for x preceding the shift operation. x << y Arithmetic left shift of complementary integer representation up to 2 for binary digits x * y. This function is only defined for non-negative integer values of y. Bits shifted to the least significant bit (LSB) as a result of a left shift have a value of 0.

Assignment operators

The following arithmetic operators are defined as follows:

= assignment operator ++ Increment up, that is, x+ + is equivalent to x = x + 1; when used in a matrix index, evaluates the value of the variable before incrementing up. - - Increment down, that is, x- - is equivalent to x = x - 1; when used in a matrix index, evaluates the value of the variable before incrementing down. += Increment up by the specified amount, that is, x += 3 is equivalent to x = x + 3, and x += (-3) is equivalent to x = x + (-3). -= Increment down by the specified amount, that is, x -= 3 is equivalent to x = x - 3, and x -= (-3) is equivalent to x = x - (-3).

Range designation

The following notation is used to indicate a range of values.

x = y..z x takes integer values from y to z, inclusive, where x, y, and z are integers and z is greater than y. _.

Math functions

The following mathematical functions are defined:

asin(x) trigonometric inverse sine function, acting on the x argument, which is in the range -1.0 to 1.0, inclusive, with an output value in the range -π÷2 to π÷2, inclusive, in units of raadians. Atan(x) trigonometric inverse tangent function, acting on the argument x, with an output value in the range -π÷2 to π÷2, inclusive, in units of radians.

Ceil(x) is the smallest integer greater than or equal to x.

Clip1 _Y (x) = Clip3(0, (1 << BitDepth _Y ) - 1, x)

Clip1 _C (x) = Clip3(0, (1 << BitDepth _C ) - 1, x)

Cos(x) is a trigonometric cosine function acting on the argument x in units of radians.

Floor(x) is the largest integer less than or equal to x.

Ln(x) - natural logarithm of x (logarithm to base e, where e is the base constant of the natural logarithm 2.718 281 828…).

Log2(x) - base 2 logarithm of x.

Log10(x) - base 10 logarithm of x.

Round(x) = Sign(x) * Floor(Abs(x) + 0.5)

Sin(x) is a trigonometric sine function acting on the x argument in units of radians.

Sqrt(x) =

Swap(x, y) = (y, x)

Tan(x) is a trigonometric tangent function acting on the x argument in units of radians.

Order of precedence of operations

When the order of operations in an expression is not explicitly specified using parentheses, the following rules apply.

- Operations with a higher priority are evaluated before any operation with a lower priority.

- Operations of equal precedence are evaluated sequentially from left to right.

The table below lists the priority of operations in order from highest to lowest; a higher position in the table indicates a higher priority

For those operations that are also used in the C programming language, the order of precedence used in this specification is the same as that used in the C programming language.

Table: Order of execution of operations from highest (top of the table) to lowest (bottom of the table).

operations (with operands x, y and z) "x++", "x- -" "!x", "-x" (as unary prefix operator) x ^y "x * y", "x / y", "x ÷ y", "", "x % y" "x + y", "x - y" (as a two-argument operator), " " "x << y", "x >> y" "x < y", "x <= y", "x > y", "x >= y" "x == y", "x != y" "x & y" "x|y" "x && y" "x||y" "x? y: z" "x..y" "x = y", "x += y", "x -= y"

Text description of logical operations

In the text, the operator of logical operations, as it should be described mathematically in the following form:

if(condition 0)

statement 0

else if(condition 1)

statement 1

…

else /* informative remark on remaining condition */

statement n

can be written like this:

… as follows / … the following applies:

- If condition 0, statement 0

Otherwise, if condition 1, statement 1

- ...

- Otherwise (informational note on the remaining condition), operator n

Each "If … Otherwise, if … Otherwise, …" statement in the text is entered with "… as follows" or "… the following applies" immediately followed by "If … ". The last condition "If … Otherwise, if … Otherwise, …" always means "Otherwise, …". Interleaved "If … Otherwise, if … Otherwise, …", which can be identified by matching "… as follows" or "… the following applies" ending in "Otherwise, …".

In the text, the operator of logical operations, as it should be described mathematically in the following form:

if(condition 0a && condition 0b)

statement 0

else if(condition 1a | | condition 1b)

statement 1

…

else

statement n

can be written like this:

… as follows / … the following applies:

- If all of the following conditions are true, statement 0:

- condition 0a

- condition 0b

- Otherwise, if one or more of the following conditions are true, statement 1:

- condition 1a

- condition 1b

- ...

-Otherwise, statement n

In the text, the operator of logical operations, as it should be described mathematically in the following form:

if(condition 0)

statement 0

if(condition 1)

statement 1

can be written like this:

When condition 0, statement 0

When condition 1, statement 1

Although embodiments of the invention have been described primarily based on video coding, it should be noted that embodiments of encoding system 10, encoder 20 and decoder 30 (and thus system 10) and other embodiments described herein may also be performed with the possibility of application in photo processing or in encoding, that is, in the processing or encoding of an individual image, regardless of any previous or subsequent image, as in video encoding. In general, only the inter prediction blocks 244 (encoder) and 344 (decoder) may be unavailable in the case where image processing coding is limited to a single image 17.

All other functionality (also referred to as tools or technologies) of video encoder 20 and video decoder 30 can equally be used for photo processing, e.g. ) transform 212/312, partitioning 262/362, intra prediction 254/354, and/or loop filtering 220, 320, entropy coding 270, and entropy decoding 304.

Embodiments of, for example, encoder 20 and decoder 30, and the functions described herein, for example, with reference to encoder 20 and decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on a computer-readable medium or transmitted via communications as one or more instructions or code and executed by a processor in hardware. The computer-readable medium may comprise a computer-readable storage medium corresponding to a physical medium, such as a storage medium for data storage, or a communication medium comprising any medium facilitating the transfer of a computer program from one place to another, such as according to a communication protocol. Thus, computer-readable media may generally correspond to (1) physical, computer-readable storage media that is non-portable, or (2) a communication medium such as a signal or carrier. A storage media can be any available media that can be accessed by one or more computers, one or more computers, or one or more processors to retrieve instructions, control program, and/or data structures for implementing the technologies described in this disclosure. . The computer program product may include a computer-readable medium.

By way of example, and not limitation, such computer-readable storage media may comprise random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM), or other storage device on optical disks, magnetic disk storage or other magnetic storage devices, flash memory or any other medium that can be used to store the desired control program in the form of instructions or data structures and can be accessed by a computer. In addition, any connection is appropriately referred to as a computer-readable medium. For example, if commands are transmitted from a website, server, or other remote source using coaxial cable, optical cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio wave, or microwave, then then coaxial cable, optical cable, twisted pair, DSL, or wireless technologies such as infrared, radio wave, or microwave are contained in the media definition. However, it should be understood that computer-readable storage media and storage media may not contain connections, carriers, signals, or other portable media, but be non-transferable physical storage media. The discs include compact discs (CD) black discs, optical discs, digital versatile discs (DVDs), floppy disks and Blu-ray discs, where some discs typically reproduce data magnetically while other discs reproduce data optically using lasers. Combinations of the above may also be included within the scope of computer-readable media.

Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application-specific integrated circuits (ASICs), field programmable gate arrays, FPGA) or other equivalent integrated or discrete logic circuits. Accordingly, the term "processor" as used herein may refer to any of the above structures, or to any other structure suitable for implementing the technologies described here. In addition, in some embodiments, the functionality presented herein may be provided within dedicated hardware and/or software modules capable of encoding and decoding, or may be contained in a combined codec. In addition, technologies may be fully implemented in one or more circuits or logic elements.

The technologies described in this disclosure may be implemented by a wide variety of devices, including a wireless mobile phone, an integrated circuit (IC), or an IC stack (eg, a stack of chips). Various components, modules, or blocks are described in this disclosure to emphasize the functional aspects of devices capable of implementing the disclosed technologies, but not necessarily requiring implementation with different hardware blocks. More specifically, as described above, the various blocks may be combined into an encoder/decoder hardware block, or may be provided by a set of interoperable hardware comprising one or more processors as described above, in combination with appropriate software and firmware.

Claims (58)

1. An encoding method implemented by a decoding device or an encoding device, comprising the steps of: obtaining indication information for the position (cbWidth/2, cbHeight/2) of the luminance of the current coding block relative to the top-left position (xCb, yCb) of the luminance samples of the current coding block, where cbWidth represents the width of the current coding block in luma samples, cbHeight represents the height of the current block coding in luma samples; set the value of the intra-frame luminance prediction mode associated with the current coding block as the first default value when the value of the indication information indicates that matrix-based intra-frame prediction (MIP) is applied to the luminance samples at position (cbWidth/2, cbHeight/ 2) luminance relative to the position (xCb, yCb) of the top-left luminance sample of the current coding block (xCb, yCb); And obtaining an intra-frame chrominance prediction mode value based on the intra-frame luminance prediction mode value of the current coding block. 2. The method of claim 1, wherein the first default value is equal to the planar mode value or the first default value is equal to the DC mode value. 3. The method of claim 1 or 2, further comprising: setting an intra-frame luminance prediction mode value, associated with the current coding block as the second default when the Intra Block Copy (IBC) block copy mode or palette mode is applied to the position (cbWidth/2, cbHeight/2) of the luminance, and the indication information indicates that the MIP does not apply to luminance samples at position (cbWidth/2, cbHeight/2) of luminance relative to position (xCb, yCb) of the top-left luma sample of the current coding block. 4. The method of claim 3, wherein the second default value is the DC mode value or the planar mode value. 5. The method according to any one of paragraphs. 1-4, in which the luma component division and the chroma component division of the current coding block do not match each other. 6. The method according to any one of paragraphs. 1-5, in which the absolute position of the brightness position (cbWidth/2, cbHeight/2) is (xCb+cbWidth/2, yCb+cbHeight/2), and the absolute position is (xCb+cbWidth/2, yCb+cbHeight/2) determines the position relative to the top left sample of the current image; the position (cbWidth/2, cbHeight/2) of the luminance specifies a position relative to the position (xCb, yCb) of the upper left luminance sample of the current coding block. 7. The method according to any one of paragraphs. 3-5, wherein when the IBC mode or palette mode is applied to the luminance samples at the luminance position (cbWidth/2, cbHeight/2), CuPredMode[0][xCb+cbWidth/2][yCb+cbHeight/2] is equal to MODE_IBC or MODE_PLT, respectively. 8. The method according to any one of paragraphs. 1-7, wherein when the indication information indicates that the MIP is applied to the luminance samples at the luminance position (cbWidth/2, cbHeight/2), the value of intra_mip_flag[xCb+cb Width/2][yCb+cbHeight/2] is 1. 9. An encoding method implemented by a decoding device or an encoding device, comprising the steps of: obtaining indication information for the position (cbWidth/2, cbHeight/2) of the luminance of the current coding block relative to the top-left position (xCb, yCb) of the luminance samples of the current coding block, where cbWidth represents the width of the current coding block in luma samples, cbHeight represents the height of the current block coding in luma samples; set the value of the intra-frame prediction luma prediction mode associated with the current coding block as the first default value when the value of the indication information indicates that the intra-frame prediction block (IBC) copy mode or palette mode is applied to the luminance samples at position (cbWidth/2, cbHeight /2) luminance relative to the position (xCb, yCb) of the top-left luminance sample of the current coding block; And obtaining an intra-frame chrominance prediction mode value based on the intra-frame luminance prediction mode value of the current coding block. 10. The method of claim 9, wherein the first default value is either a planar mode value or a DC mode value. 11. The method according to claim 9 or 10, wherein the luma component split and the chroma component split of the current coding block do not match each other. 12. The method according to any one of paragraphs. 9-11, wherein the method further comprises: set the luminance intra-frame prediction mode value of the current coding block as the second default value when the matrix-based intra-frame prediction (MIP) mode is applied to the luminance samples at the luminance position (cbWidth/2, cbHeight/2) and the indication information indicates that IBC mode or paletted mode does not apply to the position (cbWidth/2, cbHeight/2) of the brightness. 13. The method of claim 12, wherein the second default value is equal to the planar mode value. 14. The method according to any one of paragraphs. 9-13, in which the absolute position for the brightness position (cbWidth/2, cbHeight/2) is (xCb+cbWidth/2, yCb+cbHeight/2), and the absolute position is (xCb+cbWidth/2, yCb+cbHeight/2 ) defines the position relative to the top left selection of the current image; the position (cbWidth/2, cbHeight/2) of the luminance specifies a position relative to the position (xCb, yCb) of the upper left luminance sample of the current coding block. 15. The method of claim 12 or 13, wherein when the MIP is applied to the position (cbWidth/2, cbHeight/2) of the brightness, the value of intra_mip_flag[xCb+cbWidth/2][yCb+cbHeight/2] is 1. 16. The method according to any one of paragraphs. 9-15, in which, when the IBC mode or palette mode is applied to the position (cbWidth/2, cbHeight/2) of the brightness, CuPredMode[0][xCb+cbWidth/2][yCb+cbHeight/2] is equal to MODE_IBC or MODE_PLT, respectively. 17. A device for obtaining an intra-frame color prediction mode, comprising: one or more processors; And a non-volatile computer-readable storage medium connected to the processors and storing programs for execution by the processors, causing, when executed by the processors, the execution of a device: obtaining first indication information for the position (cbWidth/2, cbHeight/2) of the luminance of the current coding block relative to the top-left position (xCb, yCb) of the luminance samples of the current coding block, where cbWidth represents the width of the current coding block in luminance samples, cbHeight represents the height of the current coding unit in luma samples; setting the value of the intra-frame luminance prediction mode associated with the current coding block as the first default value when the value of the indication information indicates that matrix-based intra-frame prediction (MIP) is applied to the luminance samples at position (cbWidth/2, cbHeight/ 2) luminance relative to the position (xCb, yCb) of the top-left luminance sample of the current coding block; obtaining second indication information for the luminance position (cbWidth/2, cbHeight/2) of the current coding block, when the first indication information indicates that MIP is not applied to the luminance samples at the luminance position (cbWidth/2, cbHeight/2) relative to the top-left position (xCb, yCb) luminance samples of the current coding block; setting the value of the intra-frame prediction luma prediction mode associated with the current coding block as the second default value when the second value of the indication information indicates that the intra-frame prediction block (IBC) copy mode or palette mode is applied to the luminance samples at the position (cbWidth/2, cbHeight/2) luminance relative to the position (xCb, yCb) of the top-left luminance sample of the current coding block; And obtaining an intra-frame chrominance prediction mode value based on the intra-frame luminance prediction mode value of the current coding block. 18. The apparatus of claim 17, wherein the first indication information is intra_mip_flag[xCb+cbWidth/2][yCb+cbHeight/2], wherein when intra_mip_flag[xCb+cbWidth/2][yCb+cbHeight/2] is equal to 1, the first indication information indicates that MIP is applied to the brightness samples. 19. The device according to claim 17 or 18, in which the second information is CuPredMode[0] [xCb+cbWidth/2][yCb+cbHeight/2], while when CuPredMode[0][xCb+cbWidth/2][yCb+cbHeight/2] is MODE_IBC or MODE_PLT, the second indication information indicates that the IBC mode or palette mode is applied to the brightness samples. 20. The device according to any one of paragraphs. 17-19, in which the first default value is the planar mode value and the second default value is the DC mode value. 21. The device according to any one of paragraphs. 17-20, wherein the device is a decoder or encoder. 22. The device according to any one of paragraphs. 17-21, in which the luma component division and the chroma component division of the current coding block do not match. 23. Encoder (20), containing the processing circuit for performing the method according to any one of paragraphs. 1-16. 24. Decoder (30), containing the processing circuit for performing the method according to any one of paragraphs. 1-16. 25. A non-volatile computer-readable storage medium that stores a computer program product containing a control program for performing the method according to any one of paragraphs. 1-16. 26. A device for obtaining an intra-frame color prediction mode, comprising: obtaining block (1001) configured to obtain first indication information for the position (cbWidth/2, cbHeight/2) of the brightness of the current coding block relative to the top-left position (xCb, yCb) of the brightness sample of the current coding block, where cbWidth represents the width of the current block coding in luma samples, cbHeight represents the height of the current coding block in luma samples; a setting unit (1002), configured to set the value of the intra-frame luminance prediction mode associated with the current coding block as a first default value when the first indication information indicates that matrix-based intra-frame prediction (MIP) is applied to the luminance samples at the position (cbWidth/2, cbHeight/2) of the brightness relative to the top-left position (xCb, yCb) of the brightness sample of the current coding block; acquisition unit (1001), further configured to obtain second indication information for the luminance position (cbWidth/2, cbHeight/2) of the current coding block when the first indication information indicates that MIP is not applied to the luminance component at position (cbWidth/2, cbHeight/2) brightness relative to the top-left position (xCb, yCb) of the brightness sample of the current coding block; a setting unit (1002), further configured to set the value of the intra-frame prediction luma prediction mode associated with the current coding block as a second default value when the second indication information indicates that the intra-frame prediction block (IBC) copy mode is applied to the luminance samples, or the palette mode at the position (cbWidth/2, cbHeight/2) of the brightness relative to the top-left position (xCb, yCb) of the brightness sample of the current coding block; And an intra chroma prediction mode block (1003), configured to obtain an intra chroma prediction mode value based on an intra luminance prediction mode value of the current coding block. 27. The apparatus of claim 26, wherein the first indication information is intra_mip_flag[xCb+cbWidth/2][yCb+cbHeight/2], wherein when intra_mip_flag[xCb+cbWidth/2][yCb+cbHeight/2] is equal to 1, the first indication information indicates that MIP is applied to the brightness samples. 28. The device according to claim 26 or 27, in which the second information is CuPredMode[0][xCb+cbWidth/2][yCb+cbHeight/2], while when CuPredMode[0][xCb+cbWidth/2][yCb+cbHeight/2] is MODE_IBC or MODE_PLT, the second indication information indicates that the IBC mode or palette mode is applied to the brightness samples. 29. The device according to any one of paragraphs. 26-28, in which the first default value is the planar mode value and the second default value is the DC mode value. 30. The device according to any one of paragraphs. 26-29, wherein the device is a decoder or encoder.