RU2786022C1 - Device and method for limitations of block level depending on mode and size - Google Patents

Abstract

FIELD: image encoding.

SUBSTANCE: invention relates to means for image encoding. A value of a predicted sample of one or more values of reference samples is determined by means of an internal prediction, using an internal prediction mode selected from a DC internal prediction mode, a planar internal prediction mode, and an angular internal prediction mode. The first weight is determined based on the internal prediction mode and one of a block height, a block width, or both. The second weight is determined based on the internal prediction mode and one of the block height, the block width, or both. An additional value as the weighted sum of a value of an upper reference sample and a value of a left reference sample is calculated by means of weighing of the value of the upper reference sample with the first weight and the value of the left reference sample with the second weight. The value of the predicted sample is multiplied by a weight coefficient of the sample to obtain a weighted value of the predicted sample. The additional value is added to the weighted value of the predicted sample to obtain an unnormalized modified value of the predicted sample.

EFFECT: increase in the efficiency of encoding.

21 cl, 16 dwg, 18 tbl

Description

The technical field to which the invention belongs

Embodiments of the present application (disclosure) generally relate to the field of image processing and, in particular, to intra prediction with a position-dependent prediction combination.

State of the art

Video encoding (video encoding and decoding) is used in a wide range of digital video applications, such as digital television broadcast, video transmission over the Internet and mobile networks, real-time conversational applications such as video chat, video conferencing, DVD and Blu-ray discs. , video content collection and editing systems and security camera applications. The amount of video data required to display even a relatively short video can be significant, which can make it difficult to transfer data over a communication network with limited bandwidth. Thus, in modern communication networks, video data is usually compressed prior to transmission. The size of the video also matters because memory resources may be limited to store the video on the storage device. Video compression devices often use source software and/or hardware to encode video data prior to transmission or storage, thereby reducing the amount of data required to represent digital video images. The compressed data is then received at the destination by a video decompressor, which decodes the video data. With limited network resources and ever-increasing demands for higher video quality, advanced compression and decompression technologies are required that increase the compression ratio with little or no degradation in image quality.

Disclosure of the essence of the invention

In view of the above technical problems, the present disclosure provides a solution to the above technical problems.

Embodiments of the present disclosure provide devices and methods for intra prediction of an image block in accordance with the independent claims.

Solutions to the above and other objectives of the present disclosure are set forth in the independent claims. Additional forms of implementation are obvious from the dependent claims, description and drawings.

This disclosure suggests:

an image block intra-prediction method, contains, for sampling from a plurality of block samples:

determining a predicted sample value from one or more reference sample values by intra prediction using an intra prediction mode selected from one of DC intra prediction mode, planar intra prediction mode, and angular intra prediction mode;

determining a first weight based on the intra prediction mode and one of a block height, a block width, or both;

determining a second weight based on the intra prediction mode and one of the block height, block width, or both;

calculating an additional value as a weighted sum of the upper reference sample value and the left reference sample value by weighting the upper reference sample value with a first weight and the left reference sample value with a second weight;

multiplying the predicted sample value by the sample weight to obtain a weighted predicted sample value;

adding an additional value to the weighted value of the predicted sample to obtain a non-normalized modified value of the predicted sample; and

normalizing the non-normalized modified predicted sample value by arithmetic right shifting an integer representation of the non-normalized modified predicted sample value to obtain a normalized modified predicted sample value.

In particular, herein, the first weight and the second weight may be different.

In an exemplary form of implementing the method according to the foregoing aspect, as such, wherein the method comprises intra-prediction of an image block containing each sample of a plurality of block samples.

In a possible form of implementation of the method according to any previous implementation of the previous aspect or the previous aspect as such, in which

determining the first weight and determining the second weight comprises determining a scaling factor based on the intra prediction mode and one of a block height, a block width, or both; in which the first weight and the second weight are determined based on the scaling factor, respectively.

In a possible form of implementation of the method according to any previous implementation of the previous aspect or the previous aspect as such; wherein

determining the first weight comprises determining a vertical scale factor, denoted nScaleY, based on the block height and an intra prediction mode in which the first weight is determined based on the vertical scale factor, nScaleY;

determining the second weight comprises determining a horizontal scale factor, denoted nScaleX, based on the block width and an intra prediction mode in which the second weight is determined based on the horizontal scale factor, nScaleX, normalizing the denormalized predicted sample value by arithmetic right shifting the integer representation of the denormalized predicted sample value to obtain the normalized value of the predicted sample.

In the present invention, the definition of the first weight contains the definition of the vertical scaling factor, while the definition of the second weight contains the definition of the horizontal scaling factor, in which it should be understood that depending on the type of intra prediction mode, these coefficients may be different.

In an exemplary form of implementing the method according to any prior implementation of the preceding aspect or the preceding aspect as such, wherein nScaleX and nScaleY are derived based on a mode class index, respectively, wherein the mode class index is derived using an intra prediction mode group.

In an exemplary implementation form of the method according to any prior implementation of the preceding aspect, or the preceding aspect as such, wherein a mode class index is mapped to a particular group of intra prediction modes according to one of the following tables:

wherein DC_IDX denotes a DC mode which has an intra prediction mode index of 1, HOR_IDX and VER_IDX denotes horizontal and vertical modes which have intra prediction mode indices of 18 and 50 respectively, VDIA_IDX denotes a diagonal mode which has an intra prediction mode index of 66.

The present invention shows different mappings of intra prediction mode groups to different classes.

In an exemplary form of implementing the method according to any prior implementation of the preceding aspect, or the preceding aspect as such, wherein nScaleX and nScaleY are obtained according to the following mapping:

in which nTbW and nTbH denote the width and height of the predicted block, respectively, and ">>" denotes a right shift operation;

where Floor is a function of rounding down to the nearest integer, Log2 normalizes the integer logarithm to base 2, Min means the smallest of its arguments, and ">>" denotes a right shift operation.

In an exemplary form of implementation of the method according to any prior implementation of the preceding aspect, or the preceding aspect as such, wherein a mode class index is mapped to a particular group of intra prediction modes according to the following table:

In an exemplary form of implementing the method according to any prior implementation of the preceding aspect, or the preceding aspect as such, wherein nScaleX and nScaleY are obtained according to the following mapping:

in which nTbW and nTbH denote the width and height of the predicted block, respectively, and ">>" denotes a right shift operation;

where Floor is a function of rounding down to the nearest integer, Log2 normalizes the integer logarithm to base 2, Min means the smallest of its arguments, and ">>" denotes a right shift operation; where invAngle is the inverse angle parameter which is obtained using intraPredAngle like this:

wherein IntraPredAngle is an angle parameter that is obtained from the intra prediction mode specified by the predModelntra variable using a lookup table, LUT.

In a possible form of implementation of the method according to any prior implementation of the preceding aspect or the preceding aspect as such, in which the LUT is given by the following table

In an exemplary form of implementing the method of any prior implementation of the preceding aspect, or the preceding aspect as such, wherein normalizing the de-normalized modified predicted sample value comprises computing the predicted sample value as

wherein

P(x, y) is the unnormalized modified sample predicted value,

представляют собой значения ближайших опорных выборок, расположенных выше и слева от каждой выборки,

are the values of the closest reference samples located above and to the left of each sample,

wL=V>>((x<<1)>>nScaleX) is the horizontal weighting factor,

wT=V>>((y<<1)>>nScaleY) is the vertical weight,

V is the base value;

x is the horizontal coordinate of each sample, and

y is the vertical sample coordinate;

in which cliplCmp is set like this:

if cIdx is 0, clip1Cmp is set to CliplY;

otherwise clip1Cmp is set equal to Clip1C;

in which cIdx is a variable indicating the color component of the current block; and

where BitDepthy is the bit depth of the luma samples.

where BitDepthc is the bit depth of the chrominance samples.

In a possible form of implementing the method according to the preceding aspect as such, wherein for planar mode wTL=0 while for horizontal mode wTL=wT and for vertical mode wTL=wL.

In an exemplary form of implementing the method of any prior implementation of the preceding aspect, or the preceding aspect as such, wherein normalizing the non-normalized modified predicted sample value comprises computing the normalized modified predicted sample value as

wherein

является нормализованным модифицированным предсказанным значением выборки,

is the normalized modified sample predicted value,

P(x, y) is the unnormalized modified sample predicted value,

представляют значения ближайших опорных выборок, расположенных выше и слева от каждой выборки,

represent the values of the closest reference samples located above and to the left of each sample,

wL=V>>((x<<1)>>nScaleX) is the horizontal weighting factor,

wT=V>>((y<<1)>>nScaleY) is the vertical weight,

x is the horizontal coordinate of each sample, and

y is the vertical coordinate of the sample.

In an exemplary form of implementing the method of any prior implementation of the preceding aspect, or the preceding aspect as such, in which each sample is not updated

if the sum of the block width and block height given in the samples, respectively, exceeds 64 samples, and

either the intra prediction mode is single-ended non-diagonal, or the intra prediction mode is a planar intra prediction mode.

In an exemplary form of implementing the method of any prior implementation of the preceding aspect, or the preceding aspect as such, wherein intraPredMode is non-symmetric non-diagonal if intraPredMode is not equal to 2 or VDIA_IDX and one of the following is true: intraPredMode is not less than VDIA_IDX - numModes or intraPredMode is not greater than 2 +numModes.

In an exemplary form of implementation of the method according to any prior implementation of the preceding aspect, or the preceding aspect as such, the image is part of a video sequence.

In a possible form of implementation of the method according to any previous implementation of the previous aspect or the previous aspect as such, further comprising

adding a residual value to the normalized modified predicted sample value to obtain a reconstructed sample value.

The present disclosure further provides an encoder (20) containing a processing circuit for performing a method according to any prior implementation of any prior aspect, or any prior aspect as such.

The present disclosure further provides a decoder (30) comprising a processing circuit for performing a method according to any prior implementation of any prior aspect, or any prior aspect as such.

The present disclosure further provides a computer program comprising program code for executing a method according to any prior implementation of any prior aspect, or any prior aspect as such.

The present disclosure may further provide a computer program product containing program code for executing a method according to any prior implementation of any prior aspect, or any prior aspect as such.

The present disclosure further provides a decoder comprising:

one or more processors; and

a persistent computer-readable medium coupled to the processors and storing programs for execution by the processors, wherein the programs, when executed by the processors, configure a decoder to execute a method according to any prior implementation of any prior aspect, or any prior aspect as such.

The present disclosure further provides an encoder comprising: one or more processors; and

a persistent computer-readable medium coupled to the processors and storing programs for execution by the processors, wherein the programs, when executed by the processors, configure an encoder to execute a method according to any prior implementation of any prior aspect, or any prior aspect as such.

A detailed description of one or more embodiments is set forth in the accompanying drawings and the description below. Other features, objects and advantages will be apparent from the description, drawings and claims.

Brief description of the drawings

The following is a detailed description of embodiments of the invention with reference to the accompanying drawings, in which:

fig. 1A schematically illustrates an example of a video coding system capable of implementing embodiments of the invention;

fig. 1B schematically illustrates another example of a video coding system capable of implementing embodiments of the invention;

fig. 2 schematically illustrates an example of a video encoder capable of implementing embodiments of the invention;

fig. 3 schematically illustrates an example of the structure of a video decoder capable of implementing embodiments of the invention;

fig. 4 schematically illustrates an example of an encoding device or a decoding device;

fig. 5 schematically illustrates another example of an encoding device or a decoding device;

fig. 6 schematically illustrates intra prediction angular directions and associated intra prediction modes in HEVC;

fig. 7 schematically illustrates intra prediction angular directions and associated intra prediction modes in JEM;

fig. 8 schematically illustrates intra prediction angular directions and associated intra prediction modes in draft v.3 of the VTM-3.0 specification and VVC.

fig. 9 schematically illustrates intra prediction angular directions and associated intra prediction modes in draft v.3 of the VTM-3.0 specification and VVC;

fig. 10 schematically illustrates an example of PDPC mode DC weights for (0, 0) and (1, 0) positions within a 4×4 block;

fig. 11 schematically illustrates an example of intra-prediction of a block from reference samples of the main reference side;

fig. 12 illustrates an example of PDPC mode DC weights for (0, 0) and (1, 0) positions within a 4×4 block;

fig. 13 schematically illustrates an example of a method for determining the vertical and horizontal scaling factor;

fig. 14 schematically illustrates an example of a method for determining the vertical and horizontal scaling factor;

fig. 15 schematically illustrates an alternative example of a method for determining vertical and horizontal scaling factors.

In the following description, identical reference numerals refer to identical or at least functionally equivalent features, unless expressly stated otherwise.

Implementation of the invention

In the following description, reference is made to the accompanying drawings, which form part of the present disclosure and show, by way of illustration, specific aspects of embodiments of the invention or specific aspects in which embodiments of the present disclosure may be used. Obviously, embodiments of the invention can be used in other aspects and contain structural or logical changes not shown in the drawings. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims.

For example, it is obvious that the present disclosure with the described method can also be true for the corresponding device or system configured to perform the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more blocks, such as functional blocks, to perform the one or more method steps described (for example, one block performing one or more steps, or a plurality of blocks , each of which performs one or more of a plurality of steps) even if such one or more blocks are not described and illustrated explicitly in the drawings. On the other hand, for example, if a particular device is described in terms of one or more blocks, such as functional blocks, the corresponding method may include one step to perform the functionality of one or more blocks (for example, one step to perform the functionality of one or more blocks). blocks, or a plurality of steps, each of which performs the functionality of one or more of the plurality of blocks) even if such one or more steps is not explicitly described or illustrated in the drawings. In addition, it is understood that features of the various exemplary embodiments and/or aspects described herein may be combined with one another unless specifically noted otherwise.

Video coding generally refers to the processing of a sequence of images that form a video or video sequence. Instead of the term "image", the term "frame" or "image" can be used as synonyms in the field of video coding. The video coding used in the present invention (or the present disclosure) indicates either video coding or video decoding. Video encoding is performed at the source side, typically involving processing (eg, by compressing) the source video images to reduce the amount of data needed to represent the video images (for more efficient storage and/or transmission). Video decoding is performed on the destination side and usually contains inverse processing compared to the encoder for recovering video images. Embodiments relating to "encoding" video images (or images in general, as will be explained later) should be understood to refer to either "encoding" or "decoding" a video sequence. The combination of an encoding part and a decoding part is also called CODEC (Coding and Decoding).

In the case of lossless video coding, the original video images can be reconstructed, that is, the reconstructed video images have the same quality as the original video images (assuming no transmission loss or other data loss during storage or transmission). In the case of lossy video coding, additional compression is performed, for example by quantization, to reduce the amount of data representing video images that cannot be fully reconstructed in the decoder, i.e., the quality of the reconstructed video images is lower or worse compared to the quality of the original video images.

Several video coding standards belong to the group of "hybrid lossy video codecs" (ie, combine spatial and temporal sample-domain prediction and 2D transform coding to apply transform-domain quantization). Each picture of a video sequence is usually divided into a set of non-overlapping blocks, and encoding is usually done at the block level. In other words, in the encoder, video is usually processed, that is, encoded, at the block (video block) level, for example, using spatial (inner picture) prediction and temporal (outer picture) prediction to generate a prediction block, subtracting the prediction block from the current block ( block currently being processed/to be processed) to obtain a residual block, transforming the residual block, and quantizing the residual block in the transform domain to reduce the amount of data to be transmitted (compressed), while the decoder partially performs inverse processing compared to with the encoder to the encoded or compressed block to reconstruct the current block for presentation. In addition, the encoder duplicates the processing cycle of the decoder so that both will generate identical predictions (eg, intra and inter prediction) and/or re-restore for processing, ie, encoding, subsequent blocks.

In the following embodiments, with reference to FIG. 1-3, the encoder 20, the decoder 30 and the encoding system 10 are described.

Fig. 1A is a block diagram illustrating an example of a coding system 10, such as a video coding system 10 (or coding system 10 for short), that may use the methods of this disclosure. Video encoder 20 (eg, encoder 20) and video decoder 30 (eg, decoder 30) of video coding system 10 are examples of devices that can be configured to perform methods in accordance with the various examples described in the present invention.

As shown in FIG. 1A, encoding system 10 includes a source device 12 configured to provide encoded image data 13 to, for example, destination device 14 for decoding encoded image data 13.

The source device 12 includes an encoder 20 and may additionally, i. e. possibly contain an image source 16, a preprocessor 18 (pre-processor), for example, an image preprocessor 18 and a communication interface or communication unit 22.

Image source 16 may comprise or be any type of image capture device, such as for capturing a real image and/or any kind of image or commentary (for screen encoding, some texts on the screen are also considered part of the image or image to be encoded), device generating, such as a computer graphics processor for generating a computer animated image, or any device for receiving and/or rendering a real image, a computer animated image (e.g., screen content, a virtual reality (VR) image), and/or any combination thereof (e.g., an image augmented reality (AR)). The image source can be any type of memory or storage that stores any of the above images.

Unlike the preprocessor 18 and the processing performed by the preprocessor 18, the image or image data 17 may also be referred to as raw image or original image data 17.

The preprocessor 18 is configured to receive the (raw) image data 17 and perform pre-processing of the image data 17 to obtain a pre-processed image 19 or pre-processed image data 19. Pre-processing is performed by the preprocessor 18, which includes, for example, cropping, color format conversion (eg from RGB to YCbCr), color correction or denoising. It can be understood that the preprocessor 18 may be a possible component.

Video encoder 20 is configured to receive pre-processed image data 19 and provide encoded image data 21 (more details will be described below, for example, with reference to FIG. 2).

Communication interface 22 of source device 12 may be configured to receive encoded image data 21 and transmit encoded image data 21 (or any further processed version thereof) over communication channel 13 to another device, such as destination device 14 or any other device for storage or direct recovery.

Destination device 14 includes a decoder 30 (eg video decoder 30) and may additionally, i. e. possibly contain a communication interface or a communication unit 28, a post-processor 32 (post-processing unit 32) and a display device 34.

The communication interface 28 of the destination device 14 is configured to receive the encoded image data 21 (or any further processed version thereof), for example, directly from the source device 12 or from any other source, for example, a storage device, for example, an encoded image data storage device, and provide encoded picture data 21 to decoder 30.

Communication interface 22 and communication interface 28 may be configured to transmit or receive encoded image data 21 or encoded data 13 via a direct link between source device 12 and destination device 14, such as a direct wired or wireless connection, or over any network, such as wired or wireless network or any combination thereof, or any private and public network or any combination thereof.

The communication interface 22 may, for example, be configured to pack the encoded image data 21 into an appropriate format, such as packets, and/or process the encoded image data using any kind of transmission coding or processing for transmission over a communication channel or communication network.

Communication interface 28 analogous to communication interface 22 may, for example, be configured to receive transmitted data and process transmitted data using any kind of appropriate transmission decoding or processing and/or decompression to obtain encoded image data 21.

Both the communication interface 22 and the communication interface 28 can be configured as unidirectional communication interfaces, as indicated by the arrow for the encoded image data 13 in FIG. 1A pointing from source device 12 to destination device 14 or as bi-directional communication interfaces, and may be configured to, for example, send and receive messages, for example, to establish a connection, acknowledge and exchange any other information related to the communication channel and /or data transmission, for example, transmission of encoded image data.

The decoder 30 is configured to receive encoded image data 21 and provide decoded image data 31 or decoded image 31 (more details will be described below, for example, based on FIG. 3 or FIG. 5).

The post processor 32 of the destination device 14 is configured to post-process the decoded image data 31 (also referred to as reconstructed image data), such as the decoded image 31, to obtain post-processed image data 33, such as the post-processed image 33. The post-processing performed by the post-processing unit 32, may include, for example, color format conversion (eg, from YCbCr to RGB), color correction, cropping or resampling, or any other processing, for example, to prepare the decoded image data 31 for display, for example, by the display device 34.

The display device 34 of the destination device 14 is configured to receive the image data 33 after being processed to display the image to, for example, a user or a viewer. The display device 34 may be or comprise any kind of display for presenting the reconstructed image, such as a built-in or external display or monitor. The displays may, for example, consist of liquid crystal displays (LCD), organic light emitting diode (OLED) displays, plasma displays, projectors, micro LED displays, liquid crystals on silicon (LCoS), digital light processor (DLP) or any other display.

Although FIG. 1A depicts source device 12 and destination device 14 as separate devices, device embodiments may also comprise both or both source device 12 functionality or corresponding functionality and destination device 14 or corresponding functionality. In such embodiments, source device 12 or related functionality and destination device 14 or related functionality may be implemented using the same hardware and/or software, or using separate hardware and/or software, or any combination thereof.

As will be apparent to those skilled in the art based on the description, the presence and (precise) separation of the functionality of the various blocks or functions in source device 12 and/or destination device 14, as shown in FIG. 1A may vary depending on the actual device and implementation.

Encoder 20 (eg, video encoder 20) and decoder 30 (eg, video decoder 30) may each be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application programs, application specific integrated circuits ( ASICs), Field Programmable Gate Arrays (FPGAs), Discrete Logic, Hardware, or any combination thereof. Encoder 20 may be implemented by processing circuitry 46 to implement various modules, as discussed with respect to encoder 20 in FIG. 2 and/or any other coding system or subsystem described herein. Decoder 30 may be implemented via processing circuitry 46 to implement various modules, as discussed with respect to decoder 30 in FIG. 3 and/or any other decoder system or subsystem described herein. The processing circuit may be configured to perform various operations as described below. As shown in FIG. 5, if the methods are partially implemented in software, the device may store instructions for the software on a suitable persistent computer-readable storage medium and may execute instructions in hardware using one or more processors to execute the methods of this disclosure. Either of video encoder 20 and video decoder 30 may be integrated as part of a combined encoder/decoder (CODEC) in a single device, such as shown in FIG. 1B.

Source device 12 and destination device 14 may comprise any of a wide range of devices, including any kind of portable or stationary devices, such as laptops or laptops, mobile phones, smartphones, tablets or tablet computers, cameras, desktop computers, set-top boxes, televisions, display devices, digital media players, game consoles, video streaming devices (eg, content service servers or content delivery servers), broadcast receiver, broadcast transmitter, and the like. and may or may not use any type of operating system. In some cases, source device 12 and destination device 14 may be equipped for wireless communication. Thus, source device 12 and destination device 14 may be wireless communication devices.

In some cases, the video coding system 10 illustrated in FIG. 1A is merely an example, and the methods of the present disclosure may apply to video encoding settings (eg, video encoding or video decoding) that do not necessarily involve any communication between encoding and decoding devices. In other examples, data is retrieved from local memory, transferred over a network, and the like. The video encoding device may encode and store data in memory, and/or the video decoding device may retrieve and decode data from memory. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data in memory and/or retrieve and decode data from memory.

For convenience of description, embodiments of the invention are described herein, for example, with reference to High Efficiency Video Coding (HEVC) or Universal Video Coding (VVC) reference software, the next generation video coding standard developed by the Collaborative Video Coding Group (JCT-VC). ) the ITU-T Video Coding Expert Group (VCEG) and the ISO/IEC Moving Picture Expert Group (MPEG). For a person skilled in the art it is obvious that the embodiments of the invention are not limited to HEVC or VVC.

Encoder and encoding method

Fig. 2 shows a block diagram of an exemplary video encoder 20 that is configured to implement the methods of the present disclosure. In the example in FIG. 2, video encoder 20 comprises an input 201 (or input interface 201), a residual calculation block 204, a transform processing block 206, a quantization block 208, an inverse quantization block 210, and an inverse transform processing block 212, a reconstruction block 214, a loop filter block 220, a buffer 230 decoded pictures (DPB), mode selection block 260, entropy encoding block 270, and output block 272 (or output interface 272). The mode selector 260 may include an inter-predictor 244, an intra-predictor 254, and a splitter 262. Inter prediction block 244 may include a motion estimator and a motion compensation block (not shown). Video encoder 20 shown in FIG. 2 may also be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec.

The residual calculator 204, the transform processing block 206, the quantizer 208, the mode selector 260 form the direct signal path of the encoder 20, while, for example, the inverse quantizer 210, the inverse transform processor 212, the reconstruction block 214, the buffer 216, the loop filter 220, a decoded picture buffer (DPB) 230, inter predictor 244, and intra prediction 254 form an encoder signal return path, in which the encoder return signal path corresponds to the decoder signal path (see decoder 30 in FIG. 3). An inverse quantization block 210, an inverse transform processing block 212, a reconstruction block 214, a loop filter 220, a decoded picture buffer (DPB) 230, an inter prediction block 244, and an intra prediction block 254 are also related to the formation of the “embedded decoder” of the video encoder 20.

Images and Image Sharing (Images and Blocks)

Encoder 20 is configured to receive, for example, via input 201 an image 17 (or image data 17), for example an image of a sequence of images constituting a video or video sequence. The received image or image data may also be a pre-processed image 19 (or pre-processed image data 19). For ease of description, hereinafter referred to as picture 17. Picture 17 is also referred to as the current picture or the picture to be encoded (in particular, when encoding video to distinguish the current picture from other pictures, for example, previously encoded and/or decoded pictures of one and the same video sequence, that is, a video sequence that also contains the current image).

A (digital) image is, or can be viewed as, a two-dimensional array or matrix of samples with intensity values. A sample in an array can also be called a pixel (short for picture element) or a pixel. The number of samples in the horizontal and vertical direction (or axis) of the array or image determines the size and/or resolution of the image. Three color components are typically used to represent a color, that is, an image can be represented or contain three arrays of samples. In RBG format or color space, an image contains a corresponding array of red, green, and blue color samples. However, in video encoding, each pixel is usually represented in a luma and chrominance format or color space, such as YCbCr, which contains a luma component labeled Y (sometimes L is also used instead) and two chrominance components labeled Cb and Cr. The Y component of luma (or briefly, luminance) represents the brightness or intensity of the gray level (eg, as in a grayscale image), while the two chrominance (or briefly, chrominance) components Cb and Cr represent chroma components or color information. Accordingly, an image in YCbCr format contains an array of luma samples of luminance sample values (Y) and two chrominance sample arrays of chrominance values (Cb and Cr). RGB images can be converted to YCbCr format and vice versa, a process also known as color conversion. If the image is monochrome, it can only contain an array of brightness samples. Accordingly, the image may be, for example, an array of luminance samples in monochrome format, or an array of luminance samples and two corresponding arrays of chrominance samples in 4:2:0, 4:2:2 and 4:4:4 color format.

Embodiments of encoder 20 may include a splitter (not shown in FIG. 2) configured to split image 17 into multiple (generally non-overlapping) image blocks 203. These blocks may also be referred to as root blocks, macroblocks (H.264/AVC) or coding tree units (CTBs) or coding tree units (CTUs) (H.265/HEVC and VVC). The partition block may be configured to use the same block size for all images of the video sequence and the corresponding grid defining the block size, or to change the block size between images, subsets or groups of images and divide each image into appropriate blocks.

In additional embodiments, the video encoder may be configured to directly receive block 203 of image 17, such as one, several or all of the blocks that make up image 17. Image block 203 may also be referred to as the current image block or the image block to be encoded.

Like image 201, block 203 is again, or can be viewed as, a two-dimensional array or matrix of samples with intensity values (sample values), albeit smaller than image 17. In other words, block 203 may contain, for example, one array of samples (for example, a luminance array in the case of a monochrome image 17 or a luminance or chrominance array in the case of a color image) or three arrays of samples (for example, a luminance array and two chrominance arrays in the case of a color image 17) or any other number and/or types of arrays depending on the applied color format. The number of samples in the horizontal and vertical direction (or axis) of block 203 determines the size of block 203. Accordingly, the block may, for example, be an array of MxN samples (M - column by N - row) or an MxN array of transform coefficients.

Embodiments of video encoder 20, as shown in FIG. 2 may be further configured to encode image 17 block by block, such as splitting and encoding block 203.

Remainder Calculation

Residual block 204 is configured to calculate residual block 205 (also referred to as residual 205) based on image block 203 and prediction block 265 (more details about predictor 265 are provided later), for example, by subtracting the sample values of predictor 265 from the sample values. image block 203, sample by sample (pixel by pixel) to obtain a residual block 205 in the sample area.

transformation

Transform processing unit 206 is configured to apply transforms, such as discrete cosine transform (DCT) or discrete sine transform (DST), on sample values of residual block 205 to obtain transform coefficients 207 in the transform domain. The transform coefficients 207 may also be referred to as residual transform coefficients and represent the residual block 205 in the transform region.

The transform processing unit 206 may be configured to apply DCT/DST integer approximations such as the transforms specified for HEVC/H.265. Compared to the orthogonal DCT, such integer approximations are usually scaled by a certain factor. To preserve the norm of the residual block, which is processed by direct and inverse transformations, additional scaling factors are applied during the transformation. The scaling factors are usually chosen based on certain constraints, such as the scaling factors being a power of two for the shift operation, the bit width of the transform coefficients, the trade-off between precision and implementation cost, and so on. Specific scaling factors are, for example, specified for the inverse transform, for example, by the inverse transform processing unit 212 (and the corresponding inverse transform, for example, by the inverse transform processing unit 312 in the decoder 30), and the corresponding scaling factors for the forward transform, for example, by the transform processing unit 206 in encoder 20, respectively.

Embodiments of video encoder 20 (respectively, transform processing unit 206) may be configured to output transform parameters, such as the type of transform or transform, such as directly or encoded or compressed by entropy encoding unit 270, such that, for example, video decoder 30 may receive and use conversion parameters for decoding.

Quantization

Quantizer 208 is configured to quantize transform coefficients 207 to obtain quantized transform coefficients 209, for example by applying scalar quantization or vector quantization. The quantized transform coefficients 209 may also be referred to as quantized residual coefficients 209.

The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207 . For example, an n-bit transform factor may be rounded down to an m-bit transform factor during quantization, where n is greater than m. The degree of quantization can be changed by adjusting the quantization parameter (QP). For example, for scalar quantization, different scaling may be applied to achieve finer or coarser quantization. Smaller quantization step sizes correspond to finer quantization, while larger quantization step sizes correspond to coarser quantization. The applicable quantization step size may be indicated by a quantization parameter (QP). The quantization parameter may, for example, be an index to a predetermined set of applicable quantization step sizes. For example, small quantization parameters may correspond to fine quantization (small quantization step sizes), and large quantization parameters may correspond to coarse quantization (large quantization step sizes), or vice versa. Quantization may include division by the quantization step size and corresponding or inverse dequantization, for example by inverse quantizer 210, may include multiplication by the quantization step size. Embodiments according to some standards, such as HEVC, may be configured to use a quantization parameter to determine the quantization step size. Typically, the quantization step size can be calculated based on the quantization parameter using a fixed-point approximation of an equation involving division. Additional scaling factors may be introduced for quantization and dequantization to recover the residual block norm, which may be modified due to the scaling used in the fixed-point approximation of the equation for the quantization step size and quantization parameter. In one exemplary implementation, inverse transform scaling and dequantization may be combined. Alternatively, customized quantization tables can be used and signaled from the encoder to the decoder, for example, in a bitstream. Quantization is a lossy operation in which the loss increases with increasing quantization step sizes.

Embodiments of video encoder 20 (respectively, quantizer 208) may be configured to output quantization parameters (QPs), for example, directly or encoded by entropy coding unit 270, such that, for example, video decoder 30 may receive and apply quantization parameters for decoding.

Inverse quantization

Inverse quantizer 210 is configured to apply the inverse quantization of quantizer 208 to the quantized coefficients to obtain dequantized coefficients 211, for example, by applying the inverse quantization scheme applied by quantizer 208 based on or using the same quantization step size as block 208 quantization. The dequantized coefficients 211 may also be referred to as the dequantized residual coefficients 211 and correspond, although not usually identical to the transform coefficients due to quantization loss, to the transform coefficients 207.

Reverse transformation

The inverse transform processing unit 212 is configured to apply an inverse transform applied by the transform processing unit 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST) to obtain a reconstructed residual block 213 (or corresponding dequantized coefficients 213) in the region samples. The recovered residual block 213 may also be referred to as a transformation block 213 .

Recovery

Restoration block 214 (e.g., adder or adder 214) is configured to add transform block 213 (i.e., reconstructed residual block 213) to prediction block 265 to obtain reconstructed block 215 in the sample area, for example, by addition, sample by sample, the sample values of the reconstructed residual block 213 and the sample values of the prediction block 265 .

Filtration

The loop filter block 220 (or "loop filter" 220 for short) is configured to filter the reconstructed block 215 to obtain a filtered block 221 or, typically, to filter the reconstructed samples to obtain filtered samples. The edge filter block is designed, for example, to smooth out pixel transitions or otherwise improve video quality. The loop filter block 220 may comprise one or more loop filters, such as a deblocking filter, an adaptive shift (SAO) filter, or other filters, such as a two-sided filter or an adaptive loop filter (ALF), or sharpening or smoothing filters, or collaborative filters. or any combination of them. Although the loop filter unit 220 is shown in FIG. 2 as a loop filter, in other configurations, the loop filter unit 220 may be implemented as a post loop filter. The filtered block 221 may also be referred to as the filtered reconstructed block 221.

Embodiments of video encoder 20 (respectively, loop filter unit 220) may be configured to output loop filter parameters (such as adaptive sample offset information), for example, directly or encoded by entropy encoding unit 270, such that, for example, decoder 30 may receive and apply the same loop filter parameters or corresponding loop filters for decoding.

Embodiments of video encoder 20 (respectively, loop filter block 220) may be configured to output loop filter parameters (such as adaptive sample offset information), for example, encoded directly or by entropy encoding block 270 or any other entropy encoding block, such that, for example, decoder 30 may receive and apply the same loop filter parameters for decoding.

Decoded picture buffer

Decoded picture buffer (DPB) 230 may be a reference picture memory that stores reference picture data for use in encoding video data by video encoder 20. DPB 230 may be formed by any of a variety of memories, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of storage devices. The decoded picture buffer (DPB) 230 is configured to store the filtered block 221. The decoded picture buffer 230 can be further configured to store other previously filtered blocks, such as previously reconstructed and filtered blocks 221 of the same current picture or different pictures, such as , previously reconstructed images, and may provide complete previously reconstructed, i.e., decoded, images (and corresponding reference blocks and samples) and/or partially reconstructed current image (and corresponding reference blocks and samples), for example, for inter prediction. The decoded picture buffer (DPB) 230 is also configured to store one or more unfiltered reconstructed blocks 215, or more generally unfiltered reconstructed samples, for example, if the reconstructed block 215 is not filtered by the loop filter block 220 or any other further processed version of the reconstructed blocks or samples. .

Mode selection (separation and prediction)

The mode selection section 260 comprises a splitter 262, an inter prediction section 244, and an intra prediction section 254, and is configured to receive or obtain original image data, such as the original block 203 (the current block 203 of the current image 17) and reconstructed image data, such as filtered and/or unfiltered reconstructed samples or blocks of the same (current) picture and/or from one or more previously decoded pictures, eg decoded picture buffer 230 or other buffers (eg, linear buffer, not shown). The reconstructed image data is used as reference image data for prediction, such as inter prediction or intra prediction, to obtain prediction block 265 or predictor 265.

The mode selector 260 may be configured to determine or select a split for the current block prediction mode (including no split) and the prediction mode (e.g., intra or inter prediction mode) and generate a corresponding prediction block 265 that is used to calculate the residual block. 205 and to restore the restored block 215.

Embodiments of mode selector 260 may be configured to select a split and prediction mode (e.g., from those supported or available to mode selector 260) that provides the best match, or in other words, minimum residual (minimum residual means better compression). for transmission or storage), or minimal signaling overhead (minimum signaling overhead means better compression for transmission or storage), or which takes into account or balances both. The mode selector 260 may be configured to determine the division and prediction mode based on rate distortion optimization (RDO), that is, select the prediction mode that provides the minimum rate distortion. Terms such as "best", "minimum", "optimum", etc. in this context does not necessarily refer to the general "best", "minimum", "optimal", etc., but may also refer to termination performance or selection criteria such as a value exceeding or falling below a threshold, or other restrictions , potentially leading to a "non-optimal choice" but reducing complexity and processing time.

In other words, partitioner 262 may be configured to partition block 203 into smaller blocks or subblocks (which again form blocks), for example, iteratively using quadtree partitioning (QT), binary partitioning (BT), or ternary tree (TT), or any combination thereof, and to perform, for example, prediction for each of the partitions of blocks or subblocks, wherein the mode selection comprises selecting a tree structure of the partitioned block 203 and the prediction modes are applied to each of the partitions or subblocks.

The following explains in more detail the splitting process (for example, splitter 260) and prediction processing (inter predictor 244 and intra predictor 254) performed by exemplary video encoder 20.

Separation

Partitioner 262 may divide (or split) the current block 203 into smaller sections, such as smaller square or rectangular blocks. These smaller blocks (which may also be referred to as sub-blocks) can be further subdivided into even smaller sections. This is also called tree splitting or hierarchical tree splitting, in which a root block, e.g. at root level 0 of the tree (hierarchy level 0, depth 0) can be recursively split, e.g. split into two or more next lower tree level boxes, e.g. , nodes at level 1 of the tree (hierarchy level 1, depth 1) in which these blocks can be split again into two or more next lower level blocks, e.g. tree level 2 (hierarchy level 2, depth 2), etc. . until the split is complete, for example, when a termination criterion is met, such as the maximum tree depth or the minimum block size is reached. Blocks that are not split further are also called leaf blocks or leaf nodes of the tree. A tree using two partitions is called a binary tree (BT), a tree using three partitions is called a ternary tree (TT), and a tree using four partitions is called a quadtree (QT).

As mentioned earlier, the term "block" as used herein may be part, specifically, part of a square or rectangular shape of an image. With reference to, for example, HEVC and VVC, a block may either correspond to a coding tree block (CTU), a coding unit (CU), a prediction block (PU), and a transform unit (TU), and/or correspond to blocks, for example, a coding tree block (CTB), a coding block (CB), a transformation block (TV), or a prediction block (PB).

For example, a coding tree unit (CTU) can be either a luma sample CTB, two corresponding chrominance sample CTBs of an image that has three sample arrays, or a monochrome image CTB or an image that is encoded using three separate color planes and the syntax structures used for coding samples. Accordingly, a coding tree block (CTB) may be a block of NxN samples for some value of N, such that splitting a component into CTBs is a split. A coding unit (CU) may be or comprise a luminance sample coding unit, two corresponding chrominance sample coding units of an image that has three sample arrays, or a monochrome image sample coding unit or an image that is encoded using three separate color planes and the syntax structures used for coding samples. Accordingly, a coding block (CB) may be a block of MxN samples for some values of M and N, so that the division of the CTB into coding blocks is a division.

In embodiments, for example in accordance with HEVC, a coding tree unit (CTU) may be partitioned into CUs using a square tree structure referred to as a coding tree. The decision as to whether to encode an image region using extrinsic (temporal) or intrinsic (spatial) prediction is made at the CU layer. Each CU may be further partitioned into one, two or four PUs according to the partition type of the PU. Within one PU, the same prediction process is applied, and the corresponding information is transmitted to the decoder based on the PU. After obtaining the residual block by applying a prediction process based on the division type of the PU, the CU may be divided into transform units (TUs) according to a different quadtree structure similar to the coding tree for the CU.

In embodiments, for example, in accordance with the latest video coding standard currently being developed, called universal video coding (VVC), square tree and binary tree (QTBT) splitting using coding block splitting. In the QTBT block structure, the CU can be either square or rectangular. For example, a coding tree unit (CTU) is first divided by a quadtree structure. The leaf nodes of a quadtree can be further subdivided using a binary tree or ternary (or ternary) tree structure. The leaf nodes of the quadtree are called coding units (CUs) and segmentation is used for prediction and transformation without any additional partitioning. This means that in most cases CU, PU and TU have the same block size in the QTBT coding block structure. In parallel, many partitions have also been proposed, for example, ternary tree partitioning has been used in conjunction with the QTBT block structure.

In one example, mode selector 260 of video encoder 20 may be configured to perform any combination of the separation methods described herein.

As described above, encoder 20 is configured to determine or select the best or optimal prediction mode from a set of (predefined) prediction modes. The set of prediction modes may include, for example, intra prediction modes and/or inter prediction modes.

Internal prediction

The intra prediction mode set may comprise 35 different intra prediction modes, such as non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in HEVC, or may comprise 67 different intra prediction modes such as , non-directional modes such as DC (or medium) mode and planar mode, or directional modes such as those defined for VVC.

VTM (Multipurpose Test Model) version 3.0 uses 93 intra-prediction modes and several internal dithering tools, including four-tap sub-pixel internal interpolation filtering and position-dependent intra prediction combination (PDPC). PDPC has been proposed as a unified mechanism for modifying predicted samples resulting from intra prediction using DC, PLANAR or intra angle modes.

The intra prediction block 254 is configured to use the reconstructed samples of adjacent blocks of the same current picture to generate the intra prediction block 265 according to the intra prediction mode of the intra prediction mode set.

Block 254 intra prediction (or, in General, block 260 mode selection) is further configured to output parameters of intra prediction (or, in General, information indicating the selected intra prediction mode for the block) in block 270 entropy encoding in the form of elements 266 syntax for including an encoded image in data 21 so that, for example, video decoder 30 can receive and use prediction parameters for decoding.

external prediction

The set (or possible) inter prediction modes depends on the available reference pictures (i.e., previous at least partially decoded pictures, e.g. stored in DBP 230) and other inter prediction parameters, e.g. , for example, an area search box around the area of the current block, a reference frame is used to find the best match of the reference block and/or, for example, whether pixel interpolation is applied, such as half/half pixel, quarter pixel and/or 1/16 pixel interpolation or not.

In addition to the aforementioned prediction modes, a skip mode and/or a direct mode may be applied.

Inter prediction block 244 may include a motion estimator (ME) and a motion compensation (MC) block (both not shown in FIG. 2). The motion estimator is configured to receive or obtain an image block 203 (the current image block 203 of the current image 201) and a decoded image 231 or at least one or a plurality of previously reconstructed blocks, e.g., reconstructed blocks of one or a plurality of other/different previously decoded 231 images for motion estimation. For example, the video sequence may comprise the current picture and previously decoded pictures 231 or, in other words, the current picture and previously decoded pictures 231 may be part of or form a sequence of pictures that make up the video sequence.

The encoder 20 may, for example, be configured to select a reference block from a plurality of reference blocks of the same or different pictures from a plurality of other pictures and provide a reference picture (or reference picture index) and/or a shift (spatial shift) between position (x, y coordinates ) of the reference block and the position of the current block as inter-prediction parameters to the motion estimation block. This shift is also called motion vector (MV).

The motion compensation block is configured to obtain, for example, receive an inter prediction parameter, and perform inter prediction based on or using the inter prediction parameter to obtain inter prediction block 265. The motion compensation performed by the motion compensator may comprise sampling or generating a prediction block based on the motion vector/block determined by the motion estimate, possibly performing interpolation to sub-pixel precision. Interpolation filtering can generate additional pixel samples from known pixel samples, thus potentially increasing the number of prediction block candidates that can be used to encode an image block. After receiving the motion vector for the PU of the current picture block, the motion compensator may locate the prediction block pointed to by the motion vector in one of the reference picture lists.

The motion compensation block may also generate syntax elements associated with blocks and a video segment for use by video decoder 30 when decoding image blocks of a video segment.

Entropy coding

The entropy encoding unit 270 is configured to apply an entropy encoding algorithm or schemes (e.g., variable length coding (VLC) schemes, context adaptive VLC scheme (CALVC), arithmetic coding scheme, context adaptive binary arithmetic coding (CABAC) based on context adaptive binary arithmetic coding (SBAC), probability interval entropy coding (PIPE), or other entropy coding technologies or methods) or bypass scheme (no compression) on quantized residual coefficients 209, inter prediction parameters, intra prediction parameters, and/or loop filter parameters and/or other syntax elements for obtaining encoded image data 21 that can be output by output 272, for example in the form of encoded bitstream 21, so that, for example, decoder 30 can receive and use parameters for decoding. The encoded bitstream 21 may be transmitted to video decoder 30 or stored in memory for later transmission or retrieval by video decoder 30.

Other structural variations of video encoder 20 may be used to encode the video stream. For example, non-transform-based encoder 20 may quantize the residual signal directly without transform processor 206 for specific blocks or frames. In another implementation, encoder 20 may have quantizer 208 and inverse quantizer 210 combined into a single block.

Decoder and decoding method

In FIG. 3 shows an example of a video decoder 30 that is configured to implement the methods of the present disclosure. Video decoder 30 is configured to receive encoded picture data 21 (e.g., encoded bitstream 21), such as encoded by encoder 20, to obtain a decoded picture 331. The encoded picture data or bitstream contains information for decoding encoded picture data, such as data that are image blocks of the encoded video segment and associated syntax elements.

In the example in FIG. 3, the decoder 30 includes an entropy decoding block 304, an inverse quantization block 310, an inverse transform processing block 312, a reconstruction block 314 (e.g., adder 314), a loop filter 320, a decoded picture (DBP) buffer 330, an inter prediction block 344, and a block 354 internal prediction. The inter prediction block may be or include a motion compensation block. Video decoder 30 may, in some examples, perform a decoding process that is generally the reverse of the encoding step described with respect to video encoder 100 in FIG. 2.

As explained with respect to the encoder 20, the inverse quantization block 210, the inverse transform processing block 212, the reconstruction block 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter prediction block 344, and the intra prediction block 354 are also called "inline decoder" generation. of the video encoder 20. Accordingly, the inverse quantizer 310 may be identical in function to the inverse quantizer 110, the inverse transform processor 312 may be identical in function to the inverse transform processor 212, the restorer 314 may be identical in function to the restorer 214, the loop filter 320 may be identical in function to the loop filter 220 and the buffer 330 of the decoded image may be identical in function to the buffer 230 of the decoded image. Therefore, the explanations provided for the respective blocks and functions of the video encoder 20 apply to the respective blocks and functions of the video decoder 30, respectively.

Entropy decoding

The entropy decoding unit 304 is configured to parse the bitstream 21 (or, more generally, the encoded image data 21) and perform, for example, entropy decoding of the encoded image data 21 to obtain, for example, quantized coefficients 309 and/or decoded coding parameters ( not shown in Figure 3), such as any or all inter prediction parameters (eg, reference picture index and motion vector), intra prediction parameters (eg, intra prediction mode or index), transform parameters, quantization parameters, loop filter parameters, and /or other syntax elements. Entropy decoding unit 304 may be configured to apply decoding algorithms or schemes corresponding to coding schemes as described with respect to entropy encoding unit 270 of encoder 20. Entropy decoding unit 304 may be further configured to provide inter prediction parameters, an intra prediction parameter, and/ or other syntax elements to mode select block 360 and other parameters to other blocks of decoder 30. Video decoder 30 may receive syntax elements at the video segment level and/or video block level.

Inverse quantization

The inverse quantizer 310 may be configured to receive quantization parameters (QPs) (or inverse quantization related information in general) and quantized coefficients from the encoded picture data 21 (e.g., by parsing and/or decoding, e.g., by entropy decoding block 304) and apply, based on the quantization parameters, inverse quantization to the decoded quantized coefficients 309 to obtain dequantized coefficients 311, which may also be referred to as transform coefficients 311. The inverse quantization process may include using a quantization parameter determined by video encoder 20 for each video block in a video segment to determine the degree of quantization, and likewise the degree of inverse quantization. Reverse transformation

The inverse transform processor 312 may be configured to receive the dequantized coefficients 311, also referred to as transform coefficients 311, and apply transforms to the dequantized coefficients 311 to obtain reconstructed residual blocks 213 in the sample area. The restored residual blocks 213 may also be referred to as transform blocks 313 . The transformation may be an inverse transformation, eg, inverse DCT, inverse DST, inverse integer transformation, or a conceptually similar inverse transformation process. The inverse transform processing unit 312 may be further configured to receive transform parameters or corresponding information from the encoded image data 21 (e.g., by parsing and/or decoding, e.g., by entropy decoding block 304) to determine the transform to be applied to the dequantized coefficients 311.

Recovery

A reconstructor 314 (e.g., adder 314) is configured to add the reconstructed residual block 313 to the prediction block 365 to obtain a reconstructed block 315 in the sample area, for example, by adding the sample values of the reconstructed residual block 313 and the sample values of the prediction block 365.

Filtration

The loop filter block 320 (either in the encoding loop or after the encoding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321, for example, to smooth pixel transitions or otherwise improve video quality. The loop filter block 320 may comprise one or more loop filters, such as a deblocking filter, an adaptive shift filter (SAO), or one or more other filters, such as a bidirectional filter, an adaptive loop filter (ALF), an anti-aliasing filter, a sharpening filter, or collaborative filters or any combination thereof. Although in FIG. 3 shows the loop filter block 320 as a loop filter, in other configurations the loop filter block 320 may be implemented as a post loop filter.

Decoded image buffer

The decoded video blocks 321 in that frame or picture are then stored in the decoded picture buffer 330, which stores the decoded pictures 331 as reference pictures, used for subsequent motion compensation for other pictures and/or outputting the corresponding picture for display.

The decoder 30 is configured to output the decoded image 331, for example, through the output 332 for presentation or viewing to the user.

Prediction

Inter prediction block 344 may be identical to inter prediction block 244 (in particular, a motion compensation block) and intra prediction block 354 may be identical to inter function prediction block 254 and performs split or split decisions and predictions based on split and/or prediction parameters. or corresponding information received from the encoded image data 21 (eg, by parsing and/or decoding, eg, by entropy decoding block 304). The mode selector 360 may be configured to perform prediction (intra or inter prediction) for each block based on the reconstructed images, blocks, or corresponding samples (filtered or unfiltered) to obtain prediction block 365.

When a video segment is encoded as an intra-coded (I) segment, the intra prediction block 354 of the mode selector 360 is configured to generate a prediction block 365 for a picture block of the current video segment based on the signaled intra prediction mode and data from previously decoded blocks of the current picture. When a video image is encoded as an externally coded (i.e., B or P) segment, the inter prediction block 344 (e.g., motion compensation block) of the mode select block 360 is configured to generate prediction blocks 365 for the video block of the current video segment based on motion vectors and others. syntax elements received from block 304 entropy decoding. For inter prediction, prediction blocks may be generated from one of the reference pictures in one of the reference picture lists. Video decoder 30 may construct reference frame lists, list 0 and list 1 using default construction methods based on reference pictures stored in DPB 330.

The mode selector 360 is configured to determine prediction information for the video block of the current video segment by analyzing motion vectors or other syntax elements, and uses the prediction information to generate prediction blocks for the current video block being decoded. For example, the mode selector 360 uses some of the received syntax elements to determine the prediction mode (eg, intra or inter prediction) used to encode the video blocks of a video segment, the inter prediction segment type (eg, B-segment, P-segment, or GPB- segment), construction information for one or more reference picture lists for the segment, motion vectors for each externally coded video block of the segment, inter prediction status for each externally coded video block of the segment, and other information for decoding video blocks in the current video segment.

Other variants of the video decoder 30 may be used to decode the encoded image data 21. For example, the decoder 30 may generate an output video stream without the loop filter block 320 . For example, the non-transform-based decoder 30 can perform the inverse transform of the residual signal quantization directly without the inverse transform processing unit 312 for certain blocks or frames. In another implementation, video decoder 30 may have an inverse quantizer 310 and an inverse transform processor 312 combined into one block.

It should be understood that in encoder 20 and decoder 30, the processing result of the current step may be further processed and then output to the next step. For example, after the interpolation filtering, motion vector output, or loop filtering, an additional operation such as clipping or shifting may be performed on the result of the interpolation filtering, motion vector output, or loop filtering processing.

It should be noted that additional operations can be applied to the received motion vectors of the current block (including, but not limited to, affine mode control point motion vectors, sub-block motion vectors of affine, planar, ATMVP modes, temporal motion vectors, etc.). . For example, the motion vector value is limited to a given range according to its representing bit. If the representing bit of the motion vector is bitDepth, then the range is -2 ^∧ (bitDepth-1)~2 ^∧ (bitDepth-1)-1, in which "∧" means exponentiation. For example, if bitDepth is set to 16, then the range is -32768~32767; if bitDepth is set to 18, then the range is -131072 ~ 131071. For example, the value of the received motion vector (for example, MVs of four 4×4 subblocks within one 8×8 block) is limited such that the maximum difference between the integer parts of four 4× 4 subblock MVs does not exceed N pixels, for example, not more than 1 pixel. The present disclosure proposes two methods for limiting the motion vector according to bitDepth.

Method 1: Remove MSB (Most Significant Bit) Overflow Through Expiration Operations

in which mvx is the horizontal component of the motion vector of the image block or sub-block, mvy is the vertical component of the motion vector of the image block or sub-block, and ux and uy indicate an intermediate value;

For example, if the mvx value is -32769, after formula (1) and (2) is applied, the resulting value is 32767. In a computer system, decimal numbers are stored as a binary component. The binary component of -32769 is 1.0111.1111.1111.1111 (17 bits) and the MSB is discarded, so the resulting binary component is 0111.1111.1111.1111 (decimal 32767), which is the same as the output value after applying formulas (1) and (2).

Operations can be applied during summation of mvp and mvp as shown in formula (5) to (8).

Method 2: Remove MSB overflow via value constraint

in which vx is the horizontal component of the motion vector of the image block or sub-block, vy is the vertical component of the motion vector of the image block or sub-block; x, y and z correspond to the three input values of the clipping process MV and the definition of the Clip3 function is as follows:

Fig. 4 is a diagram of a video encoding apparatus 400 according to an embodiment of the present disclosure. The video encoding device 400 is suitable for implementing the disclosed embodiments as described herein. In an embodiment, video encoding device 400 may be a decoder, such as video decoder 30 of FIG. 1A or an encoder such as video encoder 20 of FIG. 1A.

The video encoding device 400 includes input ports 410 and receiver (Rx) units 420 for receiving data; a processor, logical unit, or central processing unit (CPU) 430 for processing data; transmitter blocks 440 (Tx) and output ports 450 for data transmission; and a memory 460 for storing data. The video encoder 400 may also include optical-electrical (OE) components and electrical-optical (EO) components connected to input ports 410, receiver units 420, transmitter units 440, and output ports 450 for outputting or inputting optical or electrical signals.

Processor 430 is implemented in hardware and software. Processor 430 may be implemented as one or more CPU chips, cores (eg, as a multi-core processor), FPGAs, ASICs, and DSPs. Processor 430 is connected to input ports 410, receiver units 420, transmitter units 440, output ports 450, and memory 460. Processor 430 includes an encoder 470. Encoding module 470 implements the disclosed embodiments described above. For example, encoding module 470 implements, processes, prepares, or provides various encoding operations. Thus, the use of the encoding module 470 provides a significant improvement in the functionality of the video encoding device 400 and causes the video encoding device 400 to be converted to another state. Alternatively, encoding module 470 is implemented as instructions stored in memory 460 and executed by processor 430.

Memory 460 includes one or more disks, tape drives, and solid state drives, and can be used as an overflow storage device, for storing programs when such programs are selected for execution, and for storing instructions and data that are read during program execution. Memory 460 may be volatile and/or non-volatile and may be read only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), and/or static random access memory (SRAM).

Fig. 5 is a simplified block diagram of a device 500 that can be used as either or both source device 310 and destination device 320 of FIG. 1 according to an exemplary embodiment.

Processor 502 in device 500 may be a central processing unit. Alternatively, processor 502 may be any other type of device or a plurality of devices capable of manipulating or processing information that currently exists or will be developed in the future. Although the disclosed implementations may be implemented with a single processor, as shown with processor 502, for example, speed and efficiency advantages may be achieved using more than one processor.

In an implementation, memory 504 in device 500 may be read only memory (ROM) or random access memory (RAM). The memory 504 may be any other suitable type of storage device. Memory 504 may include code and data 506 accessed by processor 502 using bus 512. Memory 504 may further include an operating system 508 and application programs 510, application programs 510 include at least one program, which allows the processor 502 to perform the methods described herein. For example, application programs 510 may include applications 1 through N, which further include a video encoding application that performs the methods described herein.

The device 500 may also include one or more output devices such as a display 518. The display 518 may be, in one example, a touch display that combines a display with a touch element capable of recognizing touch inputs. Display 518 may be connected to processor 502 via bus 512.

Although depicted as a single bus, bus 512 of device 500 may be comprised of multiple buses. In addition, secondary storage 514 may be directly connected to other components of the device 500 or may be accessed via a network and may contain one integrated unit, such as a memory card, or multiple units, such as multiple memory cards. Thus, device 500 can be implemented in a wide variety of configurations.

The specific implementation of simplified PDPC can be done in different ways, depending on the intra prediction mode:

For planar, DC, HOR/VER intra prediction modes (denoted as 0, 1, 18, 50 respectively in FIG. 7 and FIG. 8), the following steps are performed:

The predicted sample P(x, y) located on (x, y) is calculated as follows:

представляют собой опорные выборки, расположенные сверху и слева от текущей выборки (х, у), и

представляет собой опорную выборку, расположенную в верхнем левом углу текущего блока, функция clip1Cmp устанавливается следующим образом:wherein

are the pivot samples above and to the left of the current sample (x, y), and

is a reference sample located in the upper left corner of the current block, the clip1Cmp function is set as follows:

- if cIdx is 0, clip1Cmp is set to Clip1Y.

- otherwise, clip1Cmp is set equal to Clip1C

BitDepthy is the bit depth of the luma samples.

BitDepthc is the bit depth of the chrominance samples.

BitDepthy and BitDepthc may be signaled in the Sequence Parameter Set (SPS) of the bitstream.

Alternative definitions of Clip1Y (x) and Clip1C (c) are possible. In particular, as described by F. Galpin, P. Bordes and F. Le Léannec in JVET-C0040 "Adaptive constraint in

GEM2.0",

where min _C is the lower bound of the constraint used in the current segment for component ID C,

max _C is the upper bound of the constraint used in the current segment for component ID C,

C represents the color component (e.g. Y for luminance, Cb and Cr for chrominance),

"x >> y" is the arithmetic right shift of the integer representation of the binary complement of x by y binary digits. This function is only defined for non-negative integer values of y. The bits shifted to the most significant bits (MSBs) as a result of a right shift have a value equal to MSB x before the shift operation.

DC mode weights are calculated as follows:

wT=32>>((y<<l)>>shift),

vL=32>>((x<<l)>>shift),

wTL=-(wL>> 4)-(wT>>4).

wherein

For planar mode wTL=0, while for horizontal mode wTL=wT and for vertical mode wTL=wL. PDPC weights (wL, wT, wTL) DC mode for (0, 0) and (1, 0) positions within one 4x4 block are shown in FIG. nine.

For the diagonal (denoted as 2 and 66 in Fig. 7 and Fig. 8) and adjacent modes (directional modes no less than 58 and no more than 10 in Fig. 7 or Fig. 8), processing is performed as described below using the same formula ( one).

для расширения PDPC до верхнего правого диагонального режима. Выборка предсказания pred(x', у') находится в (х', у') в блоке предсказания. Координата х опорной выборки R_x,-1 задана:Fig. 10A illustrates the definition of reference samples.

to expand PDPC to upper right diagonal mode. The prediction sample pred(x', y') is at (x', y') in the prediction block. The x-coordinate of the reference sample R _x,-1 is given by:

x=x'+y'+1,

and the y-coordinate of the reference sample R _{1,y is} given in the same way: y=x'+y'+1.

The PDPC weights for the upper diagonal mode are:

wT=16>>((y'<<1)>>shift),

wL=16>>((x'<<1)>>shift),

wTL=0.

Similarly, FIG. 10B illustrates the definition of reference R _{x, -1} , R _{-1, y} and R _{-1, -1} for PDPC extension to the lower left diagonal mode. The x-coordinate of the reference sample R _x,-1 is given by:

x=x'+y'+1,

and the coordinate of the reference sample R _-1,y is given:

y=x'+y'+1.

PDPC for upper diagonal mode:

wT=16>>((y'<<1)>>shift),

wL=16>>((x'<<1)>>shift),

wTL=0.

In FIG. 10C shows the case of the adjacent upper right diagonal mode. The PDPC weights for the adjacent upper right diagonal mode are:

wT=32>>((y'<<1)>>shift),

wL=0,

wTL=0.

Similarly, in FIG. 10D shows the case of the neighboring lower left diagonal mode. The PDPC weights for the adjacent lower left diagonal mode are:

wL=32>>((x'<<1)>>shift),

wT=0,

wTL=0.

The reference sample coordinates for the last two cases are calculated using the tables already used for the intra-prediction angular mode. Linear datum interpolation is used if fractional datum coordinates are computed.

Simplified PDPC can be done as specified in the VVC specification. The following notation is used below:

является значением обратного угла, Round(х)=Sign(х)*Floor(Abs(x)+0.5),

is the value of the inverse angle, Round(x)=Sign(x)*Floor(Abs(x)+0.5),

Floor(x) is the largest integer less than or equal to x,

Log2(x) is the logarithm of x with base 2.

IntraPredAngle is the angle parameter specified in Table 6,

A=C?B:D is a triple assignment operation in which A is set equal to B if condition C is true. Otherwise, if condition C is false, A is set equal to D.

INTRA_PLANAR is planar intra prediction mode (),

INTRA_DC is the DC intra prediction mode,

INTRA_ANGULARXX is one of the directional intra prediction modes, in which XX denotes its number and the corresponding direction shown in FIG. eight.

If the term is not explained in the present invention, it is obvious that its definition can be found in the VVC specification or HEVC/H.265 standard specification.

Given the above description, the steps of a simplified PDPC can be defined as follows:

The input to this process are:

- intra prediction mode predModeIntra,

- variable nTbW indicating the width of the transform block,

- variable nTbH indicating the height of the transform block,

- variable refW indicating the width of the reference samples,

- variable refH indicating the height of the reference samples,

- predicted samples predSamples[x][y], with x=0…nTbW-1, y=0….nTbH-1,

- neighboring samples p[x][y], with x=-1, y=-1…refH-1 and x=0…refW-1, y=-1,

- variable cIdx indicating the color component of the current block.

The output of this process is the modified predicted samples predSamples[x][y] with x=0...nTbW-1, y=0...nTbH-1.

Depending on the value of cIdx, the clip1Cmp function is set as follows:

- if cIdx is 0, clip1Cmp is set to Clip1 _Y .

- otherwise, clip1Cmp is set equal to Clip1c.

The nScale variable is set to ((Log2(nTbW)+Log2(nTbH)-2)>>2).

Reference sample arrays mainRef[x] and sideRef[y], with x=0...refW-1 and y=0...refH-1 are obtained as follows:

mainRef[x]=p[x][-1] sideRef[y]=p[-1][y]

Variables refL[x][y], refT[x][y], wT[y], wL[x] and wTL[x][y] with x=0…nTbW-1, y=0…nTbH-1 received as follows:

- if predModeIntra is equal to INTRA_PLANAR, INTRA_DC, INTRA_ANGULAR 18 or INTRA_ANGULAR50, the following applies:

refL[x][y]=p[-1][y]

refT[x][y]=p[x][-1]

wT[y]=32>>((y<<1)>>nScale)

wL[x]=32>>((x<<1)>>nScale)

wTL[x][y]=(predModeIntra=INTRA_DC) ? ((wL[x]>>4)+(wT[y]>>4)): 0

- otherwise, if predModeIntra is equal to INTRA_ANGULAR2 or INTRA_ANGULAR66, the following applies:

refL[x][y]=p[-1][x+y+l]

refT[x][y]=p[x+y+1][-1]

wT[y]=(32>>1)>>((y<<1)>>nScale)

wL[x]=(32>>1)>>((x<<1)>>nScale)

wTL[x][y]=0

- otherwise, if predModeIntra is less than or equal to INTRA_ANGULAR10, the following ordered steps apply:

1. Variables dXPos[y], dXFrac[y], dXInt[y] and dX[x][y] are obtained as follows using invAngle:

2. Variables refL[x][y], refT[x][y], wT[y], wL[x] and wTL[x][y] are obtained as follows:

- otherwise, if predModeIntra is greater than or equal to INTRA_ANGULAR58 (see FIG. 8), the following ordered steps apply:

1. Variables dYPos[x], dYFrac[x], dYInt[x], and dY[x][y] are obtained as follows, using invAngle as below depending on intraPredMode:

2. Variables refL[x][y], refT[x][y], wT[y], wL[x] and wTL[x][y] are obtained as follows:

- otherwise refL[x][y], refT[x][y], wT[y], wL[x] and wTL[x][y] are all set to 0.

The values of the modified predicted samples predSamples[x][y] with x=0…nTbW-1, y=0…nTbH-1 are obtained as follows:

Equation 1 is used for simplified PDPC for nearest neighbor interpolation instead of linear:

refT[x][y]=(dX[x][y]<refW-1) ? mainRef[dX[x][y]]: 0

Similarly, Equation 2 assignment can also use nearest neighbor interpolation:

refL[x][y]=(dY[x][y]<refH-1) ? sideRef[dY[x][y]]: 0

Thus, on both sides of the encoder and decoder, the proposed method uses the following as input:

directional intra prediction mode (further referred to as predModeIntra, which is shown in FIG. 7 and FIG. 8)

block size parameter nTbS, which is set to (log2(nTbW)+Log2(nTbH))>>1, where nTbW and nTbH denote the width and height of the predicted block, respectively, and ">>" denotes a right shift operation.

A modification of the VVC specification that allows the proposed method to be used may include replacing "neighbor samples p[x][y]" with "reference samples p[x][y]" in the section describing the simplified PDPC.

The angular parameter intraPredAngle denotes the sub-pixel offset between two adjacent rows of predicted samples in a fixed-point representation having a fractional length of 5-bits. This parameter may be derived from the intraprediction mode derived from predModeIntra, and an exemplary derivation of intraPredAngle from predModeIntra may be determined by the LUT, for example, as shown in Table 1.

The present invention proposes to update the predicted samples obtained from the reference samples using directional, planar, or DC intra prediction modes. In FIG. 11 shows the steps in the update procedure. In the first step 1101, the scale values nScaleX and nScaleY are calculated depending on the shape and size of the block. Additionally, in FIG. 13 illustrates the implementation form of the calculation.

Steps 1103 and 1104 are looped over all positions of the predicted samples. In step 1102, loop iterations x and y are initialized, in steps 1105 and 1107 iteration boundary checks are performed, in steps 1106 and 1108 the iterations are incremented. At step 1108, iterations x and y are reset to the beginning of the next line, i. e. x is set to zero and iteration over row y is incremented by 1.

In step 1103, the position-dependent weight values wL[x] and wT[y] are calculated. These weights are obtained by right-shifting the base value V. The value of V is associated with the right-shift operation of step 4. If the right-shift is set to s, then the practical value of V can be determined, for example, as 2 ^s , 2 ^s-1 or 2 ^s . In the exemplary embodiment shown in FIG. 11, the value of s is set to 6 and the value of V is set to 32, that is, 2 ⁵ .

Unlike current methods, at step 1103 wL[x] and wT[y] use different scale values nScaleX and nScaleY, respectively. These weight values wL[x] and wT[y] are further used in step 1104 to compute a modified value of the predicted samples.

In FIG. 12 shows an alternative implementation of updating the predicted samples. In this embodiment, the weight wT[y] is recalculated only once per row, that is, as the iteration of y increases.

Due to QTBT and MTT separation, prediction blocks may not be square (mostly rectangular). However, modern PDPC does not consider the potential disparity in the width and height of the prediction block. A weighting mechanism based only on the value of the distance to the left and top sides of the predicted block may provide a too smooth operation for obtaining predicted samples if the predicted block aspect ratio differs from 1. The present invention proposes a solution for balancing the effect of the block size and the aspect ratio of the block using the mode internal prediction. Using the direction information provided by the intra prediction mode, the propagation distance and directionality of the PDPC smoothing effect can be determined. Therefore, it is proposed to use an intra prediction mode group and apply a dedicated weight scaling adaptation for each group. In this case, even elongated prediction blocks will use dithering according to side length, where appropriate, depending on the intra prediction mode and prediction block size.

In the proposed invention, it is proposed to use separate nScale values when obtaining the weights wT[y] and wL[x]. In particular, in the description above (step 1103), the following formulas will be used (the value of V is set to 16 or 32, respectively):

wT[y]=V>>((y<<1)>>nScaleY)

wL[x]=V>>((x<<1)>>nScaleX)

As shown in FIG. 14, scaling may be performed (block 1101). This method is an alternative calculation to that shown in FIG. 12. In a first step 1301, the value of the intraPredAngle intra prediction mode of the predicted block is obtained, eg, by bitstream parsing. In the next step 1302, the intra prediction mode is classified into one of the groups.

Further in the description, the additional definition of non-symmetric off-diagonal intra prediction mode is used:

intraPredMode is asymmetric off-diagonal if intraPredMode is not equal to 2 or VDIA_IDX and one of the following is true: intraPredMode is not less than VDIA_IDX - numModes or intraPredMode is not greater than 2+numModes

Table 2 shows an example classification.

Depending on the mode class index, getting nScaleX and nScaleY can be different. Steps 1303, ..., 1312 represent the mapping of the class index to derivation formulas, as shown in Table 3.

Tables 4, 5 and 6 show alternative classifications.

In Table 6, class 1 indicates not only diagonal modes, but also wide-angle modes, which have a non-fractional value of IntraPredAngle (ie, a multiple of 32 in Table 1).

An additional limitation that may be implied by the present invention does not apply to the steps of updating the predicted samples in the case of specific combinations of block size conditions and intra-prediction mode (intraPredMode).

Assuming the given numModes parameter is 8, a particular implementation may contain the following condition to disable PDPC, i.e., not performing a predicted sample update:

- the sum of the block width and the block height exceeds 64 samples and either intraPredMode is asymmetric off-diagonal or equal to PLANAR_IDX.

Particular implementations of the invention may demonstrate improvements in compression performance. The modifications to the VTM-3.0 software described in Table 4 and Table 3 result in improved compression efficiency (Table 7).

In other embodiments, the nScale value is set dependent on the inverse angle value that is obtained from the intra prediction mode. According to an embodiment, the following steps are performed to obtain the coefficient value nScale according to the intra prediction mode (predModeIntra) value:

- get the value of the intraPredAngle parameter based on the value of the prediction mode predModeIntra;

- get the value of the inverse angle (invAngle) based on the intraPredAngle parameter; and

- get the value of the nScale coefficient based on the block size and the value of invAngle.

Deriving the intraPredAngle parameter value based on the predModeIntra prediction mode value may be implemented via a lookup table. An exemplary lookup table is shown in Table 8.

The invAngle parameter of the inverse angle is derived from the intraPredAngle as follows:

Table 9 and table 10 are used to indicate the value of nScale based on the predModelntra and invAngle values.

Fig. 15 shows the steps of the present disclosure according to the classification given in Table 9 and Table 10. The variable denoted in FIG. 13 as "IntraPredAngle" is referred to as "predModeIntra" in FIG. 15. This variable indicates the ability to obtain the predicted values of the samples in the intra prediction mode, which are input values in the PDPC filtering process. In FIG. 8 shows the correspondence between predModeIntra values and intra prediction directions.

Although embodiments of the invention have been primarily described in terms of video coding, it should be noted that embodiments of encoding system 10, encoder 20, and decoder 30 (and system 10, respectively) and other embodiments described herein may also be configured for photo processing or image encoding, i.e. processing or encoding an individual image, independent of any previous or subsequent image, as in video encoding. In general, only inter prediction blocks 244 (encoder) and 344 (decoder) may be unavailable if the picture encoding process is limited to one picture 17. All other functions (also called tools or technologies) of video encoder 20 and video decoder 30 can be equally used for photographic processing, such as residual calculation 204/304, transform 206, quantize 208, inverse quantize 210/310, (inverse) transform 212/312, split 262/362, intra prediction 254/354, and/or loop filtering 220, 320, and entropy encoding 270; and entropy decoding 304.

Embodiments of, for example, encoder 20 and decoder 30 and the functions described in the present invention, for example, with reference to encoder 20 and decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on a computer-readable medium or transmitted over a communication medium as one or more instructions or code and executed by a hardware processing unit. A computer-readable medium may include a computer-readable medium that corresponds to a tangible medium such as a data medium or a medium including any medium that facilitates the transfer of a computer program from one place to another, for example, in accordance with a communication protocol. Thus, a computer-readable medium can typically correspond to (1) a tangible computer-readable medium that is permanent, or (2) a communication medium such as a signal or carrier wave. A storage medium can be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, code and/or data structures for implementing the methods described in the present invention. The computer program product may include a computer readable medium.

By way of example and without limitation, such a computer-readable medium may comprise a RAM, ROM, EEPROM, CD-ROM or other optical disk, a magnetic disk or other magnetic storage devices, flash memory, or any other medium that can be used to store the desired program code in in the form of instructions or data structures, and which can be accessed by a computer. In addition, any connection is also referred to as a computer-readable storage medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology such as infrared, radio, and microwave, then the coaxial cable , fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of media. However, it should be understood that the computer-readable storage medium and the computer-readable storage medium do not include connections, carriers, signals, or other transient media, but instead, should be considered as a permanent tangible medium. A disc as used herein includes a compact disc (CD), a laser disc, an optical disc, a digital versatile disc (DVD), a floppy disc, a Blu-ray disc, in which discs typically reproduce data magnetically while how discs reproduce data optically by means of a laser. Combinations of the above are also included within the scope of the computer readable media definitions.

The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Accordingly, the term "processor", as used herein, may refer to any of the aforementioned structure, or any other structure suitable for implementing the methods described in the present invention. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules configured to encode and decode or be contained within a combined codec. In addition, the methods may be fully implemented in one or more circuits or logic elements.

The methods of the present disclosure may be implemented in a wide variety of devices or appliances, including a cordless handset, an integrated circuit (IC), or a set of ICs (eg, a chipset). Various components, modules, or blocks are described in the present invention to provide functional aspects of devices capable of performing the described methods, but do not necessarily require implementation by different hardware blocks. Rather, as described above, the various blocks may be combined in a codec hardware block, or provided by a set of interacting hardware blocks, including one or more processors, as described above, in combination with suitable software and/or firmware.

The present disclosure includes the following additional aspects.

The first aspect of the image block intra-prediction method contains, for each sample from a plurality of block samples:

determining a predicted sample value from one or more reference sample values by intra prediction using one of DC intra prediction mode, PLANAR intra prediction mode, and intra prediction angle mode;

multiplying the predicted sample value by the sample weight to obtain an elevated predicted sample value;

determining a vertical scaling factor (nScaleY) based on the block height and intra prediction mode;

determining a horizontal scaling factor (nScaleX) based on the block width and intra prediction mode;

determining a first weight based on a vertical scaling factor (nScaleY);

determining a second weight based on a horizontal scale factor (nScaleX);

calculating an additional value as a weighted sum of the upper reference sample value and the left reference sample value by weighting the upper reference sample value with a first weight and the left reference sample value with a second weight;

adding an additional value to the weighted value of the predicted sample to obtain an unnormalized value of the predicted sample; and

normalizing the denormalized predicted sample value by arithmetic right shifting an integer representation of the denormalized predicted sample value to obtain a normalized predicted sample value.

The second aspect of the method according to the first aspect, wherein the image is part of a video sequence.

A third aspect of the method according to the first or second aspect, wherein normalizing the denormalized predicted sample value comprises calculating the normalized predicted sample value as

wherein

является нормализованным значением предсказанной выборки,

is the normalized value of the predicted sample,

P(x,y) is the unnormalized value of the predicted sample,

представляют значения ближайших опорных выборок, расположенных выше и слева от каждой выборки,

represent the values of the closest reference samples located above and to the left of each sample,

wL=V>>((x<<1)>>nScaleX) is the horizontal weighting factor,

wT=V>>((y<<1)>>nScaleY) is the vertical weighting factor,

x is the horizontal coordinate of the sample, and

y is the vertical coordinate of the sample.

A fourth aspect of the method according to any of the preceding aspects, wherein each sample is not updated if the sum of the block width and block height exceeds 64 samples and either intraPredMode is asymmetric off-diagonal or is PLANAR_IDX.

The fifth aspect of the encoder (20), containing the processing circuit for performing the method according to any of the first to fourth aspects.

The sixth aspect of the decoder (30), containing the processing circuit for performing the method according to any of the first to fourth aspects.

A seventh aspect of a computer program product, comprising program code for performing the method according to any one of the first to fourth aspects.

The eighth aspect of the decoder, containing: one or more processors; and

a persistent computer-readable medium coupled to the processors and storing programs for execution by the processors, wherein the programs, when executed by the processors, configure the decoder to execute the method of any one of the first to fourth aspects.

The ninth aspect of the encoder, containing:

one or more processors; and

a persistent computer-readable medium coupled to the processors and storing programs for execution by the processors, wherein the programs, when executed by the processors, configure the decoder to execute the method of any one of the first to fourth aspects.

Claims (86)

1. A method for intra prediction of an image block, comprising, for sampling from a plurality of block samples, the steps of: determining a predicted sample value from the one or more reference sample values by intra prediction using an intra prediction mode selected from DC intra prediction mode, planar intra prediction mode, and angular intra prediction mode; determining a first weight based on the intra prediction mode and one of the block height, block width, or both; determining a second weight based on the intra prediction mode and one of the block height, block width, or both; calculating additional values as a weighted sum of the upper reference sample value and the left reference sample value by weighting the upper reference sample value with a first weight and the left reference sample value with a second weight; multiplying the predicted sample value by the sample weight to obtain a weighted predicted sample value; adding an additional value to the weighted value of the predicted sample to obtain a non-normalized modified value of the predicted sample; and normalizing the non-normalized modified predicted sample value by arithmetic right-shifting an integer representation of the non-normalized modified predicted sample value to obtain a normalized modified predicted sample value. 2. The method of claim 1, further comprising performing intra prediction of an image block containing each sample from the plurality of block samples. 3. The method according to claim 1 or 2, in which the step of determining the first weight and determining the second weight comprises a sub-step of determining a scaling factor based on the intra prediction mode and one of the block height, block width, or both; wherein the first weight and the second weight are determined based on the scaling factor, respectively. 4. The method according to any one of paragraphs. 1-3, in which the first weight determination step comprises a sub-step of determining a vertical scale factor, denoted nScaleY, based on the block height and the intra prediction mode, wherein the first weight is determined based on the vertical scale factor nScaleY; the second weight determination step comprises a sub-step of determining a horizontal scale factor, denoted nScaleX, based on the block width and the intra prediction mode, wherein the second weight is determined based on the horizontal scale factor, nScaleX; a normalizing the denormalized predicted sample value by arithmetic right shifting the integer representation of the denormalized predicted sample value is to obtain a normalized predicted sample value. 5. The method of claim 4, wherein nScaleX and nScaleY are obtained depending on a mode class index, respectively, wherein the mode class index is obtained using an intra prediction mode group. 6. The method of claim 5, wherein the mode class index is mapped to a particular group of intra prediction modes according to one of the following tables: Group of intra prediction modes 2,
DC_IDX, HOR_IDX, VDIA_IDX Other modes class index one 2 0 or Group of intra prediction modes DC_IDX, HOR_IDX, VER_IDX 2, VDIA_IDX Other modes class index one 2 0 or Group of intra prediction modes DC_IDX, HOR_IDX, VER_IDX 2, VDIA_IDX Other modes class index 2 one 0 or Group of intra prediction modes DC_IDX, HOR_IDX, VER_IDX Diagonal modes with integer subpixel pitch:
-14 -12 -10 -6, 2, VDIA_IDX, 72, 76, 78, 80 Other modes class index 2 one 0 where DC_IDX denotes the DC mode having an intra prediction mode index of 1, HOR_IDX and VER_IDX denote the horizontal and vertical modes having intra prediction mode indices of 18 and 50, respectively, VDIA_IDX denotes the diagonal mode having an intra prediction mode index of 66. 7. The method of claim 4, wherein nScaleX and nScaleY are obtained according to the following mapping: class index How to obtain 0 nScaleX = (Log2(nTbW) – 2 + 1) >> 1
nScaleY = (Log2(nTbH) – 2 + 1) >> 1 one nScaleY = nScaleX = (Log2(nTbH) + Log2(nTbW)) > 6 ? 10 2 nScaleX = Log2(nTbW) > 3 ? 10
nScaleY = Log2(nTbH) > 3 ? 10 where nTbW and nTbH denote the width and height of the predicted block, respectively, and ">>" denotes a right shift operation; where Floor is a function of rounding down to the nearest integer, Log2 denotes the natural logarithm of a number with base 2, Min is the smallest of its arguments, and ">>" denotes a right shift operation. 8. The method of claim 5, wherein the mode class index is mapped to a specific group of intra prediction modes according to the following table: Group of intra prediction modes predModeIntra>50 predModeIntra<18 &&
predModeIntra!=0 && predModeIntra!=1 Other modes class index one 2 0 9. The method of claim 8, wherein nScaleX and nScaleY are obtained according to the following mapping: class index How to obtain 0 nScaleY = nScaleX = (Log2(nTbH) + Log2(nTbW) - 2 ) >> 2 one nScaleY=nScaleX=
Min( 2, Log2(nTbH) − Floor(Log2(3 * invAngle − 2)) + 8) 2 nScaleY=nScaleX=
Min( 2, Log2(nTbW) − Floor( Log2(3 * invAngle − 2)) + 8) Log2(nTbH) > 3 ? 10 where nTbW and nTbH denote the width and height of the predicted block, respectively, and ">>" denotes a right shift operation; where Floor is a function of rounding down to the nearest integer, Log2 denotes the natural logarithm of a number with base 2, Min is the smallest of its arguments, and ">>" denotes a right shift operation; invAngle is the inverse angle parameter obtained using intraPredAngle as follows:

; where intraPredAngle is the angle parameter obtained from the intra prediction mode specified by the predModeIntra variable using a lookup table, LUT. 10. The method of claim 8, wherein the LUT is given by the following table

11. The method according to any one of paragraphs. 4-10, wherein the step of normalizing the non-normalized modified predicted sample value comprises a sub-step of calculating the predicted sample value as

, where

is the unnormalized modified value of the predicted sample, R _x,-1 , R _-1,y represent the values of the nearest reference samples above and to the left of each sample, wL = V >> ((x << 1) >> nScaleX) is the horizontal weighting factor, wT = V >> ((y << 1) >> nScaleY) is the vertical weight, V is the base value; x is the horizontal coordinate of each sample, and y is the vertical sample coordinate; where clip1Cmp is set as follows: if cIdx is 0, clip1Cmp is set to Clip1Y; otherwise clip1Cmp is set equal to Clip1C; wherein cIdx is a variable indicating the color component of the current block; and Clip1 _Y (x) = Clip3(0, (1 << BitDepth _Y ) − 1, x) Clip1 _C (x) = Clip3(0, (1 << BitDepth _C ) − 1, x)

where BitDepthy is the bit depth of the luma samples, where BitDepthc is the bit depth of the chrominance samples. 12. The method of claim 11, wherein for planar mode, wTL = 0, for horizontal mode, wTL = wT, and for vertical mode, wTL = wL. 13. The method according to any one of paragraphs. 4-10, wherein normalizing the non-normalized modified predicted sample value comprises calculating the normalized modified predicted sample value as

, where

is the normalized modified value of the predicted sample,

is the unnormalized modified value of the predicted sample, R _x,-1 , R _-1,y represent the values of the nearest reference samples above and to the left of each sample, wL = V >> ((x << 1) >> nScaleX) is the horizontal weighting factor, wT = V >> ((y << 1) >> nScaleY) is the vertical weight, V is the base value; x is the horizontal coordinate of each sample, and y is the vertical coordinate of the sample. 14. The method according to any one of paragraphs. 1-13, in which each sample is not updated, if the sum of the box width and box height, given in samples, respectively, is more than 64 samples, and either the intra prediction mode is non-symmetric off-diagonal, or the intra prediction mode is a planar intra prediction mode. 15. The method of claim 14, wherein intraPredMode is asymmetric off-diagonal if intraPredMode is not equal to 2 or VDIA_IDX and one of the following is true: intraPredMode is not less than VDIA_IDX - numModes or intraPredMode is not greater than 2 + numModes. 16. The method according to any one of paragraphs. 1-15, further comprising the step of adding a residual value to the normalized modified predicted sample value to obtain a reconstructed sample value. 17. Encoder (20), containing the processing circuit for performing the method according to any one of paragraphs. 1-16. 18. Decoder (30), containing the processing circuit for performing the method according to any one of paragraphs. 1-16. 19. A computer-readable storage medium that stores program code that causes the execution of the method according to any one of paragraphs. 1-16. 20. Decoder, containing: one or more processors; and a persistent computer-readable medium coupled to the processors and storing programs for execution by the processors, wherein the programs, when executed by the processors, cause the decoder to execute the method of any one of claims. 1-16. 21. An encoder containing: one or more processors; and a persistent computer-readable medium coupled to the processors and storing programs for execution by the processors, wherein the programs, when executed by the processors, cause the encoder to execute the method of any one of claims. 1-16.

Images