RU2800681C2 - Coder, decoder and corresponding methods for intra prediction - Google Patents

Abstract

FIELD: coding or decoding a video signal.

SUBSTANCE: invention relates to intra prediction. Devices and methods for intra prediction are proposed. The method includes: obtaining a value of the first indication information of the current block, wherein the value of the first indication information indicates whether the intra prediction mode of the current block is contained in the most likely mode set; and getting the reference row value of the index of the current block. When the value of the first indication information indicates that the intra prediction mode of the current block is contained in the most likely mode set, and when the value of the index reference row indicates that the closest adjacent reference row to the current block is used, the method further includes obtaining a value of the second indication information of the current block, wherein the value of the second current block indication information indicates whether the intra prediction mode of the current block is a planar mode or not.

EFFECT: increase in the efficiency and speed of coding, in particular, an increase in the efficiency of determining the intra prediction mode of the current block.

26 cl, 19 dwg, 14 tbl

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of U.S. provisional application no. 62/822,865, filed March 23, 2019, U.S. provisional application no. 62/824,282, filed March 26, 2019, and U.S. provisional application no. 62/824360, filed March 27, 2019, all of which are incorporated herein by reference.

Technical field

Embodiments of the present application (disclosure) generally relate to the field of picture processing and, in particular, to intra prediction.

State of the art

Video encoding (video encoding and decoding) is used in a wide range of digital video applications, such as digital television broadcast, video transmission over the Internet and mobile networks, real-time conversational applications such as video chat, video conferencing, DVD and Blu-ray discs, systems for capturing and editing video content; and camcorder security applications.

The amount of video data required to represent even a relatively short video can be substantial, which can lead to difficulties when the data is to be streamed or otherwise transmitted over a limited bandwidth communication network. Thus, video data is typically compressed before being transmitted over modern telecommunication networks. Video size can also be an issue when the video is stored on a storage device, as memory resources can be limited. Video compression devices often use source software and/or hardware to encode video data prior to transmission or storage, thereby reducing the amount of data needed to represent digital video images. The compressed data is then received at the destination by a video decompressor which decodes the video data. With limited network resources and ever-increasing demands for higher video quality, improved compression and decompression techniques are desired that improve compression rates with little or no loss in image quality.

In particular, there is a continuing need to reduce the processor load in the context of intra prediction processing. It is known in the art to use a Most Likely Mode List (MPM) for encoding in intra prediction mode. The MPM list reduces the number of bits needed to encode the intra prediction mode of the current block. When the intra prediction mode of the current block matches an entry in the MPM list, an index is encoded rather than the actual mode, resulting in fewer bits being required. However, selecting the entries to generate the MPM list with the most efficient intra-prediction encoding is still a bit of a challenge. The correct choice of entries greatly affects the efficiency of encoding with the intra prediction mode based on the MPM list.

In view of this, the object underlying the present invention is to provide a technique for obtaining an MPM list-based intra prediction mode that results in more efficient coding compared to the prior art.

The essence of the invention

Embodiments of the present application provide devices and methods for encoding and decoding according to the independent claims.

The above and other objects are achieved by the essence of the invention according to the independent claims. Additional forms of implementation are apparent from the dependent claims, description and figures.

According to a first aspect, the invention relates to a method performed by a decoding apparatus. The method includes: obtaining a value of the first indication information of the current block, wherein the value of the first indication information indicates whether the intra prediction mode of the current block is contained in the most likely mode set; and getting the reference row value of the index of the current block. The set of most probable modes contains 5 candidate intra prediction modes and a planar mode. When the value of the first indication information indicates that the intra prediction mode of the current block is contained in the most likely mode set, and when the value of the index reference row indicates that the closest adjacent reference row to the current block is used, the method further includes obtaining a value of the second indication information of the current block , wherein the value of the second current block indication information indicates whether the intra prediction mode of the current block is a planar mode or not.

In the conventional method, the decoding complexity for MPM processing is high. Three different MPM lists (6 entries) are generated depending on the use of the Internal Subsection Encoding Mode (ISP) and Multiple Reference Lines (MRL) tools. In aspects or any possible embodiments of aspects of this invention, the MPM list (5 entries) design is unified with reduced decoding complexity, regardless of whether an MRL or ISP encoding tool is used.

In addition, the planar mode is considered a special mode due to its high frequency, and a special flag is assigned to check whether the current mode is a planar mode. Although not in the MPM list (with 5 invented entries), it is always checked when the mpm flag is true. In other words, the mpm flag indicates whether the intra prediction mode of the current block is in a 6-entry list consisting of a planar mode and 5 entries of the MPM list. These 5 MPM list entries are built based on the intra prediction modes of the left and top block).

As an implementation of the first aspect, the first indication information is indicated by the first flag, for example, intra_luma_mpm_flag. The reference row value of the index of the current block can be specified using intra_luma_ref_idx. The second indication information may be indicated by a second flag, such as intra_luma_not_planar_flag.

When the MRL encoding is not enabled, that is, when the value of the index reference string indicates that the nearest adjacent reference string to the current block is used, the devices parse the bit stream to obtain the value of the second indication information. When the MRL encoding is enabled, the value of the second indication information is obtained as a default value. In this situation, the planar mode is excluded and its corresponding flag (eg, intra_luma_not_planar_flag) is not parsed from the bitstream, but output as a default value (eg, value 1). In this case, a bit is stored to indicate the value of the planar flag (i.e., Intra_luma_not_planar) when the MRL is enabled).

According to a second aspect, the invention relates to a method performed by an encoding apparatus. The method includes: determining whether an intra prediction mode of the current block is included in a set of most likely modes; and determining whether the nearest adjacent reference row to the current block is used for intra prediction. The set of most probable modes contains 5 candidate intra prediction modes and a planar mode. The method further includes encoding a bit stream, wherein the bit stream includes information indicating whether the intra prediction mode of the current block is a planar mode when the intra prediction mode of the current block is contained in the most likely mode set, and the closest adjacent reference row to the current block is used for intra prediction.

As an implementation of the second aspect, the bitstream includes a first flag indicating whether the intra prediction mode of the current block is included in the set of most likely modes and a second flag indicating whether the intra prediction mode of the current block is a planar mode. For example, the first flag is intra_luma_mpm_flag and the second flag is intra_luma_not_planar_flag.

The method according to the first aspect of the invention may be performed by the decoding apparatus according to the third aspect of the invention. The decoding device includes a first acquisition unit and a second acquisition unit. A first acquisition unit configured to obtain a reference row value of the current block index and a value of the first indication information of the current block, wherein the value of the first indication information indicates whether the intra prediction mode of the current block is included in the most likely mode set. The set of most probable modes contains 5 candidate intra prediction modes and a planar mode. The second acquisition unit, configured to obtain the value of the second indication information of the current block, when the value of the first indication information indicates that the intra prediction mode of the current block is contained in the most likely mode set, and when the index reference row value indicates that the nearest adjacent reference row to the current is used block, wherein the value of the second current block indication information indicates whether the intra prediction mode of the current block is a planar mode.

Additional features and forms of implementation of the method according to the third aspect of the invention correspond to the features and forms of implementation of the apparatus according to the first aspect of the invention.

The method according to the second aspect of the invention may be performed by the encoder according to the fourth aspect of the invention. The encoding device includes a determination unit and an encoding unit. A determination unit configured to determine whether the intra prediction mode of the current block is contained in the most likely mode set and to determine whether the nearest adjacent reference row to the current block is used for intra prediction. The set of most probable modes includes 5 candidate intra prediction modes and a planar mode. A coding block configured to encode a bitstream, the bitstream including information indicating whether the intra prediction mode of the current block is a planar mode when the intra prediction mode of the current block is contained in the most likely mode set, and the closest adjacent reference row to the current block is used for intra prediction.

Additional features and forms of implementation of the method according to the fourth aspect of the invention correspond to the features and forms of implementation of the apparatus according to the second aspect of the invention.

According to a fifth aspect, the invention relates to equipment for decoding a video stream, including a processor and a memory. The memory stores instructions that cause the processor to execute the method according to the first aspect.

According to a sixth aspect, the invention relates to a video stream encoding apparatus including a processor and a memory. The memory stores instructions that cause the processor to execute the method according to the second aspect.

According to a seventh aspect, a computer-readable medium is provided that stores instructions that, when executed, operate one or more processors capable of encoding video data. The instructions cause one or more processors to execute a method according to the first or second aspect, or any possible embodiment of the first or second aspect.

According to an eighth aspect, the invention relates to a computer program containing program code for executing the method according to the first or second aspect, or any possible embodiment of the first or second aspect, when executed on a computer.

The planar mode has a high frequency used as the intra prediction mode. The encoding efficiency of the encoding process can be improved by using information to indicate whether the intra prediction mode of the current block is a planar mode, as compared to the prior art, when the intra prediction mode of the current block is contained in the set of most likely modes, and the nearest adjacent reference row to the current block is used for intra prediction.

Details of one or more embodiments are set forth in the accompanying drawings and in the following description. Other features, purposes and advantages will be apparent from the description, drawings and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Further embodiments of the invention are described in more detail with reference to the accompanying figures and drawings, in which:

Fig. 1A is a block diagram showing an example of a video encoding system capable of implementing embodiments of the invention;

Fig. 1B is a block diagram showing another example of an implementable video coding system;

Fig. 2 is a block diagram showing an example of a video encoder capable of implementing embodiments of the invention;

Fig. 3 is a block diagram showing an example of the structure of a video decoder capable of implementing embodiments of the invention;

Fig. 4 is a block diagram illustrating an example of an encoding apparatus or a decoding apparatus;

Fig. 5 is a block diagram illustrating another example of encoding apparatus or decoding apparatus;

Fig. 6 is a block diagram illustrating an example of a multiple reference string (MRL) encoding tool;

Fig. 7 is a block diagram illustrating an example of division of 4×8 and 8×4 blocks;

Fig. 8 is a block diagram illustrating an example of division of all blocks except 4×8, 8×4 and 4×4;

Fig. 9 is a block diagram illustrating an example of CABAC context modeling/coding;

Fig. 10 is a block diagram illustrating an example of left (L) and top (A) neighboring blocks of the current block;

Fig. 11 is a block diagram illustrating intra prediction modes;

Fig. 12A illustrates embodiments of a current block intra prediction method 1200 implemented by a decoder according to the present invention;

Fig. 12B illustrates embodiments of a current block intra prediction method 1210 implemented by a decoder according to the present invention;

Fig. 13 illustrates embodiments of a current block intra prediction method 1300 implemented by an encoder according to the present invention;

Fig. 14 illustrates embodiments of a decoding apparatus 1400 for use in an image decoder according to the present invention;

Fig. 15 illustrates embodiments of an encoder 1500 for use in an image encoder according to the present invention;

Fig. 16 is a block diagram showing an exemplary structure of a content delivery system 3100 that implements a content delivery service; And

Fig. 17 is a block diagram showing the structure of an exemplary terminal device.

The following like reference numerals refer to the same or at least functionally equivalent features, unless expressly stated otherwise.

DETAILED DESCRIPTION OF EMBODIMENTS

In the following description, reference is made to the accompanying drawings, which form part of this disclosure and which show, by way of illustration, specific aspects of embodiments of the invention or specific aspects in which embodiments of the present invention may be used. It is clear that embodiments of the present invention can be used in other aspects and contain structural or logical changes not shown in the figures. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

For example, it is understood that the disclosure associated with the described method may also be true for the corresponding device or system configured to perform the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more blocks, such as functional blocks, to perform the one or more method steps described (for example, one block performing one or more steps, or a plurality of blocks each of which performs one or more of a plurality of steps), even if such one or more blocks are not explicitly described or illustrated in the figures. On the other hand, for example, if a particular device is described in terms of one or more blocks, such as functional blocks, the corresponding method may include one step to perform the functionality of one or more blocks (for example, one step to perform the functionality of one or more blocks , or a plurality of steps, each of which performs the functionality of one or more of the plurality of blocks), even if such one or more steps are not explicitly described or illustrated in the figures. In addition, it is understood that features of the various exemplary embodiments and/or aspects described herein may be combined with one another unless SPECIFICLY indicated otherwise.

Video encoding generally refers to the processing of a sequence of pictures that form a video or video sequence. Instead of the term "picture", the terms "frame" or "image" may be used interchangeably in the field of video encoding. Video encoding (or encoding in general) contains two parts: video encoding and video decoding. Video encoding is performed on the source side and typically involves processing (eg, by compressing) the source video images to reduce the amount of data required to represent the video images (for more efficient storage and/or transmission). Video decoding is performed on the destination (destination) side and usually contains inverse processing compared to the encoder for recovering video images. Embodiments relating to "encoding" video pictures (or pictures in general) should be understood as referring to "encoding" or "decoding" video pictures or corresponding video sequences. The combination of an encoding part and a decoding part is also called CODEC (encoding and decoding).

In the case of lossless video encoding, the original video images can be restored, i.e. restored video images have the same quality as the original video images (assuming there are no transmission losses or other data loss during storage or transmission). In the case of lossy video coding, additional compression is performed, for example by quantization, to reduce the amount of data representing video pictures that cannot be fully reconstructed at the decoder, i.e. the quality of the restored video images is lower or worse than the quality of the original video images.

Several video encoding standards belong to the group of "hybrid lossy video codecs" (ie, combine spatial and temporal sample-domain prediction and 2D transform encoding to apply transform-domain quantization). Each picture of a video sequence is usually divided into a set of non-overlapping blocks, and encoding is usually done at the block level. In other words, in the encoder, the video is usually processed, that is, encoded, at the block (video block) level, for example, using spatial (intra-picture) prediction and/or temporal (inter-picture) prediction to generate a prediction block, the prediction block is subtracted from the current block (the block currently being processed/to be processed) to obtain a residual block, the residual block is transformed and this residual block is quantized in the transformation region to reduce the amount of data (compression) to be transmitted, while in the decoder the inverse processing compared to applied to the encoded or compressed block by the encoder to reconstruct the current block for presentation. In addition, the encoder duplicates the processing cycle of the decoder, so that they will both generate identical predictions (eg, intra- and inter-prediction) and/or reconstruction for processing, ie encoding (coding), subsequent blocks.

In the following embodiments of video encoding system 10, video encoder 20 and video decoder 30 are described based on FIG. 1 to 3.

Fig. 1A is a schematic block diagram illustrating an exemplary encoding system 10, such as a video encoding system 10 (or encoding system 10 for short), that may use the techniques of the present application. Video encoder 20 (or encoder 20 for short) and video decoder 30 (or decoder 30 for short) of video encoding system 10 represent examples of devices that can be configured to perform techniques in accordance with the various examples described herein.

As shown in FIG. 1A, the encoding system 10 includes a source device 12 configured to provide the encoded picture data 21 to, for example, a sink device 14 for decoding the encoded picture data 13.

Source device 12 includes an encoder 20 and may additionally, i. e. optionally, contain a picture source 16, a preprocessor (or preprocessor unit) 18, such as a picture preprocessor 18, and a communication interface or communication unit 22.

The picture source 16 may comprise or be any type of picture capture device, such as a camera for capturing a real world picture, and/or any type of picture generating device, such as a computer graphics processor for generating a computer animated picture, or any type of other device for capturing and/or providing a real-world image, a computer-generated image (eg, screen content, virtual reality (VR) image), and/or any combination thereof (eg, augmented reality (AR) image). The source of the picture can be any type of memory or storage that stores any of the above pictures.

Unlike the preprocessor 18 and the processing performed by the preprocessor unit 18, the picture or picture data 17 may also be referred to as a raw picture or raw picture data 17 .

The preprocessor 18 is configured to receive the (raw) picture data 17 and perform pre-processing on that picture data 17 to obtain a pre-processed picture 19 or pre-processed picture data 19. The pre-processing performed by the preprocessor 18 may, for example, include cropping, color format conversion (eg from RGB to YCbCr), color correction or denoising. It can be understood that the preprocessor block 18 may be an optional component.

Video encoder 20 is configured to receive pre-processed picture data 19 and provide encoded picture data 21 (more details will be described below, for example, based on FIG. 2).

Communication interface 22 of source device 12 may be configured to receive encoded picture data 21 and transmit that encoded picture data 21 (or any further processed version thereof) over a communication channel 13 to another device, such as a destination device 14 or any other device. for saving or immediate restoration.

Destination device 14 includes a decoder 30 (eg, video decoder 30) and may additionally, i. e. optionally, contain a communication interface or communication unit 28, a post-processor 32 (or post-processing unit 32) and a display device 34.

Communication interface 28 of sink device 14 is configured to receive encoded picture data 21 (or any further processed version thereof), such as directly from source device 12 or from any other source, such as a storage device, such as an encoded picture data storage device, and provide encoded picture data 21 to decoder 30.

Communication interface 22 and communication interface 28 may be configured to transmit or receive encoded picture data 21 or encoded data 13 via a direct link between source device 12 and destination device 14, such as a direct wired or wireless connection, or over any type of network. , such as a wired or wireless network, or any combination thereof, or any private and public network, or any combination thereof.

The communication interface 22 may, for example, be configured to package the encoded picture data 21 into a suitable format, such as packets, and/or process the encoded picture data using any type of transmission coding or processing for transmission over a communication link or network.

The communication interface 28, analogous to the communication interface 22, may, for example, be configured to receive the transmitted data and process the transmission data using any kind of appropriate transmission decoding or processing and/or decompression to obtain encoded picture data 21.

Both communication interface 22 and communication interface 28 may be configured as unidirectional communication interfaces, as indicated by the arrow for communication channel 13 in FIG. 1A pointing from source device 12 to destination device 14, or as bi-directional communication interfaces, and can be configured to, for example, send and receive messages, for example, to establish a connection, to acknowledge and exchange any other information related to the line communication and/or data transmission, for example, the transmission of encoded picture data.

The decoder 30 is configured to receive encoded picture data 21 and provide decoded picture data 31 or decoded picture 31 (more details will be described below, for example, based on Fig. 3 or Fig. 5.

The post-processor 32 of the destination device 14 is configured to post-process the decoded picture data 31 (also referred to as reconstructed picture data), such as the decoded picture 31, to obtain post-processed picture data 33, such as the post-processed picture 33. Post-processing performed by block 32 post-processing may include, for example, color format conversion (for example, from YCbCr to RGB), color correction, cropping or resampling, or any other processing, for example, to prepare the decoded picture data 31 for display, for example, using the display device 34.

The display device 34 of the recipient device 14 is configured to receive post-processed picture data 33 for displaying the picture to, for example, a user or viewer. The display device 34 may be or include any type of display for presenting the reconstructed picture, such as an integrated or external display or monitor. The displays may, for example, comprise liquid crystal displays (LCD), organic light emitting diode (OLED) displays, plasma displays, projectors, micro-LED displays, liquid crystal on silicon (LCoS), digital light processor (DLP), or any other type of display. .

Although Fig. 1A illustrates source device 12 and sink device 14 as separate devices, device embodiments may also include both or both functionality, source device 12 or related functionality, and sink device 14 or related functionality. In such embodiments, source device 12 or related functionality and sink device 14 or corresponding functionality may be implemented using the same hardware and/or software, or using separate hardware and/or software, or any combination thereof. .

As will be apparent to those skilled in the art based on the description, the presence and (exact) separation of functionality of different blocks or functionality in source device 12 and/or sink device 14 as shown in FIG. 1A may vary depending on the actual device and applications.

Encoder 20(e.g. video encoder 20) or decoder 30 (e.g. video decoder 30) or both encoder 20 and decoder 30 may be implemented via a processing circuit as shown in FIG. 1B, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, dedicated video encoding hardware, or any combinations thereof. Encoder 20 may be implemented via processing circuitry 46 to implement various modules, as discussed with respect to encoder 20 of FIG. 2 and/or any other encoder system or subsystem described herein. Decoder 30 may be implemented via processing circuitry 46 to implement various modules, as discussed in relation to decoder 30 of FIG. 3 and/or any other decoder system or subsystem described herein. The processing circuit may be configured to perform various operations, which will be described below. As shown in FIG. 5, if the techniques are partially implemented in software, the device may store instructions for the software on a suitable non-volatile computer-readable storage medium and may execute the instructions in hardware using one or more processors to execute the techniques of this disclosure. Any of video encoder 20 and video decoder 30 may be integrated as part of a combined encoder/decoder (CODEC) in a single device, eg as shown in FIG. 1B.

Source device 12 and destination device 14 may comprise any of a wide range of devices, including any type of portable or stationary device, such as laptops or laptops, mobile phones, smartphones, tablets or tablet computers, cameras, desktop computers, televisions, set-top boxes, televisions, display devices, digital media players, game consoles, video streaming devices (for example, content service servers or content delivery servers), broadcast receiver, broadcast transmitter or the like, and may use any type of operating system or do without it . In some cases, source device 12 and destination device 14 may be equipped for wireless communication. Thus, source device 12 and destination device 14 may be wireless communication devices.

In some cases , the video encoding system 10 illustrated in FIG. 1A is only an example, and the techniques of the present application may apply to video encoding settings (such as video encoding or video decoding) that do not necessarily involve any data transfer between encoding and decoding devices. In other examples, data is retrieved from local memory, streamed over a network, or the like. The video encoder may encode and store data in the memory, and/or the video decoder may retrieve and decode the data from the memory. In some examples, encoding and decoding is performed by devices that do not communicate with each other, but simply encode data into memory and/or retrieve and decode data from memory.

For convenience of description, embodiments of the present invention are described herein, for example, with reference to High Efficiency Video Coding (HEVC) or Universal Video Coding (VVC) reference software, the next generation video coding standard being developed by the Joint Video Coding Collaborative (JCT-VC) Expert Group on ITU-T Video Coding (VCEG) and the ISO/IEC Moving Picture Expert Group (MPEG). One of ordinary skill in the art will appreciate that the embodiments of the present invention are not limited to HEVC or VVC.

Encoder and encoding method

Fig. 2 shows a schematic block diagram of an exemplary video encoder 20 that is configured to implement the techniques of the present application. In the example of FIG. 2, video encoder 20 comprises an input 201 (or input interface 201), a residual calculation block 204, a transform processing block 206, a quantization block 208, an inverse quantization block 210, and an inverse transform processing block 212, a reconstruction block 214, a contour block 220 filtering, decoded picture buffer (DPB) 230, mode selection block 260, entropy encoding block 270, and output 272 (or output interface 272). The mode selector 260 may include an inter-predictor 244, an intra-predictor 254, and a splitter 262. Inter prediction block 244 may include a motion estimator and a motion compensation block (not shown). The video encoder 20 shown in FIG. 2 may also be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec.

The residual calculator 204, the transform processing block 206, the quantizer 208, the mode selector 260 may relate to generating the forward signal path of the encoder 20, while the inverse quantizer 210, the inverse transform processor 212, the reconstruction block 214, the loop buffer 216 filter 220, decoded picture buffer (DPB) 230, inter predictor 244, and intra predictor 254 may relate to generating the feedback signal path of video encoder 20, wherein the feedback signal path of video encoder 20 corresponds to the signal path of the decoder (see video decoder 30 in FIG. .3). An inverse quantization block 210, an inverse transform processing block 212, a reconstruction block 214, a loop filter 220, a decoded picture buffer (DPB) 230, an inter prediction block 244, and an intra prediction block 254 are also related to the formation of the “embedded decoder” of the video encoder 20.

Pictures and picture sharing (pictures and blocks)

The encoder 20 may be configured to receive, for example, via input 201 a picture 17 (or picture data 17), for example a picture from a sequence of pictures constituting a video or video sequence. The received picture or picture data may also be a pre-processed picture 19 (or pre-processed picture data 19). For simplicity, the following description refers to picture 17. Picture 17 may also be referred to as the current picture or the picture to be encoded (particularly in video encoding, to distinguish the current picture from other pictures, such as previously encoded and/or decoded pictures of the same video sequence, i.e. a video sequence that also contains the current picture).

A (digital) picture is, or can be viewed as, a two-dimensional array or matrix of samples with intensity values. A sample in an array can also be referred to as a pixel (short form of picture element) or pel (picture element). The number of samples in the horizontal and vertical direction (or axis) of the array or picture determines the size and/or resolution of the picture. Three color components are commonly used to represent a color, i.e. the picture can be represented or include three arrays of samples. In RBG format or color space, a picture contains a corresponding array of red, green, or blue samples. However, in video encoding, each pixel is usually represented in a luma and chrominance format or color space, such as YCbCr, which contains a luminance component, denoted Y (sometimes L is also used instead), and two chrominance components (chroma components), denoted Cb and Cr. The Y component of luminance (or luma for short) represents the brightness (brightness) or gray level intensity (e.g., as in a grayscale picture), while the two components Cb and Cr are chrominance (or chroma for short) represent components of chromaticity or color information. Accordingly, a picture in YCbCr format contains an array of luma samples with (Y) luminance samples and two arrays of chrominance samples with chroma values (Cb and Cr). Pictures in RGB format can be converted or converted to YCbCr format and vice versa, the process is also known as color conversion or conversion. If the picture is monochrome, it can only contain an array of brightness samples. Accordingly, the picture may be, for example, an array of luminance samples in monochrome format, or an array of luminance samples and two corresponding arrays of chrominance samples in 4:2:0, 4:2:2 and 4:4:4 color format.

Embodiments of video encoder 20 may include a picture splitter (not shown in FIG. 2) configured to split picture 17 into multiple (generally non-overlapping) picture blocks 203. These blocks may also be referred to as root blocks, macroblocks (H.264/AVC), or coding tree blocks (CTBs) or coding tree units (CTUs) (H.265/HEVC and VVC). The picture dividing block may be configured to use the same block size for all pictures in a video sequence and a corresponding grid defining the block size, or to change the block size between pictures or subsets or groups of pictures and divide each picture into appropriate blocks.

In further embodiments, the video encoder may be configured to directly receive block 203 of picture 17, such as one, more or all of the blocks forming picture 17. Picture block 203 may also be referred to as the current picture block or the picture block to be encoded.

Like picture 17, picture block 203 is again, or can be thought of as, a two-dimensional array or matrix of samples with intensity values (sample values), albeit smaller than picture 17. In other words, block 203 may contain, for example, one array of samples (for example , a luma array in the case of a monochrome image 17 or an array of luminance or chrominance in the case of a color image) or three arrays of samples (for example, luminance and two chrominance arrays in the case of a color image 17) or any other number and/or type of arrays depending on the applied color format. The number of samples in the horizontal and vertical direction (or axis) of block 203 determines the size of block 203. Accordingly, a block may be, for example, an array of M×N samples (M-column per N-row) or an array of M×N transform coefficients.

The embodiments of video encoder 20 shown in FIG. 2 may be configured to encode picture 17 block by block, such as encoding and prediction is performed for each block 203.

Embodiments of video encoder 20, as shown in FIG. 2 may be further configured to split and/or encode a picture using slices (also referred to as video slices), wherein the picture may be divided into or encoded using one or more slices (generally non-overlapping), and each slice may contain one or more units (eg CTU).

The embodiments of video encoder 20 shown in FIG. 2 may be further configured to split and/or encode a picture using groups of tiles (tiles) (also referred to as video/video tile groups) and/or tiles (also referred to as video tiles), wherein the picture may be divided into or encoded using one or more tile groups (usually non-overlapping), and each tile group may contain, for example, one or more blocks (for example, CTU) or one or more tiles, each tile, by way of example, may be rectangular in shape and may contain one or more units (eg CTU) such as full or partial units.

Remainder Calculation

Residual block 204 may be configured to calculate residual block 205 (also referred to as residual 205) based on picture block 203 and prediction block 265 (more details on predictor 265 are provided below), for example, by subtracting the sample values of predictor 265 from the values samples of the picture block 203, sample by sample (pixel by pixel) to obtain a residual block 205 in the samples area.

transformation

The transform processing unit 206 may be configured to apply a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values of the residual block 205 to obtain transform coefficients 207 in the transform domain. Transform coefficients 207 may also be referred to as residual transform coefficients and represent a residual block 205 in the transform region.

Transform processing unit 206 may be configured to apply DCT/DST integer approximations such as those defined for H.265/HEVC. Compared to the orthogonal DCT, such integer approximations are usually scaled by some factor. To preserve the norm of the residual block that is processed by forward and backward transforms, additional scaling factors are applied as part of the transform process. The scaling factors are usually chosen based on some constraints, such as the scaling factors being a power of two for shift operations, the bit depth of the transform coefficients, the trade-off between accuracy and implementation cost, and so on. Specific scaling factors, for example, are specified for the inverse transform, such as by the inverse transform processor 212 (and the corresponding inverse transform, such as the inverse transform processor 312 in video decoder 30), and the corresponding scaling factors for the forward transform, such as by the transform processor 206, may be set appropriately in encoder 20.

Embodiments of video encoder 20 (respectively, transform processing unit 206) may be configured to output transform parameters, such as the type of transform or transforms, such as directly or encoded or compressed via entropy encoding unit 270, such that, for example, video decoder 30 may receive and use conversion parameters for decoding.

Quantization

Quantizer 208 may be configured to quantize transform coefficients 207 to obtain quantized coefficients 209, for example by applying scalar quantization or vector quantization. The quantized coefficients 209 may also be referred to as quantized transform coefficients 209 or quantized residual coefficients 209.

The quantization process may reduce the bit depth associated with some or all of the transform coefficients 207 . For example, an n-bit transform factor may be rounded to an m-bit transform factor during quantization, where n is greater than m. The degree of quantization can be changed by adjusting the quantization parameter (QP). For example, for scalar quantization, different scaling may be applied to achieve finer or coarser quantization. Smaller quantization step sizes correspond to finer quantization, while larger quantization step sizes correspond to coarser quantization. The applicable quantization step size may be indicated by a quantization parameter (QP). The quantization parameter may, for example, be an index to a predefined set of applicable quantization step sizes. For example, small quantization parameters may correspond to fine quantization (small quantization step sizes), and large quantization parameters may correspond to coarse quantization (large quantization step sizes), or vice versa. Quantization may include dividing by the quantization step size, and corresponding and/or inverse dequantization, for example by inverse quantizer 210, may include multiplying by the quantization step size. Embodiments in accordance with some standards, such as HEVC, may be configured to use a quantization parameter to determine the quantization step size. Typically, the quantization step size can be calculated based on the quantization parameter using a fixed-point approximation of an equation involving division. Additional scaling factors may be introduced for quantization and dequantization to recover the residual block rate that may have changed due to the scaling used in the fixed point approximation of said equation for quantization step size and quantization parameter. In one exemplary implementation, inverse transform scaling and dequantization may be combined. Alternatively, customized quantization tables may be used and signaled from the encoder to the decoder, eg in a bitstream. Quantization is a lossy operation in which the loss increases as the size of the quantization steps increases.

Embodiments of video encoder 20 (respectively, quantizer 208) may be configured to output quantization parameters (QPs), for example, directly or encoded via entropy encoder 270, such that, for example, video decoder 30 may receive and apply quantization parameters for decoding.

Inverse quantization

The inverse quantizer 210 is configured to apply the inverse quantization of the quantizer 208 to the quantized coefficients to obtain the dequantized coefficients 211, for example by applying an inverse quantization scheme to that applied by the quantizer 208 based on or using the same quantization step size as the block 208 quantization. The dequantized coefficients 211 may also be referred to as the dequantized residual coefficients 211 and correspond - although they are usually not identical to the transform coefficients due to quantization loss - to the transform coefficients 207.

Reverse transformation

The inverse transform processor 212 is configured to apply an inverse transform with respect to the transform applied by the transform processor 206, such as an inverse discrete cosine transform (DCT) or an inverse discrete sine transform (DST) or other inverse transforms to obtain a reconstructed residual block 213 (or corresponding dequantized coefficients 213) in the sample area. The recovered residual block 213 may also be referred to as a transformation block 213 .

Recovery

Reconstructor 214 (e.g., adder or adder 214) is configured to add transform block 213 (i.e., reconstructed residual block 213) to predictor 265 to obtain reconstructed block 215 in the sample domain, such as by sample-by-sample addition. - sample values of the reconstructed residual block 213 and sample values of the prediction block 265 .

Filtration

The loop filter block 220 (or "loop filter" 220 for short) is configured to filter the reconstructed block 215 to obtain a filtered block 221, or more generally to filter the reconstructed samples to obtain filtered samples. The loop filter unit, for example, is configured to smooth out pixel transitions or otherwise improve video quality . The loop filter block 220 may comprise one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a two-sided filter, an adaptive loop filter (ALF), anti-aliasing, enhancement filters. sharpening or joint filters, or any combination of them. Although the loop filter block 220 is shown in FIG. 2 as a loop filter, in other configurations, the loop filter block 220 may be implemented as a post loop filter. The filtered block 221 may also be referred to as the filtered reconstructed block 221.

Embodiments of video encoder 20 (respectively, loop filter unit 220) may be configured to output loop filter parameters (such as sample-adaptive offset information), for example, directly or encoded through entropy encoding unit 270, such that, for example, decoder 30 may receive and apply similar loop filter parameters or appropriate loop filters for decoding.

Decoded picture buffer

The decoded picture buffer (DPB) 230 may be a memory that stores reference pictures or, more generally, reference picture data for encoding video data by video encoder 20. DPB 230 may be formed by any of a variety of memories, such as dynamic random access memory ( DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The decoded picture buffer (DPB) 230 may be configured to store one or more filtered blocks 221. The decoded picture buffer 230 may be further configured to store other previously filtered blocks, such as previously reconstructed and filtered blocks 221, the same current picture, or different images, for example, previously restored images, and can provide complete previously restored ones, i.e. decoded, pictures (and corresponding reference blocks and samples) and/or partially reconstructed current picture (and corresponding reference blocks and samples), for example, for inter prediction. The decoded picture buffer (DPB) 230 may also be configured to store one or more unfiltered reconstructed blocks 215, or more generally unfiltered reconstructed samples, such as if the reconstructed block 215 is not filtered by the loop filter block 220, or any other further processed version of the reconstructed blocks. or samples.

Mode selection (separation and prediction)

The mode selection unit 260 comprises a splitter 262, an inter prediction unit 244, and an intra prediction unit 254, and is configured to receive or obtain original picture data, such as the original block 203 (the current block 203 of the current picture 17), and reconstructed picture data, such as filtered and /or unfiltered reconstructed samples or blocks of the same (current) picture and/or from one or a plurality of previously decoded pictures, for example from decoded picture buffer 230 or other buffers (for example, a linear (line) buffer, not shown). The reconstructed picture data is used as reference picture data for prediction, such as inter prediction or intra prediction, to obtain prediction block 265 or predictor 265.

The mode selector 260 may be configured to determine or select a split for the current block prediction mode (including no split) and prediction mode (eg, intra or inter prediction mode) and generate a corresponding prediction block 265 that is used to calculate the residual block 205 and to restore the restored block 215.

Embodiments of mode selector 260 may be configured to select a division and prediction mode (e.g., those supported by or available to mode selector 260) that provide the best match, or in other words, minimum residual (minimum residual means best compression for transmission or storage), or minimum signaling overhead (lowest signaling overhead means better compression for transmission or storage), or which considers or balances both. The mode selector 260 may be configured to determine the division and prediction mode based on distortion ratio optimization (RDO), that is, selecting a prediction mode that provides minimal distortion of the coefficient. Terms such as "best", "minimum", "optimal", etc. in this context does not necessarily refer to the all-encompassing "best", "minimum", "optimum", etc., but may also refer to the fulfillment of a selection or termination criterion, such as when a value exceeds or falls below a threshold value, or other limitations, potentially leading to "sub-optimal selection" but reducing complexity and processing time.

In other words, partitioner 262 may be configured to partition block 203 into smaller block partitions or subblocks (which form blocks again), such as iteratively using quadtree partitioning (QT), binary partitioning (BT), or ternary tree partitioning (TT). ) or any combination thereof, and performing, for example, prediction for each of the block or subblock divisions, wherein the mode selection comprises selecting a tree structure of the block to be divided 203, and the prediction modes are applied to each of the block or subblock divisions.

The following explains in more detail the division (for example, by the division block 260) and the prediction processing (by the inter prediction block 244 and the intra prediction block 254) performed by the exemplary video encoder 20.

Separation

Partitioner 262 may divide (or split) the current block 203 into smaller partitions, such as smaller square or rectangular size blocks. These smaller blocks (which may also be referred to as sub-blocks) may be further divided into even smaller divisions. This is also called tree splitting or hierarchical tree splitting, in which a root box, such as at root level 0 of the tree (hierarchy level 0, depth 0), can be split recursively, such as split into two or more boxes of the next lower level of the tree, such as nodes at tree level 1 (hierarchy level 1, depth 1), and these blocks can again be divided into two or more blocks of the next lower level, for example, tree level 2 (hierarchy level 2, depth 2), etc. until the partition is complete, for example due to the fulfillment of a termination criterion, such as reaching the maximum tree depth or minimum block size. Blocks that are not further separated are also called leaf blocks or leaf nodes of the tree. A tree using two-partitioning is called a binary tree (BT), a tree using three-partitioning is called a ternary tree (TT), and a tree using four-partitioning is called a quadtree (QT).

As mentioned earlier, the term "block" as used herein can be a part, in particular a square or rectangular part of an image. With respect to, for example, HEVC and VVC, a block may be either a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transformation unit (TU) and/or corresponding blocks, such as a coding tree block (CTB). ), a coding block (CB), a transform block (TB), or a prediction block (PB).

For example, a coding tree unit (CTU) can be either a luminance sample CTB, two corresponding chrominance sample CTBs of a picture that has three sample arrays, or a monochrome picture sample CTB or a picture that is encoded using three separate color planes and the syntax structures used for coding samples. Accordingly, a coding tree block (CTB) may be an N×N block of samples for some value of N, such that dividing a component by a CTB is a split. A coding unit (CU) can be either a luminance sample coding unit, two corresponding chrominance sample coding units of a picture that has three sample arrays, or a monochrome picture or picture sample coding unit that is encoded using three separate color planes and the syntax structures used. for coding samples. Accordingly, a coding block (CB) may be an M×N block of samples for some values of M and N, so dividing the CTB into coding blocks is a split.

In embodiments, for example according to HEVC, a coding tree unit (CTU) may be partitioned into CUs using a quadtree structure designated as a coding tree. The decision whether to encode a picture area using out-of-picture (temporal) or in-picture (spatial) prediction is made at the CU layer. Each CU may be further partitioned into one, two, or four PUs, according to the partition type per PU. Within one PU, the same prediction process is applied, and the relevant information is passed to the decoder based on the PU. After obtaining the residual block by applying a prediction process based on the division type to the PU, the CU may be divided into transform units (TU) according to another quadtree structure similar to the coding tree for the CU.

In embodiments, for example, in accordance with the newest video encoding standard currently being developed, called universal video coding (VVC), for example, a combined quadtree-binary tree (QTBT) split is used to split a coding block. In the block structure of QTBT, the CU can be either square or rectangular. For example, a coding tree unit (CTU) is first divided into a quadtree structure. The leaf nodes of the quadtree are further separated by a binary tree or a ternary (or ternary) tree structure. The leaf nodes of the partitioning tree are called coding units (CUs) and this segmentation is used for prediction and transformation processing without any additional partitioning. This means that CU, PU and TU have the same block size in the QTBT encoding block structure. In parallel, multiple partitioning, such as ternary tree partitioning, can be used along with the block structure of QTBT.

In one example, mode selector 260 of video encoder 20 may be configured to perform any combination of the separation methods described herein.

As described above, video encoder 20 is configured to determine or select the best or optimal prediction mode from a set of (eg, predefined) prediction modes. The set of prediction modes may include, for example, intra prediction modes and/or inter prediction modes.

Internal prediction

The intra prediction mode set may comprise 35 different intra prediction modes, such as non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in HEVC, or may comprise 67 different intra prediction modes such as , non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in VVC.

The intra prediction block 254 is configured to use the recovered samples of adjacent blocks of the same current picture to generate the intra prediction block 265 according to the intra prediction mode of the intra prediction mode set.

Intra prediction block 254 (or mode selector 260 more generally) is further configured to output intra prediction parameters (or information indicative of the selected intra prediction mode for the block in general) to entropy encoding block 270 in the form of syntax elements 266 for including the encoded picture in the data 21 so that, for example, the video decoder 30 can receive and use the prediction parameters for decoding.

external prediction

The set of (possible) inter prediction modes depends on the available reference pictures (i.e. previous, at least partially decoded pictures, e.g. stored in DBP 230) and other inter prediction parameters, e.g. search windows around the area of the current block, the reference picture to find the most matching reference block, and/or, for example, whether pixel interpolation is applied, such as half-pixel (half/semi-pel) and/or quarter-pel (quarter-pel) interpolation or not.

In addition to the aforementioned prediction modes, a skip mode and/or a direct mode may be applied.

Inter prediction block 244 may include a motion estimator (ME) and a motion compensation (MC) block (both not shown in FIG. 2). The motion estimator may be configured to receive or obtain a picture block 203 (the current picture block 203 of the current picture 17) and a decoded picture 231, or at least one or a plurality of previously reconstructed blocks, such as reconstructed blocks of one or a plurality of other/different previously decoded pictures 231, for motion evaluation. For example, the video sequence may contain the current picture and previously decoded pictures 231 or, in other words, the current picture and previously decoded pictures 231 may be part of or form a sequence of pictures that make up the video sequence.

Encoder 20 may, for example, be configured to select a reference block from a plurality of reference blocks of the same or different pictures from a plurality of other pictures and provide a reference picture (or reference picture index) and/or offset (spatial offset) between position (x, y coordinates ) of the reference block and the position of the current block as inter-prediction parameters to the motion estimation block. This displacement is also called motion vector (MV).

The motion compensation block is configured to obtain, for example, receive an inter prediction parameter and perform inter prediction based on or using the inter prediction parameter to obtain the inter prediction block 265 . The motion compensation performed by the motion compensator may include deriving or generating a prediction block based on the motion vector/block determined by the motion estimation, possibly performing subpixel-accurate interpolations. Interpolation filtering can generate additional pixel samples from known pixel samples, thus potentially increasing the number of candidate prediction blocks that can be used to encode a picture block. After receiving the motion vector for the PU of the current picture block, the motion compensator may locate the prediction block pointed to by the motion vector in one of the reference picture lists.

The motion compensation block may also generate syntax elements associated with video blocks and slices for use by video decoder 30 when decoding picture blocks of a video slice. In addition or alternatively, for slices and corresponding syntax elements, groups of tiles and/or tiles and corresponding syntax elements can be generated or used.

Entropy encoding

Entropy encoding unit 270 is configured to apply, for example, an entropy encoding algorithm or scheme (for example, variable length coding (VLC) scheme, context adaptive VLC (CAVLC) scheme, arithmetic coding scheme, binarization, context adaptive binary arithmetic coding ( CABAC) syntax-based context adaptive binary arithmetic coding (SBAC), probability interval division entropy coding (PIPE), or other entropy coding method or methodology) or traversal (no compression) with respect to quantized coefficients 209, inter prediction parameters, parameters intra prediction, loop filter parameters, and/or other syntax elements to obtain encoded picture data 21 that can be output via output 272, for example in the form of an encoded bitstream 21, so that, for example, video decoder 30 can receive and use these parameters for decoding. The encoded bitstream 21 may be transmitted to video decoder 30 or stored in memory for later transmission or retrieval by video decoder 30.

Other changes to the structure of video encoder 20 may be used to encode the video stream. For example, non-transform encoder 20 may quantize the residual signal directly without transform processing block 206 . In another implementation, encoder 20 may have quantizer 208 and inverse quantizer 210 combined into a single block.

Decoder and decoding method

Fig. 3 shows an example of a video decoder 30 that is configured to implement the techniques of the present application. Video decoder 30 is configured to receive encoded picture data 21 (e.g., encoded bitstream 21), e.g. represent picture blocks of an encoded video slice (and/or tiles or groups of tiles) and associated syntax elements.

In the example in FIG. 3, the decoder 30 includes an entropy decoder 304, an inverse quantizer 310, an inverse transform processor 312, a reconstruction block 314 (e.g., an adder 314), a loop filter 320, a decoded picture buffer (DPB) 330, a mode application block 360, a block 344 external prediction and block 354 intra prediction. Inter prediction block 344 may be or include a motion compensation block. Video decoder 30 may, in some examples, perform a decoding step that is generally the reverse of the encoding step described with respect to video encoder 100 in FIG.

As described with respect to the encoder 20, the inverse quantization block 210, the inverse transform processing block 212, the reconstruction block 214, the loop filter 220, the decoded picture buffer (DPB) 230, the inter prediction block 344, and the intra prediction block 354 are also related to the formation of an “embedded decoder » of the video encoder 20. Accordingly, the inverse quantizer 310 may be identical in function to the inverse quantizer 110, the inverse transform processing unit 312 may be identical in function to the inverse transform processing unit 212, the restorer 314 may be identical in function to the restorer 214, loop filter 320 may be identical in function to loop filter 220, and decoded picture buffer 330 may be identical in function to decoded picture buffer 230. Therefore, the explanations provided for the respective blocks and functions of the video encoder 20 apply, respectively, to the corresponding blocks and functions of the video decoder 30.

Entropy decoding

The entropy decoding unit 304 is configured to parse the bitstream 21 (or, in general, the encoded picture data 21) and perform, for example, entropy decoding on the encoded picture data 21 to obtain, for example, quantized coefficients 309 and/or decoded encoding parameters (not shown in FIG. 3), e.g., any or all of the inter prediction parameters (e.g., reference picture index and motion vector), intra prediction parameter (e.g., index or intra prediction mode), transform parameters, quantization parameters, contour parameters filters and/or other syntax elements. Entropy decoding unit 304 may be configured to apply decoding algorithms or schemes corresponding to coding schemes as described in relation to entropy encoding unit 270 of encoder 20. Entropy decoding unit 304 may be further configured to provide inter prediction parameters, an intra prediction parameter, and/ or other syntax elements to the mode application block 360 and other parameters to other blocks of the decoder 30. Video decoder 30 may receive syntax elements at the video slice level and/or the video block level. In addition to or as an alternative to slices and corresponding syntax elements, groups of tiles and/or tiles and corresponding syntax elements may be accepted and/or used.

Inverse quantization

The inverse quantizer 310 may be configured to receive quantization parameters (QPs) (or inverse quantization related information in general) and quantized coefficients from the encoded picture data 21 (e.g., by parsing and/or decoding, e.g., by entropy decoding block 304) and applying, based on the quantization parameters, inverse quantization to the decoded quantized coefficients 309 to obtain dequantized coefficients 311, which may also be referred to as transform coefficients 311. The inverse quantization process may include using a quantization parameter determined by video encoder 20 for each video block in a video slice (or tile or group of tiles) to determine the amount of quantization and, likewise, the amount of inverse quantization to be applied.

Reverse transformation

The inverse transform processor 312 may be configured to receive the dequantized coefficients 311, also referred to as transform coefficients 311, and apply a transform to the dequantized coefficients 311 in order to obtain reconstructed residual blocks 213 in the sample domain. The restored residual blocks 213 may also be referred to as transform blocks 313 . The transformation may be an inverse transformation, such as inverse DCT, inverse DST, inverse integer transformation, or a conceptually similar inverse transformation process. The inverse transform processing unit 312 may be further configured to receive transform parameters or corresponding information from the encoded picture data 21 (eg, by parsing and/or decoding, eg by entropy decoding unit 304) to determine the transform to be applied to dequantized coefficients 311.

Recovery

Reconstructor 314 (eg, adder or adder 314) may be configured to add the reconstructed residual block 313 to the prediction block 365 to obtain a reconstructed block 315 in the sample domain, for example, by adding the sample values of the reconstructed residual block 313 and the sample values of block 365 predictions.

Filtration

The loop filter block 320 (either in the encoding loop or after the encoding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321, for example, to smooth pixel transitions or otherwise improve video quality. The loop filter block 320 may comprise one or more loop filters, such as a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as a two-sided filter, an adaptive loop filter (ALF), anti-aliasing, enhancement filters. sharpening or joint filters, or any combination of them. Although the loop filter block 320 is shown in FIG. 3 as a loop filter, in other configurations, the loop filter block 320 may be implemented as a post loop filter.

Decoded picture buffer

The decoded video blocks 321 of the picture are then stored in the decoded picture buffer 330, which stores the decoded pictures 331 as reference pictures for subsequent motion compensation for other pictures and/or for display output, respectively.

The decoder 30 is configured to output the decoded picture 311, for example, via output 312 for presentation or viewing to a user.

Prediction

The inter prediction block 344 may be identical to the inter prediction block 244 (in particular, the motion compensation block), and the intra prediction block 354 may be identical to the inter prediction block 254 in function, and makes split or split decisions and performs prediction based on split parameters and /or prediction or corresponding information received from the encoded picture data 21 (for example, by parsing and/or decoding, for example, by entropy decoding block 304). Mode apply block 360 may be configured to perform prediction (intra or inter prediction) for each block based on reconstructed pictures, blocks, or corresponding samples (filtered or unfiltered) to obtain prediction block 365.

When a video slice is encoded as an intra-coded (I) slice, the intra prediction block 354 of the mode applying block 360 is configured to generate a prediction block 365 for a picture block of the current video slice based on the signaled intra prediction mode and data from the previously decoded blocks of the current picture. When a video image is encoded as an inter-coded (i.e., B or P) slice, the inter prediction block 344 (e.g., motion compensation block) of the mode apply block 360 is configured to create prediction blocks 365 for the video block of the current video slice based on the motion vectors and others. syntax elements received from block 304 entropy decoding. For inter prediction, prediction blocks may be created from one of the reference pictures within one of the reference picture lists. Video decoder 30 may construct the key frame lists, List 0 and List 1 using default construction methods based on the reference pictures stored in the DPB 330. The same or similar may apply for or through embodiments using tile groups (e.g., video tile groups ) and/or tiles (eg, video tiles) in addition to or alternatively to slices (eg, video slices), for example, video may be encoded using groups I, P, or B of tiles and/or tiles.

The mode applyer 360 is configured to determine prediction information for a video block of the current video slice by parsing motion vectors or related information and other syntax elements, and uses the prediction information to generate prediction blocks for the current video block being decoded. For example, the mode applicator 360 uses some of the received syntax elements to determine the prediction mode (e.g., intra or inter prediction) used to encode the video blocks of the video slice, the type of inter prediction slice (e.g., B slice, P slice, or GPB slice) , construction information for one or more reference picture lists for the slice, motion vectors for each inter-coded video block of the slice, inter-prediction status for each inter-coded video block of the slice, and other information for decoding video blocks in the current video slice. The same or similar may apply to or through embodiments using groups of tiles (eg, video tile groups) and/or tiles (eg, video tiles) in addition to or alternatively to slices (eg, video slices), e.g., video may be encoded with using groups of I, P or B tiles and/or tiles.

Embodiments of video decoder 30, as shown in FIG. 3 may be configured to split and/or decode a picture using slices (also referred to as video slices), wherein the picture may be divided into or encoded using one or more slices (generally non-overlapping), and each slice may contain one or more blocks (for example, CTU).

Embodiments of video decoder 30 shown in FIG. 3 may be configured to split and/or decode a picture using tile groups (also referred to as video tile groups) and/or tiles (also referred to as video tiles), wherein the picture may be divided into or decoded using one or more tile groups ( usually non-overlapping) and each group of tiles may contain, for example, one or more blocks (for example, CTU) or one or more tiles, each tile, by way of example, may have a rectangular shape and may contain one or more blocks (for example, , CTU) such as full or partial units.

Other variants of video decoder 30 may be used to decode encoded picture data 21 . For example, decoder 30 may produce an output video stream without loop filtering unit 320 . For example, a non-transform-based decoder 30 may perform inverse quantization of the residual signal directly without the inverse transform processing unit 312 for some blocks or frames. In another implementation, video decoder 30 may have an inverse quantizer 310 and an inverse transform processor 312 combined into one block.

It should be understood that in encoder 20 and decoder 30, the processing result of some current stage may be further processed and then output to the next stage. For example, after interpolation filtering, motion vector acquisition, or contour filtering, an additional operation such as Clip (truncation) or offset may be performed on the result of the interpolation filtering, motion vector acquisition, or contour filtering processing.

It should be noted that additional operations may be applied to the current block's inferred motion vectors (including, but not limited to, affine mode checkpoint motion vectors, sub-block motion vectors in affine, planar, ATMVP modes, temporal motion vectors, and the like). For example, the motion vector value is limited to a predefined range according to its representing bit. If the representing bit of the motion vector is bitDepth (bit depth), then the range is -2^(bitDepth-1)~2^(bitDepth-1)-1, where "^" means exponentiation. For example, if bitDepth is 16, the range is -32768~32767; if bitDepth is 18, the range is -131072~131071. For example, the output motion vector value (e.g., MVs of four 4x4 subblocks in one 8x8 block) is limited so that the maximum difference between integer parts of the MVs of four 4x4 subblocks does not exceed N pixels, for example, is no more than 1 pixel. Here are two ways to limit the motion vector according to bitDepth.

Method 1: Removing the MSB (Most Significant Bit) of the Overflow via Stream Operations

ux= ( mvx+2 ^bitDepth ) % 2 ^bitDepth (1)

mvx = ( ux >= 2 ^bitDepth-1 ) ? (ux − 2 ^bitDepth ) : ux(2)

uy= ( mvy+2 ^bitDepth ) % 2 ^bitDepth (3)

mvy = ( uy >= 2 ^bitDepth-1 ) ? (uy − 2 ^bitDepth ) : uy (4)

where mvx is the horizontal component of the motion vector of the image block or sub-block, mvy is the vertical component of the motion vector of the image block or sub-block, and ux and uy indicate an intermediate value;

For example, if the value of mvx is -32769, after formulas (1) and (2) are applied, the resulting value is 32767. In a computer system, decimal numbers are stored as two's complement. The two's complement of -32769 is 1.0111,1111,1111,1111 (17 bits), then the MSB is discarded, so the resulting two's complement is 0111.1111.1111.1111 (decimal is 32767), which is the same as the output from the application of formulas (1) and (2).

ux= ( mvpx + mvdx +2 ^bitDepth ) % 2 ^bitDepth (5)

mvx = ( ux >= 2 ^bitDepth-1 ) ? (ux − 2 ^bitDepth ) : ux(6)

uy= ( mvpy + mvdy +2 ^bitDepth ) % 2 ^bitDepth (7)

mvy = ( uy >= 2 ^bitDepth-1 ) ? (uy − 2 ^bitDepth ) : uy (8)

Operations can be applied during the summation of mvp and mvd, as shown in formulas (5) to (8).

Method 2: Removing the MSB Overflow by Truncating the Value

vx = Clip3(-2 ^bitDepth-1 , 2 ^bitDepth-1 -1, vx)

vy = Clip3(-2 ^bitDepth-1 , 2 ^bitDepth-1 -1, vy)

where vx is the horizontal component of the motion vector of the image block or sub-block, vy is the vertical component of the motion vector of the image block or sub-block; x, y, and z respectively correspond to the three input values of the MV clipping process, and the Clip3 function definition is as follows:

Clip3(x, y, z) =

(where otherwise - otherwise).

Fig. 4 is a schematic block diagram of a video encoding apparatus 400 according to an embodiment of the present disclosure. The video encoding device 400 is suitable for implementing the disclosed embodiments that are described herein. In an embodiment, video encoding device 400 may be a decoder, such as video decoder 30 of FIG. 1A, or an encoder such as video encoder 20 of FIG. 1A.

The video encoder 400 includes input ports 410 (or input ports 410) and receiver units (Rx) 420 for receiving data; a processor, logical unit, or central processing unit (CPU) 430 for processing data; blocks 440 (Tx) transmitter and output ports 450 (or ports 450 output) for data transmission; and a memory 460 for storing data. The video encoder 400 may also include optical-to-electrical (OE) and electrical-to-optical (EO) components connected to input ports 410, receiver units 420, transmitter units 440, and output ports 450 to provide input or output of optical or electrical signals.

Processor 430 is implemented in hardware and software. Processor 430 may be implemented as one or more CPU chips, cores (eg, as a multi-core processor), FPGA, ASIC, and DSP. The processor 430 communicates with input ports 410, receiver units 420, transmitter units 440, output ports 450, and memory 460. Processor 430 includes an encoding module 470. Encoding module 470 implements the disclosed embodiments described above. For example, encoding module 470 implements, processes, prepares, or provides various encoding operations. Therefore, the inclusion of the encoding module 470 provides a significant improvement in the functionality of the video encoding device 400 and allows the transformation of the video encoding device 400 to another state. Alternatively, encoding module 470 is implemented as instructions stored in memory 460 and executed by processor 430.

Memory 460 may comprise one or more disks, tape drives, and solid state drives, and may be used as an overflow storage device for storing programs when such programs are selected for execution, and for storing instructions and data that are read during program execution. Memory 460 may be, for example, volatile and/or non-volatile and may be Read Only Memory (ROM), Random Access Memory (RAM), Ternary Associative Memory (TCAM) and/or Static Random Access Memory (SRAM).

Fig. 5 is a simplified block diagram of hardware 500 that can be used as one or both of source device 12 and sink device 14 of FIG. 1 according to an exemplary embodiment .

Processor 502 in apparatus 500 may be a central processing unit. Alternatively, processor 502 may be any other type of device or a plurality of devices capable of manipulating or processing information that currently exists or will be developed in the future. Although the disclosed implementations may be practiced with a single processor, as shown with processor 502, for example, speed and efficiency advantages may be achieved using more than one processor.

The memory 504 in the hardware 500 may be a read only memory (ROM) or a random access memory (RAM) device in some implementation. Memory 504 may be any other suitable type of storage device. Memory 504 may include code and data 506 accessed by processor 502 using bus 512. Memory 504 may further include an operating system 508 and application programs 510, application programs 510 including at least one program, which allows the processor 502 to perform the methods described herein. For example, application programs 510 may include applications 1 through N, which further include a video encoding application that performs the methods described herein.

Hardware 500 may also include one or more output devices, such as a display 518. Display 518 may be, in one example, a touch-sensitive display that combines a display with a touch-sensitive element capable of receiving touch inputs (by touch). Display 518 may be coupled to processor 502 via bus 512.

Although depicted here as a single bus, bus 512 of apparatus 500 may be composed of multiple buses. In addition, secondary storage 514 may be directly connected to other hardware components 500, or may be accessible via a network, and may comprise a single pluggable unit, such as a memory card, or multiple units, such as a plurality of memory cards. Thus, hardware 500 can be implemented in a wide variety of configurations.

State of the art about encoding with intra prediction mode

In the current version of VTM 4.0, a list of 6-MPM (most probable modes) is created for encoding with intra prediction mode. The MPM list is introduced to reduce the number of bits required to encode the intra prediction mode of the current block. The 6-MPM list is built based on the intra prediction modes of neighboring blocks of the current block. When the intra prediction mode of the current block enters the MPM list, an index is encoded rather than the actual mode, which may use fewer bits. When the intra prediction mode of the current block does not fall into the MPM list, truncated binarization is used to encode the intra prediction mode of the current block. The MPM list provides good prediction for the intra prediction mode of the current block.

List of MPMs for multiple reference strings (MRLs).

In VTM 4.0, the multiple reference row (MRL) coding tool can use one of several adjacent reference rows to predict the samples of the current block. When the MRL row index value is 0 (ie, the nearest adjacent reference row is used, ie, reference row 0, as shown in Fig. 6), a normal list of 6 MPMs containing planar and DC is used. When the MRL row index value is not 0, a list of 6-MPM is used, excluding planar (ie, value 0) and DC (ie, value 1).

List of MPMs for Internal Sub-Section Encoding (ISP) Mode

The Internal Sub-Section Encoding (ISP) mode is a tool recently adopted in VTM4.0 (JVET-M0102). Blocks with intra luminance prediction are divided vertically or horizontally into 2 or 4 subsections, depending on block sizes, as shown in Table 1. FIG. 7 and 8 show examples of two possibilities. All subsections satisfy the condition of having at least 16 samples.

Table 1: Number of subsections by block size

Block size Number of subsections Not divided And 2 All other cases 4

When the internal subkey encoding mode is enabled, VTM 4.0 uses a different list of MPMs, with the exception of DC mode. The intra-subsection mode may be applied when the index value of the multiple reference rows is 0 (ie, MRL is not applied to the current intra prediction block). All subpartitions share one intra prediction mode, so the MPM list is created once per intrablock and shared by all subpartitions.

The construction of the MPM may depend on the partitioning mode of the ISP. Two partitioning modes are defined: horizontal and vertical. They are shown in Fig. 7 and 8 where the horizontal split/split is shown at the top and the vertical split/split is shown at the bottom.

The following table 2 lists the characteristics of the 3 MPM list used in VTM 4.0.

Table 2: Various 6-MPMs used in VTM 4.0 under different circumstances

MRL Configuration ISP configuration Characteristics of the MPM List - 6 MPM, but excluding planar mode and DC mode 1 6 MPM but excluding DC mode 0 6 MPM including planar mode and DC mode

Background information on CABAC context modeling/coding.

As shown in FIG. 9, CABAC encoding consists of binarization, context modeling, and binary arithmetic encoding. Binarization maps syntax elements to binary characters (bins). Context modeling estimates the probability of each non-bypassed (ie regularly encoded) bin based on some particular context. Finally, binary arithmetic encoding compresses bins to bits according to the estimated probability.

By decomposing each value of a non-binary syntax element into a sequence of bins, the additional processing of each value of a bin in CABAC depends on the appropriate encoding mode decision, which can be chosen as either regular or bypassed mode. The latter is chosen for bins that are assumed to be evenly distributed and for which, therefore, the entire process of regular binary arithmetic encoding (and decoding) is simply ignored. In the regular encoding mode, each bin value is encoded using the regular binary arithmetic encoding mechanism, where the corresponding probability model is determined either by a fixed choice, based on the syntax element type and bin position, or by a bin index (binIdx) in the binarized representation of the syntax element, or adaptively selected from two or more probabilistic models depending on the associated additional information.

The choice of a probabilistic model is called context modeling. Each probability model in CABAC is addressed using a unique context index (ctxIdx), which is either determined by a fixed assignment or computed by the context inference logic, which in turn specifies that context model.

State of the art on how to determine left and top neighbors.

An encoding with a VVC intra prediction mode depends on the intra prediction modes of its neighboring blocks. Namely, the left and top blocks of the current block. Their position is shown in Fig. 10.

MPM index alarm

A flag (ie, an MPM flag) is used to indicate whether the intra prediction mode of the current block is in the MPM list or not. When the MPM flag is set to true (ie, the value is 1), then the intra prediction mode can be determined using the MPM index. The MPM index is encoded using a truncated unary code as shown in the following Table 3 when the length of the MPM list is 6. When the MPM flag is not true, the intra prediction mode of the current block is not in the MPM list, and the mode is encoded using a truncated binary code. Another example of an MPM index when the MPM list consists of 5 modes is shown in Table 4. The bin string for the encoding index is called the truncated unary code (not truncated binary for non-MPM mode encoding). Note that for the truncated unary code, bins 0 and 1 are interchangeable without loss of generality. In one example, the MPM index when the MPM list consists of 5 modes may also be encoded as shown in Table 5.

Table 3: Encoding of the MPM flag and MPM index with 6 entries in the MPM list

Intra prediction modes MPM Flag MPM Index MPM index bean string MPM Modes (6) 1 0 0 1 10 2 110 3 1110 4 11110 5 11111 0 Truncated Binary (TB)

Table 4: Encoding of the MPM flag and MPM index when there are 5 entries in the MPM list

Intra prediction modes MPM Flag MPM index MPM index bean string MPM modes (5) 1 0 0 1 10 2 110 3 1110 4 1111 0 Truncated Binary (TB)

Table 5: Encoding of the MPM flag and MPM index when there are 5 entries in the MPM list, and an alternative codeword for the MPM index

Intra prediction modes MPM Flag MPM index MPM index bean string MPM modes (5) 1 0 1 1 01 2 001 3 0001 4 0000 0 Truncated Binary (TB)

Reference information about planar, DC, horizontal, vertical and angular modes: intra prediction modes: planar (value 0) and DC (value 1). Other intra prediction modes are called angle prediction modes.

In VTM4.0, the following syntax structure shown in Table 6 is used to convey intra prediction mode information.

Table 6: parsing in VTM4.0

if ( intra_luma_ref_idx[ x0 ][ y0 ] == 0 &&
intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 0 ) intra_luma_mpm_flag [ x0] [y0] ae (v) if(intra_luma_mpm_flag[x0][y0]) { intra_luma_mpm_idx[ x0][y0] ae (v) } otherwise intra_luma_mpm_remainder [ x0] [y0] ae (v)

intra_luma_ref_idx[x0][ y0] can have three values: 0, 1 or 2 indicating which reference string is used. When intra_luma_ref_idx[x0][ y0] is absent, it is assumed to be 0.

intra_subpartitions_mode_flag [x0] [ y0] equal to 1 indicates that the current internal encoding unit is sub-partitioned (i.e. ISP mode enabled). intra_subpartitions_mode_flag [x0] [y0] equal to 0 indicates that the current internal encoding unit is not divided into rectangular transform block subpartitions. When intra_subpartitions_mode_flag [x0] [y0] is absent, it is assumed to be 0.

The intra_luma_mpm_flag[ x0][y0], intra_luma_mpm_idx[ x0][y0], and intra_luma_mpm_remainder [x0][y0] syntax elements define the intra-prediction mode for luma samples.

intra_luma_mpm_flag [x0] [y0] indicates whether the intra prediction mode of the current block is deduced from the MPM list or not. If the valueintra_luma_mpm_flag [x0] [y0] is 1, this indicates that the intra prediction mode of the current block is removed from the MPM list; If the valueintra_luma_mpm_flag [x0] [y0] is 0, this indicates that the intra prediction mode of the current block is not deduced from the MPM list. Whenintra_luma_mpm_flag [x0] [y0] is missing, it is assumed to be 1.intra_luma_mpm_flag [x0] [y0] matches MPM flag in Table 3.

intra_luma_mpm_idxcorresponds MPM index in Table 3 and uses a truncated unary code as shown in Table 3.

intra_luma_mpm_remainder represents a truncated mode binary that is not in the MPM list.

Embodiment 1 discloses the intra_luma_planar_flag and ctx syntax elements.

According to one embodiment of the invention, the intra_luma_planar_flag flag is introduced.

If the value of intra_luma_mpm_flag is true (for example, the value is 1), the decoder analyzes the intra_luma_planar_flag to determine whether the intra prediction mode of the current block is a planar mode or not. When the value of intra_luma_planar_flag is 1, the intra prediction mode of the current block is the planar mode. When the value of intra_luma_planar_flag is 0, the intra prediction mode of the current block is NOT a planar mode. If intra_luma_planar_flag is not present (for example, this flag cannot be parsed from the bitstream ifintra_luma_mpm_flagequals 0), the value of intra_luma_planar_flag is output as 0.

If written in the VVC Working Draft specification style, the intra-prediction mode of the current block

IntraPredModeY [xCb] [yCb] is derived by applying the following procedure:

If intra_luma_mpm_flag [xCb] [yCb] is 1,

If intra_luma_planar_flag [xCb] [yCb] is 1, intraPredModeY [xCb] [yCb] is set to INTRA_PLANAR.

Otherwise, IntraPredModeY[xCb][yCb] is set to candModeList[intra_luma_mpm_idx[xCb][yCb]].

If the intra prediction mode is not a planar mode (ie, the intra_luma_planar_flag value is 0), the decoder further parses the intra_luma_mpm_idx syntax element.

In one example, the intra_luma_planar_flag value is CABAC context encoded (ie not CABAC bypass encoded) and the decoder parses the flag using the context based on whether the current block is a normal intra block or not.

In one example, the intra_luma_planar_flag value is CABAC context-encoded (i.e., not CABAC bypass-encoded), and the decoder parses the flag using the context based on whether the current block is a multiple reference row (MRL) intra block or not.

In one example, the value of intra_luma_planar_flag is CABAC context-encoded (i.e., not CABAC-bypass-encoded), and the decoder parses the flag using the context based on whether the current box is an intra-block with Intra-Section Encoding (ISP) mode enabled.

In the example, the intra_luma_planar_flag position is placed as in table 7.

Table 7: Syntax with intra_luma_planar_flag

if (intra_luma_ref_idx[x0][y0] == 0 && intra_subpartitions_mode_flag[x0][y0] == 0) intra_luma_mpm_flag [x0] [y0] ae (v) if(intra_luma_mpm_flag[x0][y0]) { intra_luma_planar_flag [x0] [y0] ae (v) if(! intra_luma_planar_flag[ x0][y0]) intra_luma_mpm_idx[ x0][y0] ae (v) } otherwise intra_luma_mpm_remainder [ x0] [y0] ae (v)

Implementation 2: intra_luma_not_planar_flag and ctx

According to one embodiment of the invention, the intra_luma_not_planar_flag syntax element is introduced.

If the value of intra_luma_mpm_flag is true (eg, the value is 1), the decoder analyzes the intra_luma_not_planar_flag to determine whether the intra prediction mode of the current block is planar mode or not. When the value of intra_luma_not_planar_flag is 1, the intra prediction mode of the current block is NOT a planar mode; when the value of intra_luma_not_planar_flag is 0, the intra prediction mode of the current block is the planar mode. If intra_luma_not_planar_flag is not present in the bitstream, the value of intra_luma_not_planar_flag is output as 1. In the example, the intra_luma_not_planar_flag position is placed as in Table 8.

If the intra prediction mode of the current block is not a planar mode (ie, the intra_luma_not_planar_flag value is 1), the decoder further parses the intra_luma_mpm_idx syntax element.

VVC working draft specification style entry, current block intra prediction mode

IntraPredModeY [xCb] [yCb] is derived by applying the following procedure:

If intra_luma_mpm_flag [xCb] [yCb] is 1,

If intra_luma_not_planar_flag [xCb] [yCb] is 0, intraPredModeY [xCb] [yCb] is set to INTRA_PLANAR.

Otherwise, IntraPredModeY[xCb][yCb] is set to candModeList[intra_luma_mpm_idx[xCb][yCb]].

In one example, the value intra_luma_not_planar_flag is CABAC context encoded (ie, not CABAC bypass encoded) and the decoder parses the flag using the context based on whether the current block is a normal intra block or not.

In one example, the intra_luma_not_planar_flag value is encoded in a CABAC context (i.e. not CABAC bypass encoded) and the decoder parses the flag using the context based on whether the current block is a multiple reference row (MRL) intra block or not.

In one example, the intra_luma_not_planar_flag value is CABAC context-encoded (i.e., not CABAC-bypass-encoded), and the decoder parses the flag using the context based on whether the current block is being used by an intra block with an intra subsection encoding (ISP) mode.

Table 8: Syntax with intra_luma_not_planar_flag

if (intra_luma_ref_idx[x0][y0] == 0 && intra_subpartitions_mode_flag[x0][y0] == 0) intra_luma_mpm_flag [x0] [y0] ae (v) if(intra_luma_mpm_flag[x0][y0]) { intra_luma_not_planar_flag [x0] [y0] ae (v) if(intra_luma_not_planar_flag[x0][y0]) intra_luma_mpm_idx[x0][y0] ae (v) } more intra_luma_mpm_remainder [x0] [y0] ae (v)

Embodiment 3: intra_luma_planar_flag and MRL and bypass decoding

According to one embodiment of the invention, the intra_luma_planar_flag flag is introduced.

If the value of intra_luma_mpm_flag is true (for example, the value is 1), the decoder analyzes whether the value of the index reference row ((intra_luma_ref_idx) is 0 or not. If the value of the index reference row is 0, the valueintra_luma_planar_flag analyzed,to determine whether the intra prediction mode of the current block is a planar mode or not. When the valueintra_luma_planar_flag equals 1, mode intra-prediction of the current block is planar mode. When the valueintra_luma_planar_flag equals 0, mode Intra prediction of the current block is NOT a planar mode. Ifintra_luma_planar_flagmissing, valueintra_luma_planar_flag output like 0.

If the intra prediction mode of the current block is not a planar mode (ie, the intra_luma_planar_flag value is 0 ), the decoder further parses the intra_luma_mpm_idx.

In one example, the valueintra_luma_planar_flag encoded with CABAC bypass (ie, not encoded in a CABAC context), and the decoder parses the flag without using any CABAC context.

The position of the intra_luma_planar_flag is shown in Table 9.

Table 9: Syntax with intra_luma_planar_flag and it's coded with bypass

if (intra_luma_ref_idx[x0][y0] == 0 && intra_subpartitions_mode_flag[x0][y0] == 0) intra_luma_mpm_flag [ x0] [y0] ae (v) if(intra_luma_mpm_flag[x0][y0]) { if (intra_luma_ref_idx[x0][y0] == 0) intra_luma_planar_flag [x0] [y0] ae (v) if(! intra_luma_planar_flag[ x0][y0]) intra_luma_mpm_idx[ x0][y0] ae (v) } otherwise intra_luma_mpm_remainder [ x0] [y0] ae (v)

Embodiment 4: intra_luma_not_planar_flag and MRL and bypass decoding

According to one embodiment of the invention, the intra_luma_not_planar_flag flag is introduced.

If the value of intra_luma_mpm_flag is true (for example, the value is 1), the decoder analyzes whether the value of the index reference string is 0 or not. If the index reference row value is 0,intra_luma_not_planar_flag analyzed,to determine whether the intra prediction mode of the current block is a planar mode or not. When the valueintra_luma_not_planar_flag equals 1, mode Intra prediction of the current block is NOT a planar mode. When the valueintra_luma_planar_flag equals 0, mode intra-prediction of the current block is planar mode. Ifintra_luma_not_planar_flagmissing, meaningintra_luma_not_planar_flag output like 1.

If the intra prediction mode of the current block is not a planar mode (that is, the valueintra_luma_not_planar_flag equals 1), decoder additionally analyzesintra_luma_mpm_idx.

In one example, the intra_luma_not_planar_flag value is encoded with CABAC bypass (ie, not encoded by the CABAC context), and the decoder parses the flag without using any CABAC context.

The position of the intra_not_luma_planar_flag is placed as in Table 10.

Table 10: Syntax with intra_luma_not_planar_flag and it is coded with bypass

if (intra_luma_ref_idx[x0][y0] == 0 && intra_subpartitions_mode_flag[x0][y0] == 0) intra_luma_mpm_flag [ x0] [y0] ae (v) if(intra_luma_mpm_flag[x0][y0]) { if (intra_luma_ref_idx[x0][y0] == 0) intra_luma_not_planar_flag [x0] [y0] ae (v) if (intra_luma_not_planar_flag[ x0][y0]) intra_luma_mpm_idx[ x0][y0] ae (v) } otherwise intra_luma_mpm_remainder [ x0] [y0] ae (v)

Embodiment 5: intra_luma_not_planar_flag and MRL and cabac context encoding (the difference from Embodiment 4 is that intra_luma_not_planar_flag is cabac context encoded rather than bypass encoded).

According to one embodiment of the invention, the intra_luma_not_planar_flag flag is introduced.

The decoder parses a value indicating the index of the reference string to be used. When the index reference row value is 0, the decoder parses the intra_luma_mpm_flag value.

If the value of intra_luma_mpm_flag is true (for example, the value is 1), the decoder checks whether the value of the index reference string is 0 or not. If the index reference row value is 0,intra_luma_not_planar_flagis analyzed to determine whether the intra prediction mode of the current block is the Planar mode or not. When the valueintra_luma_not_planar_flag equal to 1, mode Intra prediction of the current block is NOT a planar mode. When the valueintra_luma_not_planar_flag is 0, mode internal prediction current block is planar mode. Ifintra_luma_not_planar_flagmissing, intra_luma_not_planar_flag valueoutputlike 1.

If the intra prediction mode of the current block is not a planar mode (ie, the value of intra_luma_not_planar_flag is equal to 1), the decoder further analyzes intra_luma_mpm_idx.

In one example, the value intra_luma_not_planar_flag is CABAC context encoded (ie, not CABAC bypass encoded) and the decoder parses the flag using the context based on whether the current block is a normal intra block or not.

In one example, the intra_luma_not_planar_flag value is encoded in a CABAC context (i.e. not CABAC bypass encoded) and the decoder parses the flag using the context based on whether the current block is a multiple reference row (MRL) intra block or not.

In one example, the intra_luma_not_planar_flag value is CABAC context-encoded (i.e., not CABAC bypass-encoded), and the decoder parses the flag using the context based on whether the current block is used by an intra block with an intra subsection encoding (ISP) mode.

The position of the intra_not_luma_planar_flag is shown in Table 11.

Table 11: syntax with intra_luma_not_planar_flag and it is encoded by cabac context

if (intra_luma_ref_idx[x0][y0] == 0 && intra_subpartitions_mode_flag[x0][y0] == 0) intra_luma_mpm_flag [ x0] [y0] ae (v) if(intra_luma_mpm_flag[x0][y0]) { if (intra_luma_ref_idx[x0][y0] == 0) intra_luma_not_planar_flag [x0] [y0] ae (v) if (intra_luma_not_planar_flag[ x0][y0]) intra_luma_mpm_idx[ x0][y0] ae (v) } otherwise intra_luma_mpm_remainder [ x0] [y0] ae (v)

For example,

Add intra_luma_not_planar_flag

if(intra_luma_mpm_flag[x0][y0]) { if (intra_luma_ref_idx[x0][y0] == 0) intra_luma_not_planar_flag [ x0] [y0] ae (v) if(intra_luma_not_planar_flag[x0][y0]) intra_luma_mpm_idx[ x0][y0] ae (v) } otherwise intra_luma_mpm_remainder [ x0] [y0] ae (v) }

intra_luma_mpm_flag[][] FL cMax=1 intra_luma_not_planar_flag[][] FL cMax=1 intra_luma_mpm_idx[][] Tr cMax=4, cRiceParam=0

When intra_luma_not_planar_flag [x0] [y0] is absent, it is assumed to be 1.

Table 12 below shows an example of assigning ctxInc to syntax elements with context-encoded beans.

Table 12 - Assignment of ctxInc to syntax elements with context-coded beans Syntax element binIdx 0 1 2 3 4 >= 5 intra_subpartition_split_flag 0 nd nd nd nd nd intra_luma_mpm_flag[][] 0 nd nd nd nd nd intra_luma_not_planar_flag[][] ! intra_subpartitions_mode_flag nd nd nd nd nd intra_luma_mpm_idx[][] detour detour detour detour nd nd intra_luma_mpm_remainder[][] detour detour detour detour detour detour

In an embodiment, the MPM list is created according to the following process.

8.4.2 Inference process for intra luminance prediction mode

The input to this process is the following:

a luminance location ( xCb , yCb ) indicating the top left sample of the current luminance encoding block relative to the top left luminance sample of the current picture,

cbWidth variable indicating the width of the current encoding block in brightness samples,

cbHeight variable indicating the height of the current encoding block in brightness samples.

In this process, the IntraPredModeY [xCb] [yCb] intraprediction luminance mode is output.

Table 13 defines the value for intraprediction mode IntraPredModeY [xCb] [yCb] and associated names.

Table 13 - Intra prediction mode specification and associated names

Intra prediction mode Related names 0 INTRA_PLANAR 1 INTRA_DC 2..66 INTRA_ANGULAR2..INTRA_ANGULAR66 81..83 INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

NOTE: The intra prediction modes INTRA_LT_CCLM, INTRA_L_CCLM and

INTRA_T_CCLM apply only to chrominance components.

IntraPredModeY [xCb] [yCb] is output like this:

If intra_luma_not_planar_flag [xCb] [yCb] is 1, the following ordered steps apply:

1. Adjacent locations ( xNbA, yNbA ) and ( xNbB, yNbB ) are set to ( xCb − 1, yCb+cbHeight − 1 ) and ( xCb+cbWidth − 1, yCb − 1 ), respectively.

2. If X is replaced by A or B, the candIntraPredModeX variables are output as follows:

The block availability inference process is called: The neighbor block availability check process is called with location (xCurr, yCurr) set to (xCb, yCb) and neighbor location (xNbY, yNbY) set to (xNbX, yNbX) as input, and the output is assigned to availableX.

Candidate intra prediction mode candIntraPredModeX is derived as follows:

If one or more of the following conditions is true, candIntraPredModeX is set to INTRA_PLANAR.

The value of availableX is FALSE.

CuPredMode [xNbX] [yNbX] is not equal to MODE_INTRA and ciip_flag [xNbX] [yNbX] is not equal to 1.

pcm_flag [xNbX] [yNbX] is 1.

X is equal to B and yCb − 1 is less than ( ( yCb >> CtbLog2SizeY ) << CtbLog2SizeY ).

Otherwise, CandIntraPredModeX is set to IntraPredModeY [xNbX] [yNbX].

where X is replaced by either adjacent block A or B, the luma location (xCb, yCb) defining the top left sample of the current luminance encoding block relative to the top left luma sample of the current picture, the location (xNbA, yNbA ) of neighbor A is set to (xCb - 1, yCb+cbHeight - 1), location (xNbB, yNbB) of B's neighbor is set to (xCb+cbWidth - 1, yCb - 1), cbWidth variable defining the width of the current encoding block in luma samples, cbHeight variable defining the height of the current encoding block in luma samples, CuPredMode representing the current prediction mode, SizeY representing the Y size of the encoding tree block components.

3. CandModeList [x] with x=0..4 is output like this:

If CandIntraPredModeB is equal to candIntraPredModeA and candIntraPredModeA is greater than INTRA_DC, candModeList [x] with x=0..4 is output as follows:

candModeList[0] = candIntraPredModeA(8 10)

candModeList[1] = 2 + ((candIntraPredModeA+61)% 64) (8 12)

candModeList[2] = 2 + ((candIntraPredModeA - 1)%64) (8 13)

candModeList[3] = INTRA_DC(8 11)

candModeList[4] = 2 + ((candIntraPredModeA+60)% 64) (8 14)

Otherwise, if candIntraPredModeB is not equal to candIntraPredModeA and candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC, the following applies:

The variables minAB and maxAB are output as follows:

minAB=Min(candIntraPredModeA, candIntraPredModeB) (8 24)

maxAB=Max (candIntraPredModeA, candIntraPredModeB) (8 25)

If candIntraPredModeA and candIntraPredModeB are both greater than INTRA_DC, candModeList[x] with x=0..4 is output as follows:

candModeList[0] = candIntraPredModeA(8 27)

candModeList[1] = candIntraPredModeB(8 29)

candModeList[2] = INTRA_DC(8 29)

If maxAB - minAB is between 2 and 62 inclusive, the following applies:

candModeList[3] = 2 + ((maxAB+61)% 64) (8 30)

candModeList[4] = 2 + ((maxAB - 1)%64)(8 31)

Otherwise, the following applies:

candModeList[3] = 2 + ((maxAB+60)% 64) (8 32)

candModeList[4] = 2 + ((maxAB)%64)(8 33)

Otherwise (candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC), candModeList [x] with x=0..4 is output like this:

candModeList[0] = maxAB(8 65)

candModeList[1] = INTRA_D(8 66)

candModeList[2] = 2 + ((maxAB+61)% 64) (8 66)

candModeList[3] = 2 + ((maxAB - 1)%64) (8 67)

candModeList[4] = 2 + ((maxAB+60)% 64) (8 68)

Otherwise, the following applies:

candModeList[0] = INTRA_DC(8 71)

candModeList[1] = INTRA_ANGULAR50(8 72)

candModeList[2] = INTRA_ANGULAR18(8 73)

candModeList[3] = INTRA_ANGULAR46(8 74)

candModeList[4] = INTRA_ANGULAR54(8 75)

4. IntraPredModeY [xCb] [yCb] is derived by applying the following procedure:

If intra_luma_mpm_flag [xCb] [yCb] is 1, IntraPredModeY [xCb] [yCb] is set to candModeList [intra_luma_mpm_idx [xCb] [yCb]].

Otherwise, IntraPredModeY [xCb] [yCb] is derived by performing the following ordered steps:

1. When candModeList[i] is greater than candModeList[j] for i=0..3 and for each i, j = (i+1) .. 4, both values are swapped as follows:

(candModeList[i], candModeList[j]) = Swap(candModeList[i], candModeList[j]) (8 94)

2. IntraPredModeY [xCb] [yCb] is derived using the following ordered steps:

IntraPredModeY [xCb] [yCb] is set to intra_luma_mpm_remainder [xCb] [yCb].

The value of IntraPredModeY [xCb] [yCb] is increased by one.

For i equal to 0 to 4 inclusive, when IntraPredModeY [xCb] [yCb] is greater than or equal to candModeList [i], the value of IntraPredModeY [xCb] [yCb] is increased by one.

Otherwise (intra_luma_not_planar_flag [xCb] [yCb] is 0) IntraPredModeY [xCb] [yCb] is set to INTRA_PLANAR.

The variable IntraPredModeY [x] [y] with x=xCb..xCb+cbWidth - 1 and y=yCb..yCb+cbHeight - 1 is set to IntraPredModeY [xCb] [yCb].

In one example, the intra prediction mode of the current block (e.g., variable name Luma_intra_mode) is neither a planar mode nor an intra prediction mode in the 5-MPM list, its value is output, and prediction of this intra prediction mode is performed as follows:

1. The value of intra_luma_mpm_remainder is output from the bitstream and luma_intra_mode is set to the value of intra_luma_mpm_remainder.

2. The value of IntraPredModeY [xCb] [yCb] is increased by one.

3. For i equal to 0 to 4 (compared to each entry in the 5-MPM list), when luma_intra_mode is greater than or equal to MPM[i] (here MPM[0] corresponds to the first entry in the MPM list, MPM[1] corresponds to second entry in the MPM list, and so on), the value of luma_intra_mode is increased by one.

4. Using the derived intra prediction mode (luma_intra_mode) and corresponding reference samples to perform intra prediction.

In one implementation of the above embodiments, the planar mode may implicitly be treated as always the first entry in the implicit 6-MPM list. This implicit 6-MPM list consists of planar (always the first entry) and 5 other modes (which can be explicitly described as a 5-MPM list).

This is because if the planar mode is always in the first position of the implicit 6-MPM list, it can be separated from the implicit MPM list and assigned a special flag (for example, the flagintra_luma_planar_flag or flagintra_luma_not_planar_flag) to indicate whether the intra prediction mode of the current block is planar or not.

One example considers introducing intra_luma_not_planar_flag[x0][y0] as a special flag to indicate if the planar mode is equal to the intra prediction mode of the current block. When it is planar mode, intra_luma_not_planar_flag is set to 0, otherwise it is set to 1. In this case, the first bin of mpm_idx is equivalent to the intra_luma_not_planar_flag special flag.

When the intra prediction mode of the current block is planar, the MPM index is 0 and the coded bin is 0. When the intra prediction mode of the current block is nonplanar, the MPM index is not 0, and the first bin of the MPM index is always 1.

In the example, intra_luma_mpm_idx is encoded using a truncated unary (TR) code,

TR value

mpm_idx 0 0

mpm_idx 1 1 0

MPm_idx 2 1 1 0

MPm_idx 3 1 1 1 0

mpm_idx 4 1 1 1 1.

According to one embodiment of the invention ( non-angular branch ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

Left_mode is not angular mode and above_mode is not angular mode.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

MPM[1]: DC

MPM [2]: vertical mode (i.e. 50)

MPM [3]: horizontal mode (i.e. 18)

MPM[4]: vertical-4 (i.e. 46)

MPM [5]: vertical+4 (i.e. 54)

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention ( non-angular branch ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

Left_mode is not angular mode and above_mode is not angular mode.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

MPM[1]: DC

MPM [2]: vertical mode (i.e. 50)

MPM [3]: horizontal mode (i.e. 18)

MPM[4]: 66

MPM[5]: 2

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention ( non-angular branch ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

Left_mode is not angular mode and above_mode is not angular mode.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

MPM[1]: DC

MPM [2]: vertical mode (i.e. 50)

MPM [3]: horizontal mode (i.e. 18)

MPM[4]: 2

MPM[5]: 34

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention ( non-angular branch ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are corner modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

Left_mode is not angular mode and above_mode is not angular mode.

then, an intra prediction mode list is created that contains N entries of intra prediction modes as follows (for example, N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], and so on):

MPM[1]: DC

MPM [2]: vertical mode (i.e. 50)

MPM [3]: horizontal mode (i.e. 18)

MPM[4]: 66

MPM[5]: 34

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (one-angled branch, one angled, the other non-angled ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode, and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

One of left_mode and above_mode is angular mode, the other is non-angular mode.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

define angular mode as ang and get this:

if left_mode is angular mode, ang=left mode

otherwise (top mode is angular mode) ang=top mode

MPM[1]: ang

MPM[2]: DC

MPM[3]: 2 + ((ang +61)% 64)

MPM[4]: 2 + ((ang - 1)%64)

MPM [5]: 2 + ((ang +60)% 64).

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention ( one-angled branch, one angled, the other non-angled ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

One of left_mode and above_mode is angular mode, the other is non-angular mode.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

define angular mode as ang and get this:

if left_mode is angular mode, ang=left mode

otherwise (top mode is angular mode) ang=top mode

MPM[1]: DC

MPM[2]:ang

MPM[3]: 2 + ((ang +61)% 64)

MPM[4]: 2 + ((ang - 1)%64)

MPM [5]: 2 + ((ang +60)% 64).

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (one-angled branch, one angled, the other non-angled ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

One of left_mode and above_mode is angular mode, the other is non-angular mode.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

define angular mode as ang and get this:

if left_mode is angular mode, ang=left mode

otherwise (top mode is angular mode) ang=top mode

MPM[1]: ang

MPM[2]: 2 + ((ang+61)% 64)

MPM[3]: DC

MPM[4]: 2 + ((ang - 1)% 64)

MPM [5]: 2 + ((angl+60)% 64)

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (one-angled branch, one angled, the other non-angled):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

One of left_mode and above_mode is angular mode, the other is non-angular mode.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

define angular mode as ang and get this:

if left_mode is angular mode, ang=left mode

otherwise (top mode is angular mode) ang=top mode

MPM[0]: Planar

MPM[1]: ang

MPM[2]: 2 + ((ang+61)% 64)

MPM[3]: 2 + ((ang - 1)% 64)

MPM[4]: DC

MPM [5]: 2 + ((ang+60)% 64).

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (one-angled branch, one angled, the other non-angled):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

One of left_mode and above_mode is angular mode, the other is non-angular mode.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

define angular mode as ang and get this:

if left_mode is angular mode, ang=left mode

otherwise (top mode is angular mode) ang=top mode

MPM[1]: ang

MPM[2]: 2 + ((ang+61)% 64)

MPM[3]: 2 + ((ang - 1)% 64)

MPM[4]: 2 + ((angl+60)% 64)

MPM[5]: DC.

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (one-angled branch, two identical corners ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

left_mode and above_mode are both angular modes and they are the same.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

MPM[1]: left mode

MPM[2]: 2 + ((left mode+61)%64)

MPM[3]: 2 + ((left mode - 1)%64)

MPM[4]: DC

MPM [5]: 2 + ((left mode+60)% 64).

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (one-angled branch, two identical corners ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

left_mode and above_mode are both angular modes and they are the same.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

MPM[1]: DC

MPM [2]: left mode

MPM[3]: 2 + ((left mode+61)%64)

MPM[4]: 2 + ((left mode - 1)%64)

MPM[5]: 2 + ((left mode+60)%64)

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (one-angled branch, two identical corners ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

left_mode and above_mode are both angular modes and they are the same.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

MPM[1]: left mode

MPM[2]: DC

MPM[3]: 2 + ((left mode+61)%64)

MPM[4]: 2 + ((left mode - 1)%64)

MPM[5]: 2 + ((left mode+60)%64)

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (one-angled branch, two identical corners ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

left_mode and above_mode are both angular modes and they are the same.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

MPM[1]: left mode

MPM[2]: 2 + ((left mode+61)%64)

MPM[3]: DC

MPM[4]: 2 + ((left mode - 1)%64)

MPM [5]: 2 + ((left mode+60)% 64).

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (one-angled branch, two identical corners ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra prediction modes of neighboring blocks are angle modes or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

left_mode and above_mode are both angular modes and they are the same.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

MPM[1]: left mode

MPM[2]: 2 + ((left mode+61)%64)

MPM[3]: 2 + ((left mode - 1)%64)

MPM[4]: 2 + ((left mode+60)%64)

MPM[5]: DC.

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (biangular branch ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra-neighbor-block prediction mode is an angle mode or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

left_mode and above_mode are two different angular modes.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

MPM[1]: left_mode

MPM[2]: above_mode

MPM[3]: DC

The variables minAB and maxAB are output as follows:

minAB = Min( above_mode, left_mode )

maxAB = Max( above_mode, left_mode ).

If maxAB - minAB is between 2 and 62 inclusive, the following applies:

MPM[4] = 2 + ((maxAB+61)% 64) (8-30)

MPM[5] = 2 + ((maxAB - 1)% 64) (8-31)

Otherwise, the following applies:

MPM[4] = 2 + ((maxAB+60)% 64) (8-32)

MPM[5] = 2 + ((maxAB)% 64) (8-33).

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (biangular branch ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra-neighbor-block prediction mode is an angle mode or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

left_mode and above_mode are two different angular modes.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

MPM[1]: above_mode

MPM[2]: left_mode

MPM[3]: DC

The variables minAB and maxAB are output as follows:

minAB = Min( above_mode, left_mode )

maxAB = Max( above_mode, left_mode ).

If maxAB - minAB is between 2 and 62 inclusive, the following applies:

MPM[4] = 2 + ((maxAB+61)% 64) (8-30)

MPM[5] = 2 + ((maxAB - 1)% 64) (8-31)

Otherwise, the following applies:

MPM[4] = 2 + ((maxAB+60)% 64) (8-32)

MPM[5] = 2 + ((maxAB)% 64) (8-33).

Step 6: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (biangular branch ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra-neighbor-block prediction mode is an angle mode or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

left_mode and above_mode are two different angular modes.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

MPM[1]: left_mode

MPM[2]: DC

MPM[3]: above_mode

The variables minAB and maxAB are output as follows:

minAB = Min( above_mode, left_mode )

maxAB = Max( above_mode, left_mode ).

If maxAB - minAB is between 2 and 62 inclusive, the following applies:

MPM[4] = 2 + ((maxAB+61)% 64) (8-30)

MPM[5] = 2 + ((maxAB - 1)% 64) (8-31)

Otherwise, the following applies:

MPM[4] = 2 + ((maxAB+60)% 64) (8-32)

MPM[5] = 2 + ((maxAB)% 64) (8-33).

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

According to one embodiment of the invention (biangular branch ):

Step 1: obtain the intra prediction mode of the left neighbor block and the intra prediction mode of the upper neighbor block according to Fig.10. If the neighbor block intra prediction mode is not available (eg, the neighbor block is outside the picture boundaries or the neighbor block is outer-coded), the neighbor block intra prediction mode is set to planar. The left neighbor intra prediction mode is denoted as left_mode and the upper neighbor intra prediction mode is denoted as above_mode.

Step 2: determining whether the intra-neighbor-block prediction mode is an angle mode or not.

Step 3: If all of the following conditions (checkboxes) evaluate to true,

left_mode and above_mode are two different angular modes.

then, an intra prediction mode list is created that contains N intra prediction mode entries as follows (e.g., N is 5, where the 1st entry corresponds to MPM[1], the 2nd entry corresponds to MPM[2], etc.) :

MPM[1]: above_mode

MPM[2]: DC

MPM[3]: left_mode

The variables minAB and maxAB are output as follows:

minAB = Min( above_mode, left_mode )

maxAB = Max( above_mode, left_mode ).

If maxAB - minAB is between 2 and 62 inclusive, the following applies:

MPM[4] = 2 + ((maxAB+61)% 64) (8-30)

MPM[5] = 2 + ((maxAB - 1)% 64) (8-31)

Otherwise, the following applies:

MPM[4] = 2 + ((maxAB+60)% 64) (8-32)

MPM[5] = 2 + ((maxAB)% 64) (8-33).

Step 4: prediction of the current block according to the index (denoted as the MPM index) and the entries of the built list.

In one implementation of the embodiment, N is set to 6.

In one implementation of the embodiment, N is set to 6. In this case, said list of intra prediction modes includes the first 5 entries as above rather than the sixth entry.

In one implementation of the embodiment, N is set to 4. In this case, said list of intra prediction modes includes the first 4 entries as above rather than the fifth entry and the sixth entry.

In one implementation of the embodiment, said list of intra prediction modes is referred to as an MPM list.

In one implementation, said list index (MPM index) is signaled in the bitstream as an indicator. Advantageously, the MPM index may be encoded with a context adaptive entropy encoder (CABAC). The MPM index can be encoded according to a different number of probability models (in other words, context) using CABAC.

For example, the first bin of the encoded word for mpm_index is encoded by the CABAC context.

In one example, its context is determined based on whether the current multi-reference inner block, ISP, or normal inner blocks is applied. A total of three contexts are created for the CABAC context of the first codeword bin for mpm_index.

If the MPM list contains both planar and DC modes, then an MPM list that excludes planar and DC modes from the generated MPM list is used by the multiple reference string encoding tool. In this case, the 4-MPM list is used when encoding in the multi-reference intra-prediction mode. In one example, the MPM list is: {Planar (value 0), DC (value 1), VER (value 50), HOR (value 18), VER-4 (value 46), VER+4 (value 54)} after building the list MPM, then a 4-MPM list of {VER, HOR, VER-4, VER+4} is used by encoding in intra prediction mode when multiple reference string is involved.

If the MPM list contains planar mode but no DC mode, then the MPM list excluding planar from the generated MPM list is used by the multiple reference string encoding tool. In this case, the 5-MPM list is used when encoding in intra prediction mode with multiple reference rows. In one example, the MPM list is {Planar, VER, HOR, 43, 60, 3} after constructing the MPM list, then the 5-MPM list of {VER, HOR, 43, 60, 3} is used by intra-prediction mode encoding when multiple references are involved. lines.

In another example, a syntax modification for an MPM list without DC mode may be required. If the MPM list does not contain a DC, it is likely that the DC mode will be used very frequently in the non-MPM branch (i.e., the mpm flag is false). In this case, a new DC_mode syntax is introduced. DC_mode is 1, indicates that the intra prediction mode of the current block is the DC mode. DC_mode equal to 0 indicates that the intra prediction mode of the current block is not a DC mode.

Namely, the old non-MPM syntax is changed from

If (mpm_flag)

…

otherwise

intra_mode (using TB)

on

If (mpm_flag)

…

otherwise

DC_mode

If (DC_mode == 0)

intra_mode (using TB)

The DC_mode syntax may be encoded with a context adaptive entropy encoder (CABAC). DC_mode may be encoded according to a different number of probability models (in other words, context) using CABAC. With the introduction of DC_Mode, the maximum value for a truncated binary is 60, i.e. 67 (total number of modes) - 6 (MPM) -1 (DC). Whereas without the introduction of DC_mode, the maximum value for the truncated binary is 61, i.e. 67 (total number of modes) - 6 (MPM).

Specifically, the following current block prediction encoding methods implemented by a decoding apparatus or an encoding apparatus are provided here. The decoding device may be video decoder 30 in FIG. 1A or decoder 30 in FIG. The encoder may be video encoder 20 in FIG. 1A or encoder 20 in FIG. 2.

According to embodiment 1200 (see FIG. 12A), the current block intra prediction method implemented by the decoding apparatus includes the following steps. In step 1201, the device obtains the value of the first indication information of the current block, wherein the value of the first indication information indicates whether the intra prediction mode of the current block is included in the set of most likely modes. The set of most probable modes contains 5 candidate intra prediction modes and a planar mode. The first indication information is indicated by the first flag, eg intra_luma_mpm_flag. As described above, when the value of intra_luma_mpm_flag is 1, the intra prediction mode of the current block is contained in the most likely mode set.

At 1202, the device obtains the value of the current block index reference row. The reference row value of the index of the current block can be specified using intra_luma_ref_idx. When the reference row value of the current block index is 0, i.e. intra_luma_ref_idx[x0][y0] == 0, the closest adjacent reference row to the current block is used. There is no sequence between steps 1201 and 1202.

In step 1203, the device determines whether the value of the first indication information indicates that the intra prediction mode of the current block is contained in the most likely mode set.

At 1204, the device determines if the index reference row value indicates that the nearest adjacent reference row to the current block is being used. There is no sequence between steps 1203 and 1204.

Step 1205, the device obtains the value of the second indication information of the current block, when the value of the first indication information indicates that the intra prediction mode of the current block is contained in the most likely mode set, and the index reference row value indicates that the closest adjacent reference row to the current block is used, the value the second indication information of the current block indicates whether the intra prediction mode of the current block is a planar mode or not. The value of the second indication information is a context-adaptive binary arithmetic coding (CABAC) decoded by context. The second indication information may be indicated by a second flag, such as intra_luma_not_planar_flag. The device parses the second flag using the context based on whether the current block is an intra-partition (ISP) encoding mode enabled intrablock or not.

When the MRL encoding is not enabled, that is, when the index reference row value indicates that the nearest adjacent reference row to the current block is used, the devices parse the bitstream to obtain the value of the second indication information.

Planar mode has been found to be inefficient with MRL enabled (ie, reference row index not equal to 0) in the conventional method. In an embodiment of the invention, when the MRL encoding is enabled, the value of the second indication information is obtained as a default value. In this situation, the Planar mode is excluded and its corresponding flag (eg, intra_luma_not_planar_flag) is not parsed from the bitstream, but output as a default value (eg, value 1). In this case, a bit is stored to indicate the value of the planar flag (i.e., Intra_luma_not_planar) when the MRL is enabled).

Details of this embodiment 1200 are shown in the above embodiments.

According to embodiment 1210 (see FIG. 12B), the current block intra prediction method implemented by the decoder includes the following steps.

In step 1211, the device obtains the reference row value of the current block index. The reference row value of the index of the current block can be specified using intra_luma_ref_idx. When the reference row value of the current block index is 0, i.e. intra_luma_ref_idx[x0][y0] == 0, the closest adjacent reference row to the current block is used.

At 1212, the device determines if the index reference row value indicates that the nearest adjacent reference row to the current block is being used.

Step 1213, when the index reference row value indicates that the nearest adjacent reference row to the current block is used, the device obtains the value of the first indication information of the current block, the value of the first indication information indicates, The prediction mode of the current block is contained in the set of most likely modes. The set of most probable modes includes 5 candidate intra prediction modes and a planar mode. The first indication information is indicated by the first flag, eg intra_luma_mpm_flag. As described above, when the value of intra_luma_mpm_flag is 1, the intra prediction mode of the current block is contained in the most likely mode set.

In step 1214, the device determines whether the value of the first indication information indicates that the intra prediction mode of the current block is contained in the most likely mode set.

Step 1215, the device obtains the value of the second indication information of the current block, when the value of the first indication information indicates that the intra prediction mode of the current block is contained in the most likely mode set, and the index reference row value indicates that the nearest adjacent reference row to the current block is used, value the second indication information of the current block indicates whether the intra prediction mode of the current block is a planar mode or not. The value of the second indication information is a context-adaptive binary arithmetic coding (CABAC) decoded by context. The second indication information may be indicated by a second flag, such as intra_luma_not_planar_flag. The device parses the second flag using the context based on whether the current block is an intra-partition (ISP) encoding mode enabled intrablock or not.

Details of this embodiment 1210 are shown in the above embodiments.

According to embodiment 1300 (see FIG. 13), the current block intra prediction method implemented by the encoder includes the following steps. In step 1301, the device determines whether the intra prediction mode of the current block is included in the set of most likely modes. The set of most probable modes contains 5 candidate intra prediction modes and a planar mode. In step 1302, the device determines whether the reference row closest to the current block is used for intra prediction. There is no sequence between steps 1301 and 1302.

Step 1303, the apparatus encodes a bitstream, the bitstream including information indicating whether the intra prediction mode of the current block is a planar mode when the intra prediction mode of the current block is contained in the most likely mode set, and the nearest adjacent reference row to the current block used for intra prediction and sending the encoded bitstream to the decoder.

The bitstream includes a first flag indicating whether the intra prediction mode of the current block is included in the set of most likely modes and a second flag indicating whether the intra prediction mode of the current block is a planar mode. As described above, as an example, the first flag is intra_luma_mpm_flag. When the value of intra_luma_mpm_flag is 1, the intra prediction mode of the current block is contained in the set of most likely modes.

The value of the second indication information is a context-adaptive binary arithmetic coding (CABAC) decoded by the context. The second indication information may be indicated by a second flag, such as intra_luma_not_planar_flag. The device parses the second flag using the context based on whether the current block is an intra-partition (ISP) encoding mode enabled intrablock or not.

The bitstream may also include an indication for indicating the reference row value of the index of the current block. The reference row value of the index of the current block is indicated, for example, using intra_luma_ref_idx. When the reference row value of the current block index is 0, i.e. intra_luma_ref_idx[x0][y0] == 0, the closest adjacent reference row to the current block is used.

Details of this embodiment 1300 are shown in the above embodiments.

Fig. 14 illustrates embodiments of a decoder 1400. The decoder 1400 may be video decoder 30 in FIG. 1A or decoder 30 in FIG. Decoder 1400 may be used to implement option 1200 and other embodiments described above.

The apparatus includes a first acquisition unit 1401 and a second acquisition unit 1402. The first obtaining unit 1401, configured to obtain the value of the reference row index of the current block and the value of the first indication information of the current block, wherein the value of the first indication information indicates whether the intra prediction mode of the current block is included in the most likely mode set. The set of most probable modes contains 5 candidate intra prediction modes and a planar mode. The second acquisition unit 1402 is configured to obtain the value of the second indication information of the current block when the value of the first indication information indicates that the intra prediction mode of the current block is contained in the most likely mode set, and when the index reference row value indicates that the closest adjacent reference row to the current is used. block, wherein the value of the second current block indication information indicates whether the intra prediction mode of the current block is a planar mode.

The apparatus 1400 further includes a parser 1403 (not shown in FIG. 14). The parser 1403 is configured to parse the second flag using the context based on whether the current block is an inner block with an inner subsection (ISP) encoding mode enabled or not.

Fig. 15 illustrates embodiments of an intra prediction encoder 1500. Encoder 1500 may be video encoder 20 in FIG. 1A or encoder 20 in FIG. 2. Encoder 1500 may be used to implement embodiment 1300 and other embodiments described above.

The apparatus 1500 includes a determiner 1501 and an encoder 1502. A determination unit 1501, configured to determine whether the intra prediction mode of the current block is included in the set of most likely modes, and to determine whether the nearest adjacent reference line to the current block is used for intra prediction. The set of most probable modes contains 5 candidate intra prediction modes and a planar mode. An encoding unit 1502, configured to encode a bitstream, the bitstream including information indicating whether the intra prediction mode of the current block is a planar mode when the intra prediction mode of the current block is contained in the most likely mode set, and the closest adjacent reference row to the current block is used for intra prediction.

The following is an explanation of the applications of the encoding method as well as the decoding method as shown in the above embodiments, and the system using them.

Fig. 16 is a block diagram showing a content supply system 3100 for realizing a content distribution service. This content delivery system 3100 includes a capture device 3102, a terminal device 3106, and optionally includes a display 3126. The capture device 3102 communicates with the terminal device 3106 over a communication line 3104. The communication link may include the communication channel 13 described above. Link 3104 includes, but is not limited to, WIFI, Ethernet, cable, wireless (3G/4G/5G), USB, or any combination thereof, or the like.

The capture device 3102 generates data and may encode the data in the encoding method shown in the above embodiments. Alternatively, the capture device 3102 may distribute the data to a streaming server (not shown in the Figures), and the server encodes the data and transmits the encoded data to the terminal device 3106. The capture device 3102 includes, but is not limited to, a camera, smartphone, or tablet , a computer or laptop, a videoconferencing system, a PDA, a vehicle-mounted device, or a combination of any of these, or the like. For example, capture device 3102 may include source device 12 described above. When the data includes video, the video encoder 20 included in the capture device 3102 may actually perform video encoding processing. When the data includes audio (ie, speech), the audio encoder included in the capture device 3102 may actually perform audio encoding processing. For some practical scenarios, the capture device 3102 distributes encoded video and audio data by co-multiplexing them. For other practical scenarios, such as in a videoconferencing system, encoded audio data and encoded video data are not multiplexed. The capture device 3102 distributes the encoded audio data and the encoded video data to the terminal device 3106 separately.

In the content delivery system 3100, the terminal device 310 receives and reproduces the encoded data. The terminal device 3106 may be a device with the ability to receive and retrieve data, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR)/digital video recorder (DVR) 3112, a television 3114, a set-top box (STB) 3116, a videoconferencing system 3118 , a video surveillance system 3120, a personal digital assistant (PDA) 3122, a vehicle-mounted device 3124, or a combination thereof, or the like, capable of decoding the aforementioned encoded data. For example, terminal device 3106 may include sink device 14 described above. When the encoded data includes video, the video decoder 30 included in the terminal device is given priority to perform video decoding. When the encoded data includes audio, an audio decoder included in the terminal device is given priority to perform audio decoding processing.

For a terminal device with its own display, such as a 3108 smartphone or tablet, 3110 computer or laptop, 3112 Network Video Recorder (NVR)/Digital Video Recorder (DVR), 3114 TV, 3122 Personal Digital Assistant (PDA), or 3124 Vehicle Mounted Device, terminal the device can send the decoded data to its display. For a non-display terminal device, such as an STB 3116, a video conferencing system 3118, or a video surveillance system 3120, it is contacted with an external display 3126 to receive and display decoded data.

When each device in this system performs encoding or decoding, a picture encoding device or a picture decoding device may be used as shown in the above embodiments.

Fig. 17 is a diagram showing the structure of an exemplary terminal device 3106. After the terminal device 3106 receives a stream from the capture device 3102, the protocol processing unit 3202 parses the transmission protocol of said stream. The protocol includes, but is not limited to, Real Time Streaming Protocol (RTSP), Hypertext Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH, Real Time Transport Protocol (RTP), Interchange Protocol real-time messaging (RTMP) or any combination thereof, or the like.

After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexer 3204 . A demultiplexer 3204 may separate the multiplexed data into encoded audio data and encoded video data. As described above, in other practical scenarios, such as in a videoconferencing system, encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is transmitted to the video decoder 3206 and audio decoder 3208 without using the demultiplexer 3204.

Through the demultiplexing processing, a video elementary stream (ES), audio ES, and optionally subtitles are generated. The video decoder 3206, which includes the video decoder 30 described in the above embodiments, decodes the video ES using the decoding method as shown in the above embodiments to generate a video frame, and supplies this data to the synchronous block 3212. The audio decoder 3208 decodes the audio ES to generate audio frame and feeds the data to synchronous block 3212. Alternatively, a video frame may be stored in a buffer (not shown in FIG. Y) before being fed to synchronous block 3212. Similarly, an audio frame may be stored in a buffer (not shown in FIG. Y) before it is fed into the synchronous block 3212.

The synchronous block 3212 synchronizes the video frame and the audio frame and supplies video/audio to the video/audio display 3214. For example, the synchronous block 3212 synchronizes the presentation of video and audio information. The information may be encoded in a syntax using timestamps regarding the presentation of the encoded audio and video data, as well as timestamps regarding the delivery of the data stream itself.

If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles and synchronizes them with the video frame and the audio frame and passes the video/audio/subtitles to the video/audio/subtitle display 3216.

The present invention is not limited to the above system, and either the picture encoding device or the picture decoding device in the above embodiments may be included in another system such as an automobile system.

Mathematical Operators

The mathematical operators used in this application are similar to those used in the C programming language. However, the results of integer division and arithmetic shift operations are defined more precisely, and additional operations such as exponentiation and real-valued division are also defined. Numbering and counting conventions define starting at 0, e.g. "first" is equivalent to 0, "second" is equivalent to 1, and so on.

Arithmetic operators

The following arithmetic operators are defined as follows:

+ Addition − Subtraction (as a two-argument operator) or negation (as a unary prefix operator) * Multiplication, including matrix multiplication x ^y Exponentiation. Specifies x to the power of y. In other contexts, such a designation is used as a superscript, which should not be interpreted as exponentiation. / Integer division truncating the result towards zero. For example, 7/4 and -7/-4 are truncated to 1, and -7/4 and 7/-4 are truncated to -1. ÷ Used to indicate division in mathematical equations where truncation or rounding is not expected. Used to indicate division in mathematical equations where truncation or rounding is not expected. Summation f( i ), where i takes all integer values from x to y inclusive. x % y Modulo operation. Modulo x divided by y, defined only for integers x and y where x >= 0 and y > 0.

Logical operators

The following logical operators are defined as follows:

x && y Boolean logical "and" for x and y

x || y Boolean logical "or" for x and y

! Boolean logical "not"

x?y:z If x is TRUE or not 0, the value of y is evaluated; otherwise, the z value is calculated.

Relational Operators

The following relational operators are defined as follows:

> More

>= Greater than or equal

< Less

<= Less than or equal

= == Equals

!=Not equal.

When a comparison operator is applied to a syntax element or variable that has been assigned the value "na" (not applicable), the value "na" is treated as a separate value for the syntax element or variable. The value "na" is not considered equal to any other value.

Bitwise Operators

The following bitwise operators are defined as follows:

& Bitwise "and". When operating on integer arguments, operates on the two's complement representation of an integer value. When working on a binary argument that contains fewer bits than another argument, the shorter argument is expanded by adding more significant bits equal to 0.

| When operating on integer arguments, operates on the two's complement representation of an integer value. When working on a binary argument that contains fewer bits than another argument, the shorter argument is expanded by adding more significant bits equal to 0.

^ When operating on integer arguments, operates on the two's complement representation of an integer value. When working on a binary argument that contains fewer bits than another argument, the shorter argument is expanded by adding more significant bits equal to 0.

x >> y Arithmetic right shift of the two's complement integer representation of x by y bits. This function is only defined for non-negative integer y values. The bits shifted to the most significant bits (MSB) as a result of a right shift have a value equal to the MSB of x before the shift operation.

x << y Arithmetic left shift of the two's complement integer representation of x by y bits. This function is only defined for non-negative integer y values. Bits shifted to the least significant bits (LSB) as a result of a left shift have a value of 0.

Assignment operators

The following arithmetic operators are defined as follows:

= assignment operator

+ ++ + Increment, i.e. x+ + is equivalent to x = x+ 1; when used in an array index, evaluates to the value of the variable before the increment operation.

− − Decreasing, i.e. x − − is equivalent to x = x - 1; when used in an array index, evaluates the value of the variable before the decrement operation.

+=+ = Increment by the specified amount, i.e. x += 3 is equivalent to x=x+3 and x += (−3) is equivalent to x=x + (−3).

−=− = Decrease by the specified amount, i.e. x −= 3 is equivalent to x=x − 3, and x −= (−3) is equivalent to x=x − (−3).

Range designation

The following notation is used to indicate a range of values:

x=y..z x takes integer values from y to z, inclusive, where x, y, and z are integers and z is greater than y.

Math functions

The following mathematical functions are defined:

Abs( x ) =

Asin( x ) is a trigonometric arcsine function operating on an argument x that ranges from - to inclusive, with an output value in the range -π÷2 to π÷2 inclusive in units of radians.

Atan( x ) is a trigonometric arctangent function operating on argument x, with an output value in the range -π÷2 to π÷2 inclusive in units of radians.

Atan2( y, x ) =

Ceil( x ) is the smallest integer greater than or equal to x

Clip1Y( x ) = Clip3( 0, ( 1 << BitDepthY ) − 1, x )

Clip1C( x ) = Clip3( 0, ( 1 << BitDepthC ) − 1, x )

Clip3(x, y, z) =

Cos( x ) is a trigonometric cosine function that operates on the x argument in units of radians.

Floor( x ) is the largest integer less than or equal to x.

GetCurrMsb( a, b, c, d ) =

Ln( x ) is the natural logarithm of x (the logarithm to the base e, where e is the base constant of the natural logarithm 281 828...).

log2( x )

Log10( x ) base 10 logarithm of x.

Min( x, y ) =

max( x, y ) =

Round( x ) = Sign( x ) * Floor( Abs( x ) + )

Sign( x ) =

Sin( x ) is a trigonometric sine function that operates on the x argument in units of radians.

Sqrt( x ) =

Swap( x, y ) = ( y, x )

Tan( x ) is a trigonometric tangent function that operates on an x argument in units of radians.

Operation Priority Order

When the order of precedence in an expression is not explicitly specified using parentheses, the following rules apply:

-- Operations with a higher priority are evaluated before any operations with a lower priority.

-- Operations with the same priority are evaluated sequentially from left to right.

Table 14 below lists the priority of operations from highest to lowest; a higher position in table 14 indicates a higher priority.

For those operators that are also used in the C programming language, the order of precedence used in this description is the same as in the C programming language.

Table 14: Priority of operations from highest (at the top of the table) to lowest (at the bottom of the table)

operations (with operands x, y and z) "x++", "x−−" "!x", "−x" (as unary prefix operator) x ^y "x * y", "x/y", "x ÷ y", " x % y" "x+y", "x − y" (as a two-argument operator), " " "x << y", "x >> y" "x < y", "x <= y", "x > y", "x >= y" "x == y", "x != y" "x & y" "x|y" "x && y" "x||y" "x? y: z" "x..y" "x=y", "x += y", "x −= y"

Text description of logical operations

In the text, the definition of logical operations, which would be mathematically described in the following form:

if ( condition 0 ) definition 0 else if ( condition 1 ) definition 1 ... otherwise /* informative comment on the remaining condition */ definition n

can be described as follows:

... as below / ... the following applies:

- if condition 0, definition 0

- Otherwise, if condition 1, definition 1

- ...

-- Otherwise (informative comment on the remaining condition), the definition of n.

Each definition of "if... Else if... Otherwise,..." in the text is introduced by the words "... as specified below" or "... the following applies", immediately followed by "if...". The last condition "if... Else if... Else, ..." is always "Else, ...". The alternating definitions of "if... Else if... Else,..." can be identified by matching "... as below" or "... the following applies" followed by "Else,...".

In the text, the definition of logical operations, which would be mathematically described in the following form:

if (condition 0a && condition 0b)

definition 0

else if (condition 1a | | condition 1b)

definition 1

...

otherwise the definition of n

can be described as follows:

... as below / ... the following applies:

-- If all of the following conditions are true, definition 0:

- condition 0a

- condition 0b

-- Otherwise, if one or more of the following conditions are true, definition 1:

- condition 1a

- condition 1b

- ...

- Else definition of n

In the text, the definition of logical operations, which would be mathematically described in the following form:

if (condition 0)

definition 0

if (condition 1)

definition 1

can be described as follows:

When condition is 0, definition is 0

When the condition is 1, the definition is 1.

Although embodiments of the present invention have been primarily described in terms of video encoding, it should be noted that embodiments of encoding system 10, encoder 20, and decoder 30 (and system 10, respectively), as well as other embodiments described herein, may also be configured to process or encode a still picture, i. e. processing or encoding a single picture, regardless of any preceding or subsequent picture, as in video encoding. In general, only inter prediction (encoder) and 344 (decoder) blocks 244 may not be available if the encoding for picture processing is limited to one picture 17. All other functionality (also referred to as tools or technologies) of video encoder 20 and video decoder 30 may equally be used for still picture processing, e.g., residual calculation 204/304, transform 206, quantize 208, inverse quantize 210/310, (inverse) transform 212/312, split 262/362, intra prediction 254/354 and/or loop filtering 220, 320; and entropy encoding 270 and entropy decoding 304.

Embodiments such as encoder 20 and decoder 30, as well as the functions described herein, such as encoder 20 and decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on a computer-readable medium or transmitted over a communication medium as one or more instructions or code and executed by a hardware processing unit. Computer-readable media may include computer-readable media that corresponds to a tangible medium, such as a storage medium, or a communication medium, including any medium that allows a computer program to be transferred from one place to another, such as according to a communication protocol. Thus, computer-readable media can typically correspond to (1) tangible computer-readable media that is durable, or (2) a communication medium such as a signal or carrier wave. Storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the techniques described in this disclosure. The computer program product may include a computer-readable medium.

By way of example, and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other media that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. However, it should be understood that computer-readable and storage media do not include connections, carrier waves, signals, or other short-lived media, but are instead directed to long-term, tangible storage media. The disc (disk) and disc (disc) used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where discs (disks ) usually reproduce data magnetically, while disks (discs) reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Accordingly, the term "processor" as used herein may refer to any of the aforementioned structures, or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within specialized hardware and/or software modules capable of encoding and decoding or included in a combined codec. In addition, the techniques may be fully implemented in one or more circuits or logic elements.

The techniques of this disclosure may be implemented in a wide variety of devices or hardware components, including a wireless telephone, an integrated circuit (IC), or an IC stack (eg, a chipset). This disclosure describes various components, modules, or blocks to emphasize the functional aspects of devices capable of performing the disclosed technologies, but not necessarily requiring implementation in different hardware blocks. Rather, as described above, various blocks may be combined into a codec hardware block, or provided by a collection of cooperating hardware blocks, including one or more processors, as described above, along with suitable software and/or firmware.

Claims (66)

1. An intra prediction method implemented by a decoding device, comprising: obtaining a value of the first indication information of the current block, the value of the first indication information indicating whether the intra prediction mode of the current block is contained in the most likely mode set, the most likely mode set comprising 5 candidate intra prediction modes and a planar mode; getting the value of the reference row of the index of the current block; when the value of the first indication information indicates that the intra prediction mode of the current block is contained in the most likely mode set, and when the value of the index reference row indicates that the closest adjacent reference row to the current block is used, obtaining the value of the second indication information of the current block, wherein the value of the second information specifying the current block indicates whether the intra prediction mode of the current block is a planar mode or not, wherein the first indication information is indicated by the first flag intra_luma_mpm_flag and the second indication information is indicated by the second flag intra_luma_not_planar_flag, when the value of the second current block indication information indicates that the intra prediction mode of the current block is not a planar mode, the intra_luma_mpm_idx corresponding to index 5 of candidate intra prediction modes is parsed; And predicting the current block based on the intra prediction mode of the current block. 2. The method of claim 1, wherein the value of the second indication information is context-adaptive binary arithmetic coding (CABAC) decoded by context. 3. The method of claim 1, wherein when the value of intra_luma_mpm_flag is 1, the intra prediction mode of the current block is contained in the most likely mode set, and when the reference row value of the current block index is 0, the nearest adjacent reference row to the current block is used. 4. The method according to any one of paragraphs. 3-5, the method further comprising: parsing the second flag using the context based on whether the current block is an inner block with the inner subsection (ISP) encoding mode enabled or not. 5. The method according to any one of paragraphs. 1-4, wherein when the value of the second current block indication information indicates that the intra prediction mode of the current block is not a planar mode, and when one or more of the following conditions are true, the intra prediction candidate mode candIntraPredModeX is set to INTRA_PLANAR: availableX is FALSE; CuPredMode [xNbX] [yNbX] is not equal to MODE_INTRA; or X is equal to B and yCb − 1 is less than (( yCb >> CtbLog2SizeY ) << CtbLog2SizeY), wherein X is replaced by either adjacent block A or B, the luma location (xCb, yCb) indicates the top left sample of the current luminance encoding block relative to the top left luminance sample of the current picture, the location (xNbA, yNbA) of neighbor A is set to (xCb - 1, yCb +cbHeight -1), the location (xNbB, yNbB) of B's neighbor is set to (xCb+cbWidth - 1, yCb - 1), the cbWidth variable specifies the width of the current encoding block in luminance samples, the cbHeight variable indicates the height of the current encoding block in luma samples, CuPredMode represents the current prediction mode, SizeY represents the Y size of the coding tree block components. 6. The method according to any one of paragraphs. 1-5, wherein when the left neighbor intra prediction mode and the top neighbor intra prediction mode are both non-angular modes, the MPM list contains: candModeList[0] = INTRA_DC candModeList[1] = INTRA_ANGULAR50 candModeList[2] = INTRA_ANGULAR18 candModeList[3] = INTRA_ANGULAR46 candModeList[4] = INTRA_ANGULAR54. 7. The method according to any one of paragraphs. 1-6, wherein the step of obtaining the value of the second indication information comprises: parsing the bitstream to obtain the value of the second indication information. 8. The method according to any one of paragraphs. 1-8, when multiple reference string (MRL) encoding is enabled, the value of the second indication information is obtained from a default value. 9. The method of intra prediction implemented by the coding device, comprising: Determining whether the intra prediction mode of the current block is contained in the most likely mode set, wherein the most likely mode set contains 5 candidate intra prediction modes and a planar mode; determining whether the closest adjacent reference row to the current block is used for intra prediction; bitstream coding, wherein the bitstream includes information indicating whether the intra prediction mode of the current block is a planar mode when the intra prediction mode of the current block is contained in the most likely mode set, and the nearest adjacent reference row to the current block is used for intra prediction , wherein the bit stream includes a first flag intra_luma_mpm_flag indicating whether the intra prediction mode of the current block is included in the set of most likely modes, and a second flag intra_luma_not_planar_flag indicating whether the intra prediction mode of the current block is a planar mode; when the value of the second flag indicating that the intra prediction mode of the current block is not a planar mode, the bitstream includes intra_luma_mpm_idx corresponding to index 5 of candidate intra prediction modes. 10. The method of claim 9, wherein when the value of intra_luma_mpm_flag is 1, the intra prediction mode of the current block is contained in the most likely mode set, and when the current block index reference row value is 0, the closest adjacent reference row to the current block is used for intra prediction. 11. The method according to any one of paragraphs. 9-10, wherein said information indicates whether the intra prediction mode of the current block is context-dependent binary arithmetic coding (CABAC) encoded by context. 12. The method of claim 11, wherein the context includes whether the current block is an intra block with an internal subpartition (ISP) encoding mode enabled. 13. A decoding device for intra prediction, comprising: a first acquisition unit configured to obtain a reference row value of the index of the current block and a value of the first indication information of the current block, wherein the value of the first indication information indicates whether the intra prediction mode of the current block is contained in the most likely mode set, the most likely mode set contains 5 candidates intra-prediction and planar modes; a second acquisition unit configured to obtain the value of the second indication information of the current block when the value of the first indication information indicates that the intra prediction mode of the current block is contained in the most likely mode set, and when the index reference row value indicates that the closest adjacent reference row to the current is used to the block, wherein the value of the second current block indication information indicates whether the intra prediction mode of the current block is a planar mode, wherein the first indication information is indicated by the first flag intra_luma_mpm_flag and the second indication information is indicated by the second flag intra_luma_not_planar_flag, when the value of the second current block indication information indicates that the intra prediction mode of the current block is not a planar mode, the intra_luma_mpm_idx corresponding to index 5 of candidate intra prediction modes is parsed; And a prediction block configured to predict the current block based on the intra prediction mode of the current block. 14. The apparatus of claim 13, wherein the value of the second indication information is context-adaptive binary arithmetic coding (CABAC) decoded by context. 15. The apparatus of claim 13, wherein when the value of intra_luma_mpm_flag is 1, the intra-prediction mode of the current block is contained in the set of most likely modes, and when the reference row value of the current block index is 0, the nearest adjacent reference row to the current block is used. 16. The device according to any one of paragraphs. 13-15, wherein the device further comprises: a parser configured to parse the second flag using the context based on whether the current block is an inner block with an inner subpartition (ISP) encoding mode enabled or not. 17. The device according to any one of paragraphs. 13-16, in which the second obtaining unit is configured to obtain the value of the second indication information by parsing the bitstream. 18. The device according to any one of paragraphs. 13-16, in which the second acquisition unit is configured to set a default value as the value of the second indication information when multiple reference row (MRL) encoding is enabled. 19. An encoding device for intra prediction, comprising: a determining unit configured to determine whether the intra prediction mode of the current block is contained in the most likely mode set, and to determine whether the nearest adjacent reference row to the current block is used for intra prediction, wherein the most likely mode set contains 5 candidate intra prediction modes, and planar mode; a coding block configured to encode a bitstream, the bitstream including information indicating whether the intra prediction mode of the current block is a planar mode when the intra prediction mode of the current block is contained in the most likely mode set, and the closest adjacent reference row to the current block is used for intra prediction, wherein the bit stream includes a first flag intra_luma_mpm_flag indicating whether the intra prediction mode of the current block is included in the set of most likely modes, and a second flag intra_luma_not_planar_flag indicating whether the intra prediction mode of the current block is a planar mode; when the value of the second flag indicating that the intra prediction mode of the current block is not a planar mode, the bitstream includes intra_luma_mpm_idx corresponding to index 5 of candidate intra prediction modes. 20. The apparatus of claim 19, wherein when the value of intra_luma_mpm_flag is 1, the intra-prediction mode of the current block is contained in the set of most likely modes, and when the current block index reference row value is 0, the closest adjacent reference row to the current block is used for intra prediction. 21. The device according to any one of paragraphs. 19-20, wherein information indicating whether the intra prediction mode of the current block is context-dependent binary arithmetic coding (CABAC) encoded by context. 22. The apparatus of claim 21, wherein the context includes whether the current block is an internal block with an internal subpartition (ISP) encoding mode enabled. 23. Decoder (30), containing a processing circuit for performing the method according to any one of paragraphs 1 to 8. 24. An encoder (20) containing a processing circuit for performing the method according to any one of paragraphs 9 to 12. 25. Decoder for intra prediction, containing: one or more processors; And a durable computer-readable medium coupled to the processors and storing a program for execution by said processors, wherein the program, when executed by the processors, configures a decoder to perform the method of any one of claims. 1-8. 26. Encoder for intra prediction, containing: one or more processors; And a durable computer-readable medium coupled to the processors and storing a program for execution by said processors, wherein the program, when executed by the processors, configures a decoder to perform the method of any one of claims. 9-12.

Images