WO2020159990A1 - Methods and apparatus on intra prediction for screen content coding - Google Patents

Methods and apparatus on intra prediction for screen content coding Download PDF

Info

Publication number
WO2020159990A1
WO2020159990A1 PCT/US2020/015411 US2020015411W WO2020159990A1 WO 2020159990 A1 WO2020159990 A1 WO 2020159990A1 US 2020015411 W US2020015411 W US 2020015411W WO 2020159990 A1 WO2020159990 A1 WO 2020159990A1
Authority
WO
WIPO (PCT)
Prior art keywords
intra
flag
block
category
blocks
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2020/015411
Other languages
French (fr)
Inventor
Xiaoyu XIU
Yi-Wen Chen
Xianglin Wang
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Dajia Internet Information Technology Co Ltd
Original Assignee
Beijing Dajia Internet Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Dajia Internet Information Technology Co Ltd filed Critical Beijing Dajia Internet Information Technology Co Ltd
Priority to CN202080022164.8A priority Critical patent/CN113615202A/en
Publication of WO2020159990A1 publication Critical patent/WO2020159990A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117Filters, e.g. for pre-processing or post-processing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/86Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness

Definitions

  • the present disclosure relates generally to coding (e.g., encoding and decoding) video data. More specifically, this disclosure relates to a method, a computing device, and a non- transitory computer readable storage medium for selectively enabling and/or disabling intra smoothing operations for video coding.
  • Video coding is performed according to one or more video coding standards.
  • video coding standards include versatile video coding (VVC), joint exploration test model coding (JEM), high-efficiency video coding (H.265/HEVC), advanced video coding (H.264/AVC), moving picture experts group coding (MPEG), or the like.
  • Video coding generally utilizes prediction methods (e.g., inter-prediction, intra-prediction, or the like) that take advantage of redundancy present in video images or sequences.
  • An important goal of video coding techniques is to compress video data into a form that uses a lower bit rate, while avoiding or minimizing degradations to video quality.
  • the first version of the HEVC standard was finalized in October 2013, which offers approximately 50% bit-rate saving or equivalent perceptual quality compared to the prior generation video coding standard H.264/MPEG AVC.
  • the HEVC standard provides significant coding improvements over its predecessor, there is evidence that superior coding efficiency can be achieved with additional coding tools over HEVC.
  • both VCEG and MPEG started the exploration work of new coding technologies for future video coding standardization.
  • One Joint Video Exploration Team (JVET) was formed in October 2015 by ITU-T VECG and ISO/IEC MPEG to begin significant study of advanced technologies that could enable substantial enhancement of coding efficiency.
  • JVET obtained a reference software called joint exploration model (JEM) by integrating several additional coding tools on top of the HEVC test model (HM).
  • JEM joint exploration model
  • VVC test model VTM
  • a method for coding video data performed at a computing device having one or more processors and memory storing a plurality of programs to be executed by the one or more processors, includes selectively disabling intra smoothing operations.
  • a hash value of each block of a plurality of non- overlapped blacks in a tile group is calculated by an encoder.
  • the plurality of non-overlapped blocks in the tile group are classified into two categories: a first category and a second category.
  • the first category includes a set of blocks of the plurality of non-overlapped blocks of hash values that are within a first set of hash values.
  • the second category includes all remaining blocks of the plurality of non-overlapped blocks that are in the tile group but not of the first category. It is then determined, for a first block in the second category, whether there is a second block in the second category representing the same hash value as the first block. Based on determining whether there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks in the tile group is disabled.
  • a computing device includes one or more processors, a non-transitory storage coupled to the one or more processors, and a plurality of programs stored in the non-transitory storage.
  • the computing device selectively disables intra smoothing operations.
  • a hash value of each block of a plurality of non-overlapped blacks in a tile group is calculated by an encoder.
  • the plurality of non-overlapped blocks within the tile group are classified into two categories: a first category and a second category.
  • the first category includes one or more blocks of the plurality of non-overlapping blocks of hash values that are within a first set of hash values.
  • the second category includes all remaining blocks of the plurality of non-overlapped blocks that are in the tile group but not of the first category. It is then determined, for a first block in the second category, whether there is a second block in the second category representing the same hash value as the first block. Based on determining whether there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks in the tile group is disabled.
  • a non-transitory computer readable storage medium stores a plurality of programs for execution by a computing device having one or more processors.
  • the plurality of programs when executed by the one or more processors, cause the computing device to selectively disable intra smoothing operations.
  • a hash value of each block of a plurality of non-overlapped blacks in a tile group is calculated by an encoder.
  • the plurality of non-overlapped blocks within the tile group are classified into two categories: a first category and a second category.
  • the first category includes a set of blocks of the plurality of non-overlapped of hash values that are within a first set of hash values.
  • the second category includes all remaining blocks of the plurality of non-overlapped blocks that are in the tile group but not of the first category. It is then determined, for a first block in the second category, whether there is a second block in the second category representing the same hash value as the first block. Based on determining whether there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks in the tile group is disabled.
  • FIG.1 is a block diagram setting forth an illustrative block-based video encoder which may be used in conjunction with many video coding standards including VVC;
  • FIG.2 is a block diagram setting forth an illustrative block-based video decoder which may be used in conjunction with many video coding standards including VVC;
  • FIGS.3A– 3E show example splitting types, namely, quaternary partitioning (FIG. 3A), horizontal binary partitioning (FIG.3B), vertical binary partitioning (FIG.3C), horizontal ternary partitioning (FIG.3D), and vertical ternary partitioning (FIG.3E), according to some examples;
  • FIG.4 illustrates intra modes in VVC
  • FIG.5 illustrates multiple reference lines for the intra prediction in VVC
  • FIG.7 illustrates locations of neighboring reconstructed samples that are used for the position-dependent intra prediction combination (PDPC) of one coding block
  • FIG.8 illustrates a comparison between pictures that are obtained from screen content and camera-captured video
  • FIG.9 illustrates an example of disabling intra fractional sample interpolation by clipping one fractional reference sample position to a left integer sample position
  • FIG.10A illustrates an example of disabling intra fractional sample interpolation by clipping one fractional reference sample position to its left neighboring integer reference sample when its left neighboring integer reference sample is the nearest neighbor;
  • FIG.10B illustrates an example of disabling intra fractional sample interpolation by clipping one fractional reference sample position to its right neighboring integer reference sample when its right neighboring integer reference sample is the nearest neighbor;
  • FIG.11 is a flowchart of a method for coding video data according to an example. DETAILED DESCRIPTION
  • first,“second,”“third,” etc. may be used herein to describe various information, the information should not be limited by these terms. These terms are only used to distinguish one category of information from another. For example, without departing from the scope of the present disclosure, first information may be termed as second information; and similarly, second information may also be termed as first information.
  • the term“if” may be understood to mean“when” or“upon” or “in response to,” depending on the context.
  • the present disclosure relates generally to coding (e.g., encoding and decoding) video data. More specifically, this disclosure relates to a method, a computing device, and a non- transitory computer readable storage medium for selectively enabling and/or disabling intra smoothing operations for video coding.
  • FIG. 1 is a block diagram setting forth an illustrative block-based video encoder 100 which may be used in conjunction with many video coding standards including VVC.
  • the input video signal is processed block by block (called coding units (CUs)).
  • CUs coding units
  • VTM-1.0 a CU can be up to 128x128 pixels.
  • one coding tree unit (CTU) is split into CUs to adapt to varying local characteristics based on quad/binary/ternary-tree.
  • each CU is always used as the basic unit for both prediction and transform without further partitions.
  • the multi-type tree structure one CTU is firstly partitioned by a quad-tree structure. Then, each quad-tree leaf node can be further partitioned by a binary and ternary tree structure.
  • a video frame is partitioned into a plurality of blocks for processing. For each given video block, a prediction is formed based on either an inter prediction approach or an intra prediction approach.
  • one or more predictors are formed through motion estimation and motion compensation, based on pixels from previously reconstructed frames.
  • predictors are formed based on reconstructed pixels in a current frame. Through mode decision, a best predictor may be chosen to predict a current block.
  • a prediction residual representing the difference between a current video block and its predictor, is sent to a transform circuitry 102.
  • the term“circuitry” as used herein includes hardware and software to operate the hardware.
  • Transform coefficients are then sent from the transform circuitry 102 to a quantization circuitry 104 for entropy reduction.
  • Quantized coefficients are then fed to an entropy coding circuitry 106 to generate a compressed video bitstream.
  • prediction-related information 110 from an inter prediction circuitry and/or an intra prediction circuitry 112, such as block partition info, motion vectors, reference picture index, and intra prediction mode, etc. are also fed through the entropy coding circuitry 106 and saved into a compressed video bitstream 114.
  • decoder-related circuitries are also needed in order to reconstruct pixels for the purpose of prediction.
  • a prediction residual is reconstructed through an inverse quantization circuitry 116 and an inverse transform circuitry 118.
  • This reconstructed prediction residual is combined with a block predictor 120 to generate un-filtered reconstructed pixels for a current block.
  • intra prediction also referred to as“spatial prediction” and/or inter prediction (also referred to as“temporal prediction” or“motion compensated prediction”) may be performed.
  • Intra prediction uses pixels from the samples of already coded neighboring blocks (which are called reference samples) in the same video picture or slice to predict the current video block.
  • Intra prediction reduces spatial redundancy inherent in the video signal.
  • Inter prediction uses reconstructed pixels from the already coded video pictures to predict the current video block.
  • Inter prediction reduces temporal redundancy inherent in the video signal.
  • An inter prediction signal for a given CU is usually signaled by one or more motion vectors (MVs) which indicate an amount and a direction of motion between the current CU and its temporal reference. Also, if multiple reference pictures are supported, one reference picture index is additionally sent, which is used to identify from which reference picture in the reference picture store the temporal prediction signal comes.
  • MVs motion vectors
  • the intra/inter mode decision circuitry 126 in the encoder 100 chooses the best prediction mode, for example based on the rate-distortion optimization method.
  • the prediction block is then subtracted from the current video block; and the prediction residual is de-correlated using transform and quantized.
  • the quantized residual coefficients are inverse quantized and inverse transformed to form the reconstructed residual, which is then added back to the prediction block to form the reconstructed signal of the CU. Further in-loop filtering, such as deblocking filter or sample adaptive offset (SAO) and adaptive in-loop filter (ALF) may be applied on the reconstructed CU before the reconstructed CU is put in the reference picture store and used to code future video blocks.
  • coding mode inter or intra
  • prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding circuitry 106 to be further compressed and packed to form the bit-stream.
  • FIG.2 is a block diagram setting forth an illustrative block-based video decoder 200 which may be used in conjunction with many video coding standards including VVC.
  • decoder 200 is similar to the reconstruction-related section residing in encoder 100 of FIG.1.
  • an incoming video bitstream 201 is first decoded through an entropy decoding circuitry 202 to derive quantized coefficient levels and prediction-related information.
  • the quantized coefficient levels are then processed through an inverse quantization circuitry 204 and an inverse transform circuitry 206 to obtain a reconstructed prediction residual.
  • the coding mode and prediction information are sent to either the spatial prediction circuitry (if intra coded) or the temporal prediction circuitry (if inter coded) to form the prediction block.
  • the residual transform coefficients are sent to inverse quantization circuitry 204 and inverse transform circuitry 206 to reconstruct the residual block.
  • the prediction block and the residual block are then added together.
  • the reconstructed block may further go through in-loop filtering before it is stored in reference picture store.
  • the reconstructed video in the reference picture store is then sent out to drive a display device, as well as used to predict future video blocks.
  • a block predictor mechanism which may be implemented in an intra/inter mode selection circuitry 208, includes an intra prediction circuitry 210 configured to perform an intra-prediction process and/or a motion compensation circuitry 212 configured to perform a motion compensation process based on decoded prediction information.
  • a set of unfiltered reconstructed pixels are obtained by summing the reconstructed prediction residual from inverse transform circuitry 206 and a predictive output generated by the block predictor mechanism, using a sum 214. In situations where an in-loop filter 216 is turned on, a filtering operation is performed on these reconstructed pixels to derive the final reconstructed video for output.
  • FIGS.3A– 3E shows five example splitting types, namely, quaternary partitioning (FIG.3A), horizontal binary partitioning (FIG.3B), vertical binary partitioning (FIG.3C), horizontal ternary partitioning (FIG.3D), and vertical ternary partitioning (FIG.3E).
  • FIG.4 illustrates intra modes in VVC.
  • VVC uses a set of previously decoded samples neighboring to one current CU (i.e., above or left) to predict the samples of the CU.
  • the amount of angular intra modes is extended from 33 in HEVC to 93 in VVC.
  • the same planar mode which assumes a gradual changing surface with horizontal slope and vertical slope derived from boundaries
  • DC mode which assumes a flat surface
  • all the intra modes (i.e., planar, DC and angular directions) in VVC utilize a set of neighboring reconstructed samples above and left to the predicted block as the reference for intra prediction.
  • FIG.5 illustrates multiple reference lines for the intra prediction in VVC. Different from HEVC where only the nearest row or column (i.e., line 0 in Figure 5) of reconstructed samples as reference, multi reference lines are introduced in VVC where two additional rows or columns (i.e., line 1 and line 3 in Figure 5) are used for the intra prediction.
  • the index of the selected reference row or column is signaled from encoder 100 to decoder 200.
  • planar and DC modes are excluded from the set of intra modes that can be used to predict the current block.
  • FIGS.6A– 6C illustrates locations of the reference samples that are used in VVC to derive the predicted samples of one intra block.
  • FIGS.6A– 6C because the quad/binary/ternary tree partition structure is applied, besides the coding blocks in square shape, rectangular coding blocks also exist for the intra prediction of VVC. Due to the unequal width and height of one given block, various sets of angular directions are selected for different block shapes, which is also called wide-angle intra prediction. Particularly, for both square and rectangular coding blocks, besides planar and DC modes, 65 out of 93 angular directions are also supported for each block shape, as shown in Table 1 below.
  • Such design not only efficiently captures the directional structures that are typically present in video (by adaptively selecting angular directions based on block shapes) but also ensures that a total of 67 intra modes (i.e., planar, DC and 65 angular directions) are enabled for each coding block. This can achieve a good efficiency of signaling intra modes while providing a consistent design across different block sizes.
  • Table 1 shows selected angular directions for the intra prediction of different block shapes in VCC.
  • VVC applies a 3-tap smoothing filter [1, 2, 1]/4 to the reference samples used for the intra prediction.
  • the filtering operation is applied for each reference sample using its two neighboring reference samples on the left and right except for the bottom-left and top-right reference samples where the filtering operation is skipped.
  • the decision on whether to apply the smoothing filter to intra reference samples is dependent on coding block size and the directionality of the angular intra mode. More specifically, for coding block sizes with less than 64 samples, the reference smoothing filter is always disabled for all angular directions. For coding block sizes with at least 64 samples, the smoothing filter is applied to three diagonal angular directions (i.e., mode 2, 34 and 66) but disabled for all other angular directions. For DC mode, the smoothing filter is always disabled. For planar mode, the smoothing filter is applied to the intra reference samples when the coding block contains more than 32 samples.
  • an interpolation process is applied to obtain intra prediction samples when the target sample is predicted from one reference sample at a fractional position.
  • a DCT-IF filter the same filter that is applied for motion compensation of chroma components
  • a gaussian filter is applied to generate the luma samples at fractional positions.
  • Table 2 shows the interpolation filters used for the intra prediction of fractional luma samples.
  • w y is the weighting parameter between two integer reference samples R i, 0 and Ri+1, 0, which are calculated as:
  • the intra prediction samples are generated from either a non-filtered or a filtered set of neighboring reference samples, which may introduce discontinuities along the block boundaries between the current coding block and its neighbors.
  • boundary filtering is applied in HEVC by combing the first row or column of prediction samples of DC, horizontal (i.e., mode 18) and vertical (i.e., mode 50) prediction modes with the unfiltered reference samples utilizing a 2-tap filter (for DC mode) or a gradient-based smoothing filter (for horizontal and vertical prediction modes).
  • the position-dependent intra prediction combination (PDPC) tool in VVC extends the above idea by employing a weighted combination of intra prediction samples with unfiltered reference samples.
  • the PDPC is enabled for the following intra modes without signaling: planar, DC, horizontal (i.e., mode 18), vertical (i.e., mode 50), angular directions close to the bottom-left diagonal directions (i.e., mode 2, 3, 4,..., 10) and angular directions close to the top-right diagonal directions (i.e., mode 58, 59, 60,..., 66).
  • planar DC
  • horizontal i.e., mode 18
  • vertical i.e., mode 50
  • angular directions close to the bottom-left diagonal directions i.e., mode 2, 3, 4,..., 10
  • angular directions close to the top-right diagonal directions i.e., mode 58, 59, 60,..., 66.
  • pred(x,y) (wL ⁇ R-1,y + wT ⁇ Rx,-1– wTL ⁇ R-1,-1 + (64– wL– wT + wTL) ⁇ pred(x,y) + 32) >> 6
  • FIG.7 illustrates locations of neighboring reconstructed samples that are used for the position-dependent intra prediction combination (PDPC) of one coding block. It shows the locations of the reference samples that are used to combine with the current prediction sample during the PDPC process.
  • the weights wL, wT and wTL in Eq. (5) are adaptively selected depending on prediction mode and sample position, as described as follows where the current coding block is assumed to be in the size of W ⁇ H:
  • FIG.8 shows a comparison between pictures that are obtained from screen content and camera-captured video.
  • the screen content is shown on the left and the camera-captured content is shown on the right.
  • Screen content coding (SCC) is becoming increasingly important in various video applications, such as desktop sharing, video conferencing, and remote education.
  • screen content shows very different characteristics.
  • the video signals captured by cameras usually show smooth boundaries across different objects.
  • screen content presents shape edges.
  • the intra smoothing operations are efficient to improve the intra coding efficiency of camera-captured content due to the gradual transition edges.
  • the content has a lot of sharp edges such that those smoothing operations can potentially make the prediction inaccurate and therefore compromise the overall coding performance.
  • the smoothing of reference samples could remove the useful high- frequency information existing in the reconstructed neighboring samples of one block. Due to the sharp edges in screen content, the interpolated samples are less accurate than the reference samples at an integer position (i.e., the reference sample without interpolation) for predicting the samples of the current block.
  • the weighted combination of the intra prediction samples and the unfiltered reference samples can potentially remove the useful high-frequency information in the neighboring reconstructed region of the current block, therefore reducing the intra prediction efficiency.
  • signaling methods are proposed in this disclosure to control the enabling and/or disabling of the smoothing operations, e.g., intra reference sample smoothing, fractional intra sample interpolation and PDPC in the existing intra prediction design of VVC.
  • Table 3 illustrates the modified syntax table of sequence parameter set (SPS), where three syntax elements, i.e., sps_intra_reference_smooth_enabled_flag,
  • sps_intra_frac_sample_interp_enabled_flag and sps_pdpc_enabled_flag, are signaled to indicate if the intra reference smoothing, intra fractional sample interpolation or PDPC are applied in the intra prediction process of all coding blocks in the tile groups that refer to the SPS.
  • sps_intra_frac_sample_interp_enabled_flag By setting one of those three flags (e.g., sps_intra_frac_sample_interp_enabled_flag) to zero, one encoder can indicate to one decoder that on the sequence level the corresponding intra smoothing (e.g., intra fractional sample interpolation) is not applied.
  • intra_frac_sample_interp_override_enabled_flag, and pdpc_override_enabled_flag are also signaled in the SPS to indicate if it allows to change the enabling and/or disabling decisions of intra reference smoothing, intra fractional sample interpolation and PDPC at a tile group level.
  • intra_frac_sample_interp_override_enabled_flag When one of those three flags (e.g., intra_frac_sample_interp_override_enabled_flag) is set equal to 1, it indicates that another control flag to enable and/or disable the corresponding intra smoothing operation (e.g., intra fractional sample interpolation) will be signaled at a tile group header; otherwise (i.e., the flag is equal to 0), it indicates that the separate control to enable and/or disable the corresponding intra smoothing operation at the tile group level is disallowed, i.e., the enabling and/or disabling of that intra smoothing operation is controlled by the corresponding flag (e.g., intra_frac_sample_interp_override_enabled_flag) is set equal to 1, it indicates that another control flag to enable and/or disable the corresponding intra smoothing operation (e.g., intra fractional sample interpolation) will be signaled at a tile group
  • Table 3 shows the modified SPS syntax table as follows:
  • sps_intra_reference_smooth_enabled_flag specifies whether intra reference smoothing is applied for intra prediction.
  • sps_intra_reference_smooth_enabled_flag 0 specifies that intra reference smoothing is not used in the CVS. Otherwise, sps_intra_reference_smooth_enabled_flag equal to 1 specifies that intra reference smoothing is used in the CVS.
  • intra_reference_smooth_override_enabled_flag 1 specifies the presence of intra_reference_smooth_enabled_flag in the tile group header of the pictures referring to the SPS.
  • intra_reference_smooth_override_enabled_flag 0 specifies the absence of intra_reference_smooth_enabled_flag in the tile group header of the pictures referring to the SPS.
  • sps_intra_frac_sample_interp_enabled_flag specifies whether intra fractional sample interpolation is applied for intra prediction. sps_intra_frac_sample_interp_enabled_flag equal to 0 specifies that intra fractional sample interpolation is not used in the CVS. Otherwise, sps_intra_frac_sample_interp_enabled_flag equal to 1 specifies that intra fractional sample interpolation is used in the CVS.
  • intra_frac_sample_interp_override_enabled_flag 1 specifies the presence of intra_frac_sample_interp_enabled_flag in the tile group header of the pictures referring to the SPS.
  • intra_frac_sample_interp_override_enabled_flag 0 specifies the absence of intra_frac_sample_interp_enabled_flag in the tile group header of the pictures referring to the SPS.
  • sps_pdpc_enabled_flag specifies whether a position-dependent intra prediction combination is applied for intra prediction. sps_pdpc_enabled_flag equal to 0 specifies that a position-dependent intra prediction combination is not used in the CVS. Otherwise, sps_pdpc_enabled_flag equal to 1 specifies that a position-dependent intra prediction combination is used in the CVS.
  • pdpc_override_enabled_flag 1 specifies the presence of pdpc_enabled_flag in the tile group header of the pictures referring to the SPS.
  • Pdpc_override_enabled_flag 0 specifies the absence of pdpc_enabled_flag in the tile group header of the pictures referring to the SPS.
  • Table 4 shows the modified syntax table of the tile group header.
  • the tile group level adaptation of enabling and/or disabling one intra smoothing process is allowed by setting the corresponding SPS control flag (i.e., sps_intra_reference_smooth_enabled_flag, sps_intra_frac_sample_interp_enabled_flag, or sps_pdpc_enabled_flag) to 1, another flag, i.e., intra_reference_smooth_enabled_flag, intra_frac_sample_interp_enabled_flag, or pdpc_enabled_flag, is further signaled in the tile group header to indicate that the corresponding intra smoothing process (i.e., intra reference smoothing, intra fractional sample interpolation, or PDPC) is enabled or disabled for the coding blocks that belongs to the tile group.
  • the corresponding intra smoothing process i.e., intra reference smoothing,
  • intra_reference_smooth_enabled_flag specifies whether intra reference smoothing is applied for intra prediction.
  • intra_reference_smooth_enabled_flag 0 specifies that intra reference smoothing is not used for the intra prediction of all coding blocks in the current tile group. Otherwise, (intra_reference_smooth_enabled_flag is equal to 1), intra reference smoothing is used for the intra prediction of all coding blocks in the current tile group. When not present, its value is inferred to be the value of
  • intra_frac_sample_interp_enabled_flag specifies whether intra fractional sample interpolation is applied for intra prediction.
  • intra_frac_sample_interp_enabled_flag 0 specifies that intra fractional sample interpolation is not used for the intra prediction of all coding blocks in the current tile group. Otherwise, (intra_frac_sample_interp_enabled_flag is equal to 1), intra fractional sample interpolation is used for the intra prediction of all coding blocks in the current tile group. When not present, its value is inferred to be the value of sps_intra_frac_sample_interp_enabled_flag.
  • pdpc_enabled_flag specifies whether PDPC is applied for intra prediction.
  • pdpc_enabled_flag 0 specifies that PDPC is not used for the intra prediction of all coding blocks in the current tile group. Otherwise, (pdpc_enabled_flag is equal to 1), PDPC is used for the intra prediction of all coding blocks in the current tile group. When not present, its value is inferred to be the value of sps_pdpc_enabled_flag.
  • three sets of syntax elements in Table 3 and Table 4 are proposed in the SPS and the tile group header to separately control the enabling and disabling of intra reference smoothing, intra fractional sample interpolation and PDPC.
  • one same syntax element is used in the SPS and the tile group header to control the enabling and/or disabling of all three intra smoothing operations.
  • intra sample interpolation should not be applied to intra reference samples when the target reference sample locates at one fractional sample position for screen content coding.
  • the positions of the referred intra reference samples that locate at fractional sample positions should be clipped to the reference samples at integer sample positions before intra prediction, for which different methods can be applied.
  • the integer reference sample that is left to the actual fractional sample position (for angular intra modes that refer to the left neighboring samples of the current block) or below the actual fractional position (for angular intra modes that refer to the above neighboring samples of the current block) is used to generate the predicted sample.
  • the value of one intra predicted sample is calculated by the proposed method as
  • FIG.9 illustrates the generation of one intra prediction sample when the above method is applied. As shown in FIG.9, intra fractional sample interpolation is disabled by clipping one fractional reference sample position to its left integer sample position.
  • FIG.11 is a flowchart of a method 1100 for coding video data according to an example.
  • the method may adaptively or selectively make a decision on whether to enable or disable intra smoothing processes (e.g., intra reference smoothing, intra fractional sample interpolation, and PDPC) based on an analysis of the video content to be coded, e.g., camera- captured video content or screen video content.
  • intra smoothing processes e.g., intra reference smoothing, intra fractional sample interpolation, and PDPC
  • the method is described using a tile group as the basic unit to adaptively switch the enabling and disabling of the intra smoothing processes.
  • the method described herein may be applicable to other coding levels, e.g., sequence-level, picture/slice-level, or even region level (e.g., each region may contain a certain number of CTUs).
  • coding levels e.g., sequence-level, picture/slice-level, or even region level (e.g., each region may contain a certain number of CTUs).
  • an encoder such as encoder 100 calculates a hash value of (e.g., 32-bit CRC) each non-overlapped block of a plurality of non-overlapped blocks in a tile group. Meanwhile, for each hash value, the encoder counts the number of blocks (i.e., usage) associated with the hash value.
  • the size of the blocks is 4x4.
  • the method is not limited to using a block size of 4x4. Other block sizes may also be used, including both square blocks and/or rectangular blocks of difference sizes in some examples.
  • all non-overlapped blocks in the tile group are classified into two categories, such as a first category and a second category.
  • the first category may comprise one or more blocks of the plurality of non-overlapped blocks having a hash value covered by a first set of hash values, such as the most-used hash values.
  • a first set of most-used hash values may be first N most-used hash values.
  • the second category may include all remaining blocks of the plurality of non-overlapped blocks that do not belong to the first category.
  • step 1105 for each block in the second category, it is determined whether there is another block in the same category which presents the same hash value. For example, for a first block in the second category, it is determined whether there is a second block in the second category representing the same hash value as the first block in some examples.
  • At step 1107 based on the determination at step 1105, at least one intra smoothing operation for intra prediction of the blocks in the tile group is disabled. For example, based on determining there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks is disabled in some examples.
  • the first block When there is at least one matching second block, the first block is regarded as a screen-content block; otherwise, if there is no matching second block, the first block is regarded as a non-screen-content block.
  • the intra smoothing operations e.g., intra reference smoothing, intra fractional sample interpolation and PDPC
  • the intra smoothing operations will be disabled for the intra prediction of the coding blocks in the tile group. Otherwise, the intra smoothing operations will be enabled for the coding blocks in the tile group.
  • a method for coding video data comprising calculating, by an encoder, a hash value of each block of a plurality of non-overlapped blocks in a tile group and classifying the plurality of non-overlapped blocks into at least two categories comprising a first category and a second category.
  • the first category comprises one or more blocks of the plurality of non-overlapped blocks representing one or more hash values covered by a first set of hash values
  • the second category comprises all remaining blocks of the plurality of non-overlapped blocks. For a first block in the second category, it is determined whether there is a second block in the second category representing the same hash value as the first block. And based on determining whether there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks is disabled.
  • the method when there is at least one second block, the method further includes determining that the first block is a screen-content block and, when there is no second block representing the same hash value as the first block, determining the first block is a non-screen-content block.
  • disabling at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks in the tile group comprises: determining whether a first enabling control flag is set to 1, wherein the first enabling control flag is sent in a sequence parameter set corresponding to the tile group, and determining, when it is determined that the first enabling control flag is set to 1, a second enabling control flag for the plurality of non-overlapped blocks within the tile group, wherein the second enabling control flag is set in a header of the tile group.
  • the enabling control flag may be one of the control flags provided in Table 3 and Table 4.
  • the control flag may be the corresponding SPS control flag, such as sps_intra_reference_smooth_enabled_flag,
  • the enabling flag may be flag, i.e., intra_reference_smooth_enabled_flag,
  • the at least one intra smoothing operation comprises one of following operations: intra reference smoothing, intra fractional sample interpolation, and/or position-dependent intra prediction combination (PDPC).
  • intra reference smoothing intra fractional sample interpolation
  • PDPC position-dependent intra prediction combination
  • the first enabling control flag may further comprise a first enabling control sub-flag for controlling an intra reference smoothing operation, a second enabling control sub-flag for controlling an intra fractional sample interpolation operation, and a third enabling control sub-flag for controlling a PDPC operation.
  • the second enabling flag may further comprise a first enabling control sub-flag for enabling or disabling an intra reference smoothing operation, a second enabling control sub-flag for enabling or disabling an intra fractional sample interpolation operation, and a third enabling control sub-flag for enabling or disabling a PDPC operation.
  • control override flag may comprise a first control override flag for an intra reference smoothing operation, a second control override flag for an intra fractional sample interpolation operation, and a third control override flag for a PDPC operation.
  • disabling at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks within the tile group comprises: determining whether a first enabling control flag is set to 1, wherein the first enabling control flag is sent in a picture parameter set corresponding to the tile group, and determining, when it is determined that the first enabling control flag is set to 1, a second enabling control flag for the plurality of non-overlapped blocks within the tile group, wherein the second enabling control flag is sent in a header of the tile group.
  • the first enabling control flag may further comprise a first enabling control sub-flag for controlling an intra reference smoothing operation, a second enabling control sub-flag for controlling an intra fractional sample interpolation operation, and a third enabling control sub-flag for controlling a PDPC operation.
  • the second enabling control flag may further comprise a first enabling control sub- flag for enabling or disabling an intra reference smoothing operation, a second enabling control sub-flag for enabling or disabling an intra fractional sample interpolation operation, and a third enabling control sub-flag for enabling or disabling a PDPC operation.
  • the method when it is determined to disable the intra fractional interpolation, the method further comprises clipping an intra fractional reference sample position to an integer reference sample position that is left to or above an actual fractional sample position or clipping the intra factional refence sample position to a nearest integer reference sample position.
  • a computing device comprises one or more processors, a non- transitory storage coupled to the one or more processors, and a plurality of programs stored in the non-transitory storage that, when executed by the one or more processors, cause the computing device to perform acts comprising: for each block of a plurality of non-overlapped blocks in a tile group, calculating, by an encoder, a hash value of the block; classifying the plurality of non-overlapped blocks in the tile group into two categories comprising a first category and a second category, wherein the first category comprises one or more blocks of hash values covered by a first set of hash values, and the second category comprises all remaining blocks of the plurality of non-overlapped blocks in the tile group.
  • At least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks in the tile group may be disabled.
  • a non-transitory computer readable storage medium storing a plurality of programs for execution by a computing device having one or more processors, wherein the plurality of programs, when executed by the one or more processors, cause the computing device to perform acts comprising: calculating, by an encoder, a hash value of each block of a plurality of non-overlapped blocks in a tile group; classifying the plurality of non-overlapped blocks into at least two categories comprising a first category and a second category, wherein the first category comprises one or more blocks of the plurality of non- overlapped blocks representing one or more hash values covered by a first set of hash values, and the second category comprises all remaining blocks of the plurality of non-overlapped blocks; for a first block in the second category, determining whether there is a second block in the second category representing the same hash value as the first block; and disabling, based on the determining whether there is the second block in the second category representing the same hash value as the first block
  • disabling, based on determining whether there is the second block in the second category, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks may comprise determining whether a first enabling control flag is set to 1, wherein the first enabling control flag is sent in a sequence parameter set corresponding to the tile group, and determining, when it is determined that the first enabling control flag is set to 1, a second enabling control flag for the plurality of non-overlapped blocks, wherein the second enabling control flag is sent in a header of the tile.
  • Computer- readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol.
  • computer-readable media generally may correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a
  • Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the implementations described in the present application.
  • a computer program product may include a computer- readable medium.
  • the above methods may be implemented using an apparatus that includes one or more circuitries, which include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components.
  • the apparatus may use the circuitries in combination with the other hardware or software components for performing the above described methods.
  • Each module, sub-module, unit, or sub-unit disclosed above may be implemented at least partially using the one or more circuitries.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

A method for coding video data comprises calculating, by an encoder, a hash value of a block of a plurality of non-overlapped blocks in a tile group. The plurality of non-overlapped blocks in the tile group are classified into a first category or a second category. The first category comprises blocks of hash values covered by a first set of hash values and the second category comprises all remaining blocks within the tile group. For a first block in the second category, it is determined whether there is a second block in the second category representing the same hash value as the first block. Based on the determination, at least one intra smoothing operation is disabled for intra prediction of the plurality of non-overlapped blocks within the tile group. Also disclosed are an associated computing device and a non-transitory computer readable storage medium.

Description

Methods and Apparatus on Intra Prediction for Screen Content Coding CROSS-REFERENCE TO RELATED APPLICATION
[0001] This application claims the benefit of U.S. Provisional Application No.62/797,756 filed on January 28, 2019. The entire disclosure of the aforementioned application is incorporated herein by reference in its entirety. FIELD
[0002] The present disclosure relates generally to coding (e.g., encoding and decoding) video data. More specifically, this disclosure relates to a method, a computing device, and a non- transitory computer readable storage medium for selectively enabling and/or disabling intra smoothing operations for video coding. BACKGROUND
[0003] This section provides background information related to the present disclosure. The information contained within this section should not necessarily be construed as prior art.
[0004] Various video coding techniques may be used to compress video data. Video coding is performed according to one or more video coding standards. For example, video coding standards include versatile video coding (VVC), joint exploration test model coding (JEM), high-efficiency video coding (H.265/HEVC), advanced video coding (H.264/AVC), moving picture experts group coding (MPEG), or the like. Video coding generally utilizes prediction methods (e.g., inter-prediction, intra-prediction, or the like) that take advantage of redundancy present in video images or sequences. An important goal of video coding techniques is to compress video data into a form that uses a lower bit rate, while avoiding or minimizing degradations to video quality.
[0005] The first version of the HEVC standard was finalized in October 2013, which offers approximately 50% bit-rate saving or equivalent perceptual quality compared to the prior generation video coding standard H.264/MPEG AVC. Although the HEVC standard provides significant coding improvements over its predecessor, there is evidence that superior coding efficiency can be achieved with additional coding tools over HEVC. Based on that, both VCEG and MPEG started the exploration work of new coding technologies for future video coding standardization. One Joint Video Exploration Team (JVET) was formed in October 2015 by ITU-T VECG and ISO/IEC MPEG to begin significant study of advanced technologies that could enable substantial enhancement of coding efficiency. The JVET obtained a reference software called joint exploration model (JEM) by integrating several additional coding tools on top of the HEVC test model (HM).
[0006] In October 2017, ITU-T and ISO/IEC issued a joint call for proposals (CfP) on video compression with capability beyond HEVC. In April 2018, 23 CfP responses were received and evaluated at the 10-th JVET meeting, which demonstrated compression efficiency gain over the HEVC of around 40%. Based on such evaluation results, the JVET launched a new project to develop the new generation video coding standard named Versatile Video Coding (VVC). In the same month, one reference software codebase, called VVC test model (VTM), was established for demonstrating a reference implementation of the VVC standard. SUMMARY
[0007] This section provides a general summary of the disclosure, and is not a
comprehensive disclosure of its full scope or all of its features.
[0008] According to a first aspect of the present disclosure, a method for coding video data, performed at a computing device having one or more processors and memory storing a plurality of programs to be executed by the one or more processors, includes selectively disabling intra smoothing operations. A hash value of each block of a plurality of non- overlapped blacks in a tile group is calculated by an encoder. The plurality of non-overlapped blocks in the tile group are classified into two categories: a first category and a second category. The first category includes a set of blocks of the plurality of non-overlapped blocks of hash values that are within a first set of hash values. The second category includes all remaining blocks of the plurality of non-overlapped blocks that are in the tile group but not of the first category. It is then determined, for a first block in the second category, whether there is a second block in the second category representing the same hash value as the first block. Based on determining whether there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks in the tile group is disabled.
[0009] According to a second aspect of the present disclosure, a computing device includes one or more processors, a non-transitory storage coupled to the one or more processors, and a plurality of programs stored in the non-transitory storage. When the plurality of programs are executed by the one or more processors, the computing device selectively disables intra smoothing operations. A hash value of each block of a plurality of non-overlapped blacks in a tile group is calculated by an encoder. The plurality of non-overlapped blocks within the tile group are classified into two categories: a first category and a second category. The first category includes one or more blocks of the plurality of non-overlapping blocks of hash values that are within a first set of hash values. The second category includes all remaining blocks of the plurality of non-overlapped blocks that are in the tile group but not of the first category. It is then determined, for a first block in the second category, whether there is a second block in the second category representing the same hash value as the first block. Based on determining whether there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks in the tile group is disabled.
[0010] According to a third aspect of the present disclosure, a non-transitory computer readable storage medium stores a plurality of programs for execution by a computing device having one or more processors. The plurality of programs, when executed by the one or more processors, cause the computing device to selectively disable intra smoothing operations. A hash value of each block of a plurality of non-overlapped blacks in a tile group is calculated by an encoder. The plurality of non-overlapped blocks within the tile group are classified into two categories: a first category and a second category. The first category includes a set of blocks of the plurality of non-overlapped of hash values that are within a first set of hash values. The second category includes all remaining blocks of the plurality of non-overlapped blocks that are in the tile group but not of the first category. It is then determined, for a first block in the second category, whether there is a second block in the second category representing the same hash value as the first block. Based on determining whether there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks in the tile group is disabled. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Hereinafter, sets of illustrative, non-limiting embodiments of the present disclosure will be described in conjunction with the accompanying drawings. Variations of structure, method, or functionality may be implemented by those of ordinary skill in the relevant art based on the examples presented herein, and such variations are all contained within the scope of the present disclosure. In cases where no conflict is present, the teachings of different embodiments may, but need not, be combined with one another.
[0012] FIG.1 is a block diagram setting forth an illustrative block-based video encoder which may be used in conjunction with many video coding standards including VVC; [0013] FIG.2 is a block diagram setting forth an illustrative block-based video decoder which may be used in conjunction with many video coding standards including VVC;
[0014] FIGS.3A– 3E show example splitting types, namely, quaternary partitioning (FIG. 3A), horizontal binary partitioning (FIG.3B), vertical binary partitioning (FIG.3C), horizontal ternary partitioning (FIG.3D), and vertical ternary partitioning (FIG.3E), according to some examples;
[0015] FIG.4 illustrates intra modes in VVC;
[0016] FIG.5 illustrates multiple reference lines for the intra prediction in VVC;
[0017] FIG.6A illustrates reference samples and angular directions that are used for the intra prediction of a flat rectangular block (W/H=2);
[0018] FIG.6B illustrates reference samples and angular directions that are used for the intra prediction of a tall rectangular block (W/H=1/2);
[0019] FIG.6C illustrates reference samples and angular directions that are used for the intra prediction of a square block (W=H);
[0020] FIG.7 illustrates locations of neighboring reconstructed samples that are used for the position-dependent intra prediction combination (PDPC) of one coding block;
[0021] FIG.8 illustrates a comparison between pictures that are obtained from screen content and camera-captured video;
[0022] FIG.9 illustrates an example of disabling intra fractional sample interpolation by clipping one fractional reference sample position to a left integer sample position;
[0023] FIG.10A illustrates an example of disabling intra fractional sample interpolation by clipping one fractional reference sample position to its left neighboring integer reference sample when its left neighboring integer reference sample is the nearest neighbor;
[0024] FIG.10B illustrates an example of disabling intra fractional sample interpolation by clipping one fractional reference sample position to its right neighboring integer reference sample when its right neighboring integer reference sample is the nearest neighbor; and
[0025] FIG.11 is a flowchart of a method for coding video data according to an example. DETAILED DESCRIPTION
[0026] The terms used in the present disclosure are directed to illustrating particular examples, rather than to limit the present disclosure. The singular forms“a”“an” and“the” as used in the present disclosure as well as the appended claims also refer to plural forms unless other meanings are definitely contained in the context. It should be appreciated that the term "and/or" as used herein refers to any or all possible combinations of one or more associated listed items.
[0027] It shall be understood that, although the terms“first,”“second,”“third,” etc. may be used herein to describe various information, the information should not be limited by these terms. These terms are only used to distinguish one category of information from another. For example, without departing from the scope of the present disclosure, first information may be termed as second information; and similarly, second information may also be termed as first information. As used herein, the term“if” may be understood to mean“when” or“upon” or “in response to,” depending on the context.
[0028] Reference throughout this specification to“one embodiment,”“an embodiment,” “another embodiment,” or the like in the singular or plural means that one or more particular features, structures, or characteristics described in connection with an embodiment are included in at least one embodiment of the present disclosure. Thus, the appearances of the phrases“in one embodiment” or“in an embodiment,”“in another embodiment,” or the like in the singular or plural in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics in one or more embodiments may be combined in any suitable manner.
[0029] The present disclosure relates generally to coding (e.g., encoding and decoding) video data. More specifically, this disclosure relates to a method, a computing device, and a non- transitory computer readable storage medium for selectively enabling and/or disabling intra smoothing operations for video coding.
[0030] Like HEVC, VVC is built upon the block-based hybrid video coding framework. FIG. 1 is a block diagram setting forth an illustrative block-based video encoder 100 which may be used in conjunction with many video coding standards including VVC. The input video signal is processed block by block (called coding units (CUs)). In VTM-1.0, a CU can be up to 128x128 pixels. However, different from the HEVC which partitions blocks only based on quad-trees, in VVC, one coding tree unit (CTU) is split into CUs to adapt to varying local characteristics based on quad/binary/ternary-tree. Additionally, the concept of multiple partition unit type in the HEVC is removed, i.e., the separation of CU, prediction unit (PU) and transform unit (TU) does not exist in VVC anymore; instead, each CU is always used as the basic unit for both prediction and transform without further partitions. In the multi-type tree structure, one CTU is firstly partitioned by a quad-tree structure. Then, each quad-tree leaf node can be further partitioned by a binary and ternary tree structure. [0031] In the encoder 100, a video frame is partitioned into a plurality of blocks for processing. For each given video block, a prediction is formed based on either an inter prediction approach or an intra prediction approach. In inter prediction, one or more predictors are formed through motion estimation and motion compensation, based on pixels from previously reconstructed frames. In intra prediction, predictors are formed based on reconstructed pixels in a current frame. Through mode decision, a best predictor may be chosen to predict a current block.
[0032] A prediction residual, representing the difference between a current video block and its predictor, is sent to a transform circuitry 102. The term“circuitry” as used herein includes hardware and software to operate the hardware. Transform coefficients are then sent from the transform circuitry 102 to a quantization circuitry 104 for entropy reduction. Quantized coefficients are then fed to an entropy coding circuitry 106 to generate a compressed video bitstream. As shown in FIG.1, prediction-related information 110 from an inter prediction circuitry and/or an intra prediction circuitry 112, such as block partition info, motion vectors, reference picture index, and intra prediction mode, etc., are also fed through the entropy coding circuitry 106 and saved into a compressed video bitstream 114.
[0033] In the encoder 100, decoder-related circuitries are also needed in order to reconstruct pixels for the purpose of prediction. First, a prediction residual is reconstructed through an inverse quantization circuitry 116 and an inverse transform circuitry 118. This reconstructed prediction residual is combined with a block predictor 120 to generate un-filtered reconstructed pixels for a current block.
[0034] As shown in FIG.1, intra prediction (also referred to as“spatial prediction”) and/or inter prediction (also referred to as“temporal prediction” or“motion compensated prediction”) may be performed. Intra prediction uses pixels from the samples of already coded neighboring blocks (which are called reference samples) in the same video picture or slice to predict the current video block. Intra prediction reduces spatial redundancy inherent in the video signal. Inter prediction uses reconstructed pixels from the already coded video pictures to predict the current video block.
[0035] Inter prediction reduces temporal redundancy inherent in the video signal. An inter prediction signal for a given CU is usually signaled by one or more motion vectors (MVs) which indicate an amount and a direction of motion between the current CU and its temporal reference. Also, if multiple reference pictures are supported, one reference picture index is additionally sent, which is used to identify from which reference picture in the reference picture store the temporal prediction signal comes. [0036] After intra and/or inter prediction, the intra/inter mode decision circuitry 126 in the encoder 100 chooses the best prediction mode, for example based on the rate-distortion optimization method. The prediction block is then subtracted from the current video block; and the prediction residual is de-correlated using transform and quantized. The quantized residual coefficients are inverse quantized and inverse transformed to form the reconstructed residual, which is then added back to the prediction block to form the reconstructed signal of the CU. Further in-loop filtering, such as deblocking filter or sample adaptive offset (SAO) and adaptive in-loop filter (ALF) may be applied on the reconstructed CU before the reconstructed CU is put in the reference picture store and used to code future video blocks. To form the output video bit-stream, coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding circuitry 106 to be further compressed and packed to form the bit-stream.
[0037] FIG.2 is a block diagram setting forth an illustrative block-based video decoder 200 which may be used in conjunction with many video coding standards including VVC. In some examples, decoder 200 is similar to the reconstruction-related section residing in encoder 100 of FIG.1.
[0038] Referring to FIG.2, in decoder 200 an incoming video bitstream 201 is first decoded through an entropy decoding circuitry 202 to derive quantized coefficient levels and prediction-related information. The quantized coefficient levels are then processed through an inverse quantization circuitry 204 and an inverse transform circuitry 206 to obtain a reconstructed prediction residual. The coding mode and prediction information are sent to either the spatial prediction circuitry (if intra coded) or the temporal prediction circuitry (if inter coded) to form the prediction block. The residual transform coefficients are sent to inverse quantization circuitry 204 and inverse transform circuitry 206 to reconstruct the residual block. The prediction block and the residual block are then added together. The reconstructed block may further go through in-loop filtering before it is stored in reference picture store. The reconstructed video in the reference picture store is then sent out to drive a display device, as well as used to predict future video blocks.
[0039] A block predictor mechanism, which may be implemented in an intra/inter mode selection circuitry 208, includes an intra prediction circuitry 210 configured to perform an intra-prediction process and/or a motion compensation circuitry 212 configured to perform a motion compensation process based on decoded prediction information. A set of unfiltered reconstructed pixels are obtained by summing the reconstructed prediction residual from inverse transform circuitry 206 and a predictive output generated by the block predictor mechanism, using a sum 214. In situations where an in-loop filter 216 is turned on, a filtering operation is performed on these reconstructed pixels to derive the final reconstructed video for output.
[0040] FIGS.3A– 3E shows five example splitting types, namely, quaternary partitioning (FIG.3A), horizontal binary partitioning (FIG.3B), vertical binary partitioning (FIG.3C), horizontal ternary partitioning (FIG.3D), and vertical ternary partitioning (FIG.3E).
[0041] FIG.4 illustrates intra modes in VVC. Like HEVC, VVC uses a set of previously decoded samples neighboring to one current CU (i.e., above or left) to predict the samples of the CU. However, to capture finer edge directions present in natural video (especially for video content in high resolutions, e.g., 4K), the amount of angular intra modes is extended from 33 in HEVC to 93 in VVC. In addition to angular directions, the same planar mode (which assumes a gradual changing surface with horizontal slope and vertical slope derived from boundaries) and DC mode (which assumes a flat surface) of HEVC are also applied in the VVC standard. Similar to the intra prediction in HEVC, all the intra modes (i.e., planar, DC and angular directions) in VVC utilize a set of neighboring reconstructed samples above and left to the predicted block as the reference for intra prediction.
[0042] FIG.5 illustrates multiple reference lines for the intra prediction in VVC. Different from HEVC where only the nearest row or column (i.e., line 0 in Figure 5) of reconstructed samples as reference, multi reference lines are introduced in VVC where two additional rows or columns (i.e., line 1 and line 3 in Figure 5) are used for the intra prediction. The index of the selected reference row or column is signaled from encoder 100 to decoder 200. When a non-nearest row or column is selected, planar and DC modes are excluded from the set of intra modes that can be used to predict the current block.
[0043] Assuming the nearest neighbors, FIGS.6A– 6C illustrates locations of the reference samples that are used in VVC to derive the predicted samples of one intra block. As shown in FIGS.6A– 6C, because the quad/binary/ternary tree partition structure is applied, besides the coding blocks in square shape, rectangular coding blocks also exist for the intra prediction of VVC. Due to the unequal width and height of one given block, various sets of angular directions are selected for different block shapes, which is also called wide-angle intra prediction. Particularly, for both square and rectangular coding blocks, besides planar and DC modes, 65 out of 93 angular directions are also supported for each block shape, as shown in Table 1 below. Such design not only efficiently captures the directional structures that are typically present in video (by adaptively selecting angular directions based on block shapes) but also ensures that a total of 67 intra modes (i.e., planar, DC and 65 angular directions) are enabled for each coding block. This can achieve a good efficiency of signaling intra modes while providing a consistent design across different block sizes.
[0044] Table 1 shows selected angular directions for the intra prediction of different block shapes in VCC.
Figure imgf000011_0001
[0045] VVC applies a 3-tap smoothing filter [1, 2, 1]/4 to the reference samples used for the intra prediction. The filtering operation is applied for each reference sample using its two neighboring reference samples on the left and right except for the bottom-left and top-right reference samples where the filtering operation is skipped. For angular intra modes, the decision on whether to apply the smoothing filter to intra reference samples is dependent on coding block size and the directionality of the angular intra mode. More specifically, for coding block sizes with less than 64 samples, the reference smoothing filter is always disabled for all angular directions. For coding block sizes with at least 64 samples, the smoothing filter is applied to three diagonal angular directions (i.e., mode 2, 34 and 66) but disabled for all other angular directions. For DC mode, the smoothing filter is always disabled. For planar mode, the smoothing filter is applied to the intra reference samples when the coding block contains more than 32 samples.
[0046] Additionally, an interpolation process is applied to obtain intra prediction samples when the target sample is predicted from one reference sample at a fractional position.
Particularly, for a luma component, according to the intra prediction mode, one 4-tap filter is applied to interpolate the value of one fractional sample at 1/32-pel accuracy by utilizing the closest reference samples at integer positions, as depicted as: Px, y = ( f[0] × Ri-1, 0 + f[1] × Ri, 0 + f[2] × Ri+1, 0 + f[3] × Ri+2, 0 + 32) >> 6 Eq. (1) where f[i], i = 0, 1,…,2, and Ri-1, 0, Ri, 0, Ri+1, 0 and Ri+2, 0 are the integer reference samples that are used to predict the samples inside the current coding block and >> denotes right bit-shift operation. The index i of the integer reference sample is calculated based on the projection displacement d corresponding to the selected intra prediction mode, as shown as: cy = (y × d)>>5, i = x + cy Eq. (2)
In the current VVC working draft, depending on the selected intra mode, either a DCT-IF filter (the same filter that is applied for motion compensation of chroma components) or a gaussian filter is applied to generate the luma samples at fractional positions.
[0047] Table 2 shows the interpolation filters used for the intra prediction of fractional luma samples.
Figure imgf000012_0001
Figure imgf000013_0001
[0048] For chroma components, a bilinear interpolation filter is applied to obtain the intra predicted samples when the target sample is generated from one fractional reference sample at 1/32-pel, as depicted as: Px, y = ((32 - wy)× Ri, 0 + wy × Ri+1, 0 + 15) >> 5 Eq. (3)
where wy is the weighting parameter between two integer reference samples Ri, 0 and Ri+1, 0, which are calculated as:
cy = (y × d)>>5, wy = (y × d)&31, i = x + cy Eq. (4)
[0049] Furthermore, the intra prediction samples are generated from either a non-filtered or a filtered set of neighboring reference samples, which may introduce discontinuities along the block boundaries between the current coding block and its neighbors. To resolve this problem, boundary filtering is applied in HEVC by combing the first row or column of prediction samples of DC, horizontal (i.e., mode 18) and vertical (i.e., mode 50) prediction modes with the unfiltered reference samples utilizing a 2-tap filter (for DC mode) or a gradient-based smoothing filter (for horizontal and vertical prediction modes).
[0050] The position-dependent intra prediction combination (PDPC) tool in VVC extends the above idea by employing a weighted combination of intra prediction samples with unfiltered reference samples. In the current VVC working draft, the PDPC is enabled for the following intra modes without signaling: planar, DC, horizontal (i.e., mode 18), vertical (i.e., mode 50), angular directions close to the bottom-left diagonal directions (i.e., mode 2, 3, 4,…, 10) and angular directions close to the top-right diagonal directions (i.e., mode 58, 59, 60,…, 66). Assuming the prediction sample located as coordinate (x, y) is pred(x,y), its corresponding value after the PDPC is calculated as
pred(x,y) = (wL × R-1,y + wT × Rx,-1– wTL × R-1,-1 + (64– wL– wT + wTL) × pred(x,y) + 32) >> 6
Eq. (5) where Rx,-1, R-1,y represent the reference samples located at the top and left of current sample (x, y), respectively, and R-1,-1 represents the reference sample located at the top-left corner of the current block. [0051] FIG.7 illustrates locations of neighboring reconstructed samples that are used for the position-dependent intra prediction combination (PDPC) of one coding block. It shows the locations of the reference samples that are used to combine with the current prediction sample during the PDPC process. The weights wL, wT and wTL in Eq. (5) are adaptively selected depending on prediction mode and sample position, as described as follows where the current coding block is assumed to be in the size of W × H:
[0052] For the DC mode,
wT = 32 >> ( ( y<<1 ) >> shift ), wL = 32 >> ( ( x<<1 ) >> shift ), wTL = ( wL>>4 ) + ( wT>>4 )
Eq. (6)
[0053] For the planar mode,
wT = 32 >> ( ( y<<1 ) >> shift ), wL = 32 >> ( ( x<<1 ) >> shift ), wTL = 0 Eq. (7)
[0054] For horizontal mode:
wT = 32 >> ( ( y<<1 ) >> shift ), wL = 32 >> ( ( x<<1 ) >> shift ), wTL = wT Eq. (8)
[0055] For vertical mode:
wT = 32 >> ( ( y<<1 ) >> shift ), wL = 32 >> ( ( x<<1 ) >> shift ), wTL = wL Eq. (9)
[0056] For bottom-left diagonal directions:
wT = 16 >> ( ( y<<1 ) >> shift ), wL = 16 >> ( ( x<<1 ) >> shift ), wTL = 0 Eq. (10)
[0057] For top-right diagonal directions:
wT = 16 >> ( ( y<<1 ) >> shift ), wL = 16 >> ( ( x<<1 ) >> shift ), wTL = 0 Eq. (11) where shift = (log2(W)– 2 + log2(H)– 2 + 2) >> 2.
[0058] FIG.8 shows a comparison between pictures that are obtained from screen content and camera-captured video. The screen content is shown on the left and the camera-captured content is shown on the right. Screen content coding (SCC) is becoming increasingly important in various video applications, such as desktop sharing, video conferencing, and remote education. Compared to natural video content, screen content shows very different characteristics. Particularly, due to the nature of camera lens, the video signals captured by cameras usually show smooth boundaries across different objects. However, screen content, as shown in FIG.8, presents shape edges.
[0059] The intra smoothing operations, for example, intra reference sample smoothing, fractional intra sample interpolation, and PDPC, are efficient to improve the intra coding efficiency of camera-captured content due to the gradual transition edges. However, for screen content video or computer-generated graphics, the content has a lot of sharp edges such that those smoothing operations can potentially make the prediction inaccurate and therefore compromise the overall coding performance. For example, for screen content which contains rich sharp edges, the smoothing of reference samples could remove the useful high- frequency information existing in the reconstructed neighboring samples of one block. Due to the sharp edges in screen content, the interpolated samples are less accurate than the reference samples at an integer position (i.e., the reference sample without interpolation) for predicting the samples of the current block. For screen content video, the weighted combination of the intra prediction samples and the unfiltered reference samples can potentially remove the useful high-frequency information in the neighboring reconstructed region of the current block, therefore reducing the intra prediction efficiency.
[0060] In some examples, signaling methods are proposed in this disclosure to control the enabling and/or disabling of the smoothing operations, e.g., intra reference sample smoothing, fractional intra sample interpolation and PDPC in the existing intra prediction design of VVC.
[0061] Table 3 illustrates the modified syntax table of sequence parameter set (SPS), where three syntax elements, i.e., sps_intra_reference_smooth_enabled_flag,
sps_intra_frac_sample_interp_enabled_flag, and sps_pdpc_enabled_flag, are signaled to indicate if the intra reference smoothing, intra fractional sample interpolation or PDPC are applied in the intra prediction process of all coding blocks in the tile groups that refer to the SPS. By setting one of those three flags (e.g., sps_intra_frac_sample_interp_enabled_flag) to zero, one encoder can indicate to one decoder that on the sequence level the corresponding intra smoothing (e.g., intra fractional sample interpolation) is not applied.
[0062] Additionally, as shown in Table 3, three other flags, i.e.,
intra_reference_smooth_override_enabled_flag,
intra_frac_sample_interp_override_enabled_flag, and pdpc_override_enabled_flag, are also signaled in the SPS to indicate if it allows to change the enabling and/or disabling decisions of intra reference smoothing, intra fractional sample interpolation and PDPC at a tile group level. When one of those three flags (e.g., intra_frac_sample_interp_override_enabled_flag) is set equal to 1, it indicates that another control flag to enable and/or disable the corresponding intra smoothing operation (e.g., intra fractional sample interpolation) will be signaled at a tile group header; otherwise (i.e., the flag is equal to 0), it indicates that the separate control to enable and/or disable the corresponding intra smoothing operation at the tile group level is disallowed, i.e., the enabling and/or disabling of that intra smoothing operation is controlled by the corresponding flag (e.g.,
sps_intra_frac_sample_interp_enabled_flag) in the SPS.
[0063] Table 3 shows the modified SPS syntax table as follows:
Figure imgf000017_0001
[0064] In particular, sps_intra_reference_smooth_enabled_flag specifies whether intra reference smoothing is applied for intra prediction. sps_intra_reference_smooth_enabled_flag equal to 0 specifies that intra reference smoothing is not used in the CVS. Otherwise, sps_intra_reference_smooth_enabled_flag equal to 1 specifies that intra reference smoothing is used in the CVS.
[0065] intra_reference_smooth_override_enabled_flag equal to 1 specifies the presence of intra_reference_smooth_enabled_flag in the tile group header of the pictures referring to the SPS. intra_reference_smooth_override_enabled_flag equal to 0 specifies the absence of intra_reference_smooth_enabled_flag in the tile group header of the pictures referring to the SPS.
[0066] sps_intra_frac_sample_interp_enabled_flag specifies whether intra fractional sample interpolation is applied for intra prediction. sps_intra_frac_sample_interp_enabled_flag equal to 0 specifies that intra fractional sample interpolation is not used in the CVS. Otherwise, sps_intra_frac_sample_interp_enabled_flag equal to 1 specifies that intra fractional sample interpolation is used in the CVS.
[0067] intra_frac_sample_interp_override_enabled_flag equal to 1 specifies the presence of intra_frac_sample_interp_enabled_flag in the tile group header of the pictures referring to the SPS. intra_frac_sample_interp_override_enabled_flag equal to 0 specifies the absence of intra_frac_sample_interp_enabled_flag in the tile group header of the pictures referring to the SPS.
[0068] sps_pdpc_enabled_flag specifies whether a position-dependent intra prediction combination is applied for intra prediction. sps_pdpc_enabled_flag equal to 0 specifies that a position-dependent intra prediction combination is not used in the CVS. Otherwise, sps_pdpc_enabled_flag equal to 1 specifies that a position-dependent intra prediction combination is used in the CVS.
[0069] pdpc_override_enabled_flag equal to 1 specifies the presence of pdpc_enabled_flag in the tile group header of the pictures referring to the SPS. Pdpc_override_enabled_flag equal to 0 specifies the absence of pdpc_enabled_flag in the tile group header of the pictures referring to the SPS.
[0070] Table 4 shows the modified syntax table of the tile group header.
Figure imgf000019_0001
[0071] In some examples, if the tile group level adaptation of enabling and/or disabling one intra smoothing process is allowed by setting the corresponding SPS control flag (i.e., sps_intra_reference_smooth_enabled_flag, sps_intra_frac_sample_interp_enabled_flag, or sps_pdpc_enabled_flag) to 1, another flag, i.e., intra_reference_smooth_enabled_flag, intra_frac_sample_interp_enabled_flag, or pdpc_enabled_flag, is further signaled in the tile group header to indicate that the corresponding intra smoothing process (i.e., intra reference smoothing, intra fractional sample interpolation, or PDPC) is enabled or disabled for the coding blocks that belongs to the tile group.
[0072] In some examples, intra_reference_smooth_enabled_flag specifies whether intra reference smoothing is applied for intra prediction. intra_reference_smooth_enabled_flag equal to 0 specifies that intra reference smoothing is not used for the intra prediction of all coding blocks in the current tile group. Otherwise, (intra_reference_smooth_enabled_flag is equal to 1), intra reference smoothing is used for the intra prediction of all coding blocks in the current tile group. When not present, its value is inferred to be the value of
sps_intra_reference_smooth_enabled_flag.
[0073] intra_frac_sample_interp_enabled_flag specifies whether intra fractional sample interpolation is applied for intra prediction. intra_frac_sample_interp_enabled_flag equal to 0 specifies that intra fractional sample interpolation is not used for the intra prediction of all coding blocks in the current tile group. Otherwise, (intra_frac_sample_interp_enabled_flag is equal to 1), intra fractional sample interpolation is used for the intra prediction of all coding blocks in the current tile group. When not present, its value is inferred to be the value of sps_intra_frac_sample_interp_enabled_flag.
[0074] pdpc_enabled_flag specifies whether PDPC is applied for intra prediction.
pdpc_enabled_flag equal to 0 specifies that PDPC is not used for the intra prediction of all coding blocks in the current tile group. Otherwise, (pdpc_enabled_flag is equal to 1), PDPC is used for the intra prediction of all coding blocks in the current tile group. When not present, its value is inferred to be the value of sps_pdpc_enabled_flag.
[0075] In some examples, three sets of syntax elements in Table 3 and Table 4 are proposed in the SPS and the tile group header to separately control the enabling and disabling of intra reference smoothing, intra fractional sample interpolation and PDPC.
[0076] In some examples, one same syntax element is used in the SPS and the tile group header to control the enabling and/or disabling of all three intra smoothing operations.
[0077] In some examples, instead of placing the high-level control flags (i.e.,
sps_intra_reference_smooth_enabled_flag, sps_intra_frac_sample_interp_enabled_flag, sps_pdpc_enabled_flag, intra_reference_smooth_override_enabled_flag, intra_frac_sample_interp_override_enabled_flag and pdpc_override_enabled_flag) in the SPS, these flags are set in the picture parameter set (PPS). Correspondingly, compared to the sequence level control of the SPS based signaling method in Table 3, the highest granularity that encoder can adaptively enable and/or disable one intra smoothing operation is at picture level after such method is applied.
[0078] Furthermore, in order to avoid the blurring of the sharp edges in the reconstructed neighbors, intra sample interpolation should not be applied to intra reference samples when the target reference sample locates at one fractional sample position for screen content coding. To disable the intra fractional interpolation, the positions of the referred intra reference samples that locate at fractional sample positions should be clipped to the reference samples at integer sample positions before intra prediction, for which different methods can be applied.
[0079] In some examples, the integer reference sample that is left to the actual fractional sample position (for angular intra modes that refer to the left neighboring samples of the current block) or below the actual fractional position (for angular intra modes that refer to the above neighboring samples of the current block) is used to generate the predicted sample. In particular, taking same notations in Eq. (1), the value of one intra predicted sample is calculated by the proposed method as
Px, y = Ri, 0, where cy = (y x d)>>5, i = x + cy Eq. (12)
[0080] FIG.9 illustrates the generation of one intra prediction sample when the above method is applied. As shown in FIG.9, intra fractional sample interpolation is disabled by clipping one fractional reference sample position to its left integer sample position.
[0081] As shown in FIGS.10A– 10B, nearest-neighbor based clipping methods are applied. Instead of using the left neighboring integer reference sample, the neighboring integer reference sample that is closer to the hypothetical fractional sample position will be selected as the predictor. Particularly, with such clipping applied, the derivation of one intra prediction sample is mathematically described as:
Px, y = Ri, 0, if 0 £ wy £ 16; Px, y = Ri+1, 0, if 16 < wy < 32 Eq. (13)
where cy = (y x d)>>5, wy = (y x d)&31, i = x + cy.
[0082] FIG.11 is a flowchart of a method 1100 for coding video data according to an example. The method may adaptively or selectively make a decision on whether to enable or disable intra smoothing processes (e.g., intra reference smoothing, intra fractional sample interpolation, and PDPC) based on an analysis of the video content to be coded, e.g., camera- captured video content or screen video content. Without loss of generality, the method is described using a tile group as the basic unit to adaptively switch the enabling and disabling of the intra smoothing processes. However, to a personal skilled in the art of video coding, the method described herein may be applicable to other coding levels, e.g., sequence-level, picture/slice-level, or even region level (e.g., each region may contain a certain number of CTUs).
[0083] In particular, at step 1101, an encoder, such as encoder 100, calculates a hash value of (e.g., 32-bit CRC) each non-overlapped block of a plurality of non-overlapped blocks in a tile group. Meanwhile, for each hash value, the encoder counts the number of blocks (i.e., usage) associated with the hash value. In some examples, the size of the blocks is 4x4. However, the method is not limited to using a block size of 4x4. Other block sizes may also be used, including both square blocks and/or rectangular blocks of difference sizes in some examples.
[0084] At step 1103, all non-overlapped blocks in the tile group are classified into two categories, such as a first category and a second category. In some examples, the first category may comprise one or more blocks of the plurality of non-overlapped blocks having a hash value covered by a first set of hash values, such as the most-used hash values. In some examples, a first set of most-used hash values may be first N most-used hash values. The second category may include all remaining blocks of the plurality of non-overlapped blocks that do not belong to the first category.
[0085] At step 1105, for each block in the second category, it is determined whether there is another block in the same category which presents the same hash value. For example, for a first block in the second category, it is determined whether there is a second block in the second category representing the same hash value as the first block in some examples.
[0086] At step 1107, based on the determination at step 1105, at least one intra smoothing operation for intra prediction of the blocks in the tile group is disabled. For example, based on determining there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks is disabled in some examples.
[0087] When there is at least one matching second block, the first block is regarded as a screen-content block; otherwise, if there is no matching second block, the first block is regarded as a non-screen-content block. When a percentage of screen-content first blocks is greater than a pre-defined threshold, the intra smoothing operations (e.g., intra reference smoothing, intra fractional sample interpolation and PDPC) will be disabled for the intra prediction of the coding blocks in the tile group. Otherwise, the intra smoothing operations will be enabled for the coding blocks in the tile group.
[0088] In some examples, a method for coding video data comprising calculating, by an encoder, a hash value of each block of a plurality of non-overlapped blocks in a tile group and classifying the plurality of non-overlapped blocks into at least two categories comprising a first category and a second category. The first category comprises one or more blocks of the plurality of non-overlapped blocks representing one or more hash values covered by a first set of hash values, and the second category comprises all remaining blocks of the plurality of non-overlapped blocks. For a first block in the second category, it is determined whether there is a second block in the second category representing the same hash value as the first block. And based on determining whether there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks is disabled.
[0089] In some examples, when there is at least one second block, the method further includes determining that the first block is a screen-content block and, when there is no second block representing the same hash value as the first block, determining the first block is a non-screen-content block.
[0090] In some examples, when there is at least one second block representing the same hash value as the first block and the percentage of first blocks relative to the total number of non- overlapped blocks is greater than a pre-determined threshold value, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks in the tile group is disabled.
[0091] In some examples, disabling at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks in the tile group comprises: determining whether a first enabling control flag is set to 1, wherein the first enabling control flag is sent in a sequence parameter set corresponding to the tile group, and determining, when it is determined that the first enabling control flag is set to 1, a second enabling control flag for the plurality of non-overlapped blocks within the tile group, wherein the second enabling control flag is set in a header of the tile group.
[0092] In particular, the enabling control flag may be one of the control flags provided in Table 3 and Table 4. For example, the control flag may be the corresponding SPS control flag, such as sps_intra_reference_smooth_enabled_flag,
sps_intra_frac_sample_interp_enabled_flag, or sps_pdpc_enabled_flag. The enabling flag may be flag, i.e., intra_reference_smooth_enabled_flag,
intra_frac_sample_interp_enabled_flag, or pdpc_enabled_flag, as shown in Table 4.
[0093] In some examples, the at least one intra smoothing operation comprises one of following operations: intra reference smoothing, intra fractional sample interpolation, and/or position-dependent intra prediction combination (PDPC).
[0094] In some examples, the first enabling control flag may further comprise a first enabling control sub-flag for controlling an intra reference smoothing operation, a second enabling control sub-flag for controlling an intra fractional sample interpolation operation, and a third enabling control sub-flag for controlling a PDPC operation.
[0095] In some examples, the second enabling flag may further comprise a first enabling control sub-flag for enabling or disabling an intra reference smoothing operation, a second enabling control sub-flag for enabling or disabling an intra fractional sample interpolation operation, and a third enabling control sub-flag for enabling or disabling a PDPC operation.
[0096] In some examples, the control override flag may comprise a first control override flag for an intra reference smoothing operation, a second control override flag for an intra fractional sample interpolation operation, and a third control override flag for a PDPC operation.
[0097] In some examples, disabling at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks within the tile group comprises: determining whether a first enabling control flag is set to 1, wherein the first enabling control flag is sent in a picture parameter set corresponding to the tile group, and determining, when it is determined that the first enabling control flag is set to 1, a second enabling control flag for the plurality of non-overlapped blocks within the tile group, wherein the second enabling control flag is sent in a header of the tile group. The first enabling control flag may further comprise a first enabling control sub-flag for controlling an intra reference smoothing operation, a second enabling control sub-flag for controlling an intra fractional sample interpolation operation, and a third enabling control sub-flag for controlling a PDPC operation. The second enabling control flag may further comprise a first enabling control sub- flag for enabling or disabling an intra reference smoothing operation, a second enabling control sub-flag for enabling or disabling an intra fractional sample interpolation operation, and a third enabling control sub-flag for enabling or disabling a PDPC operation.
[0098] In some examples, when it is determined to disable the intra fractional interpolation, the method further comprises clipping an intra fractional reference sample position to an integer reference sample position that is left to or above an actual fractional sample position or clipping the intra factional refence sample position to a nearest integer reference sample position.
[0099] In some examples, a computing device comprises one or more processors, a non- transitory storage coupled to the one or more processors, and a plurality of programs stored in the non-transitory storage that, when executed by the one or more processors, cause the computing device to perform acts comprising: for each block of a plurality of non-overlapped blocks in a tile group, calculating, by an encoder, a hash value of the block; classifying the plurality of non-overlapped blocks in the tile group into two categories comprising a first category and a second category, wherein the first category comprises one or more blocks of hash values covered by a first set of hash values, and the second category comprises all remaining blocks of the plurality of non-overlapped blocks in the tile group. For a first block in the second category, it is determined whether there is a second block in the second category representing the same hash value as the first block. Based on determining whether there is the second block in the second category, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks in the tile group may be disabled.
[00100] In some examples, a non-transitory computer readable storage medium storing a plurality of programs for execution by a computing device having one or more processors, wherein the plurality of programs, when executed by the one or more processors, cause the computing device to perform acts comprising: calculating, by an encoder, a hash value of each block of a plurality of non-overlapped blocks in a tile group; classifying the plurality of non-overlapped blocks into at least two categories comprising a first category and a second category, wherein the first category comprises one or more blocks of the plurality of non- overlapped blocks representing one or more hash values covered by a first set of hash values, and the second category comprises all remaining blocks of the plurality of non-overlapped blocks; for a first block in the second category, determining whether there is a second block in the second category representing the same hash value as the first block; and disabling, based on the determining whether there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks.
[00101] In some examples, disabling, based on determining whether there is the second block in the second category, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks may comprise determining whether a first enabling control flag is set to 1, wherein the first enabling control flag is sent in a sequence parameter set corresponding to the tile group, and determining, when it is determined that the first enabling control flag is set to 1, a second enabling control flag for the plurality of non-overlapped blocks, wherein the second enabling control flag is sent in a header of the tile.
[00102] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over, as one or more instructions or code, a computer-readable medium and executed by a hardware-based processing unit. Computer- readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol. In this manner, computer-readable media generally may correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a
communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the implementations described in the present application. A computer program product may include a computer- readable medium.
[00103] Further, the above methods may be implemented using an apparatus that includes one or more circuitries, which include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components. The apparatus may use the circuitries in combination with the other hardware or software components for performing the above described methods. Each module, sub-module, unit, or sub-unit disclosed above may be implemented at least partially using the one or more circuitries.
[00104] Other examples of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed here. This application is intended to cover any variations, uses, or adaptations of the invention following the general principles thereof and including such departures from the present disclosure as come within known or customary practice in the art. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
[00105] It will be appreciated that the present invention is not limited to the exact examples described above and illustrated in the accompanying drawings, and that various modifications and changes can be made without departing from the scope thereof. It is intended that the scope of the invention only be limited by the appended claims.

Claims

Claims WHAT IS CLAIMED IS:
1. A method for coding video data, comprising:
calculating, by an encoder, a hash value of each block of a plurality of non-overlapped blocks in a tile group;
classifying the plurality of non-overlapped blocks into at least two categories comprising a first category and a second category, wherein the first category comprises one or more blocks of the plurality of non-overlapped blocks representing one or more hash values covered by a first set of hash values, and the second category comprises all remaining blocks of the plurality of non-overlapped blocks;
for a first block in the second category, determining whether there is a second block in the second category representing the same hash value as the first block; and
disabling, based on determining whether there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks.
2. The method of claim 1, further comprising:
upon determining there is the second block in the second category representing the same hash value as the first block, determining the first block is a screen-content block; and when there is no second block, determining the first block as a non-screen-content block.
3. The method of claim 1, further comprising:
disabling the at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks when the percentage of first blocks relative to the total number of non-overlapped blocks is greater than a pre-determined threshold value.
4. The method of claim 1, wherein disabling, based on determining whether there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks comprises:
determining whether a first enabling control flag is set to 1, wherein the first enabling control flag is sent in a sequence parameter set corresponding to the tile group; and determining whether a control override flag is set to 1, wherein the control override flag is sent in the sequence parameter set corresponding to the tile group; and
determining, when it is determined that the control override flag is set to 1, a second enabling control flag for the plurality of non-overlapped blocks, wherein the second enabling control flag is sent in a header of the tile group.
5. The method of claim 4, wherein the first enabling control flag comprises a first enabling control sub-flag for controlling an intra reference smoothing operation, a second enabling control sub-flag for controlling an intra fractional sample interpolation operation, and a third enabling control sub-flag for controlling a position-dependent intra prediction combination (PDPC) operation.
6. The method of claim 4, wherein the control override flag comprises a first control override flag for an intra reference smoothing operation, a second control override flag for an intra fractional sample interpolation operation, and a third control override flag for a PDPC operation.
7. The method of claim 4, wherein the second enabling control flag comprises a first enabling control sub-flag for enabling or disabling an intra reference smoothing operation, a second enabling control sub-flag for enabling or disabling an intra fractional sample interpolation operation, and a third enabling control sub-flag for enabling or disabling a PDPC operation.
8. The method of claim 1, wherein the at least one intra smoothing operation comprises one of the following operations: intra reference smoothing, intra fractional sample interpolation, or position-dependent intra prediction combination (PDPC).
9. The method of claim 1, wherein disabling, based on determining whether there is the second block in the second category, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks comprises:
determining whether a first enabling control flag is set to 1, wherein the first enabling control flag is sent in a picture parameter set corresponding to the tile group;
determining whether a control override flag is set to 1, wherein the control override flag is sent in the picture parameter set corresponding to the tile group; and determining, when it is determined that the control override flag is set to 1, a second enabling control flag for the plurality of non-overlapped blocks, wherein the second enabling control flag is sent in a header of the tile group.
10. The method of claim 9, wherein the first enabling control flag comprises a first enabling control sub-flag for controlling an intra reference smoothing operation, a second enabling control sub-flag for controlling an intra fractional sample interpolation operation, and a third enabling control sub-flag for controlling a PDPC operation.
11. The method of claim 9, wherein the control override flag comprises a first control override flag for an intra reference smoothing operation, a second control override flag for an intra fractional sample interpolation operation, and a third control override flag for a PDPC operation.
12. The method of claim 9, wherein the second enabling control flag comprises a first enabling control sub-flag for enabling or disabling an intra reference smoothing operation, a second enabling control sub-flag for enabling or disabling an intra fractional sample interpolation operation, and a third enabling control sub-flag for enabling or disabling a PDPC operation.
13. The method of claim 5, further comprising:
determining to disable the intra fractional interpolation; and
clipping one intra fractional reference sample position to an integer reference sample position that is left to or above an actual fractional sample position.
14. The method of claim 5, further comprising:
determining to disable the intra fractional interpolation; and
clipping one intra factional refence sample position to a nearest integer reference sample position.
15. A computing device, comprising:
one or more processors;
a non-transitory storage coupled to the one or more processors; and a plurality of programs stored in the non-transitory storage that, when executed by the one or more processors, cause the computing device to perform acts comprising:
calculating, by an encoder, a hash value of each block of a plurality of non-overlapped blocks in a tile group;
classifying the plurality of non-overlapped blocks into at least two categories comprising a first category and a second category, wherein the first category comprises one or more blocks of the plurality of non-overlapped blocks representing one or more hash values covered by a first set of hash values, and the second category comprises all remaining blocks of the plurality of non-overlapped blocks;
for a first block in the second category, determining whether there is a second block in the second category representing the same hash value as the first block; and
disabling, based on determining whether there is the second block in the second category representing the same hash value as the first block, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks.
16. The computing device of claim 15, wherein disabling, based on determining whether there is the second block in the second category, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks comprises:
determining whether a first enabling control flag is set to 1, wherein the first enabling control flag is sent in a sequence parameter set corresponding to the tile group;
determining whether a control override flag is set to 1, wherein the control override flag is sent in the sequence parameter set corresponding to the tile group; and
determining, when it is determined that the control override flag is set to 1, a second enabling control flag for the plurality of non-overlapped blocks, wherein the second enabling control flag is sent in a header of the tile group.
17. The computing device of claim 15, wherein the at least one intra smoothing operation comprises one of following operations: intra reference smoothing, intra fractional sample interpolation, or position-dependent intra prediction combination (PDPC).
18. The computing device of claim 15, wherein disabling, based on determining whether there is the second block in the second category, at least one intra smoothing operation for intra prediction of the plurality of non-overlapped blocks comprises: determining whether a first enabling control flag is set to 1, wherein the first enabling control flag is sent in a picture parameter set corresponding to the tile group;
determining whether a control override flag is set to 1, wherein the control override flag is sent in the picture parameter set corresponding to the tile group; and
determining, when it is determined that the control override flag is set to 1, a second enabling control flag for the plurality of non-overlapped blocks, wherein the second enabling control flag is sent in a header of the tile group.
19. The computing device of claim 15, when it is determined to disable an intra fractional interpolation, the computing device is caused to perform acts comprising:
clipping one intra fractional reference sample position to an integer reference sample position that is left to or below an actual fractional sample position; or
clipping one intra factional refence sample position to a nearest integer reference sample position.
20. A non-transitory computer readable storage medium storing a plurality of programs for execution by a computing device having one or more processors, wherein the plurality of programs, when executed by the one or more processors, cause the computing device to perform acts comprising:
calculating, by an encoder, a hash value of each block of a plurality of non-overlapped blocks in a tile group;
classifying the plurality of non-overlapped blocks into at least two categories comprising a first category and a second category, wherein the first category comprises one or more blocks of the plurality of non-overlapped blocks representing one or more hash values covered by a first set of hash values, and the second category comprises all remaining blocks of the plurality of non-overlapped blocks;
for a first block in the second category, determining whether there is a second block in the second category representing the same hash value as the first block; and
disabling, based on determining whether there is the second block in the second category, at least one intra smoothing operation for intra prediction of the plurality of non- overlapped blocks.
PCT/US2020/015411 2019-01-28 2020-01-28 Methods and apparatus on intra prediction for screen content coding Ceased WO2020159990A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202080022164.8A CN113615202A (en) 2019-01-28 2020-01-28 Method and apparatus for intra prediction for screen content encoding and decoding

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201962797756P 2019-01-28 2019-01-28
US62/797,756 2019-01-28

Publications (1)

Publication Number Publication Date
WO2020159990A1 true WO2020159990A1 (en) 2020-08-06

Family

ID=71841908

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2020/015411 Ceased WO2020159990A1 (en) 2019-01-28 2020-01-28 Methods and apparatus on intra prediction for screen content coding

Country Status (2)

Country Link
CN (1) CN113615202A (en)
WO (1) WO2020159990A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220312003A1 (en) * 2019-06-19 2022-09-29 Electronics And Telecommunications Research Institute Intra prediction mode, and entropy encoding/decoding method and device

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015131325A1 (en) * 2014-03-04 2015-09-11 Microsoft Technology Licensing, Llc Hash table construction and availability checking for hash-based block matching
WO2015139165A1 (en) * 2014-03-17 2015-09-24 Microsoft Technology Licensing, Llc Encoder-side decisions for screen content encoding

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015131325A1 (en) * 2014-03-04 2015-09-11 Microsoft Technology Licensing, Llc Hash table construction and availability checking for hash-based block matching
WO2015139165A1 (en) * 2014-03-17 2015-09-24 Microsoft Technology Licensing, Llc Encoder-side decisions for screen content encoding

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
BIN LI ET AL.: "An HEVC-based Screen Content Coding Scheme", MICROSOFT RESEARCH LAB, 31 January 2015 (2015-01-31), pages 1 - 13, XP055245403 *
XIAOZHONG XU ET AL.: "Description of Core Experiment 8: Screen Content Coding Tools", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/WG 11, JVET-L1028-V3, 12TH MEETING, MACAO, CN, 12 October 2018 (2018-10-12), pages 1 - 14 *
YAWEI XU ET AL.: "Pattern-matching scheme for HEVC screen content coding", 5TH INTERNATIONAL CONFERENCE ON COMPUTER SCIENCES AND AUTOMATION ENGINEERING (ICCSAE 2015), 29 February 2016 (2016-02-29), pages 916 - 921, XP055724238 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220312003A1 (en) * 2019-06-19 2022-09-29 Electronics And Telecommunications Research Institute Intra prediction mode, and entropy encoding/decoding method and device
US12225191B2 (en) * 2019-06-19 2025-02-11 Electronics And Telecommunications Research Institute Intra prediction mode, and entropy encoding/decoding method and device

Also Published As

Publication number Publication date
CN113615202A (en) 2021-11-05

Similar Documents

Publication Publication Date Title
AU2019398124B2 (en) Method and apparatus for deblocking an image
US12425624B2 (en) Video coding using intra sub-partition coding mode
WO2017084577A1 (en) Method and apparatus for intra prediction mode using intra prediction filter in video and image compression
US11375199B2 (en) Interpolation filter for an inter prediction apparatus and method for video coding
US20130044814A1 (en) Methods and apparatus for adaptive interpolative intra block encoding and decoding
US10412402B2 (en) Method and apparatus of intra prediction in video coding
CN121664982A (en) Video encoding and decoding using the cross-component linear model
WO2018050601A1 (en) Method and apparatus for video coding with adaptive clipping
KR20190058632A (en) Distance Weighted Bidirectional Intra Prediction
CN109845252A (en) Apparatus and method for video encoding
CN114143548B (en) Coding and decoding of transform coefficients in video coding and decoding
CN114339223B (en) Decoding method, device, equipment and machine readable storage medium
US20210289202A1 (en) Intra prediction method and apparatus for performing adaptive filtering on reference pixel
WO2020159990A1 (en) Methods and apparatus on intra prediction for screen content coding
WO2024006231A1 (en) Methods and apparatus on chroma motion compensation using adaptive cross-component filtering
US12184846B2 (en) Intra prediction method and apparatus for performing adaptive filtering on reference pixel
WO2020264221A1 (en) Apparatuses and methods for bit-width control of bi-directional optical flow
WO2025148956A1 (en) Regression-based blending for improving intra prediction with neighboring template
HK40064089A (en) Decoding method, apparatus, device, and machine-readable storage medium
KR20240153950A (en) Intraprediction method and apparatus for performing adaptive filtering on reference pixels
WO2024010832A1 (en) Methods and apparatus on chroma motion compensation using adaptive cross-component filtering
WO2015051920A1 (en) Video encoding and decoding

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20749678

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20749678

Country of ref document: EP

Kind code of ref document: A1