CN113812161A - Scaling method in video coding and decoding - Google Patents

Abstract

An apparatus, system and scaling method in a video codec are disclosed for digital video codec, the method comprising performing a conversion between a current video block of the video and a codec representation of the video, the current video block comprising a luminance component and at least one chrominance component; the luminance component is converted from the original domain to the reshaped domain using a chroma scaled Luminance Mapping (LMCS) scheme, and chroma samples of at least one of the chroma components are predicted based on reconstructed luminance samples of the luminance component in the original domain.

Description

Scaling method in video coding and decoding

Cross Reference to Related Applications

Under applicable patent laws and/or in accordance with the rules of the paris convention, the present application is intended to claim in time the priority and benefit of international patent application No. PCT/CN2019/086789 filed 5, 14, 2019. The entire disclosure of the above-mentioned patent application is incorporated herein by reference as part of the disclosure of the present application.

Technical Field

This patent document relates to video encoding and decoding techniques, devices and systems.

Background

Despite advances in video compression, digital video still accounts for the largest bandwidth usage on the internet and other digital communication networks. As the number of interconnected user devices capable of receiving and displaying video increases, the demand for bandwidth for digital video usage is expected to continue to grow.

Disclosure of Invention

To apparatus, systems, and methods for digital video codec, and more particularly to scaling and division operations in video codec. The described method may be applied to existing video codec standards, such as High Efficiency Video Codec (HEVC), and future video codec standards or video codecs.

In one representative aspect, the disclosed technology can be used to provide a method for video processing, comprising: a conversion is performed between a current video block of a video and a coded representation (coded representation) of the video, the current video block comprising a luminance component and at least one chrominance component, wherein the luminance component is converted from an original domain to a reshaped domain using a luminance mapping with chrominance scaling (LMCS) scheme, and chrominance samples of the at least one chrominance component are predicted based on reconstructed luminance samples of the luminance component in the original domain.

In yet another representative aspect, the above-described methods are embodied in the form of processor-executable code and stored in a computer-readable program medium.

In yet another representative aspect, an apparatus configured or operable to perform the above-described method is disclosed. The apparatus may include a processor programmed to implement the method.

In yet another representative aspect, a video decoder device may implement a method as described herein.

The above and other aspects and features of the disclosed technology are described in more detail in the accompanying drawings, the description and the claims.

Drawings

Fig. 1 shows a flow chart of an example of a decoding flow with shaping.

Fig. 2 shows an example of sample (sample) positions for deriving parameters in cross-component linear model (CCLM) prediction mode.

Fig. 3 shows an example of neighboring spots used to derive Illumination Compensation (IC) parameters.

Fig. 4 shows a flow diagram of an example method for video processing.

Fig. 5 is a block diagram of an example of a hardware platform for implementing the visual media decoding or visual media codec techniques described in this document.

Detailed Description

Embodiments of the disclosed techniques may be applied to existing video codec standards (e.g., HEVC, h.265) and future standards to improve compression performance. Section headings are used in this document to enhance readability of the description, and do not limit the discussion or embodiments (and/or implementations) in any way to only the corresponding sections.

2 video codec introduction

Due to the increasing demand for higher resolution video, video coding methods and techniques are ubiquitous in modern technology. Video codecs typically include electronic circuits or software that compress or decompress digital video and are continually being improved to provide higher codec efficiency. Video codecs convert uncompressed video into a compressed format and vice versa. There is a complex relationship between video quality, the amount of data used to represent the video (determined by the bit rate), the complexity of the encoding and decoding algorithms, susceptibility to data loss and errors, ease of editing, random access, and end-to-end delay (latency). The compression format typically conforms to a standard video compression specification, such as the High Efficiency Video Codec (HEVC) standard (also known as h.265 or MPEG-H part 2), a general video codec standard to be finalized, or other current and/or future video codec standards.

The video codec standards have evolved largely through the well-known development of the ITU-T and ISO/IEC standards. ITU-T has established H.261 and H.263, ISO/IEC has established MPEG-1 and MPEG-4 video, and both organizations have jointly established the H.262/MPEG-2 video and H.264/MPEG-4 Advanced Video Codec (AVC) and H.265/HEVC standards. Since h.262, the video codec standard is based on a hybrid video codec structure, in which temporal prediction plus transform coding is utilized. To explore future video codec technologies beyond HEVC, VCEG and MPEG united in 2015 into the joint video exploration team (jfet). Since then, several new methods have been adopted by JVET and put into the reference software named Joint Exploration Model (JEM). In month 4 of 2018, a joint video exploration team (jfet) was created between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11(MPEG) to study the VVC standard with the goal of reducing the bit rate by 50% compared to HEVC.

2.1 Loop shaping (ILR) in JFET-M0427

The basic idea of loop shaping (ILR) is to convert the original (in the first domain) signal (predicted/reconstructed signal) into the second domain (reshaped domain).

Loop brightness shaper quiltImplemented as a pair of look-up tables (LUTs), but only one of the two LUTs needs to be signaled, since the other can be computed from the signaled LUT. Each LUT is a one-dimensional, 10-bit, 1024-entry mapping table (1D-LUT). One LUT is a forward LUT (FwdLUT) which will input a luminance code value Y_iMapping to a changed value Y_r:Y_r＝FwdLUT[Y_i]. The other LUT is an inverse LUT (InvLUT) that converts the changed code value Y_rMapping to

(

Represents Y_iThe reconstructed value of).

ILR is also known in VVC as luma map with chroma scaling (LMCS).

2.1.1 PWL model

Conceptually, piece-wise linear (PWL) is realized in the following manner.

Let x1, x2 be two input axis points (pivot points) and y1, y2 be their respective output axis points for a segment. The output value y of any input value x between x1 and x2 may be interpolated by the following equation:

y＝((y2-y1)/(x2-x1))*(x-x1)+y1。

in a fixed point implementation, the equation may be rewritten as:

y＝((m*x+2FP_PREC-1)>>FP_PREC)+c。

here, m is a scalar, c is an offset, and FP _ PREC is a constant value specifying precision.

Note that in CE-12 software, the PWL model is used to pre-compute the FwdLUT and InvLUT mapping tables for 1024 entries; the PWL model also allows implementations that compute the same mapping values in real time without pre-computing the LUT.

2.1.2 testing CE12-2

2.1.2.1 Brightness shaping

Test 2 of loop luma shaping (i.e., CE12-2 in the proposal) provides a lower complexity pipeline (pipeline) that also eliminates the decoding latency of block-wise intra prediction in inter-slice reconstruction. Intra prediction is performed in the shaping domain for both inter and intra slices.

Intra prediction is always performed in the shaping domain, regardless of slice type. With such an arrangement, intra prediction can start immediately after the previous TU reconstruction is completed. This arrangement may also provide a uniform process for intra mode rather than on a slice basis. FIG. 1 shows a block diagram of a mode-based CE12-2 decoding process.

CE12-2 also tested a 16-segment piecewise linear (PWL) model for luma and chroma residual scaling, rather than testing the 32-segment PWL model of CE 12-1.

Inter-band reconstruction with loop luma shaper in CE12-2 (lighter shaded blocks represent signal in the reshaped domain: luma residual; predicted intra luma; and reconstructed inter luma).

2.1.2.2 luma-based chroma residual scaling

Luma-based chroma residual scaling is a multiplication process implemented with fixed-point integer arithmetic. Chroma residual scaling compensates for the interaction of the luma signal with the chroma signals. Chroma residual scaling is applied to the TU level. More specifically, the average of the blocks is predicted using the corresponding luminance.

The average value is used to identify an index in the PWL model. The index identifies the scaling factor cscalelnv. The chroma residual is multiplied by a scaling factor.

Note that the chroma scaling factor is calculated from the forward mapped predicted luma values rather than from the reconstructed luma values.

2.1.2.3 use of ILR

At the codec side, each picture (or tile) group is first converted to the reshaped domain. And all the codec processes are performed in the shaping domain. For intra prediction, the neighboring blocks are in the shaping domain; for inter prediction, the reference block (generated from the original domain of the decoded image buffer) is first converted into the reshaped domain. The residual is then generated and encoded into a bitstream.

After the entire picture (or slice group) is encoded/decoded, samples in the reshaped domain are converted to the original domain, and then a deblocking filter and other filters are applied.

Forward shaping of the prediction signal is disabled for the following cases:

-the current block is intra coded;

the current block is coded as CPR (current image reference, also called intra block copy, IBC);

the current block is coded as combined intra mode (CIIP) and forward shaping is disabled for intra prediction blocks.

Syntax and semantics in 2.1.2.4 VVC working draft 5

7.3.2.3 sequence parameter set RBSP syntax

7.3.2.5 adaptive parameter set syntax

7.3.5 stripe header syntax

7.3.5.1 generic slice header syntax

7.3.5.4 luma map with chroma scaling data syntax

LMCS APS: APS _ params _ type is equal to APS of LMCS _ APS.

sps _ lmcs _ enabled _ flag equal to 1 specifies that luminance mapping with chroma scaling is used in CVS. sps _ lmcs _ enabled _ flag equal to 0 specifies that the luma map with chroma scaling is not used in CVS.

7.4.3.5 adaptive parameter set semantics

The adaptation _ parameter _ set _ id provides an identifier for the APS to refer to for other syntax elements.

Note that APS may be shared across images and may be different within different stripes of an image.

The APS _ params _ type specifies the type of APS parameter carried in the APS, as shown in Table 7-2.

TABLE 7-2 APS parameter type code and type of APS parameter

aps_params_type	Name of aps _ params _ type	Type of APS parameter
			0	ALF_APS	ALF parameters
1	LMCS_APS	LMCS parameters
			2.7	Reservation	Reservation

slice _ lmcs _ enabled _ flag equal to 1 specifies that the current slice enables luma mapping with chroma scaling. slice _ lmcs _ enabled _ flag equal to 0 specifies that luma mapping with chroma scaling is not enabled for the current slice. When slice _ lmcs _ enabled _ flag is not present, it is inferred to be equal to 0.

slice _ LMCS _ APS _ id specifies the adaptation _ parameter _ set _ id of the LMCS APS to which the slice refers. The temporalld of LMCS APS NAL units with adaptation _ parameter _ set _ id equal to slice _ lmcs _ aps _ id should be less than or equal to the temporalld of the coded slice NAL unit.

When two or more slices of the same picture refer to a plurality of LMCS APSs having the same adaptation _ parameter _ set _ id value, the plurality of LMCS APSs having the same adaptation _ parameter _ set _ id value should have the same contents.

7.4.6.4 luma map with chroma scaling data semantics

lmcs _ min _ bin _ idx specifies the minimum bin (bin) index used in the luma map and chroma scaling construction process. The value of lmcs _ min _ bin _ idx should be in the range of 0 to 15 (including 0 and 15).

lmcs _ delta _ max _ bin _ idx specifies the delta value between 15 and the maximum bin index LmcsMaxBinIdx used during the luma map and chroma scale construction process. The value of lmcs _ delta _ max _ bin _ idx should be in the range of 0 to 15 (including 0 and 15). The value of LmscMaxBinIdx is set to 15-lmcs _ delta _ max _ bin _ idx. The value of LmscMaxBinIdx should be greater than or equal to lmcs _ min _ bin _ idx.

lmcs _ delta _ cw _ prec _ minus1 plus 1 specifies the number of bits used to represent the syntax lmcs _ delta _ abs _ cw [ i ]. The value of lmcs _ delta _ cw _ prec _ minus1 should be in the range of 0 to BitDepthY-2 (including 0 and BitDepthY-2).

lmcs _ delta _ abs _ cw [ i ] specifies the absolute delta codeword value for the ith bin.

lmcs _ delta _ sign _ cw _ flag [ i ] specifies the sign of variable lmcsDeltaCW [ i ], as follows:

-if lmcs _ delta _ sign _ cw _ flag [ i ] is equal to 0, then lmcsDeltaCW [ i ] is positive.

-otherwise (lmcs _ delta _ sign _ cw _ flag [ i ] is not equal to 0), lmcsDeltaCW [ i ] is negative.

When lmcs _ delta _ sign _ cw _ flag [ i ] is not present, it is inferred to be equal to 0.

The variable OrgCW is derived as follows:

OrgCW＝(1<<BitDepth_Y)/16 (7-77)

the variable lmcsDeltaCW [ i ], i ═ lmcs _ min _ bin _ idx … LmcsMaxBinIdx, is derived as follows:

lmcsDeltaCW[i]＝(1-2*lmcs_delta_sign_cw_flag[i])*lmcs_delta_abs_cw[i]

(7-78)

the variable lmcsCW [ i ] is derived as follows:

-for i equal to 0 … lmcs _ min _ bin _ idx-1, lmcsCW [ i ] is set equal to 0.

For i ═ lmcs _ min _ bin _ idx … LmcsMaxBinIdx, the following equation applies:

lmcsCW[i]＝OrgCW+lmcsDeltaCW[i] (7-79)

the value of lmcsCW [ i ] should be in the range of (OrgCW > >3) to (OrgCW < <3-1) (inclusive of (OrgCW > > 3).

-for i ═ LmcsMaxBinIdx +1 … 15, lmcsCW [ i ] is set equal to 0.

The following condition is satisfied as a requirement of bitstream conformance (conformance):

the variables inputpdivot [ i ], i ═ 0 … 16, are derived as follows:

InputPivot[i]＝i*OrgCW (7-81)

variables lmcspinvot [ i ], i-0 … 16, variables ScaleCoeff [ i ] and InvScaleCoeff [ i ], i-0 … 15, derived as follows:

the variable ChromaScaleCoeff [ i ], i ═ 0 … 15, is derived as follows:

the variables ClipRange, LmscMinVal, and LmscMaxVal are derived as follows:

ClipRange＝((lmcs_min_bin_idx>0)&&(LmcsMaxBinIdx<15) (7-84)

LmcsMinVal＝16<<(BitDepth_Y-8) (7-85)

LmcsMaxVal＝235<<(BitDepth_Y-8) (7-86)

note that the arrays InputPivot [ i ] and LmscPivot [ i ], ScalCoeff [ i ] and InvScalCoeff [ i ], ScalCoeff [ i ], ClipRange, LmscMinVal and LmscMaxVal are updated only when slice _ lmcs _ model _ present _ flag is equal to 1. Thus, for example, the lmcs model may be sent with an IRAP image, but the lmcs is disabled for that IRAP image.

2.1.3 JVET-N0220

JFET-N0220 proposes four aspects:

1. a reduction in pipeline delay for calculating chroma residual scaled average luma;

2.a reduction in the size of a local buffer required to store chroma residual samples;

3. unification of fixed point precision used in luminance mapping and chroma residual scaling;

4. unification of the method of computing chroma residual scaling with the method of computing inverse luma scaling, and removal of pre-computed LUTs.

Items 2-4 have been adopted by VTM-5.

According to the adopted part, the working draft is revised as follows.

7.4.5.4 luma map with chroma scaling data semantics

lmcs _ delta _ sign _ cw _ flag [ i ] specifies the sign of variable lmcsDeltaCW [ i ], as follows:

-if lmcs _ delta _ sign _ cw _ flag [ i ] is equal to 0, then lmcsDeltaCW [ i ] is positive.

-otherwise (lmcs _ delta _ sign _ cw _ flag [ i ] is not equal to 0), lmcsDeltaCW [ i ] is negative.

When lmcs _ delta _ sign _ cw _ flag [ i ] is not present, it is inferred to be equal to 0.

The variable OrgCW is derived as follows:

OrgCW＝(1<<BitDepthy)/16 (7-70)

the variable lmcsDeltaCW [ i ], i ═ lmcs _ min _ bin _ idx … LmcsMaxBinIdx, is derived as follows:

lmcsDeltaCW[i]＝(1-2*lmcs_delta_sign_cw_flag[i])*lmcs_delta_abs_cw[i]

(7-71)

the variable lmcsCW [ i ] is derived as follows:

-for i equal to 0 … lmcs _ min _ bin _ idx 1, lmcsCW [ i ] is set equal to 0.

For i ═ lmcs _ min _ bin _ idx … LmcsMaxBinIdx, the following equation applies:

lmcsCW[i]＝OrgCW+lmcsDeltaCW[i] (7-72)

the value of lmcsCW [ i ] should be in the range of (OrgCW > >3) to (OrgCw < <3-1), including (OrgCW > >3) and (OrgCw < < 3-1).

-for i ═ LmcsMaxBinIdx +1 … 15, lmcsCW [ i ] is set equal to 0.

The following condition is satisfied for bitstream conformance:

the variables inputpdivot [ i ], i ═ 0.. 16, are derived as follows:

InputPivot[i]＝i*OrgCW (7-74)

variables lmcspinvot [ i ], i 0.. 16, variables ScaleCoeff [ i ] and InvScaleCoeff [ i ], i 0.. 15, derived as follows:

LmcsPivot[0]＝0；

for(i＝0；i<＝15；i++){

LmcsPivot[i+1]＝LmcsPivot[i]+lmcsCW[i]

ScaleCoeff[i]＝(lmcsCW[i]*(1<<11)+(1<<(Log2(OrgCW)-1)))>>

(Log2(OrgCW))

(7-75)

if(lmcsCW[i]＝＝0)

InvScaleCoeff[i]＝0

else

InvScaleCoeff[i]＝OrgCW*(1<<11)/lmcsCW[i]

}

the variable ChromaScaleCoeff [ i ], i ═ 0 … 15, is derived as follows:

if(lmcsCW[i]＝＝0)

ChromaScaleCoeff[i]＝(1<<11)

else{

ChromaScaleCoeff[i]＝InvScaleCoeff[i]

}

the variables ClipRange, LmscMinVal, and LmscMaxVal are derived as follows:

ClipRange＝((lmcs_min_bin_idx>0)&&(LmcsMaxBinIdx<15) (7-77)

LmcsMinVal＝16<<(BitDepth_Y-8) (7-78)

LmcsMaxVal＝235<<(BitDepth_Y-8)

8.5.7.6 weighted sample prediction process for combined merge and intra prediction

…

idxY＝predSamplesInter[x][y]>>Log2(OrgCW)

predSamplesInter[x][y]＝Clip1_Y(LmcsPivot[idxY]+ (8-888)

(ScaleCoeff[idxY]*(predSamplesInter[x][y]-InputPivot[idxY])+(1<<10))>>11)

8.7.4.2 image reconstruction with a mapping process of intensity samples

…

idxY＝predSamples[i][j]>>Log2(OrgCW)

PredMapSamples[i][j]＝LmcsPivot[idxY]

+(ScaleCoeff[idxY]*(predSamples[i][j]-InputPivot[idxY])+(1<<10))>>11 (8-1058)

i＝0...nCurrSw-1,j＝0…nCurrSh–1

8.7.4.3.1 inverse image mapping process of luminance samples

…

The variable invSample is derived as follows:

invSample＝InputPivot[idxYInv]+(InvScaleCoeff[idxYInv]*(8-1060)

(lumaSample[xP][yP]-LmcsPivot[idxYInv])+(1<<10))>>11

8.7.4.4 image reconstruction with luma-based chroma residual scaling procedure of chroma samples

…

The variable idxYInv is derived by calling the identification of the piecewise function index (with invAvgLuma as input and idxYInv as output) as specified in clause 8.7.5.3.2.

1. The variable varScale is derived as follows:

varScale＝ChromaScaleCoeff[idxYInv] (8-1065)

-recSamples is derived as follows:

-if tu _ cbf _ cIdx [ xCyrr ] [ yCurr ] is equal to 1, then the following equation applies:

resSamples[i][j]＝Clip3(-(1<<BitDepth_C),1<<BitDepth_C-1,resSamples[i][j])

recSamples[xCurr+i][yCurr+j]＝ClipCidx1(predSamples[i][j]+

(8-1066)

Sign(resSamples[i][j])*((Abs(resSamples[i][j])*varScale+(1<<10))>>11))

(8-1067)

else (tu _ cbf _ cIdx [ xCurr ] [ yCurr ] equals 0), the following equation applies:

recSamples[xCurr+i][yCurr+j]＝ClipCidx1(predSamples[i][j])

(8-1067)

2.2 Adaptive Parameter Set (APS)

The VVC employs an Adaptive Parameter Set (APS) to carry the ALF parameters. The slice group header includes aps _ id, which is conditionally present when ALF is enabled. The APS contains APS _ id and ALF parameters. APS (from jfet-M0132) are assigned new NUT (NAL unit types, such as AVC and HEVC). For common test conditions in (upcoming) VTM-4.0, it is recommended to use APS _ id ═ 0 only, and to transmit APS with each image. Currently, APS ID values will range from 0 to 31, and APS may be shared across images (and may be different in different slice groups within the same image). When present, the ID value should be fixed length codec. Different contents within the same image cannot reuse the ID value.

7.3.5.3 adaptive Loop Filter data syntax

7.3.7.2 coding and decoding tree unit syntax

slice _ alf _ enabled _ flag equal to 1 specifies that the adaptive loop filter is enabled and applicable to Y, Cb or Cr color components in the slice. slice _ alf _ enabled _ flag equal to 0 specifies that the adaptive loop filter is disabled for all color components in the slice.

num _ ALF _ APS _ ids _ minus1 plus 1 specifies the number of ALF APS referenced by the stripe. The value of num _ alf _ aps _ ids _ minus1 should be in the range of 0 to 7 (including 0 and 7).

slice _ ALF _ APS _ id [ i ] specifies the adaptation _ parameter _ set _ id of the ith ALF APS to which the slice refers. The Temporalld of ALF APS NAL units with adaptation _ parameter _ set _ id equal to slice _ alf _ aps _ id [ i ] should be less than or equal to the Temporalld of the coded slice NAL unit.

When two or more slices of the same picture refer to a plurality of ALF APs having the same adaptation _ parameter _ set _ id value, a plurality of LMCS APs having the same adaptation _ parameter _ set _ id value should have the same contents.

an alf _ ctb _ flag [ cIdx ] [ xCtbb > > Log2CtbSize ] [ yCtb > > Log2CtbSize ] equal to 1 specifies that the adaptive loop filter is applied to the codec tree block for the color component indicated by the cIdx of the codec tree unit at luma position (xCtb, yCtb). an alf _ ctb _ flag [ cIdx ] [ xCtbb > > Log2CtbSize ] [ yCtb > > Log2CtbSize ] equal to 0 specifies that the adaptive loop filter is not applied to the codec tree block for the color component indicated by the cIdx of the codec tree unit at luma position (xCtb, yCtb).

When alf _ ctb _ flag [ cIdx ] [ xCtbb > > Log2CtbSize ] [ yCtb > > Log2CtbSize ] is absent, it is inferred to be equal to 0.

2.3 Cross Component Linear Model (CCLM) Intra prediction in VVC

To reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in VTM4 for which chroma samples are predicted based on reconstructed luma samples of the same CU by using the following linear model:

pred_C(i,j)＝α·rec_L′(i,j)+β (3-1)

here pred_C(i, j) represents the predicted chroma sampling in the CU, and rec_L(i, j) represents reconstructed luma samples of the same CU after downsampling. The linear model parameters α and β are derived from the relationship between the luminance and chrominance values of two samples, the luminance sample having the smallest and largest sample value within the down-sampled set of adjacent luminance samples, and their corresponding chrominance samples. The linear model parameters α and β are obtained according to the following equations.

β＝Y_b-á·X_b (3-3)

Here, Y_aAnd X_aRepresenting the luminance and chrominance values of the luminance sample having the largest luminance sample value. And X_bAnd Y_bRespectively, the luminance value and the chrominance value of the luminance sample having the smallest luminance sample. Fig. 2 shows an example of the positions of the left and upper samples and the samples of the current block involved in CCLM mode.

The division operation for calculating the parameter α is implemented using a look-up table. In order to reduce the memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are represented in an exponential representation. For example, diff is approximated by a 4-bit significance and exponent. Thus, for 16 values of the significand (significant), the table of 1/diff is reduced to 16 elements, as follows:

DivTable[]＝{0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0}。

this will advantageously reduce the computational complexity and the memory size required to store the required tables.

In addition to the top and left templates that can be used together to compute the linear model coefficients, they can also be used alternately in the other 2 LM modes (referred to as LM _ a and LM _ L modes).

In LM _ A mode, linear model coefficients are calculated using only the upper template. To get more spots, the top template is expanded to (W + H). In LM _ L mode, only the left template is used to compute the linear model coefficients. To get more samples, the left template is expanded to (H + W).

For non-square blocks, the top template is expanded to W + W and the left template is expanded to H + H.

To match the chroma sample positions of a 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a downsampling ratio of 2 to 1 in both the horizontal and vertical directions. The selection of the downsampling filter is specified by the SPS level flag. The two downsampling filters correspond to the "type-0" and "type-2" contents, respectively, as shown below.

Note that when the upper reference line is located at the CTU boundary, only one luma line (a general line buffer in intra prediction) is used to obtain downsampled luma samples.

The parameter calculation is performed as part of the decoding process, not just as an encoder search operation. Therefore, no syntax is used to convey the α and β values to the decoder.

For intra chroma mode coding, a total of 8intra modes are allowed for intra chroma mode coding. These modes include five conventional intra modes and three cross-component linear model modes (CCLM, LM _ a, and LM _ L). The chroma mode signaling and derivation procedure is shown in the following table. Chroma mode coding depends directly on the intra prediction mode of the corresponding luma block. Since a separate block division (partition) structure of luminance and chrominance components is enabled in I stripes, one chrominance block may correspond to a plurality of luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited (inherit).

The decoding procedure specified in JFET-N1001-v 2 is as follows.

8.4.4.2.8 specification of INTRA prediction modes INTRA _ LT _ CCLM, INTRA _ L _ CCLM and INTRA _ T _ CCLM

…

8. The variables a, b, and k are derived as follows:

if numSampL is equal to 0 and numSampT is equal to 0, then the following equation applies:

k＝0 (8-208)

a＝0 (8-209)

b＝1<<(BitDepth_C-1) (8-210)

otherwise, the following equation applies:

diff＝maxY-minY (8-211)

if diff is not equal to 0, the following equation applies:

diffC＝maxC–minC (8-212)

x＝Floor(Log2(diff)) (8-213)

normDiff＝((diff<<4)>>x)&15 (8-214)

x+＝(normDiff！＝0)？1:0 (8-215)

y＝Floor(Log2(Abs(diffC)))+1 (8-216)

a＝(diffC*(divSigTable[normDiff]|8)+2^y-1)>>y (8-217)

k＝((3+x-y)<1)？1:3+x-y (8-218)

a＝((3+x-y)<1)？Sign(a)*15:a (8-219)

b＝minC-((a*minY)>>k) (8-220)

wherein divSigTable [ ] is specified as follows:

divSigTable[]＝{0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0}(8-221)

otherwise (diff equals 0), the following equation applies:

k＝0 (8-222)

a＝0 (8-223)

b＝minC (8-224)

9. the predicted sample points predSamples [ x ] [ y ], x ═ 0.. nTbW-1, y ═ 0.. nTbH-1 are derived as follows:

predSamples[x][y]＝Clip1C(((pDsY[x][y]*a)>>k)+b)(8-225)

2.4 MV scaling in VVC

MV scaling is applied to Temporal Motion Vector Prediction (TMVP) and AMVP.

In JFET-N1001, MV scaling is described as follows:

8.5.2.12 derivation of collocated motion vectors

…

Otherwise, mvLXCol is derived as a scaled version of the motion vector mvCol as follows:

tx＝(16384+(Abs(td)>>1))/td (8-421)

distScaleFactor＝Clip3(-4096,4095,(tb*tx+32)>>6)(8-422)

mvLXCol＝Clip3(-131072,131071,Sign(distScaleFactor*mvCol)*

((Abs(distScaleFactor*mvCol)+127)>>8))

(8-423)

where td and tb are derived as follows:

td＝Clip3(-128,127,colPocDiff) (8-424)

td＝Clip3(-128,127,currPocDiff) (8-425)

the division operator used to derive tx can be implemented by a look-up table. The size of the lookup table MV _ SCALE _ T [ poc _ diff _ idx ] may be 128, and MV _ SCALE _ T [ poc _ diff _ idx ] (16384+ (Abs (poc _ diff _ idx) > >1))/poc _ diff _ idx, from 1 to 128.

2.5 local illumination Compensation

Local Illumination Compensation (LIC) is based on a linear model of the illumination variation, using a scaling factor a and an offset b. And adaptively enable or disable Local Illumination Compensation (LIC) for each inter mode Codec Unit (CU).

When LIC is applied to a CU, a least square error method is used to derive the parameters a and b by using neighboring samples of the current CU and their corresponding reference samples (also referred to as reference neighboring samples). More specifically, as shown in fig. 3, sub-sampled (2:1 sub-sampled) neighboring samples of the CU and corresponding samples in the reference image (identified by motion information of the current CU or sub-CU) are used. IC parameters are derived and applied separately for each prediction direction.

When a CU is coded in merge mode, copying LIC flags from neighboring blocks in a manner similar to motion information copying in merge mode; otherwise, the LIC flag is signaled to the CU to indicate whether LIC is applicable.

When LIC is enabled for an image, an additional CD level RD is needed to determine if LIC is applicable to the CU. When LIC is enabled for a CU, instead of using SAD and SATD, the sum of absolute differences after mean removal (MR-SAD) and the sum of absolute Hadamard transform differences after mean removal (MR-SATD) are used for integer-pixel (pel) motion search and fractional-pixel motion search, respectively.

In order to reduce the codec complexity, the following codec scheme is applied in JEM.

When there is no significant illumination change between the current image and its reference image, LIC is disabled for the entire image. To identify this situation, a histogram of the current picture and each reference picture of the current picture are computed at the codec. Disabling LIC for the current picture if the histogram difference between the current picture and each reference picture of the current picture is less than a given threshold; otherwise, the LIC is enabled for the current image.

3 disadvantages of the prior implementations

The current designs for IRL (also known as LMCS) and CCLM may suffer from the following problems:

(1) there is a division operation in the process of LMCS, which is not desirable in hardware design.

(2) In CCLM mode, chroma samples are in the original domain, but luma samples are in the reshaped domain of LMCS, which may be inefficient.

(3) The signaling method of the LMCS model may be inefficient.

Exemplary method for lossless codec of 4-vision media codec

Embodiments of the presently disclosed technology overcome the shortcomings of existing implementations, thereby providing video codecs with higher codec efficiency. Methods for scaling and division operations in video codecs based on the disclosed techniques may enhance existing and future video codec standards and are set forth in the examples described below for various implementations. The examples of the disclosed technology provided below illustrate the general concepts and are not intended to be construed as limiting. In examples, various features described in these examples may be combined unless explicitly indicated to the contrary.

In the discussion that follows, SatShift (x, n) is defined as

In one example, offset0 and/or offset1 is set to (1< < n) > > 1or (1< < n-1). In another example, offset0 and/or offset1 are set to 0.

In another example, offset0 ═ offset1 ═ ((1< < n) > >1) -1or ((1< (n-1))) -1.

Clip3(min, max, x) is defined as

Floor (x) is defined as the largest integer less than or equal to x.

Ceil (x) is the smallest integer greater than or equal to x.

Log2(x) is defined as the base 2 logarithm of x.

Division replacement in LMCS

1. It is proposed that division operations in the video/image codec/decoding process (such as InvScaleCoeff [ i ] ═ OrgCW (1< <11)/lmcsCW [ i ] in LMCS method) can be replaced or approximated by an operation or a multiple operation process.

a. In one example, the process of operation or operations may include an operation that indexes an entry of a lookup table.

i. In an alternative example, it may include an operation that queries multiple entries of one or more tables with an index.

b. In one example, the process of operation or operations may include operations that are not division operations.

i. In one example, it may comprise a multiplication operation.

in one example, it may comprise an addition operation.

in one example, it may include the operation of SatShift (x, n).

in one example, it may include the operation of Shift (x, n).

v. in one example, it may include a left shift operation.

In one example, it may include the operations of floor (x).

In one example, it may include the operation of Log2 (x).

In one example, it may include an operation of "logical OR" (in C language |).

in one example, it may comprise "logical AND" (AND in C language)

And &) is calculated.

2. In one example, a table denoted T [ idx ] may be used to replace or approximate the division operation in term 1.

a. In one example, the table size may be equal to 2^MAnd idx can be in [0,2 ]^M-1]Within a range of (1), including 0 and 2^M-1。

b. In one example, T [ idx ]]＝Rounding(2^P/(idx + offset0)) -offset1, wherein

offset0 and offset1 are integers. P is an integer. For example, P-8 or P-9 or P-10 or

P-11 or P-12 or P-13 or P-14.

i. In one example, round (x/y) ═ Floor ((x + y/2)/y);

in one example, Rounding (x/y) is set equal to the integer Q, such that for any integer Q 'in the set, | Q y-x | < | Q'. y-x |. For example, the set may be { Floor (x/y) -1, Floor (x/y) +1 }.

in one example, offset0 may be equal to 2^WFor example, W is 0 or W is 1or W is 4, or W is 5, or W is 6, or W is 7.

1) Alternatively, offset0 may be 0.

in one example, offset1 may be equal to 2^ZFor example, Z is 3, or Z is 4, or Z is 5, or Z is 6, or Z is 7.

1) Alternatively, offset1 may be 0.

c. In one example, M as defined in item 2.a can be equal to W as defined in item 2. b.

d. In one example, Z can be equal to P-W-1, where Z, P and W are defined in item 2. b.

e. In one example, T [ idx ]]Should be less than 2^Z。

i. In one example, if T [ idx ]]Is equal to 2^ZIt may be set equal to 0.

ii, e.g., T [ idx ]]＝(Rounding(2^P/(idx+offset0))-offset1)％offset1。

3. Some examples of tables are shown below.

a.W＝M＝7，P＝15，

T＝{0,126,124,122,120,118,117,115,113,111,109,108,106,104,103,101,100,98,96,95,93,92,90,89,88,86,85,83,82,81,79,78,77,76,74,73,72,71,69,68,67,66,65,64,63,61,60,59,58,57,56,55,54,53,52,51,50,49,48,47,46,45,44,44,43,42,41,40,39,38,37,37,36,35,34,33,33,32,31,30,30,29,28,27,27,26,25,24,24,23,22,22,21,20,20,19,18,18,17,16,16,15,14,14,13,13,12,11,11,10,10,9,9,8,7,7,6,6,5,5,4,4,3,3,2,2,1,1}。

b.W＝M＝7，P＝14，

T＝{0,63,62,61,60,59,58,57,56,56,55,54,53,52,51,51,50,49,48,47,47,46,45,45,44,43,42,42,41,40,40,39,38,38,37,37,36,35,35,34,34,33,32,32,31,31,30,30,29,29,28,28,27,27,26,26,25,25,24,24,23,23,22,22,21,21,20,20,20,19,19,18,18,18,17,17,16,16,16,15,15,14,14,14,13,13,13,12,12,12,11,11,10,10,10,9,9,9,8,8,8,8,7,7,7,6,6,6,5,5,5,5,4,4,4,3,3,3,3,2,2,2,2,1,1,1,1,0}。

c.W＝M＝6，P＝13，

T＝{0,0,31,31,30,30,29,29,28,28,27,27,27,26,26,25,25,24,24,24,23,23,23,22,22,22,21,21,21,20,20,20,19,19,19,18,18,18,17,17,17,16,16,16,16,15,15,15,15,14,14,14,14,13,13,13,13,12,12,12,12,11,11,11,11,10,10,10,10,10,9,9,9,9,9,8,8,8,8,8,7,7,7,7,7,6,6,6,6,6,6,5,5,5,5,5,5,4,4,4,4,4,4,3,3,3,3,3,3,3,2,2,2,2,2,2,2,1,1,1,1,1,1,1,1,0,0,0}。

d.W＝M＝6，P＝14，

T＝{0,124,120,117,113,109,106,103,100,96,93,90,88,85,82,79,77,74,72,69,67,65,63,60,58,56,54,52,50,48,46,44,43,41,39,37,36,34,33,31,30,28,27,25,24,22,21,20,18,17,16,14,13,12,11,10,9,7,6,5,4,3,2,1}。

e.W＝M＝6，P＝13，

T＝{0,62,60,58,56,55,53,51,50,48,47,45,44,42,41,40,38,37,36,35,34,32,31,30,29,28,27,26,25,24,23,22,21,20,20,19,18,17,16,16,15,14,13,13,12,11,10,10,9,8,8,7,7,6,5,5,4,4,3,3,2,2,1,1}。

f.W＝M＝6，P＝12，

T＝{0,31,30,29,28,27,27,26,25,24,23,23,22,21,21,20,19,19,18,17,17,16,16,15,15,14,14,13,13,12,12,11,11,10,10,9,9,9,8,8,7,7,7,6,6,6,5,5,5,4,4,4,3,3,3,2,2,2,2,1,1,1,1,0}。

g.W＝M＝6，P＝11，

T＝{0,0,15,15,14,14,13,13,12,12,12,11,11,11,10,10,10,9,9,9,8,8,8,8,7,7,7,7,6,6,6,6,5,5,5,5,4,4,4,4,4,4,3,3,3,3,3,2,2,2,2,2,2,2,1,1,1,1,1,1,1,0,0,0}。

h.W＝M＝6，P＝10，

T＝{0,0,0,7,7,7,7,6,6,6,6,6,5,5,5,5,5,5,4,4,4,4,4,4,4,4,3,3,3,3,3,3,3,3,2,2,2,2,2,2,2,2,2,2,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,0,0,0,0}。

i.W＝M＝5，P＝13，

T＝{0,120,113,106,100,93,88,82,77,72,67,63,58,54,50,46,43,39,36,33,30,27,24,21,18,16,13,11,9,6,4,2}。

j.W＝M＝5，P＝12，

T＝{0,60,56,53,50,47,44,41,38,36,34,31,29,27,25,23,21,20,18,16,15,13,12,10,9,8,7,5,4,3,2,1}。

k.W＝M＝5，P＝11，

T＝{0,30,28,27,25,23,22,21,19,18,17,16,15,14,13,12,11,10,9,8,7,7,6,5,5,4,3,3,2,2,1,1}。

l.W＝M＝5，P＝10，

T＝{0,15,14,13,12,12,11,10,10,9,8,8,7,7,6,6,5,5,4,4,4,3,3,3,2,2,2,1,1,1,1,0}。

m.W＝M＝5，P＝9，

T＝{0,0,7,7,6,6,5,5,5,4,4,4,4,3,3,3,3,2,2,2,2,2,1,1,1,1,1,1,1,0,0,0}。

n.W＝M＝4，P＝10，

T＝{0,28,25,22,19,17,15,13,11,9,7,6,5,3,2,1}。

o.W＝M＝4，P＝9，

T＝{0,14,12,11,10,8,7,6,5,4,4,3,2,2,1,1}。

p.offset0＝0，M＝8，P＝14

T[k]＝(2¹⁴+ k/2)/k, k is from 1 to 256;

q.offset0＝0，M＝8，P＝11

T[k]＝(2¹¹+ k/2)/k, k is from 1 to 256;

r.offset0＝0，M＝9，P＝14

T[k]＝(2¹⁴+ k/2)/k, k is from 1 to 512;

s.offset0＝0，M＝9，P＝11

T[k]＝(2¹¹+ k/2)/k, k is from 1 to 512;

t.offset0＝0，M＝10，P＝14

T[k]＝(2¹⁴+ k/2)/k, k is from 1 to 1024;

u.offset0＝0，M＝10，P＝11

T[k]＝(2¹¹+ k/2)/k, k is from 1 to 1024;

v.T＝{0,248,240,233,226,219,212,206,199,193,187,181,175,170,164,159,154,149,144,139,134,130,125,121,116,112,108,104,100,96,93,89,85,82,78,75,72,68,65,62,59,56,53,50,47,45,42,39,37,34,31,29,26,24,22,19,17,15,13,10,8,6,4,2}

4. when T [ idx ] is used in the course of multiple operations to replace or approximate the division operation in term 2, idx can be derived in the course of multiple operations.

a. In one example, the process of multiple operations may include addition, Log2(x), floor (x), left Shift, Shift (x, n), or SatShift (x, n), "logical or" and "logical and".

b. In one example, the process of the plurality of operations may depend on M or W as defined in item 2.

c. In one example, idx may be set equal to Shift (D)<<M,Floor(Log2(D)))&(2^M-1), where D is the denominator, such as lmcsCW [ i ] in LMCS procedures]。

idx can be determined by adding the offset idx ═ Shift (D)<<M,Floor(Log2(D)))&(2^M-1)) + Off to be further modified.

1) The modified idx can be clipped (clip) to [0,2 ]^M-1]Range of (1), including 0 and 2^M-1。

2) For example, idx ═ Min (2)^M-1,idx0+ Off), where idx0 ═ Shift (D)<<M,Floor(Log2(D)))&(2^M-1) and Off ═ Shift (D)<<(M+1),Floor(Log2(D))))&1。

d. In one example, the variable M in item 4.c can be replaced with W as defined in item 2.

e. Alternatively, in addition, clipping may be further applied to the result values.

5. It is proposed to modify T idx to an intermediate value V to be used during multiple operations to replace or approximate the division operation in term 2.

a. In one example, V ═ T [ idx ].

b. In one example, V ═ T [ idx ] < < m, where m is an integer.

c. In one example, V ═ T [ idx ] × m, where m is an integer.

d. In one example, V ═ T [ idx ] + m, where m is an integer.

e. In one example, V ═ Shift (T [ idx ], n) or V ═ SatShift (T [ idx ], n), where m is an integer such as 1or 2.

1) In one example, the value n may depend on the value of D as defined in item 4.

a) For example, if D<＝2^MAnd n is 0, wherein M is defined in item 2.

b) For example, if D>2^M，n＝Ceil(Log2(D))-M。

c) For example, if D>2^M，n＝Floor(Log2(D))-M+1。

f. In one example, the modification method may depend on the value of T [ idx ] and/or the value of idx.

i. In one example, the modification method may depend on whether the value of T [ idx ] is equal to a fixed value such as 0.

in one example, the modification method may depend on whether the value of idx is less than the value TT, e.g., TT ═ 1or 2^M-1. In another example, the modification method may depend on whether the value of idx is equal to a fixed value such as 0.

in one example, the modification method can be dependent on T [ 2 ]idx]Whether the value of (a) is equal to 0 and whether the value of idx is less than the value TT, e.g. TT ═ 1or 2^M-1. In another example, the modification method may depend on T [ idx ]]Is equal to a fixed value such as 0, and idx is equal to a fixed value such as 0.

g. In one example, V ═ T [ idx ═ T [ ]]|2^ZWherein Z is defined in item 2.

i. Alternatively, if T [ idx ]]Is equal to 0 and idx is smaller than a threshold value TT, e.g. TT ═ 1or 2^{M -1}When V is 2^Z+1(ii) a Otherwise, V ═ T [ idx [ ]]|2^Z。

Alternatively, if T [ idx ]]Is equal to 0 and idx is equal to 0, then V is 2^Z+1(ii) a Otherwise, V ═ T [ idx [ ]]|2^Z。

h. In one example, V ═ T [ idx ═ T [ ]]+2^ZWherein Z is defined in item 2.

i. Alternatively, if T [ idx ]]Is equal to 0 and idx is smaller than a threshold value TT, e.g. TT ═ 1or 2^{M -1}When V is 2^Z+1(ii) a Otherwise, V ═ T [ idx [ ]]+2^Z。

Alternatively, if the value of idx is equal to 0, then V ═ 2^Z+1(ii) a Otherwise, V ═ T [ idx [ ]]+2^Z。

6. It is proposed that multiplication of the numerator to be replaced or approximated in the division operation (such as OrgC in LMCS methods, denoted N) by the intermediate value V defined in item 5 can be used as a replacement or approximation for the division result. In a formulaic manner, R ═ N × V may be used as an alternative or approximation to the division result.

a. Alternatively, a modified R, denoted as R', may be used as an alternative or approximation to the division result.

i. In one example, R' ═ R < < m, where m is an integer such as 11.

1) Alternatively, R '═ Shift (R, m) or R' ═ SatShift (R, m) where m is an integer.

2) In one example, m may depend on the value n defined in item 5. e.

a) For example, m is offset-n, where offset is an integer such as 11.

in one example, the modification method can depend on P and {. sup.

Or W (or M) as defined in item 2, and/or D as defined in item 4, and

or T and/or idx.

1) In one example, the modification method may depend on the relationship between the fixed numbers S and the function f of P, W (or M), T, idx, and D.

a) For example, f (P, W, D, T, idx) is P-W-1+ Log2(D), or f (P, M, D, T, idx) is P-M-1+ Log2 (D).

b) For example, f (P, W, D, T, idx) ═ P-W-1+ Log2(D) + off, or f (P, M, D, T, idx) ═ P-M-1+ Log2(D) + off.

i. In one example, if T [ idx ]]Is equal to 0 and idx is smaller than a threshold value TT, e.g. TT ═ 1or 2^M-1Off is equal to 0; otherwise, off is 1.

in one example, if the value of idx is equal to 0, off is equal to 0; otherwise, off is 1.

c) For example, f (P, W, D, T, idx) is P-W + Log2(D), or f (P, M, D, T, idx) is P-M + Log2 (D).

d) For example, f (P, W, D, T, idx) is P-W +1+ Log2(D), or f (P, M, D, T, idx) is P-M +1+ Log2 (D).

e) For example, S ═ 11 or 14.

f) For example, if f (P, W, D, T, idx) < ═ S, R' ═ R < (S-f (P, W, D, T, idx)); otherwise, R' ═ Shift (R, f (P, W, D, T, idx) -S).

g) For example, f (P, M, D, T, idx) < ═ S, R' ═ R < (S-f (P, M, D, T, idx)); otherwise, R' ═ Shift (R, f (P, M, D, T, idx) -S).

b. Alternatively, modified N (denoted N ') may be used to derive R, since R ═ N' ×

V。

i. In one example, N < < m, where m is an integer such as 11.

1) Alternatively, N '═ Shift (N, m) or N' ═ SatShift (N, m), where m is an integer.

2) In one example, m may depend on the value n defined in item 5. e.

a) For example, m is offset-n, where offset is an integer such as 11.

in one example, the modification method can depend on P and {. sup.

Or W (or M) as defined in item 2, and/or D as defined in item 4, and

or T and/or idx.

1) In one example, the modification method may depend on the relationship between the fixed numbers S and the function f of P, W (or M), T, idx, and D.

a) The function f can be defined as in item 6. a.

b) For example, S ═ 11.

c) For example, f (P, W, D) < ═ S, N' < (S-f (P, W, D)); otherwise, N' is Shift (N, f (P, W, D) -S).

d) For example, f (P, M, D) < ═ S, N' < (S-f (P, M, D));

otherwise, N' Shift (N, f (P, M, D) -S).

7. The result of a process that proposes an operation or operations that replace or approximate the division operator, denoted R, may be associated with a precision value, denoted Q. When R is used to calculate the variable B, B may be calculated as B ═ Shift (g (R), Q) or B ═ SatShift (g (R), Q), where g (R) is an arbitrary function with respect to R.

a. In one example of the LMCS method, R may be InvScaleCoeff [ idxYInv ], and the samples after inverse luma reconstruction may be derived as follows:

invSample＝InputPivot[idxYInv]+difference，

wherein the content of the first and second substances,

difference is Shift (g (InvScaleCoeff [ idxYInv ]), Q), and

g(InvScaleCoeff[idxYInv])＝InvScaleCoeff[idxYInv]*

(lumaSample[xP][yP]-LmcsPivot[idxYInv])，

Q＝11。

b. in one example of the LMCS method, R may be ChromaScaleCoeff [ idxYInv ], and the inverse chroma residual samples may be derived as follows:

invResSample[i][j]＝

sign (resSamples [ i ] [ j ]) (Shift (ChromaScaleCoeff [ idxYInv ], Q)), wherein,

g(ChromaScaleCoeff[idxYInv])＝Abs(resSamples[i][j])*ChromaScaleCoeff[idxYInv]，Q＝11。

c. in one example, Q may be a fixed number, such as 11 or 14.

d. In one example, Q may depend on P as defined in item 2 and/or W (or M) as defined in item 2, and/or D and/or table T as defined in item 4, and/or idx.

i. In one example, Q ═ P-W-1+ Log2(D), or Q ═ P-M-1+ Log2 (D).

For example, Q ═ P-W-1+ Log2(D) + off, or Q ═ P-M-1+ Log2(D) + off.

1) In one example, if T [ idx ]]Is equal to 0 and idx is smaller than a threshold value TT, e.g. TT ═ 1or 2^M-1Off is equal to 0; otherwise, off is 1.

For example, Q ═ P-W + Log2(D), or Q ═ P-M + Log2 (D).

For example, Q ═ P-W +1+ Log2(D), or Q ═ P-M +1+ Log2 (D).

e. In one example, Q may depend on n as defined in item 5. e.

i. For example, Q is offset-n, where offset is an integer such as 11 or 14.

8. It is proposed that D or the absolute value of D is always greater than the value G (such as 8), then the table size may depend on G.

a. In one example, G may depend on the sample bit depth.

b. In one example, the table size may depend on the sample bit depth.

Unification of division substitutions

9. In one example, the division operation used in MV scaling may be used in the course of multiple operations to replace or approximate the division operation in term 1.

a. In one example, D may be converted to an intermediate variable D' within a range [ minD, maxD ] (e.g., [ -128,127 ]), where D is the denominator (e.g., lmcsCW [ i ] in term 1).

i. In one example, the conversion from D to D' may be a linear or non-linear quantization process.

in one example, D' ═ Clip3(D, minD, maxD);

in one example, D' ═ Clip3(D,0, maxD);

in one example, D '═ SatShift (D, n) or D' ═ Shift (D, n), where n is

Is an integer such as 1or 2.

1) Alternatively, D 'Clip 3(Shift (D, n), minD, maxD) or D' Clip3(SatShift (D, n), -128,127), where n is an integer such as 1or 2.

2) Alternatively, D '═ Clip3(Shift (D, n),0, maxD) or D' ═ Clip3(SatShift (D, n),0, maxD), where n is an integer such as 1or 2.

3) In one example, the value of n may depend on the value of D.

a) For example, if D < ═ maxD, then n is 0.

b) For example, if D > maxD, then n ═ Ceil (Log2(D)) -Log2 (maxD).

c) For example, if D > maxD, then n ═ Floor (Log2(D)) - (Log2(maxD) -1).

b. In one example, the intermediate variable R is derived as R ═ (16384+ (Abs (D ') > > 1))/D', which can be used in the course of multiple operations to replace or approximate the division operation in term 1.

i. Alternatively, the table MV _ SCALE _ T may be used to derive R.

c. In one example, R in item 8.b can be associated with a precision value (denoted as Q). When R is used to calculate the variable B, B may be calculated as B ═ Shift (g (R), Q) or B ═ SatShift (g (R), Q), where g (R) is an arbitrary function with respect to R.

i. In one example, Q can depend on n in item 8. a.

1) Alternatively, Q may be equal to Offset-n, where Offset is an integer such as 14.

d. The disclosed MV scaling method may be used for MV scaling of Temporal Motion Vector Prediction (TMVP) incorporating inter mode or Advanced Motion Vector Prediction (AMVP) mode.

e. The disclosed MV scaling method can be used for MV scaling in affine prediction mode.

f. The disclosed MV scaling method may be used for MV scaling based on temporal merging mode of sub-blocks.

10. In one example, the same table may be used in the course of multiple operations to replace or approximate the division operations of CCLM and LMCS.

a. For example, the table may be any table shown in item 3, such as item 3. e.

11. In one example, the same table may be used in the process to replace or approximate the MV scaling and division operations in LMCS.

a. For example, the table may be any table shown in item 3, such as item 3. e.

12. In one example, a method of replacing or approximating a division operation in a CCLM may be used in a process to replace or approximate a division operation in Local Illumination Compensation (LIC).

a. Alternatively, a method of replacing or approximating a division in LMCS may be used in a process to replace or approximate a division in LIC.

b. Alternatively, a method that replaces or approximates the division operation in MV scaling may be used in the process to replace or approximate the division operation in LIC.

CCLM, LIC and LMCS

13. Even if the luma block is coded using LMCS, the chroma samples are predicted from the reconstructed luma samples in the original domain.

a. In one example, for CCLM mode, neighboring (neighboring or non-neighboring) reconstructed luminance samples in the reshaped domain may first be converted to the original domain, and then the converted luminance samples are used to derive linear model parameters, such as a and b.

b. In one example, for the CCLM mode, the collocated luma samples in the reshaped domain may be first converted to the original domain, and then the converted luma samples used with the linear model to generate the predicted chroma samples.

Signaling of LMCS model

14. An indication of the scaling factor used at the LMCS (e.g., ScaleCoeff [ i ] in the VVC) and/or the inverse scaling factor (e.g., InvScaleCoeff [ i ] in the VVC) may be signaled.

a. In one example, the scaling factor may be directly codec. Alternatively, the quantized value of the scaling factor may be signaled.

b. It may be signaled in a predictive manner.

c. It may be signaled in a fixed length codec, a unary codec, a truncated unary codec, or an exponential Golomb (Golomb) code.

d. It can be signaled in video units such as VPS/SPSS/PPS/APS/picture header/slice header.

15. An indication of the chroma scaling factor in LMCS, such as ChromaScaleCoeff [ i ], may be signaled.

a. In one example, the chroma scaling factor may be directly codec. Alternatively, the quantized value of the scaling factor may be signaled.

b. It may be signaled in a predictive manner.

c. It may be signaled using fixed length codecs, unary codecs, truncated unary codecs, or exponential golomb codes.

d. It can be signaled in video units such as VPS/SPSS/PPS/APS/picture header/slice header.

i. Alternatively, further, if chroma scaling is applied to the video blocks, all video blocks in the video unit may use the same signaled scaling factor.

16. It is proposed that signaling information may be used to indicate whether chroma scaling in LMCS is applied or not.

a. It can be signaled in video units such as VPS/SPSS/PPS/APS/picture header/slice header.

i. In one example, when luma and chroma samples are coded with a dual tree coding structure and a current slice or slice group is an intra coded slice or slice group, it may be signaled in a slice or slice group header.

The above examples may be incorporated in the context of a method, such as method 400, which may be implemented at a video decoder or a video encoder.

Fig. 4 shows a flow diagram of an exemplary method for video processing. The method 400 includes, at step 402, performing a conversion between a current video block of a video and a codec representation of the video, the current video block including a luma component and at least one chroma component, wherein the luma component is converted from an original domain to a reshaped domain using a luma mapping with chroma scaling (LMCS) scheme, and chroma samples of the at least one chroma component are predicted based on reconstructed luma samples of the luma component in the original domain.

In one aspect, a method for video processing is disclosed. The method comprises the following steps: a conversion between a current video block of video and a codec representation of the video is performed, the current video block comprising a luma component and at least one chroma component, wherein the luma component is converted from an original domain to a reshaped domain using a luma mapping with chroma scaling (LMCS) scheme, and chroma samples of the at least one chroma component are predicted based on reconstructed luma samples of the luma component in the original domain.

In an example, chroma samples of at least one chroma component are predicted in a cross-component linear model (CCLM) prediction mode.

In an example, adjacent reconstructed luminance samples are converted from the reshaped domain to the original domain, and the converted adjacent luminance samples are then used to derive linear model parameters in a linear model to be used in the CCLM prediction mode.

In an example, the neighboring reconstructed luma samples comprise neighboring or non-neighboring reconstructed luma samples in the reshaped domain.

In an example, the collocated reconstructed luma samples are converted from the reshaped domain to the original domain, and then the converted collocated luma samples are used in a linear model to predict chroma samples.

In an example, the first indication is signaled for indicating at least one of a scaling factor and an inverse scaling factor to be used in the LMCS scheme.

In an example, at least one of the scaling factor and the inverse scaling factor is codec.

In an example, a value of at least one of the scaling factor and the inverse scaling factor is quantized, and the quantized value of the at least one of the scaling factor and the inverse scaling factor is signaled.

In an example, the first indication is signaled in a predictive manner.

In an example, the first indication is coded using a fixed length code, a unary code, a truncated unary code, or an exponential golomb code.

In an example, the first indication is signaled in a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), an Adaptive Parameter Set (APS), a picture header, a slice header, or a slice group header.

In an example, the scaling factor comprises ScaleCoeff [ i ] to be used in the LMCS scheme.

In an example, the inverse scaling factor comprises InvScaleCoeff [ i ] to be used in the LMCS scheme.

In an example, the scaling factor comprises a chroma scaling factor to be used in the LMCS scheme.

In an example, the chroma scaling factor comprises ChromaScaleCoeff [ i ], and wherein i is an integer.

In an example, the second indication is signaled to indicate whether chroma scaling is applied to the current video block in the LMCS scheme.

In an example, if chroma scaling is applied to the current video block, all video blocks in the video unit that cover the current video block use the same chroma scaling factor signaled for the current video block.

In an example, the second indication is signaled in one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), an Adaptive Parameter Set (APS), a picture header, a slice header, or a slice group header.

In an example, if the luminance component and at least the chrominance component of the current video block are coded with a dual tree coding structure and a current slice or slice group covering the current video block is intra coded, the second indication is signaled in a slice header or a slice group header.

In an example, performing the conversion includes generating a codec representation from the current video block.

In an example, performing the conversion includes generating the current video block from the codec representation.

In another aspect, an apparatus in a video system is disclosed, the apparatus comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the above method.

In yet another aspect, a computer program product stored on a non-transitory computer readable medium is disclosed, the computer program product comprising program code for the above method.

5 example implementation of the disclosed technology

5.1 example #1

Variables lmcspinvot [ i ], i 0.. 16, variables ScaleCoeff [ i ] and InvScaleCoeff [ i ], i 0.. 15, derived as follows:

divTable [ ] is specified as follows:

divTable[]＝{0,62,60,58,56,55,53,51,50,48,47,45,44,42,41,40,38,37,36,35,34,32,31,30,29,28,27,26,25,24,23,22,21,20,20,19,18,17,16,16,15,14,13,13,12,11,10,10,9,8,8,7,7,6,5,5,4,4,3,3,2,2,1,1}。

5.2 example #2

Variables lmcspinvot [ i ], i 0.. 16, variables ScaleCoeff [ i ] and InvScaleCoeff [ i ], i 0.. 15, derived as follows:

divTable [ ] is specified as follows:

divTable[]＝{0,62,60,58,56,55,53,51,50,48,47,45,44,42,41,40,38,37,36,35,34,32,31,30,29,28,27,26,25,24,23,22,21,20,20,19,18,17,16,16,15,14,13,13,12,11,10,10,9,8,8,7,7,6,5,5,4,4,3,3,2,2,1,1}。

5.3 example #3

Variables lmcspinvot [ i ], i 0.. 16, variables ScaleCoeff [ i ] and InvScaleCoeff [ i ], i 0.. 15, derived as follows:

divTable [ ] is specified as follows:

divTable[]＝{0,124,120,117,113,109,106,103,100,96,93,90,88,85,82,

79,77,74,72,69,67,65,63,60,58,56,54,52,50,48,46,44,43,41,39,37,36,34,33,31,30,28,27,25,24,22,21,20,18,17,16,14,13,12,11,10,9,7,6,5,4,3,2,1}。

5.4 example #4

Variables lmcspinvot [ i ], i ═ 0.. 16, variables ScaleCoeff [ i ] and InvScaleCoeff [ i ], i ═ 0 … 15, were derived as follows:

divTable [ ] is specified as follows:

divTable[]＝{0,124,120,117,113,109,106,103,100,96,93,90,88,85,82,79,77,74,72,69,67,65,63,60,58,56,54,52,50,48,46,44,43,41,39,37,36,34,33,31,30,28,27,25,24,22,21,20,18,17,16,14,13,12,11,10,9,7,6,5,4,3,2,1}。

example #5

Based on JFET-1001-v 8, the text can be changed to:

7.4.6.4 luma map with chroma scaling data semantics

…

Variables lmcspinvot [ i ], i-0 … 16, variables ScaleCoeff [ i ] and InvScaleCoeff [ i ], i-0 … 15, derived as follows:

divTable [ ] is specified as follows:

divTable[]＝{0,248,240,233,226,219,212,206,199,193,187,181,175,170,164,159,154,149,144,139,134,130,125,121,116,112,108,104,100,96,93,89,85,82,78,75,72,68,65,62,59,56,53,50,47,45,42,39,37,34,31,29,26,24,22,19,17,15,13,10,8,6,4,2}。

fig. 5 is a block diagram of a video processing apparatus 500. The apparatus 500 may be used to implement one or more of the methods described herein. The apparatus 500 may be implemented in a smartphone, tablet, computer, internet of things (IoT) receiver, and the like. The apparatus 500 may include one or more processors 502, one or more memories 504, and video processing hardware 506. Processor(s) 602 may be configured to implement one or more methods described in this document (including, but not limited to, method 400). The memory(s) 504 may be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 506 may be used to implement some of the techniques described in this document in hardware circuits.

In some embodiments, the video codec method may be implemented using an apparatus implemented on a hardware platform as described with reference to fig. 5.

From the foregoing, it will be appreciated that specific embodiments of the presently disclosed technology have been described herein for purposes of illustration, but that various modifications may be made without deviating from the scope of the invention. Accordingly, the presently disclosed technology is not limited, except as by the appended claims.

Implementations of the subject matter and the functional operations described in this patent document can be implemented in various systems, digital electronic circuitry, or computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or combinations of one or more of them. Implementations of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances which affect a machine-readable propagated signal, or a combination of one or more of them. The term "data processing unit" or "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer does not require such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

The specification and drawings are to be regarded in an illustrative manner, with an exemplary meaning being exemplary. As used herein, the use of "or" is intended to include "and/or" unless the context clearly indicates otherwise.

While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims (23)

1. A method for video processing, comprising:

a conversion is performed between a current video block of a video and a coded representation of the video, the current video block comprising a luma component and at least one chroma component, wherein the luma component is converted from an original domain to a reshaped domain with a luma mapping with chroma scaling (LMCS) scheme, and chroma samples of the at least one chroma component are predicted based on reconstructed luma samples of the luma component in the original domain.

2. The method of claim 1, wherein,

chroma samples of the at least one chroma component are predicted in a cross-component linear model (CCLM) prediction mode.

3. The method of claim 2, wherein,

adjacent reconstructed luminance samples are converted from the reshaped domain to the original domain and the converted adjacent luminance samples are then used to derive linear model parameters in a linear model to be used in the CCLM prediction mode.

4. The method of claim 3, wherein the neighboring reconstructed luma samples comprise neighboring or non-neighboring reconstructed luma samples in the reshaped domain.

5. The method of claim 3 or 4,

the collocated reconstructed luma samples are transformed from the reshaped domain to the original domain, and then the transformed collocated luma samples are used in the linear model to predict the chroma samples.

6. The method of any of claims 1-5, wherein a first indication is signaled indicating at least one of a scaling factor and an inverse scaling factor to be used in the LMCS scheme.

7. The method of claim 6, wherein at least one of the scaling factor and the inverse scaling factor is coded.

8. The method of claim 7, wherein a value of at least one of the scaling factor and the inverse scaling factor is quantized and the quantized value of at least one of the scaling factor and the inverse scaling factor is signaled.

9. The method of claim 7, wherein the first indication is signaled in a predictive manner.

10. The method of any of claims 6-9, wherein the first indication is coded using a fixed length code, a unary code, a truncated unary code, or an exponential golomb code.

11. The method of any of claims 6-9, wherein the first indication is signaled in a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), an Adaptive Parameter Set (APS), a picture header, a slice header, or a slice group header.

12. The method according to any of claims 6 to 11, wherein the scaling factor comprises a ScaleCoeff [ i ] to be used in the LMCS scheme.

13. The method of any of claims 6 to 11, wherein the inverse scaling factor comprises an InvScaleCoeff [ i ] to be used in the LMCS scheme.

14. The method of any of claims 6 to 12, wherein the scaling factor comprises a chroma scaling factor to be used in the LMCS scheme.

15. The method of claim 14, wherein the chroma scaling factor comprises ChromaScaleCoeff [ i ], and wherein i is an integer.

16. The method of claim 14 or 15, wherein a second indication is signaled indicating whether chroma scaling is applied to the current video block in an LMCS scheme.

17. The method of claim 16, wherein, if the chroma scaling is applied to the current video block, all video blocks in a video unit that covers the current video block use the same chroma scaling factor signaled for the current video block.

18. The method of claim 16 or 17, wherein the second indication is signaled in one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), an Adaptive Parameter Set (APS), a picture header, a slice header, or a slice group header.

19. The method of claim 18, wherein the second indication is signaled in a slice header or a slice group header if a luma component and at least one chroma component of the current video block are coded with a dual tree coding structure and a current slice or a slice group covering the current video block is intra coded.

20. The method of any of claims 1-19, wherein the performing a conversion comprises generating the codec representation from the current video block.

21. The method of any of claims 1-19, wherein the performing a transform comprises generating the current video block from the codec representation.

22. An apparatus in a video system comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any of claims 1-21.

23. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for performing the method of any of claims 1 to 21.

Images