WO2020094153A1

WO2020094153A1 - Component based loop filter

Info

Publication number: WO2020094153A1
Application number: PCT/CN2019/117145
Authority: WO
Inventors: Li Zhang; Kai Zhang; Hongbin Liu; Yue Wang
Original assignee: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc.
Priority date: 2018-11-09
Filing date: 2019-11-11
Publication date: 2020-05-14
Also published as: WO2020094154A1; CN112997504A; CN112997500B; CN112997504B; CN112997500A

Abstract

Component based loop filter is described. In an exemplary aspect, a method for video processing includes making a decision, for a current coding tree unit (CTU) of video, regarding values of first flags associated with adaptive loop filter for a first component; and signaling second flags associated with adaptive loop filter for a second component based on the decision.

Description

COMPONENT BASED LOOP FILTER

Under the applicable patent law and/or rules pursuant to the Paris Convention, this application is made to timely claim the priority to and benefits of International Patent Application No. PCT/CN2018/114834, filed on November 9, 2018. The entire disclosures of International Patent Application No. PCT/CN2018/114834 is incorporated by reference as part of the disclosure of this application.

TECHNICAL FIELD

This patent document relates to video coding techniques, devices and systems.

BACKGROUND

In spite of the advances in video compression, digital video still accounts for the largest bandwidth use on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth demand for digital video usage will continue to grow.

SUMMARY

Devices, systems and methods related to digital video coding, and specifically, to adaptive loop filtering for video coding are described. The described methods may be applied to both the existing video coding standards (e.g., High Efficiency Video Coding (HEVC) ) and future video coding standards (e.g., Versatile Video Coding (VVC) ) or codecs.

In one representative aspect, the disclosed technology may be used to provide a method for video processing. This method includes determining a first set of filter coefficients for a current region of video based on a second set of filter coefficients for a second region of video that is collocated with the current region of video, and reconstructing, based on performing a filtering operation using the first set of filter coefficients, the current region of video from a corresponding bitstream representation.

In another representative aspect, the disclosed technology may be used to provide a method for video processing. This method includes determining, for a first chroma component of a current region of video, a value of one or more flags in a bitstream representation of the current region of video based on a value corresponding to another color component, configuring a filtering operation based one the value of the one or more flags, and reconstructing, using the filtering operation, the current region of video from the bitstream representation.

In yet another representative aspect, the disclosed technology may be used to provide a method for video processing. This method includes determining, based on a color format of a current region of video, a set of filter coefficients for a filtering operation, and reconstructing, using the filtering operation, the current region of video from a corresponding bitstream representation.

In another representative aspect, the disclosed technology may be used to provide a method for video processing. This method includes determining, for a conversion between a current region of video and a bitstream representation of the current region of video, a first set of filter coefficients for the current region of video based on a second set of filter coefficients for a second region of video that is collocated with the current region of video; and performing the conversion by performing a filtering operation using the first set of filter coefficients.

In another representative aspect, the disclosed technology may be used to provide a method for video processing. This method includes determining, for a conversion between a current processing unit of video and a bitstream representation of the current processing unit of video, a first flag indicating on or off condition of an adaptive loop filter for the current processing unit of video based on a second processing unit of video that is collocated with the current processing unit of video; and performing the conversion by performing a filtering operation using the first flag.

In another representative aspect, the disclosed technology may be used to provide a method for video processing. This method includes signaling, for a conversion between a picture of video and a bitstream representation of the video, information on region numbers and/or size for the picture of video; splitting the picture into regions based on the information; and performing the conversion based on the split regions.

In another representative aspect, the disclosed technology may be used to provide a method for video processing. This method includes parsing, for a conversion between a picture of video and a bitstream representation of the video, the bitstream representation of the video to obtain information on region numbers and/or size for the picture of video; and performing the conversion based on the information.

In another representative aspect, the disclosed technology may be used to provide a method for video processing. This method includes determining, for a conversion between a first region of video and a bitstream representation of the first region of video, a first set of filter coefficients for the first region of video based on a second set of filter coefficients for a second region of video and a set of differences between the first and second sets of filter coefficients; and performing the conversion by performing a filtering operation using the first set of filter coefficients.

In another representative aspect, the disclosed technology may be used to provide a method for video processing. This method includes merging at least two different regions of video to obtain a merged region; and performing a conversion between the merged region of video and a bitstream representation of the merged region by performing a filtering operation using same selected filter coefficients, wherein an index of a first one region in the at least two different regions of video is non-consecutive to an index of a second one region in the at least two different regions of video.

In another representative aspect, the disclosed technology may be used to provide a method for video processing. This method includes making a decision, for a current coding tree unit (CTU) of video, regarding values of first flags associated with adaptive loop filter for a first component; and signaling second flags associated with adaptive loop filter for a second component based on the decision.

In another representative aspect, the disclosed technology may be used to provide a method for video processing. This method includes parsing a bitstream representation of a current coding tree unit (CTU) of video to determine values of first flags for a first component of the CTU of video based on values of second flags corresponding to a second component of the CTU; configuring a filtering operation based on the values of the first flags; and performing, using the filtering operation, a conversion between the current CTU of video and the bitstream representation of the video including the current CTU.

In another representative aspect, the disclosed technology may be used to provide a method for video processing. This method includes making a determination regarding a color format of a current region of video; and determining adaptive loop filters for one or more chroma components based on the determination.

In yet another representative aspect, the above-described method is embodied in the form of processor-executable code and stored in a computer-readable program medium.

In yet another representative aspect, a device that is configured or operable to perform the above-described method is disclosed. The device may include a processor that is programmed to implement this method.

In yet another representative aspect, a video decoder apparatus may implement a method as described herein.

The above and other aspects and features of the disclosed technology are described in greater detail in the drawings, the description and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of an encoder block diagram for video coding.

FIGS. 2A, 2B and 2C show examples of geometry transformation-based adaptive loop filter (GALF) filter shapes.

FIG. 3 shows an example of a flow graph for a GALF encoder decision.

FIGS. 4A-4D show example subsampled Laplacian calculations for adaptive loop filter (ALF) classification.

FIG. 5 shows an example of a luma filter shape.

FIG. 6 shows an example of region division of a Wide Video Graphic Array (WVGA) sequence.

FIG. 7 shows a flowchart of an example method for linear model derivations for cross-component prediction in accordance with the disclosed technology.

FIG. 8 shows a flowchart of another example method for linear model derivations for cross-component prediction in accordance with the disclosed technology.

FIG. 9 shows a flowchart of yet another example method for linear model derivations for cross-component prediction in accordance with the disclosed technology.

Fig. 10 is a flowchart of an example method of video processing.

Fig. 11 is a flowchart of an example method of video processing.

Fig. 12 is a flowchart of an example method of video processing.

Fig. 13 is a flowchart of an example method of video processing.

Fig. 14 is a flowchart of an example method of video processing.

Fig. 15 is a flowchart of an example method of video processing.

Fig. 16 is a flowchart of an example method of video processing.

Fig. 17 is a flowchart of an example method of video processing.

Fig. 18 is a flowchart of an example method of video processing.

FIG. 19 is a block diagram of an example of a hardware platform for implementing a visual media decoding or a visual media encoding technique described in the present document.

DETAILED DESCRIPTION

Due to the increasing demand of higher resolution video, video coding methods and techniques are ubiquitous in modern technology. Video codecs typically include an electronic circuit or software that compresses or decompresses digital video, and are continually being improved to provide higher coding efficiency. A video codec converts uncompressed video to a compressed format or vice versa. There are complex relationships between the video quality, the amount of data used to represent the video (determined by the bit rate) , the complexity of the encoding and decoding algorithms, sensitivity to data losses and errors, ease of editing, random access, and end-to-end delay (latency) . The compressed format usually conforms to a standard video compression specification, e.g., the High Efficiency Video Coding (HEVC) standard (also known as H. 265 or MPEG-H Part 2) , the Versatile Video Coding (VVC) standard to be finalized, or other current and/or future video coding standards.

In some embodiments, future video coding technologies are explored using a reference software known as the Joint Exploration Model (JEM) . In JEM, sub-block based prediction is adopted in several coding tools, such as affine prediction, alternative temporal motion vector prediction (ATMVP) , spatial-temporal motion vector prediction (STMVP) , bi-directional optical flow (BIO) , Frame-Rate Up Conversion (FRUC) , Locally Adaptive Motion Vector Resolution (LAMVR) , Overlapped Block Motion Compensation (OBMC) , Local Illumination Compensation (LIC) , and Decoder-side Motion Vector Refinement (DMVR) .

Embodiments of the disclosed technology may be applied to existing video coding standards (e.g., HEVC, H. 265) and future standards to improve runtime performance. Section headings are used in the present document to improve readability of the description and do not in any way limit the discussion or the embodiments (and/or implementations) to the respective sections only.

1 Examples of color space and chroma subsampling

Color space, also known as the color model (or color system) , is an abstract mathematical model which simply describes the range of colors as tuples of numbers, typically as 3 or 4 values or color components (e.g. RGB) . Basically speaking, color space is an elaboration of the coordinate system and sub-space.

For video compression, the most frequently used color spaces are YCbCr and RGB.

YCbCr, Y′CbCr, or Y Pb/Cb Pr/Cr, also written as YCBCR or Y'CBCR, is a family of color spaces used as a part of the color image pipeline in video and digital photography systems. Y′is the luma component and CB and CR are the blue-difference and red-difference chroma components. Y′ (with prime) is distinguished from Y, which is luminance, meaning that light intensity is nonlinearly encoded based on gamma corrected RGB primaries.

Chroma subsampling is the practice of encoding images by implementing less resolution for chroma information than for luma information, taking advantage of the human visual system's lower acuity for color differences than for luminance.

1.1 The 4: 4: 4 color format

Each of the three Y'CbCr components have the same sample rate, thus there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and cinematic post production.

1.2 The 4: 2: 2 color format

The two chroma components are sampled at half the sample rate of luma, e.g. the horizontal chroma resolution is halved. This reduces the bandwidth of an uncompressed video signal by one-third with little to no visual difference.

1.3 The 4: 2: 0 color format

In 4: 2: 0, the horizontal sampling is doubled compared to 4: 1: 1, but as the Cb and Cr channels are only sampled on each alternate line in this scheme, the vertical resolution is halved. The data rate is thus the same. Cb and Cr are each subsampled at a factor of 2 both horizontally and vertically. There are three variants of 4: 2: 0 schemes, having different horizontal and vertical siting.

○ In MPEG-2, Cb and Cr are cosited horizontally. Cb and Cr are sited between pixels in the vertical direction (sited interstitially) .

○ In JPEG/JFIF, H. 261, and MPEG-1, Cb and Cr are sited interstitially, halfway between alternate luma samples.

○ In 4: 2: 0 DV, Cb and Cr are co-sited in the horizontal direction. In the vertical direction, they are co-sited on alternating lines.

2 Examples of the coding flow of a typical video codec

FIG. 1 shows an example of encoder block diagram of VVC, which contains three in-loop filtering blocks: deblocking filter (DF) , sample adaptive offset (SAO) and ALF. Unlike DF, which uses predefined filters, SAO and ALF utilize the original samples of the current picture to reduce the mean square errors between the original samples and the reconstructed samples by adding an offset and by applying a finite impulse response (FIR) filter, respectively, with coded side information signaling the offsets and filter coefficients. ALF is located at the last processing stage of each picture and can be regarded as a tool trying to catch and fix artifacts created by the previous stages.

3 Examples of a geometry transformation-based adaptive loop filter in JEM

In the JEM, an geometry transformation-based adaptive loop filter (GALF) with block-based filter adaption is applied. For the luma component, one among 25 filters is selected for each 2×2 block, based on the direction and activity of local gradients.

3.1 Examples of filter shape

In the JEM, up to three diamond filter shapes (as shown in FIGS. 2A, 2B and 2C for the 5×5 diamond, 7×7 diamond and 9×9 diamond, respectively) can be selected for the luma component. An index is signalled at the picture level to indicate the filter shape used for the luma component. For chroma components in a picture, the 5×5 diamond shape is always used.

3.1.1 Block classification

Each 2×2 block is categorized into one out of 25 classes. The classification index C is derived based on its directionality D and a quantized value of activity

as follows:

To calculate D and

gradients of the horizontal, vertical and two diagonal direction are first calculated using 1-D Laplacian:

Indices i and j refer to the coordinates of the upper left sample in the 2×2 block and R (i, j) indicates a reconstructed sample at coordinate (i, j) .

Then D maximum and minimum values of the gradients of horizontal and vertical directions are set as:

and the maximum and minimum values of the gradient of two diagonal directions are set as:

To derive the value of the directionality D, these values are compared against each other and with two thresholds t ₁ and t ₂:

Step 1. If both

and

are true, D is set to 0.

Step 2. If

continue from Step 3; otherwise continue from Step 4.

Step 3. If

D is set to 2; otherwise D is set to 1.

Step 4. If

D is set to 4; otherwise D is set to 3.

The activity value A is calculated as:

A is further quantized to the range of 0 to 4, inclusively, and the quantized value is denoted as

For both chroma components in a picture, no classification method is applied, i.e. a single set of ALF coefficients is applied for each chroma component.

3.1.2 Geometric transformations of filter coefficients

Before filtering each 2×2 block, geometric transformations such as rotation or diagonal and vertical flipping are applied to the filter coefficients f (k, l) depending on gradient values calculated for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks to which ALF is applied more similar by aligning their directionality.

Three geometric transformations, including diagonal, vertical flip and rotation are introduced:

Diagonal: f _D (k, l) =f (l, k) ,

Vertical flip: f _V (k, l) =f (k, K-l-1) , (9)

Rotation: f _R (k, l) =f (K-l-1, k) .

Herein, K is the size of the filter and 0≤k, l≤K-1 are coefficients coordinates, such that location (0, 0) is at the upper left corner and location (K-1, K-1) is at the lower right corner. The transformations are applied to the filter coefficients f (k, l) depending on gradient values calculated for that block. The relationship between the transformation and the four gradients of the four directions are summarized in Table 1.

Table 1: Mapping of the gradient calculated for one block and the transformations

Gradient values	Transformation
g _d2 < g _d1 and g _h < g _v	No transformation
g _d2 < g _d1 and g _v < g _h	Diagonal
g _d1 < g _d2 and g _h < g _v	Vertical flip
g _d1 < g _d2 and g _v < g _h	Rotation

3.1.3 Signaling of filter parameters

In the JEM, GALF filter parameters are signaled for the first CTU, i.e., after the slice header and before the SAO parameters of the first CTU. Up to 25 sets of luma filter coefficients could be signaled. To reduce bits overhead, filter coefficients of different classification can be merged. Also, the GALF coefficients of reference pictures are stored and allowed to be reused as GALF coefficients of a current picture. The current picture may choose to use GALF coefficients stored for the reference pictures, and bypass the GALF coefficients signaling. In this case, only an index to one of the reference pictures is signaled, and the stored GALF coefficients of the indicated reference picture are inherited for the current picture.

To support GALF temporal prediction, a candidate list of GALF filter sets is maintained. At the beginning of decoding a new sequence, the candidate list is empty. After decoding one picture, the corresponding set of filters may be added to the candidate list. Once the size of the candidate list reaches the maximum allowed value (i.e., 6 in current JEM) , a new set of filters overwrites the oldest set in decoding order, and that is, first-in-first-out (FIFO) rule is applied to update the candidate list. To avoid duplications, a set could only be added to the list when the corresponding picture doesn’t use GALF temporal prediction. To support temporal scalability, there are multiple candidate lists of filter sets, and each candidate list is associated with a temporal layer. More specifically, each array assigned by temporal layer index (TempIdx) may compose filter sets of previously decoded pictures with equal to lower TempIdx. For example, the k-th array is assigned to be associated with TempIdx equal to k, and it only contains filter sets from pictures with TempIdx smaller than or equal to k. After coding a certain picture, the filter sets associated with the picture will be used to update those arrays associated with equal or higher TempIdx.

Temporal prediction of GALF coefficients is used for inter coded frames to minimize signaling overhead. For intra frames, temporal prediction is not available, and a set of 16 fixed filters is assigned to each class. To indicate the usage of the fixed filter, a flag for each class is signaled and if required, the index of the chosen fixed filter. Even when the fixed filter is selected for a given class, the coefficients of the adaptive filter f (k, l) can still be sent for this class in which case the coefficients of the filter which will be applied to the reconstructed image are sum of both sets of coefficients.

The filtering process of luma component can controlled at CU level. A flag is signaled to indicate whether GALF is applied to the luma component of a CU. For chroma component, whether GALF is applied or not is indicated at picture level only.

3.1.4 Filtering process

At decoder side, when GALF is enabled for a block, each sample R (i, j) within the block is filtered, resulting in sample value R′ (i, j) as shown below, where L denotes filter length, f _m, n represents filter coefficient, and f (k, l) denotes the decoded filter coefficients.

3.1.5 Determination process for encoder side filter parameters

Overall encoder decision process for GALF is illustrated in FIG. 3. For luma samples of each CU, the encoder makes a decision on whether or not the GALF is applied and the appropriate signalling flag is included in the slice header. For chroma samples, the decision to apply the filter is done based on the picture-level rather than CU-level. Furthermore, chroma GALF for a picture is checked only when luma GALF is enabled for the picture.

4 Examples of a geometry transformation-based adaptive loop filter in VVC

The current design of GALF in VVC has the following major changes compared to that in JEM:

1) The adaptive filter shape is removed. Only 7x7 filter shape is allowed for luma component and 5x5 filter shape is allowed for chroma component.

2) Temporal prediction of ALF parameters and prediction from fixed filters are both removed.

3) For each CTU, one bit flag is signaled whether ALF is enabled or disabled.

4) Calculation of class index is performed in 4x4 level instead of 2x2. In addition, as proposed in JVET-L0147, sub-sampled Laplacian calculation method for ALF classification is utilized. More specifically, there is no need to calculate the horizontal/vertical/45 diagonal /135 degree gradients for each sample within one block. Instead, 1: 2 subsampling is utilized.

5 Examples of a region-based adaptive loop filter in AVS2

ALF is the last stage of in-loop filtering. There are two stages in this process. The first stage is filter coefficient derivation. To train the filter coefficients, the encoder classifies reconstructed pixels of the luminance component into 16 regions, and one set of filter coefficients is trained for each category using wiener-hopf equations to minimize the mean squared error between the original frame and the reconstructed frame. To reduce the redundancy between these 16 sets of filter coefficients, the encoder will adaptively merge them based on the rate-distortion performance. At its maximum, 16 different filter sets can be assigned for the luminance component and only one for the chrominance components. The second stage is a filter decision, which includes both the frame level and LCU level. Firstly the encoder decides whether frame-level adaptive loop filtering is performed. If frame level ALF is on, then the encoder further decides whether the LCU level ALF is performed.

5.1 Filter shape

The filter shape adopted inAVS-2 is a 7×7 cross shape superposing a 3×3 square shape, just as illustrated in FIG. 5 for both luminance and chroma components. Each square in FIG. 5 corresponds to a sample. Therefore, a total of 17 samples are used to derive a filtered value for the sample of position C8. Considering overhead of transmitting the coefficients, a point-symmetrical filter is utilized with only nine coefficients left, {C0, C1, ..., C8} , which reduces the number of filter coefficients to half as well as the number of multiplications in filtering. The point-symmetrical filter can also reduce half of the computation for one filtered sample, e.g., only 9 multiplications and 14 add operations for one filtered sample.

5.2 Region-based adaptive merge

In order to adapt different coding errors, AVS-2 adopts region-based multiple adaptive loop filters for luminance component. The luminance component is divided into 16 roughly-equal-size basic regions where each basic region is aligned with largest coding unit (LCU) boundaries as shown in FIG. 6, and one Wiener filter is derived for each region. The more filters are used, the more distortions are reduced, but the bits used to encode these coefficients increase along with the number of filters. In order to achieve the best rate-distortion performance, these regions can be merged into fewer larger regions, which share the same filter coefficients. In order to simplify the merging process, each region is assigned with an index according to a modified Hilbert order based on the image prior correlations. Two regions with successive indices can be merged based on rate-distortion cost.

The mapping information between regions should be signaled to the decoder. In AVS-2, the number of basic regions is used to represent the merge results and the filter coefficients are compressed sequentially according to its region order. For example, when {0, 1} , {2, 3, 4} , {5, 6, 7, 8, 9} and the left basic regions merged into one region respectively, only three integers are coded to represent this merge map, i.e., 2, 3, 5.

5.3 Signaling of side information

Multiple switch flags are also used. The sequence switch flag, adaptive_loop_filter_enable, is used to control whether adaptive loop filter is applied for the whole sequence. The image switch flags, picture_alf_enble [i] , control whether ALF is applied for the corresponding ith image component. Only if the picture_alf_enble [i ] is enabled, the corresponding LCU-level flags and filter coefficients for that color component will be transmitted. The LCU level flags, lcu_alf_enable [k] , control whether ALF is enabled for the corresponding kth LCU, and are interleaved into the slice data. The decision of different level regulated flags is all based on the rate-distortion cost. The high flexibility further makes the ALF improve the coding efficiency much more significantly.

In some embodiments, and for a luma component, there could be up to 16 sets of filter coefficients.

In some embodiments, and for each chroma component (Cb and Cr) , one set of filter coefficients may be transmitted.

6 Drawbacks of existing implementations

In some existing implementations (e.g., region-based ALF design in AVS-2) , the following problems are encountered:

(1) For the region-based ALF design, the correlation of filter coefficients between regions in current picture and previously coded pictures is not utilized.

(2) The region size is fixed for all kinds of videos regardless the video resolution. For a video with high resolution, e.g., 4096x2048, splitting into 16 regions may result in a too big region.

(3) For each LCU, one bit flag for each color component is signaled to indicate whether ALF is applied or not. However, there is some dependency between luma and chroma, when ALF is not applied to luma, typically, it won’t be applied to the corresponding chroma blocks.

The GALF design in VVC has the following problems:

(1) It was designed for 4: 2: 0 color format. For 4: 4: 4 color format, luma and chroma components may be of similar importance. How to better apply GALF is unknown.

7 Exemplary methods for improvements in adaptive loop filtering

Embodiments of the presently disclosed technology overcome the drawbacks of existing implementations, thereby providing video coding with higher coding efficiencies. The improvement of adaptive loop filtering, based on the disclosed technology, may enhance both existing and future video coding standards, is elucidated in the following examples described for various implementations. The examples of the disclosed technology provided below explain general concepts, and are not meant to be interpreted as limiting. In an example, unless explicitly indicated to the contrary, the various features described in these examples may be combined.

Example 1. It is proposed that filter coefficients of one region within current slice/picture/tile group may be predicted/derived from that used in a (e.g., collocated) region in different pictures.

(a) In one example, one flag for a region may be firstly signaled to indicate whether the filter coefficients are predicted/derived from those used in a collocated region.

(b) In one example, the collocated region should be located in a reference picture of current picture.

(c) Alternatively, furthermore, an index may be signaled to indicate from which picture the filter coefficients may be predicted/derived.

(d) In one example, another flag for a region may be signaled to indicate whether its filter coefficients are predicted/derived from the same picture as another region (e.g., its neighboring region) .

(i) In one example, an additional information is signaled to indicate which region the filter coefficients are predicted/derived from.

Example 2. One flag may be signaled in a higher level (i.e., a larger set of video data, such as picture/slice/tile group/tile) to indicate whether all regions’ filter coefficients are predicted/derived from their corresponding collocated regions in different pictures.

(a) In one example, the different pictures should be reference pictures of current picture.

(b) Alternatively, furthermore, an index may be signaled to indicate from which picture the filter coefficients may be predicted/derived.

Example 3. The ALF on/off flags of a region or CTU may be inherited from (e.g., collocated) region/ (e.g., collocated) CTU in different pictures.

(a) In one example, the collocated region should be located in a reference picture of current picture.

(b) One flag is signaled in a higher level (i.e., a larger set of video data, such as picture/slice/tile group/tile) to indicate whether all regions’ on/off flags are inherited from their corresponding collocated regions in different pictures.

(c) An index may be signaled in picture/slice header/tile group header to indicate from which picture the on/off flags may be inherited.

Example 4. Region size or the number of regions may be signaled in a SPS, a VPS, a PPS, a picture header or slice header.

(a) In one example, several sets of region numbers/sizes may be pre-defined. An index to the sets may be signaled.

(b) In one example, the number of regions or region sizes may be dependent on the width and/or height of the picture, and/or picture/slice types .

Example 5. Predictive coding of filter coefficients associated with two regions may be utilized.

(a) When signaling the filter coefficients of a first region, the differences compared to that of a second region may be signaled.

(i) The second region may be the one with successive index of the first region.

(ii) The second region may be the one with the largest index value of the previously coded regions with ALF enabled.

Example 6. It is proposed that different regions even with non-successive indices could be merged.

(a) Merged regions may share the same set of selected filters.

(b) In one example, it is signaled in picture header which regions are merged.

(c) In one example, for each region, an index of a set of selected filter coefficients may be transmitted.

Example 7. For a given CTU, the signaling of ALF on/off flags for chroma component may be dependent on the on/off values for the luma component.

(a) The signaling of ALF on/off flags for chroma component may be dependent on the on/off values for another chroma component, e.g., Cb depend on Cr, or Cr depend on Cb.

(b) In one example, when ALF is disabled for a color component, it is automatically disabled for another one or more color components for a CTU without any signaling.

(c) In one example, the ALF on/off values of one color component may be used as context for coding the ALF on/off values of another color component.

Example 8. How to handle ALF for chroma color components may depend on the color format.

(a) Whether to apply classification for chroma components is dependent on the color format. For example, for 4: 4: 4, block-based classification for chroma component may be applied while for 4: 2: 0, it is disallowed.

(b) Whether to allow multiple sets of filters for chroma components is dependent on the color format. For example, for 4: 4: 4, multiple sets of filters for chroma component may be applied while for 4: 2: 0, one set of filter is applied to both color components.

(c) Whether to allow different filters for two chroma components is dependent on the color format. For example, for 4: 4: 4, at least two sets of filters for chroma component may be applied, with at least one set for one color component, respectively. while for 4: 2: 0, one set of filter is applied to both color components

(d) In one example, when the color format is 4: 4: 4, two chroma components may use different filters, or different sets of filters, or selection of filters are based on classification results of each color component.

The examples described above may be incorporated in the context of the method described below, e.g.,

methods

700, 800 and 900, which may be implemented at a video decoder or a video encoder.

FIG. 7 shows a flowchart of an exemplary method for video processing. The method 700 includes, at step 710, determining a first set of filter coefficients for a current region of video based on a second set of filter coefficients for a second region of video that is collocated with the current region of video.

The method 700 includes, at step 720, reconstructing, based on performing a filtering operation using the first set of filter coefficients, the current region of video from a corresponding bitstream representation. In some embodiments, the filtering operations include loop filtering (or adaptive loop filtering) .

In some embodiments, and in the context of Example 1, the second region of video is from a different picture than a current picture of the current region of video. In other embodiments, the different picture is a reference picture of the current picture.

In some embodiments, and in the context of Example 5, the first set of filter coefficients is predicted from the second set of filter coefficients using a prediction operation. In an example, the prediction operation is controlled based on a flag in the bitstream representation.

In some embodiments, and in the context of Example 5, the first set of filter coefficients is based on the second set of filter coefficients and a set of differences between the first and second sets of filter coefficients. In an example, an index of the second region of video is consecutive to an index of the current region of video. In another example, an index of the second region of video corresponds to a largest index value of previously coded regions for which the filtering operation was enabled. In the context of Example 6, an index of the second region of video is non-consecutive to an index of the current region of video.

FIG. 8 shows a flowchart of an exemplary method for video processing. The method 800 includes, at step 810, determining, for a first chroma component of a current region of video, a value of one or more flags in a bitstream representation of the current region of video based on a value corresponding to another color component. In some embodiments, the color component may be a luma component or another chroma component, e.g., Y, Cb and Cr for YUV files.

The method 800 includes, at step 820, configuring a filtering operation based one the value of the one or more flags. In some embodiments, the filtering operations include loop filtering (or adaptive loop filtering) .

The method 800 includes, at step 830, reconstructing, using the filtering operation, the current region of video from the bitstream representation.

In some embodiments, and in the context of Example 7, the value of the one or more flags corresponding to the first chroma component are based on a value of one or more flags corresponding to a luma component of the current region of video.

In some embodiments, and in the context of Example 7, the value of the one or more flags corresponding to the first chroma component are based on a value of one or more flags corresponding to a second chroma component of the current region of video. In an example, the first chroma component is a blue-difference chroma component and the second chroma component is a red-difference chroma component. In another example, the first chroma component is a red-difference chroma component and the second chroma component is a blue-difference chroma component.

In some embodiments, and in the context of Example 8, the value of the one or more flags corresponding to the first chroma component are based on a color format of the current region of video.

FIG. 9 shows a flowchart of an exemplary method for video processing. The method 900 includes, at step 910, determining, based on a color format of a current region of video, a set of filter coefficients for a filtering operation. In some embodiments, the filtering operations include loop filtering (or adaptive loop filtering) .

The method 900 includes, at step 920, reconstructing, using the filtering operation, the current region of video from a corresponding bitstream representation.

In some embodiments, and in the context of Example 8, different sets of filter coefficients are used for the filtering operation for different chroma components of the current region of video. In other embodiments, multiple sets of filter coefficients are used for the filtering operation for at least one chroma component of the current region of video. In an example, the color format is 4: 4: 4.

FIG. 10 shows a flowchart of an exemplary method for video processing. The method 1000 includes, determining (1002) , for a conversion between a current region of video and a bitstream representation of the current region of video, a first set of filter coefficients for the current region of video based on a second set of filter coefficients for a second region of video that is collocated with the current region of video; and performing (1004) the conversion by performing a filtering operation using the first set of filter coefficients.

In some example, the first set of filter coefficients is predicted or derived from the second set of filter coefficients.

In some example, the filtering operation comprises loop filtering, and the first set of filter coefficients is the filter coefficients for adaptive loop filters of the loop filtering.

In some example, the current region of video is from a first set of video data, the second region of video is from a second set of video data different from the first set of video data, the set of video data including one of slice, tile, tile group, picture.

In some example, the second region of video is from a different picture than a current picture of the current region of video.

In some example, the different picture is a reference picture of the current picture.

In some example, the method 1000 further comprises: for at least one region of the video, signaling a first flag for the region to indicate whether a set of filter coefficients for the region is predicted/derived based on a corresponding set of filter coefficients for a collocated region that is collocated with the region.

In some example, the method 1000 further comprises: for at least one region of the video, parsing the bitstream representation of the region to obtain a first flag for the region to indicate whether a set of filter coefficients for the region is predicted or derived based on a corresponding set of filter coefficients for a collocated region that is collocated with the region.

In some example, the method 1000 further comprises: for at least one region of the video, signaling an index of picture to indicate from which picture the set filter coefficients of the region is predicted or derived.

In some example, the method 1000 further comprises: for at least one region of the video, parsing the bitstream representation of the region to obtain an index of picture to indicate from which picture the set filter coefficients of the region is predicted or derived.

In some example, the method 1000 further comprises: for at least one region of the video, signaling a second flag for the region to indicate whether the set of filter coefficients of the region is predicted or derived from the same picture as another region.

In some example, the another region is a neighboring region of the region.

In some example, the method 1000 further comprises: signaling an additional information for the region to indicate from which region the set of filter coefficients is predicted or derived.

In some example, the method 1000 further comprises: for at least one region of the video, parsing the bitstream representation of the region to obtain a second flag for the region to indicate whether the set of filter coefficients of the region is predicted or derived from the same picture as another region.

In some example, the another region is a neighboring region of the region.

In some example, the method 1000 further comprises: parsing the bitstream representation of the region to obtain an additional information for the region to indicate from which region the set of filter coefficients is predicted or derived.

In some example, the method 1000 further comprises: signaling a third flag, at a level of the set of video data, to indicate whether filter coefficients of all regions within the first set of video data are predicted or derived from their corresponding collocated regions in different pictures.

In some example, the different pictures are reference pictures of current picture.

In some example, the method 1000 further comprises: signaling an index of picture to indicate from which picture the filter coefficients of all region are predicted or derived.

In some example, the method 1000 further comprises: parsing the bitstream representation of the region to obtain a third flag, at a level of the set of video data, to indicate whether filter coefficients of all regions within the first set of video data are predicted or derived from their corresponding collocated regions in different pictures.

In some example, the method 1000 further comprises: parsing the bitstream representation of the region to obtain an index of picture to indicate from which picture the filter coefficients of all region are predicted or derived.

FIG. 11 shows a flowchart of an exemplary method for video processing. The method 1100 includes, determining (1102) , for a conversion between a current processing unit of video and a bitstream representation of the current processing unit of video, a first flag indicating on or off condition of an adaptive loop filter for the current processing unit of video based on a second processing unit of video that is collocated with the current processing unit of video; and performing (1104) the conversion by performing a filtering operation based on the first flag.

In some example, the first flag for the current processing unit of video is inherited from the second processing unit of video.

In some example, the filtering operation comprises loop filtering.

In some example, the processing unit includes one of region and coding tree unit (CTU) .

In some example, the current processing unit of video is from a first set of video data, the second processing unit of video is from a second set of video data different from the first set of video data, the set of video data including one of slice, tile, tile group, picture.

In some example, the second processing unit of video is from a different picture than a current picture of the current processing unit of video.

In some example, the method 1100 further comprises: signaling a second flag, at a level of the set of video data, to indicate whether the first flags of all processing units within the set of video data are inherited from their corresponding collocated processing units in different pictures.

In some example, the method 1100 further comprises: parsing the bitstream representation of the region to obtain a second flag, at a level of the set of video data, to indicate whether the first flags of all processing units within the set of video data are inherited from their corresponding collocated processing units in different pictures.

In some example, the method 1100 further comprises: signaling an index of picture in picture header, slice header, tile group header to indicate from which picture the first flags of the first processing unit are inherited.

In some example, the method 1100 further comprises: parsing the bitstream representation of the region to obtain an index of picture in picture header, slice header, tile group header to indicate from which picture the first flags of the first processing unit are inherited.

FIG. 12 shows a flowchart of an exemplary method for video processing. The method 1200 includes, signaling (1202) , for a conversion between a picture of video and a bitstream representation of the video, information on region numbers and/or size for the picture of video; splitting (1204) the picture into regions based on the information; and performing (1206) the conversion based on the split regions.

In some example, signaling the information on region numbers and/or size in at least one of Sequence Parameter Set (SPS) , Video Parameter Set (VPS) , Picture Parameter Set (PPS) , picture header, slice header.

In some example, the method 1200 further comprises: signaling an index to at least one of a plurality of sets of region numbers and/or size, wherein plurality of sets of region numbers and/or size are pre-defined.

In some example, the region numbers and/or size is dependent on width and/or height of the picture and/or slice types.

FIG. 13 shows a flowchart of an exemplary method for video processing. The method 1300 includes, parsing (1302) , for a conversion between a picture of video and a bitstream representation of the video, the bitstream representation of the video to obtain information on region numbers and/or size for the picture of video; and performing (1304) the conversion based on the information.

In some example, parsing the bitstream representation of the video to obtain the information on region numbers and/or size in at least one of Sequence Parameter Set (SPS) , Video Parameter Set (VPS) , Picture Parameter Set (PPS) , picture header, slice header.

In some example, the method 1300 further comprises: parsing the bitstream representation of the video to obtain an index to at least one of a plurality of sets of region numbers and/or size, wherein plurality of sets of region numbers and/or size are pre-defined.

FIG. 14 shows a flowchart of an exemplary method for video processing. The method 1400 includes, determining (1402) , for a conversion between a first region of video and a bitstream representation of the first region of video, a first set of filter coefficients for the first region of video based on a second set of filter coefficients for a second region of video and a set of differences between the first and second sets of filter coefficients; and performing (1404) the conversion by performing a filtering operation using the first set of filter coefficients.

In some example, when determining the first set filter coefficients of the second region, the set of differences is signaled.

In some example, when determining the first set filter coefficients of the second region, parsing the bitstream representation of the first region of video to obtain the set of differences.

In some example, an index of the second region of video is consecutive to an index of the first region of video.

In some example, an index of the second region of video corresponds to a largest index value of previously coded regions for which the filtering operation was enabled.

In some example, the filtering operation includes adaptive loop filtering.

FIG. 15 shows a flowchart of an exemplary method for video processing. The method 1500 includes, merging (1502) at least two different regions of video to obtain merged regions; and performing (1504) a conversion between the merged regions of video and a bitstream representation of the merged regions by performing a filtering operation using same selected filter coefficients, wherein an index of a first one region in the at least two different regions of video is non-consecutive to an index of a second one region in the at least two different regions of video.

In some example, the merged regions share one same set of selected filters coefficients.

In some example, the method 1500 further comprises: signaling in picture header which regions of video are merged.

In some example, the method 1500 further comprises: for each region, an index of a set of selected filter coefficients is transmitted.

FIG. 16 shows a flowchart of an exemplary method for video processing. The method 1600 includes making a decision (1602) , for a current coding tree unit (CTU) of video, regarding values of first flags associated with adaptive loop filter for a first component; and signaling (1604) second flags associated with adaptive loop filter for a second component based on the decision.

In some example, the first component comprises luma component and the second component comprises one or more chroma components.

In some example, in response to the decision indicating the adaptive loop filter for luma component is disabled, automatically disabling the adaptive loop filter for one or more chroma components for the CTU without any signaling.

In some example, the first component is a blue-difference (Cb) chroma component and the second component is a red-difference (Cr) chroma component.

In some example, the first component is a red-difference (Cr) chroma component and the second component is a blue-difference (Cb) chroma component.

In some example, in response to the decision indicating the adaptive loop filter for one chroma component is disabled, automatically disabling the adaptive loop filter for another one or more color components for the CTU without any signaling.

In some example, the values of the first flags associated with adaptive loop filter for one color component is used as context for coding the values of second flags associated with adaptive loop filter for another color component.

In some example, the method 1600 further comprises: determining an enabling/disabling of a filtering operation using the second flags, performing, based on the determination, a conversion between the current CTU of video and a bitstream representation of the video including the current CTU.

FIG. 17 shows a flowchart of an exemplary method for video processing. The method 1700 includes, parsing (1702) a bitstream representation of a current coding tree unit (CTU) of video to determine values of first flags for a first component of the CTU of video based on values of second flags corresponding to a second component of the CTU; configuring (1704) a filtering operation based on the values of the first flags; and performing (1706) , using the filtering operation, a conversion between the current CTU of video and the bitstream representation of the video including the current CTU.

In some example, the second component comprises luma component and the first component comprises one or more chroma components.

In some example, in response to the values of the second flags indicating the adaptive loop filter for luma component is disabled, automatically disabling the adaptive loop filter for one or more chroma components for the CTU.

In some example, the second component is a blue-difference (Cb) chroma component and the first chroma component is a red-difference (Cr) chroma component.

In some example, the second component is a red-difference (Cr) chroma component and the first chroma component is a blue-difference (Cb) chroma component.

In some example, in response to the values of the second flags indicating the adaptive loop filter for one chroma component is disabled, automatically disabling the adaptive loop filter for another one or more color components for the CTU.

In some example, the values of the first flags associated with adaptive loop filter for one color component is used as context for decoding the values of first flags associated with adaptive loop filter for another color component.

FIG. 18 shows a flowchart of an exemplary method for video processing. The method 1800 includes, making a determination (1802) regarding a color format of a current region of video; and determining (1804) adaptive loop filters for one or more chroma components based on the determination.

In some example, whether to use applying classification for the one or more chroma components is based on the determination.

In some example, whether to use multiple sets of filter for the one or more chroma components is based on the determination.

In some example, whether to use different sets of filter for two chroma components is based on the determination.

In some example, in response to the determination that the color format is 4: 4: 4, two chroma components use different filters, or different sets of filters, or selection of filters are based on classification results of each color component.

In some example, the method 1800 further comprises: performing a conversion between the current region of video and a bitstream representation of the current region by performing a filtering operation using the adaptive loop filters for one or more chroma components.

In some example, the filtering operation comprises loop filtering.

In some example, the conversion generates the region of video from the bitstream representation.

In some example, the conversion generates the bitstream representation from the region of video.

8 Example implementations of the disclosed technology

FIG. 19 is a block diagram of a video processing apparatus 1900. The apparatus 1900 may be used to implement one or more of the methods described herein. The apparatus 1900 may be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, and so on. The apparatus 1900 may include one or more processors 1902, one or more memories 1904 and video processing hardware 1906. The processor (s) 1902 may be configured to implement one or more methods (including, but not limited to,

methods

700, 800 and 900) described in the present document. The memory (memories) 1904 may be used for storing data and code used for implementing the methods and techniques described herein. The video processing hardware 1906 may be used to implement, in hardware circuitry, some techniques described in the present document.

In some embodiments, the video coding methods may be implemented using an apparatus that is implemented on a hardware platform as described with respect to FIG. 19.

From the foregoing, it will be appreciated that specific embodiments of the presently disclosed technology have been described herein for purposes of illustration, but that various modifications may be made without deviating from the scope of the invention. Accordingly, the presently disclosed technology is not limited except as by the appended claims.

Implementations of the subject matter and the functional operations described in this patent document can be implemented in various systems, digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term “data processing unit” or “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document) , in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code) . A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit) .

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Computer readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

It is intended that the specification, together with the drawings, be considered exemplary only, where exemplary means an example. As used herein, the singular forms “a” , “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. Additionally, the use of “or” is intended to include “and/or” , unless the context clearly indicates otherwise.

While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims

A method for video processing, comprising:

making a decision, for a current coding tree unit (CTU) of video, regarding values of first flags associated with adaptive loop filter for a first component; and

signaling second flags associated with adaptive loop filter for a second component based on the decision.
The method of claim 1, wherein the first component comprises luma component and the second component comprises one or more chroma components.
The method of claim 2, wherein in response to the decision indicating the adaptive loop filter for luma component is disabled, automatically disabling the adaptive loop filter for one or more chroma components for the CTU without any signaling.
The method of claim 1, wherein the first component is a blue-difference (Cb) chroma component and the second component is a red-difference (Cr) chroma component.
The method of claim 1, wherein the first component is a red-difference (Cr) chroma component and the second component is a blue-difference (Cb) chroma component.
The method of claim 1, 4 or 5, wherein in response to the decision indicating the adaptive loop filter for one chroma component is disabled, automatically disabling the adaptive loop filter for another one or more color components for the CTU without any signaling.
The method of any one of claims 1 to 6, wherein the values of the first flags associated with adaptive loop filter for one color component is used as context for coding the values of the second flags associated with adaptive loop filter for another color component.
The method of any one of claims 1-7, further comprising:

determining an enabling/disabling of a filtering operation using the second flags,

performing, based on the determination, a conversion between the current CTU of video and a bitstream representation of the video including the current CTU.
A method for video processing, comprising:

parsing a bitstream representation of a current coding tree unit (CTU) of video to determine values of first flags for a first component of the CTU of video based on values of second flags corresponding to a second component of the CTU;

configuring a filtering operation based on the values of the first flags; and

performing, using the filtering operation, a conversion between the current CTU of video and the bitstream representation of the video including the current CTU.
The method of claim 9, wherein the second component comprises luma component and the first component comprises one or more chroma components.
The method of claim 10, wherein in response to the values of the second flags indicating the adaptive loop filter for luma component is disabled, automatically disabling the adaptive loop filter for one or more chroma components for the CTU.
The method of claim 9, wherein the second component is a blue-difference (Cb) chroma component and the first component is a red-difference (Cr) chroma component.
The method of claim 9, wherein the second component is a red-difference (Cr) chroma component and the first component is a blue-difference (Cb) chroma component.
The method of claim 9, 12 or 13, wherein in response to the values of the second flags indicating the adaptive loop filter for one chroma component is disabled, automatically disabling the adaptive loop filter for another one or more color components for the CTU.
The method of any one of claim 9-14, wherein the values of the second flags associated with adaptive loop filter for one color component is used as context for decoding the values of first flags associated with adaptive loop filter for another color component.
A method for video processing, comprising:

making a determination regarding a color format of a current region of video; and

determining adaptive loop filters for one or more chroma components based on the determination.
The method of claim 16, wherein whether to apply classification for the one or more chroma components is based on the determination.
The method of claim 16, wherein whether to use multiple sets of filter for the one or more chroma components is based on the determination.
The method of claim 16, wherein whether to use different sets of filter for two chroma components is based on the determination.
The method of claim 16, wherein in response to the determination that the color format is 4: 4: 4, two chroma components use different filters, or different sets of filters, or selection of filters are based on classification results of each color component.
The method of any one of claims 16-20, further comprising:

performing a conversion between the current region of video and a bitstream representation of the current region by performing a filtering operation using the adaptive loop filters for one or more chroma components.
The method of any of claims 1-21, wherein the filtering operation comprises loop filtering.
The method of any one of claims 1-22, wherein the conversion generates the current region of video from the bitstream representation.
The method of any one of claims 1-22, wherein the conversion generates the bitstream representation from the current region of video.
An apparatus in a video system comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to implement the method in any one of claims 1 to 24.
A computer program product stored on a non-transitory computer readable media, the computer program product including program code for carrying out the method in any one of claims 1 to 24.